64.68.84.9: Googlebot gone crazy?
Been getting some 404s generated by a bot claiming to be googlebot. According to Whois Source, this address is owned by Google. So what is their bot doing making up random requests? I guess they are seeking out XML feeds, but you’d think they would look for feeds linked to from the page in the header meta tags…
Anyway, I’ve made some modifications to my .htaccess file. Boy this file is getting long now!
RewriteRule ^mark/blog/feed/rss2/atom.xml$ /mark/blog/feed/atom/ [R=301,L] RewriteRule ^mark/blog/atom.xml$ /mark/blog/feed/atom/ [R=301,L] RewriteRule ^mark/blog/rss.xml$ /mark/blog/feed/rss2/ [R=301,L]










August 17th, 2004 21:08
Looks all too cofumsing for me! hehe
I have a thing about those bots man! who knows what they collect? No one raelly, I dont even think the programmers know what they collect hehe
August 17th, 2004 21:16
oh Marky Moo
I updated your link on me website 
August 17th, 2004 21:25
Yeah, mod_rewite can get pretty complicated! I probably have over 40 lines of rules in mine! It’s bloody clever stuff though
August 17th, 2004 21:25
Thanks for the link update btw