ReachCustomersOnline.com

How to block spambots, ban spybots, and tell unwanted robots to go to hell

Posted by TimSlavin at May 8, 2003

How to block spambots, ban spybots, and tell unwanted robots to go to hell: "I recently woke up to the staggering level of abuse occurring on this web site. This is old news to some, but we all wake up at different times. I’m talking, of course, about automated robots and spiders. They come at all hours, they take as much as they can, and they leave me with the (bandwidth) bill. They do so without respecting the Robot Exclusion Standard, now almost 10 years old. Some come to gather email addresses, which are then sold to spammers; some come to steal images or other content, and republish it without my consent; some come to spy on me and sell information to their clients about perceived violations of copyright, trademark, or some nebulous concept of brand identity." Excellent discussion of how rogue bots can be blocked with lots of useful links to more information. He has to have the funniest URL that I have ever seen. From Dive Into Mark. Found it on Jonathan Peterson's Way.nu weblog.

Comments

Post a comment

Name:

Email Address:

URL:
Remember Me? YesNo

Comments: