Fed up of bad bots?

Bad bots, ie robots and spiders that index your site in an aggressive mannger with little benefit to yourself, can easily cause problems with bandwidth. To ban the bots, you have a few options. ROBOTS.TXT Good bots should obey robots.txt, but bad bots usually don’t so whilst it’s good practice to use this file, it …

Fed up of bad bots? Read More »