A month ago we were overrun by the Bytespider bot and I added it to the robots.txt file. It worked for a month but a few days ago they came back in full force. We are on cloud hosting and as such have no access to the .htaccess file, so I decided to put the entire IP range the robot is using in discouragement mode, redirecting them 100% to some search engine. That worked for a day and a half. Now they're back and I don't know how this is possible.
Looking at the log, it says the...
Read more
Continue reading...
Looking at the log, it says the...
Read more
Continue reading...