This is a technical web development post.
The other night I was up until 1:30AM fixing a bug in concharto.com. At 10:00pm I took a look at the latest changes log and noticed hundreds of “reverts” had happened and were continuing to happen. All coming from the same IP address. I panicked! I had to shut the site down temporarily while I fixed the problem and repaired the damage.
The problem was caused by a combination of a badly behaved robot (web spider) and a bug in the “undo revision” code.
2. The Robot. All robots are supposed to follow the instructions in the robots.txt at the root of your web site. This file provides directives on how you want robots to follow links. Our robots.txt file explicitly prohibits following links to the changes page (which contained the bug). Unfortunately, all robots don’t follow this advice. This particular robot found its way to the changes page and started following all of the links there, including each “undoevent.htm” link. Ugh!
I had to do three things to recover:
- Revert the changes that the robot made. Database backups are a good thing.
- Block the robot’s IP address.
3. The Lesson. Keep your GET methods idempotent. It is sometimes easier (less coding) to use the query string to pass parameters but it is a bug – you will probably have to fix it later.