We just literally experienced rebellion of the machines. We have a Cursor agent that runs on each PR and conducts a review. At some point we noticed mysterious commits that appear on the PRs signed by “Cursor bot”.

About 2 weeks ago we added a hook named verify.sh that does not let an agent to exit until the PR it generated satisfies certain rules, or until the check fails 5 times. The reviewing agent, who was not supposed to modify anything, became subject of that hook as well. It watched the hook hammering it and decided to act. The effect was amplified by memory: the bot keeps notes between runs.

At first it was sublimating putting corrective commits on its own branch. But then it decided it’s not enough and started putting corrective commits on the PRs themselves.

It gets worse, folks. At some point it quietly modified verify.sh itself: the very hook that was supposed to control its behavior.

Reference reading: Robert Sheckley. “Watchbird”. 1953