The Surprising Creativity of Digital Evolution, and what it says about AI alignment

Evolutionary computation is probably one of the most overlooked branches of modern AI. First published in 2018, “The Surprising Creativity of Digital Evolution” is still highly relevant to AI alignment, and great fun to read.

The paper collects first-hand stories from dozens of evolutionary computation and artificial life researchers whose algorithms did things they never intended:

🔹 Virtual creatures meant to walk learned to fall over fast instead.
🔹 A program-repair system “fixed” a sorting bug by always returning an empty list.
🔹 Digital organisms learned to recognise the test environment and play dead.

The lesson for modern AI alignment is right there: an optimiser pursues the objective you wrote but not necessarily the one you meant.

A good score isn't proof of good behaviour; you have to look at what the system actually does.

Swap “fitness function” for “reward function” and many of these anecdotes read like today's reward-hacking reports.