AlphaGo Beats Lee Sedol, 2016

Go had resisted computers for decades. The board has 19 by 19 intersections and a branching factor roughly ten times chess. Experts had predicted a decade more before a machine could beat a top human.

DeepMind's AlphaGo, led by David Silver and team, defeated Lee Sedol 4 to 1 in a televised five-game match in Seoul in March 2016. Over 200 million people watched online. Lee won game four with a move now called the divine move, a rare reminder that humans still had tricks.

What AlphaGo combined

1. 1Policy network trained on millions of human professional games
2. 2Value network that estimated the probability of winning from a position
3. 3Monte Carlo tree search that looked ahead thousands of moves
4. 4Self-play reinforcement learning to improve beyond human games

The following year, AlphaGo Zero started from random play, used only self-play and the rules of Go, and surpassed the Lee Sedol version in a matter of days. It proved that a system could reach superhuman play without imitating human games at all.

“It's not a human move. I've never seen a human play this move.”

The big idea: reinforcement learning plus deep networks plus self-play produced superhuman play in domains humans had studied for millennia. The technique generalizes far beyond games.