Thinking About Chess Engine Search Depth

We saw in an earlier post that running a chess analysis on a much more powerful computer often doesn't result in a different answer than running it on more modest hardware. This brings up the related question: how long should you let a chess engine analyze a position?

In some chess communities I've occasionally seen an enthusiastic comment by someone who proposes to do "deep analysis" of a chess position by letting their chess engine analyze it for an hour. That kind of time commitment is rarely warranted, however.

Very typical outcomes are shown in the table below comparing Stockfish evaluations at different depths using 9 chess positions randomly sampled from the 15th moves of some ICCF correspondence games:

I've highlighted the depth in green when Stockfish first found the answer it also reached in its depth 50 analysis. You can see in this sample that the "best move" was actually discovered much earlier in all but two cases. And in those two cases, the depth 50 solution engine score wasn't greatly different than shallower depth move choices.

In some cases the engine's view of your opponent's best follow up move changes with increased depth.  And the engine evaluation score for the position continues to evolve. But in terms of your next best move, going to great depths typically doesn't change the answer.  

There are certainly are cases where going to depth 50 or greater might matter. There are cases in winning positions (advantages of 100 cp or more) when it might take some time to find the optimal line. It can also be time consuming for an engine to find the shortest mating sequence for a winning endgame position (hence the value of Syzygy end game tables). But in general, analysis times of more than 10 minutes don't make a lot of sense.

In chess engine vs. chess engine tournaments, this very question of how to decide when you've already reached the optimal best move comes up. Engines are only given so much time to make a move.  Generally if they don't use all their allotted time on a move, the unused time is available for future moves. So there is an incentive to not waste time on simple moves and spend more time evaluating more difficult positions. Stockfish's time control code (only relevant in engine vs. engine tournaments) looks at the same thing we just analyzed: how stable is the solution as greater depth is reached?

The wisdom built into that time control can be generalized: If the best move solution seems to be stable, it's time to move onto your next position analysis!