Alpha Zero will be coming back! Who will be the boss , SF 10, Alpha Zero and Leela

https://twitter.com/demishassabis?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor

Someone asked Deepmind CEO, Dennis Hassabis, on yesterday whether Alpha Zero is coming back. He said yes.

So who will be the best 

1. Stockfish 10 running on 64 cores, with 7 men TB and brainfish opening book

2. Alpha Zero on 4 TPU

3. Leela on 2x 2080Ti

My estimate, is SF 10 will be +50 above Alpha Zero, and Leela will be -100 below Alpha Zero., based on this rating list.

https://docs.google.com/spreadsheets/d/1ZAIuHR6n-5JTxKQc0XUSx1jyUrgVEcj8DNLKA7-urBw/edit?ts=5b7d9014#gid=714087199

I assume that an improved AlphaZero version with a more trained network will play, and we know from Go that there was a huge difference between AlphaGo Master and AlphaGo Zero. But because NN are not that good for chess then for Go and because we saw nearly no gain in the playing strength with more learning it is still questionable if AlphaZero can be clearly improved.

But we still don‘t know how strong AlphaZero was the last time. You can compare it with the ELF network in the Leela Zero project (Go): even if the Leela networks became stronger and stronger, the ELF network still won 65% (or something like that) of the games. It seemed like ELF, which is a final network, became stronger too by the same amount than the Leela networks. You can't say that it's the same for chess with AlphaZero - but it can't be excluded either.

Because there were too much Alpha* surprises in the last years I bet on AlphaZero.

Very exciting news.

Woow, that's great!

@Godeka, what are top 3 or top 5" Go" NN engines now?

Hard to say, because there are so many engines and different NN. The strongest or one of the strongest open source engines is LeelaZero. Maybe Phoenix Go is stronger, and since May it is open source too, but I don’t know if the released NN is the strongest one. 

Golaxy is very strong too, maybe stronger than Leela, but it isn’t available. The same is true for other candidates like FineArt. By far the strongest engine is AlphaGo Zero or AlphaZero. In fact the other named engines are a rebuild of one of DeepMinds Alpha* engines.

Other than in chess, in Go the NN engines are clearly and without any doubt the strongest engines. There are other issues with that engines. For example a weaker player gets handicap stones placed on the board, but the networks are not trained for games with handicap stones. Or  the NN cannot say which player leads with how many points. Or NN are trained with a fixed Komi (points given as compensation for being the second player), but there are a lot of different used Komis, also depending on the difference of the player‘s strengths.

So the issues with NN are quite different. In chess the problem is that NN are very weak in the end game with AB engines. And maybe AB engines are better in finding the shortest solution for a checkmate problem.

Today, I saw someone tested newest 30xxx series network from Alpha Zero vs SF games. New Leela ID 31252 can reproduce exactly the same as Alpha Zero move,  21. Bg5!! at around 15 secs, 26k nodes in my GTX 1060 GPU. Really impressive new networks.

rn3r1k/pn1p1ppq/bpp4p/7P/4N1Q1/6P1/PP3PB1/R1B1R1K1 w - - 3 21

Meanwhile, I saw disaster in latest SF in rook mobility problem vs Leela. 

SF developers will be doing rook mobility patch urgently.(their plan is severe penalty if rook mobility is <3 or attacking own piceces)   I will be following whether that rook mobility patch will pass or not!! 

Of course, chess is extremely massive to search and every player has flaws, every chess engine has flaws.

But this position is too funny for current top 1 engine!!

@drmrboss:

Thank you for sharing the information with us.

Has SF been improved based on the games against AlphaZero? I can imagine the games gave valuable information about SF’s weaknesses.

Yes, SF has been improving continuously.

SF 9 is  58.3 elo stronger than SF 8.

SF 10 is 50.4 elo stronger than SF 9. (SF 10 is ready , just pending for official announcement)

https://github.com/glinscott/fishtest/wiki/Regression-Tests

Your prediction is exactly correct. Here is the video of interview on CEO of Alpha Zero,

Important facts about interview,

1. he estimated that alpha zero rating was around 3600.

2. he said , alpha Zero would be 3500 rating(-100 elo)  in case it was running on home computer with 2x,3x GPU.

3. he said, there will be newer, stronger version of Alpha Zero.

Note. My impression goes into Leela Tester, Csculie,  who correctly estimate Alpha Zero rating to be around 3700. 

https://www.youtube.com/watch?v=OP_3h__MleY

Another Alpha Zero's show. Someone using Alpha Zero machine for analysis.

https://youtu.be/v6ojLMdNJeo

Alpha Zero news are coming out. This is one game where Alpha Zero died as a patzer. ( AlphaZero vs Stockfish 8 in a new 1,000-game match, scoring +155 -6 =839) that means Alpha Zero is +50 elo stronger than SF 8 and possibly somewhere between SF 9 and SF 10.

New beta Leela engine is getting a bit stronger now thanks for Deep Mind's tips.

http://science.sciencemag.org/content/sci/362/6419/1140.full.pdf

Conclusion

1) Stockfish 10 is still the best engine in the world 
2) LC0 guys did manage to reverse engineer A0 successfully
3) LC0 and A0 roughly at the same strength 
4) NN are not less resource hungry than Alpha Beta
5) Scalability is about the same in both methods 
6) Google has unacceptable behaviour, hiding data, obfuscating opponents and hyping results

Beta Leela is a bit stronger now thanks to A0 developer Matthew's tips( they have discusion in talkchess.com in yesterday)

According to latest tests as of today, 

Beta (new ) Leela on 2080Ti ($1200 hardware ) vs AMD 14 cores 1950X ($800 hardware)  SF 10 is exactly same strength in short time control.  

You will likely see Leela in this TCEC super final, where they are using $2000 GPU for Leela vs $1200-1600 CPU for Stockfish. (old CPU)

Keep waiting LOL

5 years later still nothing

5 years later we found out it was a lie

I wish alpha0 was public