Efforts have been made over the past decade to teach artificial intelligence to beat humans at almost every game that involves strategy. All AI breakthroughs in previous benchmark games, be it checkers, chess, Go, 2 player poker, StarCraft 2 or Dota 2, AI was successful because it attempted to estimate widely popular strategy: Nash equilibrium.
Chess has long been a benchmark of progress in machine intelligence, from Alan Turing’s 1951 program for playing the game (written on paper) through Garry Kasparov’s defeat at the hands of IBM’s Deep Blue.
Deep Blue is the chess machine that defeated then-reigning World Chess Champion Garry Kasparov in a six-game match in 1997. There were a number of factors that contributed to this success including:
- a single-chip chess search engine
- a massively parallel system with multiple levels of parallelism
- a strong emphasis on search extensions
- a complex evaluation function and
- effective use of a Grandmaster game database
One less talked about aspect of this entire ‘machine beats man’ grand tale is that Deep Blue-I was defeated by Kasparov in 1996. The lessons learnt here helped Deep Blue- II to triumph in 1997. This achievement was a culmination of efforts spanning over decades of making machines to learn logic step by step.
More recently, the Alphabet subsidiary DeepMind demonstrated a chess variant of AlphaGo, the program capable of teaching itself to play the ancient Chinese board game Go.
Now, a team of researchers at University College London went a step ahead and developed a chess algorithm, called SentiMATE, which can evaluate the quality of chess moves by analysing the reaction of expert commentators. Yes.
All one has to do is to make the algorithm to read the commentaries of top games of grandmasters. Winning moves and blunders obviously get a stark contrast reaction from the commentators and these highs and lows can be identified by the machine and use it to make a decision on whether to make a move or not.
Designing SentiMATE
For this experiment, the team analysed the text of 2,700 chess game commentaries available online. They pruned out commentary that didn’t relate to high-quality moves and examples that were too ambiguous. Then they used a special type of recurrent neural network and word embeddings, trained on another state-of-the-art model for analysing language.
Data from different chess websites was scraped, which included information regarding the moves being made, and a qualitative assessment of the moves themselves in the form of commentary, written by a wide range of chess players; resulting in a large database of moves with annotated commentary.
The data was cleaned so that only commentaries which were deemed to describe the ‘quality’ of the moves were employed, through the use of a preliminary ‘Quality vs Non- Quality’ classifier.
A novel Sentiment Analysis model was built, given that the nature of chess commentary is different to that of, for example, movie reviews, thus making sentiment classifiers such as that trained on the IMDb’s database unsuitable for this problem. 2,090 moves were annotated for their ‘sentiment’, and a bi-directional LSTM using stacked BERT and GloVe embeddings was trained on this commentary. The resulting dataset was then fed into novel evaluation function for chess moves trained on sentiment, which is tested against other chess engines.
The results of this research demonstrate that annotated moves and their respective sentiment is sufficient for the model to outperform a random player and a (restrained) depth-one DeepChess engine. Furthermore, the provided classification models exhibit high accuracy, making the expansion of the dataset both feasible and timely.
Future Direction
Strategic games like chess or GO are known for having a peek into the human intellect. So it is only obvious that if AI is modelled around these games, they only get better eventually being able to perform not so sophisticated jobs. For instance, NLP used for chess as discussed above can find itself get smarter at applications like conversational agents, chatbots, home pods, speech assistants, translation and many more.
