Tic Tac Toe Heuristic or Neural Network

I’ve finally I’ve decided to delve a bit into neural works as I’ve seen some amazing things done with them.

In order to learn more about neural networks I decided to code a tictactoe Artificial Intelligence (AI) and maybe apply what I’ve learned to something a bit more complicated. I used neural networks by having each board as input neurons with the proper/best move coded as output neurons. Which means I have 9 inputs 10 hidden neurons and 9 outputs. The inputs and outputs are between -1 and 1 where -1 represents letter x and 1 represents the letter o and zero is an empty spot on the tictactoe board. Coding it this way works fine, I was able to train the AI by entering perfect training data.
Having each input producing an exact move/output. The problem with this way of training the AI is that you have to precalculate everything. The AI isn’t really learning as it plays and in the end you might as well be using a heuristic as the rules (training data) to win

So what would be the best dynamic way to code a tictactoe neural network?

Why do humans learn so much faster, and by faster I mean play fewer games to understand the rules. Part of the reason could be that they are learning from their mistakes/losses. So if your AI loses, make sure to record what your opponent did. This way you learn even when you lose. This will cut down the amount of time required to train the neural network.

So this is how I would do this:
let NN = empty neural network
let b = current empty board
let b_array = array of boards (previous moves)

while move = NN->calculate(b) && !done
{
// no move trained? make a random move
if( !move ) {
b = random_move( b );
}
if( xs_won( b ) )
{
done = true;
}
else if( ys_won( b ) )
{
// add opponents data, learn from his win.
done = true;
}
else if( draw(b) )
{
// there could be a draw, and you both did nothing wrong.. perhaps.
}
}

This method of learning is interesting, however I can see there are flaws if a player makes a bad move, but your opponent makes a worse move. This could cause your neural network to learn moves that are not necessarily good moves and for draw games maybe you both made poor moves. How do you make sure data that is trained that is wrong doesn’t poison the whole network?

That’s all for now.

One thought on “Tic Tac Toe Heuristic or Neural Network”

1. curtisr says:

Coding the Heuristic for tictactoe:
First write a function to win or block your opponent from winning as they both check the same thing. This function checks horizontal, verticle and diagonal rows for 2x’s or 2o’s and 1 space. Depending on if you are winning or blocking.
You then call this function with the “winning” flag to attempt to win first. If you can’t win try to block next.

Then finally place your x or o in a position that will give you the best advantage.