ONLamp.com
oreilly.comSafari Books Online.Conferences.

advertisement


Four Cool Ways to Use Neural Networks in Games

by David M. Bourg, Glenn Seemann, authors of AI for Game Developers
09/30/2004

In our book, AI for Game Developers, we cover many different AI techniques that are used in games. Many of the techniques we cover, such as chasing and evading, pathfinding, finite state machines, and rules-based systems, among others, have obvious applications in games. However, some of the other techniques we cover, such as neural networks, genetic algorithms, and Bayesian techniques, are not as familiar and thus their applications in games may not be as obvious. Nonetheless, these latter techniques offer compelling capabilities when applied in games and they are quickly gaining popularity, as evidenced by their appearances in game development literature, conferences, and indeed the games.

Throughout our book we give you multiple code examples and additional ideas of how you can apply all of the techniques we cover in your own games. In this article we'll look at several examples of how one of the less familiar techniques, namely neural networks, can be applied in games.

Neural Networks

Most writings on neural networks start with a biological description of neurons in our brains as a metaphor for how artificial neural networks function. Indeed, we give just such a metaphorical description in our book. However, sometimes it's more helpful to think of neural networks in a less biological sense. Specifically, you can think of a neural network as a mathematical function approximator. Input to the network represents independent variables, while the output represents the dependant variable or variables. The network itself is then a function giving one unique set of output for the given input.

Related Reading

AI for Game Developers
By David M. Bourg, Glenn Seemann

You can also think of a neural network as a way to map parameters from one space, an input space, to another space, the output space. The mapping may be highly nonlinear and depends on the structure of the neural network and how it was trained.

For games, neural networks offer some key advantages over more traditional AI techniques. First, using a neural network may allow game developers to simplify the coding of complex state machines or rules-based systems by relegating key decision-making processes to one or more trained neural networks. Second, neural networks offer the potential for the game's AI to adapt as the game is played. This is a rather intriguing possibility and is a very popular subject in the game AI community at this time.

In Chapter 14, "Neural Networks," of our book, we cover the most widely applied type of neural network architecture--multi-layer, feed-forward networks with back propagation training--in detail, giving you both descriptions and code samples (examples.oreilly.com/ai) of every part of the system. For the purposes of this article, let's focus on how we can apply such neural networks without getting into implementation details.

Control

Neural networks are often used as neural controllers for robotics applications. In these cases, the robot's sensory system provides relevant inputs to the neural controller and the neural controller's output, which can consist of one or more output nodes, sends the proper responses to the robot's motor control system. For example, a neural controller for a robot tank might take three inputs, each indicating whether or not an obstacle is sensed in front of or to either side of the robot. (The range to each sensed obstacle may also be input.) The neural controller may have two outputs that control the direction of motion of its left and right tracks. One output node may set the left track to move forward or reverse, while the other may set the right track to move forward or reverse. The combination of the resulting outputs will have the robot either move forward or backwards, turn left, or turn right.

Very similar situations to this robot arise in games. You may in fact have a computer-controlled, half-track mechanized unit in your game. Or perhaps, you want to use a neural network to handle the flight controls for a space ship or an aircraft. In each of these cases, you'll have one or more input neurons and one or more output neurons that will control the unit's thrust, wheels, tracks, or whatever means of locomotion you're simulating.

Indeed, some popular racing games have used neural networks to control the driving of the computer opponent's race car. Moreover, these neural networks were trained by actually observing game developers race around the courses. Colin McRae Rally is an often-cited example of the application of such learning-by-example techniques. Check out this article on the Generation5 web site for more information on Colin McRae Rally's use of artificial intelligence.

Threat Assessment

As another example, say you're writing a strategy simulation game where the player has to build technology and train units to fend off or attack the computer-controlled enemy. Let's say you decide to use a neural network in an effort to give the computer-controlled army some means of predicting the type of threat presented by the player at any give time during game play.

The inputs to this network include the number of enemy ground units (those of the player), the number of enemy aerial units, an indication if the grounds units are on the move, an indication if the aerial units are on the move, the range to the ground units, and the range to the aerial units. The outputs consist of neurons that indicate one of four possible threats, including an aerial threat, a ground threat, both an aerial and ground threat, or no threat. Given appropriate data during game play and a means of assessing the performance of the network, such a network could be used to predict what, if any, sort of attack was imminent. Once the threat has been assessed, the computer could take the appropriate action. These could include deployment of ground or aerial forces, shoring up defenses, putting foot soldiers on high alert, or carrying on as usual, assuming no threat.

This approach would require in-game training and validation of the network, but could potentially tune itself to the playing style of the player. Further, you are alleviated of the task of figuring out all of the possible scenarios and thresholds if you were to use a rules-based or finite-state-machine-type architecture for this task.

Attack or Flee

Let's say you have a persistent role-playing game and you decide to use a neural network to control how certain creatures in the game behave. Now let's assume you're going to use a neural network to handle the decision-making process of the creature; that is, whether the creature will attack, evade, or wander, depending on whether or not there's an enemy (a player) in the creature's proximity.

We have four inputs in this example: the number of like-creatures in proximity to the creature who's making the decision (this is an indication of whether or not the creature is traveling in a group or alone); a measure of the creature's hit points or health; an indication as to whether or not the enemy is currently engaged in combat with another creature; and finally, the range to the enemy.

We could make this example a little more sophisticated by adding more inputs such as the class of the enemy; for example, whether or not the enemy is a mage or fighter. Such a consideration would be important to a creature whose attack strategies and defenses are better suited against one type of class or another. Actually determining the enemy's class could be achieved by "cheating." Better yet, another neural network or Bayesian analysis could be used to predict the enemy's class, adding a bit more uncertainty to the whole process.

During game play we could allow the neural network to evolve using a sort of positive or negative reinforcement training. If the creature takes an action suggested by the network and its death results, then we can pretty much assume that it was a bad decision. Therefore, we could train the neural network a little to suppress that decision given the particular set of input conditions. We could take a similar approach in the event the decision was found to be a good one. We discuss and give examples of how to train such neural networks in Chapter 14 of our book. Also, we discuss a similar example (examples.oreilly.com/ai) in Chapter 15 of the book, where we use genetic algorithms techniques to evolve suitable computer opponents.

Anticipation

In this example, we aim to predict a human opponent's next strike in a martial arts fighting game. The basic assumption is that the player will try and use combinations of strikes in an effort to find the most effective combination. These combinations could be something like low kick, low kick, high kick; or punch, punch, power kick; and so on. We'd like the computer opponent to somehow learn to anticipate which strike the player will throw next, given some history of the player's strike patterns. If the computer can anticipate the next strike, then it can throw an appropriate counter-strike, or block, or take evasive action like side-stepping or back-stepping. This will add a higher level of realism to the combat simulation and present new challenges for the player.

In our book, we show how this problem can be solved using a purely rules-based approach (Chapter 11) and alternatively, a Bayesian approach (Chapter 13). This same problem can also be solved using neural networks. To keep things simple for discussion purposes, let's assume that the player's strikes can be classified as punch, low kick, or high kick. And we're going to track three-strike combinations. We can set up a neural network that takes the three previously thrown strikes as input and outputs a single prediction of the next anticipated strike. Training of such a neural network, using a method like the one we discuss in the book, would have to take place during game play so the neural network can learn the player's strike pattern.

There are many options here for constructing such a neural network. We mention using a history of three thrown strikes as input, but that's not set in stone. You could try using more in an effort to achieve more accurate results.

Conclusion

There are many other examples of how you can use neural networks in games. We discuss these four here because we find them rather compelling and well suited to neural networks. If you'd like to learn more about the underlying details for actually implementing neural networks in games, then we encourage you to check out AI for Game Developers.

There are also many other resources available on the Internet related to AI in general, and game AI in particular, that you might want to check out. Some of the sites we found most interesting include:

David M. Bourg performs computer simulations and develops analysis tools that measure such things as hovercraft performance and the effect of waves on the motion of ships and boats.

Glenn Seemann is a veteran game programmer with over a dozen games to his credit, for Mac and Windows systems. He's a co-founder with David Bourg of Crescent Vision Interactive, a game development company specializing in cross-platform games.


In July 2004, O'Reilly Media, Inc., released AI for Game Developers.


Return to ONLamp.com



Sponsored by: