ArcadeImages/Alamy Stock Photo
By Matthew Hutson
STOCKHOLM—Artificial intelligence (AI) can kick butt in games such as Pong and Space Invaders, but it comes off like a common n00b when playing Ms. Pac-Man (pictured). Now, by making AI play six classic arcade games, researchers are closer to figuring out why thinking machines excel at some games and stink at others, they reported last month at the International Conference on Machine Learning here.
The team developed a new system for visualizing how Atari-playing AIs operate. They chose Atari because the games are relatively simple and a frequent focus for researchers developing “reinforcement learning” algorithms, AIs that learn behaviors through trial and error. An AI “sees” the screen (as an input of ones and zeroes) and at first randomly responds with commands for “left,” “right,” “fire,” and so on, slowly shaping its strategy as it receives points for certain actions. In Space Invaders, the AI moves a ship back and forth across the bottom of a screen while shooting descending aliens and dodging their projectiles.
After thousands of practice games, an AI can best human performance at Space Invaders. To understand its strategy, the team blurred little sections of the screen, obscuring the ship or aliens or projectiles or shields or empty space. If blurring a section threw off an AI, the AI must have been paying a lot of attention to that area of the screen. The system then creates “saliency maps,” videos where the most critical screen areas are highlighted with colored blobs so that an observer can see where the game-playing AI is focusing.
The researchers knew that an AI playing Space Invaders appeared to aim its gun at incoming aliens, but they didn’t know whether it was spraying gunfire at clusters of aliens or aiming at individual targets. The saliency maps revealed that the AI tracks specific aliens, the team reported. (Check out the red blob in the center-left of the video below.)
You might not care about an AI’s strategies at Atari games, but the system may someday be used to highlight how other algorithms see and act on the world. What does an autonomous car focus on when it changes lanes? How does a home care robot search for a pill bottle?
The saliency maps also help debug algorithms. In one example, you can see that an AI playing Ms. Pac-Man—in which a yellow character must eats pellets in a maze while avoiding ghosts—fails because it doesn’t pay attention to the ghosts. If a car or robot doesn’t do its job well, you want to know what it’s missing so you can train it better. How to teach a robot to fear ghosts is another problem entirely.
Source: Science Mag