This GTA V AI Learning To Drive Is Absolute Chaos And We Can’t Stop Watching

“Charles” is a convolutional neural network that basically learns to drive by crashing into stuff and trying things out – watching the AI car smashing around Los Santos in GTA V makes for surprisingly addictive viewing

Jack Leslie

25 May 2017

Twitch channel sentdex has been live streaming Charles in action, crashing into whatever it can in GTA V in a bid to learn how to drive.

Some of the streams have lasted over 24 hours, in part due to Charles now being able to be respawned when he drives into lakes or rivers, or gets stuck in someone’s back garden.

Sentdex also posts footage of the AI self-driving on YouTube - there’s a whole series about it. The whole thing makes for hilarious viewing, especially with Michael (who’s sat in the car) making snarky comments about other people’s driving.

Charles “takes all actions based on single frames at a time, and bases his decisions on just pixel data. Charles only sees exactly what you see”.

The aim in the future is to give Charles some short-term memory, which may result in some less-crashy driving. For now, though, it definitely makes for amusing viewing – the tens of thousands watching on Twitch and YouTube clearly agree with us, too.

Comments

It already drives better than some people I know in real life

So is it constantly learning?

It’s a bit hard to learn anything when you don’t have a memory, don’t you think? :P

EDIT: Scratch that, it is indeed learning.

Next thing you know they’ll start using GTA V as a simulation in the UK hazard perception test.

They actually might considering how it is being used for other things such as autonomous vehicle development

I can approve

As a game dev student, this really amazes me. Making in-game AI is one thing (it basically knows everything that’s happening, and has a set of points to follow), but making an AI that plays the game based only on “visual” info, now that’s impressive.

It might be going all over the place and crashing into almost everything, but you can clearly see it’s trying to follow lines and avoid anything resembling a vehicle.