Machines vs. Humans: Who do you want navigating your train?

The reality of the world we live in today is that there are machines currently in existence, and still being developed, that are able to perform tasks much more efficiently than we are as humans. The ability of these machines to continuously learn from their mistakes and make improvements is a concept called reinforcement learning; a concept researchers at the Harris Institute for Assured Information (HIAI) are very interested in. A recent experiment using model trains, a newly created visualization system, and a reinforcement learning algorithm helped to show just how much efficiency can be increased when these learning machines are put into action.

The experiment was simple. A path was created for the trains to follow. Neither the humans nor the machine had any prior knowledge of the path. Both were given 20 opportunities to navigate their train through the course, as quickly as possible, without the train falling off the track. The machine used the reinforcement learning algorithm, the humans used their brains.

“The big idea was to show that the algorithm is more efficient than the humans,” said Tapas Joshi, a research assistant working closely with this experiment. “Humans did 20 trials and failed 10 times. The trains did 20 trials and failed only once, but got faster each time.”

Because one of the main goals of this experiment was for the train's performance to improve over time with the reinforcement learning algorithm, the fact that each machine-controlled lap continually got faster indicates success. The algorithm was able to identify where the train fell over, why the train fell over, and adjust its speed accordingly for the next round. This learning allowed the algorithm to out-perform the humans not only in laps completed but also in efficiency.

“The machine took 19 seconds and the best for the humans was 21 seconds; a whole two second difference,” said Joshi.

This was a learning exercise for both the algorithm and the human. Having no prior knowledge, both were expected to come in and adapt to the situation. As we can see from the results, the algorithm’s ability to learn surpassed the human’s. This experiment is just one example of the advancements being made to machines and machine learning. Joshi and Nima Aghli, the research assistant leading this project, are making big steps in the field of reinforcement learning. If these trends continue, there’s no telling what capabilities will be discovered.