As capable as robots are, the original animals they were designed from are always much, much better. This is partly because it is difficult to learn how to walk like a dog directly from a dog. However, these studies from Google’s AI labs make it a lot easier.
The aim of this research, in collaboration with UC Berkeley, was to find a way to efficiently and automatically transfer “agile behavior” such as a light-footed trot or spin from its source (a good dog) to a four-legged friend. Something like this has been done before, but as the researchers' blog post points out, the established training process can often "require a lot of expertise and often involve a lengthy reward tuning process for any skill you want".
Of course, this doesn't scale well, but this manual tuning is necessary to ensure that the animal's movements are well approximated by the robot. Even a very dog-like robot isn't actually a dog, and the way a dog moves may not exactly match the way the robot should, which may cause it to fall, enclose, or otherwise fail .
The Google AI project fixes this problem by adding a little controlled chaos to the normal order. Usually the dog's movements are recorded and important points such as feet and joints are carefully monitored. These points would be approximated in a digital simulation to that of the robot, in which a virtual version of the robot tries to imitate the movements of the dog with its own and learn in the process.
So far, so good, but the real problem arises when you try to use the results of this simulation to control an actual robot. The real world is not a 2D plane with idealized rules of friction and all that. Unfortunately, this means that uncorrected simulation-based gaits tend to lead a robot straight into the ground.
To prevent this, the researchers introduced a random element into the physical parameters used in the simulation, which means that the virtual robot weighs more or has weaker motors or experiences greater friction with the ground. The machine learning model, which describes how to walk, must take into account all kinds of small deviations and the complications that they cause across the board – and how to counteract them.
Learning to adapt to this randomness made the walking method learned much more robust in the real world, which resulted in a passable imitation of the walking of the target dog and even more complicated movements such as twists and turns, without manual intervention and just a little more virtual training.
Of course, the mix could still be adjusted manually if desired, but it looks like this is a big improvement over what could previously be done fully automatically.
In another research project described in the same post, another group of researchers describes a robot that teaches itself to walk independently, but is infused with intelligence to prevent it from walking and walking outside its designated area picks up when falling. With these basic skills, the robot was able to stroll through its training area continuously without human intervention and learn respectable locomotion skills.
The paper on learning agile animal behavior can be read here, while the one on robots learning to walk independently (a collaboration with Berkeley and the Georgia Institute of Technology) is here.
