Teaching robots the human way
Problem being addressed
Natural language is perhaps the most versatile and intuitive way for humans to communicate tasks to a robot. However, each task must be specified with a goal image—something that is not practical in open-world environments.
Combining the setting of open-ended robotic manipulation with open-ended human language conditioning. The researchers train a single policy to solve image or language goals, then use only language conditioning at test time. They introduce a simple transfer learning augmentation, applicable to any language conditioned policy and find that this significantly improves downstream robotic manipulation.
Advantages of this solution
The method reduces the cost of language pairing to less than 1% of collected robot experience, with the majority of control still learned via self-supervised imitation. At test time, a single agent trained in this manner can perform many different robotic manipulation skills in a row in a 3D environment, directly from images, and specified only with natural language. Importantly, this same technique allows the agent to follow thousands of novel instructions in zero shot, across 16 different languages.
Solution originally applied in these industries
Possible New Application of the Work
Robots are currently used in the epicentres of natural disasters, and with better human control they will be able to execute more complex rescue tasks.
Voice-controlled robots can be widely used by the patients with physical restrictions that need assistance with simple tasks; the robots should be able to understand the request, since the actions can't be fully programmed and automated like in a manufacturing cycle.
Source URL: #############