Smarter robot with better grip

Smarter robot with better grip

Problem being addressed

One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments. Despite this potential​,​ most of the robot learning systems today are deployed as a fixed policy and they are not being adapted after their deployment.

Solution

Adapting an image-based grasping policy to changes in background, object shape and appearance, lighting conditions, and robot morphology and kinematics. The researchers purposefully modify the robot and its environment, characteristic of the persistent change of the real world, and investigate its ability to adapt. Likewise, rather than proposing a new adaptation algorithm, with new complexity and caveats, they show how to successfully adapt robotic policies to substantial changes, using only the most basic components of existing off-policy reinforcement learning algorithms.

Advantages of this solution

This work is the first to demonstrate that simple fine-tuning of off-policy reinforcement learning can successfully adapt to substantial task, robot, and environment variations which were not present in the original training distribution. The suggested adaptation uses less than 0.2% of the data necessary to learn the task from scratch. This leads to substantial performance gains over the course of fine-tuning. These positive results hold in a limited continual learning setting, in which one can repeatedly fine-tune a single lineage of policies using data from a succession of new tasks.

Solution originally applied in these industries

engineering

Engineering

Possible New Application of the Work

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAYAAABw4pVUAAAOlXpUWHRSYXcgcHJvZmlsZSB0eXBlIGV4aWYAAHjarZprcuMwDoT/8xR7BL4fxyEJsmpvsMffD5STSTz2TJxMUrEcSYZINNBokDbrf//d5j/8pOyzianU3HK2/MQWm++8qfb6uY7OxvN6fnK8XXOfz5v3C55TgWO4fWDd7u+cT78+UG73u/H5vCnzZqfeDN0uvBkM+mTPG7kN8mYo+Ou8u/1vmr/e9PxhOre/PX3TU2lcl+7/jwVnSMJe8Mav4II9r/56Urj+On/5vCb/diaGdF7j7/4z76574MD3d3f+s/N2Pvxyx2XobVr5zk+38y499t/x0scROf/+ZP9xRMu/ufKB/7bUvdc1ux6JoxbzbVJvUznvuBGXxnA+lvkt/CXel/Pb+K2228mjhKkOYwf/NOfx+HbRietuu3WO002GGP3yhaP304dzrobim5+436njQ3TbFxNakFBBZYJc4LR/H4s7z236PB5WebI47vQOY2D8+dfcn/ju7ydDe2uYO2fru68Yl9f4YhiKnL5yFxC4ffNpOv515jrY+x8FNoBgOm6uTLDbcZkYyf2KrXBwDjYZbo32AtkVuRnARTw7MRgXQMBmF5LLzhbvi3P4sYJPZ+Q+RD9AwCWTvDBKH0PIgFO9PpvPFHfu9clfp6GXkyA5FKBpoQNWjClm8q0SQt2kkGJKKaeSamqp55BjTjnnkpWnegklllRyKaWWVnoNNdZUcy211lZ78y1AY8m03EqrrbXeeWiPHVud+zsnhh9hxJFGHmXU0UafhM+MM808y6yzzS5egkABRrIUqdKkL7cIpRVXWnmVVVdbfRNrO+y408677Lrb7u+o3VD9jNo9cn9Gzd1Q8wcova/8Qo3TpbyZcEonSTEDMR8diBdFgID2ipmtLkavyClmtvlgArzFKJOCI04RA8G4nE/bvWP3C7mnuBm8+ypu/hFyRqH7F8gZhe4Dcr/j9gA16YduwwFIsxCfwpCB9JOypc2yV4xbYthr8JZP4KVUhhZRTswVNveSJgvS4l3aENFeddQ+jHQqmuXlh0fzrQ+GmXREuTCw5sJOdhuds7XtDDqmmdISlxbUNELVc7PilsrMy55t7w7x9p1r19qsZt+O5v7Ed49fMLTdYySA6AMW5gbGGxRn7u9QvICEecnTw60FV0rOgdqVeqoR2bMJwGjiIKXaTnGP7P1uftc8fWkxZqKyjln30lEO1+V+4mPKnok4XGFSafsmZdbaKQy7ay29rxbyCHlKy0kRzZFHpxTqjL0HhFBbNXQEU+kxgTGu2Vr7oc+Z8U9MuaS5Z88UdfICwBGC2yn4LjUpYfsGKdWtNoZNmQcPPiJxR2PFM0byDufvJaXJgnYGZTj3NIr0nhLjlTktzyMW3eK2NPuuaSw3S21as51xople9nZ2pySpS3ayx5q9OJ9ng4hmD6G1VIuNNYjdsRMAMwpY2yNJBj4xK2YN+Rn2x4B/Pd7Nx4AfPJDpitgySwGKOJjqSL4WjFu8ER1s3EG+9pRb9VOdb2eszixwKQczgcoQV/vfpciYW4e/IHIGT4CtwPBRRngyM0OIX5/ezl0AO/VoetvMA2fUa/YLIaXHmlI75BXWrgsTthwyk5pKRxZE63CTzy2hAVsPZjowk7Eu984msonHskNpI4SVCJQ5hWIyc7FBkHti72fgxozmM4hgOHI+V+Sk77nEOG8XPwIMqjeI8T/Ephh/Qpi80lneIaw5+4vTxrz8o6wNkSzvzZn3DFfEkMZDHt12fxcjk88omRdxHmFJSGmPtOADCpbsbMeqxayJR8vucYpdUvMKpRNWIiGXlWWIG/Q7ZGmuQKVoARmAEpB9UOHazsebJmJvhD28yGwZEm0FpiqrZJmF6NR6uepctIfEQUj9MvVm8v1orjcog9qUilEXxx9oD9wtcNLnS28XNCBr+UDT5hFPDy1bDyD+hLA+X6O7Rn1KN8Q1RLRHJR2IxHHi9sMNv11XXnwwN/PbZD8eHSqmMbNympTl8GuKkiGI4Zo0ekFYItFHumBcms016yq5NtHNYU3gmG6FuEkHuBAGRoosn7CVGxAz3VlgP7TNXDOeLjrQZTdfKtOmwPgyw+DjMDFCsZ7qASP7AQWX5Gb01MM9IpVnITp9JGjRNpKoNKYQEwChhGxbz7kHxBKe7hQpPp2H71GsRswqeEhi9b0gaJCokRRk+llLxzLhZD3VBj4My1UBPjQa/E63hamdM0wwKdorcsBTzF3CwDTiEIWJzNaUNiqlmcohKQ/O2NIuk4JJOMgopKnV9MBvZ6QIt5Yll5CEFkRU7i2ePI2HRez2BAbXe+euaQnmoFWoo/q2o8rNlgjrQhgUdxKXcSfVF/FdMZk3WnVeq+xVNcI1vqo2uBZOqVS9fAJz1X7ug+09aTazEsbJfoh0BtcjlRv9SmIzBabbycACj6l3TmTbSALqaPHQWo7XwEgH6b2HGGLtLha/l3em/EUffTXvzB/z6oW8M0/z7MGRKN+VMYaTgWnH3PocRIqzYjSawoCPYx/adbhB49pScsOi/nOoNsV2REOpCKejjRnPYXHCAklZkW87k7QcNC9wP0/DezEtJ5CmszCyj8lPcE6bg/dCPaQfdiuCGMKJfMuFpJnW1JRJeoIiAnVyShUoIJU8nCPbG3w+In+kGRldZE00IxGcpOErP1FImnWGLgif5Xk5dZSbUzUFRz31DLd2Uadyi7ZGi3l5odvJgwyOQ58h2+TS2gqW0oifLBqmbj+aLqacJBjH00nX0p4cU5Xls1ZahnNCPgkp4kMa4XZNI+e6eLsEZyxi8KHYMr+6iy8ofB0ACUWjSkD6IdqtbrKlHVXbI+SD6Hl2y3WHjqb+YZ7mzw74+tF88thvXvngsHtffnQY7jJPpMszhz10F4/35pM7fnOX3vLbHQ/dZf6Be54YeuaTv8SXeRZgr8aX+XP0fD2+3uMou7lv9b4LvQP1y1313jVd0Ww+FsiEjm1WCpaFMUaOzmq5p8qcfg1WqOEqNFpaDmXZHOwpSi7xTDn0eFP6dCnXPd31zUzpCeoMR9XC4Ifcf2LS/LL5M5Pm0TC/Y9L8eeZfN2m+6sy/mTSv4/PYpPkJ5B9Nmn8TRScgX8TnST03P28eLg1jft48XBrFfLN5oO4jHzyScCD9nRczE8Wb4otCKS3NMUnzlp0LCDwX9B8KL6+obHXl1KWay2sd/REv5FyJRv0e5QhAnnjJUDiivLmd3uPm9iQ3t1/i9M6gebP4U4PmfojfNWiezflVg+bZnF81aP7mxK8aNF9F5W8GzaswPzNovhs39wbNlwTiF+SOeaZ3XpU75qPeqbsXXbqYO9J0ujVSXzF4CjMKd+zmMEWrReKW2aOnhdaOnh69tGngh5poMtJZ3IvI9U7HFkdNzXU7SO7GaGKiTnulQDpsG7S/Xlb3KfAbuWzR2ZWWPuiOR4hJis8tpKTbF3VPXXXszLm4gdJ3dCnWq/yGvCbdAd75sFhrFB/Y7g0R2/I4XlEFcjWiqhFmwHUK7Kqe+eKn05sf/VRFA8AM+7b+q3sdl0Wl9pvFy96dNQ2Td3tqDVvmWge+bP1kdObj8H4yOvNxeD8ZnXnmvFdHZ54579XRma9A+5XRma9A+5XRmVcD79nozKuB92x05idpcVkkWZ0mbdGNGmwmdy04O7tHq17XolfLQfcudJujePF83m56/tJy7wF1siI9NwJ+Yaggl35f5n59lds8XuZ+eZU7mqbLuZJlpQHPMJeabW32qBJoR6nMHoWiy42hTTlfCkrUjW6j9CqbofaRTU9bsBh4TC24lOes2V2uq+/FeJIfIaeGcJpDN7YCWrD0EjsQfNwyMz/eDv1syLdFe5Vn6HNVKdKTLgRV8F3CLLNWmRxttn31FJXvYy62CJIzIHPB1cwiLgAfXC5IS5m3IrxLKyvHUYgW+BlJt/Q7L30RXbO52esSKV0l6tVBvl4KH1dC8xjOWKmEScLEr7rVWHb2c3emv/ooqjeJiejnHMSk8yBtNhGYbKzJBj6/hgBtG3/eK2UwLQ3RRWVfJKsb6Wlt7mFRVVsdOXUhdnAkXhhVKJC1Cbp2TwqtE8ZyVHKuuTbdN5LlprilK64mtqHLxWpgDEJlFN36kDl1RWrGFojG0s7uwx/XBM3XFw+fHFEOG72gX/aKOq9MHo81W0k+xtWsTHLcx+3brN6lndty4/oWUweO+bYl4DNpQDQZYEVeSO4CCQlAOOeVQLzLUaYHA9on+o/GRFtOebcyyEZyp9MoFN1VS7q6acLMzYIynRAQ6Koxzk1tZUn0DHQTRHR1upwncdgwiEpdHtcv+BS4qRFTJwzN95ZUfl9RMZ+XVCzsBfFI8MTRCLyPFWqkdZswIVOa0Nhpcqavy54tjI2icvQiCLmcBY7GRrJw9Ai6FqubExWNpmlYzgZDJ/DgYXq6sYfsoptdItqejZ7oab+wYrk7rVhXOWdLcuLXDCmMSeCjDmODw0YwCrDoDgP5PHOKCDtCYI3KPSQPvqH3nfoFHe16k24M6Lo+UZwrs2EidunuttFtOvJBhV5bIctoMNFxcR+6EVpUUlp72OJupAkDWXNBArKG6sA0ty7cIhYZMFYWXt0MUodAMBEls02Ye7XWx7GSy6qtlEPZFW0/l4EZru+AJNmBQYIXAwU5Iq9mb8tcY7qp3+5zAqlpJD7KFvNgLwyOx2+6qs3g7UrCSLI7q9qRcYtWgzVdk+2y7kARrbObgsilYoQuum9FxunGUNRq6Jk+IbN9gRFKxbiHe0nJ1pix9atoODUfdEs+mdVzXykXkTWUDqtum8KBAVbHGM7zG36HoPLMYWVSGfNkgn4Zzq5yDPVqzflG27eK9PpVo2Escwi7pnzWYZP2JWQjYCjX3vbZI/Rt63BnPzlDK3LYHUJ1Vqv1UQ7mX3yv5iuGhkPvDBGr3/twkH8eBdoDrLUCQ1wgtK0rxWimwnR5LKsbFcU1XIDXIZKX1m7NHdNQZ6WZ/wMq4r6i7/lgQQAAAYVpQ0NQSUNDIHByb2ZpbGUAAHicfZE9SMNAHMVfU2tFKg7tIOKQoYqDBVFRR61CESqEWqFVB5NLP4QmDUmLi6PgWnDwY7Hq4OKsq4OrIAh+gDg5Oim6SIn/SwotYjw47se7e4+7d4BQLzHN6hgFNL1iphJxMZNdEYOv6EQAYQxjSmaWMStJSXiOr3v4+HoX41ne5/4cPWrOYoBPJJ5hhlkhXiee3KwYnPeJI6woq8TnxCMmXZD4keuKy2+cCw4LPDNiplNzxBFisdDGShuzoqkRTxBHVU2nfCHjssp5i7NWqrLmPfkLQzl9eYnrNAeQwAIWIUGEgio2UEIFMVp1UiykaD/u4e93/BK5FHJtgJFjHmVokB0/+B/87tbKj4+5SaE4EHix7Y9BILgLNGq2/X1s240TwP8MXOktf7kOTH+SXmtp0SOgdxu4uG5pyh5wuQP0PRmyKTuSn6aQzwPvZ/RNWSB8C3Svur0193H6AKSpq+QNcHAIDBUoe83j3V3tvf17ptnfD6IScrrpa2XVAAAABmJLR0QA/wD/AP+gvaeTAAAACXBIWXMAAC4jAAAuIwF4pT92AAAAB3RJTUUH5AYFEy0s9Q3dgwAABSdJREFUeNrtnUtoHVUch79zk5pESauVLkQrDbcboT4R8QXVbqqWdiFNxWq1WhUqqCRaHxhURDfaUHcVHwsfqBvFFqFqrEgD2oggiCjUWFNTbKBoizSt+MjfxZ20N7e5d85M5nXT37fKTc6ZO+f/zXnO5IwDGB4eplwuIxLjKuAVoCNCnj+B9a2KXSqsAZbEyLeypNgVihYJKRgSUjDUh9TnTGALMN8j7d/AU8CPEpIey4H1EdLvB3rUZBWnOS/l8aVCnbqECAkRElJMnISkw/cx832neUg6vA58ArRHyDPunDsgIemxP2qGgYEBzdQbsA+YiNDP7jneETjn/SVmNuWzhNTrXZ370swWAXM9kv8D/FQV5KuBN2h8g+ov4C5gUEL8r9zRmNlXA4s90nVLSPSaEqfp8W2zTkonIeGBXg5sDml+DgFrq5utuMQV0g7cDFzu2cbmzavAUFWZN1D/PscfwGvAf8HntfjdH18DPJ+pEOccZnYZ8CGwsIku9M+rhNwBvOyxgrE1o4s7/kHM7BxgJ5W7ac1Kp0eaeXmdXClK7QAebnIZhSfqWtaNClmxhHQqZF74zl9+07A3G14CvgHaQmb3g2kJOQLcVDVULBp7Mv6+CWBXlvOQWv4FdgfWi85BjzRjVT//7nnckcyHvbMB59w7ZjYGnFEnyVHn3M6qpZC+YB7TaDnkWDAdkJAYSyGTE8WwNMcFAR9ldX7q1COuVEQQKSFhwTSzFcCd0zQ/DngL2BYS8FUN0jwLPC0hIVdwzVXbB1xZJ/v5zrltIVf5RMy/nZJCLjCzrUxdJh+lsph4NPg8p0H+liIUYjYJ6QaW1vzuCqAf+KpZCjGbnstyEX8vIUI15KQ+vuiFmE19yA5gXdUM3AF7mfpY53vAedPIM+B9CUl2yDtkZotrgm2TtcLMcM71m9mWesPWJCZ2EjJ1vmGNmqUgzUSRy6FOXUKEhEiIkBAJERIiIUJCJERIiJAQCRESIiFCQiRESIiEiMKQ5D11i7ILTjOSxUMQSQmZC4xYER7bSJeOZhFSAs5Vg6M+REKEhEiIaJ5h7xDwi8I2LfsyFRKMaLsVdzVZarLqMdtn4kWYyUcRcpaZfQGcptBPyw7nXO9MpbT61gwzuwe4SHGvS5eZvQgcSL0PMbM2YJNi3pA24MGZNuu+nfrtwALFPJSNZtaZtpAS8Lhi7cU8YMNMaomPkFX47WMuKvSa2ZxUhASmn1SMI7GQymb8yQsxs+uobCcuovEEMbf0KIXUjkcV21hcCCxLVIiZXQzcoNjGpi9O515qUDt6abKddArGUjO7JBEhwat+blNMZ4QDHotaS0p1ascDFGSHtSZndXBxe9M6Te2YD9yfYyEOUdn0/2BCx1sHPJNTWVqBh5xzPb6Ljq21tcPMNhLthYhJ8zEnXr6SBP1UNr/Ma6Od+8zsOTx3yC7V1I4OoCfnaj6Y8PGOAN/mWJ7TgXt9+5LaPuRu4OychXyaaM9aCcSunMvUa2btcYQ8kvOJjzrnfk7ygEHb/VnO5VoA3OotpKWlhSDDopxP/OuUjpt3DYHKmmBou9UK0NXVhZkV4QbUCjPbzImNj5NiSQHKVgZWAttDhZjZMuDSApx0O5UXj81Wep1z2xsNgSf7kGs1h8uEa4Lb4aFC2hSrTCgR8j8mk0J+UKwyYcQ5d9hHyAfAr4pX6rwQtoRSAhgfHz8G3AIcVsxS410qL0kOH/aOjY1RLpd3U3kQbhNwPRn8P90pwl7gTeBtn8T/A2LPQqttuQqjAAAAAElFTkSuQmCC

Technology Companies

The base model achieves a success rate of 86% on the baseline grasping task and is able to pick up labeled objects with 98% success rate; this could be a great automation solution for warehouses with constantly changing environment to minimize operational errors.

Author of original research described in this blitzcard:

question-mark-icon

Name of the author who conducted the original research that this blitzcard is based on.

Source URL: #############check-icon


search-iconBrowse all blitzcards