{"id":686778,"date":"2025-08-23T18:45:15","date_gmt":"2025-08-23T15:45:15","guid":{"rendered":"https:\/\/buradabiliyorum.com\/en\/robots-can-now-learn-to-use-tools-just-by-watching-us\/"},"modified":"2025-08-23T18:45:15","modified_gmt":"2025-08-23T15:45:15","slug":"robots-can-now-learn-to-use-tools-just-by-watching-us","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/robots-can-now-learn-to-use-tools-just-by-watching-us\/","title":{"rendered":"Robots can now learn to use tools\u2014just by watching us"},"content":{"rendered":"
\n
\n
\n
\n \"Robots
\n Credit: UIUC HCA LAB
\n <\/figcaption><\/figure>\n<\/p><\/div>\n<\/div>\n

Despite decades of progress, most robots are still programmed for specific, repetitive tasks. They struggle with the unexpected and can’t adapt to new situations without painstaking reprogramming. But what if they could learn to use tools as naturally as a child does by watching videos?<\/p>\n

I still remember the first time I saw one of our lab’s robots flip an egg in a frying pan. It wasn’t pre-programmed. No one was controlling it with a joystick. The robot had simply watched a video of a human doing it, and then did it itself. For someone who has spent years thinking about how to make robots more adaptable, that moment was thrilling.<\/p>\n

Our team at the University of Illinois Urbana-Champaign, together with collaborators at Columbia University and UT Austin, has been exploring that very question. Could robots watch someone hammer a nail or scoop a meatball, and then figure out how to do it themselves, without costly sensors, motion capture suits, or hours of remote teleoperation?<\/p>\n

That idea led us to create a new framework we call “Tool-as-Interface<\/a>,” currently available<\/a> on the arXiv<\/i> preprint server. The goal is straightforward: teach robots complex, dynamic tool-use skills using nothing more than ordinary videos of people doing everyday tasks. All it takes is two camera views of the action, something you could capture with a couple of smartphones.<\/p>\n

\n
\n
\n
\n
\n
\n <\/p>\n

\n