User Tools

Site Tools


In the last years I was part of several complex public demonstrations where we programmed our robots to carry out various household chores. We believe that by building such systems and thus getting confronted with the real-world problems, we will find solutions that will scale better and do so also faster. Below is the list with the demonstrations and links to the papers and code (as far as possible).

Robots Making Sandwiches and Popcorn.
To come.

Autonomous Drink Serving This video shows a demonstration of a PR2 and a TurtleBot working together in an autonomous drink serving application. A drink is ordered through a web interface and immediately the TurtleBut – called BusBot – is called to the kitchen. Meanwhile the PR2 already opens the fridge, finds the ordered drink and retrieves it. The PR2 then places the drink in a drinkholder on top of the BusBot. Now, the BusBot gets the autonomously navigates to the person's location and delivers the drink. This demonstration was done at Bosch RTC and also shown during the industrial exhibition at IROS2011. [ code ] [ media ]

Robotic Roommates Shopping for and Preparing Bavarian Breakfast In the first part of the video below, you can see TUM-James, a PR2 Beta Programm robot, and TUM-Rosie cooperatively preparing the traditional Bavarian Weisswurst Frühstück (Bavarian Sausage Breakfast). TUM-Rosie is collecting the sausages, putting them into the pot with boiling water, waiting for them to be cooked and, finally, finding and getting them out of the pot into the serving bowl. TUM-James is meanwhile slicing the french baguette using a regular electric bread slicer and in the end serving the sausages and the bread to the class of highly regarded roboticists. In the second part of the video, TUM-James is shown simulating the shopping task, bringing the groceries home and placing them to the places they are meant to be stored in. It uses 3D perception algorithms from PCL in order to detect object candidates in the storage rack and ODUfinder system to recognize them. Grasping and manipulation of the shopping basket is done using James's haptic capabilities. Finally, to infer where do the objects belong to, TUM-James queries KnowRob system and computes the maximum WUP similarity (aka relatedness) to the other objects in the kitchen's ontology. Shopping part of the demonstration was accepted as video publication at AAAI Video Competition 2011.

Robotic Roommates Making Pancakes In this video we show a public experiment that shows two robots making pancakes using web instructions. In the experiment, the robots retrieve instructions for making pancakes from the World Wide Web and generate robot action plans from the instructions. This task is jointly performed by two autonomous robots: The TUM-James opens and closes cupboards and drawers, takes a pancake mix from the refrigerator, and hands it to the TUM-Rosie. TUM-Rosie cooks and flips the pancakes, and then delivers them back to the TUM-James. While the robot plans in the scenario are all percept-guided, they are also limited in different ways and rely on manually implemented sub-plans for parts of the task. [ pdf ] [ media ]

people/pangercic/demos.txt · Last modified: 2011/11/20 19:52 by pangerci