A group of researchers at New York University has developed and launched Anysense, an iOS app optimized for collecting visual training data for generalizable robotics models. Raunaq Bhirangi, Zeyu Bian, Venkatesh Pattabiraman, Haritheja Etukuru, Mehmet Enes Erciyes, Nur Muhammad Mahi Shafiullah, and Professor Lerrel Pinto have developed the Anysense app.
Late last year, Professor Pinto and his team launched an open source research project called the Robot Utility Model (RAM), which attempts to generalize robot training, avoiding the need for developers to train thousands of examples of tasks. Instead, you can successfully operate in zero-shot situations or in previously invisible environments.
To support the capture of ram training data, NYU’s generalizable robotics and AI lab (Grail Lab) has created “The Stick.” This is an open source 3D printed gripper device that uses the iPhone for visual feedback.
“The Stick” is an inexpensive gripper used to train ram data. I use the iPhone 12 Pro and standard ready-made “Reacher.” |Credit: Robot Utility Model Team
AnySense app released on the Apple App Store
The AnySense app is designed for the robotics community for the collection and learning of multisensory data, and is a direct result of the RUM project. Training such generalizable models is bottlenecked by the inability to collect diverse and high quality data in the real world, NYU researchers argued.
One way to deal with this bottleneck is to create a tool that combines cheap, accessible sensors with a scalable, intuitive data collection interface, they said.
Below are three notable features of the Anysense iPhone application:
Integrates the iPhone’s sensor with external multisensory inputs via Bluetooth and wired interfaces. It can interface with Anyskin, a versatile tactile sensor that can measure multi-axis contact forces. It is completely open source and is available to the robotics community for the collection and learning of multisensory data.
The AnySense app can be downloaded from the Apple App Store.
Anyskin provides tactile feedback
The research team developed a touch-sensitive pad called Anyskin. This can be used in the Anysense app to provide gripping feedback. Anyskin is easy to assemble, compatible with a variety of robot end effectors, and is designed to be generalized to new skin instances.
Anyskin contacts through distortions in the magnetic field produced by magnetized iron particles on sensory surfaces. NYU researchers say the flexible surface is physically isolated from the electronics, allowing for easy replacement in the event of breakage.
