Multimodal AI makes this robot vacuum cleaner smart enough to pick up your socks
As long ago as 1962, The Jetsons imagined the vacuum cleaner as more than a simple appliance. Rosie was a friendly, helpful, and intelligent presence in the household, helping the family tidy up after themselves. That vision is no longer some far-flung fantasy of the future. It’s real and ready for mass production thanks to a specific development in AI.
This May, Roborock is launching Saros Z70, the world’s first mass-produced robot vacuum cleaner with an OmniGrip intelligent, foldable robotic arm that can pick up your socks and put them back where they need to be.
What makes this possible is “multimodal AI” — an intelligence capable of making sense of data from multiple modalities, such as infrared cameras and sensors. Instead of relying on a single input such as a laser, it learns from a fusion of information from all sorts of data points.
In the case of the Saros Z70, it uses 3D point cloud data (similar to what’s used for facial recognition) to see and measure the depth between objects in your home. It then continuously updates its map of every room in real time so the device can rapidly adapt to new furniture layouts and obstacles.
But as astonishing as this technology is, it didn’t come from nowhere. The first big shift in this type of AI was when Roborock applied its StarSight Autonomous System, which is a navigation and obstacle avoidance system, so that robot vacuums could use a laser to navigate around the house.
This next iteration of that system uses a sampling frequency that’s 21 times higher than traditional LDS solutions, scanning 21,600 sensor points to identify objects as small as two square centimeters, so it can not just vacuum but actively organize your home for you. The result is that if it comes across a piece of crumpled paper on your floor, it’s intelligent enough to know it needs to be treated differently than a sock.
At the heart of this massive leap forward is a multimodal AI that’s designed to ensure this generation of robot vacuum will continue to become even smarter, safer and friendlier than it already is.
“The Saros Z70 can help users grasp the future with intelligence,” said Richard Chang, founder and CEO, Roborock. “It can be a user’s home helper, companion, and assistant — whatever it needs to be.”
The magic of AI powered by multimodal perception
Robotic arms are already widely used in all sorts of industrial settings. But for a robot vacuum to operate a 5-axis robotic arm in a home, it needs to be intelligent enough to navigate not just all your furniture but also whatever surprises your kids and pets might throw its way. That takes multimodal perception – an ability to see the home in a multitude of different ways and the intelligence to make sense of all that data.
The Saros Z70 accomplishes this by combining 3D Time-of-Flight (ToF) sensors and visual recognition to create a real-time 3D map of everything around it so it can detect objects, recognize spatial relationships between them, and intelligently adapt.
It uses RGB and infrared cameras in the arm and on its chassis to perceive depth more accurately than previous LIDAR systems. The lateral wall sensor extends the field of vision made possible by a single-point laser to eliminate blind spots so the vacuum can keep cleaning even when it needs to move alongside something like a cable on the floor. And its AI uses a range of sensors to make sure the foldable arm is only used in the safest possible ways.
For instance, it can clean areas that were previously obstructed and lift socks, small towels, tissue papers, and sandals. But its sensors also prevent potential issues.
The weight sensor in the arm prevents its gentle grip from lifting anything heavier than 300 grams, the anti-pinch and pressure sensors make sure you can interact with the arm safely, and the upward spatial sensor keeps the arm from unfurling if a child, cat, or coffee table is 45 cm above it.
The combination of all these different data sources ensures the device can dynamically react to the specific objects and obstacles in any given home. It needs multimodal AI to align and fuse all these different inputs into a unified view of reality that is greater than the sum of its parts.
Each one of these high-definition sensors provides valuable context, but it’s the AI in the middle that can turn all that data into the intelligence needed to safely clean a home that’s constantly changing with kids and pets running around.
A friendly robot that grows smarter all the time
Remarkably, all those sensors and cameras are packed into the thinnest Roborock yet, a mere 7.98 cm (3.14 inches) tall. So it’s smart enough to tell the difference between a cat and a lost slipper, but it’s also thin enough to clean under the sofa.
The upshot of all this intelligence is a robot vacuum that’s capable of learning your habits automatically so it knows when to pick something up, where to put it and whether it needs to use suction or mopping for optimal results on all the different routes it takes.
The device can also be voice-activated and used to keep an eye on your pets, using its cameras to find them wherever they are in the house and even send you pictures.
It’s a truly friendly assistant that is constantly becoming more intelligent the more it gets to know you. It adapts to your home layouts when they change, reacts to objects when they’re out of place and is continuously updated over-the-air to detect new objects.
It already knows, for instance, not to pick up a pair of scissors while being smart enough to put a crumpled piece of paper in the bin.
All of which means that the vision of the multi-tasking robot tending to your home is no longer just a vision. There’s already a friendly AI-powered robot you can talk to while it puts your socks away and it’s called the Saros Z70.
Explore the possibilities of the Saros Z70
