Ecovacs robot vacuums, which have been found to suffer from critical cybersecurity flaws, are collecting photos, videos and voice recordings – taken inside customers' houses – to train the company's AI models.
The Chinese home robotics company, which sells a range of popular Deebot models in Australia, said its users are "willingly participating" in a product improvement program.
When users opt into this program through the Ecovacs smartphone app, they are not told what data will be collected, only that it will "help us strengthen the improvement of product functions and attached quality".
Users are instructed to click "above" to read the specifics, however there is no link available on that page.
Ecovacs's privacy policy – available elsewhere in the app – allows for blanket collection of user data for research purposes, including:
- The 2D or 3D map of the user's house generated by the device
- Voice recordings from the device's microphone
- Photos or videos recorded by the device's camera
It also states that voice recordings, videos and photos that are deleted via the app may continue to be held and used by Ecovacs.
An Ecovacs spokesperson confirmed the company uses the data collected as part of its product improvement program to train its AI models.
Critical cybersecurity flaws – allowing some Ecovacs models to be hacked from afar – have cast doubt on the company's ability to protect this sensitive information.
Cybersecurity researcher Dennis Giese reported the problems to the company last year after he found a series of basic errors putting Ecovacs customers' privacy at risk.
"If their robots are broken like that," he asked, "how does their back-end [server] look?
"Even if the company's not malicious, they might be the victim themselves of corporate espionage or nation state actors."
Ecovacs — which is valued at $4.6 billion — said it is "proactively exploring more comprehensive testing methods" and committed to fixing the security issues in its flagship robot vacuum in November.
Ecovacs says it anonymises the data
In a blog post from 2020, two engineers in the Ecovacs Robotics AI department described a problem they'd been facing.
"Building a deep learning model without large amounts of data is like making a house without blueprints," wrote Liang Bao and Chengqi Lv.
"Due to the unique ground-view perspective and uncommon object categories, we cannot find any public dataset which fit our needs.
"Therefore, we first cooperated with many institutions to collect data from all over the world."
A company spokesperson told the ABC this pre-launch dataset did not involve "real user household information".
But since the products have been launched, they confirmed that data from users who had opted into its "Product Improvement Program" was being used for training its AI model.
"During this data collection, we anonymise user information at the machine level, ensuring that only the anonymised data is uploaded to our servers," the spokesperson said in a statement.
"We have implemented strict access management protocols for viewing and utilising this anonymised user data."
Intimate photos shared on social media
Imagery from robot vacuums has been leaked before. In 2022, intimate photos taken by iRobot devices were shared on Facebook, infamously including one of a person sitting on the toilet.
The robots that had taken them were, in this case, part of a testing program that users had opted into.
A company spokesperson told MIT Tech Review that they were "special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase".
The devices were physically labelled with bright green stickers (they said "video recording in progress") and users had consented to them sending data to iRobot for research purposes.
It's one thing to allow a company, based in the US, to access device imagery. But it's another for the photos to end up on a social media site.
And then there's the question of how they ended up there.
Images leaked by contract workers
iRobot had contracted an AI training data company called Scale AI to analyse the raw footage for use training its object detection algorithm.
Scale AI's founder Alex Wang described his company – which is valued at $20 billion – as solving "the data problem" for the AI industry.
"Our data engine generates nearly all the data needed to fuel the leading large language models," he said in an interview with CNBC.
The reality for its millions of contract workers, as described in a 2023 report from the Washington Post, is far less glamorous.
"The workers differentiate pedestrians from palm trees in videos used to develop the algorithms for automated driving; they label images so AI can generate representations of politicians and celebrities; they edit chunks of text to ensure language models like ChatGPT don't churn out gibberish."
iRobot ended its relationship with Scale AI after its contractors leaked the photos on social media.
Do cleaning robots even need high-definition cameras?
Researchers at the Australian Centre for Robotics have developed a solution that can avoid this problem altogether.
To keep sensitive images out of reach of hackers, they've developed technology that changes "how a robot sees the world".
In effect, it's an inherently "privacy-preserving" camera.
By scrambling the image taken by the camera beyond recognition before it is even digitised, there is no way that remote attacks can access the raw imagery.
And yet, enough information is still retained in the scrambled image for the robot to navigate by.
"There's no potential for a breach of security," explains Donald Dansereau, a senior lecturer at Sydney University who supervised the project.
The technology isn't quite ready for commercialisation, but Dr Dansereau is confident it will see uptake from tech companies.
He stresses there's "no magic bullet on the technological side – good policy and good literacy are still required".
