The Office of the eSafety Commissioner has unveiled plans to enforce proactive scanning of online messaging, cloud storage, and other internet-based services for child sexual abuse material and pro-terror material, with the release on Monday of two draft standards for public consultation.
The failure to commit to using technologies that detect and remove the child sexual abuse material and pro-terror material from Relevant Electronic Services (RES) and Designated Internet Services (DIS) was a key sticking point for eSafety Commissioner Julie Inman Grant when she rejected two industry-designed codes earlier this year.
RES covers a broad range of electronic services – such as those providing end-to-end encrypted messaging services, gaming services, and dating services.
Designated Internet Services, meanwhile, covers internet-based services that includes generative AI capable of producing harmful material, enterprise services, machine learning model platform services, and end-user managed services such as cloud storage for files or photos.
The regulator’s two draft standards published Monday would also commit industry to “disrupt and deter ‘known’ and ‘new’” harmful material, with some providers required to “invest in systems, processes and technologies that enhance the ability of the service to detect, disrupt and deter ‘known’ and ‘new’ [material]”.
This would also apply to ‘synthetic’ harmful material produced by a generative AI tool.
Six other codes, developed by a consortium of six peak industry bodies representing software, communications, electronics, digital platforms, and gaming were registered by the eSafety Commissioner earlier this year.
If registered, the eSafety Commissioner has the power to enforce the RES and DIS standards in the same way as the industry-designed codes that are now registered and will come into effect in the next few months.
The six bodies putting forward industry codes were the Australian Mobile Telecommunications Association, The Software Alliance, Communications Alliance, Consumer Electronics Suppliers Association, Digital Industry Gorup Inc (DIGI), and the Interactive Games and Entertainment Association.
In a statement, Ms Inman Grant said the RES and DIS codes drafted by industry had lacked “a strong commitment to identify and remove known child sexual abuse material”.
‘Known’ harmful material is that which has been identified and verified on databases held by global CSAM organisations and law enforcement agencies.
“There are already widely available tools, like Microsoft’s PhotoDNA, used by over 200 organisations and most large companies, that automatically match child sexual abuse images against these databases of ‘known’ and verified material,” Ms Inman Grant said.
She said PhotoDNA is “privacy protecting as it only matches and flags known child sexual abuse imagery” and stressed that it “doesn’t scan text in emails or messages, or analyse language, syntax, or meaning”.
“Many large companies providing online services take similar steps in other contexts, processing webmail traffic using natural language processing techniques to filter out spam, or apply other categorisation rules.
The eSafety Commissioner has pushed on with its plans to require digital platform providers to scan content, even as Apple abandoned development of its CSAM scanning tool last December due to what it called privacy risks.
When the industry body consortium released the first draft of their industry codes in September 2022, it argued that “the extension of proactive detection measures could have a negative impact on the privacy and security of end-users of private communications and file storage services, including services used by businesses and government enterprises”.
A similar view was shared by Digital Rights Watch who have described the extension of proactive detection measures from public platforms to private communications and file storage services as “an unreasonable invasion of privacy and creates additional security and safety risk for individuals, businesses and governments”.
In her statement, Ms Inman Grant sought to allay these fears and insisted that “eSafety takes the privacy of all Australians very seriously”.
“I want to be very clear on this – eSafety is not requiring companies to break end-to-end encryption through these standards nor do we expect companies to design systematic vulnerabilities or weaknesses into any of their end-to-end encrypted services,” she said.
“But operating an end-to-end encrypted service does not absolve companies of responsibility and cannot serve as a free pass to do nothing about these criminal acts.”
She highlighted some companies are already taking steps to prevent the spread of CSAM and other harmful material.
This includes Meta’s end-to-end encrypted WhatsApp messaging service which scans the “non-encrypted parts of its service including profile and group chat names and pictures that might indicate accounts are providing or sharing child sexual abuse material”.
“These and other interventions enable WhatsApp to make 1 million reports of child sexual exploitation and abuse each year. This is one example of measures companies can take,” Ms Inman Grant said.
DIGI’s director for policy, regulatory affairs and research Dr Jennifer Duxbury – a member of the industry group consortium that designed the rejected RES and DIS – told InnovationAus.com that the group is considering the draft standards.
“DIGI and its members share the government’s strong commitment to online safety. We are closely reviewing the draft standards and look forward to engaging with the Office of the eSafety Commissioner throughout the consultation process,” Dr Duxbury said.
Draft industry standards were released for consultation on Monday, with the public able to make submissions until December 21.
