There will never be enough people to fix the vulnerabilities in critical social and economic systems, with AI touted as the answer to the growing "security debt".
"The pace of change is only increasing from here on out," GitHub's chief technology officer Mike Hanley told AAP in San Francisco.
"This is a permanent structural change to everything ... there's no going back," he said, as the use of artificial intelligence ramps up.
But as some seek to use generative AI to cause harm and chaos rather than human progress, he used the main stage at a tech convention to urge governments, companies and emerging coders to work together.
Sharing information and having common tools could mean vulnerabilities would be eliminated without the need for widespread technical expertise, which is already in short supply in Australia and other advanced economies.
"Security is a team sport, it's not a trade secret," Mr Hanley said.
He said the benefits of generative AI were "so overwhelming" that it was worth the investment to overcome abuse, misuse and malicious use.
"The next wave that's happening is agentic AI - that is the new thing as of the last few months. People should start thinking about what that means for their organisation," he said.
This phase of AI would bring an era of autonomous agents - not merely overworked humans - capable of crunching data, hunting for vulnerabilities in code, and performing complex business processes.
"Most companies don't have enough security people on their teams, let alone enough security people to look at all the vulnerability alerts they have," he said.
"So in most organisations they do the most important ones and the rest just goes onto the pile."
He said having AI as a teammate, or agent, would clean up this pile of "security debt" on a scale that was not possible before.
In fact a human team could have a fleet of auto-fixing "agents" who were experts in specific tasks and capable of working across all models and systems.
He said energy security was another challenge as the rapidly growing warehouses of data behind generative AI need to run hardware and cooling systems.
GitHub's parent company Microsoft is turning to nuclear power, as are Amazon and Google, with the tech giants announcing energy deals in recent months.
AI-driven transformation has also delivered a solid start to the fiscal year for Microsoft, with cloud revenue of US$38.9 billion, up 22 per cent, according to a quarterly earnings update.
But citizens - and governments - need to be kept informed about how AI is trained and how it's used, so that it's not seen as "black box" to be feared and resisted.
Nor were the challenges unique to AI, as every technology had the potential to do great good and great harm, he said.
A few years ago, researchers were able to trick a Tesla into speeding by putting electrical tape over speed limit signs, which critics said illustrated the weakness in new machine-learning technology.
But the industry saw there were threats and fixed it, Mr Hanley said.
* AAP travelled with the assistance of GitHub.
