Key judgements
- Artificial intelligence is changing the speed, scale and complexity of cyber operations. While malicious use is increasingly evident, its overall contribution to the frequency and severity of cyberattacks remains difficult to quantify.
- AI presents significant opportunities to strengthen defensive cyber capability. However, increasing autonomy introduces new operational dependencies and risks that extend beyond conventional cyber security.
- Australia’s national AI initiatives are advancing adoption, technical safety research and government accountability. The challenge is connecting these efforts with evidence of how AI systems perform in operational environments.
- As AI becomes more deeply integrated into cyber operations, confidence will increasingly depend on demonstrated capability, observable behaviour and effective controls, rather than assumed performance.
The changing cyber threat landscape
The Australian Signals Directorate’s (ASD) Annual Cyber Threat Report 2025–26 identifies artificial intelligence as a significant influence on Australia’s cyber threat environment.
ASD reports that AI is enabling malicious actors to accelerate vulnerability discovery, automate elements of cyber activity and operate with greater speed, scale and sophistication.
The report also highlights continued pressure on Australia’s cyber resilience. During 2025–26, ASD’s Australian Cyber Security Centre responded to more than 1,500 cyber security incidents, an increase of over 20% from the previous year, and issued more than 4,400 notifications concerning potentially malicious cyber activity.
These figures reflect an increasingly demanding operational environment. They do not, however, establish that AI caused the increases.
This distinction matters.
International research supports ASD’s assessment that AI is increasingly being used in cyber operations, including by malicious actors. Yet the extent to which AI has independently increased the overall volume or severity of cyberattacks remains uncertain.
What is more clearly established is the expansion of technical capability.
AI-assisted tools can accelerate software analysis, vulnerability discovery, code generation and aspects of exploitation. Activities that previously required substantial manual effort may increasingly be supported by automation.
For defenders, this creates pressure to detect, assess and respond to threats more quickly.
But increased speed is only part of the challenge.
As cyber operations become more automated, the quality of the decisions made within those operations becomes just as important as the speed at which they are executed.
AI is becoming part of the cyber force
ASD’s report is not exclusively concerned with malicious applications of AI.
It also identifies significant opportunities for AI to strengthen cyber defence, including vulnerability management, threat detection, malware analysis, incident response and cyber threat intelligence.
These capabilities could improve operational efficiency and allow human specialists to focus on activities requiring deeper investigation, contextual judgement and operational oversight.
However, an important distinction is emerging between AI systems that assist cyber operators and those capable of undertaking sequences of actions with increasing autonomy.
An AI assistant may analyse a suspicious file or summarise an incident. An autonomous agent may select tools, investigate systems, pursue an objective and initiate actions that change its operating environment.
These applications present different levels of operational risk.
ASD’s report identifies concerns arising from unintended actions by AI agents, including behaviour influenced by ambiguous instructions, insufficient constraints and interactions with vulnerable systems.
This is not simply another form of malicious cyber activity.
It introduces the possibility that an authorised AI system could undertake technically permitted actions that are inconsistent with its intended purpose or operational boundaries.
ASD’s separate guidance on the careful adoption of agentic AI, published in May 2026, reinforces the importance of least-privilege access, secure design, continuous monitoring, human oversight and controlled deployment.
More recent guidance on agentic AI harnesses also emphasises that security cannot be established through the model alone. The surrounding tools, integrations, permissions and governance mechanisms are fundamental to controlling agent behaviour.
These are necessary safeguards.
But they raise another question:
How do organisations establish whether an AI agent is capable of performing its assigned mission reliably, even when appropriate security controls are in place?
Preventing unauthorised activity and demonstrating operational competence are related, but different challenges.
Australia’s national AI response
Australia’s approach to AI is developing across several complementary areas: economic adoption, technical safety research, policy and governance.
The Australian Government’s National AI Plan, released in December 2025, identifies three priorities: capturing the opportunities of AI, spreading its benefits and keeping Australians safe.
The National AI Centre (NAIC), established in 2021, supports practical AI adoption across Australian industry. Its work provides organisations with guidance on identifying opportunities, understanding risks and implementing responsible AI practices.
The Australian AI Safety Institute (AISI) performs a distinct function.
Established as an initiative under the National AI Plan, AISI analyses and tests emerging AI capabilities, risks and harms, supports government agencies and regulators, and contributes to international AI safety research.
Its published work includes research on multi-agent systems, automated AI evaluation and the oversight of advanced AI systems.
This technical focus is particularly relevant as AI moves towards increasingly autonomous operations.
While the National AI Centre supports organisations in understanding and adopting AI, the AI Safety Institute contributes to the scientific understanding of what advanced systems can do, where risks emerge and how those risks can be measured.
These functions are complementary, rather than interchangeable.
They also reflect a broader development in AI governance: responsible adoption increasingly depends on both organisational preparedness and reliable technical evidence.
For cyber operations, this distinction has particular significance.
An organisation may understand the potential value of deploying an AI agent and have established an appropriate governance process, while still lacking sufficient evidence about how that agent will perform under realistic operational conditions.
From governance to demonstrated assurance
Australia’s government AI policies are also becoming more structured.
The Commonwealth’s Policy for the Responsible Use of AI in Government, updated in October 2026, requires entities within its scope to establish accountability arrangements, maintain AI use-case registers, undertake impact assessments and implement responsible-use processes.
The policy excludes Defence portfolio AI use and the national intelligence community, which operate under separate governance arrangements.
Across jurisdictions, state governments are also introducing AI assessment and oversight mechanisms, reflecting growing recognition that AI-related risks require documented accountability.
These developments are significant.
Governance frameworks establish responsibilities, identify risks and create processes for reviewing the use of AI. They provide an important foundation for responsible adoption.
However, governance assurance and technical assurance are not the same.
An impact assessment can establish that an organisation has considered the consequences of deploying an AI system. It cannot independently demonstrate that the system will perform a particular cyber task successfully.
Similarly, a benchmark result may demonstrate technical capability under controlled conditions without establishing that the system is suitable for deployment in a sensitive operational environment.
A high-performing model is not necessarily a reliable operational agent.
The distinction becomes increasingly important as AI systems are given greater access to tools, information and operational infrastructure.
The question is not simply whether an AI system has been assessed, but what the assessment actually proves.
The emerging evidence requirement
Taken together, ASD’s threat assessment and Australia’s evolving AI policy landscape point towards a shared challenge.
AI capability is advancing. Adoption is accelerating. Governance requirements are becoming more established. Technical safety research is expanding.
Yet important uncertainties remain about how increasingly autonomous AI systems behave under complex operational conditions.
Traditional benchmarks can measure whether a model completes a defined task. They may reveal less about how it reaches that outcome, the decisions it makes along the way or its response to unexpected conditions.
Operational environments introduce additional considerations: incomplete information, competing objectives, changing system states, tool constraints and consequences that may extend beyond the immediate task.
For cyber operations, understanding these behaviours is essential to interpreting capability.
Three questions warrant particular attention:
What has actually been demonstrated? A model completing a technical benchmark provides evidence of performance under the tested conditions. It does not automatically establish capability across different cyber disciplines or missions.
How was the outcome achieved? Successful execution may conceal unnecessary actions, incomplete analysis or departures from an expected operational method. Understanding the sequence of actions can be as important as verifying the result.
Under what conditions can the capability be relied upon? Model behaviour may change with different tools, permissions, instructions or environments. Reliability must therefore be considered in relation to the intended mission and operating conditions.
These are not questions that governance frameworks alone can resolve.
They require technical evaluation capable of producing observable, repeatable and interpretable evidence, supported by appropriate operational controls and human accountability.
Assessment: From AI adoption to operational confidence
ASD’s Annual Cyber Threat Report 2025–26 highlights a cyber environment in which AI is influencing both adversarial activity and defensive capability.
At the same time, Australia’s national AI initiatives are strengthening the foundations for adoption, safety research and accountable use.
These developments represent important progress, but they address different parts of the challenge.
The National AI Centre supports informed adoption. The Australian AI Safety Institute advances the assessment of emerging capabilities and risks. Government policies establish expectations for responsible use. ASD provides an operational view of how AI is reshaping the cyber threat landscape.
The next challenge is connecting these efforts with reliable evidence of performance.
For cyber operations, this means moving beyond assumptions based on general model capability towards understanding how AI systems act, decide and perform within defined mission environments.
It also means recognising the limits of that evidence. A successful simulation is not proof of universal reliability, just as a completed risk assessment is not proof of operational readiness.
As autonomy increases, effective cyber AI adoption will require governance, secure system design, rigorous technical evaluation and accountable human decision-making to operate together.
Australia’s challenge is not simply to accelerate AI adoption or strengthen its governance. It is to establish what these increasingly capable systems can demonstrably do, where their limitations lie and under what conditions they can be relied upon.
References
- Australian Signals Directorate (2026). Annual Cyber Threat Report 2025–26. cyber.gov.au/publication/annual-cyber-threat-report-2025-26
- Australian Signals Directorate (2026). Careful adoption of agentic AI services. cyber.gov.au/publication/careful-adoption-of-agentic-ai-services
- Australian Signals Directorate (2026). Agentic AI harnesses. cyber.gov.au/advisory/agentic-ai-harnesses
- Australian Government, Department of Industry, Science and Resources (2025). National AI Plan. industry.gov.au/publications/national-ai-plan
- Australian Government. National AI Centre. industry.gov.au — National AI Centre
- Australian Government. Australia’s AI Safety Institute. industry.gov.au — AI Safety Institute
- Digital Transformation Agency (2026). Policy for the Responsible Use of AI in Government, Version 2.1. digital.gov.au/ai/ai-in-government-policy