Cyber AI Applied Research

Data for building, testing and improving cyber AI.

AI agents are moving into cyber work faster than organisations can fully understand their behaviour.

FIFTH DOMAIN’s Cyber AI Applied Research examines how AI agents behave in realistic missions, how that behaviour can be evaluated, and how the resulting evidence can support development, assurance and improvement.

We create controlled mission environments, record agent behaviour, evaluate outcomes and, where appropriate, produce simulation outputs, trajectories, evaluation findings and training data for teams building or improving cyber AI capability.

Discuss an evaluation

Cyber AI needs evidence from the work it is expected to do.

In cyber operations, small behaviours matter.

An agent may use the wrong tool, ignore relevant evidence, take an unsafe shortcut, recover well from an error, ask for help at the right time or appear successful while reaching the outcome for the wrong reason.

These behaviours are difficult to understand through general benchmarks alone. They need to be observed in context: the task, tools, time, permissions, assistance, grading method, model and operating environment all affect what the result means.

Cyber AI Applied Research is FIFTH DOMAIN’s work to make that evidence clearer.

It helps answer questions such as:

What did the AI agent actually do?
What evidence did it use or miss?
How did it respond to uncertainty, failure or changing conditions?
What behaviours should be improved before the agent is relied on?
What data could help model builders and AI engineers improve future performance?

For teams building, testing or improving cyber AI systems.

Cyber AI Applied Research is suited to teams that need evidence about how AI systems behave in cyber tasks, not just whether they produced an answer.

Model builders

For teams developing or fine-tuning models that need cyber-specific evaluation data, annotated trajectories or behavioural feedback.

AI engineers and agent developers

For teams building AI agents that use tools, follow instructions and operate inside cyber workflows.

Cyber AI product teams

For vendors or internal teams building cyber products, services, detections or agent-enabled workflows.

Security operations teams

For teams exploring how AI agents may support triage, investigation, detection, response or analyst workflows.

Research and development teams

For groups studying AI-agent behaviour, evaluation methods, cyber simulation, human-AI teaming or model improvement.

Government, Defence and national security teams

For agencies that need evidence about AI-agent behaviour before considering operational use, assurance, controls or further development.

Transformation partners and system integrators

For teams helping organisations adopt, test or govern AI-enabled cyber capability.

We turn mission activity into usable evaluation evidence.

Our cyber AI research supports the design, observation and analysis of AI-agent behaviour in realistic cyber missions.

Evidence outputs for cyber AI teams

Depending on the question, this work can produce one or more connected outputs: simulation environments, recorded trajectories, evaluation findings or training data. Each output is scoped around the behaviour, capability or decision the team needs to understand.

Simulations

Cyber simulation environments and scenarios for product development, threat research, protection engineering, detection engineering and AI-agent evaluation.

Trajectories

Recorded model or agent behaviour from simulation runs, showing what the agent did, what tools it used, what evidence it relied on and what outcomes followed.

Evaluations

Analysis of recorded behaviour against agreed criteria and reference data, with findings provided through reports, dashboards or structured evidence outputs.

Training data

Annotated trajectories and feedback that can support model builders and AI developers seeking to fine-tune or improve their systems. Dataset scope, format and permitted use are agreed before work begins.

What this work can support.

AI-agent evaluation

Understand how an AI agent behaves in realistic cyber missions before it is relied on.

Model improvement

Generate annotated trajectories, feedback and behavioural insights that can support fine-tuning or future development.

Product development

Use evaluation evidence to improve cyber AI products, services, detections or agent-enabled workflows.

Threat and detection research

Create scenarios and simulation environments for studying cyber behaviours, detection logic or defensive coverage.

Assurance and governance

Generate evidence that supports decisions about oversight, restrictions, suitability and deployment readiness.

Human-AI capability development

Understand how AI agents may support, augment or complicate the work of cyber operators.

The result is read in context, not in isolation.

A score alone does not explain whether an AI agent is ready to be relied on.

The result needs to be read alongside the task, tools, time, assistance, grading method, limitations, model and operating environment. That context matters because two agents may reach the same answer for very different reasons. One may follow the evidence carefully. Another may guess, skip steps or rely on the wrong information.

Our cyber AI research is designed to examine the behaviour behind the outcome.

This helps teams understand not just whether something worked, but what happened, why it happened and what should change next.

Improving an AI agent for cyber investigation work.

A team is building an AI agent to support cyber investigation. Before using it in a live environment, they need to understand how it behaves when given a realistic task, relevant tools and a defined objective.

FIFTH DOMAIN can create a contained mission environment and evaluation criteria for that task. The agent’s behaviour is recorded as it works through the mission, including the actions it takes, the evidence it uses, the tools it calls, the decisions it makes and the outcome it reaches.

The recorded behaviour is evaluated against agreed criteria and reference data. The evidence may show where the agent performed well, where it missed relevant information, where instructions need to change, where tool access should be restricted or where further training data may help.

The output is not a certification. It is evidence that helps the team understand what happened and what to improve next.

Scope and permission for Cyber AI Applied Research.

Cyber AI Applied Research can involve sensitive inputs, behavioural records and derived data. Scope and permissions need to be clear before work begins.

This includes:

the task, tools, time, assistance and grading method;
the model and operating environment being evaluated;
limitations and reference sources;
outputs, access, hosting and retention;
ownership of results;
whether raw traces, derived data, commissioned processing or research reuse are permitted.

Simulations generated for a customer are private to that customer. Data use, reuse, retention and research permissions are agreed as part of the engagement.

Follow the signals from cyber AI research.

Mission Readout is FIFTH DOMAIN’s monthly briefing on cyber AI, mission evidence, evaluation methods and the development of human and AI cyber capability.

Subscribe for concise research updates, practical observations and signals from our work in Cyber AI Applied Research.

Subscribe to Mission Readout