QA Engineer AgentTests that trace to requirements. Defects that arrive triage-ready.
The QA Engineer Agent drafts test cases from acceptance criteria, maps coverage gaps, writes defects from failed runs with the evidence attached, triages incoming bugs, diagnoses flaky tests, and records fix validation step by step. It runs inside your Azure DevOps project, and generated tests and defect decisions stay with your QA team.
No card required. Your data stays in your Azure DevOps tenant.
- trigger criteria updated · STORY 4821
- check coverage · 2 untested criteria
- checkpoint awaiting QA review
Coverage checked the moment criteria change.
The best first workflow for this agent. The trigger is a change your team already makes, the check is concrete, and the gaps it finds are easy for QA to confirm.
- 01Runs when acceptance criteria are added or updated on a story, on the condition your team picked.
- 02Checks every criterion for a linked test, and looks for missing negative, boundary, and edge cases.
- 03Drafts test cases with steps and expected results for each gap, linked to the requirement, and waits for QA.
- 04A rule-shaped link check can run as an Execution Agent: deterministic code, no model call, a flat 4 credits a run.
COVERAGE CHECK · STORY 4821
- ! Every criterion has a test
- ! Negative and boundary cases
- ✓ Tests linked to the story
- ✓ Steps and expected results
- ✓ Configurations assigned
Ships with
- Generate Test Cases from Requirements
- Design Test Cases
- Identify Test Coverage Gaps
- Triage Bug
- Link Bug to Test Failure
- Investigate Test Failure
- Diagnose Flaky Test
- Validate Fix
- Reproduce Defect
- Execute Test Run
- Analyze Regression
- Generate Test Report
Six jobs your QA team can hand over.
Each one runs the same controlled sequence and leaves its output in Test Plans or on the work item, linked to the requirement, ready for QA to review.
- JOB 01
Draft test cases from acceptance criteria.
Prepare test cases from the criteria on new stories, plus the risks, interfaces, and non-functional conditions around them. Each case has steps and expected results, sits in the right suite, and links back to the requirement while QA reviews it.
dev.azure.com/contoso · Test Plansrun started · criteria updated · STORY 4821
reading 4 scenarios, 1 edge case, 2 risks
drafting 6 test cases · steps and results
linked to STORY 4821 · suite: checkout
checkpoint · cases awaiting QA review
- JOB 02
Find the coverage gaps.
Find requirements and risks without tests, tests that no longer match a current requirement, and missing negative or boundary cases, then propose the cases or links that close each gap.
dev.azure.com/contoso · Test Plans · release 2.4run started · manual: coverage, release 2.4
mapping 61 requirements to tests
2 untested requirements found
3 tests point at a retired requirement
checkpoint · gap list awaiting QA
- JOB 03
Turn failed runs into triage-ready defects.
When a test fails, correlate it with the environment, configuration, build, recent changes, related defects, and prior results. Then draft a defect with that evidence attached and linked to the failing test, instead of a one-line bug and a screenshot.
dev.azure.com/contoso · Test Plans · Runsrun started · TC-88 failed · nightly-ci
comparing with last 10 results · 9 passed
build 1187 · 3 merges since last pass
defect drafted · evidence and links attached
checkpoint · defect awaiting QA
- JOB 04
Triage incoming bugs.
Assess severity, priority, reproducibility, affected area, business impact, and likely owner against your rules. Flag duplicates and the evidence that is missing, and ask the reporter for it on the item.
dev.azure.com/contoso · Boards · Bugsrun started · BUG 6120 created
area: checkout · severity rule: 2
possible duplicate of BUG 6094
repro steps missing · asked reporter
checkpoint · triage awaiting QA
- JOB 05
Diagnose flaky tests.
Analyze recent result patterns across environments, timing, dependencies, and changes to separate a flaky test from a real regression, and post the evidence with a remediation path. Quarantine stays a QA decision.
dev.azure.com/contoso · Test Plans · Runsrun started · schedule: weekly flaky review
TC-142 · 4 fails in 20 runs · no code change
fails only on agent pool linux-2
probable cause: shared test data timing
record · finding posted on TC-142
- JOB 06
Validate the fix, step by step.
After a fix merges or deploys, rerun the reproduction or the linked test and record the outcome of the result and of every step, so the Runs view shows exactly which step passed. Regression evidence is attached to the defect.
dev.azure.com/contoso · Test Plans · Runsrun started · fix for BUG 6120 deployed
rerunning TC-88 · config: Chrome, staging
UpdateTestStepResults · 7 of 7 passed
regression evidence attached to BUG 6120
checkpoint · closure awaiting QA
run started · criteria updated · STORY 4821
reading 4 scenarios, 1 edge case, 2 risks
drafting 6 test cases · steps and results
linked to STORY 4821 · suite: checkout
checkpoint · cases awaiting QA review
ILLUSTRATIVE RUNS · EVERY RUN LANDS IN YOUR JOB LOG
It runs when quality is at stake.
Manual, event-based, or scheduled. Every run starts from a trigger a person on your team set. The agent cannot add its own, and a disabled agent does not run at all.
- EVENT
A bug is filed, criteria change, or a fix lands.
Add conditions so it runs only on what matters, for example when Acceptance Criteria Changed on a User Story, or State changes to Resolved on a Bug. No run, no credits, on everything else.
- SCHEDULE
The quality review comes round.
Schedule a manual agent once, daily, on weekdays, weekly, monthly, or yearly: the flaky-test review every week, the test summary report before the sprint review.
- MANUAL
Someone asks.
Run a coverage check on a release, a failure investigation on one run, or a sign-off report on demand from the agent, or call it from Copilot4DevOps Chat.
Detected
- scanning…
TC-88, from red to a defect the team can act on.
The seven stages every run follows, shown on one failed-test run. Click a stage or let it play.
01 Trigger
TC-88 fails in the nightly regression run on build 1187. The agent's condition matches a failed outcome in the checkout suite. Passing runs and other suites do not start it.
What it reads. What it writes.
The agent retrieves only the Azure DevOps context, and approved external context, that the workflow needs. Scope it to a project, area path, test plan, or work item type.
| Context | It reads | It writes, creates, or updates |
|---|---|---|
| Requirementsstories and criteria | Requirements, acceptance criteria, risks, and their links | Proposed or created test cases, suites, and requirement relationships |
| Test Planscases, runs, results | Test cases, suites, runs, results, attachments, and history | Test results with the outcome of each step, and coverage findings |
| Defectsbugs and failures | Bugs, their history, related defects, and related incidents | New or updated defects with structured evidence and links, plus triage findings |
| Delivery contextwhat changed | Commits, PRs, builds, environments, configurations, logs, and releases | Failure, flaky-test, regression, and fix-validation findings |
| Your processknowledge you attach | Severity and priority rules, reproduction standards, coverage expectations, release criteria, and industry evidence requirements | Nothing. Knowledge changes only when your team edits it |
| The runevery time | — | A run record, plus quality and release sign-off reports when asked |
Run it as shipped, or fork it.
Every agent is a versioned definition. Edit its instructions in plain English, attach your test standards and templates as knowledge, and re-version with rollback. You never write or see code.
Grounded in your own knowledge
With Bring Your Own Data, the agent retrieves from your uploaded documents, Azure DevOps queries, and wiki pages during the run, so its test cases and defects follow your standards and terminology.
Your model, per agent
Choose the model this agent runs on and see its credit rate at selection. Enterprise brings your own LLM and key.
Versioned, with library updates
Restore any earlier version in one step. When the library version improves, compare yours with the new one, then keep yours or review and update.
knowledge
- case_format"steps · expected result"
- severity_rules"sev 1 to 4 · v2"
- repro_standard"build · env · steps"
- naming"TC- / BUG-"
- use_byod"on · wiki + queries"
- approver"named, per workflow"
KNOWLEDGE UPDATED · SOP v3 attached
The agent tests and drafts. Your QA team decides.
Generated tests and defect decisions remain subject to QA review. Automated severity, closure, quarantine, or release sign-off actions use explicit rules and approval boundaries, never a blanket setting.
Deterministic rules
- Links between tests, requirements, and defects
- Required fields on new defects
- Duplicate checks against open defects
Drafts, then waits for review
- New test cases and suites
- Defects drafted from failed runs
- Severity, quarantine, and closure proposals
Never does
- Signs off a release or accepts a risk
- Adds its own triggers, or runs while disabled
- Trains models on your project data
Permissions
Allow and deny, per group.
Who can manage, delete, and inspect agents follows Allow and Deny per team or security group, set at project level with organization inheritance.
| Permission | Allow | Deny |
|---|---|---|
| Manage Agents | ||
| Delete Agents | ||
| Job Explorer | ||
| Manage Skills |
PROJECT LEVEL · ORGANIZATION INHERITANCE
Job log · per run
Every run, reconstructable.
- trigger
- work item 4821 updated
- checked
- criteria · links · fields
- changed
- +1 follow-up task
- approval
- awaitedgranted
- output
- draft accepted · verified
The same agent, held to your industry's evidence.
Industry variants add safety class, DAL or ASIL, GxP, risk-control, independence, coverage, conformance, or evidence requirements to triage and test workflows.
- Variant 01
Software / SaaS
- Variant 02
Automotive
- Variant 03
Medical Devices
- Variant 04
Aerospace & Defence
- Variant 05
Pharma / MedTech
- Variant 06
Finance / FinTech
- Variant 07
Telecommunications
Need formal verification and validation? The Test Engineer Agent designs, executes, and traces formal tests, with variants for automotive, medical devices, aerospace and defence, pharma, and telecommunications.
All industries
Agents prepare, check, and maintain engineering artifacts. Qualified personnel keep responsibility for safety, regulatory, certification, and release decisions.
Hands off, explicitly.
Quality work starts with good requirements and ends at a release decision. Complex workflows coordinate several agents with explicit handoffs and approval boundaries.
QA Engineer Agent
PlanBusiness Analyst Agent
Drafts and validates the acceptance criteria that test cases are generated from.
ShipDevOps Engineer Agent
Triages pipeline failures with evidence, so a red build and a red test are diagnosed together.
ReleaseRelease Manager Agent
Sweeps release scope for open criteria and pending approvals before go/no-go.
GovernCompliance Officer Agent
Walks trace chains from requirement to test to result, and freezes audit evidence packs on demand.
[ Trusted by Microsoft and industry leaders ]
Questions about the QA Engineer Agent.
It drafts them from acceptance criteria, risks, and interfaces, with steps and expected results, and links them to the requirement. With Required Approval on, the run pauses and opens a task until a QA engineer approves, edits, or rejects the draft.
The catalog includes test-run execution. Agents can create and update test results, and the UpdateTestStepResults tool call records Passed or Failed for each step of a manual test result, so the Runs view shows the outcome step by step.
Only under a deterministic, verified policy your team explicitly turns on. Most teams keep closure and risk acceptance with QA, and the agent proposes closure with the fix-validation evidence attached.
It analyzes recent result patterns, environments and configurations, timing, dependencies, and recent changes, then posts the evidence and a remediation path on the test. Quarantine remains a QA decision.
A coverage check when acceptance criteria change. The trigger is clear, the gaps are concrete, and the drafted cases are quick for QA to confirm in the first week of the trial.
Only if you want it to. Event-based agents take trigger conditions built from fields, operators, and values, so you can limit it to a suite, an area path, or a work item type, and spend no credits on the rest.
Credits are $0.01 each, pooled across your organization with no per-user fees. A rule-based link or duplicate check on an Execution Agent is a flat 4 credits, about $0.04. Investigating a failure across results, builds, and logs is a deeper run that scales with context. Agents pause cleanly at zero.
The QA Engineer Agent focuses on defects and day-to-day software quality operations. The Test Engineer Agent focuses on formal test design, configurations, execution, coverage, and verification and validation evidence.
Put it on your next test run this week.
Free trial with 2,000 credits, no card. Or bring one workflow to a 30 minute demo and watch the run live.
