AI & Automation5 min read

Test Agent Cancellation After the Tool Commits

S

Suneet Malhotra

Oct 15, 2026

β€’
1 views
Test Agent Cancellation After the Tool Commits - AI & Automation blog post
πŸ”§MCPπŸ”§QA AutomationπŸ”§Idempotency

Cancelling an agent request does not prove that its tool action was undone. A useful QA test checks three things separately: whether the interface stopped waiting, whether the worker stopped executing, and whether the destination already changed.

This follows the tool-call checklist with a narrower failure case: the user presses Stop while a tool is creating a ticket. The exercise below uses a fictional ticket service and synthetic events. It describes a test design, not results from an employer system.

Where cancellation sits

An agent application may send a tool request through Model Context Protocol, or MCP, to a server that calls another service. That gives us several boundaries: the interface, the MCP client, the tool handler, and the ticket database. A cancellation signal has to reach work that is still interruptible to prevent a change.

Pin the protocol version in your fixture. In the MCP 2025-11-25 cancellation specification, ordinary in-progress requests use a cancellation notification referencing the request ID. Receivers should stop processing and release resources, but may ignore cancellation when processing has completed or the request cannot be cancelled. Task-augmented requests use a separate mechanism. These are version-specific rules, not a universal recipe for every SDK.

The practical implication is that sending the signal is insufficient evidence for a rollback. A successful database commit can precede the signal even if the response has not reached the client. That timing window is the target of this exercise.

Define the observable contract first

Give the fictional create-ticket operation a durable operation ID, such as qa-ticket-042. Keep it distinct from the request ID used to correlate a protocol exchange. In this proposed application contract, multiple attempts for the same operation must resolve to the same ticket, and reuse with different ticket contents must be rejected.

This follows the general approach in the Amazon Builders' Library discussion of idempotent APIs: caller-provided identifiers let a service distinguish a retry from a new intent. The article also explains the ambiguity of a missing response and the need to handle an identifier reused with different parameters. MCP request correlation alone does not implement this application behavior.

For the fixture, expose a read-only lookup by operation ID. Define three user-visible outcomes: cancelled before creation, ticket created, and outcome unconfirmed. The last state matters when the client cannot reach the lookup service. Do not let the interface translate missing evidence into a claim that nothing happened.

Place barriers around the commit

Avoid relying on a short sleep to reproduce the race. Add test-only synchronization barriers to the fake service. One pauses immediately before the write transaction; another pauses after commit but before returning the response. The test controller releases each barrier explicitly.

Run these cases independently, resetting the fixture between them:

Controlled interruptionExpected destination stateExpected application behavior
Cancel before the pre-write barrier is releasedNo ticket, when the fixture honors cancellationReport cancellation after verifying no creation
Cancel after commit, before response deliveryExactly one ticketReconcile and report the existing ticket
Lose the response and make lookup unavailableState cannot yet be confirmed by the clientReport uncertainty and retain the operation ID
Repeat the same operation after a lost responseExactly one ticketReturn the existing result under the fixture contract
Reuse the operation ID with different contentsOriginal ticket unchangedReject the conflicting request

These expectations belong to this synthetic service contract. A third-party tool without lookup or idempotency support cannot be assumed to provide them. For that integration, document the unresolved outcome and prohibit an automatic retry that could create another ticket.

Assert the destination, then the message

The strongest assertion in the post-commit case reads the ticket store and counts records for the operation ID. The count should be one. Next, assert that the agent did not send a second create operation with a fresh identity. Finally, inspect the user-facing status: it must agree with the reconciled destination state.

Keep conversation rendering separate from reconciliation. A response arriving after cancellation need not resume the abandoned conversation. A dedicated status check can still establish whether the requested action happened and update an activity record.

Capture the operation ID, protocol request ID, protocol revision, barrier reached, cancellation observation, commit outcome, lookup outcome, and final displayed status. Use logical event ordering from the fixture rather than comparing wall clocks across machines. Connect these records to the trace continuity test, so missing telemetry becomes an explicit test failure.

Prove the test can catch the defect

Add a negative control: make the fake service ignore cancellation after commit and make the client immediately display "Nothing was created." The destination assertion should still find one ticket, and the message assertion should fail. If the suite stays green, it only tested that the Stop button changed appearance.

Then remove the deliberately incorrect client behavior and rerun the same controlled ordering. Require a recorded reconciliation result or an explicit unconfirmed state. A green result now has a concrete meaning: this fixture detects a false cancellation claim.

Cancellation also does not grant permission to delete a ticket that already exists. Reversal is another state-changing operation with its own authorization and failure cases. Keep that workflow outside this test unless it is explicitly part of the product contract.

For the next tool integration you test, identify the last point where work can stop without changing the destination. What evidence will your interface show when the user presses Stop just after that point?

Share this post

You Might Also Like

Stay in the Loop

Get weekly insights on AI-driven QA, engineering leadership, and automation strategies.

No spam, ever. Unsubscribe anytime.