MCP Rigor Tests: Discovery Smoke vs Acceptance Coverage
Author: Ashwin Kondapalli, Founder & CTO, LoopIQ
Updated: October 6, 2026
The MCP Rigor run is green. Someone declares the agent “acceptance tested.” The suite only listed tools. There was no approved contract, no reviewed business assertions, and no link to the requirement under change. Discovery smoke was mistaken for acceptance coverage, and both were mistaken for permission to ship.
MCP Rigor lets teams generate and run natural-language .mcpr suites against MCP servers. Discovery smoke confirms the server can list tools. Acceptance coverage requires approved contracts, human-reviewed assertions, and evidence linked to requirements. If a sufficient contract is missing, generated fallback tests remain discovery-only (Use MCP Rigor tests). A successful run does not automatically approve a release certification, and passing tests do not authorize a release.
What is the difference between discovery smoke and acceptance coverage?
Discovery smoke
Intent: Can we reach the server and enumerate tools?
Typical content: List tools and expect success.
Contract needed: Minimal connectivity configuration.
Human review: Still review before trusting it as evidence.
Release meaning: Operational smoke only.
Acceptance coverage
Intent: Does behavior match the approved contract for the requirement?
Typical content: Tool, resource, or prompt calls with expected behavior taken from the contract.
Contract needed: An approved MCP tool, resource, or prompt contract with inputs and expected behavior.
Human review: Mandatory review and approval of the generated .mcpr package before execution.
Release meaning: Informs readiness when linked and fresh. It never authorizes a release on its own.
Pair this with intent-based testing: generate review-only packages, approve what humans trust, then run.
Boundary: Hosted LoopIQ execution has documented restrictions (for example, no arbitrary stdio servers, no token shell commands, no interactive OAuth in the hosted path, and private networks blocked). These are hosted runner limits, not a claim that MCP Rigor lacks capabilities elsewhere (MCP Rigor help).
How do you turn an MCP Rigor run into acceptance evidence?
Follow the sequence contract, generate review-only, approve, run, and link evidence:
Enable the framework. An admin enables MCP Rigor and the hosted runner; select the correct org and team.
Attach the contract. The source requirement includes an approved MCP tool, resource, or prompt contract with arguments and expected behavior.
Generate review-only. Produce the .mcpr package. Browser page objects are not required for MCP Rigor.
Human review. Check the assertions and reject discovery-only fallbacks when acceptance was intended.
Protect secrets. Configure MCP_URL and MCP_TOKEN as protected parameters. Do not paste tokens into requirements, source, or reports.
Execute. Public HTTPS Streamable HTTP on port 443, with a short-lived OAuth or bot token valid for the run. Review credit consent where runs are metered.
Link evidence. Sanitized JSON evidence is linked to the release through the existing evidence workflow. This is still not automatic certification.
A companion post in this series covers why a passing MCP Rigor run can still block a release, and how to write .mcpr suites that humans can actually review.
How this works in LoopIQ
Prerequisites: MCP Rigor enabled; the correct org and team; a bearer token scoped to the needed resources; an approved contract when acceptance is the intent; and reviewers who can approve packages (MCP Rigor help; test automation).
Sequence (conceptual): Select MCP Rigor, attach the requirement and contract, generate, review and approve the .mcpr package, configure protected parameters, execute on the LoopIQ runner, review counts and sanitized evidence, then link the run to the candidate and dossier.
Approvals and outputs: Approve before you execute. A successful run is not a release certification. Package the evidence in the Release Compliance Dossier when it is used at go/no-go, and keep it traceable as described in From Test Results to Release Evidence. Pricing is $4.99 per user per month; metered credits may apply for chargeable operations.
Illustrative comparison
Illustrative demo data. Not a customer result.
mcp-discovery-smoke: contract present: connectivity only; reviewer decision: approve as smoke; coverage claim allowed: discovery only.
mcp-refund-acceptance: contract present: tool contract for a partial-order refund; reviewer decision: approve assertions; coverage claim allowed: acceptance for that intent.
mcp-generated-fallback: contract present: missing or insufficient; reviewer decision: reject as acceptance; coverage claim allowed: discovery only, recorded as a gap.
Even the acceptance suite only informs readiness. The go/no-go still belongs to people working from the release readiness checklist.
FAQ: quick answers
What is an MCP Rigor discovery smoke test?
A suite that verifies the MCP server can list tools (and related discovery checks). It confirms connectivity and enumeration, not business-requirement acceptance coverage.
When does an MCP Rigor suite provide acceptance coverage?
When generation uses an approved MCP tool, resource, or prompt contract with expected behavior, humans review and approve the .mcpr assertions, and execution evidence links to the requirement and candidate.
Do passing MCP Rigor tests authorize a release?
No. A successful run does not automatically approve release certification. Passing tests inform readiness under policy; they do not authorize a release.
What happens if the MCP tool contract is missing?
Generated fallback tests remain discovery-only and leave acceptance coverage unverified. Fix the contract before treating results as acceptance evidence.
See a requirement become reviewed tests and execution evidence
See a requirement become reviewed tests and execution evidence, from approved contract to linked MCP Rigor run. Book a walkthrough with Ashwin.
Further reading: MCP Rigor help, Intent-based testing, Test results to release evidence, Release readiness checklist, Release Compliance Dossier, Pricing.
General information for engineering, quality, release, and compliance leaders. Not legal, audit, or regulatory advice.