MCP Security Index methodology

Scope and units of comparison

The unit of assessment is a resolved MCP identity and its observed package release or endpoint scope. Providers are ranked within deployment cohorts because hosted services, installable packages and bridges expose different evidence. Functional categories such as productivity or identity support discovery; they do not change security scores or cohort ranks.

The MCP specification describes tools as model-controlled and recommends a human decision point for tool invocations. CoSAI identifies access control, tool design, human oversight and lifecycle governance as security concerns. The index adapts these themes into its own evidence requirements; the numerical weights are Durin policy. [2] [1]

MCP authorization is optional. A successful unauthenticated initialize request can serve a public endpoint or precede per-tool authentication. It does not by itself demonstrate an authentication bypass. The index records that case as unknown for the authentication-boundary check, rather than failed or not applicable. A 401 challenge can establish an observed boundary, but does not verify authorization of every action. [7] [8]

Evidence, uncertainty and confidence

Evidence is attached to an identity, a stable control ID, a source and an observation time. A criterion's freshness window determines when its evidence expires. Attributed documentation can establish a declared capability, while runtime verification requires evidence appropriate to that control. One does not silently become the other.

SLSA provenance concerns verifiable information about how an artifact was produced. The index distinguishes registry digests and provenance metadata from independent verification of their claims. Metadata inspection does not execute the artifact or establish the security of its behavior. [4]

MCP security guidance addresses authorization context, token handling and scope minimization. In this index, published OAuth and identity metadata establish observable configuration; they do not validate every runtime access decision. [3]

Missing mandatory evidence, an incomplete or expired assessment, active compromise, an unresolved affected release or insufficient assessed weight can prevent ranking. At least 20 category-budget units must have evidence. Confidence and evidence coverage are reported separately from the baseline score; a failed or partial required check cannot establish that there are no known findings.

Public posture describes what an independent observer can substantiate without publisher credentials or invoking tools. A positive documentation result establishes a published capability claim; a verified metadata result establishes the observed declaration or response. Neither establishes enforcement of a sensitive operation. Missing, unreachable or unmatched evidence remains unknown and earns no credit. A completed documentation search is not proof that a capability is absent. [7] [9]

Public evidence confidence uses the scored baseline only. High requires at least 80% known weighted coverage and 80% qualified capture coverage; Medium requires at least 50% of each. A qualified capture identifies its collector, public sources, SHA-256 digests, bounded passages or normalized observations, and collection time. Source-content receipts hash the fetched response text; advisory query receipts hash a retained normalized query manifest instead. Sources must link to the evidence record and have been collected or actually reconfirmed within 30 days. Reusing a cached fact or recalculating rankings does not renew its collection time. Old records without captures cannot automatically gain stronger confidence. The existing 180-day scoring expiry remains separate; hourly advisory freshness still applies.

Runtime assurance is reported separately as Not tested or scoped checks with test artifacts. A test must name its action, configuration and evidence artifact; it may support a pass or a failure. An observed HTTP status, registry digest, OAuth declaration, documentation match or tool annotation never becomes runtime enforcement proof. High confidence in public posture can coexist with runtime assurance Not tested. [9]

Public capture completeness and attestations

Public-source capture does not require a publisher to participate in the assessment. Collectors preserve the source URL, response digest, collector identity and observation time alongside independently observed facts. Package receipts bind to the registry, package name, exact version and recorded artifact digest. Publisher receipts bind to the endpoint and publisher identity. A changed subject, malformed response, unavailable page or incomplete required source collection cannot manufacture a qualified receipt.

Publisher attribution can be supported by a fresh exact namespace lookup in the official MCP Registry. Domain-based namespaces indicate control of the corresponding domain. For a pinned catalog attribution, a configured public document must name the publisher and exact endpoint before it can receive a public document receipt. An internal catalog record alone is insufficient capture proof. Publisher attribution does not establish that the service is safe. [12]

Advisory receipts describe a bounded query against the currently released configured corpus. The retained manifest identifies the resolved subject, source checkpoints and successful collection times, immutable advisory revision digests, matched records and remediation observations. Its digest covers that normalized query manifest, rather than claiming to hash an upstream HTTP response. Missing or stale required sources, unresolved subjects and incomplete assessments do not receive qualified query receipts. No match means no match in this corpus; it does not establish the absence of undisclosed vulnerabilities. [6]

These receipts are unsigned observer records, not publisher attestations. Existing public provenance and signed attestations can identify a producer and artifact, but their signatures and claims must be verified under an explicit trust policy before they can support stronger assurance. This collection release does not issue publisher attestations or automatically verify every public attestation. Signing an unchanged claim alone does not increase its confidence or score. [13] [14] [15]

Capture completeness describes how much observed evidence has reproducible collection material. Control evidence coverage describes how much of the scored baseline is known. Improving one cannot fill gaps in the other. Public receipts preserve documented or observed evidence states; they do not prove human approval enforcement, tenant isolation or other runtime behavior. Those claims require scoped tests and their artifacts. No tools or package code are executed to collect these receipts.

MCP definitions and discovery metadata

Definition collection examines the public, unauthenticated surface of a configured remote MCP endpoint. Initialization negotiates a supported protocol and records server information, capabilities, instructions and extension metadata. Only advertised capabilities are listed: tools, prompts, resources and resource templates. Tool descriptions, complete input and output schemas, annotations, execution hints, icons and metadata are retained within explicit bounds. Prompt bodies and resource contents are not requested; tools and package code are never executed. [8] [9]

Discovery examines the challenge-advertised protected-resource document and the path-specific and root OAuth well-known locations. It also examines permitted authorization-server and OpenID discovery documents, security.txt and two optional MCP manifest conventions. The manifest paths are collection choices, not MCP requirements. Requests are restricted to configured hosts and redirects are refused. Advertised icons, schema references, websites and extension URLs are recorded as data and are not automatically fetched. [7] [10] [11]

All collected definition text is untrusted, including descriptions, instructions, nested schema text and extension metadata. Versioned rules identify instruction-override wording, approval-bypass wording, model-role delimiters and invisible or bidirectional formatting. A pattern match is labeled Review and needs contextual investigation; legitimate documentation can match. No match means only that the configured patterns were not found in collected text. Neither result demonstrates prompt-injection resistance or establishes a vulnerability. Tool annotation hints do not verify behavior or grant human-approval credit. [9]

Complete surfaces receive canonical SHA-256 baselines. JSON key order and catalog entry order do not change a baseline. Changes to descriptions, schemas, annotations or metadata are reported for review, not classified as malicious. Partial, invalid, blocked and unavailable collections cannot replace the last complete baseline. The latest bounded snapshot and complete per-surface digests are retained; this is not an archive of every prior definition body.

Collection labels are separate from scored evidence states: Observed records what was collected, Review identifies a specific condition to investigate, and Unknown records a collection gap. Authentication refusal does not establish that a provider has no tools. Authenticated or tenant-specific definitions remain unknown, and a package without a remote endpoint is unsupported for runtime definition discovery. Missing optional metadata does not reduce a score. These additional observations preserve the existing category budgets, criterion weights, mandatory evidence requirements and ranking floor.

Independent subcriteria and weighted credit

One criterion is one real safeguard with a stable control ID and a fixed point budget. Its independent subcriteria have descriptive check IDs, explicit evidence requirements and weights that sum to the parent budget. For example, the 24-point human-approval criterion assigns 8 points each to an identified human review interface, explicitly covered sensitive actions and an approval decision independent of the agent or client prompt. These are three distinct checks, not 24 repetitions of the same result. [2] [5]

Each subcriterion is evaluated using its own evidence key, source, state and observation time. A legacy parent pass, a sibling pass or an old one-point allocation cannot satisfy another check. A missing, unreadable, stale or future-dated observation earns no credit. Metadata checks establish precisely the advertised field or observed HTTP result named by the check. They do not establish runtime token, tenant or scope enforcement. [7]

Weighted credit is the sum of each applicable subcriterion weight multiplied by its earned fraction. Unknown and failed checks contribute zero. Not-applicable evidence needs an explicit reviewed determination permitted by the criterion. Documentation can establish a capability at the parent policy rate; checks requiring an authorized runtime test receive zero documentation credit. Atomic criteria such as advisory response and release age remain single checks rather than being split artificially.

Version 2 introduced independently evidenced subcriteria; Version 3 added observational definition collection. Version 4 retains their parent budgets and scored roster while revising public evidence confidence and the interpretation of anonymous initialization. Version 5 extends reproducible public collection receipts while preserving Version 4 scoring and confidence rules. Historical editions retain their original rules. Rank movement is shown only for comparable baselines. New catalog observations cannot change a score or confidence; scored admission remains explicitly versioned and simulated.

Scoring model and stability

For each category, earned baseline credit is divided by applicable baseline points, including unknown requirements at zero credit. The effective category weight is the smaller of its fixed budget and its applicable baseline points. The index normalizes the weighted category credits to a score from 0 to 100. Criterion credit is represented in thousandths, and the final score is rounded to the nearest integer.

In notation: effective weight = min(category budget, applicable baseline points); category credit = earned baseline points / applicable baseline points; score = round(100 × sum(effective weight × category credit) / sum(effective weight)). A category with no applicable points contributes no weight.

For a simple illustrative case with all 100 budget units applicable, unknown approval evidence earns zero of the 30 approval units. If every other category earns full credit, the resulting score is 70, assuming all other ranking requirements are met. Real configurations can have a different applicable denominator.

New criteria begin as observations outside the scored roster. Adding them does not change existing scores, scored confidence or ranks, although total catalog coverage changes. Promotion into scoring requires versioned review: existing weights and budgets remain fixed, the 20-unit evidence floor remains, and newly admitted weight must bound movement for already-graded providers to five points. A larger recalibration requires a new baseline edition.

Independent human approval

The baseline allocates 24 points to an independent approval flow for named sensitive actions and 6 points to request expiry. Attributable provider or execution-host documentation can meet this capability baseline. An agent-controlled approval flag, an ordinary client tool prompt or initial OAuth consent is not sufficient for this control.

Stripe documents Dashboard review for selected actions, independently of the client's tool-approval setting. Unapproved requests expire after 24 hours. Confirmation depends on the action: a generic write tool can also perform actions outside that approval flow. This is the reference capability, not a claim that every Stripe mutation requires human review. [5]

CoSAI discusses managed host confirmation and server-requested user confirmation as approaches to human oversight. The index recognizes attributable provider and host flows. Exact-action binding, bypass resistance and stronger enforcement assurance remain distinct advanced controls, visible outside the baseline score. [1]

Advisories and remediation timing

Advisories match a resolved package version or endpoint scope. OSV represents affected version ranges with events such as introduction and a fixed release; the presence of a reported fixed version is not evidence that a provider has deployed it. Remediation timing begins at the first attributable public disclosure and ends at a confirmed fixed release publication or reviewed incident end time. [6]

MCP-MNT-002 has four points. A complete, current advisory check with no matching advisory earns full credit. When advisories do match, the worst confirmed remediation latency determines credit. Missing timing evidence stays unknown, while an active affected release earns no credit for this criterion and affects ranking eligibility. Affected providers are checked hourly; this check's evidence expires after two hours without confirmation.

Catch-all execution as an independent finding

A catch-all red flag identifies an advertised generic execution surface: code, shell commands, unrestricted SQL, dispatch or generic requests. Detection uses inspected tool schemas or configured first-party documentation. Constrained enumerations do not trigger this rule; unavailable or partial inventories remain unknown unless an attributable source independently establishes the capability. Collection never invokes the tools.

CoSAI identifies overly permissive tool capabilities as a security boundary concern. The index treats this as a capability finding, independent of vulnerability advisories and baseline points. A documented approval flow and a catch-all red flag can therefore coexist. Stripe's generic API write tool illustrates why the two observations must remain separate. [1] [5]

Interpretation and limitations

The index describes demonstrated safeguards for a stated configuration at an observation time. First-party claims do not verify all production settings, and unauthenticated metadata does not establish authorization enforcement on every action. Hosted code, authenticated tool inventories and tenant-specific policies can be unavailable. Such gaps remain explicit rather than being filled by inference.

A complete advisory collection with no match is a bounded observation, not proof that no undisclosed vulnerability exists. Package code is not executed. Definition text patterns do not test model behavior or establish prompt-injection resistance, and functional categories are not risk ratings. Operators should read the control evidence, affected versions and independent findings alongside the score.

Dated editions preserve methodology text, control definitions, confidence policy, point rules, checks, taxonomy and references. From Version 3 they also freeze the definition-collection signal policy. The undated URL points to the current edition. Historical assessments retain their original calculation policy; editions with different scoring or confidence baselines are not compared as rank movement.

Deterministic checks

Checks describe what collection can establish. Unsupported observations remain explicit. A release waiting period is not a security verdict, and unavailable hosted code is not assumed to be safe.

Appendix A. Control catalog

Control IDs identify the same safeguard across providers. Baseline controls contribute to the score; advanced assurance remains visible outside it. Each criterion has a fixed weight. Independent subcriteria have distinct evidence requirements and contribute their stated share; atomic criteria retain one evidence check. Historical editions preserve their original allocations.

Hosted services

Appendix B. Provider classification

One reviewed primary functional category per MCP identity. Classify the exposed service, not its transport, name, security controls or safety score.

Edition history

Dated links preserve released methodology content. The undated address opens the current edition.

1. Version 5 · 2026-11-12-v5Current edition
2. Version 4 · 2026-11-12-v4
3. Version 3 · 2026-11-12
4. Version 2 · 2026-11-11-v2
5. Version 1 · 2026-11-11

Edition fingerprint: sha256:3f77fa825240f25bfce57cfdf1285f6a3fda5610e83ebe45212614b212122ddf