callwitness / research
The measurement on the baseline page is only worth as much as the method behind it. This is the method, in enough detail to be attacked, and every number on this page is computed from the same raw file the results come from.
Two numbers per server, and the relationship between them.
tools/list response as it crossed the wire. This is what a client pays to have the server available at all, before anything is called, and it is the number every published estimate of MCP context cost already counts.tools/call result as it crossed the wire. This is what actually reaches the model, and as far as I could find, nobody was counting it — because counting it means running the servers rather than reading their manifests.Both are measured at the transport, not reconstructed from the source. The proxy sits between client and server, forwards every line unchanged, and hands a copy to a recorder on another thread. It cannot alter what it measures, which is the point: an instrument that can change the stream is measuring itself.
Servers were drawn from the reference implementations, community directories, and registry searches, filtered to those that can be launched by a package runner and exercised without an API key, an account, or an external service. The filter is not editorial: a server that refuses to list its tools without credentials yields no measurement, so it cannot be in the distribution however popular it is.
Of 86 attempted, 82 started and answered tools/list, declaring 1,070 tools between them. 65 of those went on to answer a real call.
What happened to the other 21. 8 need credentials or infrastructure I do not have; 2 declare only tools that execute or write, and the read-verb rule will not call those; 5 were asked and the call errored; 4 never completed a handshake; and 2 had no call defined for them in the catalogue — a gap in the catalogue rather than a fact about the server.
The full list, with the exact argv used for each, is census/servers.json in the repository. Nothing was measured that is not in that file.
A census that calls tools on 82 servers is a census that will eventually call something it should not. This one nearly did: an early version of the harness would have invoked kubectl_delete, cleanup and run_process, all of which take no required arguments and would have been selected as easy calls.
So tool selection is a rule rather than a judgement. A tool name is split into words on non-alphanumeric boundaries and camel-case transitions, then:
safe_delete_all is not safe;504 declared tools were refused by this rule across the run. Where a server's entry in servers.json names explicit calls, those were used instead; the allowlist is the floor, not the plan.
One request at a time, and nothing concurrent. A server under parallel load is measuring the harness.
declared_bytes: 0 means no tools/list was observed, never that a server declared nothing, and no ratio is computed against a denominator that was not seen.n. Inside the published document, not in a footnote beside it, so a number cannot travel without its sample size.4 servers produced no measurement. They are in the dataset with their failure reasons rather than dropped from it — a census that silently excludes what it could not measure is reporting its own filter as a finding.
| reason | servers |
|---|---|
| started, never answered initialize | 4 |
A further 35 calls were made and came back as errors — a bad argument, a missing input file, a remote service refusing. Those are excluded from the size distribution, because an error message is not a response, but they are counted here so the two numbers can be reconciled: 140 successful calls out of 140 plus 35 attempted.
A further 12 servers started and declared tools but exposed nothing that the rule above would call. They contribute a declared size and no delivered size, which is exactly how they appear in the published baseline.
| package | what happened |
|---|---|
| devdocs-mcp | no response to initialize |
| ephemeris-mcp | no response to initialize |
| mcp-scholarly | no response to initialize |
| mcp-simple-arxiv | no response to initialize |
Most published Python MCP servers currently crash on a fresh install. The mcp SDK released a 2.x that renamed FastMCP and dropped mcp.server.fastmcp. Packages that never pinned mcp<2 now fail on import, before initialize, and a census sees them as dead servers.
In the first pass this affected 21 of 29 PyPI servers. Launching them as uvx --with "mcp<2" <package> recovers nearly all of them, which is why so many entries in servers.json carry that flag.
This matters for anyone reading the results as much as for anyone repeating them: a census run without that pin would under-count Python servers by roughly seventy percent, for a reason that has nothing to do with the servers being measured.
A tool's description is prose the model reads. 42 tools across 20 servers carry descriptions written as instructions to the model — imperatives, emphasis, "you must" constructions — rather than descriptions of what the tool does.
This is recorded, not judged. It is noted because it is invisible in any count of tools or bytes, and because a description that instructs is doing something different from a description that describes.
| package | tools |
|---|---|
| @agentdeskai/browser-tools-mcp | {'tool': 'refreshBrowser', 'flags': ['ordering']} |
| @knip/mcp | {'tool': 'knip-docs', 'flags': ['ignore', 'instead-of']} |
| @modelcontextprotocol/server-filesystem | {'tool': 'read_text_file', 'flags': ['you-must']}, {'tool': 'read_multiple_files', 'flags': ['you-must']} |
| @modelcontextprotocol/server-sequential-thinking | {'tool': 'sequentialthinking', 'flags': ['ignore', 'you-must']} |
| @sinco-lab/mcp-youtube-transcript | {'tool': 'get_transcripts', 'flags': ['shouting']}, {'tool': 'get_transcript', 'flags': ['shouting']} |
| @upstash/context7-mcp | {'tool': 'resolve-library-id', 'flags': ['do-not', 'shouting', 'you-must']}, {'tool': 'query-docs', 'flags': ['do-not', 'you-must']} |
| arxiv-mcp-server | {'tool': 'semantic_search', 'flags': ['ordering', 'shouting']}, {'tool': 'citation_graph', 'flags': ['instead-of']}, {'tool': 'check_alerts', 'flags': ['ordering']} |
| astronomy-mcp | {'tool': 'search_simbad_object', 'flags': ['do-not']}, {'tool': 'search_ned_object', 'flags': ['do-not']} |
| chrome-devtools-mcp | {'tool': 'fill_form', 'flags': ['prefer-this']}, {'tool': 'take_snapshot', 'flags': ['always-call']} |
| duckduckgo-mcp-server | {'tool': 'fetch_content', 'flags': ['ignore']} |
| firecrawl-mcp | {'tool': 'firecrawl_scrape', 'flags': ['instead-of']}, {'tool': 'firecrawl_search', 'flags': ['ignore']} |
| holidays-mcp | {'tool': 'is_public_holiday', 'flags': ['prefer-this']} |
| mcp-abacus | {'tool': 'analyze', 'flags': ['instead-of']}, {'tool': 'solver', 'flags': ['instead-of']} |
| mcp-gnu-units | {'tool': 'find_units', 'flags': ['instead-of']} |
| mcp-nixos | {'tool': 'nix', 'flags': ['do-not', 'prefer-this']} |
| mcp-pandoc | {'tool': 'convert-contents', 'flags': ['you-must']} |
| rcsb-mcp | {'tool': 'rcsb_find_go_terms', 'flags': ['instead-of']}, {'tool': 'rcsb_find_interpro_domains', 'flags': ['instead-of']}, {'tool': 'rcsb_find_enzyme_classes', 'flags': ['instead-of']}, {'tool': 'rcsb_find_disease_terms', 'flags': ['instead-of']}, {'tool': 'rcsb_find_organisms', 'flags': ['instead-of']}, {'tool': 'rcsb_query_strucmotif', 'flags': ['shouting']}, {'tool': 'rcsb_search_request', 'flags': ['instead-of']}, {'tool': 'rcsb_seqcoord_alignments', 'flags': ['ignore']}, {'tool': 'rcsb_seqcoord_annotations', 'flags': ['ignore']}, {'tool': 'rcsb_seqcoord_group_alignments', 'flags': ['ignore']}, {'tool': 'rcsb_seqcoord_group_annotations', 'flags': ['ignore', 'instead-of']} |
| tavily-mcp | {'tool': 'tavily_research', 'flags': ['you-must']} |
| uniprot-mcp-server | {'tool': 'uniprot_batch_entries', 'flags': ['instead-of']}, {'tool': 'uniprot_get_processing_features', 'flags': ['instead-of']}, {'tool': 'uniprot_get_ptms', 'flags': ['instead-of']} |
| wikipedia-mcp | {'tool': 'get_summary', 'flags': ['instead-of']}, {'tool': 'wikipedia_get_summary', 'flags': ['instead-of']} |
One machine, one operator, one set of arguments. The arguments were chosen by hand to be representative and safe, and a different set would produce different delivered sizes — that is the finding, but it is also the limitation, and it cuts both ways.
Delivered size is not a fixed property of a server. The same server measured twice, with the same declared size, moved by an order of magnitude because the argument changed. Treat the maximum as a lower bound on how large a response can get, never as a ceiling.
The sample is small and the servers are not a random draw. They are the servers that can be run without credentials, which is a real population but not the whole one. Servers requiring API keys are systematically absent, and they are not obviously similar to the ones here.
Nothing here measures quality. A large response is often a server working correctly. What is being measured is how much of a context budget a server can consume without having said so at install time.
Everything below is in the repository. The census is a script, not a service:
git clone https://github.com/AditiChaudharyy14/callwitness
cd callwitness
pip install -e .
python census/census.py --root /a/folder/with/real/content
python census/baseline.py
python census/render_baseline.pyTo measure your own servers instead of these, wrap them and generate the same document from your own traffic:
callwitness run --label docs -- npx -y <your mcp server>
callwitness baseline --out mine.jsonRaw data: census/data/census.jsonl for the current run and census/data/census-r1.jsonl for the first. Both are line-delimited JSON, one record per server, and both are what the published documents are computed from.