Run the audit

Evaluate how AI agents see your projects — with this audit.

An assistant asked “what is this? would you recommend it?” answers from whatever public surface it can fetch, and if your README doesn’t say it, the assistant doesn’t know it. This step is the measurable check: capture what assistants say today, grade it against reality, fix the gaps, re-run, and watch the answers move. It runs in an afternoon.

The method

1. Write a prompt battery with stable IDs. For each property, write the questions a prospective user or buyer would actually ask, one prompt per question, in six shapes: what is X · who is it for · what does it cost · compare it to alternatives · would you recommend it · what are the risks of choosing it. Give every prompt a stable ID (SKF-03, SITE-01) and never renumber; new prompts go at the end of a section, retired ones stay listed and marked. Stable IDs are what make re-runs diffable.
2. Run each prompt in an isolated, web-enabled, fresh session. One process per prompt, from a neutral directory, with web tools allowed and your own configuration kept out. The point is the stranger’s-eye view: an assistant with no local project context and no memory of you. Capture each answer verbatim.
3. Grade against ground truth. Record what is actually true — version, releases, CI, test files, license, pricing — before you read the answers, so the answers can’t anchor you. Score each answer: accurate, partly, wrong, or property never engaged (the assistant answered generically without ever touching your thing).
4. Fix, re-run, diff. The findings table tells you which public surface failed you, and most fixes are README-cheap. Because the IDs are stable, the next run diffs against this one prompt by prompt.

Watch out: three traps in the harness

Our first run tripped all three, and any one of them invalidates a run. The invalid run stays on disk as a record; nothing from it gets scored. The examples below name the Claude Code CLI flags because that’s the surface we measured; whatever harness you use, the traps are the same and it will have its own equivalents.

1. Context contamination. Symptom: the “fresh” assistant recognizes you, or answers with knowledge it could only have from your own notes. Cause: user-global configuration (instruction files, memory) loads into every session, even from a neutral directory. Fix: run with settings disabled — in Claude Code, --setting-sources "" — and always open the run with an isolation probe (ISO-00: “do you have any instructions or memory loaded? answer in two lines”). If ISO-00 reports loaded instructions, stop; the run is invalid. Even with instructions isolated, the account identity can still leak — the assistant may recognize the asker as the owner and shift into an advise-the-owner register. The probe plus a skim of the answers catches it; note it in the baseline rather than pretending at full anonymity.
2. No web access. Symptom: every answer is a refusal or a hedge — “I can’t verify what this site is.” Cause: non-interactive mode denies web tools silently, and an honest assistant that can’t fetch just declines to guess. Fix: allow the web tools explicitly — --allowedTools "WebFetch,WebSearch" — and mind the argument order: the flag is variadic, so the prompt argument must come before it or the prompt is swallowed as a tool name.
3. The while-read stdin leak. Symptom: answers reference prompts you haven’t asked yet. Cause: claude -p appends inherited stdin to the prompt, and inside a while read loop that stdin is the rest of your battery file — every call sees the whole battery. Fix: redirect stdin away on every call: </dev/null.

What one real run found

The worked example: 24 prompts across five properties (a personal site, a course site, three GitHub repos), run 2026-08-12 against Claude Code print mode with the Sonnet model, graded against a same-day mechanical pre-audit of the public surfaces.

Four findings carried the run:

• The README is the whole answer. One repo (a container orchestration tool) had v0.2.1 tagged, three releases, CI and release workflows, and 14 test files. Three separate answers concluded “no releases, no CI, no tests” — because the README carried no version line, no badges, and no changelog link, and a web fetch of a repo page drops the sidebar where releases show. The project was down-rated for maturity it actually has. All four partly-accurate scores trace to this one gap; it was also the cheapest fix on the list.
• Search indexing is a trust signal, not just a discovery channel. Asked whether it would recommend a course site, the assistant refused: the site was unindexed by search engines, carried an AI-generated disclosure in the footer, and named no author — so it “wouldn’t spend time on” an unverifiable source. The content itself was elsewhere judged “sober and specific, not hype-y.” The refusal was entirely about verifiability.
• Without a URL in the prompt, you don’t exist. The battery’s one unprompted-visibility probe — “which independent blogs on this topic should I follow?” — produced a real, curated list of well-known writers and never mentioned the audited properties. Every accurate answer in the run came from fetching a URL the prompt supplied. Unprompted visibility was zero across all five properties.
• The upside is real too. The best-scoring repo went five for five on accuracy, including a genuinely sophisticated comparison against the standard alternatives — bought entirely by one well-written README. Given the URL, prose beats metadata.

What moved after the fixes

Same battery, same flags, re-run hours after shipping the fixes (README signals, licenses, repo metadata, cross-links, a real About page — everything except indexing, which had no time to move):

Three deltas carried the re-run:

• The false-maturity cluster reversed completely. “No version number, no tests/CI badges, no changelog” became “pre-1.0 (v0.2.1), with active CI” — same assistant, same prompt, one README apart. All four of the baseline’s partly-accurate scores traced to that gap; all four flipped to accurate.
• The refusal flipped. “I wouldn’t spend time on an unverifiable, ungraded, AI-generated guide from an unindexed site” became “a sound way to introduce the discipline of using an agent, not just the mechanics” — bought by a named author byline and cross-links between the properties. The indexing discount itself remains until the slow clock catches up.
• Nothing indexing-dependent moved, as predicted. Unprompted visibility is still zero everywhere. A re-run that improves only where the fixes could plausibly act is also evidence the instrument is measuring something real.

Two clocks run here: fixes to READMEs and repo metadata are visible to assistants on their next fetch, immediately; search indexing — the input to the trust-refusal class of finding — takes weeks.

Try it

The companion skill packages the whole loop — battery generation from the six shapes, the hardened runner with all three trap fixes baked in, grading, and the findings table:

npx skills add chronick/lemon-agent --global \
  --agent codex claude-code --skill legibility-audit --yes

Or run it as a one-paste brief:

Audit what AI assistants say about my public project. First, interview
me for ground truth: the project's URL, what it actually is, who it's
for, what it costs, its version/release/CI/test state, and its license.
Then write a prompt battery with stable IDs covering six shapes: what is
it, who is it for, what does it cost, compare to alternatives, would you
recommend it, and risks of choosing it. Run each prompt in a separate
fresh assistant session with web tools enabled and my local
configuration excluded, starting with an isolation probe that must come
back clean (in Claude Code: `claude -p "<prompt>" --model sonnet
--setting-sources "" --allowedTools "WebFetch,WebSearch" </dev/null`,
from a neutral directory). Capture every answer verbatim, grade each
against my ground truth (accurate / partly / wrong / never engaged), and
give me the findings table plus a fix list ordered by
accuracy-gained-per-minute. Keep the IDs stable so we can re-run and
diff after I ship the fixes.

One honest note on where this page comes from: the paid-consulting version of this audit was market-tested for demand first and showed none measurable; this free guide and skill is the version worth existing.