Conformance Suite v0.1
For Defining Agent-Directed Manipulation · 日本語
A standard is only a standard once everyone measuring reaches the same verdict. This suite checks whether your implementation agrees with the definition.
Running it
node run.mjs <path-to-your-implementation>Your implementation exports a function under one of default, scan, or scanHtml, in either of these shapes:
(html) => ({ findings: [{ severity: "info" | "warn" | "high" }] })
({ html }) => ({ findings: [{ severity: "info" | "warn" | "high" }] })Either is fine. Neither the argument shape nor the ordering of findings is part of the definition. Severity is compared using the maximum across findings.
No dependencies. Node 18 or later. Exit code is 0 when every case agrees and 1 otherwise.
What is in it
cases.json holds 18 cases. It contains both what must be detected and what must not be.
| Group | Cases | What it pins down |
|---|---|---|
| Four quadrants | 5 | That severity follows mechanically from region × visibility |
| Not detected | 10 | Advertising copy, structured data, AI mentions, hidden-but-harmless text, genuine reviews, conditional CSS |
| Not escalated | 1 | That an editorial review stays first_party and does not become high |
| Not concluded | 2 | That unresolvable external CSS is not declared hidden |
| Total | 18 |
13 of the 18 expect no finding at all, and that imbalance is deliberate. The substance of this standard lies more in what it does not flag than in what it does. For a judgement like this one, false positives are the only fatal failure.
Both languages are exercised
Every case carries a lang field, and the runner reports the split:
Definition v0.1 conformance: 18 / 18 agree (call form: scan(html))
by language: ja 7/7 en 9/9 ja+en 1/1 n/a 1/1Revision 1 of this suite tested several rules in Japanese only. An English-only implementation could pass it without ever being exercised — or fail quadrant-fp-hidden for a reason that was not its fault. Revision 2 adds an English counterpart for every such case. Both languages are in scope for the definition, so both are in scope for the suite.
A reference implementation is included
reference/scan.mjs implements the definition in one dependency-free file and passes all 18 cases. Diff your verdicts against it when a case disagrees.
curl -s https://example.com | node reference/scan.mjsIt is not normative. Where it and the definition disagree, the reference implementation has the bug.
If you disagree
Your implementation is not necessarily the one that is wrong.
A disagreement marks a place where the definition is open to more than one reading. Each case in cases.json carries a clause and a why describing what that provision is trying to protect. If you read them and remain unconvinced, the definition is the more likely culprit — please open an issue.
Files
cases.json— 18 cases with expected verdicts and the intent behind each provisionrun.mjs— the runner. No dependenciesREADME.md— this page as text
Every case carries clause and why in English, with clause_ja and why_ja alongside. Read why before concluding your implementation is wrong.