Audit: Human review coverage¶
What we check¶
Repositories that have the human review tracking tooling deployed
(see docs/code-review-tracking.md)
should keep the review backlog small: fewer than 5 in-scope files
needing review. Repositories without a .vscode/review-scope.toml
scope config do not have the tooling deployed and are reported as
not applicable; see the compliance page for which repositories
currently carry it.
The check runs scripts/review-tracking.py status against the
clone. A file needs review if it has never received a whole-file
review, or if its content at HEAD no longer matches the blob SHA
stamped for its last review (a stale review). Coverage is
deliberately recomputed against HEAD rather than trusted from the
committed REVIEWS.md: that file is only accurate immediately
after a prune, so a missed prune run cannot inflate the coverage
this audit sees.
Reviews imported from shakenfist/development (see "Importing
reviews from this repository" in
docs/code-review-tracking.md) count
toward coverage on the same terms as a native review: their blob
SHA must still match HEAD. They count because they attest to
identical bytes -- a review of a blob is a review of every copy of
it -- and import verified the signed commit that introduced each
one before recording it (unless explicitly told not to with
--no-verify, which REVIEWS.md then shows). The details line names how many of the
reviewed files were imported, when any were, so that coverage
arriving by import is not mistaken for a review session.
A run of review-tracking.py that does not answer -- an unusable
checkout, an interpreter that will not execute, a timeout, a non-zero
exit or output that is not its JSON -- is reported as error, naming
what went wrong, rather than as a failure of this criterion against
that repository. The tool failed, not the repository, so there is no
verdict: the open issue, if there is one, is left as it is. See "What
each status means" in
docs/consistency-audits.md.
Expect routine churn near the threshold: a single feature PR can touch five in-scope files, so the issue this audit files acts as a standing work-queue nudge -- it lists the files needing review, and closes automatically once a review session brings the backlog back under the threshold. The list is re-rendered on every run while the issue stays open, like every consistency issue's body (see Issues are the work tracking), so it names the backlog as of the latest run rather than as of filing.
Template¶
No template -- compliance is restored by doing review sessions,
not by copying files. See
docs/code-review-tracking.md
for the session workflow. Staleness on the default branch is
normally pruned automatically by the adopting repository's
prune-reviews workflow; this audit is the backstop that notices
when the backlog has grown regardless.
Projects¶
Per-project compliance for this criterion is regenerated on every run of the consistency audit: see the compliance page.