Skip to content

Bot bypass identity

Prompt

Before responding to questions or discussion points in this document, explore this repository thoroughly. Read the relevant files and ground your answers in what they actually say. Do not speculate about the repository when you could read it instead. Flag any uncertainty explicitly rather than guessing.

There is no application code here. The artifacts are the audit specifications in docs/audits/, the tooling in scripts/ that measures them, the templates in templates/ that the rest of the fleet copies, and the workflows that run all of it every week against every Shaken Fist repository.

Consult AGENTS.md for the conventions and the invariants that are not visible in the code, and ARCHITECTURE.md for the shape of the system. docs/consistency-audits.md is the reference for what a run does, how to add a criterion, how to bring a repository into scope, and how to test a change before it reaches the fleet -- read it before changing anything under scripts/ or docs/audits/. docs/code-review-tracking.md covers the review tooling, and PUSH-AUDIT.md is the pre-push review runbook that every plan's final phase runs.

Two things make planning here different from planning in a repository that holds a product, and both should shape any plan written from this template:

  • The blast radius is other people's repositories. The weekly workflow files and closes GitHub issues fleet-wide. A change that is merely wrong does not produce a red build; it produces issues in ten repositories, or silently closes ones that should have stayed open. Always pass --dry-run when running audit-manage-issues.py by hand.
  • This repository is in its own audit matrix. A standard we exempt ourselves from is a standard we stop noticing the cost of. A change to a criterion is a change we are measured against on the next run, so a plan should say how many repositories -- including this one -- it newly fails.

Plan file conventions (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/plan-file-conventions.md):

  • All planning documents live in docs/plans/.
  • Detailed planning gets one plan file per phase. Phase files are named for their master plan, sit in the same directory as it, and append -phase-NN-descriptive before the .md extension.
  • The master plan tracks its phases in a table under its Execution section. Merged is last, and the push audit is the last row, for the reasons given in plan-push-audit-phase:
Phase Plan Status Merged
1. Schema migration PLAN-thing-phase-01-schema.md Not started
2. Public API PLAN-thing-phase-02-api.md Not started
3. Push audit - Not started
  • One commit per logical change, and at minimum one commit per phase. Unrelated changes are not batched into a single commit. Each commit is self-contained: it builds, passes tests, and has a message explaining what changed and why.

In this repository. Plans here keep their phases as sections inside the master plan rather than as separate phase files, and the Execution table's Plan column is dropped accordingly; docs/plans/index.md says so. The shared convention above is the fleet default, and a plan large enough to want phase files should use them rather than argue with the block.

Situation

#211 asks for DEPENDENCIES_TOKEN to be replaced, because PLAN-workflow-ref-trust.md's step 2a would make shakenfist-bot a bypass actor on shakenfist/actions main. It frames the problem as one that step would create. Read through the GitHub API on 2026-10-07, it already exists in six repositories, and is wider than the issue says in four ways. Everything below was re-read on 2026-10-14 (Australian Eastern Daylight Time, UTC+11, which is the evening of 2026-10-13 in UTC), before the plan went to review, and the figures are from that day. The rulesets, the bypass team's membership and the bot's role were read again later on 2026-10-14, while the review was being answered, and had not changed.

The bypass is already in place. Six default-branch rulesets require a pull request and status checks, and every one of them names the same bypass actor, the team sf-can-skip-merge-queue (id 11722172), with bypass_mode: always. Five of them also require the merge queue (instar and client-python-k3s added it between the two readings).

Repository Rules on the default branch Bypass
shakenfist pull request, status checks, merge queue sf-can-skip-merge-queue
kerbside pull request, status checks, merge queue sf-can-skip-merge-queue
ryll pull request, status checks, merge queue sf-can-skip-merge-queue
instar pull request, status checks, merge queue sf-can-skip-merge-queue
hunkydory pull request, status checks sf-can-skip-merge-queue
client-python-k3s pull request, status checks, merge queue sf-can-skip-merge-queue

The team has three members: mikalstill, shakenfist-bot and sfconductor. A ruleset bypass belongs to an identity, not to a token, so every credential that authenticates as shakenfist-bot can push straight to those six default branches, skipping review, required checks and the merge queue.

The bot is also an organisation owner. orgs/shakenfist/memberships/shakenfist-bot reports role: admin, and repos/shakenfist/<r>/collaborators/shakenfist-bot/permission reports admin in all ten of shakenfist, kerbside, ryll, instar, hunkydory, client-python-k3s, actions, development, sfui and kerbside-client. (sfconductor is an ordinary member.) No ruleset names OrganizationAdmin or the admin repository role as a bypass actor today, so ownership does not bypass anything by itself. But a repository admin can rewrite the ruleset, and can widen an environment's deployment branch policy. So a bot credential with enough scope can add a bypass back, delete the ruleset, or open the environment this plan puts the App's key in, whatever team the bot is in. Taking the bot out of the team is necessary and not sufficient (open question 5).

More than one credential is the bot. DEPENDENCIES_TOKEN is not the only one. RENOVATE_TOKEN is a separate secret, but every Renovate pull request in shakenfist, kerbside and ryll (the last 14 in each) is authored by shakenfist-bot, so it is the bot's credential too. Live workflows on default branches that put one of the two in scope:

Secret Workflows Triggers
RENOVATE_TOKEN renovate.yml in 17 repositories, including all six above schedule, workflow_dispatch
DEPENDENCIES_TOKEN prune-reviews.yml in actions, development, hunkydory, kerbside, kerbside-client, ryll, sfui push, schedule, workflow_dispatch
DEPENDENCIES_TOKEN pin-indirect-dependencies.yml in shakenfist, kerbside pull_request, schedule, workflow_dispatch
DEPENDENCIES_TOKEN code-formatting.yml in shakenfist, client-python pull_request, schedule, workflow_dispatch
DEPENDENCIES_TOKEN functional-tests.yml (claude-code delinter job) in shakenfist pull_request, merge_group
DEPENDENCIES_TOKEN sync-external-docs.yml in shakenfist schedule, workflow_dispatch

None runs on pull_request_target, issue_comment or workflow_run, so nobody outside the organisation can fire one. The exposure is to whoever and whatever can run code beside the secret. That includes every Renovate run, which executes renovatebot/github-action (referenced by tag, not sha, until PLAN-workflow-ref-trust.md phase 3) and whatever package-manager code a lockfile update runs. It also includes the pull-request jobs above, which the templates already say are guarded against forks and against accident, but not against a writer (templates/pin-indirect-dependencies/pin-indirect-dependencies.yml:58-65).

Where the secrets live is not visible to this repository. actions has no repository secrets at all, yet its prune-reviews pushed as shakenfist-bot 44 times in a quarter, so at least DEPENDENCIES_TOKEN is an organisation secret. Its visibility (all repositories, or selected ones) needs admin:org to read. If it is visible to every repository, a push of a new workflow to any branch of any of them reaches it.

Only one job needs the bypass. The ruleset rule-suite history for the last month shows what the bypass is actually used for:

Repository shakenfist-bot sfconductor mikalstill
shakenfist 0 83 4
kerbside 34 14 8
ryll 49 2 4
hunkydory 16 0 0
instar 0 6 1
client-python-k3s 0 1 1 (a tag)

Every shakenfist-bot bypass is a prune-reviews push of review marks to develop: each one's commit subject is "Prune stale review marks." or, since PLAN-review-import.md added the import step, "Prune and import review marks.". The bot's other writers do not need one: Renovate, pin-indirect-dependencies, code-formatting, the delinter and sync-external-docs (through actions/tools/ci_sync_docs.sh:59-65) all push a branch and open a pull request. No workflow in the fleet merges a pull request as the bot. sfconductor's bypasses are pull request merges that skip the queue (for example shakenfist#4467, a Renovate bump); it is a separate identity, outside this plan except as open question 3.

Nothing in the fleet mints a GitHub App token today. There is no actions/create-github-app-token reference in any workflow or template.

How prune-reviews uses the token today. The template and this repository's deployed copy are byte-identical, and all seven live copies still name DEPENDENCIES_TOKEN. Its one job, prune-and-import, runs on [self-hosted, static] and has no actions/checkout. Since #254 its first step, Fetch the prune script, has no token. It clones the default branch into RUNNER_TEMP, ignoring the runner's git configuration. The second step, Prune and import review marks, runs prune-script/tools/ci-prune-reviews.sh from that clone, and hands the push credential to it as the PUSH_TOKEN environment variable (secrets.DEPENDENCIES_TOKEN || github.token). The script gives it to its one git push. scripts/tests/test_ci_prune_reviews.py holds the workflow to that shape (see step 2a). The template README's "Runner isolation" section says why: the static runners are long-lived, share one home directory across jobs and repositories, and run pull-request code. The commit's author and committer are set by the script (git config user.name 'shakenfist-bot'), not by the credential, so the commit metadata does not say which identity pushed it; the rule-suite history's actor_name does.

Mission and problem statement

Make the identity that can bypass a default-branch ruleset one that only reviewed default-branch code can use, and take that power away from every credential the bot carries elsewhere. Concretely:

  1. A GitHub App, with contents: write and nothing else, becomes the only automation bypass actor on the rulesets whose repositories run prune-reviews, and on actions when PLAN-workflow-ref-trust.md step 2a adds one.
  2. Its private key is an environment secret, in an environment whose deployment branch policy admits only the default branch. A job on any other ref, including a pull request that edits the workflow, cannot read it. That is the property a repository or organisation secret cannot have. The branch policy decides which job is given the key, not what else has run on the machine that job runs on, so the job also moves off the shared static runners (open question 6).
  3. prune-reviews mints a short-lived installation token from it, and the template, its README and every adopted copy change to match.
  4. shakenfist-bot leaves sf-can-skip-merge-queue, and stops being an organisation owner (open question 5). Its PATs keep working for what they actually do, pushing branches and opening pull requests that retrigger CI. They can no longer reach a protected branch directly, rewrite the ruleset that protects it, or widen the environment that holds the App's key.

This closes #211 and lets PLAN-workflow-ref-trust.md step 2a name the App as its bypass actor rather than a team of one.

Out of scope:

  • Replacing DEPENDENCIES_TOKEN and RENOVATE_TOKEN with fine-grained or App tokens. Once neither can bypass, what they can do is push branches and open pull requests, which still need review and CI to land. Narrowing them is worth doing and is listed in Future work.
  • sfconductor and mikalstill as bypass actors (open question 3).
  • Moving prune-reviews to open pull requests instead of pushing. PLAN-review-import.md keeps the push, and this plan does too.
  • Pinning renovatebot/github-action to a sha. PLAN-workflow-ref-trust.md phase 3 owns pinning. The one exception is the new actions/create-github-app-token reference. It handles the bypass identity's private key, so step 2a pins it to a sha with a # vX.Y.Z comment, in the form that criterion will accept, rather than waiting for phase 3.

Open questions

  1. One App for the fleet, or one per repository? One App installed on selected repositories, with the key duplicated into each repository's environment, is one thing to rotate and one bypass actor id for every ruleset and for the moving-ref-source-protection criterion to compare against. Per repository Apps contain a leaked key to one repository but multiply everything else by seven. Default: one App, installed only on the repositories whose prune-reviews must bypass a ruleset: today hunkydory, kerbside and ryll, since shakenfist, instar and client-python-k3s carry no prune-reviews, plus actions once step 2a lands and any of the six that adopts review tracking later. It is named for what it does rather than for the bot, for example shakenfist-review-marks.
  2. What do repositories without a pull-request ruleset use? development, sfui and kerbside-client push review marks today with DEPENDENCIES_TOKEN || github.token, and actions does until step 2a. A repository with no ruleset to bypass gains nothing from the App, and github.token with contents: write already pushes there. Default: they move to github.token, which needs no secret. They still get an empty review-marks environment with the default-branch policy (open question 4). The template mints the App token only when the environment's secret is present, and PUSH_TOKEN falls back to github.token otherwise, so one file still serves both kinds (as the DEPENDENCIES_TOKEN bullet in templates/review-tracking/README.md promises today). Note that github.token pushes do not start workflows, where PAT pushes did (an App installation token's pushes do): confirm nothing in those four repositories depended on the retrigger. The token comment above the job's permissions: block says the retrigger finds nothing to do, so this should be a no-op.
  3. What about sfconductor? It is a user account in the same team, with more bypasses than the bot, and this repository does not know where its credential lives. Default: out of scope. Phase 1 records what it is and where its token is held, and files an issue if the same reasoning applies, rather than changing its access here.
  4. Does a missing environment fail closed? GitHub creates an environment on first reference if it does not exist, with no branch policy. A repository adopted later, whose operator forgot to create the environment first, would run the job in a new, unprotected environment holding no secret, mint nothing, and fall back to github.token, whose push the ruleset then refuses. Default: that is failing closed, and acceptable. But an empty, unprotected environment is a trap for whoever later adds the key to it. So every repository that runs prune-reviews gets a review-marks environment with the default-branch policy before its first run of the new template, whether or not it will hold the key (steps 1b and 2b). Then "the environment exists" implies "it admits only the default branch" everywhere. The adoption skill and the template README say so. They also say the key is only ever an environment secret, never a repository or organisation secret: secrets.REVIEW_MARKS_APP_KEY resolves at all three levels, and the mint-when-present guard would work just as quietly with a key that any branch can read. Step 3c reports both an environment without the policy and the key held at repository level.

The default-branch policy, wherever this plan uses the phrase, means exactly this, and nothing looser. repos/{org}/{repo}/environments/review-marks has deployment_branch_policy equal to {"protected_branches": false, "custom_branch_policies": true}. .../deployment-branch-policies holds exactly one entry, with type: branch and name equal to the default branch, containing no glob characters. "Admits the default branch" is not the test. A protected_branches: true policy admits any protected branch, a branch name is a glob, and a policy can be type: tag, so a set such as {develop, tag *} admits the default branch and also lets any writer push a tag at a commit with an edited workflow and read the key. 5. Does shakenfist-bot stay an organisation owner? It is one today (Situation), which makes it an admin of every repository, able to edit any ruleset and any environment's deployment branch policy. The second matters as much as the first. A bot credential that widens the review-marks policy lets a branch workflow read the App's key, which mints bypass tokens for every repository the App is installed on. DEPENDENCIES_TOKEN is reachable from pull-request jobs and Renovate runs. None of its known jobs needs that: Renovate, pin-indirect-dependencies, code-formatting, the delinter, sync-external-docs and, after phase 2, nothing that pushes to a protected branch only need to push branches and open pull requests. What is not known is whether some other credential is the bot. AUDIT_TOKEN is the obvious candidate, and reading bypass_actors needs a caller that can edit the ruleset. Default: demote it to an organisation member with write, not admin, on the repositories its credentials push to. Step 1a records whose account each of DEPENDENCIES_TOKEN, RENOVATE_TOKEN and AUDIT_TOKEN is, and its scopes. If any of them is the bot's and needs admin, the demotion waits until that credential moves to an identity of its own. That wait is a hard gate, not something to record and move past. Step 1b demotes the bot before it adds the key anywhere. No repository's environment is given the key while any automation identity is an admin there, because Mission item 2 would be false in that repository from the moment the key arrived. 6. Which runner holds the key? prune-and-import runs on the static pool, which runs pull-request code from every repository and shares one home directory across jobs (template README, "Runner isolation"). The environment's branch policy decides which ref's job is given the key. It does not decide what else has run on the machine: a process an earlier pull-request job left behind can read the key from the token step's environment, or the minted token from the script's. The PAT has the same exposure today, so this is not a new hole, but Mission item 2's claim is stronger than a static runner delivers. Default: move prune-and-import to the fleet's ephemeral vm runners, as pin-indirect-dependencies uses ([self-hosted, vm, debian-13, m]), in the one template. Step 1a confirms those labels serve all seven repositories that run prune-reviews. If they do not, or the boot time makes the job impractical on every push, keep static and say so instead, in Mission, the template comment and the criterion spec's reviewer-confirmed half: anyone who can run a pull-request job on the static pool can reach the key.

Execution

Phase Status Merged
1. Operator inventory and App creation Not started
2. prune-reviews mints an App token Not started
3. Move the bypass and enforce it Not started
4. Push audit Not started

Plan status vocabulary (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/plan-status-vocabulary.md):

A status cell -- in the master plan's own Execution phase table, and in the row docs/plans/index.md carries for the plan -- holds exactly one of these terms and nothing else:

  • Proposed -- written down as a concept, not yet scheduled.
  • Not started -- scheduled, but no work has begun.
  • In progress -- work has begun and has not finished.
  • Blocked -- cannot proceed until something outside the plan changes. Say what, in the plan.
  • Complete -- the work is done.
  • Abandoned -- deliberately dropped without being done.
  • Superseded -- replaced by another plan, which the plan names.

The term is the whole cell. No dates, no phase arithmetic, no parenthetical qualifiers, no summary of what happened: a status is read to decide whether a plan still wants attention, and prose in that column has repeatedly grown until it could no longer be read either by a person scanning the table or by tooling. Detail belongs in the plan file, and a one-line summary belongs in the index's own Intent column.

Matching is case-insensitive, so In Progress is accepted, but the spelling above is the one to write.

In this repository. The same term is written twice: once in this plan's own phase table, and once in the row the plan carries in docs/plans/index.md. The index row is the whole-plan status, so it only reaches Complete once every phase has been completed, abandoned or superseded. The plan-index criterion reads that table, and this repository is inside its own audit matrix, so a status that drifts out of the vocabulary fails our own tooling before it fails anybody else's.

Push audit phase (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/plan-push-audit-phase.md):

  • Every master plan ends with a phase that runs the repository's PUSH-AUDIT.md over the whole plan's work. It is the last row of the Execution table and it is not optional. The rule binds every plan that carries the phase, which is decidable from the plan file alone: a plan that is already Complete, Abandoned or Superseded and does not carry the phase is not reopened to acquire one, and a plan that has the phase runs it even if it reaches Complete before the phase does.
  • That phase audits the accumulated diff of every phase in the plan against the default branch, not the diff of the last phase alone. Auditing one phase at a time would miss what the phases did to each other -- the duplicated helper that only exists once phases three and six have both landed, the doc page that phase two made wrong and phase five never revisited.
  • Once the plan's phases have merged, a diff against the default branch is empty and would read as a clean audit. The range is not reliably derivable after the fact either: unrelated work lands on the default branch between phases, so anything anchored on "since the plan file appeared" is far too wide. It has to be recorded. As each phase lands, what put it on the default branch goes into the plan: the merge commit of its pull request, whose diff against its first parent is the whole of what landed, or -- where the phase landed directly -- every commit of the phase, or its first..last range. A single commit is only ever enough when it is a merge commit.
  • Where the Execution phases are a table, that record is a Merged column, added last so that a row which omits it still reaches Status; where they are prose sections it is a Merged: line in the phase's own section. The Status column keeps its single vocabulary term and nothing else (see plan-status-vocabulary). A phase that landed in another repository records <repo> <sha> (#pr) and is audited against that repository's default branch, as part of the pull request that lands it; the plan's own push-audit phase cites that audit rather than re-running it.
  • Phases that landed before the plan started recording them are reconstructed rather than left blank. Recover what you can from gh pr list --state merged and git rev-list --first-parent, and say in the plan that the range was reconstructed. Do not trust a path-filtered git log on its own: it lists the commits that touched a path without saying which arrived directly and which arrived inside a pull request, and recording a commit that came in under a merge audits one commit of that pull request rather than the pull request. A reconstructed record may be a summary table in the audit phase's own section rather than a column or a line in the Execution table, which keeps retrospective archaeology out of a table that tracks live status. Where a phase accreted over months of unrelated commits and no range is recoverable, say that instead and name the paths the audit read -- an audit that says what it could not scope is a result; one that silently audits nothing is not.
  • Findings land as their own pull request against the default branch, and the plan is not complete until they are resolved or explicitly declined in writing. A finding that is declined says why, in the plan, where the next reader will find it.
  • Where the audit finds nothing, record that in the plan in one sentence. It is a real result, and a run of them is the evidence for making the phase conditional rather than mandatory.
  • A repository with no PUSH-AUDIT.md still carries the phase, and the phase says that the runbook does not exist yet and what was done instead. Silently omitting it is what let the audit go untriggered for as long as it did.

In this repository. PUSH-AUDIT.md exists at the repository root and is referenced from AGENTS.md, so the final phase runs it rather than explaining its absence. Note that every diff command in it is written against main...HEAD: a stale local main silently widens the audit to unrelated history, so fetch before starting, or read it as origin/main...HEAD.

Phase landing (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/plan-phase-landing.md):

A plan's status and a repository's review state both live in files that every branch would otherwise rewrite. Left alone, that turns each of them into a merge-conflict hot spot, and it spends a pull request and a full CI run on a change that is entirely prose. Three rules keep them out of the way.

  • A phase is closed out in the first commit of the next phase, not in a pull request of its own. By the time the next phase branches, the previous one has merged, so its merge commit is known and its Merged cell can record the thing the push-audit phase actually needs. This is the only ordering that works: a phase cannot record its own merge commit, and a separate close-out pull request buys that record at the price of a round trip. The close-out sets the finished phase's Status and Merged cells and the plan's row in docs/plans/index.md, and it is committed before the next phase's own work, so that the branch never claims the plan is further along than the default branch is.

  • The last phase closes itself out. The push-audit phase is the last row of every plan, so no next phase will carry its close-out. Where the audit raises findings, the plan is not complete until they are resolved or declined, and those land as their own pull request after the audit phase has merged -- so that pull request is the carrier, and it can record the audit phase's merge commit, which by then is known. Where the audit finds nothing there is no carrier, and no follow-up pull request is opened for the sake of one cell: the phase sets its own Status, and the plan's index row, to Complete in its own pull request, and records no Merged cell. It is the only row permitted to omit one. The column exists so that the push-audit phase can reconstruct what to audit; the audit phase is last, so nothing ever reads its own row.

  • REVIEWS.md is not pruned or regenerated in a pull request that changes code or documentation. Editing a reviewed file stales its mark, and adding or removing an in-scope file moves the header count, but neither is the landing pull request's business. prune regenerates the file whether or not it dropped anything, so the prune-reviews workflow heals both on the next push to the default branch. Pruning from a branch is also wrong more often than it is right, though not for the reason it first appears: prune compares each stamp against HEAD, which on a branch is the branch tip, so it drops the marks for the files the pull request itself touched while keeping marks the default branch has already pruned. Committing that state merges a review file computed from a stale tree, and can resurrect marks prune-reviews has already removed. Accumulated staleness is reported by the review-coverage audit, which recomputes coverage against HEAD and raises an issue once the backlog is worth a review session.

A review session is the exception, and it is not optional tidiness: stamp regenerates REVIEWS.md as well as writing the marks, and the rows, the sidecars and the marks are committed together (see docs/code-review-tracking.md). Where a repository requires a pull request to reach its default branch, that is how a review session lands, so "not in a pull request" is about the kind of change, not the mechanism.

These rules assume phases land one after another. Where two phase branches are open at once, each closes out only the phase it directly follows.

In this repository. The plan index is docs/plans/index.md. review-tracking-tests deliberately does not assert the REVIEWS.md header count, so a phase that adds or removes an in-scope file needs no regeneration commit; prune-reviews corrects the count on the next push to main.

Phase 1: operator inventory and App creation

Planning effort: medium. Review effort: high, because it changes organisation settings that no test in this repository covers.

This phase is done by the management session with the operator. It needs admin:org, which neither the audit token nor a sub-agent has, and nothing in it lands as code here except what it records in this section.

Step Effort Model Isolation Brief for sub-agent
1a high opus none Operator step. With gh auth refresh -s admin:org, record here: each organisation secret's name and visibility (gh secret list --org shakenfist and, for a selected secret, gh api orgs/shakenfist/actions/secrets/<name>/repositories); which repositories hold DEPENDENCIES_TOKEN or RENOVATE_TOKEN as a repository secret; and what sfconductor is, who operates it and where its credential is held (open question 3). Re-run the rule-suite query from Situation (repos/shakenfist/<r>/rulesets/rule-suites?time_period=month&rule_suite_result=bypass) for the six repositories and confirm that shakenfist-bot's bypasses are still only prune-reviews pushes. If anything else appears, stop: phase 3 would break it. Record shakenfist-bot's organisation role (gh api orgs/shakenfist/memberships/shakenfist-bot --jq .role, admin on 2026-10-14) and its permission on each of the ten repositories in Situation (repos/shakenfist/<r>/collaborators/shakenfist-bot/permission). Then settle, by measurement, whose account each credential is. Use a throwaway workflow on a branch of this repository, deleted afterwards, since all three secrets are in scope here. For each of DEPENDENCIES_TOKEN, RENOVATE_TOKEN and AUDIT_TOKEN, it prints gh api user --jq .login and the X-OAuth-Scopes response header (never the token). GitHub sends that header only for classic PATs and OAuth tokens: a missing header means the token is fine-grained or an App's, not that it has no scopes, so record its type and work out its reach by probing (can it read a ruleset's bypass_actors, or edit an environment?) instead. For AUDIT_TOKEN alone it also prints gh api repos/shakenfist/hunkydory/rulesets/23805020 --jq 'has("bypass_actors")'. Also record: the organisation rulesets (orgs/shakenfist/rulesets), and whether any applies to the six default branches (none had classic branch protection on 2026-10-14: branches/develop/protection reports "Branch not protected" in all six); every repository or organisation secret named REVIEW_MARKS_APP_KEY (there should be none, now or ever); and which runner labels serve the seven repositories that run prune-reviews (open question 6). Record all of it here, and settle open questions 5 and 6 from it. The bypass_actors answer decides which shape step 3c takes. PLAN-workflow-ref-trust.md step 2a asks the same question, and cites this record rather than measuring again (step 3b). The probe workflow runs on an ephemeral vm runner, never static, prints only the login, the header and the boolean, and is deleted together with its branch and its run logs once the record is written. The same applies to 3a's rejection workflow.
1b high opus none Operator step. First, change shakenfist-bot's organisation role and repository permissions as open question 5 settled (organisation member, write not admin), and record repos/shakenfist/<r>/collaborators/shakenfist-bot/permission for all ten repositories afterwards. Its bypass is through the team, not its role, so prune-reviews keeps landing. If open question 5 could not demote it, stop here: the key goes into no repository where an automation identity is still an admin. If AUDIT_TOKEN is the bot's, re-run 1a's bypass_actors probe now, since 3c runs under the state this leaves. Then create the App under the organisation as open question 1 settles it: repository permission contents: write only, no webhook, no user authorisation. Install it on selected repositories: hunkydory, kerbside and ryll (open question 1). Generate a private key. In each of the seven repositories that run prune-reviews (hunkydory, kerbside, ryll, actions, development, sfui, kerbside-client), create an environment named review-marks with the default-branch policy, exactly as open question 4 defines it. In hunkydory, kerbside and ryll alone, add the key to it as an environment secret REVIEW_MARKS_APP_KEY; never as a repository or organisation secret. Put the App's client id in an organisation variable REVIEW_MARKS_APP_CLIENT_ID; a client id is not a secret. Record the App's id and the environment settings here, and keep the key file out of every repository and scratchpad that outlives the step.
1c high opus none Operator step. On the rulesets of hunkydory, kerbside and ryll, add the App as a bypass actor beside the existing team (actor_type: Integration, the App's id, bypass_mode: always), preserving every other field, using the GET-then-PUT shape under "Template" in docs/audits/merge-queue-config.md. This changes nothing on the day: every live copy still pushes with DEPENDENCIES_TOKEN. It must be in place before phase 2 merges any copy into those three repositories, whose next push is then the App's. Add the App only to the rulesets of the repositories it is installed on. GitHub is expected to refuse an App that is not installed there as a bypass actor, so confirm that on the first one rather than assuming it. A repository that adopts review tracking later gets the bypass when it gets its environment. Record the ruleset JSON's bypass_actors for each repository here.

Definition of done. The inventory and the App's id are written into this section; all seven repositories that run prune-reviews have a review-marks environment that meets open question 4's definition of the default-branch policy, with both API responses recorded here; only hunkydory's, kerbside's and ryll's hold the key; shakenfist-bot is an organisation member and an admin of no repository; and the rulesets of those three name the App beside the team as bypass actors.

Phase 2: prune-reviews mints an App token

Planning effort: medium. The template change has a worked example in its own token comment, and the rollout follows the review-tracking-adoption skill.

Order matters. Until phase 3, the bot is still in the bypass team, so a copy that still uses DEPENDENCIES_TOKEN keeps working. A copy that has merged pushes as the App instead, which only works because step 1c made the App a bypass actor first: no copy merges into hunkydory, kerbside or ryll until 1c is recorded. With both bypasses in place, nothing breaks partway through. Phase 3 must not start until the copies in those three repositories have merged.

Step Effort Model Isolation Brief for sub-agent
2a high opus worktree In this repository, change templates/review-tracking/prune-reviews.yml and its deployed twin .github/workflows/prune-reviews.yml (keep them byte-identical). The template is the post-#254 shape described in Situation: no checkout, a token-free Fetch the prune script step, then Prune and import review marks. Add environment: review-marks to the prune-and-import job; if actionlint and GitHub accept environment: {name: review-marks, deployment: false}, use that so each run does not record a deployment, and otherwise say in the README that every run appears in the Deployments view. Change runs-on as open question 6 settled, and revisit the timeout-minutes reasoning for a vm runner's boot time if it moved. If it did, also rewrite the comment above Fetch the prune script and the README's "Runner isolation" section, which both justify the no-checkout design in static-runner terms. Keep the design and the test that forbids actions/checkout, and say why: it is defence in depth, and a repository can still move the job back to static. Add a step id: app-token between the two, using actions/create-github-app-token pinned to a sha with a # vX.Y.Z comment (a v3 release or later: its action.yml at v3.3.0, read on 2026-10-14, accepts client-id and the permission-<name> inputs and deprecates app-id) with client-id: ${{ vars.REVIEW_MARKS_APP_CLIENT_ID }}, private-key: ${{ secrets.REVIEW_MARKS_APP_KEY }}, and permission-contents: write. Leave owner and repositories unset, which limits the token to the current repository, and say why in a comment: the App is installed on several repositories, and naming the owner would widen every token to all of them. Guard it on the key being present, but note that the secrets context is not available in a step's if:: expose presence through a job-level env: entry (for example HAS_REVIEW_MARKS_KEY: ${{ secrets.REVIEW_MARKS_APP_KEY != '' }}) and test that, and confirm with actionlint that the form you choose is accepted. The script step's PUSH_TOKEN becomes ${{ steps.app-token.outputs.token \|\| github.token }}. The fetch step still has no token, and the push token still goes only to the script. scripts/tests/test_ci_prune_reviews.py's workflow_steps and test_the_workflow_never_checks_out_the_workspace hold the old shape, and will fail. Restate each of their assertions, keeping what it protects rather than loosening it. (1) The exact step-name set gains the token step's name. (2) The PUSH_TOKEN line assertion becomes the new expression. (3) text.count('secrets.') == 1 becomes: secrets.REVIEW_MARKS_APP_KEY appears only in the job-level presence flag and the token step, and neither the fetch step nor the script step contains secrets.. (4) No token or secret reaches Fetch the prune script, as now. Add assertions that the job carries environment: review-marks and that the token step has no owner: or repositories: input. test_the_workflow_steps_land_the_review_state runs the fetch and script steps by name, and must not try to run the token step. Rewrite the token comment above the job's permissions: block, saying that contents: write is there for github.token in the repositories with no pull-request ruleset, not for the App, whose token carries its own permission, (it begins "Repositories whose default branch ruleset requires a pull request") to say why: the App is the ruleset's bypass actor, its key lives in an environment only the default branch can use, a repository with no pull-request ruleset has neither and pushes with github.token, and an App push, like the PAT push before it, retriggers the workflow once into a run that finds nothing to do. Remember the &&/\|\| value semantics explained at templates/pin-indirect-dependencies/pin-indirect-dependencies.yml:154-159. Leave the script's git config user.name 'shakenfist-bot' alone: the commit identity is not the push identity, and changing it is outside this plan. Update templates/review-tracking/README.md: replace the "Optionally, a DEPENDENCIES_TOKEN secret" bullet and its "That fallback is silent" paragraph with the environment, its branch policy, the secret and the variable, in the order an operator must create them (open question 4), keeping the warning about a silent fallback in its new form. Also say that every repository running the workflow gets the environment, with its policy, before its first run, whether or not it holds the key, since one GitHub creates on first reference has no policy; and that the key is only ever an environment secret, never a repository or organisation one (open question 4); change the Token column of its projects table; and fix the top of the file and the workflow's header comment where they say the commit lands "as shakenfist-bot", so that they say who pushes. Add the environment setup, as an operator step before the first prune-reviews run, to .claude/skills/review-tracking-adoption/SKILL.md, which today says nothing about tokens, with the same two warnings. Run actionlint and pre-commit run --all-files.
2b medium sonnet worktree Roll the 2a template out with the review-tracking-adoption skill, one pull request per repository, in this order: hunkydory, ryll, kerbside (the three whose pushes need the bypass), then the four without a pull-request ruleset (actions, sfui, kerbside-client, and development, whose own copy 2a already changed). Before each pull request merges, confirm its repository's review-marks environment exists with the default-branch policy (phase 1 step 1b created all seven), so the first run never creates an unprotected one. If shakenfist, instar or client-python-k3s has adopted review tracking by then, it joins the first group and phase 1 step 1b is repeated for it. Each pull request copies the template byte for byte and changes nothing else. After each merge, confirm the next prune-reviews run either pushed as the App or found nothing to do, and record the run URL here. On the first merge, hunkydory's, also confirm from the run log that the app-token step ran rather than being skipped, before the ryll and kerbside pull requests are opened. A presence guard that always evaluates false falls back to github.token without saying so, and actionlint cannot show that. The commit's author and committer do not show this, because the script sets them; in the three repositories with a ruleset, the rule-suite history (repos/shakenfist/<r>/rulesets/rule-suites?rule_suite_result=bypass) shows the push's actor_name, and in the four without one the run's own log does. A run that fails to mint a token in a repository that has the environment means 1b's setup is wrong there: stop and report rather than adding a fallback.

Definition of done. Every live prune-reviews.yml is byte-identical to the template (compare gh api content against it; a comment may still mention the PAT by name, so grep for secrets.DEPENDENCIES_TOKEN, not the bare name, if grepping at all); hunkydory, kerbside and ryll each have a prune-reviews push to its default branch whose rule-suite actor_name is the App.

Phase 3: move the bypass and enforce it

Planning effort: high. Steps 3a and 3b change who can push to six default branches, and a mistake either locks out review marks or leaves the old path open without anything saying so.

Step Effort Model Isolation Brief for sub-agent
3a high opus none Operator step. The App has been a bypass actor since step 1c, and phase 2's definition of done shows it landing. Remove shakenfist-bot from sf-can-skip-merge-queue, which takes its bypass away on all six. Its role was already changed in 1b; re-record repos/shakenfist/<r>/collaborators/shakenfist-bot/permission for all ten repositories to confirm nothing restored it. Prove both halves. First, a push to develop in hunkydory is rejected by the ruleset, once authenticated with DEPENDENCIES_TOKEN and once with RENOVATE_TOKEN, from a throwaway workflow on a branch, deleted afterwards. Each probe pushes an empty commit (git commit --allow-empty) on the current tip of develop, so an accepted push changes no file. If one is accepted, the bypass is still there: stop, record it from the rule-suite history, open a revert pull request at once, and do not go on to the second half until it is understood. A bypass belongs to an identity, no ruleset names the admin role or OrganizationAdmin, and 1b left the bot an admin nowhere, so one repository proves it for all six. Second, the next prune-reviews run in hunkydory, kerbside and ryll lands or finds nothing to do. Record the ruleset JSON's bypass_actors for each repository here.
3b medium sonnet none In this repository, amend docs/plans/PLAN-workflow-ref-trust.md: open question 1's default becomes the App from this plan rather than a team of one, with a sentence saying the residual it described is closed; step 2a's brief installs the App on actions, adds the key to the review-marks environment that phase 1 step 1b already created in actions (as an environment secret, never a repository one), and names the App as the only bypass actor; the "documented set" that step 2b's criterion compares bypass_actors against is that App. Step 2a's question of whether the audit token sees bypass_actors cites this plan's step 1a record (and 1b's, if it re-ran the probe) rather than measuring again. Remove the #211 entry from that plan's Future work and its related-issues list, citing this plan. Touch nothing else in that plan.
3c high opus worktree Add a criterion bypass-actor-identity in scripts/audit/checks/github_config.py, following DeleteBranchOnMerge for the API-call shape, with id bypass-actor-identity, its spec and its issue_title (Bypass actor identity). For each ruleset that applies to the default branch and carries a pull_request rule, it judges every bypass actor by its actor_type, and reports each one that fails. Team fails. RepositoryRole passes only for the admin role (confirm its actor_id against a real ruleset GET rather than assuming it) and fails for any other. OrganizationAdmin passes. Integration passes only if it is the review-marks App (a module constant holding its id). DeployKey fails: a deploy key with write access is an automation credential, like the PATs this plan removes. Any other actor_type is an error, never a pass. The admin-role and OrganizationAdmin passes rest on 1b having left no automation identity as an owner or an admin, and in particular as an admin of any repository the App is installed on, where it could widen the environment's policy. The check cannot see that, so the spec puts it in its "Required, but confirmed by a reviewer" half. Separately, in every repository whether or not it has a pull-request ruleset, the check reads repos/{org}/{repo}/environments/review-marks. Where the environment exists, it fails unless the policy is exactly open question 4's default-branch policy: protected_branches false, custom_branch_policies true, and one policy, type: branch, named for the default branch with no glob characters. It also fails if REVIEW_MARKS_APP_KEY appears in repos/{org}/{repo}/actions/secrets (names only), since a repository-level copy is readable from any branch. If AUDIT_TOKEN cannot list secrets, that half moves to the spec's reviewer-confirmed half, beside the organisation-level check, which the audit cannot make either way. A ruleset response without bypass_actors is an error, never a pass, as in PLAN-workflow-ref-trust.md's phase 2. A repository with no pull-request ruleset and no review-marks environment reports not_applicable with that reason. If step 1a (or 1b, after the demotion) recorded that AUDIT_TOKEN cannot see bypass_actors, do not widen the token. Instead, the check measures only the environment and secret half above. The spec moves the bypass list to its reviewer-confirmed half, as PLAN-workflow-ref-trust.md step 2b does, and the actor-type tests below become tests of the spec's table rather than of code. Register it in CHECKS in scripts/audit/registry.py, write docs/audits/bypass-actor-identity.md following docs/audits/README.md, add its row there, add its lines to FROZEN_METADATA and FROZEN_ISSUE_TITLES in scripts/tests/test_metadata.py, and add pass, fail and not-applicable tests to scripts/tests/test_github_config.py using CheckTestCase and a stubbed GitHub client, including a team bypass, a non-admin repository role, an OrganizationAdmin actor (a pass), a DeployKey actor, an unrecognised actor_type (an error), a missing bypass_actors key, an unknown App, an environment with no branch policy in a repository with no pull-request ruleset, an environment whose policies add a tag policy beside the default branch, one whose branch policy is a wildcard, one with protected_branches: true (each a fail), a repository-level REVIEW_MARKS_APP_KEY, and a repository with neither a pull-request ruleset nor the environment (not_applicable). Before writing the team rule, decide with the operator what sfconductor's bypass becomes (open question 3). If it stays a team, the spec says the check is expected to fail there and why, rather than exempting it in code. But then 3c's first weekly run files six issues that nothing in this plan can close. So 3c does not land with the team rule until open question 3 has an outcome. Until then, land the criterion without the team rule, reporting team bypass actors in the spec's reviewer-confirmed half, and say so in the commit message. State in the commit message how many repositories it newly fails, including this one, against this prediction. If open question 3 leaves sf-can-skip-merge-queue in place for mikalstill and sfconductor, all six repositories in Situation fail on the team rule. This repository, actions (until PLAN-workflow-ref-trust.md step 2a), sfui and kerbside-client carry only the "Protect default branch history" ruleset, with no pull_request rule. They pass on the environment half, because 1b gave each one its policy. So six newly fail, none of them this one. On the fallback shape, the team rule is not measured, and if 1b and 2b did their job nothing newly fails.

Definition of done. shakenfist-bot is not a member of sf-can-skip-merge-queue, and its role is what open question 5 settled; a push to a protected default branch with either bot PAT is rejected; bypass-actor-identity runs in the weekly audit and its results match what step 3a recorded; #211 is closed from the pull request that lands 3c, with a link to this plan.

Phase 4: push audit

Planning effort: medium. The runbook and the ranges are fixed; the work is running it.

Run PUSH-AUDIT.md once per landing recorded in this repository's Merged cells, with AUDIT_RANGE set as its "How to use this runbook" section says. The rollout pull requests in step 2b land in other repositories, and each is audited there as part of landing it; this phase cites those audits rather than re-running them. Phases 1 and 3a land no code anywhere, so the audit of them is a review of what this plan recorded against what the GitHub API reports on the day the audit runs.

Agent guidance

Execution model

Sub-agent execution model (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/subagent-execution-model.md):

All implementation work is done by sub-agents, never in the management session. The management session is reserved for planning, review, and decision-making. This keeps the management context lean and avoids drowning it in implementation diffs.

The workflow is:

  1. Plan at high effort in the management session.
  2. Spawn a sub-agent for each implementation step with the brief from the plan, at the recommended effort level and model.
  3. Review the sub-agent's output in the management session. Check the actual files -- the sub-agent's summary describes what it intended, not necessarily what it did.
  4. Fix or retry if the output is wrong. Diagnose whether the brief was insufficient (improve it) or the model was too light (upgrade it), then re-run.
  5. Commit once the management session is satisfied.

This applies to all steps, including high-effort ones. If a sub-agent cannot succeed even with a detailed brief and the right model, that is a signal the brief needs improving, not that the management session should do the implementation itself.

Use isolation: "worktree" for sub-agents when the change is risky or experimental; the worktree is discarded if the output is unsatisfactory. For safe, well-understood changes, sub-agents can work directly in the main tree.

In this plan. Steps 1a, 1b, 1c and 3a are the exception: they change organisation and repository settings with admin:org, so the management session does them with the operator rather than delegating them. They are also the only steps whose effect no test here covers, which is why each has its own proof in its brief.

Planning effort

Planning effort (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/plan-planning-effort.md):

The master plan itself is always created at high effort -- it requires broad codebase understanding, cross-referencing several source files, and judgment calls about scope and sequencing.

Each phase plan states the recommended effort level for planning that phase. Phases that turn on design decisions, cross-component coordination, protocol changes, or subtle correctness questions should be planned at high effort. Phases that are mechanical, or that follow a pattern already established elsewhere in the codebase, can be planned at medium effort.

In this repository. High effort is anything that changes what a criterion means, anything that touches the scheduler in audit-check.py, and anything that reaches audit-manage-issues.py -- those decide what the fleet is held to and what lands in other people's issue trackers. Medium effort covers adding a criterion that follows the shape of an existing one, a documentation sweep, or a template change with a worked example already in the tree.

Step-level guidance

Sub-agent step guidance (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/subagent-step-guidance.md):

Each phase plan includes a table like this:

Step Effort Model Isolation Brief for sub-agent
1a medium sonnet none One-sentence summary of what to do and which files to touch
1b high opus worktree Why this needs high effort: requires understanding X to do Y

Effort levels, from cheapest to most thorough:

  • low -- Purely mechanical changes: rename, reformat, add a log line, regenerate generated code. The brief is a complete instruction.
  • medium -- The plan provides enough context to follow a clear brief. The sub-agent may read a few files, but the approach is already decided.
  • high -- Requires reading several files, making judgment calls, or understanding non-obvious invariants. The sub-agent needs to think about edge cases.
  • xhigh -- The setting for hard coding and agentic steps: long-horizon changes, or steps where the sub-agent must both research and implement.
  • max -- Correctness matters more than cost. Expect diminishing returns and occasional overthinking; reserve it for steps where a wrong answer would be expensive to detect.

Brief for sub-agent: this is the key field. Write it as if briefing a colleague who has never seen the codebase. Include what to change, which files to touch, what patterns to follow, and any non-obvious constraints.

A good brief front-loads the research the planner already did, so the implementing agent does not repeat it. Instead of "add storage functions for the new object", name the functions to add, the file they belong in, the existing equivalent to mirror (with line numbers), and any registration the change also needs.

The better the brief, the lower the effort level needed and the lighter the model that can succeed.

In this repository. A worked brief: instead of "add a check that plans are indexed", write "add a PlanIndex class to scripts/audit/checks/plans.py declaring the id plan-index, its spec and its issue_title as class attributes and implementing run(repo), register the instance in CHECKS in scripts/audit/registry.py beside the other plan checks, write docs/audits/plan-index.md following the structure in docs/audits/README.md and linking to compliance.md#plan-index, add the file to docs/audits/README.md, add its lines to FROZEN_METADATA and FROZEN_ISSUE_TITLES in scripts/tests/test_metadata.py, and add tests to scripts/tests/test_plans.py covering pass, fail and not-applicable." A brief that names four of the criterion's five files is the characteristic defect here, and the frozen tables are the one most often left out.

A check that does not apply reports not_applicable with a reason rather than being omitted, because an omitted check renders as unknown.

Model choice

Sub-agent model roster (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/subagent-model-roster.md):

The planner recommends which model is best suited to each step. This is a judgment call, not a rigid rule -- the right model depends on what the step requires, not on whether it is "planning" or "implementation". The models available to sub-agents are:

  • fable -- The most capable model available, for the hardest reasoning and the longest-horizon work: multi-step changes a single sub-agent must carry end to end, or steps whose correctness depends on holding a whole subsystem in mind at once. It costs materially more than opus, so reserve it for steps that have already defeated opus or are expected to.
  • opus -- The default for steps needing deep reasoning, architectural understanding, subtle correctness judgment (locking, state machines, migrations), or intricate implementation that would be costly to debug if it were wrong.
  • sonnet -- A good default for well-briefed implementation work. Faster and cheaper than opus, and effective when the plan front-loads the research and the brief leaves no broad judgment calls to make.
  • haiku -- Suitable for purely mechanical tasks: search-and-replace, regenerating generated code, adding log lines, running commands. The brief must be a near-complete instruction.

Model choice interacts with effort level and brief quality. A detailed brief compensates for a lighter model -- sonnet at medium effort with a thorough brief often matches opus at medium effort with a vague brief. The planner's job is to write briefs good enough that the recommended model can succeed.

The model also determines the context window: fable, opus and sonnet have 1M tokens, haiku has 200K. A step that must hold many files in context at once may need one of the larger-context models for that reason alone, even when the reasoning itself is straightforward.

When in doubt, skew to the more capable model. Saving money only matters if the outcome is still acceptable. A failed or low-quality implementation wastes more time -- and therefore more money -- than the heavier model would have cost. Recommend a lighter model only when you are confident the brief is detailed enough for it to succeed.

In this repository. The project-specific checks referred to above are:

  • pre-commit run --all-files passes. It runs actionlint, shellcheck, flake8, skillsaw and all five test suites, and ci.yml runs the same command on every pull request.
  • python3 scripts/audit-check.py --repo-path . --repo-name development still reports what it reported before the change, or the plan says why the verdict moved.
  • If the change touches issue filing, it was exercised with --dry-run only.

Management session review checklist

Management session review checklist (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/plan-review-checklist.md):

After a sub-agent completes, the management session verifies:

  • The files that were supposed to change actually changed -- read them, do not trust the summary.
  • No unrelated files were modified.
  • The changes match the intent of the brief: not merely syntactically correct, but semantically right.
  • The project's own pre-merge checks pass, including any generated code that has to be regenerated and committed (see the project-specific checks below).
  • The commit message follows project conventions, including the Co-Authored-By line recording model, context window, and effort level.

Administration and logistics

Success criteria

We will know when this plan has been successfully implemented because the following statements will be true:

  • pre-commit run --all-files passes.
  • scripts/audit-check.py run against this repository reports no new failures, or the plan states which verdicts moved and why.
  • Any new or changed criterion has all five of its files in step: the Check subclass in scripts/audit/checks/<family>.py, its registration in CHECKS in scripts/audit/registry.py, the specification under docs/audits/, its row in docs/audits/README.md, and its frozen lines in FROZEN_METADATA and FROZEN_ISSUE_TITLES in scripts/tests/test_metadata.py -- plus a FROZEN_COLUMN_NAMES line where a spec page carries more than one check. AUDIT_METADATA, ISSUE_TITLES and COLUMN_NAMES are derived from the registry rather than tables anybody edits; the issue title is the fleet-wide idempotency key, so renaming one orphans every open issue for that check across the fleet.
  • No consistency-audit marker block has been added to a criterion specification by hand, and the compliance tables in docs/audits/compliance.md have not been hand-edited.
  • Any entry added to REPO_OVERRIDES carries a stated reason.
  • Anything under templates/ is judged as the code it will become in ten other repositories: placeholders consistent, no reference to paths that only exist here, and the README beside it saying what to substitute.
  • Python is wrapped at 120 characters, single quotes for strings and double quotes for docstrings, and no script has grown a dependency outside the standard library.
  • Documentation in docs/ describes any user-visible change. AGENTS.md changes only if a convention changed; ARCHITECTURE.md only if the shape of the system changed; README.md only if the pitch, the install story or the documentation links changed.

Specific to this plan:

  • No user or team bypass actor on any default-branch ruleset belongs to an automation identity, other than sfconductor pending open question 3. The only other automation bypass is the review-marks App, and bypass-actor-identity reports compliant everywhere except where the plan says why not.
  • A push to a protected default branch authenticated with DEPENDENCIES_TOKEN or RENOVATE_TOKEN is rejected.
  • shakenfist-bot is not an organisation owner and holds no admin role on any repository. No exception is possible here: open question 5 makes it a gate on the key, not a recorded blocker.
  • The App's key exists only as an environment secret, in environments with the default-branch policy (open question 4), and every review-marks environment has that policy, key or no key.
  • prune-and-import runs where open question 6 settled, and the plan says what residual remains if that is still the static pool.
  • prune-reviews keeps landing review marks in every adopted repository.

  • 211 is closed, and PLAN-workflow-ref-trust.md step 2a names

    the App rather than a team of one.

Documentation index maintenance

When creating a new master plan from this template, add one row to the table in docs/plans/index.md: the date the plan was written, a link to it, a one-line intent, and its status from the vocabulary above. Rows run oldest first. One row per master plan, never one per phase -- the phases are tracked in the plan's own Execution table, and duplicating them in the index is how the two drift apart.

There is no phase-arithmetic column and no order.yml here; both belong to repositories whose documentation is published through a generated navigation. The plan-index criterion checks the columns this index actually has.

The index row carries the whole-plan status, so it only reaches Complete once every phase has been completed, abandoned or superseded. Update it as the plan progresses, not only at the end.

Plan close-out sections (shared block; do not edit -- the canonical copy lives in shakenfist/development at templates/shared-blocks/plan-closeout-sections.md):

Future work

We should list obvious extensions, known issues, unrelated bugs we encountered, and anything else we should one day do but have chosen to defer to here, so that we do not forget them.

...

Bugs fixed during this work

This section should list any bugs we encounter during development that we fixed. You should also scan the project's issue tracker, where one exists, for directly related issues that we should either resolve as part of this master plan or at least be aware of while planning it.

...

Back brief

Before executing any step of this plan, please back brief the operator as to your understanding of the plan and how the work you intend to do aligns with that plan.

In this repository. Future work, as of writing:

  • Narrow DEPENDENCIES_TOKEN and RENOVATE_TOKEN. After this plan neither can bypass a ruleset, but both still push branches and open pull requests in every repository the bot can write to, and DEPENDENCIES_TOKEN is in scope of pull-request jobs. A fine-grained token per use, or a second App for branch pushes, is the next step. Fine-grained tokens owned by a user expire within a year, so that is a rotation commitment, not a one-off change.
  • sfconductor merges pull requests past the merge queue and required checks in four repositories (open question 3). Phase 1 records where its credential lives; whatever follows from that is its own issue.
  • If step 1a finds AUDIT_TOKEN is the bot's, 1b's demotion may stop it listing repository secrets or reading bypass_actors, and 3c then moves those halves to its reviewer-confirmed half. The way back to measuring them is a separate read-only identity for the audit, for example a second App with secrets: read and administration: read, rather than making the bot an admin again.
  • PLAN-renovate-cadence.md (registered 2026-10-03, not started) replaces the seventeen copies of renovate.yml with one central runner in this repository, and its open question 1 asks where RENOVATE_TOKEN lives and what it can reach. Once that lands, RENOVATE_TOKEN is in scope of one workflow rather than seventeen, which makes narrowing it (the first item above) a one-repository change. Neither plan waits on the other: this one removes the bot's bypass whichever repositories Renovate runs from.

Related issues, as of writing:

  • 211 is the issue this plan closes.

  • 153, through PLAN-workflow-ref-trust.md, whose step 2a this

    plan unblocks.

  • 88 (standardise renovate groupings) and #58 (Dependency

    Dashboard) are about the same Renovate runs, but neither changes which identity they run as.

📝 Report an issue with this page