Renovate cadence¶
Prompt¶
Before responding to questions or discussion points in this document, explore this repository thoroughly. Read the relevant files and ground your answers in what they actually say. Do not speculate about the repository when you could read it instead. Flag any uncertainty explicitly rather than guessing.
There is no application code here. The artifacts are the audit
specifications in docs/audits/, the tooling in scripts/ that
measures them, the templates in templates/ that the rest of the
fleet copies, and the workflows that run all of it every week
against every Shaken Fist repository.
Consult AGENTS.md for the conventions and the invariants that
are not visible in the code, and ARCHITECTURE.md for the shape
of the system. docs/consistency-audits.md is the reference for
what a weekly run does, how to add a criterion, how to bring a
repository into scope, and how to test a change before it reaches
the fleet -- read it before changing anything under scripts/ or
docs/audits/. docs/code-review-tracking.md covers the review
tooling, and PUSH-AUDIT.md is the pre-push review runbook that
every plan's final phase runs.
Two things make planning here different from planning in a repository that holds a product, and both should shape any plan written from this template:
- The blast radius is other people's repositories. The weekly
workflow files and closes GitHub issues fleet-wide. A change
that is merely wrong does not produce a red build; it produces
issues in ten repositories, or silently closes ones that should
have stayed open. Always pass
--dry-runwhen runningaudit-manage-issues.pyby hand. - This repository is in its own audit matrix. A standard we exempt ourselves from is a standard we stop noticing the cost of. A change to a criterion is a change we are measured against at the next weekly run, so a plan should say how many repositories -- including this one -- it newly fails.
Plan file conventions (shared block; do not edit -- the canonical
copy lives in shakenfist/development at
templates/shared-blocks/plan-file-conventions.md):
- All planning documents live in
docs/plans/. - Detailed planning gets one plan file per phase. Phase files are
named for their master plan, sit in the same directory as it,
and append
-phase-NN-descriptivebefore the.mdextension. - The master plan tracks its phases in a table under its Execution
section.
Mergedis last, and the push audit is the last row, for the reasons given inplan-push-audit-phase:
| Phase | Plan | Status | Merged |
|---|---|---|---|
| 1. Schema migration | PLAN-thing-phase-01-schema.md | Not started | |
| 2. Public API | PLAN-thing-phase-02-api.md | Not started | |
| 3. Push audit | - | Not started |
- One commit per logical change, and at minimum one commit per phase. Unrelated changes are not batched into a single commit. Each commit is self-contained: it builds, passes tests, and has a message explaining what changed and why.
In this repository. Plans here keep their phases as sections
inside the master plan rather than as separate phase files, and
the Execution table's Plan column is dropped accordingly;
docs/plans/index.md says so. The shared convention above is the
fleet default, and a plan large enough to want phase files should
use them rather than argue with the block.
Situation¶
The CI cluster is busy enough that queue time is a velocity cost,
and Renovate looks like a large part of it. Measured on 2026-10-03
with gh, it is a real part but not most of it.
Pull requests. Renovate opened about 227 of the roughly 1,030 pull requests created across the organisation between 2026-09-03 and 2026-10-03. Half of them bumped one dependency, and that dependency is Renovate itself:
| Renovate pull requests, 30 days | Count |
|---|---|
renovatebot/github-action |
111 |
stbenjam/skillsaw pre-commit hook |
21 |
| Other pre-commit hooks | ~6 |
| Other GitHub Actions | 0 |
| Runtime dependencies (pip, cargo, npm, docker, lock file maintenance) | ~85 |
The last three rows split what was first counted as "everything
else, about 95". They were re-counted on 2026-10-10 from the titles
of the window's pull requests by shakenfist-bot, so they are
approximate. The 14 repositories that received a pre-commit hook
update in the window are the ones a monthly CI-tooling group would
reach. The runtime updates fall on 82 distinct repository,
dependency and ISO week triples, and on 62 distinct repository and
dependency pairs.
renovatebot/github-action is released several times a week, and
every repository runs its own copy of
templates/renovate/renovate.yml, so each release becomes about 16
identical pull requests. Each one runs that repository's full CI
and then its merge queue.
The action itself is gone since this was measured: on 2026-10-15 the
template stopped running Renovate as a container and began installing
the npm renovate package, pinned in PINNED_RENOVATE_VERSION. The
measurement above stands as history, but the dependency it names is
now that package, and it churns harder than the action did --- 204
pull requests in the 30 days to 2026-11-11 against the 111 here. Phase
1's survey, item 2, has the figures, and phase 4 re-establishes both
ends of the comparison with one method rather than measuring against
this table.
CI time. These figures are wall-clock run durations, so they
include queueing. They count each Renovate pull request's own runs
plus its merge-queue runs on gh-readonly-queue/... branches. The
runs API returns at most 1,000 runs, which is why the busiest
repositories have shorter windows:
| Repository | Renovate share of CI time | Window from |
|---|---|---|
| occystrap | 21.6% | 2026-09-03 |
| ryll | 19.8% | 2026-09-03 |
| kerbside | 16.9% | 2026-09-19 |
| kerbside-patches | 10.4% | 2026-09-23 |
| shakenfist | 9.1% | 2026-09-25 |
| instar | 8.4% | 2026-09-06 |
The Renovate runs themselves. The template schedules
renovate.yml hourly on the static runner. In the week to
2026-10-03, shakenfist, ryll, instar and occystrap each ran it 34 to
36 times (about five a day rather than 24), each run taking a
median of 0.5 to 1 minute. This is small next to the pull-request
CI, but it is roughly 85 runs a day across the fleet, all on one
runner label.
Why only five of the 24 hourly crons run is not runner contention.
On 2026-10-09 the last 100 runs in ryll and in shakenfist were
all schedule events and all succeeded; none was cancelled, so no
run was displaced from the renovate concurrency group while it
waited for a runner. Each run started the moment it was created, but
it was created anywhere from :00 to :59 past the hour. GitHub
delays scheduled events under load, documents the start of the hour
as the busiest time, and drops events when the load is high enough.
The template's cron: '0 * * * *' sits exactly there. The median gap
between runs was 4 hours 47 minutes and the longest 9 hours 35
minutes, and that gap is the floor on how soon Renovate can open a
security pull request.
"Copied verbatim" is not what the fleet holds.
templates/renovate/README.md says renovate.yml "is copied
verbatim". Read through the GitHub API on 2026-10-03, the 17 live
copies come in 14 different versions, and they pin three different
action versions (11 at v46.3.6, 2 at v46.3.5, 4 at v46.3.4).
renovate.json is a per-repository adaptation by design, so the
fleet-wide policy (schedule, minimumReleaseAge, the no-automerge
rule, the pre-commit manager) is restated in every copy. Changing
that policy is therefore a sweep of about 17 pull requests.
Security updates already flow. Renovate's token can read
Dependabot alerts: at least ten [SECURITY] pull requests were
opened in 2026 (for example kerbside's cryptography bump on
2026-08-05), and shakenfist has 30 Dependabot alerts with none
open. Renovate's documented defaults for vulnerabilityAlerts drop
both the schedule and minimumReleaseAge, so a security fix should
not wait for either. This plan relies on that, and phase 1 checks
it against the Renovate version the fleet actually runs, because
tightening the schedule is only safe if it holds.
A plan already in flight raises the stakes.
PLAN-workflow-ref-trust.md (development#209, merged 2026-10-03)
phase 3 will pin every third-party action to a commit sha, with
Renovate's helpers:pinGitHubActionDigests keeping the pins
current. The fleet has 540 third-party uses: references. Without
grouping, every upstream action release becomes a digest pull
request in each repository that uses it, and the churn this plan
removes comes straight back. That plan's phase 3 also edits
renovate.json in every repository, which is the same sweep this
plan makes.
What the audit holds the fleet to today
(scripts/audit/checks/packaging.py, Renovate.run): both
.github/workflows/renovate.yml and renovate.json must exist, and
renovate_manages_pre_commit must find the pre-commit manager
enabled in one of three local forms. That function reads only the
local file, so it does not recognise a manager enabled through a
shared preset.
Mission and problem statement¶
Cut the CI load Renovate generates by about half, without making
security fixes slower and without losing the ability to tell which
update broke a build. The measure is the number of pull requests
Renovate opens across the organisation in 30 days, because each one
costs a full CI run plus a merge-queue run: 227 in the Situation's
window, and the target is 100 or fewer. The target is derived from
the post-change shape rather than chosen, and so is unaffected by
the baseline having moved (phase 1's survey, item 2, measures it at
roughly twice the Situation's figure): about 4 bumps of the npm
renovate pin in renovate-fleet.yml (weekly, see phase 2), about
14 monthly CI-tooling pull requests, and about 82 runtime pull
requests, which stay one per pull request and which a weekly window
barely folds. Phase 4 measures the before and the after with one
method, because the Situation's number predates both the move to npm
and the drift audit. Runtime dependencies are therefore most of what
remains, and the lever that would cut further is grouping them, which
this plan declines for the bisection reason below. The Renovate share
of CI time is recorded alongside it, but it is not the target,
because the runs API's 1,000-run cap gives each repository a
different window. Three changes do this:
- Run one Renovate instead of 23. A single scheduled
workflow in this repository runs Renovate across the
organisation. The per-repository
renovate.ymlcopies go away, and with them the roughly 85 daily Renovate runs onstaticand the pull requests that bump each copy's own Renovate pin: 204 in the 30 days to 2026-11-11, where the Situation's window saw 111 for the container action the copies ran then. The copy count was seventeen when this plan was written; phase 2's survey, item 3, counted 23 on 2026-11-12. - Put the fleet policy in one shared preset, so that each
repository's
renovate.jsonholds only what is particular to it (lockstep groups, Python constraints, range strategy). The preset sets tiered schedules: security at any time, ordinary updates weekly, majors and CI tooling monthly. - Group by risk. GitHub Actions and pre-commit hook updates (CI tooling: low risk, not shipped, and the failing job names the culprit) go into one monthly pull request per repository. Runtime dependencies and majors stay one per pull request, because that is where being able to bisect a failure matters.
"Without making security fixes slower" holds for fixes that come
with an advisory, because only vulnerabilityAlerts takes the
immediate path, and it is fed by Dependabot alerts. A fixed release
of a GitHub Action or a pre-commit hook that carries no advisory, or
one in an ecosystem Dependabot does not alert on, is an ordinary
update to Renovate and waits for the monthly CI-tooling window: up
to a month, against up to about ten hours today. Step 1a measured
which of the fleet's ecosystems Dependabot covers, and the gap is
wider than this paragraph assumed. Covered: pip, cargo, npm and
github-actions. Not covered at all: docker, which is absent from
GitHub's supported-ecosystem table, and pre-commit, which is not
an ecosystem --- a hook is a git repository rather than a package in
an advisory database. The github-actions coverage also holds only for
actions referenced by semantic version, not by commit sha, so
PLAN-workflow-ref-trust.md's digest pinning moves that ecosystem
into the gap as well; that plan should know, and phase 1's Future
work says so. Those three are the accepted worst case, and the
override is manual: the dependency dashboard lists updates awaiting
their schedule, and ticking one opens its pull request on the next
run.
Out of scope:
- Automerge. Fleet policy is that every update is reviewed, and this plan keeps it.
- The pinning policy itself, which
PLAN-workflow-ref-trust.mdowns. This plan provides the grouping that policy needs, and its phase 3 must land before that plan's step 3b. - Turning other copied workflow templates into shared workflows. Renovate is a special case: a reusable workflow would still run once per repository, while a central runner runs once in total. The general question is recorded under Future work.
Open questions¶
- Where does
RENOVATE_TOKENlive, and what can it reach? The central runner needs a token that can push branches and open pull requests in every repository Renovate manages, and read Dependabot alerts in each one. Listing organisation secrets needsadmin:org, which the planning session did not have. Default: assume it is theshakenfist-botpersonal access token with organisation-wide access. Phase 2's cutover confirms the read path with aRENOVATE_DRY_RUN=lookuprun and the write path separately, because a lookup dry run creates no branch and opens no pull request, so an under-scoped token passes it. Both happen before any per-repository workflow is disabled. Mostly answered by cutover step 1 (2026-11-14, with the secret scopes read on 2026-11-15): the token pushes to all 23 managed repositories, and it lives in an organisation secret withselectedvisibility plus an environment copy on this repository, with no repository-level copy anywhere in the 23. Phase 2's Record holds the detail, including the one managed repository the organisation list is missing. Whether the runner reads Dependabot alerts is still unconfirmed, because neither dry run logs it at INFO, and that half is carried to phase 3. - Which timezone do schedules use? Renovate reads schedules in
UTC unless
timezoneis set, and today'safter 9pmis UTC. Answered on 2026-11-11: the operator choseAustralia/Sydney, which the preset sets explicitly (phase 1, decision 5). - Cadence. Default: ordinary updates
on saturday, majors and the CI-tooling groupon the first day of the month, andvulnerabilityAlertsat any time. These are easy to change later because they live in one preset. Each window is a whole day on purpose: Renovate acts only when a run lands inside the window, and the Situation measured gaps of up to 9 hours 35 minutes between scheduled runs, so a six-hour window could see no run at all and slip a week's updates silently to the next. - Where does the central runner live? Default: this
repository, beside
consistency-audit.yml. This repository already holds the workflows that act on the whole fleet, whileactionsholds code that runs inside other repositories' jobs. - Is OSV worth adding as a second source of security alerts?
Dependabot alerts already produce security pull requests (see
Situation).
osvVulnerabilityAlertswould add the OSV database as a backstop. Answered by step 1a: no. Phase 1 did find two ecosystems Dependabot does not cover, docker and pre-commit, but OSV cannot reach either: Renovate'sdatasourceToOsvEcosystemmap has no entry fordockeror for thegithub-tagsdatasource that the pre-commit manager resolves hooks to, so the option would duplicate the pip, cargo and npm coverage Dependabot already gives and add nothing. It is also still marked experimental.
Execution¶
| Phase | Status | Merged |
|---|---|---|
| 1. Shared preset and its validation | Complete | 47086db |
| 2. Central runner and cutover | In progress | |
| 3. Fleet sweep and audit change | Not started | |
| 4. Re-measure | Not started | |
| 5. Push audit | Not started |
The order matters in two places:
- Phase 2 cuts over before phase 3 deletes anything. Two
Renovate instances working on the same repository race on branch
creation; the comment in
templates/renovate/renovate.ymlrecords exactly that failure. Phase 2 therefore disables each per-repository workflow withgh workflow disable, which takes effect at once, needs no pull request and is undone bygh workflow enable, before the central runner is turned on. Phase 3 then deletes the disabled files and moves eachrenovate.jsononto the preset in a single pull request per repository, so the sweep costs one CI run per repository rather than two. PLAN-workflow-ref-trust.md's digest pinning lands in the preset before this plan's phase 3, not after it. This paragraph said the opposite until 2026-11-13, reasoning that a line added to the preset before the sweep reaches one repository while the fleet's digest pull requests would arrive one per release per repository. The second half does not follow from the first: that churn is prevented by the monthly CI-tooling group, which lives in this same preset, rather than by the ordering. What the ordering decides is where the change gets reviewed. Before the sweep it changes one repository, and this plan's own sweep pull request then carries the policy into each repository reviewably, alongside everything else that repository receives. The pin diff is not in that pull request: it arrives afterwards, as one grouped Renovate pull request per repository on the next monthly window, which is also the volume to expect from the sweep -- roughly one grouped pull request per repository, not one per action per release. After the sweep the same one-line edit changes every repository's behaviour with no pull request in any of them -- which is what the comment aboveRENOVATE_FLEET_PRESETinscripts/audit/checks/packaging.pywarns about. The matching reasoning is in that plan's phase 3. (The operator'splan-blockerstool carries a matching note, but it lives outside this repository.)
Only phase 1 carries a step table. Phases 2 to 4 get theirs when
each is planned, from the scope written here, because what phase 1
finds (the limits, the token, the schedule semantics) changes their
briefs. Phase 3's audit-change brief in particular names the
packaging test module, the converged-templates test that the
deletions break, the spec rewrite, and --dry-run for any
issue-filing run.
Plan status vocabulary (shared block; do not edit -- the canonical
copy lives in shakenfist/development at
templates/shared-blocks/plan-status-vocabulary.md):
A status cell -- in the master plan's own Execution phase table, and
in the row docs/plans/index.md carries for the plan -- holds
exactly one of these terms and nothing else:
Proposed-- written down as a concept, not yet scheduled.Not started-- scheduled, but no work has begun.In progress-- work has begun and has not finished.Blocked-- cannot proceed until something outside the plan changes. Say what, in the plan.Complete-- the work is done.Abandoned-- deliberately dropped without being done.Superseded-- replaced by another plan, which the plan names.
The term is the whole cell. No dates, no phase arithmetic, no parenthetical qualifiers, no summary of what happened: a status is read to decide whether a plan still wants attention, and prose in that column has repeatedly grown until it could no longer be read either by a person scanning the table or by tooling. Detail belongs in the plan file, and a one-line summary belongs in the index's own Intent column.
Matching is case-insensitive, so In Progress is accepted, but the
spelling above is the one to write.
In this repository. The same term is written twice: once in
this plan's own phase table, and once in the row the plan carries
in docs/plans/index.md. The index row is the whole-plan status,
so it only reaches Complete once every phase has been
completed, abandoned or superseded. The plan-index criterion
reads that table, and this repository is inside its own audit
matrix, so a status that drifts out of the vocabulary fails our
own tooling before it fails anybody else's.
Push audit phase (shared block; do not edit -- the canonical
copy lives in shakenfist/development at
templates/shared-blocks/plan-push-audit-phase.md):
- Every master plan ends with a phase that runs the repository's
PUSH-AUDIT.mdover the whole plan's work. It is the last row of the Execution table and it is not optional. The rule binds every plan that carries the phase, which is decidable from the plan file alone: a plan that is alreadyComplete,AbandonedorSupersededand does not carry the phase is not reopened to acquire one, and a plan that has the phase runs it even if it reachesCompletebefore the phase does. - That phase audits the accumulated diff of every phase in the plan against the default branch, not the diff of the last phase alone. Auditing one phase at a time would miss what the phases did to each other -- the duplicated helper that only exists once phases three and six have both landed, the doc page that phase two made wrong and phase five never revisited.
- Once the plan's phases have merged, a diff against the default
branch is empty and would read as a clean audit. The range is not
reliably derivable after the fact either: unrelated work lands on
the default branch between phases, so anything anchored on "since
the plan file appeared" is far too wide. It has to be recorded. As
each phase lands, what put it on the default branch goes into the
plan: the merge commit of its pull request, whose diff against its
first parent is the whole of what landed, or -- where the phase
landed directly -- every commit of the phase, or its
first..lastrange. A single commit is only ever enough when it is a merge commit. - Where the Execution phases are a table, that record is a
Mergedcolumn, added last so that a row which omits it still reachesStatus; where they are prose sections it is aMerged:line in the phase's own section. TheStatuscolumn keeps its single vocabulary term and nothing else (seeplan-status-vocabulary). A phase that landed in another repository records<repo> <sha> (#pr)and is audited against that repository's default branch, as part of the pull request that lands it; the plan's own push-audit phase cites that audit rather than re-running it. - Phases that landed before the plan started recording them are
reconstructed rather than left blank. Recover what you can from
gh pr list --state mergedandgit rev-list --first-parent, and say in the plan that the range was reconstructed. Do not trust a path-filteredgit logon its own: it lists the commits that touched a path without saying which arrived directly and which arrived inside a pull request, and recording a commit that came in under a merge audits one commit of that pull request rather than the pull request. A reconstructed record may be a summary table in the audit phase's own section rather than a column or a line in the Execution table, which keeps retrospective archaeology out of a table that tracks live status. Where a phase accreted over months of unrelated commits and no range is recoverable, say that instead and name the paths the audit read -- an audit that says what it could not scope is a result; one that silently audits nothing is not. - Findings land as their own pull request against the default branch, and the plan is not complete until they are resolved or explicitly declined in writing. A finding that is declined says why, in the plan, where the next reader will find it.
- Where the audit finds nothing, record that in the plan in one sentence. It is a real result, and a run of them is the evidence for making the phase conditional rather than mandatory.
- A repository with no
PUSH-AUDIT.mdstill carries the phase, and the phase says that the runbook does not exist yet and what was done instead. Silently omitting it is what let the audit go untriggered for as long as it did.
In this repository. PUSH-AUDIT.md exists at the repository
root and is referenced from AGENTS.md, so the final phase runs
it rather than explaining its absence. Note that every diff
command in it is written against main...HEAD: a stale local
main silently widens the audit to unrelated history, so fetch
before starting, or read it as origin/main...HEAD.
Phase landing (shared block; do not edit -- the canonical copy
lives in shakenfist/development at
templates/shared-blocks/plan-phase-landing.md):
A plan's status and a repository's review state both live in files that every branch would otherwise rewrite. Left alone, that turns each of them into a merge-conflict hot spot, and it spends a pull request and a full CI run on a change that is entirely prose. Three rules keep them out of the way.
-
A phase is closed out in the first commit of the next phase, not in a pull request of its own. By the time the next phase branches, the previous one has merged, so its merge commit is known and its
Mergedcell can record the thing the push-audit phase actually needs. This is the only ordering that works: a phase cannot record its own merge commit, and a separate close-out pull request buys that record at the price of a round trip. The close-out sets the finished phase'sStatusandMergedcells and the plan's row indocs/plans/index.md, and it is committed before the next phase's own work, so that the branch never claims the plan is further along than the default branch is. -
The last phase closes itself out. The push-audit phase is the last row of every plan, so no next phase will carry its close-out. Where the audit raises findings, the plan is not complete until they are resolved or declined, and those land as their own pull request after the audit phase has merged -- so that pull request is the carrier, and it can record the audit phase's merge commit, which by then is known. Where the audit finds nothing there is no carrier, and no follow-up pull request is opened for the sake of one cell: the phase sets its own
Status, and the plan's index row, toCompletein its own pull request, and records noMergedcell. It is the only row permitted to omit one. The column exists so that the push-audit phase can reconstruct what to audit; the audit phase is last, so nothing ever reads its own row. -
REVIEWS.mdis not pruned or regenerated in a pull request that changes code or documentation. Editing a reviewed file stales its mark, and adding or removing an in-scope file moves the header count, but neither is the landing pull request's business.pruneregenerates the file whether or not it dropped anything, so theprune-reviewsworkflow heals both on the next push to the default branch. Pruning from a branch is also wrong more often than it is right, though not for the reason it first appears:prunecompares each stamp againstHEAD, which on a branch is the branch tip, so it drops the marks for the files the pull request itself touched while keeping marks the default branch has already pruned. Committing that state merges a review file computed from a stale tree, and can resurrect marksprune-reviewshas already removed. Accumulated staleness is reported by thereview-coverageaudit, which recomputes coverage againstHEADand raises an issue once the backlog is worth a review session.
A review session is the exception, and it is not optional
tidiness: stamp regenerates REVIEWS.md as well as writing the
marks, and the rows, the sidecars and the marks are committed
together (see docs/code-review-tracking.md). Where a repository
requires a pull request to reach its default branch, that is how
a review session lands, so "not in a pull request" is about the
kind of change, not the mechanism.
These rules assume phases land one after another. Where two phase branches are open at once, each closes out only the phase it directly follows.
In this repository. The plan index is docs/plans/index.md.
review-tracking-tests deliberately does not assert the REVIEWS.md
header count, so a phase that adds or removes an in-scope file needs
no regeneration commit; prune-reviews corrects the count on the
next push to main.
Phase 1: shared preset and its validation¶
Planning effort: high. This phase sets the policy every repository will inherit, and it is where the security-schedule assumption either holds or does not.
Scope¶
Add a Renovate preset to this repository that holds the fleet
policy: gitAuthor, assignees, dependencyDashboard,
rollbackPrs, minimumReleaseAge, the pre-commit manager, the
customManagers:githubActionsVersions extend, the schedules from
open question 3, timezone, prHourlyLimit and
prConcurrentLimit (set explicitly, from step 1a), the
no-automerge rule, the two rules that keep node-version and the
pinned Renovate in step, the shellcheck-py/shellcheck-py
versioning rule, and the CI-tooling group (matchManagers:
["github-actions", "pre-commit"], all update types including
digest, monthly).
The preset is renovate-fleet.json at the root of this
repository, and every repository references it as
"extends": ["local>shakenfist/development:renovate-fleet"].
Steps 1b, 1c and 1d, and phase 3's audit change, all work to that
exact path and string. This repository's own github-actions
managerFilePatterns for templates/, and the customManagers
regex entry that gives the built-in preset's match string the same
reach, stay in its local renovate.json, because they apply only
here (docs/audits/renovate.md, "Repositories that ship workflow
templates").
Renovate reads a local> preset from the default branch of the
repository that holds it, so a change to the preset cannot be
tried from a pull request branch: it takes effect for the whole
fleet when it merges. Pre-commit's validator is the only check
before that, and phase 2's lookup dry run is the first live read
of the preset.
Out of scope for this phase: the central runner and the cutover
(phase 2), deleting any per-repository renovate.yml or touching
any other repository (phase 3), and the lockstep groups, which
stay local for the reason phase 3 gives. Also out of scope: the
criterion that would require the shellcheck-py rule rather than
merely permit it, which survey item 7 records as a separate plan.
What the survey found¶
The phase's premises were written on 2026-10-03. Seven changes have landed in this repository since, and four of them make a claim in this section or its step table false. Each is corrected at its source in the same commit as this plan, so a later reader does not have to find this list first.
-
The dependency this plan is mostly about no longer exists.
templates/renovate/renovate.ymlstopped usingrenovatebot/github-actionon 2026-10-15 (7682dec, "Run Renovate from npm, not in Docker"), because the action runs Renovate as a container and the static pool's ephemeral runner pods have no Docker daemon. The template now sets up Node 24, installsrenovatefrom npm with--ignore-scriptsand rebuildsre2, and runs the binary; the pin isPINNED_RENOVATE_VERSIONwith a# renovate:comment above it, read bycustomManagers:githubActionsVersions. Two further commits added conditional Rust (b24d693) andhashin(38092c4) steps for lock-file refresh. Step 1a was written to research an action version and the Renovate version inside it; there is no action and no container, so it now researches the npmrenovaterelease the template pins. The Situation's measurement window closed on 2026-10-03, before any of this, so its table is accurate history and is left alone with a note pointing here. -
The churn nearly doubled, and all of the growth is the one dependency phase 2 deletes. Counted on 2026-11-11 with
gh pr list --search 'created:>=2026-10-12 in:title Update dependency renovate'over the twenty matrix repositories that have the workflow: 204 pull requests in 30 days for the Renovate pin alone, against the 111 the Situation measured for the action. Per repository it runs 7 to 21, and the three that report zero (agent-python,client-python,client-python-k3s) are the ones still running the container action withoutcustomManagers:githubActionsVersions, so their pin is invisible rather than current. Org-wide, pull requests byshakenfist-botwhose title beginsUpdatenumber 417 in the same window, plus 10Lock file maintenance, against the Situation's 227 for all of Renovate. The Mission's baseline is therefore stale by roughly a factor of two. Its target is not:100 or fewerwas derived from the post-change shape (a handful of bumps of one central pin, about 14 monthly CI-tooling pull requests, about 82 runtime ones), not as a fraction of the baseline, so the number stands and only the thing it is measured down from has moved. Phase 4 re-establishes both ends with one method rather than comparing against the Situation's. -
The
shellcheck-pyrule is already fleet policy here.dc2f6ed(#275, 2026-10-18) put it intotemplates/renovate/renovate.jsonand this repository's rootrenovate.json, with thebuildgroup name and the$anchor the step table specifies and a fuller description thankerbside's, and documented it intemplates/renovate/README.md. So the rule moves into the preset from this repository's own template, verbatim, and not fromkerbside;kerbside's surviving local copy and itstox.inigroup are phase 3's business. The scope paragraph that saidkerbsidehad solved it alone, and listed the five dashboards carrying the warning on 2026-10-09, is corrected. -
"Copied verbatim" is now true and enforced, so there is nothing to correct in the README.
d2298b0(2026-10-31) addedrenovate_workflow_drift(scripts/audit/checks/packaging.py), which holds every copy ofrenovate.ymlbyte-identical to the template with eachPINNED_*_VERSIONvalue and eachuses:ref blanked, and95f3492extended the blanking to action refs.templates/renovate/README.mdanddocs/audits/renovate.mdboth describe that rule. This repository's.github/workflows/renovate.ymlis byte-identical to the template today. Step 1d therefore drops the instruction to correct the README's verbatim claim, and the Bugs-fixed entry that raised it records how it was actually fixed. -
A second local-only check will fail this repository, and the step table did not name it.
renovate_reads_version_comments(scripts/audit/checks/packaging.py:173) reads only the localrenovate.jsonand requirescustomManagers:githubActionsVersionsinextends; theRenovatecheck calls it at line 486. It is the same shape of bug asrenovate_manages_pre_commit, which step 1c exists to fix, and it has the same blast radius: if the preset carries the extend and step 1b strips it from the root file,developmentfails its ownrenovatecriterion on the next scheduled run. Decision 1 resolves it, decision 2 shapes the fix.renovate_manages_pre_commitis also at line 46 now, not 47, and therenovate.jsonpath it reads is resolved byrenovate_configat line 702 rather than inline at line 54. -
The root
renovate.jsonhas grown three keys the step table does not account for. Beside what it held in October it now carries acustomManagersregex entry fortemplates/(addedc6446d2), aprBodyNotesrule warning that a Renovate major can raiseengines.nodepast the workflow'snode-version(91721b7), and a rule disabling Node majors for.github/workflows/renovate.ymlandtemplates/renovate/renovate.yml(46e8d3e, after a Node 26 bump broke nine repositories' hourly runs at once). The last two are in the template as well, so they are fleet policy; decision 4 places them. Step 1b's list of what stays and what goes is rewritten accordingly. -
Everything else the phase assumes still holds.
renovate-fleet.jsondoes not exist and nothing in the tree references the preset..pre-commit-config.yamlhas no Renovate hook.renovatebot/pre-commit-hooksexists, its newest tag is44.227.4, and itsrenovate-config-validatorhook islanguage: node,language_version: lts, withadditional_dependencies: [renovate@44.227.4]and exactly thefiles:default the step table quotes --- which matches bothrenovate.jsonfiles and notrenovate-fleet.json.ci.yml'slint-and-testjob runspre-commit run --all-fileson[self-hosted, static], so the hook is exercised in CI and not only on a workstation.*.jsonis in.vscode/review-scope.toml, so the preset joins the review queue;review-tracking-testsdoes not assert theREVIEWS.mdheader count, so no regeneration commit is needed. No criterion compares anyrenovate.jsonagainst the template, so changing the template's copy fails nobody. One thing the survey found that is not in this phase's scope: a criterion requiring the shellcheck-py rule in every repository using the hook is recorded against an unwrittenPLAN-renovate-shellcheck-audit, in the operator'splan-blockersnotes rather than in this repository. If it lands after this phase it must read the preset form too, for the reason decision 1 gives; Future work now says so.
Blast radius. This phase changes no criterion's requirements --- it only widens what satisfies two of them --- so it newly fails no repository. The one repository it can break is this one, and only if step 1b lands without 1c.
Decisions¶
-
The preset carries
customManagers:githubActionsVersions, and the audit is taught to see it there. The alternative is to leave the extend in every localrenovate.json, which costs nothing today and avoids touching a second check. It is rejected because it splits the fleet policy for no reason other than the audit's convenience: the entry is either right for every repository or wrong for every repository, which is the same test the shellcheck rule was admitted under. The audit reading only the local file is the defect, and a plan whose whole purpose is to move policy into a preset should fix it once rather than route around it twice. This is the decision most likely to be argued with, so the counter-argument is in the back brief. -
One helper for both checks, and the preset is trusted by name in neither of them. Factor each check's existing tests on the parsed dict into a helper that takes the config dict ---
renovate_manages_pre_commit's three forms, andrenovate_reads_version_comments's preset tuple --- and have a single new predicate,extends_fleet_preset(config), answer the preset question for both call sites. The per-check helpers must not know the preset's name, so neither can pass a repository by trusting a string; the name is matched once, in one place, and what it vouches for is proven by the test in decision 3. -
The preset is proven by loading the real file, and the proof is mutation-tested. A
local>preset takes effect fleet-wide the moment it merges, so a commit that dropped the pre-commit manager or the version-comment extend fromrenovate-fleet.jsonwould make every repository in the fleet pass a check on a promise the preset no longer keeps. Step 1c therefore loads the realrenovate-fleet.jsonfrom the repository root withjson.loadand asserts each per-check helper accepts it on its own, with no preset special case in play, and the step runsmutation-checkover the new tests so that the guard is shown to fail when the thing it guards is removed. -
The two Node rules go into the preset, and the disable rule names both paths. They are in the template as well as here, so they are fleet policy by the test decision 1 uses. The disable rule's
matchFileNameslists.github/workflows/renovate.ymlandtemplates/renovate/renovate.yml; the second path exists only in this repository and is inert everywhere else, which is the precedent the shellcheck rule already set ("Harmless in a repository that does not use the hook"). Phase 2 addsrenovate-fleet.ymlto the same rule when the central runner arrives, and phase 3's sweep leaves both rules naming a file the fleet no longer has, which is a no-op rather than a finding. -
timezoneisAustralia/Sydney(open question 2, answered by the operator on 2026-11-11), with open question 3's cadence defaults: ordinary updateson saturday, majors and the CI-tooling groupon the first day of the month,vulnerabilityAlertsat any time. The windows are whole days, so the zone decides which 24 hours count as Saturday rather than a boundary of minutes; in a local zone the weekly batch arrives across Saturday instead of straddling Saturday night. -
The validator hook is pinned at
44.227.4, itsfiles:is overridden, and it is passed--no-global. The default pattern skipsrenovate-fleet.jsonwhile still passing, which is the failure mode the hook is being added to prevent, so the override and the deliberate-break proof are both definition-of-done items rather than suggestions. Step 1a found that the flag matters for the same reason: a file given as an argument is validated as a global self-hosted configuration, which accepts a self-hosted-only key in a file that is neither, and the Record has the measurement. The hook pinsrenovate@44.227.4inadditional_dependencies, which is a second Renovate version in the repository, bumped by the pre-commit manager rather than by the workflow's# renovate:comment. Accepted: under the preset's monthly CI-tooling group that is one pull request a month, and the two pins validating and running the same configuration can drift a few releases apart without consequence. -
The documentation is a step of its own. The step table had 1c carrying the audit change, its tests, and rewrites of two documentation pages. Splitting the prose into 1d gives each a commit of its own, which is the convention, and keeps a documentation rewrite from being reviewed as part of a diff whose correctness argument is about Python.
Steps¶
| Step | Effort | Model | Isolation | Brief for sub-agent |
|---|---|---|---|---|
| 1a | high | opus | none | Research only; change no files, report with citations, do not assume. The subject is the npm renovate package, not renovatebot/github-action: survey item 1 records that templates/renovate/renovate.yml stopped using the action on 2026-10-15 and now installs renovate from npm, pinned in PINNED_RENOVATE_VERSION (44.222.2 in the tree today; 44.227.4 is the newest tag of renovatebot/pre-commit-hooks, which tracks the same release train). Answer four questions against that version's documentation and source. (i) What vulnerabilityAlerts does by default with schedule, minimumReleaseAge and grouping --- the plan relies on it dropping all three, and tightening the schedule is only safe if it does. If a security fix would wait for the weekly window or join the CI-tooling group, say exactly which key the preset must override. (ii) Which of the ecosystems the fleet uses (pip, cargo, npm, docker, github-actions, pre-commit) GitHub Dependabot raises alerts for, since vulnerabilityAlerts is fed by them; this answers open question 5, and the github-actions and pre-commit answers are the accepted worst case the Mission's security-latency paragraph states. (iii) The defaults for prHourlyLimit and prConcurrentLimit in that version (recent versions appear to default to 2 and 10, but verify), and whether vulnerabilityAlerts pull requests are exempt from them. With a once-a-week window those limits decide how much of a week's backlog opens in it, so recommend values and say why; step 1b sets both explicitly rather than inheriting. (iv) Whether renovate-config-validator validates a preset file as an ordinary configuration, including one whose own extends names a built-in preset, so that step 1b's hook genuinely checks renovate-fleet.json. Report to the management session; 1b does not start until the operator has seen the answers (see the back brief gate). |
| 1b | medium | sonnet | none | Write renovate-fleet.json at the repository root and move the fleet policy into it, then reduce both renovate.json files to it. The preset holds: $schema; extends with customManagers:githubActionsVersions (decision 1); gitAuthor, assignees, dependencyDashboard, rollbackPrs, minimumReleaseAge; "pre-commit": {"enabled": true} with its existing description; "timezone": "Australia/Sydney"; the schedules from decision 5 and prHourlyLimit: 2 with prConcurrentLimit: 10 from step 1a's Record, which are Renovate's own defaults written down rather than inherited; no vulnerabilityAlerts object and no osvVulnerabilityAlerts, because step 1a established that the defaults already drop schedule, minimumReleaseAge and grouping for a security fix and that OSV cannot reach either ecosystem Dependabot misses; the no-automerge rule; the Renovate-major prBodyNotes rule and the Node-major disable rule, the latter with matchFileNames listing both .github/workflows/renovate.yml and templates/renovate/renovate.yml (decision 4); the shellcheck-py/shellcheck-py versioning rule copied verbatim from this repository's own templates/renovate/renovate.json, description included, not from kerbside (survey item 3) --- keep the build group name and the $ anchor, because Renovate's regex versioning (lib/modules/versioning/regex/index.ts) only appends revision to the comparison array when build also matched, so renaming the group would truncate every tag to three components and hide packaging-only releases, and the anchor is what ignores the v0.11.0.1-1 re-tag that has no matching PyPI release; and the CI-tooling group (matchManagers: ["github-actions", "pre-commit"], every update type including digest, scheduled monthly). Then reduce templates/renovate/renovate.json to $schema, "extends": ["local>shakenfist/development:renovate-fleet"], and the per-repository sections the README describes, and reduce this repository's root renovate.json to $schema, that same extends, the github-actions managerFilePatterns for templates/ with its description, and the customManagers regex entry for templates/ with its description --- those last two apply only here (survey item 6). Everything else either of them holds today is in the preset and goes. Add renovatebot/pre-commit-hooks rev 44.227.4 to .pre-commit-config.yaml with the renovate-config-validator hook, override its files: --- the default (^|/).?renovate(?:rc)?(?:\.json[c5]?)?$ matches both renovate.json files but not renovate-fleet.json, so the preset would be skipped and the hook would still pass --- and pass it args: ['--no-global'], because a file given as an argument is otherwise validated as a global self-hosted configuration, which accepts a self-hosted-only key in a file that is not one (step 1a's Record has the measurement; the flag costs nothing and both modes still catch unknown keys and bad values). Follow the file's own convention of a comment saying why each hook is there. Then prove it: break the preset on purpose (an unknown top-level key is enough), run pre-commit run renovate-config-validator --all-files, confirm it fails and names renovate-fleet.json, paste the output into your report, and restore the file. Do not commit the broken state. This makes development the preset's first live user from the moment phase 1 merges, still run by its own renovate.yml, so its dependency dashboard is the canary for the preset's schedules before the rest of the fleet adopts it. Step 1c must land in the same pull request, or this repository fails the Renovate check on the two local keys that have just been removed. |
| 1c | medium | sonnet | none | Teach two checks in scripts/audit/checks/packaging.py that the fleet preset satisfies them, which it must do before 1b's reduced configs can pass. renovate_manages_pre_commit (line 46) tests three forms on the parsed dict: pre-commit.enabled is True, pre-commit appears in enabledManagers, or an extends entry ends :enablePreCommit. renovate_reads_version_comments (line 173) tests one: an extends entry in RENOVATE_VERSION_COMMENT_PRESETS. Both read the local file only, through renovate_config (line 702). Factor each one's dict tests into a helper that takes the config dict --- so renovate_config_manages_pre_commit(config) and renovate_config_reads_version_comments(config), or whatever reads best beside the existing names --- add one new predicate extends_fleet_preset(config) that looks for local>shakenfist/development:renovate-fleet in extends, and have each public function return helper(config) or extends_fleet_preset(config). The per-check helpers must not know the preset's name (decision 2), so neither can pass a repository by trusting a string. Tests go in scripts/tests/test_packaging.py beside RenovatePreCommitManagerTest (line 265) and in a new class for the version-comment check, following CheckTestCase as the neighbours do: for each check, a passing local form, a failing config, and the preset form; and, per decision 3, a test that loads the real renovate-fleet.json from the repository root with json.load and asserts each per-check helper accepts it with no preset special case in play --- that test is the only thing standing between a future edit of the preset and every repository in the fleet passing on a promise the preset no longer keeps. Then run the mutation-check skill over the new tests and report which mutants were killed; a test that survives the removal of what it guards is not finished. Verify with python3 -m unittest discover -s scripts/tests -t scripts, and with python3 scripts/audit-check.py --repo-path . --repo-name development, which must report renovate compliant on the branch after 1b. Do not touch docs/ --- that is 1d. |
| 1d | medium | sonnet | none | Document the preset. In docs/audits/renovate.md: the "What we check" bullets for renovate.json extending customManagers:githubActionsVersions and enabling the pre-commit manager now have a fourth satisfying form each, the fleet preset, so say that both are satisfied either locally or through local>shakenfist/development:renovate-fleet, and add a short section on the preset --- what it holds, that Renovate reads it from this repository's default branch so it cannot be tried from a branch, that pre-commit's renovate-config-validator is the only check before it merges, and that the audit trusts it by name with a test that loads the real file. Leave the "Repositories that ship workflow templates" section as it stands: its github-actions and customManagers patterns are still local to development (survey item 6). In templates/renovate/README.md: the renovate.json row and the "Copying these files" section now describe a file that extends the preset and adds only what is particular to the repository, and the three-form list under "The pre-commit manager" gains the preset. Keep the shellcheck-py section but say the rule now lives in the preset rather than in each copy. Add one sentence, in whichever page the preset's contents are described on, recording step 1a's exception: a packageRules entry with enabled: false suppresses even a vulnerability fix, which the preset's Node-major disable rule is, so Node majors in the Renovate workflows are the one update type the immediate security path does not reach. State it; do not change the rule. Do not correct the claim that renovate.yml is copied verbatim: survey item 4 records that renovate_workflow_drift has since made it true and that both pages already describe the rule enforcing it. No fact about the preset may be stated differently on the two pages; the definition of done checks that by reading both. |
Record¶
Every entry cites the command or file it came from, and one that could not be read says so. Dates are Australian Eastern Daylight Time (UTC+11).
Security schedule (1a). Ran 2026-11-12. vulnerabilityAlerts
drops all three of schedule, minimumReleaseAge and groupName
by default, so the preset needs no vulnerabilityAlerts object at
all: lib/config/options/index.ts at tag 44.222.2 gives the
option a default of groupName: null, schedule: [],
minimumReleaseAge: null, prCreation: 'immediate' and
prConcurrentLimit: 0, read directly by the management session as
well as by the step. The forcing is not merely a default that a
top-level key could beat: the Dependabot path appends one
packageRules entry per alerted dependency with
force: {...config.vulnerabilityAlerts}, and mergeChildConfig
returns { ...config, ...config.force }, so the forced keys beat
the top level and every earlier rule. Renovate's documentation says
the same of schedule and prHourlyLimit and adds that it cannot
be overridden. One real exception, recorded because it applies to
this preset: a packageRules entry with enabled: false does still
suppress a vulnerability fix, and the preset has one --- the
Node-major disable rule. It is narrow (two file names, node,
major), so the exposure is Node majors in the Renovate workflows
alone, and 1d states it rather than the plan changing to avoid it.
Dependabot coverage (1a). Covered for the fleet: pip, cargo,
npm, github-actions. Not covered: docker, absent from GitHub's
supported-ecosystem table, and pre-commit, which is not an
ecosystem. Corroborated from Renovate's side by its GitHub ecosystem
enum (actions, composer, go, maven, npm, nuget, pip, rubygems,
rust), which logs and skips anything else. The github-actions
coverage holds only for actions referenced by semantic version and
not by commit sha. The Mission's security-latency paragraph is
corrected accordingly, and open question 5 is answered no, with the
reason: OSV's ecosystem map reaches neither docker nor the
github-tags datasource the pre-commit manager uses, so it would
duplicate the coverage the fleet already has.
Pull request limits (1a, and a decision against the step's
recommendation). Defaults confirmed at prHourlyLimit: 2 and
prConcurrentLimit: 10, both enforced per repository: the hourly
count filters the repository's own pull requests by branchPrefix
and createdAt within the clock hour, and both counters are
re-seeded per repository, so one central multi-repository run still
gives each repository its own budget. Vulnerability pull requests
are exempt from the hourly limit unconditionally and have a separate
concurrent budget, unlimited by the forced prConcurrentLimit: 0;
they do still increment the shared hourly counter. The step
recommended prHourlyLimit: 0, reasoning that a limit of 2 against
a once-a-week window silently defers a repository's backlog. The
management session put the opposite case to the operator, who chose
2 and 10, stated explicitly on 2026-11-12. The reasoning: phase
2's runner is hourly, and the Situation measured about five
scheduled runs actually landing per day, so a whole-day window
offers each repository roughly ten pull requests against a measured
need of about one a week; the limit binds only at cold start, which
is where smoothing a burst into the CI cluster is the wanted
behaviour rather than a cost; and a held update is visible on the
dependency dashboard and already measured by phase 4, which
prHourlyLimit: 0 would quietly remove.
The validator needs --no-global (1a). A new finding, and it
changes step 1b. renovate-config-validator treats a file given as
an argument as a global self-hosted configuration, and global mode
accepts a self-hosted-only key in what is meant to be a repository
preset. Reproduced independently by the management session with
renovate@44.222.2 in a node:24 container: a preset containing
"onboarding": false exits 0 in the default mode and 1 under
--no-global, with The "onboarding" option is a global option
reserved only for Renovate's global configuration. Both modes catch
an unknown key (notARealRenovateKey, exit 1) and a bad value, so
the flag is about the one class of mistake global mode misses. A
preset holding $schema, an extends of a built-in preset,
ordinary keys and packageRules validates offline in both modes, as
does a consumer config whose only extends is the local> preset
--- no preset resolution is attempted, so the hook needs no network
beyond its own install.
The preset (1b). Written 2026-11-12. Top-level keys, in file
order: $schema, extends (customManagers:githubActionsVersions),
gitAuthor, dependencyDashboard, assignees, timezone
(Australia/Sydney), schedule (on saturday), prHourlyLimit
(2), prConcurrentLimit (10), pre-commit (enabled, with its
description), packageRules, minimumReleaseAge (3 days),
rollbackPrs. No vulnerabilityAlerts and no
osvVulnerabilityAlerts, per 1a. Six packageRules, in order: the
no-automerge rule; majors on the monthly window; the CI-tooling
group (matchManagers: ["github-actions", "pre-commit"],
groupName: "CI tooling", no matchUpdateTypes so every type
including digest, monthly); the Renovate-major prBodyNotes
rule; the Node-major disable rule, now naming
templates/renovate/renovate.yml as well; and the shellcheck-py
versioning rule. Checked by the management session against the
template config the preset replaces: no policy key was lost, and
the preset adds exactly timezone and the two limits. The
shellcheck rule compares equal to the template's parsed rule, with
the build group and the $ anchor intact.
Two rules the plan named a schedule for but no rule: majors, and the CI-tooling group's name. Both were the step's judgement and both were kept.
The validator proof (1b). With "notARealRenovateKey": true
added to the preset, pre-commit run renovate-config-validator
--all-files failed as follows, naming the file, which is the whole
point of the override:
renovate-config-validator................................................Failed
- hook id: renovate-config-validator
- exit code: 1
INFO: Validating renovate-fleet.json as repo config
ERROR: Found errors in configuration
"file": "renovate-fleet.json",
"errors": [
{
"topic": "Configuration Error",
"message": "Invalid configuration option: notARealRenovateKey"
}
]
as repo config in that output is --no-global taking effect; the
default mode says as global config and is what would have let a
self-hosted-only key through. The preset was restored from a copy
and the hook passes again.
The audit change (1c). Done 2026-11-12. extends_fleet_preset
is the only place the preset's name is matched, by exact membership
of extends; renovate_config_manages_pre_commit and
renovate_config_reads_version_comments take the parsed dict and
know nothing of any preset; each public function is the helper or
the predicate. 2579 tests pass, up from 2569, and
python3 scripts/audit-check.py --repo-path . --repo-name
development reports renovate as pass, "Renovate workflow and
config exist", with both findings gone.
Mutation check: seven mutants, seven killed, none surviving and
none timing out. The preset losing pre-commit.enabled, losing the
pre-commit key, and losing the version-comment extend; each of the
two call sites losing its predicate; the predicate loosened to
substring matching; and a per-check helper taught the preset's name.
The substring mutant survived the step's first set of tests and a
test was added for it. The management session re-ran the substring
and dropped-manager mutants on copies of the tree and confirms both
fail.
The documentation (1d). Done 2026-11-12.
docs/audits/renovate.md and templates/renovate/README.md each
gained a section on the preset, and the inverted renovate.json
story --- the schema and one extends, with a repository adding only
what is particular to it --- replaced the per-repository adaptation
both pages described. The five facts that appear on both pages were
compared mechanically rather than by reading: the cadence, the
limits, the CI-tooling group's coverage, the security paragraph and
the enabled: false exception each appear exactly once on each page
and are identical. Nothing had to be reconciled, and neither page's
verbatim-copy claim for renovate.yml was touched, which survey
item 4 is the reason for.
Three corrections by the management session rather than the step.
The specification's "What we check" list had come to say that
renovate.json extends the preset, which the check does not verify
--- a repository keeping the policy locally still passes --- and a
criterion page that overstates its own check misleads the one reader
who consults it, the person working out what their repository is
measured on. The new section described the audit's preset handling
in terms of "tests" where the code has helper functions, which also
risks this repository's own documentation test holding prose that
names a test to naming one that exists. The rest were rewraps and
one sentence in which "the preset" meant both the built-in
customManagers:githubActionsVersions and the local fleet preset in
the same breath.
Open questions settled. 2 (Australia/Sydney) and 5 (no OSV);
3 stands as planned, confirmed by the operator on 2026-11-12.
Execution notes¶
Dates are Australian Eastern Daylight Time (UTC+11).
Step 1c touched three files, not the two it was scoped to
(2026-11-12). scripts/tests/test_hooks.py holds every suite hook
in .pre-commit-config.yaml to firing on everything its suite
reads, and decision 3's test reads renovate-fleet.json, so the
audit-package-tests hook's files: pattern had to name the
preset. The step was fenced off from that file and reported the
failure rather than reaching for it; the management session made
the one-line change. The invariant is right and the step's scope
was wrong: the preset is trusted fleet-wide by name, so a commit
that guts it has to be a commit that fails, including one that
touches nothing else.
The survey's item 5 was demonstrated, not merely argued
(2026-11-12). With 1b landed and 1c not yet, audit-check.py
against this repository reported both findings --- the
version-comment extend and the pre-commit manager --- and the unit
suite failed on test_the_template_itself_passes for the
version-comment extend alone, because that fixture has no
.pre-commit-config.yaml for the manager check to apply to. Had
step 1c shipped as the master plan first wrote it, covering only
the pre-commit manager, this repository would have started failing
its own renovate criterion at the next scheduled audit run.
The operator overrode step 1a on the pull request limits
(2026-11-12), choosing prHourlyLimit: 2 and
prConcurrentLimit: 10 over the step's recommendation of 0 and
10. The Record's limits entry has the reasoning on both sides.
Decision 1's nested merge was measured, not assumed (2026-11-12).
The review round asked what happens now that
customManagers:githubActionsVersions arrives through the preset
while this repository keeps a local customManagers entry: if
Renovate replaced the array rather than concatenating it, the pinned
Renovate in templates/renovate/renovate.yml would stop being
bumped, which is the defect the local entry exists to prevent, and
neither the validator nor any criterion would see it. Renovate
44.227.4 flags customManagers mergeable with type array, and
mergeChildConfig concatenates such an option parent-first, at every
level of preset resolution. Confirmed against the real files by
registering renovate-fleet.json in the resolver's internal preset
registry and resolving this repository's renovate.json through it:
two entries come back, the built-in first and the templates/ one
second, with all six packageRules, the schedule and the timezone
intact. Not left as a test, because reproducing it needs Node and the
renovate package, which the audit suite does not have.
Risks and mitigations¶
- The preset cannot be tried before it is live. Renovate
reads a
local>preset from the default branch, so the merge is the first fleet-wide effect and there is no staging. Checked by: the validator hook, which is proven to fail on a broken preset before anyone relies on it (step 1b); phase 1 leavingdevelopmentas the preset's only user, so its dependency dashboard is the canary; and phase 2'sRENOVATE_DRY_RUN=lookuprun being the first live read. The management session reads the preset's key list against the scope paragraph before the commit is proposed. - The audit now trusts a preset by name. Widening two checks
to accept
local>shakenfist/development:renovate-fleetmeans a preset that silently dropped the pre-commit manager or the version-comment extend would be invisible to the fleet's own measurement. Checked by: decision 3's test, which loads the real file and asserts the per-check helpers accept it without the special case, and the mutation check that shows the test fails when the preset is gutted. The management session reads that test rather than trusting the step's summary. - Step 1b without 1c fails this repository. Removing the
pre-commitkey and thecustomManagersextend from the rootrenovate.jsonmakesdevelopmentnon-compliant on its ownrenovatecriterion at the next scheduled run, which files an issue against this repository. Checked by: both steps landing in one pull request, and the definition of done requiringaudit-check.pyto reportrenovatecompliant on the branch before the commit is proposed. - A week's updates could slip silently.
on saturdaymeans Renovate acts only when a run lands inside the window, and the Situation measured gaps of up to 9 hours 35 minutes between scheduled runs. Mitigated by the window being a whole day (open question 3) rather than six hours, and by this repository still running hourly in phase 1, so the canary has roughly 24 chances to land in it. Phase 2 records the central runner's longest gap against the same window. - The validator adds a Node toolchain fetch to every
pre-commit run.
language: nodewithlanguage_version: ltsmeans pre-commit installs Node and Renovate into a hook environment on first use, on workstations and on the[self-hosted, static]runner thatci.yml'slint-and-testjob uses. Checked by: the step reporting the hook running successfully, and CI on the pull request being the second place it runs. If the runner cannot reach the npm registry the step reports that rather than working around it, because a hook that cannot run in CI is not a check.
Definition of done¶
Every item is checked by the management session against the files and the commands, not against a sub-agent's summary.
renovate-fleet.jsonexists at the repository root, and every key the Scope paragraph lists is present in it.pre-commit run --all-filespasses, and theRecordholds the output of arenovate-config-validatorrun against a deliberately brokenrenovate-fleet.jsonthat failed and named that file. A hook that passed on a broken preset is the one outcome this item exists to exclude.- The hook passes
--no-global, and the preset setsprHourlyLimit: 2,prConcurrentLimit: 10and novulnerabilityAlertsobject, which is what step 1a's Record concluded. templates/renovate/renovate.jsonand the rootrenovate.jsoneach extendlocal>shakenfist/development:renovate-fleet, and neither restates a key the preset holds. Checked by comparing their top-level keys andpackageRulesdescriptions against the preset's, not by reading for intent.- The root
renovate.jsonstill carries thegithub-actionsmanagerFilePatternsand thecustomManagersregex entry fortemplates/, both with their descriptions. python3 -m unittest discover -s scripts/tests -t scriptspasses, andpython3 scripts/audit-check.py --repo-path . --repo-name developmentreportsrenovatecompliant on the branch.- Both
renovate_manages_pre_commitandrenovate_reads_version_commentsaccept a config whose onlyextendsentry is the fleet preset, and reject a config with neither the local form nor the preset. Both have a test for each of those three cases. - A test loads the real
renovate-fleet.jsonand asserts each per-check helper accepts it with no preset special case in play, andmutation-checkreports that test failing when the preset is edited to drop what it checks. - No fact about the preset is stated differently in
docs/audits/renovate.mdandtemplates/renovate/README.md, and neither page still describes the fleet policy as living in each repository's ownrenovate.json. - Neither page's description of
renovate.ymlas copied verbatim has been changed, because survey item 4 found it true. - Open questions 2 and 5 are answered in the plan, and open question 3 either stands or is changed with step 1a's reason recorded.
- The four stale claims in survey items 1, 3, 4 and 6 are corrected in the master plan's own prose --- the Mission's target derivation, the phase 1 scope paragraph, the step table and the two Bugs-fixed entries --- in the commit that adds this phase plan.
Back brief¶
Before step 1a, back brief the operator on decision 1, which is
the one with a real counter-argument. Putting
customManagers:githubActionsVersions in the preset means a second
audit helper has to learn about presets, and the cheap alternative
--- leave that one entry in every repository's local
renovate.json, where the audit already sees it --- costs nothing
this phase and nothing in phase 3's sweep. The argument for doing
it anyway is that the entry is fleet policy by exactly the test the
shellcheck rule was admitted under, and that a plan whose purpose
is to move policy into one file should not leave a line behind
because the measuring tool cannot read the new location. Confirm
also decision 5's timezone and cadence, since they are fleet-wide
on merge.
Then stop again after step 1a, before 1b writes the preset:
vulnerabilityAlerts' defaults, the Dependabot ecosystem coverage
and the two pull request limits all change what the preset must
say, and the preset is the one file in this plan that cannot be
tried before it is live. Report 1a's four answers and the preset's
intended key list, and get agreement on them before any file is
written.
Phase 2: central runner and cutover¶
Planning effort: high. This phase is the only one with live, fleet-wide effect, and the cutover order is what prevents duplicate pull requests.
Scope¶
In scope: .github/workflows/renovate-fleet.yml in this repository,
its global configuration file renovate-runner.json, the validator
hook change that makes a global file checkable at all, the two
preset rules that have to learn about the new workflow, and the
cutover of all 23 managed repositories in the order below.
Out of scope: deleting any repository's .github/workflows/renovate.yml
or moving its renovate.json onto the preset, which is phase 3;
re-measuring pull request volume, which is phase 4; and changing any
schedule the preset already sets, which phase 1 settled.
What the survey found¶
The master plan's phase 2 section was written on 2026-10-03, before
the fleet left renovatebot/github-action on 2026-10-15 and before
phase 1 shipped the preset. Six of its claims had moved. All of them
are corrected at their source in the commit that adds this plan, so
the prose above no longer disagrees with the tree.
-
The central runner cannot use
renovatebot/github-action. The section passed the runner's configuration "to the action through itsconfigurationFileinput", named the action as the dependency the temporary rule suppresses, and gave it a local weekly override. Phase 1's survey item 1 already recorded thattemplates/renovate/renovate.ymlstopped using the action on 2026-10-15 and installs the npmrenovatepackage instead.docs/audits/renovate.mdgives the reason, and it applies to this workflow with full force: "The static runner pool is gaining ephemeral pods on a Kubernetes cluster, and they have no Docker daemon, so the action fails on every run a pod takes." Three repositories still run the action (agent-python,client-python,client-python-k3s) and are non-compliant for it, so the action survives in the fleet but not as anything this phase should adopt. -
The runner-label question is already answered, and the answer is no. The section asked whether
staticrunners are ephemeral and said to record the answer. This repository recorded it in.github/workflows/consistency-audit.yml, which runs its issue-writing job on[self-hosted, vm, debian-13, s]rather thanstaticbecause the static runners "share one user and one home directory between jobs, so code a leg runs could leave a.pthfile or a~/.gitconfighook there for this job to run withAUDIT_TOKEN. A VM starts from the image and is thrown away after one job."RENOVATE_TOKENpushes branches to every repository, so it is strictly more powerful thanAUDIT_TOKEN, which can write none of them. -
The discovery surface is 23 repositories, and it adds nothing silently. The section sized the timeout for "about 20". Counted on 2026-11-12 across the 30 unarchived repositories in the organisation: exactly 23 carry both
renovate.jsonand.github/workflows/renovate.yml, and the other seven (divergulent-reviews,homebrew-tap,performance,private-ci,reproducables,sonobouy,uefi-latency-guest) carry neither. BecauserequireConfig: "required"manages only a repository with a config, and because the config and the workflow coincide exactly,autodiscoverovershakenfist/*discovers precisely the set that Renovate already runs in. The cutover list is therefore knowable now rather than from the dry run, and it is in Cutover below. Two of the 23,imagesandprivate-ci, are N/A for therenovateaudit criterion while still being managed;private-ciis one of the seven without a config, so the managed N/A isimagesalone. -
renovate-runner.jsonneeds nofiles:edit, and the hook rejects it. The section said to add the file to the validator hook'sfiles:and break it on purpose. Phase 1 made that pattern derived rather than a list --^(.*/)?(\.renovaterc|\.?renovate[^/]*\.json[c5]?)$-- so the file is picked up with no edit at all, which is what phase 1 intended. The consequence the section only flagged as a possibility is real and was measured on 2026-11-12: with arenovate-runner.jsonholdingautodiscover,autodiscoverFilter,requireConfigandonboardingin the tree,pre-commit run renovate-config-validator --all-filesexits 1 and reports all four as "a global option reserved only for Renovate's global configuration", because phase 1 passes--no-globaland the hook validates every match as repository configuration. One hook cannot check both kinds of file, so decision 3 splits it. -
The preset's Node-major rule does not cover the new workflow.
renovate-fleet.json's fifth rule disables majornodeupdates withmatchFileNamesnaming.github/workflows/renovate.ymlandtemplates/renovate/renovate.ymlonly. Arenovate-fleet.ymlthat sets up Node is read by the samegithub-actionsmanager as a dep namednode, so the moment this phase adds the file, major Node updates become proposable against the fleet's own runner, and merging one stops Renovate with "Unsupported node environment detected" for every repository at once. Confirmed against the pinned version:renovate@44.227.4declaresengines.nodeof^24.11.0 || ^26.13.0, so Node 25 is genuinely refused. The preset's Renovate-majorprBodyNotesentry points the reader at the same two files and so becomes incomplete in the same commit. -
Four mechanical claims hold, and one is a no-op. Checked against
renovate@44.227.4in the pre-commit hook's own installation.dryRunisglobalOnlywithallowedValues: ["extract", "lookup", "full"], soRENOVATE_DRY_RUN=lookupis valid. Global parsing merges the environment over the file (dist/workers/global/config/parse/index.js:81), andautodiscoverFiltercarries nomergeableflag, so theworkflow_dispatchfilter input replaces the file's value rather than concatenating with it -- which is what makes a single-repository canary possible.packageRulesismergeable: true, so the section's claim that the global rule concatenates with each repository's resolved rules, parent first, is right, and a later rule settingenabledwould win.requireConfig: "required"is already Renovate's default, so setting it documents rather than changes behaviour.
Five stale figures elsewhere in the plan are corrected with this
phase rather than left for a later one to trip over. The Success
criteria and the phase 4 section both still said the target was
"down from 227", which phase 1's survey item 2 superseded by
measuring 417 org-wide Update pull requests in 30 days; the
Mission was corrected then and these two were missed. The Mission's
first change, and two counts in phase 3, still said seventeen
copies of renovate.yml, which was true on 2026-10-03 and is 23
today by item 3 above. The Situation's own dated statements of the
same number are left alone: they are accurate history, and phase 1
set the convention of pointing forward from them rather than
rewriting them.
Decisions¶
-
Run Renovate from npm, the way the template does. Survey item 1 leaves no choice: an action that needs a Docker daemon fails on every run that lands on a Kubernetes pod. The workflow therefore follows
templates/renovate/renovate.yml--actions/setup-node,npm install --ignore-scriptsof a pinnedrenovate,npm rebuild re2, and the binary invoked directly -- with the global config named byRENOVATE_CONFIG_FILEinstead of an action input. The install step holds no token, as in the template, so an install script in Renovate's dependency tree cannot reachRENOVATE_TOKEN. -
The job runs on an ephemeral VM in a protected environment, not on
[self-hosted, static]. This is the decision most worth arguing with, so the argument is in Risks as well. The case for it is survey item 2: this repository has already decided that a job holding a fleet-write token does not run on a pool that shares a home directory between jobs, and it decided that for a token that can write nothing. The case against is that the 23 per-repository copies run onstatictoday, sostaticis not a regression, and that an ephemeral VM pays the Node and npm install on every hourly run with no cache to reuse. The deciding consideration is that this phase is the last cheap moment: after cutover there is one workflow holding one token with organisation-wide push, dispatchable by anyone who can dispatch a workflow, and moving it later means re-proving the cutover. The environment also buys themain-only deployment rule thataudit-writerelies on, which matters more here than there: a branch that edits this workflow must not be able to dispatch it with the organisation's push token. Namedrenovate-write, deploying only frommain. -
Keep the name
renovate-runner.jsonand split the validator hook in two. The alternative is to name the file so phase 1's pattern misses it, which buys a green hook by making the fleet's most privileged configuration the one file nobody validates -- the exact failure phase 1's derived pattern was written to prevent. So the hook becomes two entries on the same pinned repository: the existing one, with the runner config excluded, keeping--no-globalso that a repository config holding a global option is still rejected; and a second naming only the runner config, in default mode, where global options are accepted.RenovateConfigHookTestthen stops asserting that one hook matches every config and asserts the stronger invariant: every Renovate configuration in the tree is validated by exactly one of the two, neither both nor neither. That keeps the property derived rather than listed, so the next config added is still covered without anyone remembering this. -
The preset learns about
renovate-fleet.ymlin the same pull request that adds it. Survey item 5 is a defect this phase creates, so this phase fixes it:.github/workflows/renovate-fleet.ymljoins the Node-major rule'smatchFileNames, and the Renovate-majorprBodyNotesentry names it as the file whosenode-versionhas to move with the pin. Both are preset edits, so they are live fleet-wide on merge, and both are tested. -
Hourly, at minute 23. Unchanged from the section, and the reasoning stands: the Situation measured GitHub delaying and dropping top-of-hour scheduled events, and one runner means a dropped run delays all 23 repositories at once.
-
The cutover list is the 23 repositories measured in survey item 3, written into Cutover below rather than derived from the dry run. The dry run still has to agree with it; a disagreement is a finding, not a correction to make silently.
-
The temporary disable rule stays file-based, for a rewritten reason. The section justified
matchFileNamesover disabling therenovatebot/github-actionpackage by sayingrenovate-fleet.ymluses that action too. It will not. The rule is still right, and now for a better reason: between this phase and phase 3 the disabledrenovate.ymlfiles remain in 23 trees, each pinning arenovateversion that thecustom.regexmanager bumps and three also pinning the action, so a package-based rule would have to name three different dependencies to suppress churn in files that no longer run, while onematchFileNamesentry covers every dependency in them. The file list is the two the preset already names. -
No local weekly override for the central pin. The section gave
renovatebot/github-actioninrenovate-fleet.ymla localgroupNameand weekly schedule, to keep the most privileged workflow off the monthly window. Survey item 6's manager check makes most of that unnecessary: the npmrenovatepin is found bycustom.regex, notgithub-actions, so the preset's CI-tooling group never matched it and it is already weekly. What is left on the monthly window isactions/checkoutandactions/setup-nodein this one file, so a local rule gives just those two the weekly schedule, scoped bymatchFileNamesto.github/workflows/renovate-fleet.yml. That rule is local because the file exists only here.
Steps¶
Steps 2a to 2d write files and are sub-agent work. The cutover is not: every one of its actions changes a live system in other people's repositories, so it is run by the management session with the operator watching, in the order under Cutover.
| Step | Effort | Model | Isolation | Brief for sub-agent |
|---|---|---|---|---|
| 2a | high | opus | none | Write .github/workflows/renovate-fleet.yml and renovate-runner.json. Model the workflow on templates/renovate/renovate.yml, which runs a pinned npm renovate rather than renovatebot/github-action (decision 1), keeping its comments where they still apply and rewriting them where they do not -- the concurrency comment carries over, RENOVATE_AUTODISCOVER_FILTER: ${{ github.repository }} does not. Triggers are schedule with cron: '23 * * * *' and workflow_dispatch only; top-level permissions: contents: read; concurrency: {group: renovate-fleet, cancel-in-progress: false}; runs-on: [self-hosted, vm, debian-13, s] with environment: renovate-write (decision 2); timeout-minutes set high enough for a serial pass over 23 repositories, with a comment saying the number is provisional until the dry run measures it. Two workflow_dispatch inputs, passed as RENOVATE_DRY_RUN and RENOVATE_AUTODISCOVER_FILTER: survey item 6 confirms dryRun accepts extract/lookup/full and that the environment overrides the file for autodiscoverFilter, so the filter input narrows rather than widens. The checkout is this repository at the default branch and nothing else; persist-credentials: false, since Renovate clones with RENOVATE_TOKEN into RENOVATE_BASE_DIR. Install Renovate in a step holding no token, exactly as the template does, pinned behind a # renovate: datasource=npm depName=renovate comment so customManagers:githubActionsVersions bumps it. Before Renovate starts, add the write-path probe cutover step 2 needs: a script in tools/ (not inline, per the repository's convention on workflow steps longer than a few lines) that calls GET /repos/shakenfist/<repo> with RENOVATE_TOKEN for each of the 23 repositories and fails naming the repository if permissions.push is not true. renovate-runner.json holds $schema, autodiscover: true, autodiscoverFilter: ["shakenfist/*"], requireConfig: "required" (already the default; set to document it), onboarding: false, and the one temporary packageRules entry from decision 7. Do not edit .pre-commit-config.yaml -- step 2b owns it, and the hook fails on this file until it lands, so report that rather than working around it. |
| 2b | medium | sonnet | none | Split the renovate-config-validator hook in .pre-commit-config.yaml per decision 3, and move scripts/tests/test_hooks.py's RenovateConfigHookTest onto the stronger invariant. The existing entry keeps args: ['--no-global'] and gains an exclude: for the runner config; a second entry on the same rev names only that file and passes no --no-global. Give each a distinct name: and alias: so pre-commit run <alias> reaches either. Keep the file's convention of a comment saying why, and say why there are two: a repository config holding a global option must still fail, and the runner config is not a repository config. The test currently asserts that one hook's pattern matches every Renovate configuration in the tree; rewrite it to assert that each is matched by exactly one of the two hooks, keeping the derivation (the set comes from git ls-files, not a list) and the guard that the derived set is non-empty. Prove both halves: break renovate-runner.json with an unknown top-level key and confirm the global-mode hook fails naming that file; put autodiscover: true into renovate.json and confirm the repository-mode hook still rejects it. Paste both outputs into your report and restore the files; do not commit either broken state. |
| 2c | medium | sonnet | none | Teach renovate-fleet.json about the new workflow (decision 4) and add the local rule from decision 8, then test all three. In the preset: add .github/workflows/renovate-fleet.yml to the Node-major disable rule's matchFileNames, and extend the Renovate-major prBodyNotes entry so it names that file as the one whose node-version moves with the pin, without losing the two files it already names -- phase 3 removes those, not this phase. In this repository's own renovate.json: a rule scoped by matchFileNames to .github/workflows/renovate-fleet.yml and matchPackageNames to actions/checkout and actions/setup-node, giving them the weekly schedule, with a description saying why the npm pin needs no such rule (it is found by custom.regex, which the CI-tooling group's matchManagers never matched). Extend RenovateFleetPresetTest in scripts/tests/test_packaging.py with a test per change, using the existing _rule(**match) helper; the Node rule's test must assert the full matchFileNames set rather than membership, so dropping one file fails it. Then mutation-test: break each of the three additions in turn, confirm a test fails and names the right thing, keep the script under this session's scratchpad, and report the count. Restore from copies, never git checkout <dir>, and run each mutant under timeout. |
| 2d | medium | sonnet | none | Document the central runner. docs/audits/renovate.md is the fleet's Renovate specification and currently describes a per-repository workflow as the arrangement; add a section for the central runner that says what it is, where its global config lives, why it runs on a VM in a protected environment, and that the per-repository copies are disabled but not yet deleted, which phase 3 does. Do not rewrite the "What we check" bullets: the criterion still checks the per-repository files, which still exist, and changing it belongs to phase 3. templates/renovate/README.md gets a short note that the template is on its way out and why it is still there. Keep the fact in one place and point at it from the other, as phase 1 did: no fact about the runner stated differently on two pages. AGENTS.md and ARCHITECTURE.md only if a convention or the shape of the system changed -- a new fleet-wide workflow is arguably the latter, so make the judgement and say which way you went. |
Cutover¶
Operator-gated, in this order. Nothing here is a sub-agent step, and nothing in it is done without an explicit yes.
- Dry run, read path only. Dispatch with
RENOVATE_DRY_RUN=lookup. Confirm it discovers exactly the 23 repositories below, that it reads each one's Dependabot alerts (open question 1), and that every dependency in arenovate.ymlis reported as disabled by the temporary rule -- which is also the check that a globalpackageRulesentry reaches each repository's resolved config at all. Record the run's duration: it sizestimeout-minutesand tells us whether an hourly cadence is achievable at all. - Write path, before anything is disabled. The probe step from
2a fails the job naming any repository whose
permissions.pushis not true; record its result here. Then canary on this repository alone:gh workflow disable renovate.ymlhere, tick one held update on this repository's dependency dashboard, and dispatch the central workflow for real with the filter input set toshakenfist/development. Confirm it pushed the branch, opened or updated the pull request, and rewrote the dashboard. Tick an update to a workflow file specifically, because pushing one needs a capability the probe cannot see: push permission is not theworkflowscope, and most of what this fleet's Renovate proposes touches.github/workflows/. A canary that only moved a requirements pin would leave that untested. - Disable the rest.
gh workflow disable renovate.ymlin each of the other 22. The full list,developmentincluded, is here so that the rollback is mechanical:actions,agent-python,andris,client-python,client-python-k3s,client-python-ova,clingwrap,cloudgood,development,divergulent,hunkydory,images,instar,kerbside,kerbside-client,kerbside-patches,library-utilities,occystrap,ryll,sfui,shakenfist,uncalibrated-sextant,visual-digest-rust.
Not before the rust-toolchain pin lands, due about
2026-11-18. renovate-runner.json disables every dependency
in .github/workflows/renovate.yml and
templates/renovate/renovate.yml for the central runner, so
the dtolnay/rust-toolchain@stable reference in both --- held
by minimumReleaseAge because the stable branch moved on
2026-11-15 --- can only ever be pinned by each repository's own
hourly workflow, which step 2 and this step disable. Disabling
first strands those two references unpinned, and
PLAN-workflow-ref-trust.md step 3a's definition-of-done grep
then fails on them. Either wait for that pin, or narrow this
rule to drop templates/renovate/renovate.yml first, which
phase 3 has to do anyway. The reasoning is in that plan's step
3a record.
4. Turn the schedule on, and watch a full day. The workflow
lands with workflow_dispatch only, so this step is a pull
request of its own adding the schedule block its header comment
records, and it comes after step 3 because a schedule with the
copies still enabled is the duplicate pull request race. Then
watch: dependency dashboards update, no duplicate branches appear,
and open Renovate pull requests are adopted rather than reopened. Record how many times
renovate-fleet.yml ran that day, each run's duration, and the
longest gap, against the Situation's median of 4 hours 47
minutes and longest of 9 hours 35 minutes. A pass approaching 60
minutes reduces the cadence silently, because GitHub keeps one
pending run per concurrency group and cancels the older one. If
the central runner is not clearly better, stop and investigate
before phase 3: this plan's promise on security latency rests on
Renovate actually running.
Rollback is gh workflow enable per repository and disabling the
central workflow. Nothing in this phase deletes a file elsewhere in
the fleet.
Record¶
Both validator modes were proved against a broken file, not
read (2026-11-13). With an unknown key added to
renovate-runner.json, the global-mode hook fails and names the
file:
Validate Renovate global configuration...................................Failed
- hook id: renovate-config-validator
- exit code: 1
INFO: Validating renovate-runner.json as global config
ERROR: Found errors in configuration
"file": "renovate-runner.json",
"errors": [
{
"topic": "Configuration Error",
"message": "Invalid configuration option: notARealRenovateKey"
}
]
And with "autodiscover": true placed in the root renovate.json,
the repository-mode hook still rejects it, which is the property
--no-global was added for and the one the split had to preserve:
Validate Renovate repository configuration...............................Failed
- hook id: renovate-config-validator
- exit code: 1
INFO: Validating renovate.json as repo config
WARN: Found errors in configuration
"file": "renovate.json",
"warnings": [
{
"topic": "Configuration Error",
"message": "The \"autodiscover\" option is a global option
reserved only for Renovate's global configuration and
cannot be configured within a repository's config file."
}
]
Both files were restored from copies and verified byte-identical afterwards.
Mutation testing, 29 mutants and 29 killed (2026-11-13). Each
step kept a runnable script under this session's scratchpad rather
than improvising: step-2a/mutate.py for the push probe (12,
including the two defects its first run found -- an assertion reading
the wrong stream, and the bytecode cache serving a previous
same-length mutant), step-2b/mutate.py for the hook split (4: the
exclude removed, the global pattern widened, the global hook deleted,
and the global hook given --no-global), and step-2c/mutate.py for
the preset and the local rule (10: three file drops from the Node
rule, three from the note, and four ways of breaking the local rule).
The management session killed three more by hand, one per step, to
check the scripts rather than trust them: disabling the probe's
empty-set guard fails test_a_token_that_finds_nothing_to_manage_fails
with "the token can push to all 0 repositories", removing the hook's
exclude: makes the runner config matched by both hooks, and dropping
renovate-fleet.yml from the Node rule fails the whole-set assertion.
No mutant timed out.
The probe answers its own question, but not open question 1 yet
(2026-11-13). Run read-only against the organisation with an operator
token rather than RENOVATE_TOKEN, it reported 23 of the
organisation's repositories as managed and named exactly the 23 in
Cutover above, then reported the token could push to all of them.
That confirms the derivation and the 23, and says nothing about what
RENOVATE_TOKEN is or can reach: the probe's first real run, in
cutover step 1, is what answers that.
The cutover's four steps each produce a measurement that phase 3 and
phase 4 depend on, so they are recorded here as they happen: the dry
run's discovered repository list and duration, the push-permission
probe's output, the canary's branch and pull request, and the
watch day's run count, durations and longest gap. The answer to
open question 1 -- what RENOVATE_TOKEN actually is and what it can
reach -- is recorded here too, since the probe is the first thing in
this plan that establishes it rather than assuming it.
Cutover step 1: the fleet is discovered exactly, and a pass costs
under ten minutes (2026-11-14). Two dispatches, both read-only.
Run 43543283195 with dry_run: lookup took 9m22s for the job and
7m34s for Renovate itself; run 43557260170 with dry_run: full,
which computes every branch and pull request without writing any of
them, took 10m05s and 8m12s. Either leaves a wide margin under the
50-minute timeout-minutes, so the number set in 2a stands. Nothing
here settles the cadence, though: a dry run writes nothing, so 8m12s
is a floor rather than the cost of a real pass, and the queue delay
recorded under Execution notes sits outside timeout-minutes
entirely. The Situation's measurement of the hourly cron still holds,
and has drifted slightly down: re-read on 2026-11-15, the last 100
scheduled runs of renovate.yml in development, actions,
shakenfist and ryll span 23 to 24 days each, every one succeeding
and none cancelled, which is about four of the 24 hourly crons a day
where the Situation measured five. Step 4's watch day is what
measures the cadence.
The probe ran first on both, with the job's own RENOVATE_TOKEN
rather than an operator token, and reported:
40 repositories in shakenfist, 23 of them managed by Renovate
shakenfist/actions
shakenfist/agent-python
shakenfist/andris
shakenfist/client-python
shakenfist/client-python-k3s
shakenfist/client-python-ova
shakenfist/clingwrap
shakenfist/cloudgood
shakenfist/development
shakenfist/divergulent
shakenfist/hunkydory
shakenfist/images
shakenfist/instar
shakenfist/kerbside
shakenfist/kerbside-client
shakenfist/kerbside-patches
shakenfist/library-utilities
shakenfist/occystrap
shakenfist/ryll
shakenfist/sfui
shakenfist/shakenfist
shakenfist/uncalibrated-sextant
shakenfist/visual-digest-rust
the token can push to all 23 repositories
The 23 are the 23 in Cutover above, name for name, with nothing to
explain. Both runs predate step 2's canary and step 3, so every
per-repository renovate.yml was still enabled when the probe
reported, which is the second clause of the definition-of-done item
and the reason the probe runs before anything is disabled. The two
counts reconcile as 40 in the organisation, 10 of them archived, 30
autodiscovered by Renovate, 7 of those without a Renovate
configuration, 23 managed. Renovate logged each of the 7 as
disabled-no-config and onboarded none of them, so requireConfig
and onboarding: false behave as decision 5 intended. The probe's
own line is the weak part of that chain rather than its arithmetic:
the 40 it prints is the raw organisation listing, taken before the
archived filter in managed_repositories()
(scripts/renovate-check-push.py:191), so it invites a comparison
with Renovate's 30 that needs this paragraph to resolve. Filed as
issue #401 rather than fixed here, so that the probe's own log stops
needing a plan document to read.
The global rule reaches every repository's resolved config, and
will close four pull requests (2026-11-14, run 43557260170). The
full pass proposed no new branch anywhere -- the per-repository
copies have been running hourly and are current -- but it would
retire four pull requests that are already open:
| Repository | PR | Title |
|---|---|---|
actions |
251 | Update dependency renovate to v44.227.4 |
andris |
81 | Update dependency renovate to v44.227.4 |
client-python-k3s |
171 | Update renovatebot/github-action action to v46.3.13 |
kerbside-client |
24 | Update dependency renovate to v44.227.4 |
Each of the four changes .github/workflows/renovate.yml and nothing
else, which is exactly the file the temporary packageRules entry
disables. That is the proof decision 4 wanted, and a stronger one
than an absence of output: a global entry matched by matchFileNames
reaches each repository's resolved configuration, demonstrated by
Renovate retiring work it had already started on the strength of it.
The demonstration is positive for those four repositories and no
more. The other 19 had no pending renovate.yml update for the rule
to act on, so the run is consistent with the rule applying there
without evidencing it, and the 23 Would ensure Dependency Dashboard
lines show that each repository's configuration resolved rather than
that this entry matched in each. The first real pass, or a
debug-level log, is what would.
The four closures are intended and are recorded here so that whoever
finds them closed can see why. They update a workflow phase 3
deletes, so merging them would be churn. The canary in step 2 is
filtered to this repository and will not touch them; step 3 or step 4
is where they close. It also means the dependency dashboards in all
23 repositories would be rewritten by the central runner on its first
real pass, which the full pass shows as 23 Would ensure Dependency
Dashboard lines.
RENOVATE_TOKEN lives in two places, and one managed repository
can see neither (2026-11-15). Open question 1 asked where the token
lives and recorded that answering it needs admin:org, which the
planning session did not have. It is available now. There is an
organisation secret named RENOVATE_TOKEN with selected
visibility over 22 repositories, and an environment secret of the
same name on this repository's renovate-write environment, created
2026-11-13T04:27:19Z. No repository-level RENOVATE_TOKEN exists in
any of the 23 -- each was checked -- so phase 3's choice resolves to
restricting the organisation secret's repository list, with nothing
to delete per repository. Which of the two copies the fleet runner's
job received is not determinable from the run: both hold a working
token, and neither of GitHub's two secrets pages -- the concept page
and the how-to -- states a precedence between an environment secret
and an organisation secret of the same name, which is the same gap
Two claims about the environment were wrong records one level down.
The 22 are not the 23. uncalibrated-sextant is managed by Renovate,
is not on the organisation secret's list, and has no repository
secrets of its own, so secrets.RENOVATE_TOKEN is empty there. Its
hourly renovate.yml has failed every scheduled run it has ever had
-- 12 of 12, from 2026-11-11T15:40:59Z, which is its whole run
history -- each ending FATAL: Initialization error
with "You must configure a GitHub token". The other 22 each
succeeded on all of their last 10 scheduled runs. That correlation is
also the clearest evidence available that the per-repository
workflows read the organisation secret rather than something else:
every repository on the list works, and the one repository off it has
never worked. The cutover fixes it as a side effect, since the fleet
runner uses this repository's token and the probe reports it can push
to all 23, so the first real central pass would have been the first
dependency update uncalibrated-sextant ever had. It was fixed on
its own terms instead, on 2026-11-15 with the operator's approval:
the repository was added to the organisation secret's selected list
with the single-repository endpoint, taking it from 22 to 23, and the
selected set is now exactly the 23 Renovate manages. The run was not
forced, so its own cron is what proves it.
What is left is why it happened, and it is why issue #402 stays open:
nothing checks the organisation secret's selected list against the
set of repositories carrying a renovate.json. The probe is the
closest thing and cannot do it, because it reads permissions.push
for the token it already holds and cannot see secret visibility. The
next repository onboarded without being added to the list fails the
same way, silently. Both candidate homes for the check -- the
consistency audit's renovate criterion, or the probe itself --
need admin:org, which neither has today.
Open question 1's read path is still open after both dry runs (2026-11-14). Neither pass mentions Dependabot or vulnerability alerts at the default INFO log level, in either mode, so neither confirms nor denies that the central runner reads them. Absence of a log line is not absence of the fetch, and nothing here should be read as either. Answering it needs debug logging, which the workflow does not expose as an input; it is carried to phase 3 rather than left looking settled.
Execution notes¶
Dates are Australian Eastern Daylight Time (UTC+11).
The back brief's four answers (2026-11-13). Decision 2 stands: the
runner is an ephemeral VM in the renovate-write environment, with
the operator creating the environment himself, since it is a
repository setting rather than a file in a pull request. Naming it in
the workflow does not create that protection -- see Two claims about
the environment were wrong below -- so the restriction binds only
once the operator has configured it.
Phase 2 lands in two pull requests, not one (2026-11-13). GitHub
offers workflow_dispatch only for a workflow that exists on the
default branch, so none of the four cutover steps can run from the
phase branch. This pull request is steps 2a to 2d; the cutover runs
against main afterwards and its measurements land as a second pull
request on this phase. Four Definition of done items -- the dry run's
discovered list, the probe's output, the canary and the watch day --
are therefore unreachable in the first pull request and are checked
in the second.
The probe derives its repository set rather than reading the dry
run's (2026-11-13, step 2a). The phase's cutover text said the
probe checks "every repository the dry run discovered", which it
cannot know: the probe runs before Renovate, on every run. It
instead applies the rule requireConfig: "required" applies --
every unarchived repository in the organisation whose default branch
carries a renovate.json -- so it stays correct as repositories are
added and retired, where a list of 23 names would be a second copy
of the fleet. It reads only renovate.json of the longer list
Renovate accepts, and says so: covering that list partway would buy
confidence it has not earned, and the renovate audit criterion is
what keeps a repository from configuring Renovate another way
unnoticed.
Two departures from the template in renovate-fleet.yml
(2026-11-13, step 2a). The Rust and hashin setup steps are
unconditional, where the per-repository template guards each with
hashFiles(). The condition asks whether the workspace holds a
Cargo.lock or a requirements*.txt, and the repositories this
workflow manages are not in the workspace at all -- Renovate clones
them later -- so the guards would never fire and lock file refreshes
would fail at cutover for the fleet's Rust and hashed-requirements
repositories. The cost is a rustup install and a venv on every
hourly run. The probe also lives in scripts/ rather than the
tools/ the phase text named, because every Python file in this
repository is in scripts/ and every shell script in tools/.
timeout-minutes is bounded by the cadence, not the work
(2026-11-13, step 2a). 50 minutes, because GitHub keeps one pending
run per concurrency group: a pass that regularly ran past the hour
would quietly halve the schedule rather than fail anything, and
timing out under the hour makes it loud. The number is provisional
until the cutover's dry run measures a full pass.
The first review round caught a live-on-merge defect (2026-11-13).
The workflow was written with its hourly cron in place, so merging it
would have started real fleet-wide runs at the next scheduled tick
while all 23 per-repository copies were still running -- the duplicate
pull request race this phase exists to avoid, which the runner's own
concurrency group cannot prevent because the racing instances are
different workflows in different repositories. The cause was in the
planning, not the implementation: rewriting this phase's Cutover
section dropped the master plan's original "enable the central
runner's schedule" step, so nothing remained to turn the schedule on
and it shipped already on. The workflow now lands with
workflow_dispatch only, the block to add is recorded in its header
comment, and cutover step 4 turns it on in a pull request of its own.
Two claims about the environment were wrong (2026-11-13). This
phase said the workflow was inert until renovate-write existed.
GitHub's documentation says the opposite: "Running a workflow that
references an environment that does not exist will create an
environment with the referenced name", and that environment "will not
have any protection rules or secrets configured". Naming an
environment therefore protects nothing by itself, and the main-only
deployment rule binds only once an administrator has configured it.
Whether a job naming an environment still inherits a repository-level
secret of the same name is not documented either way, so nothing in
this plan or the spec asserts it. Both pages now say the protection is
verified out of band before the cutover.
The probe cannot see the capability that matters most
(2026-11-13). permissions.push is not the workflow scope, and most
of what this fleet's Renovate proposes touches .github/workflows/,
so the probe can report that the token reaches all 23 repositories
while every workflow-file branch is refused. It is documented rather
than probed, and cutover step 2's canary now ticks a workflow-file
update specifically, which is what establishes the capability.
The local weekly rule covers majors too (2026-11-13, step 2c).
The rule decision 8 asks for gives actions/checkout and
actions/setup-node in renovate-fleet.yml the weekly window, and it
is not narrowed by update type, so it also takes those two actions out
of the preset's first-of-the-month major window: a local rule is
concatenated after the preset's and the later match wins. Kept rather
than narrowed, because it is the same thing decision 8 wants for the
same reason -- the most privileged workflow in the fleet should not be
the slowest to pick up a fix -- and neither action is automerged, so a
major still waits for a human. Nothing else in the fleet is affected,
because the rule names a file that exists only here.
The validator hook fails between 2a and 2b, by design
(2026-11-13). renovate-runner.json is a global configuration and
phase 1's hook validates every Renovate file as a repository
configuration under --no-global, so the hook fails from the moment
2a creates the file until 2b splits it in two. Measured and recorded
rather than worked around: renaming the file to dodge the hook's
derived pattern would have left the fleet's most privileged
configuration the one file nobody validates.
The VM runner pool was drained under step 1 (2026-11-14). The
first dispatch queued at 01:54 and did not start until 04:21, because
sfcbr was drained for maintenance and nothing with the vm label was
answering. Three unrelated jobs were queued behind the same shortage
-- Secret scan, Mermaid lint and Prune and import review marks
-- so the cause was the pool rather than the labels or the
environment, and secret-scan.yml and prune-reviews.yml have used
the identical label set for months. The queued dispatch ran itself
when the pool returned; no retry was needed. Worth knowing for the
watch day in step 4: a drained pool does not fail the runner, it
delays it silently, and timeout-minutes does not cover queue time.
The runner's Renovate version moved between step 1's two runs
(2026-11-14). lookup ran 44.222.2 and full ran 44.227.4, because
398 merged ac1b8a6 between them, bumping¶
PINNED_RENOVATE_VERSION in renovate-fleet.yml. That is this
repository's own hourly copy finding the new file within hours of the
merge, which incidentally confirms the custom manager matches the
fleet runner as well as the template, but it means the two durations
differ in version as well as in mode.
Risks and mitigations¶
-
Disabling a repository's own Renovate strands what the central runner is configured not to see.
renovate-runner.json's temporary rule disables every dependency in.github/workflows/renovate.ymlandtemplates/renovate/renovate.yml, so those two files are maintained only by the per-repository workflow this phase turns off. The live case isPLAN-workflow-ref-trust.md's step 3a: itsdtolnay/rust-toolchain@stablepin is held byminimumReleaseAgeuntil about 2026-11-18, and disabling first means it never arrives. Mitigation: cutover steps 2 and 3 carry the date, and the alternative is to narrow the rule to drop the template path, which phase 3 has to do anyway --- the template outlives the deployed copies the rule exists for. -
The VM image may not have what Renovate needs. The
audit-writejob's comment says the image "has git and python3 but not gh", and says nothing about Node.actions/setup-nodesupplies Node, and the template's own steps supply npm, Rust when a repository has aCargo.lock, andhashinwhen it has arequirements*.txt, so a VM should be able to do everything a static runner does -- but it pays the whole install on every run. The dry run in cutover step 1 measures it. If an hourly cadence cannot be met on a VM, the fallback is a dedicated runner label rather than the shared-home static pool, and if neither is available the residual risk is recorded and accepted explicitly rather than by default. Checked by the management session against the dry run's duration, before step 3 disables anything. - A pass longer than an hour halves the cadence in silence. One pending run per concurrency group means a slow pass drops runs without failing anything. The watch day counts runs rather than assuming them, and the success criterion is comparative: at least as often as the per-repository copies managed.
- The token may not reach every repository. Disabling 23 workflows and then discovering the central runner cannot write to one of them leaves that repository unmanaged and quiet about it. The probe runs in the job, on every run and not just at cutover, and fails naming the repository; cutover step 2 runs it before step 3 disables anything.
- The preset edits in 2c are live on merge. They are
fleet-wide the moment the pull request lands, like everything in
renovate-fleet.json, and before the central runner is turned on. Both edits are conservative in that direction: adding a file to a disable rule can only remove update paths, and aprBodyNoteschange only alters prose in a pull request body. - A branch could dispatch the organisation's push token. The
renovate-writeenvironment deploying only frommainis what prevents a branch that has edited this workflow from running it withRENOVATE_TOKEN; the workflow's ownifon the ref is defence in depth, asconsistency-audit.ymldescribes.
Definition of done¶
Every item is checked by the management session against the files and the commands, not against a sub-agent's summary.
.github/workflows/renovate-fleet.ymlexists, triggers onworkflow_dispatchand nothing else, carries top-levelpermissions: contents: read, names therenovate-writeenvironment, runs on an ephemeral VM label, and contains nouses:ofrenovatebot/github-action. The absence of ascheduleis the point: it is what stops the runner working the same repositories as the copies that are still running, and the minute-23 block to add at cutover is recorded in the workflow's header comment.scripts/tests/test_renovate_fleet_workflow.pyasserts each of these, including that no third-party action beside the token runs on a ref that can move.- The workflow installs Renovate in a step whose
env:does not mentionRENOVATE_TOKEN, and the step that holds the token runs only what that step installed. renovate-runner.jsonexists withautodiscover,autodiscoverFilter,requireConfig,onboardingand the one temporarypackageRulesentry, and no other key.pre-commit run --all-filespasses with both validator hooks present, and theRecordholds two failing runs: the global-mode hook failing on a brokenrenovate-runner.jsonand naming it, and the repository-mode hook rejecting a global option placed inrenovate.json. A hook that passed on either is the outcome these two items exist to exclude.- Every file matching
^(.*/)?(\.renovaterc|\.?renovate[^/]*\.json[c5]?)$ingit ls-filesis validated by exactly one of the two hooks, asserted by a test that derives the file set rather than listing it. renovate-fleet.json's Node-major rule names all three workflow files, and a test asserts the whole set, so removing one fails it. The Renovate-majorprBodyNotesentry namesrenovate-fleet.ymlwithout having dropped the two files it already named.- This repository's
renovate.jsongivesactions/checkoutandactions/setup-nodeinrenovate-fleet.ymlthe weekly schedule, and no rule in it restates anything the preset holds. python3 -m unittest discover -s scripts/tests -t scriptspasses with more tests than before, andpython3 scripts/audit-check.py --repo-path . --repo-name developmentreports no verdict worse than it does onmaintoday, which isrenovatecompliant and the two pre-existing failuresplan-source-referencesandreview-coverage.- Mutation testing covers each of 2c's three preset and config changes, every mutant is killed, and the run names the expected string. The script's path and the count are in the commit message.
- The dry run's discovered repository list is recorded and equals the 23 in Cutover, or the difference is explained there.
- The push-permission probe's output is recorded for all 23 repositories, and it ran before any workflow was disabled.
- No fact about the central runner is stated differently in
docs/audits/renovate.mdandtemplates/renovate/README.md, and neither page claims the per-repository workflow has been deleted, because phase 3 deletes it. - The watch day's run count, durations and longest gap are recorded, and the comparison against the per-repository copies' median of 4 hours 47 minutes is stated as met or not met.
Back brief¶
Before step 2a, back brief the operator on decision 2, the runner
label, which is the one with a real counter-argument and the one
that cannot be changed cheaply afterwards. The argument against it
is that the 23 copies run on [self-hosted, static] today, so
static is no worse than the status quo, and that an ephemeral VM
pays a full Node and npm install every hour with nothing cached.
The argument for it is that this repository has already ruled that
a job holding a fleet-write token does not run on a pool sharing a
home directory, for a token that can write nothing, and that after
cutover this workflow is the single point through which every
repository in the organisation is written to. Confirm also that the
renovate-write environment may be created, since that is a
repository setting rather than a file in a pull request.
Then stop again before cutover step 3, the only irreversible-feeling step in the phase: 22 repositories stop running Renovate for themselves, and the central runner has to be known good first. Report the dry run's discovered list against the 23, the probe's output, the canary's result and the measured pass duration, and get agreement before disabling anything beyond this repository.
Phase 3: fleet sweep and audit change¶
Planning effort: high for the audit change, medium for the sweep.
The audit change reverses what the Renovate criterion means -- a
file it requires today becomes a failure -- and files issues across
the fleet at the next weekly audit run (Sunday 18:00 UTC), which is
high effort by this
repository's own rule (see Planning effort below). The sweep that
follows is the mechanical consistency-fix pattern the fleet
already runs. Plan them as two steps, and land the audit change
first.
Scope. In this repository: the Renovate check now requires
renovate.json to extend local>shakenfist/development:renovate-fleet,
and treats a .github/workflows/renovate.yml as a failure, because
if it were re-enabled it would race the central runner. Tests in the
packaging test module cover both new failures (the workflow file
present, the extends missing) as well as the pass. The issue title
Renovate does not change, since it is the fleet-wide idempotency
key. Rewrite docs/audits/renovate.md with the check: its "What we
check" list and rationale describe the required extends and the
forbidden workflow file, where today they say the workflow must
exist, runs hourly and is copied verbatim. Specifications are
written for human review and nothing regenerates them, so a spec
left alone would contradict the check and every issue it files. The
rewrite stales the spec's review mark in REVIEWS.md; that is
expected, and per the phase-landing rules above it is left for
prune-reviews rather than edited here. Delete
templates/renovate/renovate.yml and this repository's own
.github/workflows/renovate.yml, and remove the template's row
from the template README. Deleting those two files breaks
ConvergedTemplatesTest.test_this_repository_runs_renovate_verbatim
in scripts/tests/test_registry.py, which reads both and asserts
they are byte-identical, so pre-commit run --all-files fails in
the same pull request unless the test goes with them. Do not stop at
that one: run git grep -n 'renovate\.yml' and settle every hit
outside docs/plans/. On 2026-10-10 the others were the
ConvergedTemplatesTest docstring, the actionlint comment in
.pre-commit-config.yaml that cites renovate.yml and counts "the
ten copy-directly templates", the supporting-workflows list in
ARCHITECTURE.md (which should name renovate-fleet.yml from
phase 2 instead), the renovate.yml fixture in the packaging test
module, and docs/audits/renovate.md itself; the mention in
docs/audits/scheduled-workflow-health.md is history and stays.
The audit change lands first, and the
Renovate issues it files at the next weekly run are the sweep's
work queue. That is deliberate: it is the fleet's issue-driven
consistency-fix pattern, and a transitional check that accepted
both forms would spare about 23 issues at the price of letting the
sweep stall unnoticed. Before the audit change merges, preview its
verdicts against fresh clones of the fleet with audit-check.py
and audit-manage-issues.py --dry-run, as
docs/consistency-audits.md describes, and record here that the
count matched the one below. A workflow_dispatch of
consistency-audit.yml has no dry-run mode -- it files issues for
real -- so it is only for starting the sweep after the merge
without waiting for Sunday.
In each managed repository, one pull request: delete
.github/workflows/renovate.yml, and reduce renovate.json to
extends plus the repository's own rules. Keep every lockstep
group, every constraints.python and every rangeStrategy.
Lockstep groups stay local throughout this plan, because the
renovate-lockstep-groups criterion (RenovateLockstepGroups in
scripts/audit/checks/packaging.py) reads only the local file and
would report a group moved into the preset as missing; promoting
them is under Future work. Delete only rules that restate the
preset. Once every sweep pull request has merged, remove the
runner's temporary renovate.yml package rule from phase 2. Two repositories need extra care:
instar (its custom manager for cargo install pins and its frozen
release base image) and shakenfist (the most groups). In
kerbside, delete the shellcheck-py versioning rule, which now
restates the preset, but keep its shellcheck group and its
tox.ini custom manager, which are repository-specific. State in
this section how many repositories the audit change newly fails:
every managed repository until its sweep pull request merges,
except development, which moved onto the preset in phase 1 and
loses its own renovate.yml in the audit change's pull request, so
it passes from the start.
Once the sweep has merged, the preset's schedules govern the fleet for the first time. Watch the first Saturday after that and record here that the weekly window got runs, and that the week's pending updates opened or were visibly held by the preset's pull request limits: phase 2's daily run count does not show whether a one-day window was hit.
When no copy of renovate.yml remains, RENOVATE_TOKEN is needed
only by renovate-fleet.yml. If it is an organisation secret
(open question 1), restrict its repository access to
shakenfist/development; if it is a set of per-repository secrets,
delete the others. Either way, record which, so the credential's
exposure drops from 23 repositories to one.
Phase 4: re-measure¶
Planning effort: medium.
Scope. Thirty days after phase 3 completes, repeat the
Situation section's measurements: Renovate pull request count by
package and by the Situation table's rows, Renovate share of CI time
including merge-queue runs, and Renovate runs on static per day
with the longest gap between them. Record before and after here, and
say whether the Mission's target -- 100 or fewer Renovate pull
requests in 30 days, against the baseline this phase
re-establishes -- was met, and if not, which row
of the Mission's derivation missed its estimate. Record also the
latency of any [SECURITY] pull request opened in the window,
measured from the advisory's publication, and whether any
repository's dependency dashboard showed updates held by
prHourlyLimit or prConcurrentLimit, since a held update lowers
the count without being a saving. If grouping has made a failure
hard to attribute, say so here and adjust the preset; that is the
trade-off the operator asked to keep visible.
Phase 5: push audit¶
Run PUSH-AUDIT.md over the union of the Merged ranges of
phases 1 to 3 in this repository. Phase 3's per-repository sweep
pull requests each run their own repository's push audit, where one
exists, as they land; cite those audits here rather than re-running
them.
Agent guidance¶
Execution model¶
Sub-agent execution model (shared block; do not edit -- the
canonical copy lives in shakenfist/development at
templates/shared-blocks/subagent-execution-model.md):
All implementation work is done by sub-agents, never in the management session. The management session is reserved for planning, review, and decision-making. This keeps the management context lean and avoids drowning it in implementation diffs.
The workflow is:
- Plan at high effort in the management session.
- Spawn a sub-agent for each implementation step with the brief from the plan, at the recommended effort level and model.
- Review the sub-agent's output in the management session. Check the actual files -- the sub-agent's summary describes what it intended, not necessarily what it did.
- Fix or retry if the output is wrong. Diagnose whether the brief was insufficient (improve it) or the model was too light (upgrade it), then re-run.
- Commit once the management session is satisfied.
This applies to all steps, including high-effort ones. If a sub-agent cannot succeed even with a detailed brief and the right model, that is a signal the brief needs improving, not that the management session should do the implementation itself.
Use isolation: "worktree" for sub-agents when the change is
risky or experimental; the worktree is discarded if the output is
unsatisfactory. For safe, well-understood changes, sub-agents can
work directly in the main tree.
Planning effort¶
Planning effort (shared block; do not edit -- the canonical copy
lives in shakenfist/development at
templates/shared-blocks/plan-planning-effort.md):
The master plan itself is always created at high effort -- it requires broad codebase understanding, cross-referencing several source files, and judgment calls about scope and sequencing.
Each phase plan states the recommended effort level for planning that phase. Phases that turn on design decisions, cross-component coordination, protocol changes, or subtle correctness questions should be planned at high effort. Phases that are mechanical, or that follow a pattern already established elsewhere in the codebase, can be planned at medium effort.
In this repository. High effort is anything that changes what
a criterion means, anything that touches the scheduler in
audit-check.py, and anything that reaches
audit-manage-issues.py -- those decide what the fleet is held
to and what lands in other people's issue trackers. Medium effort
covers adding a criterion that follows the shape of an existing
one, a documentation sweep, or a template change with a worked
example already in the tree.
Step-level guidance¶
Sub-agent step guidance (shared block; do not edit -- the
canonical copy lives in shakenfist/development at
templates/shared-blocks/subagent-step-guidance.md):
Each phase plan includes a table like this:
| Step | Effort | Model | Isolation | Brief for sub-agent |
|---|---|---|---|---|
| 1a | medium | sonnet | none | One-sentence summary of what to do and which files to touch |
| 1b | high | opus | worktree | Why this needs high effort: requires understanding X to do Y |
Effort levels, from cheapest to most thorough:
- low -- Purely mechanical changes: rename, reformat, add a log line, regenerate generated code. The brief is a complete instruction.
- medium -- The plan provides enough context to follow a clear brief. The sub-agent may read a few files, but the approach is already decided.
- high -- Requires reading several files, making judgment calls, or understanding non-obvious invariants. The sub-agent needs to think about edge cases.
- xhigh -- The setting for hard coding and agentic steps: long-horizon changes, or steps where the sub-agent must both research and implement.
- max -- Correctness matters more than cost. Expect diminishing returns and occasional overthinking; reserve it for steps where a wrong answer would be expensive to detect.
Brief for sub-agent: this is the key field. Write it as if briefing a colleague who has never seen the codebase. Include what to change, which files to touch, what patterns to follow, and any non-obvious constraints.
A good brief front-loads the research the planner already did, so the implementing agent does not repeat it. Instead of "add storage functions for the new object", name the functions to add, the file they belong in, the existing equivalent to mirror (with line numbers), and any registration the change also needs.
The better the brief, the lower the effort level needed and the lighter the model that can succeed.
In this repository. A worked brief: instead of "add a check
that plans are indexed", write "add a PlanIndex class to
scripts/audit/checks/plans.py declaring the id plan-index, its
spec and its issue_title as class attributes and implementing
run(repo), register the instance in CHECKS in
scripts/audit/registry.py beside the other plan checks, write
docs/audits/plan-index.md following the structure in
docs/audits/README.md and linking to compliance.md#plan-index,
add the file to docs/audits/README.md, add its lines to
FROZEN_METADATA and FROZEN_ISSUE_TITLES in
scripts/tests/test_metadata.py, and add tests to
scripts/tests/test_plans.py covering pass, fail and
not-applicable." A brief that names four of the criterion's five
files is the characteristic defect here, and the frozen tables are
the one most often left out.
A check that does not apply reports
not_applicable with a reason rather than being omitted, because
an omitted check renders as unknown.
Model choice¶
Sub-agent model roster (shared block; do not edit -- the canonical
copy lives in shakenfist/development at
templates/shared-blocks/subagent-model-roster.md):
The planner recommends which model is best suited to each step. This is a judgment call, not a rigid rule -- the right model depends on what the step requires, not on whether it is "planning" or "implementation". The models available to sub-agents are:
- fable -- The most capable model available, for the hardest reasoning and the longest-horizon work: multi-step changes a single sub-agent must carry end to end, or steps whose correctness depends on holding a whole subsystem in mind at once. It costs materially more than opus, so reserve it for steps that have already defeated opus or are expected to.
- opus -- The default for steps needing deep reasoning, architectural understanding, subtle correctness judgment (locking, state machines, migrations), or intricate implementation that would be costly to debug if it were wrong.
- sonnet -- A good default for well-briefed implementation work. Faster and cheaper than opus, and effective when the plan front-loads the research and the brief leaves no broad judgment calls to make.
- haiku -- Suitable for purely mechanical tasks: search-and-replace, regenerating generated code, adding log lines, running commands. The brief must be a near-complete instruction.
Model choice interacts with effort level and brief quality. A detailed brief compensates for a lighter model -- sonnet at medium effort with a thorough brief often matches opus at medium effort with a vague brief. The planner's job is to write briefs good enough that the recommended model can succeed.
The model also determines the context window: fable, opus and sonnet have 1M tokens, haiku has 200K. A step that must hold many files in context at once may need one of the larger-context models for that reason alone, even when the reasoning itself is straightforward.
When in doubt, skew to the more capable model. Saving money only matters if the outcome is still acceptable. A failed or low-quality implementation wastes more time -- and therefore more money -- than the heavier model would have cost. Recommend a lighter model only when you are confident the brief is detailed enough for it to succeed.
In this repository. The project-specific checks referred to above are:
-
pre-commit run --all-filespasses. It runs actionlint, shellcheck, flake8, skillsaw and all five test suites, andci.ymlruns the same command on every pull request. -
python3 scripts/audit-check.py --repo-path . --repo-name developmentstill reports what it reported before the change, or the plan says why the verdict moved. - If the change touches issue filing, it was exercised with
--dry-runonly.
Management session review checklist¶
Management session review checklist (shared block; do not edit --
the canonical copy lives in shakenfist/development at
templates/shared-blocks/plan-review-checklist.md):
After a sub-agent completes, the management session verifies:
- The files that were supposed to change actually changed -- read them, do not trust the summary.
- No unrelated files were modified.
- The changes match the intent of the brief: not merely syntactically correct, but semantically right.
- The project's own pre-merge checks pass, including any generated code that has to be regenerated and committed (see the project-specific checks below).
- The commit message follows project conventions, including
the
Co-Authored-Byline recording model, context window, and effort level.
Administration and logistics¶
Success criteria¶
We will know when this plan has been successfully implemented because the following statements will be true:
- Renovate runs on an hourly schedule for the whole fleet from this
repository, and no managed repository carries
.github/workflows/renovate.yml. - Every managed
renovate.jsonextends the shared preset and holds only repository-specific rules, andrenovate-fleet.jsonpassesrenovate-config-validatorin pre-commit, which provably checks it: a deliberately broken preset fails the hook. - No managed repository's dependency dashboard reports "Can't find
version matching" for
shellcheck-py/shellcheck-py, and the repositories pinned tov0.10.0.1have been offered the bump. [SECURITY]pull requests still open outside the weekly window, measured in phase 4.- Phase 4 records the before and after numbers, and Renovate opened 100 or fewer pull requests across the organisation in its 30-day window, measured against a baseline taken with the same method (417 org-wide in the 30 days to 2026-11-11, by phase 1's survey item 2, which superseded the Situation's 227).
- The central runner runs at least as often as the per-repository copies did, and phase 4 records its daily run count and longest gap.
pre-commit run --all-filespasses.scripts/audit-check.pyrun against this repository reports no new failures, or the plan states which verdicts moved and why.- Any new or changed criterion has all five of its files in step:
the
Checksubclass inscripts/audit/checks/<family>.py, its registration inCHECKSinscripts/audit/registry.py, the specification underdocs/audits/, its row indocs/audits/README.md, and its frozen lines inFROZEN_METADATAandFROZEN_ISSUE_TITLESinscripts/tests/test_metadata.py-- plus aFROZEN_COLUMN_NAMESline where a spec page carries more than one check.AUDIT_METADATA,ISSUE_TITLESandCOLUMN_NAMESare derived from the registry rather than tables anybody edits; the issue title is the fleet-wide idempotency key, so renaming one orphans every open issue for that check across the fleet. - No
consistency-auditmarker block has been added to a criterion specification by hand, and the compliance tables indocs/audits/compliance.mdhave not been hand-edited. - Any entry added to
REPO_OVERRIDEScarries a stated reason. - Anything under
templates/is judged as the code it will become in ten other repositories: placeholders consistent, no reference to paths that only exist here, and the README beside it saying what to substitute. - Python is wrapped at 120 characters, single quotes for strings and double quotes for docstrings, and no script has grown a dependency outside the standard library.
- Documentation in
docs/describes any user-visible change.AGENTS.mdchanges only if a convention changed;ARCHITECTURE.mdonly if the shape of the system changed;README.mdonly if the pitch, the install story or the documentation links changed.
Documentation index maintenance¶
When creating a new master plan from this template, add one row to
the table in docs/plans/index.md: the date the plan was written,
a link to it, a one-line intent, and its status from the
vocabulary above. Rows run oldest first. One row per master plan,
never one per phase -- the phases are tracked in the plan's own
Execution table, and duplicating them in the index is how the two
drift apart.
There is no phase-arithmetic column and no order.yml here; both
belong to repositories whose documentation is published through a
generated navigation. The plan-index criterion checks the
columns this index actually has.
The index row carries the whole-plan status, so it only reaches
Complete once every phase has been completed, abandoned or
superseded. Update it as the plan progresses, not only at the end.
Plan close-out sections (shared block; do not edit -- the
canonical copy lives in shakenfist/development at
templates/shared-blocks/plan-closeout-sections.md):
Future work¶
We should list obvious extensions, known issues, unrelated bugs we encountered, and anything else we should one day do but have chosen to defer to here, so that we do not forget them.
- Shared workflows instead of copied templates. Renovate is the
extreme case of a wider problem: on 2026-10-03, the 17 live copies
of
codeql-analysis.ymlcame in 12 versions. Converting the other verbatim-copied templates into callers of shared workflows inactionsis phase 5 ofPLAN-workflow-ref-trust.md, because it rests on that plan's decision to trustactionsmain. - GitHub dropping top-of-hour scheduled events (see Situation) is
not specific to Renovate. Phase 2 moves the central runner off the
hour, but any other fleet workflow scheduled at minute
0is exposed to the same delays and drops, which thescheduled-workflow-healthcriterion would want to know. - Digest pinning moves github-actions out of Dependabot's reach.
Step 1a found that GitHub raises Dependabot alerts for actions
referenced by semantic version but not by commit sha, so
PLAN-workflow-ref-trust.md's step 3b takes the fleet's 540 third-partyuses:references out of the one ecosystem that feedsvulnerabilityAlerts. Nothing in this plan changes because of it --- an action fix then waits for the monthly CI-tooling window, which is the worst case the Mission already accepts for docker and pre-commit --- but that plan should record the trade as its own, since it is the change that causes it. - A criterion requiring the
shellcheck-py/shellcheck-pyversioning rule, rather than merely permitting it, is recorded against an unwrittenPLAN-renovate-shellcheck-audit. If it is written after phase 1 it must accept the rule reached throughlocal>shakenfist/development:renovate-fleetas well as a local copy, for the same reason phase 1's decision 1 gives: a criterion that reads only the local file holds the fleet to a line the policy has moved out of. - Promoting lockstep groups into the preset, which is what
development#88
ultimately asks for. A group that appears in more than one
repository could live in
renovate-fleet.json, butrenovate-lockstep-groupswould first have to resolve anextendsof the preset as well as reading the local file, with tests showing it accepts a group defined there; until then it reports a promoted group as missing.
Bugs fixed during this work¶
This section should list any bugs we encounter during development that we fixed. You should also scan the project's issue tracker, where one exists, for directly related issues that we should either resolve as part of this master plan or at least be aware of while planning it.
- Related, and partly addressed here: development#88 asks for one global set of Renovate groupings instead of rediscovering each one through CI failures in each repository. The phase 1 preset carries the groupings that are fleet policy (the CI-tooling group, the shellcheck-py versioning rule). The lockstep groups stay local in this plan, for the reason phase 3 gives, and their promotion is under Future work, so #88 stays open.
templates/renovate/README.mdanddocs/audits/renovate.mddescribedrenovate.ymlas copied verbatim while 17 copies held 14 versions. Fixed outside this plan, and in the other direction:d2298b0(2026-10-31) made the claim true by auditing every copy against the template, so phase 1 has nothing to correct and phase 3 removes the file and rewrites the spec around the preset. Phase 1's survey, item 4, has the detail.- Renovate cannot look up
shellcheck-py/shellcheck-py's four-component tags, so the hook is never bumped (found fromhunkydory#34, the dependency dashboard, on 2026-10-09). No issue tracks it. Fixed for the fleet bydc2f6ed(#275, 2026-10-18), which put the versioning rule intemplates/renovate/renovate.jsonand in this repository's own config; phase 1 moves that rule into the preset, and phase 3 removeskerbside's local copy, which predates it.
Back brief¶
Before executing any step of this plan, please back brief the operator as to your understanding of the plan and how the work you intend to do aligns with that plan.