Phase 9: Documentation sweep¶
Context¶
Phases 1–8 each ended with a focused documentation update
covering the change shipped in that phase. As a result, the
project-level documents (ARCHITECTURE.md, AGENTS.md,
docs/developer_guide/network_dispatcher.md) are already in good
shape — they were never allowed to drift more than one phase out
of date. Phase 9 is therefore not the rewrite the master plan
implied; it is a focused finalisation pass.
The scope of Phase 9 is the work that genuinely was deferred:
- The REST API reference docs under
docs/developer_guide/api_reference/were only partially updated. Phase 7 rewroteclusteroperations.mdfrom scratch, butnetworks.mdstill carries adelete_networkexample that predates the 202+poll contract — the example manually pollsget_networkin awhileloop, which is exactly the code the new client library wraps. That example needs updating. ARCHITECTURE.mdandAGENTS.mdhave accumulated a sequence of "Phase N —" subsections (Phase 7, Phase 8 in ARCHITECTURE.md; Phases 6, 7, 8 in AGENTS.md). The phase-narrative format was useful while phases were landing one-at-a-time; in the merged final state it produces a document organised around the order work happened in rather than the architecture itself. A light consolidation pass folds these into final-state descriptions while keeping the rationale intact.</li> <li>The master plan and the eight sub-phase plans need a final cross-check: any "future work" items that shipped during the project should be removed; any references to the <code>/cluster_operations/</code> URL prefix (renamed during Phase 7 review) should be <code>/clusteroperations/</code>; any references to "per-method migration flags" (documented during Phase 8 as inaccurate) should be removed.</li> </ul> <p>A handful of other docs were inspected during Phase 9 planning and confirmed to need no changes:</p> <ul> <li><code>README.md</code> — no network-facade-relevant content; nothing to update.</li> <li><code>docs/developer_guide/state_machine.md</code> — the Network state values (<code>initial</code>, <code>created</code>, <code>delete-wait</code>, <code>deleted</code>, <code>error</code>) and their transitions are unchanged by the facade refactor. The doc is about <em>which</em> states exist, not <em>how</em> transitions are driven.</li> <li><code>docs/operator_guide/networking/overview.md</code> — describes the resulting host network configuration (interfaces, bridges, VXLAN topology) rather than the orchestration mechanism. No facade-specific updates needed.</li> <li><code>docs/developer_guide/network_dispatcher.md</code> — already comprehensive. The chronological "Phase 6", "Phase 7", "Phase 8" sections in this doc are intentional and stay: this is the deep-dive technical narrative for contributors who want to understand how the architecture came to be, and the chronology is the point. Final-state summaries live in <code>ARCHITECTURE.md</code> instead.</li> <li>Existing tests are already consistent with the post-Phase-8 state. Phase 8a already converted the lock-related assertions; Phase 6c added explicit <code>InvalidStateForTask</code> tests for the retired handlers. No test-cleanup pass needed.</li> </ul> <h2 id="what-phase-9-ships">What Phase 9 ships<a class="headerlink" href="#what-phase-9-ships" title="Permanent link">¶</a></h2> <p><strong>1. API reference doc updates.</strong></p> <ul> <li><code>docs/developer_guide/api_reference/networks.md</code> — the "Python API client: delete a network" example block (around lines 59–94 of the current file) currently shows:</li> </ul> <div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="n">n</span> <span class="o">=</span> <span class="n">sf_client</span><span class="o">.</span><span class="n">delete_network</span><span class="p">(</span><span class="n">n</span><span class="p">[</span><span class="s1">'uuid'</span><span class="p">])</span> <a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="k">while</span> <span class="n">n</span><span class="p">[</span><span class="s1">'state'</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">'deleted'</span><span class="p">:</span> <a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a> <span class="nb">print</span><span class="p">(</span><span class="s1">'Waiting...'</span><span class="p">)</span> <a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a> <span class="n">n</span> <span class="o">=</span> <span class="n">sf_client</span><span class="o">.</span><span class="n">get_network</span><span class="p">(</span><span class="n">n</span><span class="p">[</span><span class="s1">'uuid'</span><span class="p">])</span> </code></pre></div> <p>Replace with an example that reflects the new client API: <code>delete_network</code> with <code>wait=True</code> (the default) returns the cluster operation's final view once it reaches a terminal state. The <code>while</code> loop disappears. Add a brief paragraph explaining that <code>wait=False</code> returns the op handle if the caller wants to do something else while the deletion happens. Reference the <code>docs/developer_guide/api_reference/clusteroperations.md</code> page for the polling contract.</p> <ul> <li> <p><code>docs/developer_guide/api_reference/networks.md</code> — the "REST API calls" entry list near the top mentions <code>DELETE /networks</code> and <code>DELETE /networks/{network_ref}</code> but doesn't note that they return HTTP 202. Add a brief parenthetical to each entry stating the response shape.</p> </li> <li> <p>Sweep the rest of <code>docs/developer_guide/api_reference/</code> for any other stale references to the old synchronous-delete contract. The phase 7 rewrite of <code>clusteroperations.md</code> is current and should not need further edits; if anything is found, fix in place.</p> </li> </ul> <p><strong>2. ARCHITECTURE.md consolidation.</strong></p> <p>The "Network Operation Error Handling" section currently has ten subsections, two of which (<code>#### Phase 7 — REST contract</code> and <code>#### Phase 8 — NodeLock removal</code>) are phase-narrative holdovers. Fold their content into the surrounding final-state descriptions:</p> <ul> <li>The Phase 7 subsection covers the 202+poll contract, the cluster-operation discovery endpoints, and the <code>redirect_to_network_node</code> removal. Most of this belongs alongside the existing <code>### Network Operation Queue Families</code> section as a "REST API surface" subsection describing the final shape.</li> <li>The Phase 8 subsection is a single paragraph noting the NodeLock removal. Fold into the <code>#### BridgedVXLanNetwork — worker-only mutation surface</code> subsection, where it belongs architecturally — the worker-only mutation surface is the <em>reason</em> the NodeLocks could be removed.</li> </ul> <p>After the consolidation, neither subsection retains a "Phase N — title" header. The information is preserved; the phase-narrative chrome is gone.</p> <p><strong>3. AGENTS.md consolidation.</strong></p> <p>AGENTS.md currently has phase-narrative subsections for Phase 6 (maintain), Phase 7 (REST contract), and Phase 8 (NodeLock removal). The doc's purpose is to brief a fresh agent on the codebase as it stands now — phase numbers are noise in that context. Replace the three subsections with a single "Network facade architecture" subsection summarising the final state in three or four short paragraphs:</p> <ul> <li>The <code>BridgedVXLanNetwork</code> worker class is the only place that mutates host network state. The single-threaded net-worker dispatcher (<code>shakenfist/daemons/network/workitem.py</code>) is the only caller. Mention the cancellation check on dequeue and the exponential back-off map; point at the single-worker safety comment in workitem.py.</li> <li><code>Network</code> methods enqueue cluster operations rather than mutating state directly. Maintain (<code>shakenfist/daemons/network/maintain.py</code>) is discovery-only with a five-guard pipeline.</li> <li>The REST API surface — the two delete endpoints return 202 with an op handle; the two discovery endpoints (<code>/clusteroperations/<uuid>/chain</code> and <code>/clusteroperations?target_*=</code>) are available; the only surviving <code>@redirect_to_network_node</code> is on <code>NetworkPingEndpoint.get</code>.</li> <li>Error handling: <code>ErrorReport</code> is the on-the-wire shape; errors are data, not rehydrated exception types.</li> </ul> <p>Keep the section concise — AGENTS.md is supposed to brief in minutes, not hours.</p> <p><strong>4. Master plan and sub-phase plan audit.</strong></p> <ul> <li><code>docs/plans/PLAN-network-facade.md</code> — audit the body (especially the "Mission", "Design decisions", and "Open questions" sections) for any remaining "TODO" / "future work" / "Phase N will..." items that have shipped. Remove the obsolete ones, leave the genuinely deferred ones (e.g. the queue-based ping endpoint, the <code>redirect_instance_request</code> / <code>redirect_to_eventlog_node</code> decorator migrations) with a note that they remain deferred beyond the master plan.</li> <li>The eight sub-phase plans (<code>PLAN-network-facade-phase-0[1-8]-*.md</code>) stay as historical record. No edits needed unless a cross-reference between them is broken.</li> <li><code>docs/plans/PLAN-recurring-operations.md</code> — confirm it still reflects the right scope (was set aside during Phase 1 planning as a separate future plan).</li> </ul> <p><strong>5. Status flip.</strong></p> <p>After steps 1–4 land and CI is green, flip Phase 9 in the master plan execution table to <code>Complete</code>. With Phase 9 complete, the entire network-facade master plan is done.</p> <h2 id="what-phase-9-does-not-do">What Phase 9 does <strong>not</strong> do<a class="headerlink" href="#what-phase-9-does-not-do" title="Permanent link">¶</a></h2> <ul> <li><strong>No structural changes to source files.</strong> Phase 9 is docs only. The single test-file touched in Phase 8a already converted lock-related assertions; no further test cleanup is needed.</li> <li><strong>No README.md changes.</strong> The README has no network-facade-relevant content. If we wanted a one-line pointer to the dispatcher dev guide, it could go here, but that's a different conversation.</li> <li><strong>No state-machine doc changes.</strong> Network states haven't changed.</li> <li><strong>No operator-guide changes.</strong> The operator's Shaken-Fist-deployment view of networking is unchanged.</li> <li><strong>No new doc files.</strong> All Phase 9 edits are to existing files.</li> <li><strong>No <code>mkdocs.yml.tmpl</code> changes.</strong> No new pages, no reorganisation.</li> <li><strong>No restructuring of <code>network_dispatcher.md</code>.</strong> Its chronological phase-by-phase narrative is the right format for that doc.</li> </ul> <h2 id="key-references">Key references<a class="headerlink" href="#key-references" title="Permanent link">¶</a></h2> <ul> <li><code>docs/developer_guide/api_reference/networks.md</code> — the file with the most concretely-broken example.</li> <li><code>docs/developer_guide/api_reference/clusteroperations.md</code> — reference target for the new polling contract.</li> <li><code>ARCHITECTURE.md</code> — sections to consolidate at lines 320 (Phase 7 subsection) and 410 (Phase 8 subsection).</li> <li><code>AGENTS.md</code> — three Phase subsections to consolidate; check the existing structure first to land the new section in the right place.</li> </ul> <h2 id="success-criteria">Success criteria<a class="headerlink" href="#success-criteria" title="Permanent link">¶</a></h2> <ul> <li><code>docs/developer_guide/api_reference/networks.md</code> <code>delete_network</code> example uses the new client API; no manual polling loop.</li> <li><code>ARCHITECTURE.md</code> "Network Operation Error Handling" section has no "Phase N — title" subsection headers; information is preserved in final-state subsections.</li> <li><code>AGENTS.md</code> has a single "Network facade architecture" subsection rather than three phase-numbered subsections.</li> <li><code>docs/plans/PLAN-network-facade.md</code> execution table shows every phase as <code>Complete</code>.</li> <li><code>grep -rn '/cluster_operations/' docs/ shakenfist/ ARCHITECTURE.md AGENTS.md</code> returns zero hits (no underscored URL references remaining).</li> <li><code>pre-commit run --all-files</code> passes.</li> <li><code>mkdocs build</code> does not warn about broken links.</li> </ul> <h2 id="step-level-guidance">Step-level guidance<a class="headerlink" href="#step-level-guidance" title="Permanent link">¶</a></h2> <table> <thead> <tr> <th>Step</th> <th>Effort</th> <th>Model</th> <th>Isolation</th> <th>Brief for sub-agent</th> </tr> </thead> <tbody> <tr> <td>9a. API reference doc updates</td> <td>low</td> <td>sonnet</td> <td>none</td> <td>Edit <code>docs/developer_guide/api_reference/networks.md</code>. Replace the "Python API client: delete a network" example block (currently shows manual <code>while n['state'] != 'deleted'</code> polling against <code>get_network</code>) with an example using the new client API: <code>n = sf_client.delete_network(uuid)</code> returns the op view once terminal; no polling loop needed. Add a brief paragraph after the example noting that <code>wait=False</code> returns the raw op handle if the caller wants fire-and-forget semantics, and pointing readers at <code>docs/developer_guide/api_reference/clusteroperations.md</code> for the polling contract details. Also: the "REST API calls" entry list near the top of the file lists <code>DELETE /networks</code> and <code>DELETE /networks/{network_ref}</code> — add a short parenthetical to each entry indicating they return HTTP 202 with an op-handle response body. Sweep the rest of <code>docs/developer_guide/api_reference/</code> (<code>agentoperations.md</code>, <code>instances.md</code>, <code>artifacts.md</code>, <code>interfaces.md</code>, etc.) for any remaining stale references to a synchronous-network-delete contract or to <code>/cluster_operations/<...></code> URLs (the underscored form). If found, fix in place. Run <code>pre-commit run --all-files</code> (which includes the doc-related linters); fix any issues. Commit message subject: <code>docs: api_reference updates for phase 9.</code> The commit body should explain the example replacement, note that <code>clusteroperations.md</code> was already updated in Phase 7 so this commit completes the API reference sweep, and reference the master plan / phase 9 plan.</td> </tr> <tr> <td>9b. ARCHITECTURE.md and AGENTS.md consolidation</td> <td>medium</td> <td>sonnet</td> <td>none</td> <td>Edit <code>ARCHITECTURE.md</code>'s "Network Operation Error Handling" section (starts around line 228). It currently contains ten subsections, two of which (<code>#### Phase 7 — REST contract</code> around line 320, and <code>#### Phase 8 — NodeLock removal</code> around line 410) are phase-narrative holdovers. <strong>Phase 7 content</strong> (the 202+poll delete contract, the two discovery endpoints, the <code>redirect_to_network_node</code> removal) should be folded into a new subsection after the existing <code>### Network Operation Queue Families</code> section, titled something like <code>### REST API surface</code> or <code>### External API contract</code>, describing the final shape (not "what Phase 7 shipped"). <strong>Phase 8 content</strong> (single paragraph noting the NodeLock removal) belongs in the existing <code>#### BridgedVXLanNetwork — worker-only mutation surface</code> subsection — fold it in as a closing paragraph explaining <em>why</em> the worker-only mutation surface lets cross-daemon serialisation be queue-based rather than lock-based. After folding, neither "Phase 7" nor "Phase 8" subsection header should remain; the information is preserved without phase-narrative chrome. Edit <code>AGENTS.md</code>: it currently has three phase-numbered subsections (Phase 6 maintain, Phase 7 REST contract, Phase 8 NodeLock removal). Replace them with a single "Network facade architecture" subsection summarising the final state in three or four short paragraphs (see the master plan's "AGENTS.md consolidation" body for the structure). Keep the section concise — AGENTS.md is for fresh-agent orientation in minutes, not deep technical history (that lives in <code>network_dispatcher.md</code>). For both files, <strong>preserve information</strong>, don't lose any architectural detail in the consolidation. Read each subsection thoroughly before consolidating. Run <code>pre-commit run --all-files</code>; confirm <code>mkdocs build</code> does not warn. Commit message subject: <code>docs: consolidate phase narrative into final-state.</code> Body should list each file touched and what was folded into what, and note that the deep-dive chronological narrative still lives in <code>docs/developer_guide/network_dispatcher.md</code>.</td> </tr> <tr> <td>9c. Master plan audit</td> <td>low</td> <td>sonnet</td> <td>none</td> <td>Audit <code>docs/plans/PLAN-network-facade.md</code>. Read the "Mission", "Design decisions", "Open questions", and "Future work" sections (or whatever the equivalent named sections are) and identify any "TODO" / "future work" / "Phase N will..." / "deferred to Phase N" items. For each, decide whether it shipped during phases 1-8 (in which case remove the deferred-work marker) or whether it remains genuinely outside the master plan's scope (e.g. the queue-based ping endpoint, the <code>redirect_instance_request</code> / <code>redirect_to_eventlog_node</code> decorator migrations — these are real future work, not phase 9 cleanups). For items that shipped, fold the description into past-tense final-state language. Also: <code>grep -rn '/cluster_operations/' docs/ shakenfist/ ARCHITECTURE.md AGENTS.md</code> should return zero hits after this step (the URL was renamed during Phase 7 review; any remaining references in the master plan or sub-phase plans are stale). Skim the eight sub-phase plans (<code>PLAN-network-facade-phase-0[1-8]-*.md</code>) for broken cross-references between them, but <strong>do not</strong> edit the sub-phase plans' content — those stay as historical record. Skim <code>docs/plans/PLAN-recurring-operations.md</code> and confirm its scope reflects what was deferred during Phase 1 planning; no edits expected. Run <code>pre-commit run --all-files</code>. Commit message subject: <code>plans: phase 9 master plan finalisation.</code> Body should list each removed-as-shipped item, list each retained-as-deferred item, and confirm the grep result.</td> </tr> <tr> <td>9d. Mark phase 9 complete</td> <td>low</td> <td>sonnet</td> <td>none</td> <td>Edit <code>docs/plans/PLAN-network-facade.md</code>. In the execution table, flip Phase 9's status from <code>Planning</code> to <code>Complete</code>. The row description <code>Documentation and tests</code> can stay as-is (the tests scope turned out to be already-handled, but the description is fine). Add a one-paragraph note at the top of the master plan or in a "Status" section noting that with Phase 9 complete, the entire network-facade master plan has landed. Use the same phrasing pattern as the earlier "plans: mark phase N of network facade complete" commits (<code>82d0af2f</code> for Phase 7, <code>90f6f3c9</code> for Phase 8). The commit body should reference the four phase-9 sub-commits, note that this is the final commit of the project, and (optionally) note what comes next (the ping-endpoint queue migration and the remaining redirect decorators are the natural follow-ons but are not part of this plan). Commit message subject: <code>plans: mark phase 9 of network facade complete.</code></td> </tr> </tbody> </table> <h2 id="step-ordering-and-dependencies">Step ordering and dependencies<a class="headerlink" href="#step-ordering-and-dependencies" title="Permanent link">¶</a></h2> <ul> <li>9a (API reference) is self-contained and lands first.</li> <li>9b (project doc consolidation) is independent of 9a; can land in either order. Either order is fine; 9b is the most substantive Phase 9 commit and benefits from going second so any 9a feedback can be folded in.</li> <li>9c (master plan audit) depends on 9a and 9b having landed because the audit cross-checks against the consolidated final-state.</li> <li>9d (status flip) lands last, after CI confirms 9a–9c.</li> </ul> <p>Recommended landing order: 9a → 9b → 9c → CI green → 9d.</p> <h2 id="back-brief">Back brief<a class="headerlink" href="#back-brief" title="Permanent link">¶</a></h2> <p>Before executing 9b, the implementing sub-agent must back brief the management session. Confirm:</p> <ul> <li> <p>The information preservation invariant. Consolidating phase-narrative subsections into final-state descriptions must not drop any architectural detail. Reading each subsection slowly before consolidating is the load-bearing step; if a paragraph's content doesn't have a natural home in the final-state structure, <strong>stop and report</strong> rather than silently dropping it.</p> </li> <li> <p>The audience distinction. ARCHITECTURE.md is the final-state reference; AGENTS.md is the fresh-agent orientation; <code>network_dispatcher.md</code> is the deep-dive chronological narrative. Each audience benefits from a different structure. Phase 9 leaves <code>network_dispatcher.md</code> alone — phase narrative is the <em>right</em> format for that file.</p> </li> <li> <p>The "Phase X subsection header" rule applies only to the phase-narrative subsections in ARCHITECTURE.md and AGENTS.md. Other phase references in those files (e.g. "Phase 6 retired the network_deploy handler" stated as historical fact) can stay — they're descriptive, not structural.</p> </li> </ul> <p>Before executing 9c, the implementing sub-agent must confirm:</p> <ul> <li> <p>The retained-as-deferred list. The queue-based ping endpoint and the two non-network redirect decorators (<code>redirect_instance_request</code>, <code>redirect_to_eventlog_node</code>) are explicitly out of scope for the network-facade master plan and remain genuine future work. <strong>Do not</strong> describe these as "shipped" or remove them from the master plan's deferred-work list.</p> </li> <li> <p>Sub-phase plans stay as-is. Only the master plan body and execution table are touched. The eight sub-phase plans are historical record of how the work was scoped and executed.</p> </li> </ul> <h2 id="review-checklist-for-the-management-session">Review checklist for the management session<a class="headerlink" href="#review-checklist-for-the-management-session" title="Permanent link">¶</a></h2> <p>After 9a: - [ ] <code>networks.md</code> <code>delete_network</code> example uses the new client API; no <code>while</code> polling loop. - [ ] <code>networks.md</code> REST entry list notes 202 on the two delete endpoints. - [ ] No other API reference page has a stale synchronous-delete reference. - [ ] <code>pre-commit run --all-files</code> passes.</p> <p>After 9b: - [ ] No <code>#### Phase N — title</code> subsections remain in <code>ARCHITECTURE.md</code>'s "Network Operation Error Handling" section. - [ ] <code>AGENTS.md</code> has a single "Network facade architecture" subsection rather than three phase-numbered ones. - [ ] Architectural detail from the consolidated subsections is preserved; agent's commit message lists what was folded into what. - [ ] <code>mkdocs build</code> does not warn.</p> <p>After 9c: - [ ] <code>grep -rn '/cluster_operations/' docs/ shakenfist/ ARCHITECTURE.md AGENTS.md</code> returns zero hits. - [ ] Master plan body has no "TODO" / "future work" markers for items that shipped during phases 1–8. - [ ] The retained-as-deferred items (ping endpoint, two non-network redirect decorators) remain documented as such. - [ ] Sub-phase plans untouched.</p> <p>After 9d: - [ ] Phase 9 status is <code>Complete</code> in the master plan execution table. - [ ] All nine rows now show <code>Complete</code>. - [ ] CI passes on the phase 9 PR.</p> <p class="md-content__report-issue"> <a href="https://github.com/shakenfist/shakenfist/issues/new?title=Docs%20issue%3A%20plans/PLAN-network-facade-phase-09-docs.md&body=%2A%2ASource%20file%3A%2A%2A%20%60plans/PLAN-network-facade-phase-09-docs.md%60%0A%2A%2APage%20URL%3A%2A%2A%20https%3A//shakenfist.com/plans/PLAN-network-facade-phase-09-docs/%0A%0A%2A%2ADescribe%20the%20issue%3A%2A%2A%0A&labels=documentation" target="_blank" rel="noopener"> 📝 Report an issue with this page </a> </p> </article> </div> <script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script> </div> </main> <footer class="md-footer"> <div class="md-footer-meta md-typeset"> <div class="md-footer-meta__inner md-grid"> <div class="md-copyright"> Made with <a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener"> Material for MkDocs </a> </div> </div> </div> </footer> </div> <div class="md-dialog" data-md-component="dialog"> <div class="md-dialog__inner md-typeset"></div> </div> <script id="__config" type="application/json">{"annotate": null, "base": "../..", "features": [], "search": "../../assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script> <script src="../../assets/javascripts/bundle.d7400e89.min.js"></script> </body> </html>