1. LandOffer
  2. Blog
  3. All articles
  4. Write a Runbook Someone Else Can Follow Under Pressure

Onboarding & Teamwork

Write a Runbook Someone Else Can Follow Under Pressure

Completed synthetic RB-06 and a fictional second-reader correction record distinguish missing prerequisites, restore verification, rollback to prior stale state and untested branches; actual offline fixture evidence is separate.

9 min readLandOffer team

A runbook someone else can use under pressure needs a clear trigger, prerequisites, authorized scope, ordered steps, expected observations, stop branches, and a way to check the result. Put continuation, stop and contact instructions beside the steps so the reader can find them without searching elsewhere. Then have another reader walk through it: the author's familiarity can conceal the very prerequisite a responder needs most.

The procedure should show the evidence needed at each checkpoint, including when the reader must stop. This LandOffer.ai guide presents a fictional second-reader walkthrough alongside an actually executed local JSON fixture. The walkthrough people and incident context are invented. The local checks touched only disposable article files; no service, production system, customer information or operational account was accessed.

An open field manual rests beside a brass Check, Act and Stop signal, a key and a closed toolbox. Conceptual editorial illustration.

Define the trigger and the authority before the steps

AWS Well-Architected's runbook guidance calls for a specific outcome, tools, permissions, error handling, ownership and escalation. It also recommends validation by someone other than the author and updating the procedure when the process changes. Use that as a writing method, not as permission to act on any workload.

A trigger identifies the symptom and evidence that make this procedure applicable. A more useful trigger names the observed symptom, evidence needed and relevant exclusion. If the observation does not match, route the reader to investigation or the appropriate owner instead of encouraging a familiar action anyway.

The completed example is RB-06, version 2, owned by fictional maintainer Sana. It restores a deliberately stale local demonstration record from an approved synthetic snapshot. It is a writing exercise, not a recovery recipe for a real service. The desired result is byte-for-byte equality with the approved example and the expected three-item state.

Its trigger is the exercise's inspect result showing matches_approved: false for fixture rb-demo-01. Exclusions include an unknown fixture identity, missing approval evidence, an unexpected snapshot, an existing backup from an earlier attempt, or any need to touch a real environment. Those situations stop the procedure rather than widen it.

Microsoft's incident-response guidance separates roles, procedures and communication responsibilities. That distinction matters even for a well-written runbook: having instructions does not give the reader change authority. A real operational procedure must identify who authorizes the action and who owns escalation under the organization's process.

Fill in the prerequisites that memory would hide

RB-06's named files belong to the article's local synthetic fixture, not a utility installed on readers' systems. Its control script operates only beside its own file and has no network or service integration. The commands below describe that bounded package; they are not instructions to substitute a production path.

The package contains control.py, manifest.json, approved.json and current.json. The manifest supplies the approved snapshot's SHA-256, and each JSON record identifies rb-demo-01. The approved record has state ready and items alpha, beta, gamma; the starting current record is stale with only alpha. These invented values make the outcome checkable without private data.

The filled prerequisite section states:

Requirement RB-06 entry and stop condition
Scope and access A disposable local exercise directory, with permission to read the four package files and write its own backup and current record; no elevated privileges or external account
Runtime An approved Python 3 interpreter, called python3 in the example commands; if unavailable, stop and request the exercise owner's setup help
Approval evidence Manifest and approved snapshot from the same fixture package; the script checks identity, snapshot hash and expected approved contents before any action
Initial state current.json identifies the same fixture; inspect it before selecting restore
Backup state before.json must not already exist when beginning restore; do not delete an unexplained backup to bypass that guard
Escalation route In the fictional exercise, record the stop reason and ask Sana in the designated exercise review thread; no public incident message or real page is authorized

A filename alone is not approval evidence. If the manifest is missing, do not reconstruct it by hashing whatever snapshot happens to be present and calling that approved. Retrieve the matching package through the exercise owner. For a real system, the trusted source and authorization route would need to be documented and independently appropriate to that environment.

Write an action, an observation and a branch together

The complete version 2 procedure below keeps each action beside its expected result. All command names refer to the small offline control script. Its preflight runs before each operation and rejects missing package records, an incorrect identity, a snapshot hash mismatch or unexpected approved contents.

Step Action within the synthetic package Expected observation and branch
1. Confirm entry conditions Check the package version, scope, four required files, writable exercise directory and absence of before.json If any prerequisite is unknown or a backup already exists, stop; preserve the directory and ask Sana
2. Inspect Run python3 control.py inspect Expect ok: true, state stale, items ['alpha'], and matches_approved: false; if already matching, no restore is needed; if unexpected, stop
3. Restore once Run python3 control.py restore Expect ok: true and restored: true; the script saves prior bytes in before.json before replacing current bytes; on failure stop rather than repeat blindly
4. Verify Run python3 control.py inspect again Expect state ready, the three approved items and matches_approved: true; a restore acknowledgement by itself is insufficient
5. Record the result Preserve the outputs, package identity and before/after evidence In the exercise, report only that the local record matches the approved snapshot; do not infer service availability or future prevention
6. Stop or use the defined rollback If verification is unexpected, retain evidence and escalate; rollback requires the exercise owner's confirmation and the saved backup Never improvise a restart, edit the approved snapshot or widen access to make the procedure appear successful

A stop is an expected outcome when the conditions do not fit. Keep the original error and state, including the step that produced it. An instruction to “retry until it works” would erase the distinction between a recoverable interruption and a different problem.

The rollback branch is specific: after confirmation in this exercise, python3 control.py rollback copies before.json back into current.json, then the reader inspects again. It returns the saved prior bytes. Because the starting example is stale, this rollback does not produce the approved healthy state. Record the restored prior state and keep the issue open for the owner.

Do not translate “rollback” into universal reassurance. A real procedure would need its own supported reversal conditions, expected observations and escalation when reversal fails. This guide supplies no generic restart or restoration command for a real system.

Show what the local checks actually established

For this article, the author executed the fixture locally and saved the outputs. The test sequence first omitted the manifest in a separate temporary exercise copy. A restore attempt stopped with missing manifest.json; byte comparison confirmed that the current file was unchanged and no backup was created.

With the complete package, inspection showed the intended stale record. Restore succeeded, and the next inspection showed ready, all three approved items and exact snapshot equality. A later demonstration of rollback returned the starting stale bytes, which were compared with the original. The final inspection remained nonmatching, as expected for that rollback demonstration.

Actual local observation Supported conclusion Limit
Missing manifest stops restore without writes This tested prerequisite failure is handled before mutation Other missing or invalid inputs were not all executed
Restore followed by matching inspection This fixture's current record became the approved snapshot No live service, concurrent writer, filesystem failure or permission problem was tested
Rollback bytes equal starting bytes The saved prior record was restored in this local demonstration Prior state is stale; this is not a successful service recovery

The script also contains branches for a wrong fixture identity, a snapshot hash mismatch, an existing backup and a missing rollback backup. Those branches were not executed in this check sequence. Retain that validation gap instead of marking every documented stop branch tested. Malformed-input and permission-failure behavior likewise remain outside the observed evidence.

A second-reader record sits beside an incomplete prerequisite checklist, a revised instruction sheet and a still-open review marker. Conceptual editorial illustration.

Complete a second-reader walkthrough and its correction

The following walkthrough is fictional. It illustrates how another reader might expose an author assumption; it is not a claim that a colleague actually operated the local fixture. The missing-manifest behavior and the corrected-package recheck described above are the author's actual offline checks.

Sana's version 1 runbook listed the script and snapshot, but omitted the approval manifest from prerequisites. Second reader Pavel follows only the written procedure and stops at preflight. He records the missing approval source and stops before creating or changing any package file. His review note says: “The package list does not tell me where the approval hash comes from. I cannot establish that this snapshot is the authorized one.”

Sana accepts the finding and completes the revision:

Finding in the simulated walkthrough Completed version 2 correction Recheck and remaining limit
Approval manifest absent from the package list Name manifest.json, require the matching approved package and specify identity/hash checks Actual missing-manifest attempt stops unchanged; actual complete package proceeds
Version 1 says only “restore and confirm success” Separate the restore acknowledgement from a second inspection with exact expected state and items Actual post-restore inspection matches; other environments are untested
“Rollback if needed” leaves the outcome ambiguous Specify saved-backup restoration, owner confirmation and reinspection; explain the prior state can remain unhealthy Actual demonstration returns prior stale bytes; no claim of restored service

In the simulated second pass, Pavel can identify the required package, the expected post-restore observation and the stop route without an unwritten explanation. He still cannot establish behavior under concurrent changes or permission failure. The fictional review record therefore accepts version 2 as an improved exercise procedure while retaining those gaps.

Sana accepts responsibility for adding the remaining branch checks to a later review. Team lead Mina allocates two hours in the next documentation slot, replacing optional diagram polishing; Sana accepts that time. The later identity, hash, backup and permission checks remain open; the simulated walkthrough grants no production-use approval.

Maintain the procedure with its evidence and owner

Google's SRE workbook discussion of playbook maintenance notes that operational instructions change with their systems and recommends agreeing on minimum structured details. Its examples distinguish general guidance from deterministic procedures that may suit automation. The maintenance lesson applies here without copying Google's alerting rules or promising reduced recovery time.

Keep the owner, last review, package or system version, verified branches and known gaps with the runbook. Revisit it after a relevant environment change or a walkthrough reveals an assumption. Someone should be able to see whether the procedure still matches the system before using it under pressure.

For your next runbook, complete one bounded outcome with its prerequisites, observations and stop branches, then ask a second reader to follow the written record. Record the exact step, missing prerequisite and preserved state, then revise that section and recheck the affected path. Return to LandOffer.ai for more workplace guides; the result to aim for is a versioned procedure with honest validation limits and a named continuation owner.

Sources and Further Reading