Job search
SRE Portfolio for Job Applications: Define an SLO and Explain One Reliability Incident
An actual loopback service experiment connects a complete request-based SLO with a traced failure, bounded recovery and separately verified configuration guard.
An SRE portfolio should show how you define a user-visible reliability objective, detect a failure, investigate its cause and verify an improvement. Pair the dashboard with the objective, observations and decisions behind it. Use a small service you own, define the denominator and reporting window, and preserve the incident evidence alongside the limits of the experiment.
This guide checks Google's primary SRE guidance on October 21, 2026. It also reports an original AI-authored local HTTP service that was actually run, faulted, recovered and stopped. All requests stayed on newly allocated loopback ports; no production service, cloud account, external load or customer traffic was involved. LandOffer publishes this guide and offers job-search tools; browse recent roles to select the reliability responsibility your case should demonstrate.

Start with the outcome the service must deliver
For a request-driven service, identify what a caller is trying to accomplish. In this lab, the caller wants to retrieve a usable status document promptly. The service reads a local JSON file and returns it through /status. That creates a dependency whose failure can affect the caller even while the HTTP process is still running.
Google's Implementing SLOs chapter recommends expressing useful indicators as good events divided by total events and distinguishes the desired outcome from its measurement implementation. Apply that distinction explicitly: a usable response within a threshold is the intended outcome; a client probe's status, parsed body and elapsed time are one way to observe it.
The probe does not represent every user journey. It cannot establish browser rendering, authentication behavior, global network reachability or real customer satisfaction. Here there are no customers at all. Use the lab to demonstrate measurement and investigation within that narrow scope.
Choose the objective before inspecting the result. A portfolio target is an engineering assumption to explain, not an industry-wide reliability requirement. If you change it later, preserve the original result and justify the new requirement rather than redefining success to hide a failed experiment.
Worked SLO: a complete objective and observed result
The executed lab used Python 3.12.14 and a standard-library HTTP server bound to 127.0.0.1, with an operating-system-assigned unused port. Its status document contained state: ready and revision: local-v1. The following record defines the first experiment's measurement completely:
| SLO field | Declared lab definition |
|---|---|
| User-visible task | Retrieve the expected usable status document |
| Eligible events | All 100 sequential client attempts to this lab's /status endpoint |
| Good event | HTTP 200, expected state and revision in the JSON body, and client elapsed time at most 100 ms |
| Bad event | Any eligible attempt not satisfying all good-event conditions, including transport failure |
| Window | One fixed 100-request experiment, not a calendar month or rolling production period |
| Target | At least 99% good events within that experiment |
| Budget | At most one bad event for this 100-request denominator |
| Decision on a miss | Preserve the miss, investigate the failure and verify the selected corrective change separately |
The probe made no automatic retries. A transport failure would have remained in the denominator rather than disappearing from the report. The service-side log was used for diagnosis, while the client observation determined whether a request was good. That avoids defining success solely from requests the server managed to log.
The observed sequence was 40 baseline requests, 20 fault-period requests and 40 recovery requests. Baseline and recovery each met all good-event conditions; every fault-period request returned HTTP 503. The original window therefore contains 80 good events and 20 bad events.
The arithmetic is explicit: SLI = 80 / 100 = 80%. Allowed bad events = 100 × (1 − 0.99) = 1. The experiment used 20 times that allowance and exceeded it by 19 events. That is budget consumption for this finite request window, not a time-based burn-rate calculation or a monthly availability result.
Recovery did not erase the original failures. The completed decision is that the first experiment missed its objective and warrants a corrective change. No statement about production availability, an SLA, customer impact or long-term compliance follows from these 100 local requests.
Inject one failure you can trace and reverse
The controlled fault changed the service's configured status-file path to a deliberately absent file inside its own new lab directory. The process continued to accept requests, but its file read raised FileNotFoundError. The handler returned HTTP 503 with status_dependency_missing instead of a usable document.
This is a configuration-dependent availability failure, not a load or capacity test. The missing file was intentional; no user file was removed, no existing server was modified and no remote dependency was targeted. The experiment changed a private in-process configuration value rather than exposing an administrative endpoint.
The saved server log connects the response to the configured filename and exception class. That is evidence for the local causal explanation: the unvalidated path change caused the read failure. It does not establish a production incident pattern, nor does it test every filesystem error, malformed document or concurrent configuration update.
Define recovery before running the fault. Here the recovery action was to restore the known-good path and make the remaining 40 requests. All 40 returned the expected usable response within the declared threshold. The maximum measured latency across the original 100 requests was approximately 9.16 ms; this is a local probe observation, not a performance benchmark or capacity claim.
Worked incident record: investigation, recovery and follow-through
Google's postmortem guidance explains why context, impact, triggers, recovery details and concrete action items matter. The portfolio should connect those elements rather than ending with “restarted the service.” This completed record describes the actual local exercise, not a fictional employer outage:
| Incident field | Recorded lab evidence and conclusion |
|---|---|
| Scope | One local status-reader process; no real users or external traffic |
| Trigger | After request 40, the controller assigned an absent status-file path |
| Detection | Requests 41–60 returned 503 and failed the client-side good-event rule |
| Investigation | Server events named the absent configured file and FileNotFoundError |
| Contributing design gap | Configuration activation accepted a path without checking that a usable document could be read |
| Recovery | Restore the known-good path before request 61; requests 61–100 were good |
| Corrective change | Validate the candidate file and expected payload before replacing the active path |
| Verification | A separate guarded service rejected the same absent path; its next 20 requests were good |
| Remaining limits | Later file removal, disk errors, malformed runtime content and concurrency were not comprehensively tested |
The request numbers show the experimental order. UTC timestamps and monotonic observations are also saved. Because one automated controller performs the sequence, this is not evidence of human detection time, on-call response time or organizational MTTR. A fast scripted rollback does not demonstrate an operational team's response speed.
The corrective action’s acceptance condition is specific: reject an unreadable candidate path before activation and leave the known-good path unchanged. The actual guarded setter reads and parses the candidate document, checks the expected fields, and only then assigns the path. A missing file therefore prevents activation rather than causing the next client request to discover it.
The regression used a newly started service and an independently allocated loopback port. The bad path raised an error before activation, and 20 subsequent probes satisfied the good-event definition. Those requests belong to a separate verification window. Combining them with the original requests would change the denominator and conceal the original experiment's declared scope.
The completed action is narrow: the demonstrated missing-path activation is blocked. The next useful test would remove or corrupt a file after a valid activation and observe the behavior. That test was not performed, so the portfolio must not claim that preflight validation eliminates every runtime dependency failure.

Explain what your alert would mean at this scale
The client probe observed failures, but this lab did not deploy a paging service, notify an on-call person or measure an alert's delivery. Keep that distinction in the case. A dashboard cell turning red is an observation; a tested notification and an actionable response procedure are additional work.
Google's Alerting on SLOs chapter discusses low-traffic services and synthetic traffic. Small denominators can make individual failures dominate a ratio, while synthetic requests cover only the behavior they exercise. This lab's intentionally concentrated fault is useful for demonstrating evidence and arithmetic, not estimating the distribution of failures in a real service.
For this portfolio, the chosen response is to stop expanding the demo, preserve the failed window and verify the configuration guard. That is an explicit experiment policy, not an organization's approved error-budget policy. A real service would need stakeholders, meaningful reporting periods and operational decisions appropriate to its users.
If you later implement alerting, specify which window and condition should notify, what the owner should inspect and how a recovery notification resets. Do not transplant a high-traffic production rule into a tiny service without explaining its behavior. Preserve the original SLI definition so that an alert implementation cannot silently redefine what counts as user success.
Give another engineer a complete route through the evidence
The useful artifact is a small reproducible case: source, controlled input, SLO definition, raw request observations, server events, ordered incident timeline, the corrective change and regression results. A static narrative should summarize the findings so that running the service is optional for understanding them.
This lab saved all 100 original request records, the separate 20-request regression, 120 server events and the timeline. The saved script passed ten assertions, including the expected phase outcomes, guarded rejection, unchanged input status-document SHA-256 hash and cleanup. Both server threads stopped, both sockets closed, and connection checks confirmed that the allocated ports were no longer listening.
The saved JSON and code remain as local evidence; no cloud resources or public endpoint were created. A reviewer can distinguish the original miss from the later regression pass without interpreting an unlabeled screenshot. Keep unsuccessful runs and corrections available when they explain the engineering decision, rather than presenting only the green result.
For your own project, attribute your role accurately. Someone may have written the service while you designed the SLI and investigation; that can still be meaningful work when clearly described. This article's code is AI-authored teaching material and does not establish a candidate's personal authorship, production tenure or ability to operate an unfamiliar system.
Turn the case into a defensible application claim
A useful project description names the service, objective, failure mode, observed impact and verified correction. Someone who genuinely performed their own equivalent work could write: “Defined a request-based SLO for a local status service, traced an injected missing-file failure through client probes and server logs, and verified a configuration-validation guard in a separate regression window.”
Add numbers only when your retained records support them, and label the environment. “80% across a controlled 100-request experiment” is understandable. “Improved company uptime to 100%” would exceed this evidence: there was no company service, and the successful regression did not repair the original reporting window.
As you compare recent roles on LandOffer, compare each role’s SLI design, investigation, recovery, validation and operational responsibilities with evidence you can explain. Show the failed objective as well as the improvement, and retain the boundary between an instrumented local lab and production reliability.



