Job search
MLOps Portfolio: Show Model Promotion, Monitoring, and Rollback Evidence
Demonstrate actual model loading, refused promotion, delayed-label batch monitoring, and rollback in an original persistent local inference service.
An MLOps portfolio should show which model is actually serving a request, who is allowed to change that version, what monitoring can establish, and whether a rollback restores the expected response. A registry label or a green notebook cell cannot answer all four questions. Preserve the commands, loaded-version evidence, predictions, and decision limits together.
This guide actually ran an original persistent Python CLI inference service with synthetic data on October 23, 2026. It demonstrated a refused promotion, an explicit model reload, two-phase batch monitoring, and rollback. It used no cloud account, external endpoint, MLflow server, Evidently installation, or real customer information. The models and approval records are authored teaching fixtures, not production credentials or trained-model achievements.

Use LandOffer’s recently posted jobs to identify model-serving, release-control, or monitoring responsibilities you want the sample to demonstrate. Select a small operating behavior you can run and explain honestly.
Define version authority separately from model quality
The current MLflow registry overview describes model versions, mutable aliases, tags, and lineage. These organize a release workflow. They do not make a tag an independent approval or prove that a running process has loaded the newly selected artifact. Our example implements its own small local mechanism rather than calling MLflow APIs or older stage-transition methods.
There are three original JSON model artifacts. Each contains a version and a manually defined threshold: v1 predicts class 1 for inputs at least 0.5; v2 uses 0.9; v3 uses 0.3. Otherwise the prediction is class 0. These simple threshold classifiers are not fitted estimators, calibrated probabilities, or useful real-world predictors. Their purpose is to make a serving change easy to inspect.
A separate local approval file contains v1 and v2, each with a fictional owner, approval identifier, and exact artifact hash. It has no entry for v3. Before selecting or loading a version, the service requires an approval entry whose recorded artifact hash matches the file it is about to use. This checks the local release rule; it does not authenticate the person who wrote the approval file. The owner label is not authenticated, and the same project author controls the file, models, and test runner.
The approval means “permitted for this bounded local exercise.” It does not mean “safe for production” or “quality validated.” V2 is deliberately admitted so the demonstration can expose its bad behavior and recover from it. A real release process would need an authorized decision-maker, protected approval storage, appropriate evaluation gates, and accountable operating owners.
Worked asset one: prove the running service changed
The service is a long-lived subprocess that receives JSON lines through standard input and returns JSON responses. It caches the loaded model in memory. It opens no network port. Every response records the process ID, selected alias, loaded version, model hash, and load generation, so a pointer change can be distinguished from an actual reload.
Startup loads v1 as generation 1. The fixed diagnostic request is same-probe with input 0.7. Its prediction is 1. The test then asks to select v3, which lacks a matching approval. The command is refused; the alias and loaded version remain v1, and the same probe still returns 1.
The next request selects approved v2. The alias changes to v2, but the service continues holding v1 in memory. Repeating the probe still returns 1 with loaded version v1 and generation 1. That is intentional evidence that changing the pointer alone did not reload this service.
The reload command reads the selected JSON artifact, checks its approval hash and declared version, replaces the in-memory model, and increments the load generation. Prediction requests then use that cached object rather than resolving the alias again. Now the probe returns 0 with loaded version v2 and generation 2. This is an observed response from the same process, not an inferred deployment result based on a filename.
The completed transition record is:
| Recorded step | Selected alias | Actually loaded | Same input 0.7 |
|---|---|---|---|
| Startup | v1 | v1, generation 1 | 1 |
| Unapproved v3 refused | v1 | v1, generation 1 | 1 |
| Approved v2 selected only | v2 | v1, generation 1 | 1 |
| Explicit v2 reload | v2 | v2, generation 2 | 0 |
| Rollback selects v1 only | v1 | v2, generation 2 | 0 |
| Explicit rollback reload | v1 | v1, generation 3 | 1 |
The last two rows occurred after the monitoring decision described below. They preserve the same distinction during recovery: selecting the old version did not itself restore the response. The explicit reload and repeated request supplied the response evidence. The later eight-row replay checks more than the single diagnostic input, while still remaining a finite functional test.
The packet includes service.py, run_demo.py, model artifacts, approval records, monitoring inputs, and raw request-response traces. From its demo directory, repeat the exercise with:
python3 run_demo.py > replay.json
Two fresh runs produced matching states, predictions, and monitoring values after excluding their different real process IDs. Within each run, all responses came from one process. Both service processes stopped normally with exit code zero. The saved trace includes the refused command and the responses before and after each reload, so the recovery conclusion does not depend on a manually filled status table. This is a local functional replay, not a latency, concurrency, availability, or production-reliability benchmark.
Design monitoring that can admit what it does not know
The independent monitoring asset uses two distinct eight-row datasets with separate A and B request identifiers. Neither includes the diagnostic probe. A reference dataset has mean input 0.5. Labels are stored separately from features and predictions. The runner first scores A0–A7 or B0–B7 and emits a proxy report with quality set to null; only the next evaluation step reads and joins that batch’s label file.
The Evidently batch-monitoring guide distinguishes independently executed evaluation runs from later quality checks when delayed labels arrive. Our demonstration follows that separation with authored standard-library code. It does not create Evidently Reports, upload data, configure a dashboard, or schedule recurring jobs.
The delay is simulated through ordered ingestion: predictions and the proxy report are produced first; a separate step then reads the authored label file. No real-time wait, external labeling service, or newly collected real-world outcome is claimed. The resulting quality metrics are actual calculations against those synthetic targets.
For this exercise, the proxy alarm is an absolute difference in feature means greater than 0.15. Quality is evaluated after all eight request IDs have matching labels, with minimum count eight and coverage 100 percent. The authored quality alarm is accuracy below 0.9. These thresholds were chosen for a small teaching case; they are not statistical significance tests or recommended production settings.
Worked asset two: two batches that require different decisions
Batch A contains inputs 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, and 0.95. The actual service scores these under loaded v1. Their mean is 0.775, an absolute shift of 0.275 from the reference. The proxy alarm fires, while quality remains unknown in the initial report.
After the separate label-ingestion step, all eight predictions match the authored targets. Accuracy is 8/8, or 1.0. The completed decision is to retain v1 in this local experiment and inspect the input change rather than automatically roll back on the proxy alert. All eight targets are positive, so this batch supplies no test of negative-class performance.
Batch B uses a separate set of request IDs and the same input values as the reference: 0.1, 0.2, 0.3, 0.4, 0.6, 0.7, 0.8, and 0.9. It is scored after the explicit v2 reload. Its mean remains 0.5, so this particular input-mean proxy does not alarm. Initially, its quality is also unknown.
When the eight labels are ingested, v2 has four true negatives, one true positive, three false negatives, and no false positives. Accuracy is 5/8, or 0.625; positive-class recall is 1/4, or 0.25. Complete label coverage allows the authored quality gate to run, and it alarms.
| Report state | Batch A under v1 | Batch B under v2 |
|---|---|---|
| Absolute input-mean shift | 0.275; proxy alarm | 0; no proxy alarm |
| Before labels | Quality unknown | Quality unknown |
| After labels | 8/8 correct; no quality alarm | 5/8 correct; quality alarm |
| Recorded local action | Retain v1; inspect shift | Trial rollback to approved v1, then verify |
These reports demonstrate why a proxy and a quality metric answer different questions. Evidently’s drift documentation describes feature and prediction drift as proxies when true labels are unavailable. Our mean-difference check is much narrower than its distribution-testing presets. No mean shift does not mean “no drift,” and an input shift does not by itself prove prediction failure.
Evidently’s classification documentation requires targets and predictions to calculate performance. Here, labels are matched by exact request ID; duplicate or mismatched IDs fail the local assertions. Coverage and denominators remain in the saved report rather than disappearing behind a percentage.

Close the rollback with a response and a batch replay
The B quality alarm prompts a bounded local recovery trial using the previously approved v1 artifact. The runner selects v1, explicitly reloads it, and repeats the diagnostic input. The response returns to 1 with loaded v1 and generation 3. The loaded hash is included so the recovery is tied to a particular artifact.
The runner then sends all eight B feature rows through the reloaded service. Against the same retained labels, the resulting confusion counts are four true positives and four true negatives, with no errors: 8/8 accuracy. The quality alarm clears for this finite replay. It does not retrospectively repair the original predictions or prove that future traffic will perform well.
The completed recovery record therefore contains the triggering labeled metric, selected fallback and approval reference, explicit load confirmation, identical probe response, replay result, and remaining scope. It does not state that an on-call operator approved a production rollback. The fictional approval and scripted decision are under the project author’s control.
No recovery-time objective was measured. There is no queue draining, concurrent in-flight traffic, database migration, feature-store compatibility, multi-replica rollout, or client cache to reconcile. An older artifact could also be incompatible with a changed input contract or downstream dependency. This demonstration did not test such changes, so its successful replay cannot authorize that fallback in another environment. Keep them as untested operating requirements rather than assuming a previous model is always safe.
Package the operating evidence separately from training claims
The closest offline portfolio exercise demonstrates training, a retained split, evaluation, and reproducibility. This project demonstrates a different responsibility: version selection versus loaded state, approval refusal, actual serving responses, batch checks, and recovery. It does not create a new trained model or borrow a test score from the earlier experiment.
The authored classifier makes the mechanics transparent, but limits the significance of the metrics. Eight rows and a deliberately changed threshold cannot establish model quality for a real population. The synthetic labels define the demonstration’s expected outcomes; they are not human-validated production ground truth or evidence about decisions affecting people.
Give a reviewer both routes through the packet: the persistent-service trace for the control change, and the two-phase reports for the monitoring decision. Include the repeat command, tested environment, input hashes, approval assumptions, and exclusions. A screenshot of a version pointer cannot replace either route.
A supported summary is: “Built a persistent local CLI inference demo with hash-bound fictional approvals, explicit model loading, separate proxy and labeled-quality reports, and a verified rollback response and batch replay.” It does not imply MLflow or Evidently integration, cloud deployment, or production MLOps ownership.
Use LandOffer’s current listings to identify relevant responsibilities, then select the actual transition and monitoring records that explain your contribution. LandOffer does not approve model versions, evaluate these batches, or operate the demonstration service. Keep the observed recovery and its untested operating limits together.
Download the worked example files
Download the curated educational companion. Read PUBLICATION-README.md after extraction for the public file layout, replay commands and requirements. The listed offline Python checks were rerun from the extracted archive in a disposable directory. The earlier local report and commands describe the original evidence environment; use the companion README for the portable layout. No whole evidence directory, credentials or compiler binaries are included.


