Part B · SLO document
1. Service Overview. One job per page (lead_sheet_page, 1 credit). The browser shrinks the photo, the job checks it again and sends it to the readLeadSheet model task, checks the answer looks like MusicXML and saves it. A page with no staff, or an answer that isn't MusicXML, fails the job and returns the credit VERIFIED. The SLO uses a four-week rolling window.
2. SLIs and SLOs, five SLI types, Google's maximum:
Read a page · LatencyASSUMED
90% within 2 min · 99% within 5 min
Revised 2026-10-25 from 90% within 90 s.
- Specification
- The share of accepted page jobs that reach ready quickly enough, from the buyer pressing submit.
- Implementation
count of lead_sheet_page jobs with ready_at − submitted_at ≤ 120 s ÷ count of lead_sheet_page jobs that ended, from jobs' own timestamps. Needs the SLI contract.- Why two grades
- Warm runs are estimated at 40 to 60 s SOURCED. At invite-stage volume most pages arrive after the endpoint has scaled to zero (15 idle minutes VERIFIED), and a cold start is held up to 50 s VERIFIED: 90 to 110 s, so 90 s would fail on most pages. A cold start over 50 s is a 503, retried after a jittered backoff, which can pass 5 minutes; how often is UNKNOWN until Step 4 measures it.
maxJobSeconds is 600 VERIFIED. - Linked setting
uxBudgetSeconds is 30 VERIFIED: every job longer than that sends a "job done" email, so at 40 to 60 s nearly every page would send one. Set it to the 90% grade (120) or decide the email is wanted on every page.
Read a page · SuccessASSUMED
95% end with a file
- Specification
- The share of accepted page jobs that end with a downloadable MusicXML file.
- Implementation
count of lead_sheet_page jobs ending ready ÷ count ending ready or failed. "No staff found" counts as a failure, because the model can be wrong about it. It can't be watched on its own today: the models adapter reports every handler refusal as one bad_request kind and never reads the body VERIFIED (models decision 20261018). A wrong photo and a model miss look the same. Adding the kind is a security-reviewed change across models.
Buy credits · FreshnessASSUMED
99% within 60 s · 100% within 30 min
- Specification
- The share of completed checkouts whose credits appear quickly.
- Implementation
count of grants with granted_at − checkout_completed_at ≤ 60 s ÷ count of completed checkouts, from billing and credits events.- Backstop
- If the webhook is lost, the heartbeat (every 10 minutes) re-reads a checkout still undecided 10 minutes after creation VERIFIED, so the slow path ends within about 20 to 30 minutes.
Failed page · Credit returnedASSUMED
99.9% within 60 s · 100% within 75 min
- Specification
- The share of failed page jobs whose credit is returned. Money, so the strictest target. A failure releases the credit in the job's own terminal transaction; a stuck job expires after 3 × (600 + 300) + 900 = 3,600 s, then the next heartbeat (10 minutes) settles it VERIFIED. 24 hours was far looser than the system already guarantees.
- Implementation
count of failed jobs with a refund within 60 s ÷ count of failed jobs, from jobs and credits.
See the page · Page experienceVERIFIED
Core Web Vitals "good" at p75 on phones
- Specification
- Confirmed by Kaali as Lead Sheet Reader's page-speed target.
- Implementation
- Field data from real phone visits. With little traffic, lab runs stand in until there is enough.
Correctness is a release gate, not an SLO
Chord accuracy can't be checked on live uploads, because nobody knows the right answer. Google measures pipeline correctness with a prober that injects data "with known correct outcomes" (
Example SLO Document). Here: a golden set of real handwritten lead sheets, used with permission, run on every model version. A new version ships only if chord accuracy doesn't fall below the current one and every file opens in MuseScore. "About half right" is the brief's claim
VERIFIED, not a measurement: the baseline is
UNKNOWN and the eval harness isn't built (AGENTS.md: "eval not built yet")
VERIFIED. So the first job is to measure the current model on the golden set. Golden-set size
ASSUMED: at least 30 pages, with a licence or permission recorded for each, since buyers' uploads can't be reused without consent.
3. Rationale. No traffic exists, so every target above was picked by the author from the model decision's estimate (40 to 60 s warm) and the Launch Kit's 95 to 99% range. None has been measured, and none has been checked against what buyers feel. Step 4 must measure warm and cold times separately and replace these numbers.
4. Error Budget. Each objective has its own budget, 100% minus its goal. At an illustrative 200 pages in four weeks ASSUMED, the 95% success target allows 10 failed pages. At this volume one failure is 0.5% of the month's budget and a quiet week gives no signal. Google's low-traffic options are synthetic traffic, grouping services, client retries, and a lower target or longer window (Alerting on SLOs). Here: below about 200 pages in four weeks, track counts and read every failed page instead of paging on percentages. Retries already exist (3 attempts VERIFIED).
Prober: withdrawn. The earlier draft proposed an hourly model prober at about US$12 a month. That breaks an existing approval: Lead Sheet Reader's model spend is approved at under C$10 a month, with budgetCapCad 10 and dailySpendCapCad 0.33 VERIFIED. Each probe wakes the endpoint for at least 15 billed minutes, about US$0.40 a day, more than the daily cap. Instead: probe everything up to the model on the chassis canary, which has its own queue VERIFIED; whether it can stop short of the model is UNKNOWN. Probe the model only on the golden set at release time.
5. Clarifications and Caveats.
- Job timestamps miss uploads that never reached a job. The upload step needs its own count.
- Jobs cancelled by the buyer are excluded from latency and success.
- Test and synthetic jobs are marked and excluded from buyer SLIs.
- Hugging Face's own limits are a dependency: one replica cannot meet the latency target under any real spike (see Part C).