AI Strategy
Olly uses AI as an assistant that narrows and surfaces, never as an autonomous decision-maker. Every model output is grounded in a deterministic structure, and every consequential decision is owned by a human or by reviewed deterministic code.
Position
Regulated surfaces (clinical triage, claims, compliance) cannot tolerate hallucination, drift, or silent confidence inflation. Every AI surface in Olly has (1) a deterministic ground-truth artifact to align to, (2) a human or rule that closes the loop, and (3) an eval suite that gates each release.
Where AI lives in Olly (triage symptom narrower; assist-only roles)
AI is in production in exactly one place today, all other candidate surfaces are assist-only.
| Surface | Role | Status |
|---|---|---|
| Triage symptom narrower (v5 orchestrator) | Captures chief complaint, picks an NHS-grounded pathway, drives dialogue, emits a care band (self-care / GP / urgent / ED) | Live |
| Claims first-pass review | Flags likely-clean vs likely-needs-review claims for a human adjuster | Under consideration |
| Broker assist | Drafts proposals over a broker's own book; broker edits and sends | Under consideration |
| Documentation generation | Drafts internal runbooks and persona summaries from structured inputs | Under consideration |
The triage narrower is the only surface that talks to a member. Even there, the model does not pick the care band freely - it walks an NHS-pathway tree and the band is a deterministic function of the captured answers.
Where AI is explicitly NOT (clinical decisions; compliance assessment; fraud flags; payment decisions)
AI does not make any of the following. These are owned by deterministic code or a credentialed human, and the audit trail must prove it.
- Clinical diagnosis or prescription. Triage produces a care band, not a Dx. Prescribing is out of scope.
- Compliance assessment. Consent capture, GDPR lawful-basis selection, FCA fair-value statements, and CQC-relevant flags are rule-based or human-attested.
- Fraud / AML flags. Pattern detection is deterministic (sanctions list, thresholds, network graph). No LLM raises a fraud signal.
- Payment decisions. Claim approval, clinician payout, refund authorisation, and policy issuance run through deterministic services. AI may surface "this looks routine" but cannot close the ticket.
- Underwriting. Quote pricing is rate-table + actuarial code. No model.
Model choice and tiering (Google Gemini Flash across call sites; tracked in Langfuse)
The production triage service runs on Google Gemini (Gemini Flash) via PydanticAI - the v5 narrower, pathway picker, disposition layer, tone scoring, and reasoning synthesis all call Gemini Flash. Inference is an external Google endpoint pinned to UK data residency (europe-west2); PHI does reach that endpoint, so the control is residency + a no-retention/no-training posture, not on-prem isolation.
| Tier | Use | Model |
|---|---|---|
| Reasoning | v5 narrower, pathway picker, disposition layer | Gemini Flash |
| Fast | Tone scoring, classification, narrow rewrites | Gemini Flash |
Every call is exported as an OTel span to Langfuse with model id, prompt, tool calls, and latency. Spend, p95 latency, and error rate are visible per tier per environment. Model swaps go through the same eval gate as code changes.
Grounding & guardrails (structured outputs; NHS pathway DB grounds disposition; deterministic post-processing; never freeform medical advice)
Three guardrails apply to every consequential AI call.
- Structured outputs. Every model call returns a PydanticAI-validated typed object. Free text is only emitted on conversational turns; structured fields drive routing.
- NHS pathway grounding. The model picks the pathway and asks the questions; a deterministic walker against the matched NHS pathway picks the band. The model cannot escalate or de-escalate freely.
- Deterministic post-processing. Safety-net text, disposition titles, and member-facing copy are templated from a vetted bank, parameterised by the structured outputs.
A captured-but-unanswered red-flag question keeps the session in dialogue; it never resolves silently to a lower band. The narrower is also designed to never reject - out-of-scope symptoms get one more clarifying turn, not a refusal.
Evaluation cadence (deep-persona regression suite per change; shadow AB before promote; KPI-gated rollout)
Every change that touches a prompt, model id, pathway, or orchestrator layer runs the deep-persona regression suite before merge - around 120 personas covering red-flag scenarios and the long-tail MSK / mental health / paediatric edges. Disposition-routing changes additionally run a shadow A/B in dev. Promotion to UAT is gated on a small set of KPIs (red-flag recall, disposition stability, completion rate, p95 latency, cost per session); a regression in any one blocks the promote.
Failure mode (fail-soft to human; never blocking; surfaces uncertainty rather than guessing)
When the model is uncertain, the orchestrator asks another question. When the LLM provider is unreachable, the session terminates with a conservative safety-net disposition ("contact GP or NHS 111") and a flag for human review - it never invents an answer. When the pathway picker cannot match, the session escalates rather than guessing. The service is fail-soft: if AI fails, a human-readable safe path is still reached.
Drift detection (Langfuse weekly review; re-eval triggers on >0.5% disposition shift)
A weekly Langfuse review compares the live distribution of dispositions, pathway picks, and tone scores against the prior week and against the eval-suite baseline. A shift greater than 0.5 percent in the disposition mix, or any movement in red-flag recall, triggers a re-run of the full regression suite and a root-cause review. Provider-side silent model updates - the most common cause - are caught here; the response is to pin the model version and re-evaluate.
Future surfaces under consideration (claims first-pass review; broker assist; documentation generation - all assist-only)
The next three candidate surfaces preserve the "human or rule closes the loop" property.
- Claims first-pass review. Model flags routine vs needs-review against policy terms; an adjuster closes the ticket.
- Broker assist. Model drafts client-facing copy from a broker's own scheme data; broker edits and sends.
- Documentation generation. Model drafts internal runbooks and persona summaries; a human owns the published version.
Each surface must ship with its own grounding artifact, eval suite, and drift watch before going live.
What we explicitly do NOT do
- Do not let AI close a claim, approve a payout, or issue a refund.
- Do not let AI override a clinical decision or emit free-text medical advice.
- Do not ship a model or prompt change without the deep-persona suite green.
- Do not train on member PHI. No fine-tuning on production conversations; no providers that retain inputs for training.
- Do not rely on the model for safety-critical routing - the disposition band is a deterministic function the model cannot bypass.