An insurer wants to cut the hours its claims team spends reading policy documents by hand. Where do you start?
Answer it out loud first, then compare. That's what makes it stick.
I'd learn the work before touching a tool. How do adjusters read a policy today, what questions are they really answering, how many documents a week, in what formats, and how often do templates change? What does a wrong reading cost and who catches it? Then I'd pick one narrow question type, like coverage-limit lookup, build it with citations to the page, and measure time saved and errors against adjusters' current work.
- Shadow the process: what question is the reader answering?
- Volume, formats, template churn
- Cost of a wrong answer, current catchers
- Pick one question type for slice one
- Cite sources; measure time and errors vs baseline
This prompt appears in a published write-up of a deployed-engineer case round, so expect something close to it. The trap is hearing 'documents' and answering 'RAG'.
Step 1: Find the real task. Adjusters are not reading for fun. They are answering questions: is this covered, what is the limit, is an exclusion triggered? Ask for the top 5 questions they answer per policy. That list is your product scope.
Step 2: Size and shape.
- Documents per week (say 1,000) and minutes per document (say 25) gives the hours at stake: about 400 hours per week.
- Formats: clean PDFs, scans, endorsements added over time? Scans need OCR and raise error rates.
- Template churn: if each carrier or state uses a different wording, extraction rules will not transfer.
Step 3: Cost of being wrong. A missed exclusion can mean paying a claim wrongly; a wrong limit can mean a complaint. Ask who catches mistakes today (peer review, audits) because that defines where a human check should stay.
Step 4: Choose slice one. Do not start with 'read all policies'. Pick one high-volume, checkable question (for example 'what is the per-claim limit and deductible?') on 2-3 policy types. Output a short answer with the page and quote it came from, so the adjuster verifies in seconds instead of rereading 40 pages.
Step 5: Prove it.
- Baseline: time 20 real cases done by hand.
- Test set: 100 historical cases with the answer the adjuster reached.
- Pilot: adjusters use the tool, you measure time per case, accuracy against the labelled set, and how often they override it.
Step 6: Risks with fixes.
- Stale or superseded endorsements: version the documents and show the date.
- Over-trust: keep human sign-off on denials.
- Data access: start with a read-only, de-identified sample while security review runs.
What a strong close sounds like: 'In two weeks I can show one question type on 100 past cases with page-level citations, and we decide with numbers whether to widen.'
- Adjusters say they don't trust AI summaries. How do you respond?
- Half the policies are scanned images. How does that change your plan?
- Which metric do you show the VP of claims after the pilot?
- How Distyl AI hires forward deployed engineers (prep, unknown) Gives this insurer prompt as an example case; says naming a tool like RAG too early signals pattern-matching.
- The four interview rounds (Educative) (prep, unknown) Decomposition prompts are ambiguous enterprise problems judged on clarifying questions, clean breakdown and root cause versus symptom.