MeridianHealth — Policy RAG Assistant

01 · Business Problem

The Problem

MeridianHealth Insurance is a mid-sized health insurance carrier headquartered in Hartford, CT, licensed across 22 states, serving ~2.4M covered lives and booking ~$4.1B in earned premiums. It sells through 4,800 direct sales reps, 28,000 independent agents/brokers, and 14 employer-group partnerships across 47 active plan designs. On a live sales call, reps get sharp, specific questions — waiting periods, deductible/copay structure, pre-existing condition coverage, exclusion status — and the answers are scattered across thousands of pages of brochures, terms & conditions, exclusion lists, and claim guides. New reps (~35% of the field force, 28–30% attrition) often don't know where to look. The back-office helpdesk absorbs 11,000–13,000 queries/month at 6–14 min each, and an estimated 18–22% of sales conversations stall at the question-handling stage — with wrong answers carrying real misrepresentation and compliance risk.

The Solution

A RAG-based policy intelligence system built in n8n: the 4 policy documents are chunked and embedded into Pinecone, a rep's question is answered in real time by GPT-4.1-mini using only retrieved, source-tagged context, and every answer is scored automatically by a GPT-4.1 LLM-as-a-Judge across 4 dimensions before being logged to an audit table — so the system can be trusted (or flagged) before wider rollout, instead of reps guessing on a live call.

02 · By the Numbers

Policy Documents Ingested

500/50

Chunk Size / Overlap

Top-K Retrieval

Benchmark Test Queries

3/5

Passed LLM-Judge Gate

4.05

Avg Judge Score /5

03 · Embedded Knowledge Sources

Source Tag	Document	Contains
[BROCHURE]	Policy_brochure.pdf	Plain-language product overview — family floater details, benefits, eligibility, buying steps, claims, renewals, premium factors
[TERMS]	Health_Insurance_Policy_Terms_and_Conditions.pdf	Formal policy wording — definitions, covered benefits, general conditions, waiting periods, claim conditions, renewal terms
[EXCLUSIONS]	List_of_Standard_Claim_Exclusions.pdf	Non-payable items and charges, items only payable under specific wording, hospital-service/room-charge items that can't be billed separately
[CLAIM_GUIDE]	Health_Insurance_Claim_Guide.pdf	Reimbursement and cashless claim process, document requirements, claim status steps, policyholder FAQ

04 · Sample Query Walkthrough — Test Case #1

Rep Question (live chat)

"If I use the cashless claim process, how long does it generally take for the insurer to issue the initial pre-authorisation decision, and how long does it take for the final settlement approval after I submit post-discharge documents?"

↓ Retrieve top-10 chunks from Pinecone (semantic search over 4 documents)

↓ Question & Answer Generation — GPT-4.1-mini, source-tagged, no-guessing system prompt

Generated Answer

Initial pre-authorisation decision is typically issued within 1 hour of receiving complete documents.
Final settlement approval is typically issued within 3 hours of receiving all post-discharge documents.

Sources: [CLAIM_GUIDE]

↓ LLM-as-a-Judge Evaluation — GPT-4.1, 4-dimension rubric

Verdict

Relevance 5/5 · Groundedness 5/5 · Context Precision 5/5 · Context Recall 5/5 — PASS

Pipeline

▶

Manual
Trigger

Read 4 PDFs

→

Chunk +
Embed

500/50, 512-dim

→

Pinecone
Store

Vector index

→

Chat Trigger
+ Retrieve

Top-K = 10

→

Answer
Generation

GPT-4.1-mini

→

LLM-as-
Judge

GPT-4.1

→

💾

Data Table

evaluation_results

Stage 1

Document Ingestion

Read/Write Files · Recursive Character Text Splitter · Embeddings OpenAI · Pinecone Vector Store

Load & Split

Reads all PDFs from /data/shared/*.pdf, extracts text via the PDF loader, and splits with a Recursive Character Text Splitter — chunk size 500, overlap 50 — to preserve continuity across clause boundaries.

Embed & Store

OpenAI embeddings at 512 dimensions, inserted into the Pinecone index meridianhealth-insurancepolicy, namespace meridianhealth-insure. Same embedding config is reused at query time so retrieval and ingestion stay aligned.

Stage 2

Retrieval

Chat Trigger · Retrieve Chunks · Vector Store Retriever (Chunk Limit)

Chat Trigger

Fires on every incoming rep question (typed in the n8n chat interface). The raw chatInput feeds both the retrieval branch and the answer-generation branch in parallel.

Semantic Search

Top-K = 10 chunks pulled from Pinecone per query — enough breadth to cover cross-document questions (e.g. exclusions + terms together) without flooding the context window.

Stage 3

Grounded Answer Generation

GPT-4.1-mini · Question and Answer Generation (chainRetrievalQa)

System Prompt Rules

Role: internal knowledge assistant for support staff on live calls — direct, technical, no soft language. No guessing: missing info → "Not in documents — escalate." Proactive exclusions: every coverage query must check [EXCLUSIONS] and [TERMS]. One-line bullets only, no source tags inside the answer body.

Source Tags

Every retrieved chunk carries one of [BROCHURE] / [EXCLUSIONS] / [TERMS] / [CLAIM_GUIDE], and the answer must cite which tags it drew from — giving the rep (and the auditor) a verifiable source trail.

Stage 4

LLM-as-a-Judge Evaluation

GPT-4.1 · scores QUERY + RETRIEVED_CHUNKS + ANSWER

4-Dimension Rubric (1–5 each)

Relevance — does the answer address the query? Groundedness — is every claim traceable to retrieved chunks? Context Precision — how much of what was retrieved was actually useful? Context Recall — did retrieval surface everything needed for a complete answer?

Hard Groundedness Floor

Verdict is FAIL if any dimension scores ≤3 — and a groundedness score of 1–2 always fails the verdict regardless of other scores, because a hallucinated policy detail on a live call is a compliance and mis-selling risk that overrides everything else.

Stage 5

Persist for Audit

Extract Metrics (Set) · Insert row · evaluation_results data table

What Gets Logged

Every query, the generated answer, all 4 metric scores, and the final PASS/FAIL verdict — giving product and compliance teams a queryable audit trail of what the system told the field, and a way to flag low-confidence responses for human review.

Design Notes

Why a Separate Judge Model?

Answer generation runs on the cheaper, faster GPT-4.1-mini since it's on the live-call critical path; evaluation runs on the larger GPT-4.1 since it's async and needs to be a stricter, more reliable check on the mini model's output.

Why the Groundedness Hard Floor?

In a regulated insurance context, a fluent but hallucinated answer is worse than an honest "escalate" — so the rubric is deliberately asymmetric: relevance and recall failures are recoverable product gaps, but a groundedness failure is a compliance failure and always fails the gate.

How to Import into n8n

01 · Download

Click "Download .json" above to get the workflow file.

02 · Import

In n8n: Menu → Workflows → Import from File. Select the downloaded JSON.

03 · Credentials

Connect your Pinecone API key and OpenAI API key on the respective nodes.

04 · Setup

Upload the 4 PDFs to /data/shared/, create the evaluation_results data table, run the ingestion trigger once, then open chat.

LLM-as-a-Judge Scores — 5/5 Test Queries (/5)

Relevance

4.6

Groundedness

4.0

Context Precision

3.6

Context Recall

4.0

Overall average: 4.05/5 · Pass rate: 3/5 (60%)

Key Finding

Both failures are compound questions — each bundles an eligibility/waiting-period sub-question with a consumables-exclusion sub-question in a single query (newborn add-age + infant formula/wipes; maternity coverage + baby food/diapers/delivery kit). The three single-topic procedural questions (claim timing, exclusion-only, device-only) all scored 4–5 across every dimension. The worst score, context precision 2/5 on the maternity query, shows retrieval pulling mostly noise when a query needs two distinct clause types satisfied at once; the newborn-age answer is even internally inconsistent ("91–180 days" vs. "Day 1 if newborn cover opted"), suggesting the top-K=10 single-pass retrieval never cleanly resolved to one specific figure. This is a classic multi-hop retrieval weakness, not a hallucination problem — the fix is decomposing compound queries into sub-retrievals rather than raising K on a single pass.

Test Case Results

#	Query Topic	Rel	Ground	CtxPrec	CtxRecall	Verdict
1	Cashless claim timing — pre-auth decision vs. final settlement approval	5	5	5	5	PASS
2	Domiciliary hospitalisation for asthma (spouse, Family Floater) + oxygen mask/nebuliser coverage	5	5	4	5	PASS
3	Newborn addition to Family Floater (earliest age) + infant formula/baby wipes coverage	4	3	3	3	FAIL
4	CPAP machine + arm sling coverage after accidental injury hospitalization	4	4	4	4	PASS
5	Pregnancy/childbirth coverage + baby food/diapers/delivery kit reimbursement	5	3	2	3	FAIL

Rel = Relevance, Ground = Groundedness, CtxPrec = Context Precision, CtxRecall = Context Recall (all /5). Verdict rule: FAIL if any dimension ≤3, with groundedness ≤2 as a hard floor. Full query/answer text and raw scores in data/evaluation_results.xlsx.