Graph RAG Apps: A Production Deep Dive and Recommended Stack

TL;DR

• Graph RAG is useful when your product questions need relationships, provenance, multi-hop reasoning, or corpus-level synthesis. It is not automatically better than a well-built vector RAG app for simple lookup.
• The production unit is not “a graph.” It is a repeatable system: source connectors, schema design, entity extraction, relationship validation, hybrid retrieval, graph traversal, answer contracts, evaluations, observability, and human review.
• A good default stack in 2026 is Neo4j for the graph and graph-aware retrieval, Postgres for product metadata, evals, and workflow state, object storage for raw source artifacts, LlamaIndex for property-graph ingestion, LangGraph for durable hand-offs, OpenAI Structured Outputs or equivalent schema-constrained output for extraction/answers, and RAGAS plus custom graph evals for quality gates.
• Use Microsoft GraphRAG-style community summaries for global questions like “What themes are emerging across this corpus?” Use Neo4j/LlamaIndex-style property graph retrieval for product questions like “Which customers are affected by this dependency chain?”
• PMs should hand off the question inventory, risk tiers, source-of-truth rules, answer contract, and launch gates. Engineers should hand back the graph schema, retrieval plans, eval results, trace examples, cost model, and operations runbook.
• Production readiness means boring things work: re-indexing is idempotent, graph changes are versioned, citations resolve to stable source spans, no side effect happens without policy checks, and every answer can be traced back to evidence.

What You Will Learn Here

• When Graph RAG is worth the extra architecture.
• How a production Graph RAG app differs from a demo notebook.
• A reference architecture for ingestion, graph storage, retrieval, generation, evaluation, and hand-off.
• A concrete support and customer-intelligence example with graph schema, retrieval flow, and output contract.
• A recommended stack for MVP, serious product, and enterprise-grade Graph RAG.
• The PM-to-engineering hand-off artifacts that make the project reviewable before anyone writes code.
• The quality gates I would require before calling a Graph RAG app production ready.

This is a follow-up to RAG Apps in Practice. That earlier article covers structured chunks, hybrid retrieval, reranking, and RAG evaluation. This one goes deeper on the graph-specific layer: entity and relationship modeling, graph traversal, community summaries, durable workflow hand-offs, and the stack I would actually recommend for a real app.

The Short Version: Graph RAG Is Relationship Retrieval

Baseline RAG usually starts from text similarity:

Question -> embedding search -> top chunks -> answer with citations

Graph RAG adds an explicit relationship layer:

Question -> entities + intent
         -> graph lookup / traversal / community summaries
         -> evidence paths + source chunks
         -> answer with citations and graph reasoning

That extra layer matters when users ask questions like:

• “Which enterprise customers are exposed if this service is deprecated?”
• “Why did this claim appear in three different policy documents?”
• “Which support macros contradict the refund policy that became active on May 1?”
• “What are the top themes across 5,000 customer escalations?”
• “Which vendors, systems, teams, and contracts are connected to this outage?”

A plain vector search can retrieve semantically similar text. A graph can retrieve connected evidence.

The important caveat: graph retrieval is not magic. It adds extraction cost, schema maintenance, graph drift, and more ways to be wrong. Use it when relationship structure is part of the answer.

When Graph RAG Is Worth It

I would consider Graph RAG when at least two of these are true:

I would avoid Graph RAG, at least initially, when the app is mostly FAQ lookup, documentation search, simple summarization, or a narrow support assistant where parent-child chunks plus hybrid search already work.

What Microsoft GraphRAG Adds

Microsoft Research introduced GraphRAG publicly in April 2024 as a structured, hierarchical approach to RAG over private datasets. Its documented pipeline extracts entities, relationships, and claims from text units, builds a graph, clusters that graph into communities using Leiden-style techniques, summarizes communities bottom-up, and then uses those structures at query time.

The key distinction is query shape:

• Local search helps with questions about specific entities and their neighborhoods.
• Global search helps with holistic questions across a corpus by using community summaries.
• DRIFT search combines local entity search with community context.
• Basic search remains useful when the question is better answered by standard top-k vector retrieval.

That last point is production wisdom. A mature Graph RAG app should route across retrieval modes. It should not force every query through the most expensive path.

Microsoft’s own getting-started guide also warns that GraphRAG can consume a lot of LLM resources during indexing. Treat that as a design constraint, not a footnote. Graph building is often the expensive part.

Production Architecture

Here is the reference architecture I would start from:

                          +----------------------+
                          | Product question set |
                          | PM + domain owners   |
                          +----------+-----------+
                                     |
                                     v
+----------------+       +----------------------+       +-------------------+
| Source systems | ----> | Ingestion workers    | ----> | Raw artifact store|
| docs, tickets, |       | parse, clean, hash,   |       | PDFs, HTML, JSON, |
| CRM, wiki, DBs |       | version, chunk        |       | snapshots         |
+----------------+       +----------+-----------+       +-------------------+
                                     |
                                     v
                          +----------------------+
                          | Extraction contract  |
                          | entities, relations, |
                          | claims, source spans |
                          +----------+-----------+
                                     |
                    +----------------+----------------+
                    v                                 v
          +-------------------+             +-------------------+
          | Graph database    |             | Product database  |
          | entities, edges,  |             | tenants, jobs,    |
          | vectors, fulltext |             | evals, traces     |
          +---------+---------+             +---------+---------+
                    |                                 |
                    +----------------+----------------+
                                     v
                          +----------------------+
                          | Retrieval router     |
                          | vector, fulltext,    |
                          | Cypher, community    |
                          +----------+-----------+
                                     |
                                     v
                          +----------------------+
                          | Answer generator     |
                          | structured output,   |
                          | citations, refusal   |
                          +----------+-----------+
                                     |
                                     v
                          +----------------------+
                          | Evals + observability|
                          | faithfulness, graph  |
                          | recall, cost, drift  |
                          +----------------------+

Notice what is not in the center: a chat UI. The UI is only one entry point. The real product is the knowledge pipeline plus the evidence contract.

The Recommended Stack

Here is the stack I would recommend for most teams building a serious Graph RAG app in 2026.

This is not the only valid stack. It is the one I would choose when a team asks for a practical default with enough structure to survive production.

MVP Stack

Use this when the team is validating whether graph retrieval improves the product:

• LlamaIndex PropertyGraphIndex
• Neo4j Aura or local Neo4j
• OpenAI or another schema-capable model for extraction
• A small FastAPI service
• A 50-to-100-question eval set
• Manual review of graph samples in Neo4j Browser

Do not build a large ingestion platform yet. Prove that relationships improve answer quality.

Production Default Stack

Use this when customers will depend on the answers:

• Neo4j for graph, vector, and full-text retrieval
• Postgres for app metadata, evals, and workflow checkpoints
• Object storage for raw source snapshots
• LangGraph for ingestion and answer workflow hand-offs
• LlamaIndex for graph construction and custom retrieval plumbing
• RAGAS or equivalent metrics for faithfulness and retrieval quality
• LangSmith, LangWatch, OpenTelemetry, or another trace store for production debugging
• CI gates that run a fixed eval set before prompt, schema, retriever, or model changes deploy

Enterprise Stack

Use this when the app touches regulated data, many teams, or critical workflows:

• Everything in the production stack
• Explicit graph schema registry and migration process
• Fine-grained source permissions and per-tenant isolation
• Human approval queues for high-risk answer categories
• Immutable audit logs for source versions, graph versions, prompts, models, and answer traces
• Red-team and abuse test suites
• Offline replay tooling for incident investigation
• Cost budgets per tenant, source, and retrieval mode

Enterprise Graph RAG is less about “bigger graph” and more about “who is allowed to trust which path of evidence.”

Example: Customer Escalation Intelligence

Imagine a B2B SaaS company wants an internal assistant that answers:

Which enterprise customers are likely affected by the billing sync incident, and what should the account team say?

The answer might require:

• incident notes from engineering
• billing service ownership data
• customer contracts
• feature usage telemetry
• support tickets
• CRM account tiers
• known workarounds
• public status-page language

Flat RAG can retrieve some similar documents. Graph RAG can model the relationships:

(Incident)-[:AFFECTS]->(Service)
(Service)-[:OWNS_DATA_FLOW]->(Integration)
(Customer)-[:USES]->(Feature)
(Feature)-[:DEPENDS_ON]->(Service)
(Customer)-[:HAS_CONTRACT]->(Contract)
(Contract)-[:HAS_SLA]->(SLA)
(Ticket)-[:MENTIONS]->(Incident)
(StatusUpdate)-[:COMMUNICATES]->(Incident)

Now the question becomes a graph problem:

Incident: billing sync delayed
        |
        v
Affected service -> dependent features -> active customers
        |                                  |
        v                                  v
Known workaround                    contract tier + SLA
        |                                  |
        +---------------+------------------+
                        v
              recommended account message

The final answer should not merely say “these customers may be affected.” It should return a structured response:

{
  "answer": "Three enterprise customers are likely affected by the billing sync delay...",
  "affected_customers": [
    {
      "customer_id": "cust_123",
      "customer_name": "Northwind Enterprise",
      "risk_level": "high",
      "evidence_path": [
        "incident.billing-sync.2026-07-02",
        "service.billing-sync",
        "feature.invoice-export",
        "usage.cust_123.invoice-export",
        "contract.cust_123.enterprise-sla"
      ],
      "recommended_message": "Acknowledge the delay, confirm invoice exports are affected, offer the workaround, and commit to the next update window."
    }
  ],
  "citations": [
    {
      "source_id": "incident.billing-sync.2026-07-02",
      "span": "Root cause and affected services"
    },
    {
      "source_id": "contract.cust_123.enterprise-sla",
      "span": "Support response obligations"
    }
  ],
  "requires_human_review": true
}

That response is product-friendly because PMs can discuss it. It is engineer-friendly because every field is testable.

The Graph Schema Is a Product Decision

Most Graph RAG projects fail quietly at the schema layer. They extract too many generic entities, too many weak relationships, or no stable IDs.

Start with the questions users need answered, then design a graph schema that supports those questions.

For the escalation example:

Node types:
  Customer
  Contract
  SLA
  Incident
  Service
  Feature
  Integration
  Ticket
  StatusUpdate
  Workaround

Relationship types:
  USES
  DEPENDS_ON
  AFFECTS
  HAS_CONTRACT
  HAS_SLA
  MENTIONS
  COMMUNICATES
  HAS_WORKAROUND
  OWNED_BY

Required properties:
  id
  source_id
  source_span
  confidence
  valid_from
  valid_to
  tenant_id
  extraction_run_id

The properties matter as much as the node labels. Without source_span, users cannot audit the answer. Without valid_from and valid_to, stale relationships survive forever. Without tenant_id, the graph becomes a data-leak risk. Without extraction_run_id, you cannot debug a bad graph update.

Ingestion: Extract Less, Validate More

The ingestion loop should be boring and replayable:

Source snapshot
      |
      v
Parse + normalize
      |
      v
Chunk with stable IDs
      |
      v
Extract entities, relationships, claims
      |
      v
Validate against schema + source spans
      |
      v
Resolve identities + dedupe
      |
      v
Write graph changes
      |
      v
Run graph quality checks

Use schema-constrained extraction where possible. OpenAI’s Structured Outputs docs describe JSON Schema adherence as stronger than JSON mode: JSON mode gives valid JSON, while Structured Outputs can enforce a supplied schema on supported models. Even then, treat the output as untrusted until your code validates it.

A minimal extraction contract might look like this:

from pydantic import BaseModel, Field

class ExtractedEntity(BaseModel):
    id_hint: str
    label: str
    name: str
    source_id: str
    source_span: str
    confidence: float = Field(ge=0, le=1)

class ExtractedRelationship(BaseModel):
    source_id_hint: str
    target_id_hint: str
    type: str
    source_id: str
    source_span: str
    confidence: float = Field(ge=0, le=1)

class ExtractionResult(BaseModel):
    entities: list[ExtractedEntity]
    relationships: list[ExtractedRelationship]
    unsupported_claims: list[str]

Production rule: never let the model invent stable IDs. Let it propose identity hints; deterministic code resolves those hints against known source systems.

Retrieval: Route Before You Retrieve

Do not build one mega-retriever. Route the question.

User question
     |
     v
+------------------+
| Retrieval router |
+--------+---------+
         |
         +--> Basic RAG: simple lookup, exact policy answer
         |
         +--> Local graph: entity neighborhood, dependencies, ownership
         |
         +--> Hybrid graph: vector + full-text + Cypher traversal
         |
         +--> Global graph: community summaries, themes, corpus synthesis
         |
         +--> Refuse / ask clarifying question

Neo4j’s GraphRAG Python docs describe vector retrievers, hybrid retrievers that combine vector and full-text search, and hybrid Cypher retrievers that add graph traversal after initial retrieval. That shape is exactly what production apps need: retrieve candidates, then use graph structure to collect context that text similarity alone would miss.

For the escalation example, the retrieval plan might be:

1. Classify query intent:
   "affected customers for incident" -> graph traversal

2. Resolve anchor entity:
   Incident("billing sync delayed")

3. Traverse:
   Incident -AFFECTS-> Service
   Service <-DEPENDS_ON- Feature
   Feature <-USES- Customer
   Customer -HAS_CONTRACT-> Contract -HAS_SLA-> SLA

4. Pack evidence:
   incident note, service metadata, usage proof, contract span, workaround

5. Generate structured answer:
   customers, risk levels, evidence paths, citations, review flag

This is also where Graph RAG becomes debuggable. If the answer is wrong, you can inspect whether entity resolution failed, the edge was missing, the traversal was too broad, or the model ignored correct evidence.

Hand-Off: What PMs and Engineers Should Exchange

Graph RAG requires tighter hand-off than a normal search feature because “better answers” is too vague.

PM Hand-Off

PMs should provide:

1. Question inventory

   • 30 to 100 real questions grouped by user intent.
   • Mark which questions require graph relationships and which are simple lookup.

2. Decision value

   • What decision will the user make from the answer?
   • What happens if the answer is incomplete or wrong?

3. Source priority

   • Which system wins when sources disagree?
   • Which sources are allowed in customer-facing answers?

4. Risk tiers

   • Which answers can be automated?
   • Which answers require human review?
   • Which answers must refuse?

5. Answer contract

   • Fields the product needs, not prose preferences.
   • Example: affected accounts, confidence, evidence paths, next action, citations, review flag.

6. Launch gates

   • Minimum quality threshold.
   • Latency and cost budget.
   • Required audit behavior.

Engineering Hand-Back

Engineers should return:

1. Graph schema

   • Node types, edge types, required properties, tenancy model, permissions model.

2. Retrieval plan matrix

   • Which query classes use basic RAG, local graph, hybrid graph, or global summaries.

3. Eval report

   • Retrieval recall, faithfulness, citation accuracy, graph path coverage, refusal correctness, latency, and cost.

4. Trace examples

   • Good answer, bad answer, refused answer, human-review answer, stale-source answer.

5. Operations runbook

   • How to re-index, roll back graph changes, replay a failed answer, inspect citations, and patch bad entities.

6. Known limitations

   • Missing sources, low-confidence relationships, unsupported question classes, and human-review boundaries.

This hand-off keeps the team honest. PMs own product risk and acceptance criteria. Engineers own system behavior and evidence.

Evaluation: Test the Graph, Not Just the Answer

RAGAS defines faithfulness as factual consistency between the response and retrieved context. That is necessary, but Graph RAG needs more layers.

Evaluate at four levels:

For Graph RAG specifically, I would add these checks:

• Entity recall: Did retrieval include the expected entities for a known question?
• Relationship precision: Are returned edges supported by source spans?
• Path coverage: Did the evidence include the expected graph path, not just one nearby chunk?
• Citation resolvability: Does every citation resolve to a stable source and span?
• Staleness rejection: Does the system prefer active relationships over expired ones?
• Permission filtering: Does the same question return different evidence for users with different access?
• Human-review routing: Are high-risk answers correctly flagged before side effects?

An eval record can be simple:

{
  "question": "Which customers are affected by the billing sync incident?",
  "expected_entities": ["incident.billing-sync.2026-07-02", "service.billing-sync"],
  "expected_path_pattern": "Incident-AFFECTS-Service-DEPENDS_ON-Feature-USES-Customer",
  "must_cite_sources": ["incident note", "usage record", "contract"],
  "requires_human_review": true
}

The model’s final prose is the last thing to score. First score whether the system found the right evidence.

Durable Hand-Offs Inside the App

Graph RAG apps often involve slow steps:

• source syncing
• parsing PDFs
• entity extraction
• graph writes
• community summary generation
• human approval
• re-answering after source updates

That means you need durable workflows. LangGraph’s docs position it as a low-level orchestration runtime for long-running, stateful agents, with persistence, human-in-the-loop, streaming, and debugging support. Its persistence model distinguishes checkpointers for thread-scoped graph state from stores for longer-lived application data.

A practical workflow might look like this:

Ingestion requested
      |
      v
Parse source snapshot
      |
      v
Extract graph candidates
      |
      v
Validate + dedupe
      |
      +--> low confidence? -> human review queue
      |
      v
Write graph mutation
      |
      v
Run eval slice
      |
      +--> eval regression? -> block promotion
      |
      v
Promote graph version

Use Postgres or another durable backend for workflow state. In-memory workflow state is fine for demos; it is not a production hand-off.

Production Readiness Checklist

Before calling the app production ready, I would want every box below checked.

Data and Graph

• Source snapshots are immutable and versioned.
• Every node and edge has a source reference or is marked as derived.
• Every relationship has confidence, source span, extraction run, and validity window.
• Entity resolution is deterministic enough to reproduce.
• Graph schema changes are migrated and reviewed.
• Tenant and permission boundaries are enforced before retrieval.

Retrieval

• Query router chooses basic, local graph, hybrid graph, global graph, or refusal.
• Hybrid retrieval uses vector and lexical signals where exact terms matter.
• Graph traversal has depth limits, type filters, and cost limits.
• Context packing preserves evidence paths and citation IDs.
• Retrieval traces are stored for debugging.

Generation

• Final answers use a typed output contract.
• The model can refuse when evidence is missing.
• Citations resolve to stable source spans.
• High-risk outputs are routed to human review.
• Prompt and model versions are stored with every answer trace.

Evaluation and Operations

• Fixed eval set runs in CI for retriever, schema, prompt, and model changes.
• Production traces can be sampled into future eval datasets.
• There are alerts for faithfulness drops, retrieval misses, latency spikes, and cost spikes.
• Re-indexing is idempotent.
• Rollback exists for graph versions, prompts, and model changes.
• There is a runbook for correcting bad entities and relationships.

Common Failure Modes

1. The Graph Is Too Generic

If every node is Person, Organization, or Concept, the graph will look impressive and answer poorly. Product-specific node and edge types matter.

2. The Graph Has No Time

Many relationships expire. A customer used a feature last quarter. A policy changed on June 1. A vendor owned a system before the migration. If time is not modeled, stale facts become permanent.

3. The App Trusts Extracted Edges Too Much

LLM-extracted relationships are candidates, not facts. High-impact edges should be validated against deterministic sources or human review.

4. Global Summaries Become Untraceable

Community summaries are useful for themes, but they can hide the underlying source path. Keep summary-to-source provenance.

5. The Team Measures Only Final Answers

A final answer score cannot tell you whether the failure came from extraction, entity resolution, retrieval, ranking, context packing, generation, or permissions.

A Practical Build Plan

Here is the build order I would use.

Phase 1: Prove Graph Value

• Pick one user workflow.
• Collect 50 real questions.
• Label which questions require relationships.
• Build a tiny schema.
• Index a small source set.
• Compare baseline hybrid RAG vs graph retrieval.
• Keep or kill Graph RAG based on evals, not excitement.

Phase 2: Productionize the Pipeline

• Add source snapshots and stable chunk IDs.
• Add deterministic entity resolution.
• Add graph schema validation.
• Add retrieval router.
• Add structured answer contract.
• Add traces and eval CI.
• Add a human-review path.

Phase 3: Scale the Knowledge System

• Add scheduled re-indexing.
• Add graph version promotion.
• Add community summaries for global questions.
• Add tenant-specific permissions.
• Add cost budgets and latency SLOs.
• Add operational runbooks and incident replay.

This phased approach prevents the most common mistake: building a beautiful graph before proving it answers product questions better than simpler retrieval.

The Recommendation

If I were handing this to a team today, I would recommend:

Frontend:
  Existing app UI or internal console

API:
  FastAPI for Python-first teams
  TypeScript API if the product stack is already TS-heavy

Workflow:
  LangGraph for durable ingestion and answer hand-offs
  Postgres-backed persistence for production

Storage:
  Neo4j for property graph, vector index, full-text index
  Postgres for product metadata, evals, traces, jobs
  Object storage for raw source snapshots

Ingestion:
  LlamaIndex PropertyGraphIndex
  Custom extractors with schema-constrained outputs
  Deterministic entity resolution

Retrieval:
  Basic RAG for simple lookup
  Neo4j hybrid retrieval for vector + full-text
  Cypher traversal for entity neighborhoods
  Microsoft GraphRAG-style community summaries for global synthesis

Generation:
  Structured output schema
  Evidence paths
  Citation IDs
  Refusal and human-review flags

Evaluation:
  RAGAS for faithfulness and context metrics
  Custom graph evals for entity recall, path coverage, citation correctness
  Production traces sampled into regression datasets

The deeper recommendation is architectural: keep the graph, retriever, workflow, and product contract separate. When those boundaries are clean, the system can evolve. You can change models, rework extraction, tune retrieval, or replace a framework without rewriting the whole product.

Final Thought

Graph RAG is not “RAG plus a graph database.” It is a way to make relationships, evidence, and hand-offs explicit.

That is why it is powerful for production apps. It gives engineers something testable and PMs something reviewable. The graph is not the deliverable. The deliverable is a system that can answer relationship-heavy questions with evidence, route uncertainty to humans, and keep improving as the knowledge base changes.

Sources

• Microsoft GraphRAG documentation: Welcome - GraphRAG process, indexing, query modes, community summaries, and versioning notes.
• Microsoft GraphRAG Getting Started - installation flow, indexing/query commands, and warning about LLM resource usage.
• Microsoft Research: GraphRAG: Unlocking LLM discovery on narrative private data - April 2024 research blog describing private-data use cases, provenance, whole-dataset reasoning, and baseline RAG limitations.
• GraphRAG paper on arXiv - research paper for the GraphRAG methodology.
• LlamaIndex Property Graph Index guide - property graph construction, labeled nodes, relations, metadata, and graph retrievers.
• LlamaIndex GraphRAG implementation with PropertyGraph abstractions - example pipeline using graph extraction, communities, and a custom GraphRAG query engine.
• Neo4j GraphRAG for Python documentation - official Neo4j package for vector indexes, retrievers, and GraphRAG pipelines.
• Neo4j GraphRAG RAG user guide - vector, hybrid, and hybrid Cypher retrieval patterns.
• LangGraph overview - durable execution, human-in-the-loop, persistence, and orchestration positioning.
• LangGraph persistence docs - checkpointers, stores, thread state, fault tolerance, and production persistence guidance.
• OpenAI Structured Outputs guide - JSON Schema adherence, Structured Outputs vs JSON mode, refusals, and typed SDK support.
• RAGAS Faithfulness metric - factual consistency metric between response and retrieved context.
• RAGAS v0.4 migration guide - current metrics including faithfulness, answer relevancy, context precision, and context recall.
• Anthropic: Contextual Retrieval in AI Systems - contextual embeddings, contextual BM25, and reranking results for retrieval quality.