Knowledge Graphs for AI Retrieval — Structured vs. Vector

The Retrieval Problem Vector Search Doesn't Solve

Vector retrieval is effective when the query and the relevant document share semantic similarity. "What is the capital of France?" retrieves a document about Paris. "How does the immune system respond to viral infection?" retrieves immunology documents. The embedding captures meaning, nearest-neighbor search finds the relevant chunks, the LLM synthesizes the answer.

This breaks down for relational queries. "What companies did the founders of Stripe work at before Stripe, and which of those companies later competed with Stripe?" is not a semantic similarity problem. It's a multi-hop traversal over a graph of relationships between entities. Vector search will retrieve documents mentioning Stripe, Patrick and John Collison, and payment companies — but it won't tell you which companies the founders worked at, connected to which companies later competed with Stripe.

Knowledge graphs answer relational queries. They model entities and the typed relationships between them as a directed graph. Queries traverse the graph structure rather than ranking by similarity. The two approaches solve different problems and are most powerful in combination.

Knowledge Graph Fundamentals

A knowledge graph is a collection of triples: (subject, predicate, object). Each triple encodes a fact about a relationship between entities:

• (Patrick Collison, co-founded, Stripe)
• (Stripe, competes-with, Adyen)
• (Patrick Collison, previously-worked-at, Auctomatic)

Entities have types (Person, Company, Product) and properties (name, founding date, industry). Predicates have types that define valid subject and object types — a co-founded predicate expects a Person subject and an Organization object.

This structure enables queries like: "Find all Person entities who co-founded Company entities that are in the same industry as Company X." In SPARQL or Cypher (graph query languages), this is a straightforward query. In vector search, it's intractable.

Building Knowledge Graphs from Documents

The pipeline from unstructured documents to a knowledge graph has three stages:

Stage 1: Entity Recognition. Extract named entities from text — people, organizations, locations, products, concepts. LLMs perform well at entity extraction with appropriate prompting:

const ENTITY_EXTRACTION_PROMPT = `Extract all named entities from the following text.
For each entity, provide:
- name: the entity's canonical name
- type: one of [Person, Organization, Product, Location, Concept, Event]
- aliases: alternative names or abbreviations mentioned in the text

Return as JSON array.

Text:
{text}`;

const EntitySchema = z.array(z.object({
  name: z.string(),
  type: z.enum(['Person', 'Organization', 'Product', 'Location', 'Concept', 'Event']),
  aliases: z.array(z.string()),
}));

Stage 2: Relation Extraction. For each pair of entities found in proximity, extract the relationship between them. This is the most challenging step — relationship types require a controlled vocabulary to be useful:

const RELATION_TYPES = [
  'co-founded', 'works-at', 'formerly-worked-at', 'invested-in',
  'acquired', 'competes-with', 'partners-with', 'created', 'uses',
] as const;

const RelationSchema = z.object({
  subject: z.string(),
  predicate: z.enum(RELATION_TYPES),
  object: z.string(),
  confidence: z.number().min(0).max(1),
  source: z.string(), // document ID the relation was extracted from
});

Stage 3: Entity Resolution. The same entity appears under different names across documents: "Patrick Collison", "Collison", "the Stripe CEO". Entity resolution maps these to a single canonical entity identifier. Naive string matching fails for this — use embedding similarity between entity descriptions combined with string similarity heuristics.

async function resolveEntities(
  rawEntities: RawEntity[],
  existingEntities: CanonicalEntity[]
): Promise<Map<string, string>> {
  const resolutionMap = new Map<string, string>();

  for (const raw of rawEntities) {
    const embedding = await embed(raw.name + ' ' + raw.context);
    const nearest = await vectorSearch(embedding, existingEntities, { limit: 5 });

    const match = nearest.find(candidate =>
      candidate.similarity > 0.9 &&
      candidate.type === raw.type
    );

    if (match) {
      resolutionMap.set(raw.name, match.id);
    } else {
      const newId = createEntity(raw);
      resolutionMap.set(raw.name, newId);
    }
  }

  return resolutionMap;
}

Graph Storage and Query

For production knowledge graphs, purpose-built graph databases outperform relational databases adapted to graph workloads:

• Neo4j: The established standard. Cypher query language is expressive and well-documented. APOC plugin library extends the built-in functionality significantly. Operational complexity is non-trivial.
• Kuzu: Embedded graph database (like SQLite for graphs). No server process, excellent for smaller graphs that fit in-process. Open-source, Python and Node.js bindings.
• Amazon Neptune: Managed service supporting both SPARQL and Gremlin. Correct choice if you're deep in AWS and don't want to manage graph database infrastructure.
• FalkorDB: Redis-compatible graph database. Correct if you're already operating Redis and don't need the full Neo4j feature set.

For most RAG applications at startup/early stage, Kuzu embedded in your application process is the lowest operational overhead path. Migrate to a dedicated graph database when your graph exceeds a few million nodes or you need multi-process access.

GraphRAG: Combining Graphs and Vectors

Microsoft Research's GraphRAG paper (2024) formalized the approach of combining knowledge graph traversal with vector retrieval. The core insight: vector retrieval finds relevant documents, knowledge graph traversal finds relevant relationships and context that span documents.

The GraphRAG pipeline for answering a question:

1. Extract entities from the query: "Which Stripe founders previously worked at companies in the payments industry?"
2. Retrieve directly relevant entities from the graph: Stripe, Patrick Collison, John Collison
3. Traverse relationships from seed entities to find related context: co-founded, previously-worked-at, industry relationships
4. Retrieve vector-similar documents for each traversed entity: documents mentioning each found entity
5. Combine graph-structured context + document chunks into the LLM context window
6. Generate the answer with full relational and documentary context

async function graphRAGQuery(question: string): Promise<string> {
  // Step 1: Extract entities from the question
  const queryEntities = await extractEntities(question);

  // Step 2-3: Graph traversal from seed entities
  const graphContext = await db.run(`
    MATCH (e:Entity)-[r]-(related:Entity)
    WHERE e.name IN $entityNames
    RETURN e, r, related
    LIMIT 50
  `, { entityNames: queryEntities.map(e => e.name) });

  // Step 4: Vector retrieval for each entity in context
  const embeddedEntities = await Promise.all(
    graphContext.entities.map(e => vectorSearch(e.embedding, { limit: 3 }))
  );

  // Step 5: Build combined context
  const context = buildContext(graphContext, embeddedEntities.flat());

  // Step 6: Generate answer
  return await generateAnswer(question, context);
}

When to Use Each Approach

Use vector retrieval when:

• Queries are primarily semantic ("explain how X works", "what is the difference between X and Y")
• Your knowledge base is unstructured prose without clear entity-relationship structure
• You need to retrieve relevant documents across a large corpus quickly
• The relationships between pieces of knowledge are not the focus of queries

Use knowledge graphs when:

• Queries require multi-hop reasoning ("what companies did X work at, and which of those competed with Y")
• Your domain has a clear taxonomy of entity types and relationship types
• You need to answer "who is connected to whom" type questions
• You need structured, verifiable provenance for retrieved facts

Use both (GraphRAG) when:

• Your domain has both structured relationships and rich prose context
• Questions can be either semantic or relational
• You need the structured reasoning of graph traversal with the document-level context of vector retrieval

The practical recommendation: start with vector retrieval. It has simpler infrastructure, lower implementation cost, and covers the majority of RAG use cases. Add knowledge graph extraction and traversal when users repeatedly ask relational questions that vector retrieval answers poorly. The signal that you need GraphRAG is specific and measurable — don't add the complexity speculatively.