Extracting Tables for RAG

By Michael Samuel Naeem · January 24, 2025 · 5 min read

Your pricing RAG returned "Starter plan: $29/mo" from a chunk that accidentally merged the Starter row with the Enterprise footnote about annual billing discounts. The table was in the PDF; the chunker treated it as plain text and sliced through row boundaries. Tables are structured data wearing plain-text clothing — standard RAG chunking undresses them into useless fragments.

How tables fail in RAG pipelines

Common failure modes:

• Row splitting — chunk boundary falls between rows; header row separated from data.
• Column splitting — a chunk contains values without column headers.
• Flattening — table converted to space-separated text, losing column relationships.
• Merged cells — PDF extraction mishandles colspan/rowspan, shifting data.
• Multi-page tables — header row appears only on page 1; subsequent pages lose context.

Each produces chunks that embed poorly and generate wrong answers.

Table extraction pipeline

import camelot

def extract_tables_from_pdf(pdf_path: str) -> list[dict]:
    tables = camelot.read_pdf(pdf_path, pages="all", flavor="lattice")
    extracted = []

    for i, table in enumerate(tables):
        df = table.df
        # Set first row as header if it looks like headers
        if is_header_row(df.iloc[0]):
            df.columns = df.iloc[0]
            df = df[1:]

        extracted.append({
            "table_id": f"{pdf_path}:table-{i}",
            "page": table.page,
            "markdown": df.to_markdown(index=False),
            "json": df.to_dict(orient="records"),
            "source": pdf_path,
        })

    return extracted

Validate extraction on a sample — PDF table extraction is imperfect, especially for borderless tables and scanned documents.

Chunking strategies for tables

Whole-table chunking — small tables (under 500 tokens) index as a single chunk with markdown formatting:

| Plan     | Price  | Uptime SLA |
|----------|--------|------------|
| Starter  | $29/mo | 99.5%      |
| Pro      | $99/mo | 99.9%      |
| Enterprise | Custom | 99.99%   |

Row-level chunking — wide or long tables split by row, with headers prepended to each row:

def row_chunks(table: dict) -> list[str]:
    headers = table["columns"]
    chunks = []
    for row in table["rows"]:
        text = " | ".join(f"{h}: {v}" for h, v in zip(headers, row))
        chunks.append(f"Table: {table['title']}\n{text}")
    return chunks

"Plan: Enterprise | Price: Custom | Uptime SLA: 99.99%" embeds and retrieves much better than a bare "99.99%."

Summary + detail — generate an LLM summary of the table for broad queries, keep row chunks for specific lookups:

summary = llm.generate(f"Summarize this table in 2-3 sentences:\n{table['markdown']}")
# Index summary as parent, rows as children (parent-document pattern)

Embedding and retrieval for tables

Tables benefit from hybrid search:

• BM25 catches exact values — prices, plan names, error codes in cells.
• Vector search catches semantic queries — "cheapest plan with 99.9% uptime."
• Metadata filters on table topic — table_type: pricing, product: payments-api.

def search_tables(query: str, filters: dict = None):
    bm25_results = bm25_search(query, filter={**filters, "chunk_type": "table"})
    vector_results = vector_search(embed(query), filter={**filters, "chunk_type": "table"})
    return reciprocal_rank_fusion([bm25_results, vector_results])

Tag table chunks with chunk_type: table and the table's section title as metadata for filtered search.

Handling complex table formats

Multi-page tables — repeat header row on every page during extraction. Most PDF extractors do not do this automatically; post-process to inject headers.

Nested tables — flatten to a single level or extract as separate tables with parent-child metadata linking them.

Tables with footnotes — attach footnotes to the relevant row chunk, not as a separate floating chunk.

Scanned PDFs — run OCR with table detection (Azure Document Intelligence, Google Document AI) before standard extraction. Quality varies; human-review a sample.

from azure.ai.formrecognizer import DocumentAnalysisClient

def extract_with_azure(pdf_bytes: bytes):
    poller = client.begin_analyze_document("prebuilt-layout", pdf_bytes)
    result = poller.result()
    for table in result.tables:
        rows = extract_rows_with_headers(table)
        yield format_as_markdown(rows)

Generation with table context

When retrieved chunks contain tables, instruct the LLM to preserve tabular formatting in answers:

When answering from table data, present comparisons in table format.
Cite the source table by name. Do not interpolate values not present
in the retrieved table.

This reduces the model inventing prices or SLA numbers not in the source data.

Evaluating table retrieval

Build eval questions specifically about tabular data:

• Lookup: "What is the Enterprise uptime SLA?"
• Comparison: "Which plan has the best uptime?"
• Filtering: "Plans under $100/month."
• Aggregation: "How many plans include SSO?"

Measure separately from prose retrieval. Table extraction quality is the ceiling — no retrieval strategy compensates for garbled extraction input.

Common production mistakes

Teams get table extraction documents wrong in predictable ways:

• Skipping failure-mode rehearsal — run a game day or fault injection exercise before peak traffic, not after the first outage.
• Missing correlation context — every error path should carry request, trace, or tenant identifiers so incidents are debuggable.
• Optimizing for demo, not steady state — load tests, cache warm-up, and cold-start paths matter more than local dev latency.
• Undocumented trade-offs — if you chose speed over strict correctness (or vice versa), write that down for the next engineer.

RAG pipelines for table extraction documents degrade when chunk boundaries split tables, embeddings go stale after doc updates, and retrieval metrics are measured offline only. Re-index incrementally and monitor answer faithfulness on live traffic samples.

Debugging and triage workflow

When table extraction documents misbehaves in production, work top-down instead of guessing:

1. Confirm scope — one tenant, region, or deployment stage? Narrow blast radius before deep diving.
2. Check recent changes — deploys, flag flips, config pushes, and schema migrations in the last 24 hours.
3. Compare golden signals — latency, error rate, saturation, and traffic for the affected surface vs. baseline.
4. Reproduce minimally — smallest input or scenario that triggers the failure; capture traces/logs with correlation IDs.
5. Fix forward or rollback — if rollback is faster than root-cause during incident, rollback first, postmortem second.
6. Add a guard — alert, integration test, or circuit breaker so the same class of failure is caught earlier next time.

Document the timeline during triage. Future you (and on-call) will need timestamps, not just conclusions.

Resources

• Unstructured.io table extraction
• Camelot PDF table extraction
• IBM Docling document parser
• Azure Document Intelligence layout model
• Tabula — PDF table extractor