← Back to Work
Generative AI (K-Bot) Per-Type Parsing & Chunking Pipeline
A preprocessing system that lifts RAG quality by routing parsing, chunking, and embedding per document type
A single text-processing path erased PDF page context, slide structure, and cross-page tables, weakening the context available to retrieval.
02 / Constraints
The existing upload API and user flow had to remain stable.
PDF, DOCX, PPT/PPTX, Excel, and text required different structural handling inside one pipeline.
Page, slide, and sheet provenance had to survive through retrieval.
03 / Decisions & approach
Route by document type
Parser and chunking strategies are selected from file type and usage context.
Preserve structure while chunking
Headings and layout define semantic units, while cross-page tables are merged into one searchable unit.
Keep provenance as data
Page, slide, and sheet positions stay in chunk metadata so evidence remains traceable after retrieval.
04 / System flow
Document processing flow
One upload entry point selects the strategy that fits each document.
-
Upload
-
Route
-
Parse
-
Chunk
-
Index
05 / Outcome
- Tables, slides, and page-based documents became structure-aware searchable units.
- New document types can be added as strategies without changing the upload flow.
Retrieval quality starts with how much source structure survives, before the embedding model enters the picture.