Why I built it
A general chat model knows an enormous amount and none of it is yours. The moment someone asks about their own contracts, runbooks, research notes, or onboarding docs, the base model is guessing from priors. Confidently. Fluently. And without any way for the reader to tell the difference.
DocuMind closes that gap. It takes a private document collection, processes it into a searchable representation, and supports conversational querying over that corpus. The workflow is deliberately narrow: upload your documents, ask questions, get answers that came from the documents.
Generic chat is useful. Grounded chat is trustworthy. Those are different products.
The constraint that shapes everything
Answers should be traceable to source content wherever possible. That single requirement decides the chunking strategy, the retrieval design, the prompt assembly, and the shape of the UI, because a response you cannot audit is worth less than one you can, even if it reads better.
The problem
Users need accurate answers from private documents, but baseline chat tools have no direct grounding in that data. Organization-specific knowledge simply is not in the base model's memory, and no amount of prompting conjures it. The fix is architectural rather than conversational: put the relevant source text in front of the model before it generates.
Architecture
Five layers, each with one job, in one direction.
Ingest
Documents must arrive intact and stay attributable to their source, or grounding is impossible downstream.
The split decides what the retriever can ever find. Highest-leverage layer in the system.
The corpus as the retriever sees it - not as the user wrote it.
Answer
Retrieval quality bounds answer quality. Nothing downstream recovers from picking the wrong evidence.
Builds the answer around retrieved evidence under real token and latency budgets.
Grounding is a UI concern too: the reader needs somewhere to look when they doubt an answer.
- User surface
- Processing
- Index / retrieval
- Model call
Five layers, one direction. Hover a block for the constraint it carries.
Upload and ingestion
The entry point for a user's corpus. Everything downstream depends on documents arriving intact and attributable back to their source.
Parsing and chunking
Where documents become retrievable units. The split decides what the retriever can ever find, which makes this the highest-leverage layer in the system.
Retrieval and index
The searchable representation of the corpus, and the layer that determines answer quality more than the model choice does.
LLM orchestration
Assembles retrieved context into a prompt and manages the conversation, under real token and latency budgets.
- Python
- LangChain
- Streamlit
- RAG
- OpenAI
The tech stack, and why
| Layer | Choice | Why it earned its place |
|---|---|---|
| Core app | TypeScript (77.6%) | Primary logic and integration language. An ingestion-and-retrieval pipeline has enough moving contracts to justify types everywhere |
| UI styling | CSS (20.6%) | Frontend presentation for a chat surface that has to make grounding visible, not just readable |
| Supplemental scripts | JavaScript (1.8%) | Minor runtime and support code |
Retrieval is the whole ballgame
Retrieval-augmented generation fetches relevant source chunks before generating an answer. Stated that way it sounds like a plumbing detail. It is not. It is the quality ceiling of the entire product.
Happy path
The model can only reason about text it actually saw.
Aligned with source content and auditable when someone needs to verify it.
Failure path
No prompt engineering downstream rescues this. The evidence is simply the wrong evidence.
It reads exactly like the grounded one, which is precisely the danger.
Every failure below produces fluent, confident output. That is what makes retrieval the quality ceiling.
The model cannot reason about text it never saw. If retrieval surfaces the wrong three chunks, no prompt engineering downstream rescues the answer. You get a fluent, well-structured response built on the wrong evidence, which is strictly worse than a refusal. So retrieval quality directly bounds answer quality, and every tuning hour spent there returns more than the same hour spent on generation.
Chunking is a tradeoff you cannot escape
How a document is split affects both retrievability and context coherence, and the two pull in opposite directions.
Smaller chunks
A tight embedding matches a tight query. Less of the token budget spent per retrieved unit.
A sentence can lose the surrounding text that made it mean anything.
Larger chunks
The model sees an argument rather than a fragment of one.
More topics per vector means fuzzier matching, and more tokens spent on each hit.
What decides it
Self-contained runbook sections and flowing narrative prose want different splits.
Retrievability and context coherence pull in opposite directions. The corpus decides where to sit.
Larger chunks preserve surrounding meaning, so the model sees an argument rather than a fragment. But they dilute the embedding, so search gets less precise and each retrieved chunk spends more of the token budget. Smaller chunks make search sharper and cheaper, but they can sever a sentence from the context that made it mean anything.
There is no globally correct chunk size
The right split depends on the shape of the corpus. Chunking that works on structured runbooks with self-contained sections will misbehave on flowing narrative prose where meaning accumulates across pages. This is a parameter to tune against your actual documents, not one to copy from someone else's blog post.
Grounding, and why it is a UI problem too
Responses linked to source evidence increase trust. That matters most in exactly the domains DocuMind targets: user-owned knowledge, where the reader often knows enough to be harmed by a confident wrong answer but not enough to catch it unaided.
Retrieve before generating
Nothing is answered from model priors alone. The corpus is consulted first.
Assemble the prompt around the evidence
Retrieved context is what the answer is built from, not a hint alongside it.
Keep the trail auditable
Responses should align with source content and be checkable against it when someone needs to verify a claim.
The honest cost of strict grounding
Tighter grounding can reduce fluency at the edges. An answer constrained to what the corpus actually says will sometimes be clipped or awkward where an unconstrained model would have produced smooth prose. I would take the awkward true answer every time, but it is a real tradeoff and worth naming rather than pretending grounding is free.
Design rationale
What is the hardest engineering problem in a document chat system? Balancing retrieval precision against generation quality under token and latency constraints. Each of the three is easy to optimize alone and they conflict pairwise: better precision usually means more retrieval work, richer context means more tokens, and both mean a slower response on a surface where users expect chat-like responsiveness. Good UX requires accurate retrieval, coherent prompts, and a responsive interaction all at once, which is why the tuning is never a single knob.
How do you build trust in the responses? By designing for grounding and traceability from the start rather than bolting citations on afterwards. Responses align with source content and can be audited when needed, which means when a user doubts an answer, they have somewhere to look instead of a choice between believing the model and abandoning the tool.
Why is chunking a design decision rather than a configuration value? Because it silently sets the ceiling on everything after it. A chunk that never gets retrieved is invisible regardless of how good the model is, and a chunk that arrives without its surrounding context produces an answer that is technically sourced and still wrong. Treating it as a tunable default rather than a corpus-specific design choice is how RAG systems quietly underperform.
Where it stands
Live and open source, grounded in the code as of the development branch, built
solo end to end: ingestion workflow, retrieval and chat orchestration, UI, and
testing. The repository is the
source of truth for the current implementation.