Margin: a document assistant you can check

Students, researchers, professionals and small teams spend hours reading contracts, papers, policies and minutes to find the few things that matter: what they owe, what was decided, what the evidence says. General chatbots can summarise, but their summaries are hard to trust. Checking a claim means hunting for it in the original, which can take as long as reading the document.

Hypotheses, not findings. These users and pains come from desk research and my own use, not from interviews. The study plan in section 10 is how I'd test them.

Read less, understand more, and check everything. Margin gives you a summary, answers and key facts, and keeps the source one click away. The design principle is that the evidence is part of the answer, not a footnote.

• Executive summary, takeaways and a section-by-section summary, with a free outline from detected headings.
• Questions answered only from the document, with a clear “the document doesn't say” when it doesn't.
• Key facts in six categories: people and organisations, dates, figures, decisions, risks, action items.
• Comparison of two documents: in common, different, and only in one.
• A private library with search across every document.

The v1 issues were found by installing and running the original repository, not just reading it.

1. Drop a file, or try one of four sample documents. Upload errors explain what to do (“this looks like a scanned PDF”).
2. The document opens as real pages beside the analysis; the outline jumps between sections.
3. The summary arrives with a grounding note: “All 23 claims are backed by quotes found in the document.”
4. Click any page chip: the viewer scrolls to the page and highlights the exact quote.
5. Ask follow-up questions; open “How this was found” to see which passages were searched and how they ranked.

• Evidence colours mean one thing. Green: quote found word for word. Amber: close match. Red: not found. Interpretations carry a label instead of a colour.
• Reading type for reading. Source Serif for document text and AI prose, Plex Sans for interface, Plex Mono for metadata, one highlighter colour for evidence.
• Every state designed: empty library, indexing, loading, AI off, offline demo, server waking up, not found, partial answers.
• Accessible by default: skip link, real tabs with ARIA, keyboard-operable upload, visible focus, reduced-motion support, light and dark themes, a phone layout that switches between the document and its insights.
• Cost is visible: each result shows the model, tokens, time and whether it came from cache.

The retrieval eval needs no API key and runs in CI. Four synthetic documents (15 pages, 47 chunks) and 47 answerable questions, each labelled with the page that holds the answer. A hit at k means a top-k passage covers that page. Run on 2026-10-21 with bge-small-en-v1.5:

What this says: on this set, dense search alone is as good as or better than hybrid; hybrid only wins at hit@6 within a document (the six passages Ask actually sends). Down-weighting BM25 closed the gap, but one question is 0.021 of any rate here, so I didn't tune to it. These documents and questions were written by the same person who built the pipeline, and a document-scoped search picks from about a dozen passages: treat this as a regression suite and a comparison, not real-world accuracy.

Answer quality is measured by a second script (evals/run_generation.py): accuracy against expected facts, correct refusals on 7 unanswerable questions, false refusals, the share of claims whose quotes verify, stated-but-unverified claims, latency and tokens. It runs the real Ask pipeline end to end. Run on 2026-10-21 with gemini-3.5-flash-lite over 54 questions:

The one wrong answer was a real error: asked how long the first contract period is, it inferred “at least twelve months” from a termination clause and missed the stated 24 months, though it did mark the answer as partial. The failed refusal: asked what Neha Kulkarni reported about sales, it said (correctly, with a verified quote) that she sent apologies, instead of saying the minutes don't cover it. Accuracy is a strict string match against facts the builder wrote, on a small synthetic set, using the free tier, where a few responses took 10–15 seconds. Read it as evidence that grounding holds up, not as a benchmark.

• Two-column pages came out interleaved. Sorted text extraction reads straight across both columns. Margin now looks for a vertical gutter between groups of lines and reads each column top to bottom.
• Highlighting needs the page, not the text. Quotes are located on the rendered page with PyMuPDF and drawn as an overlay, so the highlight lands on what the reader actually sees. When a quote wraps strangely, the viewer says so instead of highlighting the wrong thing.
• Titles. Files named “scan_0042.pdf” became their title. Margin now prefers the PDF's metadata, then a meaningful filename, then the first heading.
• Working without a key. An offline demo mode copies sentences from the document so the whole interface can be built and tested without a key. It's labelled as not AI wherever it appears, and none of the portfolio images use it.

• 42 backend tests (pytest) with a fake model: parsing (including scanned, encrypted and two-column PDFs), chunking, quote verification, retrieval, workspace isolation, path traversal, upload limits, caching, map-reduce, rate limits and errors.
• 9 frontend unit tests (Vitest), typecheck and lint.
• A scripted browser walkthrough of every screen at 1440 px and 390 px, in light and dark: no console errors, no horizontal scroll, skip link first in the tab order.
• CI runs lint, format, tests and the retrieval eval on every push.

Try it with one of the sample documents from the library. The live demo runs gemini-3.5-flash-lite on free hosting, so the first visit after a quiet spell takes up to a minute to wake the server.

• Usability study (planned, not done): 5-6 participants (students, an analyst, a paralegal) each answer 4 questions about a 20-page document, once with Margin and once with a plain PDF reader. Measure time to a correct answer, how often they click a citation, and confidence before and after. Key question: does the grounding note change how much people trust and check?
• Run the answer eval with Gemini and a second model; add an identifier-heavy retrieval set to settle hybrid vs dense.
• OCR for scanned PDFs, tables extracted as tables, and accounts with sharing for teams.
• Lesson: “add citations” isn't a feature until something checks them. The verification step and the UI that shows its result are what make the product trustworthy, more than the model choice.