Every portfolio will have a chatbot soon, and the obvious way to build one is to paste your whole knowledge base into a system prompt and let the model sort it out. That is what I did first. It is also the exact anti-pattern I would never ship at work: raw, unversioned data dropped straight into production.
Two things broke, and both were data problems wearing a chatbot costume. First, a private note leaked — the model started emitting raw [[wiki-link]] syntax, internal cross-references pointing at pages I had never published. Second, and subtler: pasting a 99,000-character build diary made answers worse, not better. The model had more text and less signal, and it started to hedge. More context, vaguer output.
The reframe fixed both at once. The corpus is not a text file — it is a data product, and I already know how to build those. So I gave it the medallion architecture I build for production pipelines: raw at the bottom, curated in the middle, business-ready on top, with tests the whole way up.
Bronze is my private second-brain wiki — everything, unfiltered, and it never ships. Membership is opt-out by folder: anything outside own-use/ is corpus, and the default is exclusion, so forgetting to classify a page keeps it out, not in. Silver is the model-facing layer: curated pages and distilled digests, each wrapped in an XML <document> tag carrying its source, type and date, with the documents placed before the instructions — the ordering Anthropic reports is worth up to a third of answer quality on long context. Gold is the answers served first: a facts sheet derived from the same data my CV renders from, so the bot and the page can never disagree, plus a hand-checked FAQ for the questions everyone asks.
What makes it engineering rather than a metaphor is that the corpus can fail the build. It is generated on every deploy and gated before a single byte reaches a visitor:
// build-knowledge.mjs — the corpus is generated and gated on every build.
const BUDGET_TOKENS = 45_000; // an ATTENTION budget, not a cost one:
const FAIL_TOKENS = 55_000; // past this, every extra token dilutes the rest.
// 1. No private term ever reaches a visitor.
const hits = findDenylistHits(corpus, loadDenylist(DENYLIST_FILE));
if (hits.length) { console.error('DENYLIST HIT:', hits.join(', ')); process.exit(1); }
// 2. Stay inside the attention budget, or the build stops and forces a choice.
if (totalTokens > FAIL_TOKENS) process.exit(1);That denylist gate is the data-quality test; the eval harness is the regression suite. Twenty-nine cases run in CI on every corpus change — outcome-first answers for recruiters, mechanism-first ones for engineers, prompt-injection and salary-bait cases the bot has to refuse, and questions in French and Chinese it has to answer in kind. And the day that [[wiki-link]] leaked, it became a permanent assertion checked on every answer of every case:
# questions.yaml — run in CI on every corpus PR (npm run eval:chat)
- id: hire-visa
question: Does Tong need visa sponsorship to work in the EU?
must_mention: ["Blue Card"]
must_not: ["cannot work", "needs a new sponsorship"]
# ...plus a global assertion the runner injects on EVERY case:
# must_not: ["[["] — a bug found once never comes back silently.The top tier is monitoring. Every real conversation is logged, and a gap miner classifies each reply — did the bot actually know this, or did it dodge? The misses cluster into topics, and the topics tell me what to write next. The corpus stops converging on what I guessed recruiters would ask and starts converging on what they actually ask. Collection, quality gates, serving, monitoring — a data product in production, pointed at myself.
Prompt context is a data product, not a text dump — so I gave it the tests I would give a pipeline.
The part I like most is recursive: this essay is Silver. It is derived from a private Bronze planning document I will never publish, and before it reached this page it passed the same denylist gate as everything the bot says. The method is not a diagram on a slide — it is running underneath this sentence, and you are reading its output.