A RAG BOT THAT CITES ITS SOURCES
← Back to Case Studies
Why I built this
I built this the year I got laid off. That is most of the why.
Read the whole story
In February 2025, Moody's let me go, along with a lot of other people. I had been building production RAG systems there, and suddenly I was building nothing, refreshing job boards, and circling a question a lot of us were asking that year: what is the point of building anything if AI can just do it better? Job searching has a way of quietly convincing you the answer is "none."
So I started building things I actually wanted, for myself, mostly to remember that I still could. A week after the layoff I did a 48-hour game jam with friends. A few months later Uma Musume launched, and I built this bot, half as a portfolio piece and half because I genuinely wanted to stop losing to the training screen. It let me prove, with zero stakes and on my own terms, that I could still ship the exact thing I used to do for a living. That October I joined IBM. The bot is a small monument to the in-between: proof that building for yourself is how you keep your hands warm when the market has gone cold.
The problem
When Uma Musume: Pretty Derby launched globally, I picked it up and immediately hit a wall that will be familiar to anyone who has tried to get good at a deep game. Every character is based on a real Japanese racehorse, down to the race types they excelled at, and the community has responded by writing 200-plus-page training guides and data-mining character traits into spreadsheets. The knowledge is all out there. It is just scattered across thirty browser tabs, and none of it answers the question you actually have at 1 a.m.
The question is never "show me the wiki." It is "how should I build Oguri Cap for Team Trials," or "I have three turns left before the Arima Kinen, what is the best move." Those answers exist inside the guides, buried in walls of text. I wanted to ask them in plain English and get a straight answer, and I wanted an excuse to get back into LangChain while I was between jobs. So I built the bot I wanted to use.
The assumption that changed
The easy version of this project is a chatbot that has "read" the guides and confidently tells you how to build your horse. I had already learned, the hard way, why that version is a trap. At Moody's I helped build a system that summarized credit memos across Morningstar, MarketWatch, and internal financial documents, and the entire game there was hallucination and fact-checking. A confident wrong answer about a credit memo is a real problem. A confident wrong answer about a stamina build just wastes someone's month of training.
So the assumption I dropped was that a good answer is a fluent answer. A good answer is a sourced one. The whole bot is built around a single non-negotiable feature: every response cites the community guide it came from, with the snippet and the page reference, so you can see why it told you what it told you. Explainability was not a nice-to-have bolted on at the end. It was the reason the thing was worth building at all.
What it actually is
It is a retrieval-augmented generation pipeline behind a Discord bot. You ask a question in a channel; it retrieves the relevant chunks of community documentation, grounds an answer in them, and replies with citations. The stack is deliberately boring in the good way:
- RecursiveCharacterTextSplitter to chunk the dense guides. The 200-page walls of text get split into passages small enough to retrieve precisely but large enough to keep their meaning.
- OpenAI embeddings to turn those chunks and every incoming question into vectors, so "how do I build a long-distance runner" matches the right passage even when it shares no exact words with it.
- FAISS as the vector store, for fast similarity search over all those embedded chunks. Ask a question, get the closest passages back in milliseconds.
- Supabase to persist user memory, so context carries between queries instead of every message starting from zero. The bot remembers what you were asking about a minute ago.
Retrieved passages go to the model with the question, the answer comes back grounded in those passages, and the citations are assembled from the same chunks that were retrieved. The sources are not a separate lookup. They are literally the evidence the answer was built from.
The hard part: making it trustworthy
The retrieval is the easy half. The hard half is everything that makes a retrieved answer actually trustworthy, and most of it happens before a single embedding is created. Community guides are written by humans for humans: inconsistent formatting, tables that turn to soup when you strip the HTML, the same character spelled three ways, advice that contradicts itself between two sections. Bad chunks produce confident nonsense, so the unglamorous data cleaning is where the quality actually lives. Fewer horses than the Moody's version, as I put it at the time, but a lot more data cleaning.
Citations are also harder than they sound. It is easy to slap a source link at the bottom of an answer. It is harder to make sure the snippet you show is really the passage the answer was drawn from, rather than a plausible-looking one retrieved alongside it. Getting that honest matters twice over here: it lets a player verify the advice before they sink twenty in-game turns into it, and it credits the community members whose guides are doing the real work. The bot is a convenient front door to their knowledge, and it should always say so.
What it does now
It works. You can ask it how to build a specific character for a specific race type and get a straight, sourced answer, with the community guide and page it came from attached. I built it while I was between jobs, partly as a portfolio piece and partly because I genuinely wanted it, and it turned into a compact demonstration of the exact thing I do professionally: retrieval over messy source documents, grounded generation, and citations that hold up. Same architecture as the financial-document work, minus the compliance stakes and plus a lot of horse girls.
What it's actually about
The horse girls are the fun part, but the pattern is the point. Retrieval over messy documents, grounded generation, and honest citations is the same shape whether the source is a fan-written training guide or a pile of financial filings. The domain changes; the discipline does not. Chunk carefully, retrieve precisely, ground the answer in what you retrieved, and always be able to show your work. A system that can tell you why it said something is worth ten that just sound confident.
There is a smaller thing I did not expect. While I was building it, I messaged my dad, who used to raise racehorses and track which environments and races his horses were suited for. He sent me a photo of an old trophy one of them won. I was sitting there modeling speed types and stamina builds for a cartoon, and realized we had been doing versions of the same thing decades apart. A quirky game about anime horses turned into a way to reconnect over the real ones.
So the question I would leave you with: the last time an AI tool gave you an answer, could it show you where the answer came from? And if not, why did you trust it?