AI Workflows - The Rise of Personal Documentation Assistants

This blog is part of the AI Workflows series and discusses a brief history of documentation-centric assistants, and how to consume product documentation more productively and effectively.

These days, many products come out, more or less fleshed out, in part thanks to AI boosting or allowing smaller teams to build smaller tools, but also in industries with a lot of innovation or funding.

The dread of onboarding

One concern that's extremely relevant to these new products, but also relevant to quite a great many products overall, is that the documentations are very very hard to consume. It's hard to search for what you want, especially when you don't know what you want. It's hard to find a precise answer to everything, or even a hint of how things work.

Not every documentation is made equal, and some are just good, or have an amazing search built-in. TailwindCSS comes to mind, or BunJS, amongst others.
Most, however, are hard to consume at best. Some are downright indomitable and induce a state of absolute madness. Even those of many major software houses and even AI builders do not have a very good documentation assistant. That says a lot.

The rise and fall or "Documentation AI Assistant"

Back when ChatGPT came out, in what feels like an eternity ago in AI age, everyone started thinking "hey, maybe I can feed it with some data and it will be a good documentation assistant"

In essence, the idea had a lot of merits. Given how smart it felt back then, it felt like nothing was impossible, so why not give it all we have and hope it will figure out the needle in a haystack.

In practice, however, it did not work out as well. I don't think I've ever talked to a documentation assistant and came out happy of my experience, and found out the answers I was looking for. Oftentimes, the answer was just worse than just reading the documentation myself. And that's when it could find out the relevant information at all.
It was, however, a very cheap way to access ChatGPT for free 😂.

RAG and riches

Then came the RAG systems, which stands for "Retrieval Augmented Generation", using some clever tricks whereby the data is chunked into small pieces of a couple dozen or hundred words and embedded, which is a process where words are transformed into a large vector - essentially a long list of floating point numbers that give successive directions across an N-dimensional space that only makes sense to the model and represents the embedding of the meaning of that sentence. The advantage of embeddings is that they have 2 properties are particularly interesting. The first one is that they embed the meaning of the sentence they carry, the second is that they have vectors, so we can leverage maths to run calculations to find how related 2 embeddings are, or, more accurately, how close they are in the embedding space. Thanks to these properties, we can sort all the embeddings by mathematical distance and the ones at the top should be the most similar to what you are searching for.
This process is the retrieval part, where you try to find chunks that are relevant to what the user is looking for, then augment the context with just these chunks.

In practice, it works just okay. Nothing crazy. It improved on the state of things a little, but not by a mile.

Devil's in the details

So, why is this not working?

Part of the problem is that you are not looking for a needle in a haystack. You are looking for a piece of your story that's missing.
What I mean by that is that if you go to a Documentation Assistant and ask it point blank to complete your story, without it know the details that define your story, it's just going to give you low quality gibberish most of the time.

The rise of LLM.txt

To the chagrin of marketing people, who would - legitimately - love to capture all that delicious traffic and maximize conversion by doing all sorts of optimizations and hooking you in, the best tool you have at your disposal in order to learn and find what you need in a documents is to stay within your personal workflow, and ask it to build the piece of your story that's missing.
Because it already knows much about what you are trying to do (your story), all it now need to help you is to learn this thing you are trying to understand, and for that, we can just "upload" it - SciFi lovers will reminisce of Neo learning kungfu in the Matrix.

To that effect, many projects have started to put together a file, oftentimes called LLM.txt, in memory of the robots.txt file that's long been used as a standard to convey informations to robotic crawlers.
This file contains all the bits from the documentation, in markdown, formatted and ideally optimized for an LLM to process and understand immediately.

This file is typically fairly big, 20k, 50k, 80k, sometimes 100k tokens (pieces of text that's about equivalent to 0.7 word), plus your conversation history, this means only modern, long context and long span attention models will be able to handle that. But models the likes of Gemini 2.0 flash or 2.5 Pro are absolute beasts at handling it.

You will, however, instantly possess a superpower: The ability to learn and query any product documentation WITHIN the context of your needs, at your disposal, and you are in full control. What a time to be alive!

All you need to do is to download one of them, upload it to your chat, and simply query away.

As a Product Builder, should I use LLM.txt?

A resounding YES. The products that will thrive in the age of AI are those that will make it much easier to use with AIs, in other words "AI-friendly products will win". Be one of them. Even if that means you are losing a little bit of your marketing control and overview, it will be worth it in the end. And, on the other hand, not doing it will be a disservice to you because your competitors will.