Home Creators Posts Import Register Favorites Logout

Content

Over the years I’ve sometimes struggled to find information in the long, detailed and complex documentation for many CG apps. Developers often publish docs using one of a few standard frameworks and search in particular can be clunky and confusing. If you type in a keyword you can expect to get dozens of results pointing to outdated pages, passing mentions in release notes, and worse.

So, I thought, wouldn’t it be great if we could interact with reams of app documentation using natural language? For example, “how do I import an image texture in Arnold?” or “what is the default keyboard shortcut for the perspective camera in Houdini?”

We’ve all got a bit of AI Fatigue by now, drowning in hype about how it’s going to kill us all, save humanity, put everyone out of work, increase prosperity, and everything in between.

That’s indeed very tiresome, so I try to filter through the noise and learn what AI actually is good for in the real world. One example, RenderMan’s denoiser, is a triumph of AI (or more, accurately, machine learning). It produces incredible results that seem almost like magic, with no downsides. All the people who worked their fingers to the bone painstakingly denoising images by hand won’t be put out of a job as a result of Pixar’s tech (they didn’t exist!).

I think one of the greatest recent advancements in AI, and large language models (LLMs) in particular, is the ability to interact with computers using normal language. Being able to say, hey, give me a recipe for muffins and be sure to include blueberries but no nuts is pretty amazing.

I decided to try creating an app that uses natural language to interact with CG docs. I’ve been researching and testing, and the results, genuinely to my surprise, show a lot of promise. Let’s take a look!

The Theory

In the section after this one I’ll talk about how to set up and use what I’ve developed so far, but first let’s look at the problems and challenges facing this project.

The main technique I’m taking advantage of is RAG (Retrieval-Augmented Generation). This is used to augment existing AI models with new information. The models from big companies have already done the hard work and we certainly don’t want to get involved in modifying or re-training them. That requires huge technical investment and lots of expensive compute time.

Instead, we can use RAG to tell an existing model: here is some new knowledge, only refer to that when producing answers. In the case of a CG app, we’ll point the model to a pre-processed version of the app’s documentation and ask it to generate answers using those docs and to never refer to its other training or make things up.

For me, the number one problem with all existing general purpose AI is that it is very susceptible to inventing complete nonsense. If you go to Gemini or Grok or whatever and say “how do I create an MPM simulation in Blender,” a lot of the time it will spit out paragraphs of text referencing nonexistent menus and tools even though you can’t do that in Blender and anybody with even a passing familiarity would know that.

RAG attempts to solve this and, in my experience with this project so far, it does a pretty decent job, depending on the model. A few older models I tested enthusiastically made stuff up even when asked not to and so I’ve removed them from the list of useable models. Recent models more or less take the instructions to heart and are able to limit themselves to the docs they’re given.

Creating a working alpha version of this documentation assistant involved a few steps:

Sanitize the existing HTML-based docs, removing all markdown, menus and any other distractions. Save the sanitized files to plain text format.

Create a vector database based on the text files that an AI model can access.

Write a script that takes a user question, injects it into a prompt that references the documentation database, then presents an answer to the user with URL references to the relevant sections of the docs.

So that’s the general concept. Overall it’s not too complicated, but there are a lot of things that can go wrong and can benefit from optimization. Curating the docs, optimizing the database, picking the right model and writing a good prompt are all crucial when creating a script that provides useful and accurate results.

Setting up DccDocsHelper

I’ve shared the first alpha build of the project on GitHub: https://github.com/BrianHanke/DccDocsHelper

It implements all the ideas discussed above, but for now I’m just sharing the main assistant.py script and a database produced from the Gaffer 1.6.20.0 documentation. Gaffer was easy to work with since the docs are structured cleanly and they’re included with all Gaffer releases. Eventually I want to add many more apps, but Gaffer is a good place to start and see how things are working. I wasn’t expecting much to begin with, but I’m actually surprised how well it’s working and I think it’s definitely worth continuing development. Once things get more advanced I’ll share all the sanitization and database creation scripts.

All that being said, here’s how to get set up!

You’ll need a recent version of Python (I’m using 3.10): https://www.python.org

You’ll need Ollama, a platform for downloading and running AI models: https://ollama.com

You'll need to install a few Python packages with pip:

pip install ollama chromadb rich

And you'll need an AI model or two. You can find the full list here: https://ollama.com/search

I'm still testing, but so far gemma4 from Google is a fine choice. I'm also liking granite4.1:8b from IBM. There are many that don’t work well at all, including deepseek-r1, mistral and gemma3.

To download a model, type:

ollama pull model_name

Generally speaking it seems that models ranging in size from 5-10GB are good enough. Anything larger is probably overkill and the small ones I tried fail some basic tests.

To run the assistant, fetch the GitHub repo either by downloading and extracting the ZIP or by typing:

git clone https://github.com/BrianHanke/DccDocsHelper.git

Edit line 12 of assistant.py to point to the model you want to use.

Finally, navigate to the repo's src folder:

cd /path/to/DccDocsHelper/src

And type:

python assistant.py

This will show a greeting with the currently active model and a prompt to type a question. Ask anything about Gaffer and after some thinking the model will respond.

It should refuse to answer any general questions and focus only on what is available in the Gaffer documentation. Many models failed this step, ignoring the prompt instruction and producing sometimes extremely elaborate muffin recipes from their general knowledge.

I’ve been trying various questions and gauging the accuracy and usefulness of the answers. Simple things like “what is the keyboard shortcut to view the alpha channel?” have been great:

Other things like “how do I create a box?” are generally right. Some models get carried away and start introducing some questionable details, but overall I didn’t find anything catastrophically wrong. Using granite4.1:8b here, it’s all good aside from point 2. Calling Tab > Utility a menu is a little misleading and there is no right-click option.

I’d never heard of IBM’s Granite models before and they’re quite impressive for this task. Their docs specifically mention that they’re optimized for RAG workflows and I think it shows.

Final Thoughts

So that’s what I’ve got so far! It would be helpful to get feedback from users. What kind of questions you ask, what responses you get, what you liked and didn’t like. Feel free to open an Issue on GitHub or message me. Thank you!

Files

Comments