Adding an AI help bot to a WordPress product is easy now. Point it at your documentation, give it an API key, done. It'll work beautifully for about a month.
Then you ship a release. A button gets renamed, a default gets retuned, a feature moves. Nothing breaks, no error appears. The bot just quietly starts giving confident, fluent, wrong answers about your own software. You find out when a customer emails to say your help system sent them hunting for a setting that doesn't exist.
I've just built one of these for a plugin I've been working on, and the AI part took an afternoon. Everything else was solving that problem. Here's the order I'd do it in if I were starting again.
Step 1: Decide who's actually asking
Before writing a word of knowledge, get clear that support splits in two, and the split is awkward.
Someone opening your plugin for the first time asks "how do I make the text bigger?" They want to know which button to press.
Someone else asks "why can't I set any font size I like?" They want the reasoning. In my case, that sizes deliberately step through the theme's presets so pages stay on the design system.
A documentation page has to pick one and disappoint the other. A bot doesn't have to. That's the actual reason to build one. Not that it's faster than search, but that it can read the question and pitch the answer.
So: write your bot's instructions to cover both, and tell it explicitly to judge from the phrasing which to lead with. Mine has three modes: automatic, beginner, developer. Automatic is the one that matters. The other two exist so I can check it has the material for both extremes.
Step 2: Split what it needs to know into two piles
This is the step everything else depends on, so it's worth doing slowly.
List everything your bot needs to know. Then sort each item into one of two piles.
Pile one: explanation. Why your plugin works the way it does. What the constraints are for. What the design is trying to achieve. Why a thing that looks like a bug is deliberate.
This is the part that makes a bot useful rather than a search box. No script can write it. And here's the good news: it barely changes between releases. The reasoning behind a feature is stable even when its label isn't.
Pile two: facts. The exact wording of every button, tooltip, error and menu item. Default settings. Version numbers. Which features are switched off unless you enable them.
This pile changes constantly. It's also the pile that's already written down somewhere that can't be out of date: your source code.
The insight: you only have to hand-write pile one.
Step 3: Let the code write the second pile
Get a script to read your plugin's own source and pull out every button label, every message, every default value, word for word. Have it write them into a file. Run it before every release.
This sounds like more work than updating a doc by hand. It's the opposite. It's work you do once instead of work you must remember forever. And the "remember forever" version has never worked for documentation in the history of software.
Mine pulls out around fifty message strings, forty-odd button labels, every default setting and the version number, all straight from the code. I don't type any of them. If I reword a tooltip on a Tuesday, the bot is quoting the new wording by Friday's release without me thinking about it.
The trap: a script that reads code will eventually break when you restructure that code. Make sure it fails loudly rather than quietly returning nothing. Mine stops the release and tells me exactly which bit it can no longer find. A help bot that's silently missing half its facts is worse than one that won't build.
Step 4: Tell the bot which pile wins
This is the small step that makes the whole thing safe, and it's one line.
Your bot's instructions should say, explicitly: where the generated facts disagree with the written explanation, the generated facts are correct.
Think about what that buys you. Your hand-written explanations will fall behind. You're a person, releases happen. But when they do, the bot's reasoning gets slightly dated while it still quotes every button label exactly right, because those came from the code this morning.
The half that can go wrong is permanently outranked by the half that can't.
Stale writing makes your bot a bit less insightful. It never makes it wrong about what's on the screen. That's the difference between a bot you can leave running unattended and one you have to babysit.
Step 5: Make it update itself
Now wire it together so nothing depends on you remembering.
The pattern that works: the bot doesn't carry its knowledge around inside it. It fetches it fresh, from wherever you publish it, and caches it for a few minutes.
That means when a release regenerates the facts, the live bot picks them up on its own. No redeploy, no button to press, no step to forget. The bot is infrastructure and changes rarely. The knowledge changes every release. Keep them separate and only one of them needs your attention.
Bonus you get for free: once facts are extracted mechanically, you can compare this release's set to last release's. Which labels vanished? Which defaults moved? That's a rather good changelog, and more usefully a nudge list: this thing you wrote three paragraphs about no longer exists. Your explanations still drift, but now something tells you where.
Step 6: Put it somewhere that can hold a secret
Here's where people get caught out, so it's worth being precise.
You cannot host this as a plain HTML page. Not on GitHub Pages, not dropped on your web host as a file. Anything served as a static page is readable by anyone who views the source, including your API key, which is the thing that pays for every question anyone asks.
You need something that runs code, so the key stays on the server. That doesn't mean a server to maintain. I used a free Cloudflare Worker, which took one click and a form.
Do check the boring option first. I wrote a ten-step deployment guide before discovering there was a one-click button that did the same job and asked for the key in a form. Six of my ten steps deleted themselves.
Step 7: Set the limits before you share the link
Two of the three things I got wrong were here, so please read this bit even if you skim the rest.
A per-person limit does not protect your wallet. I capped how many questions any one visitor could ask per day. It felt responsible. It's useless against the thing that actually costs money: a thousand people politely asking two questions each are every single one of them under the limit, and you've just paid for two thousand questions. You need a cap on the total, not per person. That's the number that bounds your bill.
Check the free tier's actual limits, don't assume. My rate limiter needed two small database writes per question. The free plan allowed a thousand writes a day. I'd set the daily question cap to fifteen hundred. So my safety mechanism would have run out of budget and started failing at precisely the moment it was needed. I only caught it by looking up the published limits rather than trusting my memory.
Set a hard spending limit on your API account. Not a rate limit, an actual spending cap, on the billing page. Rate limits bound how many questions get asked. Only a spending limit bounds what you pay. Do it before the link goes anywhere public.
Step 8: Link it from inside the product
Last step, and it's where an in-product bot beats a docs page.
Put the link where someone is when they get stuck. In my case a ? next to the undo button, not buried in a support menu.
Two small touches worth the effort:
Pass the version. Add the plugin version to the link. The bot then knows which release the person is actually running, rather than assuming they're on the newest, which they very often aren't.
Open it in a new tab. If someone has unsaved work on screen, navigating away from it is a genuinely bad moment to be helpful.
What it costs
The hosting is free. The AI usage came to a fraction of a penny per question. The bulk of what gets sent is the same knowledge base every time, and that gets billed at roughly a tenth of the normal rate after the first send.
With the caps set sensibly, the worst case is bounded and predictable. Which is really the point: I'd rather have a help bot that can tell someone it's reached its limit for today than one that can quietly empty my account.
The one question to ask on day one
Not "can it answer questions?" They all can. That's the easy part now.
Ask: what does this look like six releases from now?
If the honest answer involves you remembering to do something, it's already broken. Work out which half of your knowledge a machine can read for itself, let it, and tell the bot which half to trust.
Leave a Reply