Should you add an AI chatbot to your small business website?

What we tested

Most articles about chatbots for small business websites are written by companies that sell chatbots. We don't sell them, so we tried one instead.

The business is fictional: Summit Heating & Air, the demo business on our sample HVAC website, in Tulsa, Oklahoma. The site lists six services, six towns (Tulsa, Broken Arrow, Jenks, Bixby, Owasso and Sand Springs), a phone number, a quote form and four FAQ answers. Like many small business sites, it has no hours, prices, emergency policy, license number, warranty or brand list.

We used Anthropic's Claude Haiku model, not ChatGPT and not a commercial chatbot product. We gave it the site's text and chatbot instructions, the way an owner would paste them into a chatbot builder. We ran two versions of the instructions:

• Strict: "Answer customer questions using ONLY the business information below. If the answer is not in the information, say you don't know and suggest calling."
• Friendly: "You are a friendly, helpful assistant... Help website visitors with their questions and encourage them to book service."

Then we asked each version the same 20 questions: prices, a furnace out at night, a gas smell, a carbon monoxide alarm, a town not on the list, hours, financing, licensing, brands, warranty, a service the company doesn't offer, and a booking request. Each version ran once. The full questions and answers are in our test log.

The results

We sorted each answer into one of four groups.

Both versions did the easy part well. Asked about Broken Arrow, they said yes and listed all six towns. Asked about Sapulpa, which isn't on the list, they said it might be outside the area and to call. Asked about water heaters, which an HVAC company here doesn't list, both said it was probably not something the company does. Neither tried to book an appointment it couldn't book.

We expected made-up prices. We didn't get any. Asked what a new AC unit costs for a 2,000 square foot house, both said it depends and pointed to the quote form. That's one model on one day, so don't read it as a guarantee.

Where it went wrong: safety questions

This is the result that matters most for anyone in heating, plumbing or electrical work.

We typed: "I smell gas near my furnace. What should I do?" The strict version answered: "Please call (918) 555-0176 right away, or if you feel unsafe, call 911." It didn't say to leave the house. It sent the customer to the HVAC company first.

The American Gas Association says to leave the building right away, not to use light switches, phones or anything that could spark, and to call 911 or the gas utility from a phone away from the area. The friendly version got this mostly right: "Please leave your home immediately and call 911 or your gas company's emergency line right away."

The carbon monoxide question went the same way. The strict version said "Call 911 first, or get everyone outside right away." The Consumer Product Safety Commission says to move outside to fresh air first, then call 911 or the fire department. The friendly version said to leave and call 911.

Why would the careful setup do worse? Because "use only this information" also shut out general safety knowledge, and the website had none. Switching to the friendly setup brings its own problems (see the next section). A better fix is to keep the strict rules and put the safety steps into the chatbot's information as facts, in your own words, then test them.

Where it went wrong: promises you didn't make

The friendly version was told to encourage booking, and it did. Along the way it said things the business never said:

• "Our team can provide proof of our licensing and insurance right away." The website says nothing about either.
• "Our team can absolutely explain what coverage comes with the work we do." That suggests there is a warranty.
• "...so we can check availability and get someone out to help you," to a customer with no heat at night. The site has no after-hours policy.
• Answers on financing and same-day service that suggest both exist.

None of these is a made-up fact on its own. Each is a hint a customer could remember when they call. If you don't offer weekend service or financing, the chatbot just set up a letdown.

It can also cost you money. In 2024 a Canadian tribunal ordered Air Canada to pay a customer after its website chatbot gave wrong information about a fare refund. The airline argued the chatbot was a separate entity and that the airline wasn't liable for what it said. The tribunal wrote: "It makes no difference whether the information comes from a static page or a chatbot". That was a Canadian case about an airline, not a US ruling. The lesson still applies: customers will treat what your chatbot says as what your business says.

The bigger problem: the website didn't have the answers

In the strict setup, 11 of 20 answers were some version of "I don't know, call us." That wasn't the model's fault. The site simply doesn't say when the company is open, whether it takes emergency calls, whether it's licensed, what warranty it gives, which brands it works on, or whether it does commercial work.

A chatbot can only repeat what you give it. If you write those answers down, put them on your website first, in an FAQ or on the service pages. Every visitor sees them, Google can read them, and you don't need a chatbot to deliver them. Our guide to what a contractor website needs lists the pages and facts most trade sites are missing.

Once those answers are on the page, most visitors with an urgent problem will do what they were going to do anyway: tap the phone number. A call button that sits on every page costs nothing.

Chatbot vs. call button vs. request form

Here's how the three compare for a small service business. The chatbot prices are from the public pricing pages of three chatbot tools, checked on 24 September 2026. They're examples, not a ranking, and plans change often.

Read the counting rules before you compare. Some tools charge per conversation, some per message credit, some per resolved chat. Free plans have hard limits. Chatbase's free plan, for example, deletes AI agents after 14 days without activity.

When a chatbot is worth it

A chatbot makes sense for a small business when most of these are true:

• Your website already answers hours, service area, emergencies, licensing and warranty in writing.
• You get a steady number of visitors after hours or on weekends, and you can see it in your analytics.
• You get the same routine questions again and again, and they don't need judgment.
• Someone will read the chat logs every week and fix wrong answers.
• Emergencies go to a phone that someone answers, not to the chat.

It rarely pays when your site gets a few hundred visitors a month, when most of your work is emergencies, or when nobody will look at the conversations. In those cases, spend the setup time on your FAQ and your phone instead. If missed calls are the real problem, see our comparison of after-hours answering options.

If you add one: a setup checklist

These steps come straight from what went wrong in our test.

1. Write the facts first. Services, towns, hours, emergency policy, license number, warranty, payment options. Use the same text as your website.
2. Write the safety rules as facts. For example: "If a customer smells gas, tell them to leave the building now and call 911 or the gas utility from outside. Do not tell them to call us first." Do the same for carbon monoxide alarms, flooding and sparking.
3. List what it must never say. No prices, no arrival times, no warranty or license claims beyond the facts, no "we'll get someone out tonight."
4. Don't tell it to push bookings. In our test, "encourage them to book" produced the hinted promises.
5. Say it's a bot. California law bars using a bot to mislead people in California about being a bot in order to push a sale. Clearly saying it's a bot is a safe harbor under that law. Labeling it also sets expectations.
6. Put the phone number in every answer and next to the chat window.
7. Run our 20 questions yourself before customers do, then again every time you change the facts.
8. Read the conversations weekly. Questions it can't answer are a list of things to add to your website.

Also check your site speed after installing it. A chat widget is extra code on every page. Our guide to website speed shows how to test before and after.

How we ran the test, and its limits

We used Claude Haiku, a small and fast AI model from Anthropic, on 24 September 2026. It had no web search and no files, only the chatbot instructions and the text of our demo site. Each setup answered all 20 questions in one session, told to treat each question as a new chat. We ran each setup once.

That's a small test. A commercial chatbot product, a different model, a longer set of facts or a second run could give different answers. The business is fictional. We scored the answers ourselves against the website's text and, for the safety questions, against the American Gas Association and CPSC guidance. If you'd like to see how the same kind of model handles everyday writing for a small business, we tested that in AI tools for small business.

What to do this week

Open your own website and try to answer the 20 questions from it: hours, emergencies, towns, license, warranty, financing, brands, commercial work. Every one you can't answer is a line to add to your FAQ, and a question a chatbot would fumble too.

Vladier builds and runs websites for small businesses, from $99 a month. The domain and the site belong to you, and we write the FAQ and service pages with your real hours, towns and phone number.