The seven questions I ask an AI vendor before signing

I was a CTO for twenty years. In that time I bought technology, sold technology and cleaned up after purchases other people made. Today I am on the side that sells AI agents for customer service and sales, and I am still on the buying side too, as a board member and advisor.

Both sides taught me the same thing: almost every disappointment with an AI vendor was decided before the contract was signed, in a question nobody asked.

These are the seven I ask.

1. What happens when the model gets it wrong?

Not if. When. Every model makes mistakes, including mine.

The answer that matters is not about accuracy. It is about what the company has built around the error. Is there a rule that stops the agent from stating something that is not in the knowledge base? Is there a log of what it said, retrievable in thirty seconds? Is someone alerted when it gets things wrong, or does the company only find out when a customer complains?

A vendor who answers this question with the model's accuracy didn't understand the question.

2. Who owns the data, and where does it live?

Who is the data controller. Where the data is stored. What happens to it if the contract ends tomorrow. Whether customer conversations are used to train models, and how that is switched off.

This is not legal red tape. It is the difference between being able to switch vendors in two years and being locked in because the entire history of your customer relationships lives in someone else's house.

3. How do I turn it off?

The most revealing question on the list and the one asked least.

If the agent starts getting in the way on a busy Thursday, how long does it take to pull it without bringing down the whole service? Is there a switch? Do you have to open a ticket? Do you have to wait for someone to wake up?

A system without a shutdown plan has no contingency plan. And anyone who has spent a night trying to roll back a deployment knows exactly why this question comes before price.

4. What does the price track?

Seats, conversations, messages, resolutions, model usage. Each pricing model rewards a different behavior.

If the vendor charges per message, it has no incentive for the agent to be concise. If it charges per seat, it has no incentive to automate. If it charges per resolution, there needs to be a clear, auditable definition of what counts as resolved, or the invoice becomes an argument every month.

There is no right model. There is a model aligned with the result you want, and it is worth understanding the incentive before signing.

5. Who answers when it breaks?

Not the support channel. The person.

What response time is committed in the contract, what happens if it is missed, and whose name picks up when the problem is big. A small vendor with the founder on the phone often solves things better than a big vendor with a ticket queue. But that has to be clear beforehand, not discovered in a crisis.

6. How does it talk to what I already have?

CRM, ERP, telephony, product catalog, inventory system. Integration is where timelines go to die.

The useful question is not whether it integrates. It is how. A documented, stable API, or an automation someone stitched together that breaks when the other vendor updates its screens? Who maintains it? What happens when your CRM ships a new version?

7. Can I talk to a customer who gave up?

Every vendor has happy references.

What teaches you something is the customer who tried it and stopped. Why they stopped, what didn't work, whether it was a product problem or an expectations problem. A vendor who agrees to that conversation is confident in what it sells. A vendor who freezes at the question has just given you the information you needed.

The takeaway

None of these questions is technical. All of them can be asked by whoever signs the budget without understanding a line of code.

That is on purpose. Buying AI is not a technology decision. It is a decision about risk, dependency and reversibility, and all three fit entirely in the head of the person running the business.

If the vendor answers all seven clearly, you are probably buying well, even if its model is not the most advanced on the market. If it dodges three, the most advanced model on the market won't save you.