Four questions to ask before buying any AI tool
The demo will always look good. Demos are built to look good. These are the four questions we ask before we put any tool into one of our own businesses, and the answers tell you more than an hour of features ever will.
1. What happens when it is not sure?
This is the question almost nobody asks, and it is the one that matters most. Every AI system will eventually hit an input it cannot read, an API that times out, or a quota that ran out at 3am.
There are only two possible designs. It can fail closed — refuse to act, flag it, wait for a human. Or it can fail open — guess, and publish the guess. A system that guesses when it is unsure is worse than no system, because now you have wrong information wearing the costume of verified information.
Ask the vendor directly: “Show me what happens when your model returns garbage.” If they have not thought about it, they have shipped a fail-open system and have not noticed yet.
2. Who owns the data, and can you get it out?
Not “can you export a CSV.” Everyone can export a CSV. The real questions are whether your customer history, your notes, your pricing, and your operational record leave in a form another system could actually import — and whether your data is being used to train models that serve other customers, including your competitors.
The answer changes what the tool costs. A tool you cannot leave is not priced at its monthly fee. It is priced at whatever they decide to charge you the year you can no longer afford to migrate.
3. What can it do without a human saying yes?
Draw a hard line between work and judgment. Automating the work is the point — drafting, sorting, summarizing, scheduling, reminding. Automating the judgment is how businesses end up apologizing to customers for something nobody chose to send.
Our rule, in our own systems: anything that reaches a customer or spends a dollar passes a human approval gate. Everything upstream of that can run unattended. Ask where the vendor draws that line, and whether you can move it.
4. Does it answer questions nobody anticipated?
Every tool answers the questions its vendor built reports for. The valuable question is always the one that shows up later, specific to your business, that no report covers.
We found a serious problem in our own storefront only because we could join three things nobody designed to be joined: what search engines were being served, what the catalog contained, and what customers actually bought. No dashboard offered that. Owning the pieces is what made the question askable.
And the honest counterpoint
Sometimes the right answer is to buy the boring tool and get on with your day. Accounting, payroll, and email are solved problems — you will not out-build them and you should not try. Build when the way you work is genuinely different from everyone else in your industry, or when the thing you need to ask is the thing no vendor will ever build a report for.