Vendor evaluation for AI tooling tends to produce long feature-comparison spreadsheets that answer the wrong question. A feature matrix tells you what a vendor's product can do today. It does not tell you whether adopting it is the right decision for a specific organisation with a specific problem, timeline and internal capability.
Three questions, asked before the feature comparison, usually reveal more.
What happens if this vendor is wrong in six months?
AI tooling changes faster than most procurement cycles account for. The relevant question is not whether a vendor's current capability is good, but what it costs to leave if a better option appears, or if the vendor's roadmap diverges from what the organisation actually needs. A platform that locks proprietary data into a format only it can read is a different commitment than one built on open, portable standards, even if the two look similar in a demo.
In practice. Ask what the data export path looks like before signing anything. If the honest answer is "there isn't a clean one," treat that as a cost, not a footnote.
Is the problem well-understood enough to buy a solution for it?
Buying works well when the problem is well-defined and the vendor has solved it for organisations genuinely similar to yours. It works poorly when the problem is still being discovered — when the real requirements will only become clear after a few months of internal use. In the second case, a narrower internal proof of concept, even an imperfect one, teaches more than a vendor pilot does, because it surfaces the organisation's actual constraints rather than the vendor's assumptions about them.
Does building this in-house create a capability worth having?
Building makes sense when the resulting capability is close to what makes the organisation distinctive, and the team has the ongoing capacity to maintain it. It makes considerably less sense for commodity functionality a vendor already does well, where the "build" option quietly becomes an unstaffed internal product with no roadmap.
- If the capability is core to what the business does differently, lean toward building or heavily customising.
- If it is important but not distinctive, lean toward buying, with a clear eye on the exit cost above.
- If the requirements are still unclear, prototype before you procure — for either path.
Waiting is a legitimate answer
Not every AI opportunity needs to be decided this quarter. If the problem is not urgent, the tooling is immature, or the organisation is not yet ready to operate what it would adopt, waiting and re-evaluating in six months is a defensible strategic choice, not an admission of indecision.
References
An assessment framework used in AFC advisory engagements is available on request.