There are now many companies offering AI receptionists, chat agents, automations and custom software. Some are excellent. Some are new. Some are both. Choosing well matters, because the agency you pick will touch your customers, your data and your phone numbers.
This guide is a practical checklist for evaluating any AI automation agency, including us. It is meant to help you ask better questions, not to steer you toward a particular answer.
Start with your own goals
Before you talk to any agency, write down:
- The problem. Missed calls, slow lead response, manual data entry, scattered tools.
- What success looks like. More booked appointments, faster replies, fewer hours on admin.
- Your constraints. Budget range, timeline, internal team capacity, compliance needs.
- What you will not automate. Some conversations should always involve a person.
A clear brief makes proposals easier to compare and helps good agencies give you useful advice.
Questions to ask about the work
Discovery and scope
- How do you learn our business before building anything?
- What will you deliver in the first phase, and what comes later?
- Which parts are custom, and which use existing platforms?
- What do you recommend we not automate, and why?
A thoughtful agency will push back on some of your ideas. That is a good sign.
Build quality
- How do you test agents before they talk to customers?
- Can we hear sample calls or see sample conversations before launch?
- How do agents handle questions they cannot answer?
- How is human handoff designed and tested?
Maintenance and ownership after launch
This is where many engagements succeed or fail.
- Who monitors the agents after launch?
- How often are transcripts and call recordings reviewed?
- How are changes handled when our prices, hours or services change?
- What happens if a platform the agent depends on changes or goes down?
AI agents are not “set and forget.” Prices change, policies change, and customers ask new questions. Someone needs to own ongoing review, whether that is the agency or a person on your team with clear training.
Questions to ask about data and accounts
| Question | Why it matters |
|---|---|
| Whose accounts are the phone numbers, CRM and messaging platforms in? | If the agency owns them, leaving can be painful |
| Can we export all our data, including conversations and recordings? | Your customer history should stay yours |
| Where is customer data stored, and who can access it? | Privacy, security and regulatory obligations |
| Which AI providers process our data, and under what terms? | You should know where conversations go |
| What happens to our data if we end the engagement? | Clean exit terms prevent surprises |
A strong default is that core accounts, such as your CRM, phone numbers and messaging registrations, are in your business’s name, with the agency given access to manage them.
Questions to ask about compliance
AI agents that call and text people operate under real rules. Ask:
- How do you handle A2P 10DLC or toll-free registration for texting?
- How do agents disclose that they are AI?
- How is consent captured and stored?
- How are opt-outs honored across every channel?
- How do you approach the TCPA for outbound calls, and CASL if we serve Canada?
- Do you recommend we involve our own counsel? (A good answer is usually yes.)
An agency does not need to be a law firm, but it should understand the basics and build with compliance in mind.
Red flags to watch for
Most agencies mean well. Still, a few patterns deserve caution:
- Promises of specific results. No honest provider can promise a set number of leads, sales or rankings. Results depend on your market, offer and operations.
- No discussion of consent or disclosure. Especially for outbound calls and texts.
- Agency-owned everything. Your numbers, data and accounts locked inside someone else’s setup.
- Vague maintenance. “We’ll set it up and you’re good to go” usually means nobody watches it after launch.
- No way to hear or see the agent. You should be able to test it before customers do.
- Pressure to sign quickly. A good fit can wait a day for you to think.
- One tool for everything. Good agencies choose tools for your needs, not the other way around.
Green flags
- They ask detailed questions about your operations before quoting.
- They suggest a small first phase with clear success measures.
- They talk openly about limitations and edge cases.
- They put accounts in your name and document what they build.
- They describe a regular review process after launch.
- They are comfortable with you speaking to counsel.
Comparing proposals fairly
When you have two or three proposals, compare them on the same dimensions:
| Dimension | Agency A | Agency B | Agency C |
|---|---|---|---|
| Problem understood clearly | |||
| Phase one scope and timeline | |||
| Testing before launch | |||
| Ongoing review and maintenance | |||
| Data and account ownership | |||
| Compliance approach | |||
| Exit terms | |||
| Total cost over the first year |
Looking at the first year, not just setup, gives a more honest picture. Ongoing platform costs, usage costs and maintenance often matter more than the initial build.
A note on newer agencies
The field is young, so many capable agencies are new. That is not a problem by itself. Ask them to show their thinking: a walkthrough of how they would approach your use case, a demo agent, and a clear explanation of their process. Judge the quality of their questions as much as their portfolio.
FAQ
Should we build AI automations in-house instead?
It can make sense if you have technical staff with time to build and maintain them. Many businesses start with an agency for the first build and train an internal owner to manage day-to-day changes.
How long should a first project take?
It depends on scope. A focused first phase, such as one agent on one channel with a CRM connection, is usually much faster than a broad rollout. Ask each agency for a phased timeline.
What should be in the contract?
At minimum: scope, deliverables, account and data ownership, maintenance responsibilities, response times for issues, confidentiality, and exit terms including data export. Have counsel review it.
If you want an outside look at where AI fits in your business before choosing anyone, our AI Audit & Roadmap is a good starting point, or you can book a strategy call.