AI Readiness Before Automation

1. Define the task, not the technology

Start with the decision or piece of work the AI should help with. Good candidates are specific and frequent: classify incoming support tickets, extract fields from invoices, draft replies from a knowledge base, summarise long documents, or route requests to the right team.

Vague goals such as "use AI in operations" lead to vague results. Write the task as an input, an expected output, and who uses that output next.

2. Check the data the AI will depend on

AI features are only as good as the information they can see. List the data sources involved, where they live, how current they are, and who is allowed to access them.

• Is the knowledge written down, or does it live in people's heads?
• Are documents up to date, or are there several conflicting versions?
• Can the data be accessed through an API or export, with permissions respected?
• Does it include personal or sensitive information that must be protected or removed?

3. Agree what a good answer looks like

Collect a set of real examples with the correct output for each. Twenty to fifty examples are often enough to start. This evaluation set becomes the test that every prompt, model, or retrieval change must pass.

Also agree the acceptable error rate. Drafting an internal summary can tolerate occasional mistakes; approving a payment cannot. The tolerance decides how much automation is appropriate.

4. Place a human review point

Decide where a person checks, edits, or approves the AI output, at least until the system has proven itself. Common patterns are: AI drafts and a person sends, AI suggests and a person approves, or AI acts automatically only above a confidence threshold and escalates the rest.

Design the interface so reviewing is fast. If checking an AI result takes as long as doing the work manually, the automation will not be used.

5. Choose the right approach: prompts, RAG, or agents

Simple transformation tasks such as rewriting or classification often need only a well-designed prompt. When answers must reflect your own documents or policies, retrieval-augmented generation (RAG) lets the model look up relevant passages and cite them. When the task needs several steps across tools, an agent can call your APIs, but it needs strict permissions, logging, and approval steps.

Start with the simplest approach that meets the quality bar. Complexity can be added once the evaluation set shows it is needed.

6. Plan for cost, latency, and failure

Estimate how many requests the feature will handle and what each costs. Set per-user limits where usage could grow unexpectedly, and decide how fast responses need to be.

Plan for failure too: provider outages, timeouts, and low-confidence answers. A good AI feature degrades gracefully, for example by falling back to another model or handing the task to a person with context.

7. Measure business results

Define how you will know the AI feature is worth keeping: minutes saved per task, tickets resolved without escalation, faster turnaround, or fewer errors. Track these from the first pilot so decisions are based on evidence rather than impressions.

AI readiness checklist

If you can answer these questions, your workflow is ready for an AI pilot.

• What exact task will the AI perform, with what input and output?
• Which data sources does it need, and are they accurate and accessible?
• Do we have an evaluation set of real examples with correct answers?
• What error rate is acceptable, and where does a person review the output?
• Is a prompt, RAG, or an agent the simplest approach that works?
• What are the cost limits, fallbacks, and success metrics?