How I Work: The Approach Behind the AI Shortlist Audit.
Customized to your five best clients
Simulates your buyers telling AI their requirements
Measured against the six durable buyer questions
The AI Shortlist Audit tests what AI recommends by simulating your buyer's conversation with AI. A company cannot see the full conversations its own buyers have with AI, so every test it runs is a simulation. What matters is how close the simulation is to your buyer.
I build mine from an interview with your CEO or sales team about your five best clients, so the audit is customized to your company. Each simulated buyer conversation includes what a buyer tells AI: who they are, what problem they are trying to solve, and what they are looking for in a vendor. I then compare your website against the six buyer questions, which are the questions B2B buyers have been asking for a long time.
This is a different job from AEO, GEO, and AI visibility tracking. I am David Lee of Do What Works, and the AI Shortlist Audit page describes the meetings, the timeline, the deliverables, and the fee.
There Are Several Ways to Simulate a Buyer's Conversation With AI
A buyer's conversation with AI is private. A company cannot see the full conversations its own buyers have with AI, so it cannot test them directly. Every test it runs is a simulation of the actual conversation. In each one, the prompt is a stand-in for what the buyer said. AI's answer to it is real.
How close the simulation is to your buyer matters, because AI's recommendation depends on what the buyer tells it about their situation. Simulations differ in three ways: where they come from, what they contain, and how many turns they take.
Where a Simulation Comes From
Choosing the source for a simulation is a trade-off between how close it is to your buyer and how much effort it takes. A source is close when it is about your buyers, in the buyer's own words, and about what they needed when they built the shortlist.
Source | How close it is to your buyer | Effort |
|---|---|---|
The company's website | Far | Low |
A category or topic | Far | Low |
Search keywords, rephrased as questions | Far | Low |
Panels of real AI users | Closer in language | Low |
The company's own records, such as sales call transcripts | Close | Medium |
Interviews with the company's CEO or sales team | Close | Medium to high |
Interviews with the company's customers | Closest | High |
What a Simulation Contains
A simulation contains a short question about a topic or category, a question with a persona, or the buyer's requirements. Each has a strength and a limit.
Contents | Strength | Limit |
|---|---|---|
A short question about a topic or category | Quick and cheap to produce, so many can be tracked across a market over time | It has the topic, not the buyer. It does not say who the buyer is, what problem they are trying to solve, or what they are looking for in a vendor |
A question with a persona: a typical buyer's role, goals, and problems | Gives AI a buyer to answer for, and is still quick to produce | It describes a typical buyer, not your buyer. The persona is written by a tool or by marketing, not supplied by a buyer |
The buyer's requirements: who they are, what problem they are trying to solve, and what they are looking for in a vendor | Gives AI what a buyer would tell it about their situation | It is one wording among many, and it needs a source that knows the buyer |
How Many Turns a Simulation Takes
A simulation takes one turn or several. Each has a strength and a limit.
Turns | Strength | Limit |
|---|---|---|
Single turn: one prompt | Easy to repeat and compare over time | It stops after AI's first answer. It does not show what happens when the buyer asks a follow-up |
Multi-turn: a scripted conversation | Multi-turn, like a real conversation. Shows if a vendor stays on AI's list as the buyer narrows | Each script is one path, and the path changes the answer. Covering more paths takes many scripts and many runs |
How the AI Shortlist Audit Simulates Your Buyer's Conversation With AI
The AI Shortlist Audit is built on ten simulated buyer conversations. They come from an interview with your CEO or sales team about your five best clients. Each one includes the buyer's requirements and runs as one prompt.
Why I Interview Your CEO or Sales Team
I build the simulation from an interview with your CEO or sales team. I chose it as a balance: it is close to your buyers, and it asks for your team's time, not your customers'.
While interviewing your customers directly would be the closest source, it takes the most effort.
Sales call transcripts would be another option, but a sales call is a Job 3 conversation. By then the buyer has usually built the shortlist and is asking validation questions, not shortlisting questions.
So I treat the CEO or the sales team as a close enough proxy for the buyer's requirements. They know who the client is, what problem the client came to solve, and what the client was looking for in a vendor.
The interview takes two hours, and it is detailed on the AI Shortlist Audit page.
Why the Interview Is About Your Five Best Clients
The interview is not about all of your customers. It is about five of your best clients.
I ask for five because the goal is to simulate your best clients, not to measure a market. Five is not a statistical sample, and it is not meant to be.
I am deliberate about asking you which five the interview covers, and I am careful not to define what "best" means. I leave that to you. Best could mean:
Your most profitable clients
The clients that bring you the most revenue
The clients that are the easiest and most enjoyable to work with
I want you to define it, because these are the type of clients you want more of.
Building the audit around your five best clients keeps it customized to your company. The audit shows what AI does when buyers like your best clients ask for a vendor shortlist. The audit's recommendations for your website are written for what those buyers want, not for a typical buyer in your market or industry.
How a Single-Turn Prompt Simulates a Buyer's Multi-Turn Conversation
My single-turn prompt simulates a buyer's multi-turn conversation by including all of the buyer's requirements: who they are, what problem they are trying to solve, and what they are looking for in a vendor. The 3 Buyer Jobs Framework explains the three inputs a researcher needs.
A company cannot know the exact words its buyers used with AI, or how many turns the conversation took. It can know what any researcher, a person or AI, needs from a buyer to make a recommendation: the buyer's requirements.
I build ten simulated buyer conversations from the transcript of the interview, two for each of your five clients. The two describe the same engagement from a different angle. Each prompt is usually over 300 words, because that is how many words it takes to detail all three parts of the buyer's requirements.
One prompt also keeps the test simple enough to repeat, and it gives ChatGPT and Claude the same prompt. A multi-turn script cannot do that, because each model's later turns build on its own earlier answers.
The simulation ends at the shortlist. A buyer's follow-up questions about one vendor belong to Job 3: Validate Vendors, which the audit does not cover.
How I Run the Simulated Buyer Conversations
I run all ten simulated buyer conversations on ChatGPT and on Claude. Each one runs once on each model, which gives twenty sets of results.
One thing to note on all testing of AI responses: AI is non-deterministic by design. The same prompt can return a different answer each time it runs, and the two models sometimes return different vendors. The audit reports each model's results separately and gives a summary of both. It is a record of what AI did at one point in time.
That is enough to act on, because the audit does not ask you to chase a placement that changes from week to week. The audit asks you to answer the six durable buyer questions on your website, which are the questions that B2B buyers have been asking for a long time.
What the Audit Compares and Measures
The audit covers two of the three buyer jobs: Job 1: Discover Vendors and Job 2: Build the Shortlist.
For Job 1: Discover Vendors, the audit records what AI did in each of the twenty sets of results: if AI discovered you, which vendors it discovered and recommended, the keyword phrases it generated, and the listicles and directories it used.
For Job 2: Build the Shortlist, I compare your website against the six buyer questions. The audit gives you three scores:
Content Quality
How well your website answers the six buyer questions.Content Organization
If each answer is on the page that owns it. Website as a Database explains why that matters to AI.Recommendation Readiness
How likely AI is to recommend you once it has read your website. It is stated as a likelihood, because AI is non-deterministic.
What Do You Do With the AI Shortlist Audit?
The audit is only valuable if you act on it. Your team or agency works through the reference document and makes five changes:
Each of the six buyer questions gets one page on your website, so AI has one complete answer to work from. The Foundational Pages describes the six pages.
The answers that were missing get written, and the answers on the wrong page move to the right one.
Your website is organized like a database: one page owns each topic, and each section answers one question.
The keyword phrases AI generated from your simulated buyer conversations are worked into the copy of the page each one belongs on.
Your team pitches the listicles and claims the directory profiles AI used to discover vendors like you.
Acting on the audit increases the probability that AI discovers and recommends you when buyers like your best clients use AI to build a shortlist.