I Asked ChatGPT the Same Question Three Times. I Got Three Different Answers.
Wondering how to actually get cited and recommended by AI? As this title suggests, you may ask the same query on a different day, will you get the same answer? If not how do you actually get AI to refer to you and your business when someone searches something that you do? Read on to find out… (or skip it all scroll straight to the bottom to read ‘So what can you do?’)
There is a whole world of information out there and I don't know about you, but I long for simplicity in this digital age of information coming at me 24/7. So lets keep this post as simple as possible! You may not know that getting cited and/or recommended by AI are two different things. Let's start there in defining what are we actually trying to do:
Get cited
This means that your words were useful. AI pulled text from your page and shows your link as a source, which allows people to click that link, that will take them directly to your webpage which AI has said contains useful information in regards to their query.
Get recommended
This means your business was named as the answer to the question the person typed into their AI search. For example, someone types into search; I am looking for an AI search specialist in Kerikeri. AI answers with: "Try Luminera in Kerikeri."
There are some simple facts and commonalities here to work towards:
be useful on your website. Useful, factual, information first content.
be the answer. Content that is centred around what your customers are typing, who they are and what they want, answer their questions in their words
Where Does AI Get It’s Information??
According to Semrush;
“Several AI companies have struck licensing partnerships allowing their AI systems to use partner organizations’ content for training and live retrieval. For example, OpenAI and Google have partnered with Reddit and Stack Overflow. OpenAI also licenses content from organisations like Yelp and Time. Content from partner organisations often appears prominently in AI systems’ answers.”
We decided to do a similar test but in New Zealand to see what we would come back with. We came back with the same results, proving exactly what Semrush describes. In this case, it's that when someone is asking for recommendations in an area, ChatGPT is citing from Reddit. It is citing what real people are saying on the Reddit forum about where they think you should eat. Think about that, just how relevant it is what people are saying about you on the internet, not just what your website says about you.
Three Tests, Same Query, One Commonality: Reddit
Test conditions and method: All three queries I was logged out of ChatGPT. I copy pasted the same query. One was three hours later, one five minutes after that.
Test one: a map, and a real business named
As you can see here, Reddit has been cited when I search for 'cheap eats in nz according to locals' in ChatGPT. This also demonstrates the difference between being cited and recommended, as Tipene's Fish And Chips is what's been recommended, and Reddit is what has been cited, as it's their link that follows the recommendation here.
And this is what's fascinating when we talk about what actually works, Tipene's did nothing here except for be a great business, and was talked about on Reddit, earning this recommendation. You cannot buy your way into a recommendation on Reddit, and fake reviews get removed. Which is why being authentic, and being really great at what you do. Especially in New Zealand with so many small businesses all doing their best. This is your reminder, to keep on doing that, being you! And just how important that is.
Test two: no map, no businesses, still Reddit
To further the experiment I then asked ChatGPT the exact same query 'cheap eats in NZ according to locals' just 3 hours later. I got back a totally different format and response to the same query. No businesses were recommended this time. But still every choice ChatGPT made, was based on Reddit reviews.
As you can see, in this answer there were no business slots at all. None. No amount of good SEO could have won a place here because there was nowhere to be named. And for some reason ChatGPT didn't think I was asking for restaurant recommendations, it seems to have deduced that I was asking for cuisine recommendations in NZ. So what does this tell us? ChatGPT is inconsistent, and will make its own deductions. It recommends based on what it thinks you are asking for, and how it can best answer that question. But it is not always the information you are actually after.
Test three: back to a map, and it thinks I am in Auckland
Five minutes later I opened a new window, typed into ChatGPT the same query (stayed logged out, as with the other queries I asked) and got a different answer again. Back to the map format, seemingly presuming I am based in Auckland, which we are not, we are in Kerikeri. And some business recommendations again. Yet again, Reddit is cited as the source of the information.
What These Three Tests Actually Tell Us
This is showing that AI cannot reliably:
work out where I, the searcher, am, and that it will try to guess.
answer your same query consistently, because it is changing every time, it is making guesses based on how it thinks you want the answer, rather than asking you when it is unsure.
Noting as well that this was a very general query I asked with a large area to recommend (all of NZ).
What it does show however, is consistency in the citation layer. Reddit was consistently named as the source. This indicates that being the 'source' or in other words 'becoming the answer' contributes to your website being a very valuable asset. And is something well worth considering when making decisions around your business's online presence.
Checking Once Doesn’t Show us The Whole Picture
But it does show that how we measure visibility in AI search matters. Checking once and getting one result isn't an accurate measure. Online search has truly changed. When it was just Google, the results are the same every time, you can see what changes because you can see where your page is ranked. In AI search the foundation is built anew, with every. single. search.
This is why we choose to measure our clients' visibility differently from a lot of other digital marketing agencies. We will run the same queries multiple times over a month, to measure how often do you appear. A single screenshot one time isn't a good gauge as to how visible you are.
When you choose to work with us in an ongoing visibility partnership, we are running our tracking software compiling the results over time to consistently recalibrate, and change and update, depending on what AI is coming back with about your business.
Keyword Research for AI Search Is a Different Job
This experiment shows the queries worth targeting are the ones that reliably produce a business slot. Keyword research for AI search is essential to increase the likelihood of your business being recommended. Our [visibility tracking software](/free-ai-check) harvests specific keywords of what people are actually writing into AI. We can then put these keywords into your website.
Why There Is No Number One Rank To Promise
This experiment also reinforces why there is no number one rank to promise. There is only a probability, and it is rebuilt every time someone asks. We work through consistency over time, to increase that probability, to bring all of the pieces of the puzzle together, so your business gets found online.
Want to find out what AI Search is saying about you? Get your FREE, no obligation check
So What Can You Do?
There are some authentic ways that you can optimise for Reddit:
You can put your business in your Reddit profile
You can give really useful answers on questions that already exist
Make sure when engaging with Reddit, you always disclose your interest when you mention in your own field, this keeps you upfront, credible and honest. If we were to answer on Reddit we could stay with "I run an SEO, AEO business up north, so take this with a grain of salt, but..."
Comment below if you found this blog useful!
If you want to know more about how to optimise for Reddit, this is part of our on-going visibility partnership
