The robots are visiting your website more than people are
People ask ChatGPT now instead of clicking Google. Here's the afternoon of simple fixes that gets your site into the answer, so you don't quietly disappear.
Last week Mark Lou shared his website numbers. In 30 days, AI robots visited his site 850,000 times. That’s more than 4 times as many visits as real people. ChatGPT alone showed up 20 times more than any other AI.
So here’s the strange new truth. Robots visit your website more than humans do. The only thing that matters now is simple: can they read your site when they get there?
For a long time the game was easy. You wanted to show up high on Google so a person would click your link. That’s changing fast. Now lots of people never click any link at all. They ask ChatGPT or Perplexity a question and do whatever it tells them. If the AI names your site in its answer, you win. If it doesn’t, you’re invisible. There’s no second page to hide on. You’re in the answer, or you’re gone.
Most sites are gone. Not because they’re bad. Because nobody told the robots they were allowed in, and nobody wrote anything for a reader that skims in half a second.
Good news: fixing it takes an afternoon, not a rebuild. Here’s the list.
1. Let the robots in (10 minutes)
Every site has a tiny file called robots.txt. Think of it as the bouncer at the door. On a lot of sites the bouncer is turning AI robots away, sometimes by accident. Go check yours, and let the important ones in:
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Perplexity-User
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: Claude-User
Allow: /
User-agent: GPTBot
Allow: /
User-agent: Google-Extended
Allow: /Two kinds of robots matter here. Some read your site to learn (training). Others show up the second someone asks a question, grab your page, and use it in the answer (retrieval). The retrieval ones are the ones you can’t afford to block. Block them and you vanish at the exact moment someone’s ready to buy.
2. Leave the robots a menu (30 minutes, best bang for your buck)
Make a simple file called llms.txt and put it at yoursite.com/llms.txt. On it: your product name, one sentence on what you do, and a short list of your best pages with a line about each.
It’s like handing the AI a menu so it doesn’t have to wander around guessing. Fair warning: no AI company has promised they read it yet. But it takes half an hour and it can’t hurt you. When something’s free and safe, just do it.
3. Say the answer first
This is the big one. When AIs quote a page, about 44% of what they grab comes from the first third of it. So if your page starts with a long boring intro, the AI gives up before it finds the good part.
So don’t bury it. Put the answer in the first two sentences. Make your headings the actual question someone would ask (”How much does it cost?” not “Pricing”). Keep paragraphs short. And use real numbers. “30% more” beats “a lot more,” because specifics sound like you know what you’re talking about, to a robot and to a human.
One more trick: write each chunk so it makes sense on its own. AIs copy little pieces, not the whole page. So don’t write “it” or “this.” Name the actual thing every time.
4. Label your stuff (this is called schema)
Schema is a bit of hidden code that tells the robot what it’s looking at, so it doesn’t have to guess. Like a label on a can. Add these:
FAQ code on your question-and-answer sections. Gets quoted about 3x more.
Product code on product pages, with the name, features, and price.
Article code on your posts, with the author and date.
Company code across the whole site, with your name and your links.
Pages with these labels get quoted 30 to 40% more than pages without them. Add them, then check them with Google’s free Rich Results Test. Fifteen minutes of copy-paste.
5. Make sure it works with the lights off (no JavaScript)
Try this: open your page, turn JavaScript off, reload it. If the page goes blank, that’s exactly what a lot of robots see. Blank. If your site only shows its content after a pile of code runs, the robots often miss all of it. The fix: make your important pages show real text right away. Your homepage and your posts should have actual words in them before any fancy code runs.
6. Get talked about in other places
AIs don’t only quote your site. They quote the places that talk about you. So show up in a few.
Reddit is huge for Perplexity, almost half of what it pulls from. Actually help people in a couple of subreddits, and only mention your tool when it’s the honest answer. People sniff out spam instantly.
Get listed where AIs look for tools: G2, Capterra, Product Hunt.
Share one piece of your own data. Whoever says a number first gets quoted again and again. Mark Lou’s robot number is the whole reason you’re reading this. Your numbers can do the same job.
Also good: those “best tools for X” list posts. Find the ones already ranking for your space and ask to be added.
7. Two small extras
Describe yourself the exact same way everywhere: homepage, about page, every profile. If the words keep changing, AIs get confused about what you even do. And keep your pages fresh. A “last updated” date, plus the matching code, tells AIs your stuff is current. Old pages get pushed down.
8. Keep score, or you’re just guessing
You don’t need a fancy tool to start. In your analytics, look for visits coming from chatgpt.com, perplexity.ai, and claude.ai. Then once a week, ask ChatGPT, Claude, and Perplexity 20 to 30 questions a real customer would ask, and write down when your site shows up. That’s your score. Do everything on this list, and getting named 1 time out of 5 within two months is a solid goal.
Change one thing at a time, though. Otherwise you’ll never know what actually helped.
One honest note
Google says don’t get carried away. Make genuinely good pages first, not pages that only try to trick the AI. Sites that go too far can get punished for it. So do all of this on top of real, useful content. Not instead of it.
