Is my old website preventing me from being mentioned in AI search results?

AI search tools are reading your website right now.

If it is a mess under the hood, they may just give up and move on, and you don’t want that, so here are some of the issues that older websites might face when it comes to getting mentioned in AI searches.

If your site is old, even a few years old, it will have been built and optimised for traditional search, so it is potentially holding your business back from getting mentioned in AI searches, as it was not built with AI search in mind.

• What is an AI crawler budget?
• What slows an AI crawler down?
• The JavaScript problem.
• Code soup and why it matters.
• DOM size – the hidden culprit.
• What AI crawlers are actually looking for.
• The older site problem.
• What you can do about it.

There is a new type of search happening, and it is growing fast. Instead of typing a query into Google and clicking through links, more and more people are asking ChatGPT, Perplexity, Google’s AI Overviews, or Bing Copilot to just answer their question directly.

These AI tools read content from across the web, process it, and then give the user a composed answer – often without them visiting a single website. If your business gets referenced in that answer, great. If it does not, you are invisible to that user.

The question a lot of business owners are now asking is: why is my competitor getting mentioned and I am not?

The answer might not be about your content at all. It might be about how your website is built.

What is an AI crawler budget?

When any automated tool – whether a search engine bot or an AI training crawler – visits your website, it has a finite amount of time, processing power, and resource it is willing to spend on your site before it moves on. This is called a crawl budget.

Think of it like a postie with 500 letters to deliver in a day. If one house takes them 20 minutes to navigate because the path is blocked, the gate sticks, and the letterbox requires a specific technique, they are going to fall behind. Eventually, he starts skipping the difficult ones.

Your website is that house. And if it is old, bloated, or badly built, the crawler is going to spend far more time than necessary trying to make sense of it – and may not bother reading all of it at all.

This is not a small problem. AI crawlers from OpenAI (GPTBot), Google (Googlebot and its AI-specific variants), Anthropic (ClaudeBot), and Perplexity all operate on this principle.

They are trying to read as much of the web as possible, as efficiently as possible. A slow, cluttered website burns through their patience quickly.

What slows an AI crawler down?

There are several things that make a website hard work for a crawler. They are not all obvious, and most business owners have no idea they are sitting on a site that is fighting against itself.

Here is what the problem typically looks like.

A website built five or more years ago on a page builder like Elementor, Divi, or WPBakery – or even just on an old WordPress theme – tends to carry a lot of baggage. There is CSS that no longer applies to anything, JavaScript files that load whether they are needed or not, and HTML structure that has grown arms and legs over the years as the site has been tweaked and patched and updated.

The crawler arrives, requests the page, and then has to work through all of that before it gets to the bit that matters – your actual content.

The JavaScript problem.

JavaScript is a particularly bad one for AI crawlers, and this is worth understanding properly.

JavaScript is a programming language that runs in the browser. It is what makes things move, animate, load dynamically, or respond to clicks. Used well, it is great. Used carelessly, it causes real problems for anything trying to read your site automatically.

Here is the issue. When a crawler visits a standard HTML page, it can read the content almost instantly – the text is right there in the code, plain as day. When a crawler visits a page that relies heavily on JavaScript to load its content, it has to do something called rendering. It has to actually run the JavaScript, wait for it to execute, and then read whatever it produces.

This takes significantly more time and resource. Not every crawler even bothers. Some AI crawlers do not render JavaScript at all – they only read the raw HTML. If your content only exists after the JavaScript has run, those crawlers will see a near-empty page.

A lot of older websites – and plenty of newer ones built on the wrong foundations – are full of this. Sliders that load their content via JavaScript. Navigation menus built entirely in JS. Hero sections that pull their text in dynamically. All of it invisible to a crawler that does not bother to render.

And even when a crawler does render JavaScript, every additional JS file it has to load and execute is time and resource spent. A page loading 30 separate JavaScript files – which is not unusual for a bloated WordPress site – is going to eat a substantial chunk of that crawl budget before the crawler has read a single word of your content.

Code soup and why it matters.

Code soup is what I call HTML that has become unreadable, over-engineered nonsense.

A clean HTML page has a logical structure. There is a header, some navigation, a main content area, and a footer. The headings follow a sensible order. The paragraphs are wrapped in simple tags. A crawler can read it the way a person reads a document – top to bottom, with a clear sense of what is important.

Now take a page builder site. Every section of the page is wrapped in multiple nested divs, each carrying long and complicated class names the page builder needs to function. A single paragraph of text might sit inside six or seven nested containers, each one adding nothing meaningful to the content itself.

Here is a real-world comparison. On a page builder site I looked at recently, a basic blog archive page had just over 10,000 characters of visible, human-readable content. The HTML code behind it ran to nearly 240,000 characters. That is a ratio of roughly 24 characters of code for every 1 character of actual content.

A crawler has to wade through all of that to extract the bit that matters. It is the equivalent of being handed a 300-page document to answer a simple question that is answered in one paragraph on page 276.

Eventually, the crawler either runs out of budget, decides the signal-to-noise ratio is too poor to bother with, or simply does not rank the content as highly because the surrounding structure is so cluttered that it cannot clearly identify what the page is actually about.

DOM size – the hidden culprit.

The DOM is the Document Object Model – it is the browser’s internal representation of your page. Every element on the page, every div, every span, every heading and paragraph and image, becomes a node in the DOM.

Google recommends a DOM of fewer than 1,400 nodes for a healthy page. A well-built bespoke site might have 400 to 600 nodes on a typical page. A page builder site can easily hit 3,000, 5,000, or even 10,000 nodes on a page that does not look particularly complex from the outside.

Each of those nodes takes memory and processing time to work through. For a person using the site in a browser, an oversized DOM causes slowness. For a crawler, it causes the same problem – except the crawler has hundreds of thousands of other pages to get to today, so it is not going to hang around.

A large DOM also makes it harder for an AI crawler to understand the hierarchy of the page. Which bit is the main content? What is a navigation element versus actual information? What is a call to action versus substantive text worth extracting and referencing? The messier the structure, the harder these questions are to answer, and AI crawlers are trying to make these judgements at scale.

What AI crawlers are actually looking for.

It helps to understand what these crawlers are trying to do. They are not just indexing pages the way Google has always done. They are trying to extract factual, attributable information that they can use to answer questions.

They want to find clear, structured content. A question asked in a heading, answered directly in the text beneath it. A list of steps that explains a process. A definition that is accurate and specific. A comparison that is honest and informative.

They are looking for signals that this content was written by someone who actually knows what they are talking about. Schema markup – which is structured data you add to your site’s code to explicitly label what things are – helps them enormously. A site that uses schema to mark up its author information, its organisation details, its FAQs, its articles, is giving the crawler exactly what it needs in a format it can immediately understand.

An old, poorly structured site with no schema, bloated code, and JavaScript-dependent content delivery is the opposite of this. It is a site that makes the crawler work hard for very little reward.

The older site problem.

None of this is to say that an older site cannot be found or mentioned by AI search. Content still matters enormously – a site with brilliant, clear, authoritative content on a technically poor foundation will still do better than a fast site with nothing worth saying.

But there is a ceiling.

If your site is genuinely slow to load, if it is built on a page builder that pumps out code-heavy output, if it relies on JavaScript to render key content, if it has an enormous DOM, if it has no schema markup – you are fighting the algorithm with one hand tied behind your back.

The sites getting mentioned in AI answers tend to share certain characteristics. They load fast. Their HTML is clean and logical. Their content is structured clearly, with headings that match the questions people ask. They have schema markup that tells crawlers exactly what they are looking at. Their text-to-code ratio is healthy, meaning the crawler is not burning through budget on noise before it gets to signal.

Older sites built on heavy themes or page builders rarely have any of these things by default. The code they produce was never designed with this kind of scrutiny in mind – it was designed to look good in a browser and give non-developers an easy way to build pages. That is a different goal entirely.

If your site was built five years ago on Elementor or Divi and has never had a serious technical audit, the honest answer is that it is probably working against you in AI search right now.

What you can do about it.

The fix is not always a full rebuild, though for many older sites it genuinely is the most sensible long-term answer.

There are things that can be done in the shorter term. You can audit your crawl budget using tools like Screaming Frog or Google Search Console to understand how crawlers are currently seeing your site. You can check your PageSpeed score at Google’s PageSpeed Insights tool – anything below 90 on mobile is a concern, and scores in the 40s or 50s are a genuine problem. You can look at your DOM size in Chrome’s developer tools. You can check whether key content on your pages is visible in the raw HTML source or only appears after JavaScript has run.

You can add schema markup to your key pages – there are WordPress plugins that handle the basics, and any decent developer can implement more specific schema by hand.

You can also look at how your content is structured. Clear headings that match the questions your customers actually ask. Direct answers in the text that follows. No waffle, no filler, no burying the point.

But if the underlying code is a disaster – if your site is built on a page builder that outputs thousands of unnecessary DOM nodes, loads 40 JavaScript files on every page, and produces a text-to-HTML ratio that makes a crawler’s eyes water – adding schema and tidying the content is a bit like painting a house that has damp. You need to fix the structure first.

AI search is not a passing trend. It is changing how people find businesses, ask questions, and make decisions. If your website cannot be read efficiently and clearly by an automated crawler, it will not be referenced in those answers. And if it is not referenced, the people asking those questions will never know you exist.

Getting your site in shape technically is not just an SEO task anymore. It is a basic requirement for staying visible.