Ranking on Google doesn’t mean AI search can see your site. ChatGPT, Perplexity, Claude, and AI Overviews can only cite pages they can reach and read, and the major AI crawlers Vercel tested can’t render JavaScript. Some CDN and firewall settings also block AI bots by default, so a page that does well on Google can still be invisible to them.
This SEO audit checklist walks through three layers: technical SEO (crawlability, speed, and site architecture), on-page SEO (page elements, content quality, and structured data), and AI crawlability (bot access, rendering, and crawl checks). For every item, you’ll see what to check and what a pass looks like. At the end, you’ll learn how to sort your findings so the biggest blockers get fixed first.
Part 1: Technical SEO Audit
Technical SEO decides whether search engines can find, read, and store your pages. If a page fails here, good content won’t help it rank, so this layer comes first.
How do you check crawlability and indexability?
Crawlability means search engines can reach a page. Indexability means they may store it. A page that isn’t indexed can’t rank, so start in Google Search Console.
Two checks cover most crawl and index problems: the files that give crawlers instructions, and the signals each page sends about whether it should be indexed. Check the files first, then the pages.
- robots.txt and XML sitemap
Open yourdomain.com/robots.txt. It should block staging environments and faceted URLs, not CSS, images, or scripts. Then check the XML sitemap. It should list only canonical, indexable URLs, with no redirects, 404s, or noindex pages.
Pass: the sitemap is submitted in Search Console with zero errors.
-
Index coverage, canonicals and noindex conflicts
In the Pages report, investigate Crawled, not indexed and “Discovered, not indexed” first. Then check the signals on each page. Every page should have one canonical tag, pointing to itself unless a true duplicate exists elsewhere.
Pages you want ranked should carry no noindex tag. The X-Robots-Tag header should agree with the page’s meta tags, because a conflict between the two can keep a page out of the index.
Pass: all key pages are indexed and send no conflicting signals.
How do you audit site speed, mobile, and security?
Test your homepage, top landing pages, and best posts in PageSpeed Insights. Note which metric fails, because each one has a different fix. Speed and mobile checks use real visitor data, so run them on the pages that matter most, not on a single test URL.
Core Web Vitals thresholds (LCP, INP, CLS)
Google, via web.dev, rates LCP of 2.5 seconds or less, INP of 200 milliseconds or less, and CLS of 0.1 or less as good, measured at the 75th percentile of real visits. A slow LCP usually comes from a heavy hero image or slow server, so use WebP or AVIF and a CDN. A poor INP usually means excess JavaScript, so defer non-critical scripts. A high CLS usually comes from images or ads without a set size, so add width and height.
Pass: all three metrics rate well on your key pages.
Mobile parity and HTTPS
Google uses the mobile version of a site for indexing and ranking, so a noindex or nofollow tag that exists only on mobile can stop a page from being indexed. Mobile pages need the same content, links, and schema as desktop. Confirm HTTPS is valid on every page, with no mixed content.
Pass: mobile and desktop content match, and every page loads securely.
How do you audit site architecture and structured data?
Architecture decides how easily crawlers reach your pages. Structured data tells them what those pages contain. Crawl the site with Screaming Frog or a similar tool, then validate the markup. Check the structure first, because crawlers can only read markup on pages they find.
Click depth, orphan pages and redirect chains: Aim to keep key pages within three or four clicks of the homepage. This is a common guideline, not an official Google limit. Every important page needs at least one contextual internal link with descriptive anchor text.
Pages with no internal links are orphans, and crawlers may never find them. Google treats 3xx redirects as passing PageRank, but chains still slow crawling and page loads, so point each redirect straight to the final URL. Fix or remove every broken link.
Schema types and validation: Check for Organization schema sitewide, Article on blog posts, and Product on sellable pages. Add FAQPage only if the Q&A is visible on the page, and know it won’t earn a rich result. Validate with Google’s Rich Results Test. Markup must match the visible content, and multiple types on one page are cleaner when nested in a single @graph..
Part 2: On-Page SEO Audit
On-page SEO checks whether each page tells search engines and readers what it is about. Start with the page elements, then audit the content itself.
Which on-page elements should every page pass?
Every key page needs a unique title tag, a unique meta description, one H1, a logical heading order, descriptive alt text, and clear internal anchor text.
Crawl the site with Screaming Frog or a similar tool and filter for duplicates and gaps in each of these elements. A crawler finds in minutes what a manual check takes hours to find.
Write each title tag so it is unique, puts the primary keyword near the front, and stays within roughly 50 to 60 characters. Write each meta description to earn the click, at roughly 140 to 160 characters. These lengths are common guidelines, not official Google limits.
Use one H1 per page that matches the topic, and nest H2s and H3s in order without skipping levels. Add descriptive alt text to every meaningful image. Make internal anchor text describe the target page, not “click here.”
Pass: no duplicate titles, no missing H1s, and no empty alt text on key images
How do you audit content quality and intent match?
A page passes when it matches what searchers want, offers something the top results don’t, and is the only page on your site targeting that query.
Pull your top pages by clicks and impressions from Search Console, then check each one against the search results for its main query.
- Thin, outdated, and cannibalised content: Look for three problems. Thin pages say too little to answer the query. Outdated pages carry old figures, screenshots, or advice.
Cannibalised pages compete with another page on your site for the same query, which you can spot when Search Console shows two URLs swapping positions for one keyword. Merge overlapping pages into the stronger URL, redirect the weaker one, and rewrite or remove thin pages.
Pass: each query maps to one page, and no key page is thin or outdated.
- E-E-A-T and answer-first formatting: Add author bios, cited sources, and first-hand experience to show expertise and trust. Show a visible last updated date on evergreen pages. Then put the direct answer in the first one or two sentences under each heading, so both readers and AI systems can find it without reading further. Write each section so it still makes sense if quoted alone, which means naming the subject instead of leaning on it or this.
Pass: every key page has a named author, cited sources, a current date, and answer-first sections.
Part 3: AI Crawlability Audit
AI crawlability checks whether AI systems can reach, read and cite your pages. Work in order: access first, then rendering.
Which AI crawlers should you allow or block?
AI bots fall into three groups, and blocking one group doesn’t block the others. Decide for each group whether the visibility is worth the access.
Treating “AI crawlers” as one bot is the most common mistake in this audit. Check your robots.txt for each group separately.
Training, search and agent bots
Training bots collect content to train models, such as GPTBot and ClaudeBot. Search bots index content to answer questions later, such as OAI-SearchBot and PerplexityBot. Agent bots act on a person’s behalf in real time, such as ChatGPT-User. Blocking training bots doesn’t remove you from AI answers, but blocking search bots does.
Sample robots.txt setup
A common setup blocks training and allows search:

Verify each bot name against its vendor’s current documentation, since these lists change.
Pass: search bots are allowed, and every blocked bot is blocked on purpose.
Is your CDN or JavaScript hiding content from AI bots?
A welcoming robots.txt doesn’t help if a firewall blocks the bot or the content only appears after JavaScript runs. Check both.
Two separate problems can hide a page from AI bots even when robots.txt allows them.
CDN and firewall blocking
Check your CDN and firewall rules for AI bots. Some CDN providers now block Training and Agent bots by default on pages with ads for newly onboarded domains, while Search bots stay allowed. According to W3Techs, more than 30% of websites use a reverse proxy service, so these defaults can reach a lot of sites. Review your settings carefully, because blocking Training can also catch crawlers that do both search and training.
The no-JavaScript test
Disable JavaScript in your browser, or view the page source, and reload your key pages. Vercel’s study found that the major AI crawlers it tested don’t render JavaScript, though Gemini and Applebot do. The title, body text, prices, specifications, and schema should all appear without it. If they don’t, use server-side rendering, static generation, or pre-rendering.
Pass: AI search bots reach your pages, and all key content is readable in the raw HTML.
How do you prioritise fixes after the audit?
Sort every finding by how much it blocks visibility, then fix the blockers first. Anything that stops a page from being crawled, indexed, or read comes before polish.
Fix today: problems that keep pages out of search or AI answers, such as indexing errors, blocked search bots, invalid HTTPS, and failing Core Web Vitals on key pages.
Fix this month: gaps that weaken pages that already work, such as missing or duplicate titles, absent schema, thin content, and weak internal links.
Ongoing: recurring upkeep, such as refreshing dated content, adding new Q&A content, and checking server logs for AI bot visits.
FAQs
How often should you run an SEO audit?
Quarterly. Run a full audit every three months, plus a light monthly crawl of key pages. Add one after a migration or a Google core update.
How long does an SEO audit take?
An afternoon for a small site. A large site with many page templates can take several days.
Can you do an SEO audit for free?
Yes. Google Search Console, PageSpeed Insights, and the Rich Results Test are free, and the free version of Screaming Frog crawls a limited number of URLs, enough for a small site.