Index bloat is the gap between the number of URLs Google has indexed and the number that could plausibly earn a click. On most sites above a few thousand pages that gap is large, and it is invisible until someone measures it.
It matters for two reasons. Crawl budget spent on worthless URLs is crawl budget not spent on your money pages, so new content takes longer to appear. And a site where most indexed URLs are thin gives Google a weaker overall signal about what the domain is good at.
Measure the gap first
Two numbers, five minutes:
- Indexed URLs. Search Console → Indexing → Pages → “Indexed”. Not the sitemap count, the total.
- URLs that deserve to rank. Your real page inventory: products, categories, service pages, genuinely useful articles.
If the first number is more than about 1.3× the second, you have bloat worth investigating. If it is 5× or more, it is probably the highest-leverage technical work available to you.
Where it comes from
Almost always one of these seven, in roughly descending order of volume:
Faceted and parameter URLs. The biggest source on any ecommerce site by a wide margin. Filter combinations, sort orders, view modes and tracking parameters each multiply the URL space. This is a large enough topic to have its own guide.
Internal search results. ?s= and /search/ URLs get indexed when someone links to them or a sitemap plugin includes them. They are generated on demand, which means the URL space is effectively infinite.
Tag and author archives. A blog with 400 posts and 300 tags produces 300 archive pages, most holding one or two posts. They compete with the posts they list and rarely earn anything.
Pagination. Page 2 onward of a category or blog series, where the intro copy and title are duplicated from page 1. Sometimes these should be indexed; usually the duplicate template copy means they should not be.
Attachment and media pages. WordPress in particular generates a page per uploaded image by default. On a site with an image library this alone can double the indexed count.
Staging and development leftovers. Old subdomains, /dev/, /v2/, /old/ directories, and test pages that were never removed. These also frequently carry noindex that was removed at launch and not reinstated.
Thin programmatic pages. Location pages for cities you do not serve, category pages with two products, “compare X vs Y” pages generated for every combination. Programmatic content is not inherently bad, but the threshold for “this page is useful” has to be enforced somewhere.
Find it without guessing
The reliable method is a reconciliation, not a crawl on its own.
- Crawl the site fully, ignoring robots.txt on the second pass so blocked URLs surface too.
- Export Search Console’s page indexing report, including the “Indexed, not submitted in sitemap” bucket. That bucket is where bloat concentrates.
- Export 12 months of performance data at page level, including zero-click pages. You need impressions as well as clicks.
- Join the three. Any indexed URL with zero clicks and negligible impressions over twelve months is a candidate.
The twelve-month window matters. A page with seasonal demand can look dead in a 90-day export and be valuable in November.
Decide what to do with each group
Sort candidates into patterns, not individual URLs, because the individual list regenerates.
Delete and return 410 when the page has no purpose, no traffic and no external links. A 410 tells Google the removal is intentional and gets the URL dropped faster than a 404.
Redirect 301 when the page has no value itself but has external links pointing at it, or when a clearly better equivalent exists. Do not redirect everything to the homepage — an irrelevant redirect target is treated as a soft 404 anyway.
Noindex, follow when the page needs to exist for users but should not compete. Tag archives, filtered views and internal search pages usually land here. Remember this does not save crawl budget; Google still fetches the page to read the directive.
Block in robots.txt when you want to stop the crawling as well, and the URLs have no external links worth preserving. Never combine robots.txt disallow with noindex on the same URL — if the crawler cannot fetch the page, it cannot see the directive.
Improve when the page is thin but the topic is genuinely valuable. A category with two products is not a technical problem, it is a merchandising one.
The mistake that costs traffic
The dangerous version of this work is running a bulk noindex based on a crawl alone, without checking performance data first.
Pages that look thin by word count are sometimes earning steadily. Location pages, spec pages and small category pages often have short copy and high commercial intent. Deleting them because a crawler flagged low word count is how a cleanup turns into a traffic loss.
Rule: no URL gets removed or noindexed without first checking twelve months of impressions and clicks. If it earned anything, it goes on the “improve” list rather than the “remove” list.
What to expect afterwards
Indexed URL count will fall, often dramatically, and it should. Watch these instead:
- Crawl requests to your money templates should rise in the log files within two to four weeks.
- Time to index for new pages should shorten noticeably.
- Impressions on your real pages should be flat or up. If they drop, something valuable was caught in the cleanup and should be restored.
Give it eight weeks before judging. Deindexing is not instant, and the first few weeks will show a falling page count with no upside yet.
Index bloat is one of the nine checks in the free technical audit, and the reconciliation above is exactly how we run it. See the full audit scope, or send your domain and we will tell you what your real gap is.
