How to Check If Your Website Has Duplicate Content

What Google Actually Treats as Duplicate Content

Google defines duplicate content as substantial blocks of text that appear at more than one URL. "Substantial" is doing real work in that sentence. Google never publishes a threshold, but controlled tests and observations from crawling large sites consistently show that two pages start competing as duplicates somewhere around 60-70% body-text overlap. Two product pages sharing only a footer are not duplicates. Two articles sharing an intro paragraph, a quoted statistic, and a conclusion are borderline. Two URLs serving byte-identical HTML are unambiguous.

Three categories need separating because each requires a different fix. Exact duplicates serve identical content at two or more URLs. Near-duplicates differ only cosmetically: a different date line, a session ID in the query string, a re-sorted list, a swapped ad slot. Cross-domain duplicates are pages you placed elsewhere yourself — a syndicated post, a manufacturer description reused by fifty retailers, a press release pushed to forty outlets.

Duplicate content almost never triggers a manual penalty. Google says so directly, and the exceptions (scraping others to manipulate rankings) are rare. The real cost is dilution: Google selects one canonical copy, ignores the rest, and the backlinks, crawl frequency, and engagement signals you earned get split across URLs that can never individually rank.

The Internal Duplicates Your Site Creates Without You Noticing

Every site has internal duplicates it did not deliberately build. The usual culprits, each with the concrete URL it produces:

• Protocol and host splits: http://example.com/guide, https://example.com/guide, https://www.example.com/guide, http://www.example.com/guide — four crawlable URLs for one page
• Trailing slash inconsistency: /blog/seo-checklist versus /blog/seo-checklist/, served by WordPress and most frameworks
• Parameter URLs: ?utm_source=newsletter, ?sort=price, ?page=2, ?color=blue each create a distinct entry in Google's index
• Printer, AMP, and mobile variants: /print/article and /amp/article mirror the canonical page
• Paginated series: /category?page=1 returning the same items as /category
• Faceted navigation: ?brand=nike&size=10 combinations that multiply into thousands of near-identical filtered pages

Faceted navigation is the worst offender. A clothing store with 400 products, 12 brands, 8 sizes and 6 colors can generate tens of thousands of filter combinations, most containing the same product grid as a page you already index. Run our URL parameter cleaner against your XML sitemap to see how many parameterized URLs you are currently asking Google to crawl.

External Duplicates: Syndication, Manufacturer Copy, and Scrapers

External duplication happens when your text — or text you rely on — lives on other domains. Four scenarios cover most cases:

1. Syndication without a canonical. You publish on your blog, then cross-post to Medium, an industry publication, and LinkedIn. Four copies, four URLs, and no version accumulates links.
2. Manufacturer product descriptions. If you publish the copy the manufacturer hands you and forty other retailers do the same, your product page matches forty competitors word for word. This is precisely why template-only e-commerce sites plateau; rewriting every description is a ranking prerequisite, not a nice-to-have. See SEO for e-commerce product pages for the rewrite process.
3. Distributed press releases. A wire service places identical text on fifty news domains within an hour.
4. Scrapers. Low-quality sites republish your articles wholesale, occasionally outranking you because they publish first or carry more authority.

Step 1: Crawl Your Own Site and Group the Duplicates

Screaming Frog SEO Spider clusters duplicates automatically. Crawl the domain, then open the Duplicate filter to see every set of URLs resolving to the same content; the paid tier adds a Near Duplicate mode with a similarity threshold you can tune. For a crawler-free baseline, Siteliner (siteliner.com) scans your whole domain and reports a sitewide duplicate percentage plus the exact page pairs involved, alongside a boilerplate score showing how much of your template is being repeated on every URL.

If you want a fast pre-crawl signal, our duplicate title and description checker finds pages competing with identical or near-identical titles and meta descriptions. Pages sharing a title almost always share body content too, so this surfaces the problem in one pass without downloading a crawler.

Step 2: Read What Google Search Console Decided

Search Console → Indexing → Pages is the authoritative record of how Google resolved your duplicates. Two statuses matter, and they call for opposite responses:

• "Duplicate without user-selected canonical" — Google found duplicate URLs and you never declared a preferred version, so it guessed. Eliminate this status first; every URL here is a coin flip you lost.
• "Duplicate, Google selected different canonical than user" — you declared a canonical and Google overrode it. Typical causes: canonicalizing to a page with zero internal links, canonicalizing to a noindexed URL, or pointing at a URL behind a redirect chain.

Click into a sample URL, run URL Inspection, and compare the "User declared canonical" against the "Google selected canonical" fields. When they disagree, one of three things is wrong: the canonical target is weaker than the duplicate, your internal links point at the wrong version, or your sitemap lists URLs that contradict the canonical. The resolution logic is covered in Canonical Tags Explained.

Step 3: Hunt for Copies on Other Domains

Copyscape (copyscape.com) remains the standard external check. Paste a URL and it returns every indexed page containing matching passages. The free tier lists matching sites; Copyscape Premium (about $0.03 per search after ten free ones) shows sentence-level overlap so you can judge whether the shared block is boilerplate or your actual article.

Run it against your ten highest-traffic posts plus anything you guest-authored. Pair that with our plagiarism checker, which highlights matched passages inside a draft before it goes live — useful when several writers work from one brief and quietly converge on the same phrasing.

Choosing the Fix: 301, Canonical, or Noindex

Three tools solve duplication and choosing wrong creates a new problem:

Use a 301 when a URL should not exist for anyone. Protocol, host, and trailing-slash normalization are always redirects because they consolidate link equity permanently and remove the URL from the crawl queue:

Redirect 301 /old-guide /new-guide

Use a canonical tag when the URL must stay reachable for humans — print views, paginated pages, filtered results, syndicated copies. Every duplicate declares the same preference:

<link rel="canonical" href="https://example.com/guide" />

Use noindex when the page serves users but earns nothing in search — internal search results, tag archives, post-purchase confirmation pages. Never stack noindex and a cross-canonical on the same URL: the directives contradict each other, and Google has to guess which one you meant. It will often guess wrong.

Syndication That Does Not Backfire

Cross-domain syndication only works when the republishing site adds a rel=canonical pointing to your original within hours of publication. Google has stated that if the copy gets indexed first, your canonical may be ignored entirely — you end up being the duplicate of your own article. Negotiate the tag as part of the syndication agreement. If the publisher refuses, publish a 100-150 word excerpt plus a prominent link back to the full article instead of the complete text, and give the excerpt its own self-referencing canonical.

Verifying the Fix and Keeping It Fixed

After deploying canonicals or redirects, inspect the affected URLs in Search Console and request indexing. Watch the Pages report weekly: "Duplicate without user-selected canonical" should fall toward zero over two to four crawl cycles. Confirm each group resolves correctly by running our bulk canonical checker across your priority URL list — it verifies that every canonical returns a 200, resolves to an indexable target, and does not form a chain or loop.

Re-run Siteliner monthly. A rising duplicate percentage almost always traces back to a new plugin, template change, or URL pattern introduced since the last crawl, and catching it in week one saves you from a slow ranking bleed that is painful to diagnose later.

Frequently asked questions

What is the main takeaway from this guide on how to check if your website has duplicate content?

The guide covers how to check if your website has duplicate content end to end: what it is, why it matters, and the exact steps to apply it. Skim the key takeaways first, then follow the step-by-step parts in order — most readers can apply the first improvements the same day.

How long does it take to see results from how to check if your website has duplicate content?

For most sites, changes made from this how to check if your website has duplicate content guide start showing measurable movement within two to six weeks. Track the affected pages in Google Search Console to see when search engines pick up your updates.

Do I need paid tools to follow this how to check if your website has duplicate content guide?

No. Every step in this how to check if your website has duplicate content guide can be completed with free tools, including the free utilities on this site. Paid platforms add convenience and historical data, but they are not required to get results.