An Extensive Guide on Magento Duplicate Content

An Extensive Guide on Magento Duplicate Content

Nothing is created equal, they say. They are wrong. And if you own an online shop, you are the one who knows that perfectly well as many pages on your site are literary equal, aren’t they? Our extensive guide will help you to solve the problem of Magento duplicate content and never allow it again.

If you’re still not sure whether this guide is for you, just go to Google and search for ( should be replaced by the name of your shop). How many pages are returned? Thousands? Ok, then search for What do you see now? Much less pages are returned, right? So what’s wrong?

Those pages that are omitted during the second search are found in Google supplemental index as Google considers them thin content. In most cases this content means duplicate content (full or partial), and it is one of the greatest enemies of any Magento store. I even heard Google Panda searching for duplicate content once at night…

What are the types and most common examples of duplicate content in Magento?

Partial duplicates are the case when only minor part of the content or its layout is unique.

partial duplicates in Magento

Here are the most common examples of partial duplicates in Magento stores (you can click on any example and and jump to its detailed overview).

Full duplicates are the case when the content on two or more pages is identical.

Full duplicates in Magento

The most common example of full duplicates in Magento is:

Are you ready? Then let’s get down to the details!

Partial Duplicates in Magento


1. Product Sorting

That’s great when users can sort the products in your store by bestsellers, by newest, by price, number of reviews, etc. It’s even better if people can decide how many products should be displayed on the page: 20? 50? 100? But all these sorting options create pages with different characters (?, =, |) in the URLs:|desc|asc|desc

The problem comes out when sorting pages get indexed and even cached by Google. Imagine how many such pages can exist! Thousands! And Google crawlers spend time indexing them while they could concentrate their resources on indexing more important pages of your site: categories, products, etc.

1.2. How to find product sorting pages

First go to your product pages and sort them by any option. Now you can see the parameters added to the URL after sorting (e.g., dir, sortby). Go to Google and search for inurl:dir

Most likely you’ll see this:

Google Supplemental Index

Just click to include the omitted results and you’ll see the pages in your store containing “dir” in the URLs.

It’s bad when these pages with parameters are indexed. But it’s even worse if they are cached. You can check it by clicking Cached from the search results or by searching cache:url in Google.

1.3. How to remove product sorting duplicates

1.3.1. Google Webmaster Tools

Go to Google Webmaster Tools => Crawl => URL Parameters. Here you will see the parameters Google has found in the URLs of your store and how it crawls them. “Leg Googlebot decide” is the default option there.

URL Parameters in Google Webmaster Tools

But when it comes to crawling your Magento store, it’s you but not Google who should decide which pages should be indexed, right? So if you haven’t decided this before, it’s high time you did it! Click “edit”, choose “Yes” in the dropdown menu and then – “No URls”.

Block indexing of URLs with parameters in GWT

You can also add parameters that are not listed in GWT and set crawling options for Google. But be careful and check twice (or even three times) before blocking the URLs with these parameters.

De-indexation of URL parameters in Google

1.3.2. Rel=canonical

You can also use canonicalization for the sorting pages in your Magento store. This way they will be accessible for users but redirect the crawlers to pages without parameters.

Canonicals? Great! But how do I implement them?

You should add this code to the the sorting pages:

<link href="CategoryURL " rel="canonical" />

where CategoryURL is the address of the same category page without parameters. For example, the following pages:|desc|asc|desc

should canonicalize this page

After adding the code, give Google some time to re-index the pages and follow the new instructions, usually it takes a few days. Canonicalization set up correctly, you’ll see the cache of the canonicalized page ( even if you check the cache of the sorting pages (cache:|desc)

using rel=canonical and blocking parameters in GWT

2. Pagination

You Magento store is big as you have lots of great products there, right? But even you have only a few products (that are also great!), they are still placed on the pages with pagination options.

For example:

2.1. How to find paginated duplicates

To find paginated pages in your Magento store, go to Google and search for inurl:page

This search returns all the pages containing “page” in the address within your site.

2.2. How to remove duplicates

2.2.1. Canonicalization

You already know a lot about canonicalizing pages in your store, right? Just make rel=canonical tags on those paginated pages:

point to the category

Single rel=canonical for each page

2.2.2. Pagination with rel=”next” and rel=”prev”

This option was specifically created by Google to fight duplicate paginated results. The idea is that all paginated pages are connected like links in a chain:

how pages are connected by rel=prev and rel=next

If we take these pages as an example:

Then we should put the prev/next instructions in the following way:

<link href="" rel="next" />

in the <head> of

<link rel="prev" href=" " />
<link rel="next" href="" />

in the <head> of

<link rel="prev" href="" />

in the <head> of (let’s imagine this is the last page).

All this is not too complicated, so you can make it yourself or use a special module (our Improved Layered Navigation has such functionality) to implement rel=”next” and rel=”prev”.

rel prev next implementation

There are a few things to remember:

  1. The first paginated page should contain only rel=”next”
  2. The last paginated page should contain only rel=”prev”
  3. Google allows using both canonicals and rel=”next” and rel=”prev” on one page

3. Variations of the same product

Imagine you sell mugs (or you really sell them?) and have landing pages for each color:

variations of one product

The characteristics are the same, description is the same, layout is the same… So what’s new? Just picture! Unfortunately, it’s too little for Google to treat such pages as unique. This means that all product variations found on different pages are partial duplicates that act like a magnet for Google Panda.

3.1. How to find product variations

As a Magento shop owner, you probably know all your products and can make a list of their variations. Alternatively, you can search in Google for: “here comes a short excerpt from the product description”

This way you will find all the pages of your site containing this very excerpt.

3.2. How to remove duplicates.

3.2.1. One page for all variations

You can create a single page for a particular product and list all its variations there.

product variations on one page

This way you have one unique page instead of several duplicate ones.

3.2.2. Rel=canonical

You can also use rel=canonical. Just choose one variation page and put canonical to it from other variations. This way the content will be seen by users but Google will have a copy of only one page that you’ve chosen.

3.2.3. Make each variation page unique

The hardest way to solve duplication issues with product variations is to make each variation page unique. You will have to add different product descriptions and meta info. Yes, this is extremely time consuming. So be sure to grab a cup of coffee before starting this long journey!

Full Duplicates in Magento

Now when you’ve found all the partial duplicates, it’s time to look for full ones. Ready, steady, go!

The same product in different categories

You may have one product that can be found in two or more categories. For example:

There’s only one necklace but 3 different URLs! In this case no matter how great your product is, Google will consider it thin content. It’s so unfair! So let your great products be great to Google by making them unique.

1.1. Rel=canonical

Just like with product variations I’ve talked above, you can choose one URL to show your product (necklace, as seen in the example) and the other pages will canonicalize it.

1.2. Remove category from URL

Alternatively, you can remove the category path from the URL, so that each product will have only one address no matter in how many categories it can be found:

Don’t know how to do that in Magento? There was a good tip in one of our post on SEO in Magento. Just go to System => Configuration => Catalog => Search Engine Optimization and switch “Use Categories Path for Product URLs” field to “No” and both “Canonical Link Meta Tag” fields to “Yes”.

remove category from URL in Magento

1.3. Leave only one category path in a product URL

If you have a red T-Shirt in 2 categories at once: T-Shirts and New, you can choose which category to use in the URL: either the longest one (T-Shirt) or the shortest one (new). This is possible with Unique Product URL extension.


Now you know how Google sees duplicate content on your store. Don’t let it decide how to treat your pages, suggest the best way possible instead. Take control over your site crawling!

Kristina Azarenko

Kristina Azarenko

Kristina is the person who strongly believes in the power of SEO and does her best to bring the website to the top of search results. She pays careful attention to the latest marketing trends while developing detailed strategies for site performance improvement. Being a many-sided person she effectively combines her marketer activities with dancing classes, photo shooting and… knitting :)

You may also like...

71 Responses

  1. Darryl Collins says:

    Great helpful article, thanks for posting!

  2. Nghia says:

    Great article! What about duplicated content over multiple stores (one magento installation) ?

    • Kristina Azarenko Kristina Azarenko says:

      Thanks for your comment, Nghia!
      The best strategy is to keep content unique as much as possible. If you have the same products in different stores, you’d better write unique descriptions for them. This is quite time consuming but rewarding.

  3. Andrew Clay says:

    hey this is really interesting & helpful post. Thanks for sharing this.

  4. satyajobs says:

    Thanks for sharing fabulous information.It’ s my pleasure to read it.I have also bookmarked you for checking out new posts.

  5. scott says:

    How do you make your paginated pages canonical point back to the url without the ?page=2 for example?

  6. Judas says:

    Hello Kristina! What I really wanted to say is “Bundle of Thanks”.
    You helped me a lot by sharing such a nice and detailed post. I’m a webmaster and didn’t know the Goolge Webmasters Crawling “URL parameters” technique to remove duplicated product pages. I’ve implemented these duplication removing tips on my magento site.
    Thanks again!

  7. Kristina Azarenko Kristina Azarenko says:

    You’re welcome! I’m happy I’ve helped you!

  8. Gonzalo Janousek says:

    I blog often and I seriously thank you for your information. This great article has truly peaked
    my interest. I’m going to book mark your blog and keep checking for new information about once a week.
    I opted in for your Feed as well.

  9. Zak says:

    Great article!

    Another tip that we often use to help GWT warnings of high number of URLs indexed is to hide the layered navigation from search bots. This would also help with section 1 of this article We normally create a theme exception for bot user agents with a small layout XML file which removes the layered nav.

  10. Jack says:

    Thanks Kristina. I saved this article to study later – lots of stuff needs to be done…

  11. Julien says:

    Hi, great post ! Thank you very much, simple and efficiente :)

  12. Matt says:

    Thanks so much for this blog post, Kristina, you’re going to make me look like a hero tomorrow at work. If I read your above response to Nghia correctly and apply it to my own situation, it sounds like the best method for dealing with a large number of products whose only difference is their size would be boil them down to one product with some method of selecting the correct size before adding to a cart.
    Does that hold water?

    • Kristina Azarenko Kristina Azarenko says:

      Hello Matt,
      Placing product variations on a single page is a painless solution in most cases. However, it still depends on your niche. If you customers tend to look for particular sizes of your products, it might be better to have multiple pages with more or less unique content and meta info.
      Good luck to you tomorrow!

      • Paul says:

        Hi! Again! Thank you for the last reply relating to ! Ok here is the most confusing thing. Absolute best practices have been applied, any I now have 48000 indexed! Oh my lord! Which software are you using to identify how many are listed in SERP’s? And how would I increase the quantity sat in SERP’s? This site will mean the world to me? Eg allow me to eventually spend more time with the children. I will make it work, absolute determination as the driving force! It is still in development, however I am trying hard to stagger so that Google can see a steady progression! Also, where is the best place to put articles in Magento? Is there an extension that works well with SEO? I have started to write an article within Cms pages…. (take a look) I have added to this every 1 to two weeks for progression. I will continue with these articles, however if this is the wrong place to start growing a knowledge base, I will stop now (leave the vitamin D page for index reasons) but start growing the rest elsewhere! Help! :0)

        Kind Regards


        • Kristina Azarenko Kristina Azarenko says:

          Hi Paul,
          you can find how many pages are found in main Google index by adding /& (please see more info about it at the beginning of that article).
          I would also recommend adding your site to Google Webmaster Tools to track duplicate titles/descriptions there. It seems you have variations of 1 product on different pages, this may be an issue.
          As for a blog, you can use Magento CMS pages or use WordPress for blog and connect it with Magento.
          Hope this helps.

  13. Amy says:

    Thanks so much for this great post. Was great timing as I was cleaning up my GWT! For the URL Parameters I have fixed the ones you have listed with the examples of what to put as the Effect. I have some others but don’t want to get it wrong. Can you please tell me what I would put as the effect for the following: manuf, em_x, artist, size_materials, options. Thanks so much for your help.

    • Kristina Azarenko Kristina Azarenko says:

      Hello Amy!
      There’s one thing you should remember: when you add a new parameter to GWT, you must be sure it’s not used in canonical URLs. For example, if you add “options” as a parameter, any page containing it in the URL (like will be de-indexed. So at first make a list of “pure parameters”. In your case these ones em_x, size_materials seem to match.
      Moreover, pages usually contain more than 1 parameter, so it’s enough to block the most obvious of them. For example, for that URL it’s enough to block just sortby.
      Hope that helps!

  14. Mar says:

    Great information! Regarding removing the category from the URL on an existing site, I assume one will need to build a table of the inbound links and redirect them to the desired landing page? Is it better to handle it this way in Magento rather than doing a NOINDEX in the category landing pages?

    • Kristina Azarenko Kristina Azarenko says:

      Hello Mar,
      you actually don’t need to build a list of inbound links. What you need is to make sure all visitors will reach the necessary product page. If you decide to remove category path on an existing site, you can use a ready solution or make a list of multiple product URLs with categories in them and 301 redirect all of them to the URL without any category path (it will become a canonical one).
      This applies to product pages only. When it comes to category pages and pagination, there are some other ways to handle duplication (they are covered in the article).
      If you still have questions, don’t hesitate to ask them :)

  15. rakesh says:

    the article is helpful for me.

    can u share me how to fix step by step process ppt

    we are getting same issue

    please help waiting for reply
    my mail is

    • Ksenia Dobreva Ksenia Dobreva says:

      Hi Rakesh,

      thanks for your comment. Sorry, but we do not consult via email.
      It’s not clear what issue you are getting. The whole article is a step by step instruction, so maybe if you describe your problem we could give you a few hints.


  16. Kiran says:

    Hi Kristina Azarenko,
    This is my website ( have the duplicate content, – 164 indexed and – 97indexed in google, Please send me file resolve this issue in ppt.

    • Ksenia Dobreva Ksenia Dobreva says:

      Hi there and thanks for the comment!

      Although I don’t understand what ppt file you’re talking about. We don’t have any presentation files, the whole post is the instruction on how to get rid of the duplicate content (which you can certainly apply to your store).


  17. Karen says:

    Thanks for your great post. I looked through the mentioned items and found a lot of things to be done on my shop.

  18. Paul says:

    Hi, I have been patiently building a site in a live environment, ….. Fantastic post first of all! Ok, so I have followed all the instructions which should knock about 2300 products off the duplicate indexes. I have 17629 products, Google has indexed according to the site: command , 31400 products……pheweeeee! However the plot thickens, within the site map area of webmaster tools Google claims to have only indexed 377 pages! No errors?!!! Help!

    • Kristina Azarenko Kristina Azarenko says:

      Hello Paul,
      Thanks for commenting.
      As I can see, Google really shows that 31000+ pages of your site are indexed. However, only 26 pages of Google SERPs are available, i.e. about 260 indexed pages. The best thing you can do in this case is to relax, apply the best practices of fighting duplicate content and wait for the results. Google may be slow at showing such information, especially if your site is new.
      Good luck!

  19. Adam Allen says:

    Great article by the way, something i have been meaning to do for ages but didnt quite know how to go about it so thank you for that. Quick question though in step 1.3.2 where you say add

    to the sorting pages. in magento im sure you know most pages work off of .phtml files. What file would i add this code to to make sure it works, the layout file, the view.phtml which sets up whats on the page or the list.phtml which gets the products?

    Thank you

  20. Steffen Wirth says:

    thanks for this great article ! Please have a look in this article from GOOGLE
    if you use category canonical in Magento you’ll have the same canonical url of page 1 for all other pages, so GOOGLE just uses your page 1 and not the other content & products of the other category pages. Thats what Google says. Have you made other experiences than Google?

    • Kristina Azarenko Kristina Azarenko says:

      Hello Steffen, thanks for your comment!
      You are right about rel=canonical, it shows only the canonicalized page to Google and states that other duplicate pages should be ignored. That’s why I wrote in the article “If canonicalization set up correctly, you’ll see the cache of the canonicalized page even if you check the cache of the sorting pages” – this can be applied to sorting, pagination, etc.
      This is how canonicalization works for Google, so hardly anybody had other experiences with that :)

  21. Ricardo says:

    I have a doubt about csv file template to import coupons.
    Anyone can help me?

  22. Sebastien says:

    Hi there,

    First of all, thanks A LOT for your article. I have been searching about this issue for a while and this article is exactly what I needed!

    I just have 3 quick questions about your solutions. In case it helps, my website is tictoque[dot]com. My attributes/filters create a huge amount of duplicate content despite canonical links being long enabled in Magento Admin… So here are my questions:

    1) Do you recommend using canonicalization for the sorting pages or using URL Parameters in GWT or both? (My goal is to prevent duplicate content from bein indexed in the future AND to de-index all these sorting pages that are already indexed and cached by Google…)

    2) If I use canonicals, how do I add the canonical tags to the canonical pages and to the sorting pages? Do I have to edit phtml files via FTP (and if so, in which directories) or can I do it in Admin?

    3) If I edit URL Parameters in GWT and set them to “No URLs” like you suggest, will that make Google de-index the corresponding pages or do I have to additionnaly go to GWT “Google Index / Delete URL” and ask for URL deletion?

    Thanks a lot again!!

    • Kristina Azarenko Kristina Azarenko says:

      Hi Sebastien,
      I’ve looked through your site and will try to answer your questions.
      1. I would recommend using both solutions just to be on the safe side.

      2. You’d better ask your developer about that. I’m not sure that it is possible in Magento without any customization or extension.

      3. Yes, if you let Google know that the pages with particular parameters are duplicates and should not be indexed, it will de-index them. It will take time but at the end they will be removed from the search. Of course, you can use URLs removal requests on the URL-by-URL basis but it’s quite time-consuming.

      As for your site, I see that the URLs like “__store=default&couleur_vin=30&dir=asc&order=position&___from_store=default” were last visited more than a month ago, so if your canonicals were implemented not long ago, Google hasn’t found them yet. Your site is quite new, so be patient :)

      • Sebastien says:

        Hi Kristina,

        Thanks a lot for your answer. I will follow your advice then 😉

        And actually, I can already see the canonical links pointing to my main category in the head section of my sorting pages, so you are right, I just have to wait for Google to find them…

        What worries me is that canonicals were actually implemented quite a long time ago, before the last crawl from Googlebot. Do you advise me to wait for a few weeks or to already use GWT and set the relevant parameters to “No URLs”?

        Thanks again :)

        • Kristina Azarenko Kristina Azarenko says:

          Hi Sebastien,
          It’s up to you. But I would rather wait for the canonicals to get noticed by Google and add parameters exclusions in GWT after that.

  23. Kristian says:


    Thank you for the helpful article! We have category paths for URLs currently set to “yes”. So the full category path is being indexed. We realize that it should have been be set to no. Our site has been indexed and is getting SEO traction. However, we would like to improve and since everyone states this should be set to no we feel we should change the setting. Our concern is what the affect will be on page rank after we make the switch. Any advice or suggestions?

    Thanks again!

    • Kristina Azarenko Kristina Azarenko says:

      Hi Kristian,
      If you remove duplicates, the effect will be positive. In your case you have 2 options: exclude all the category paths from the products URLs or leave only 1 category path per product, so it’s up to you which way to go. You can get back to the above article for more details on these options.

  24. Jeff Moriarty says:

    Hi Kristina,

    I have been dealing with many canonical issues for awhile, but had a question. Take a look at See the canonical url, it appears wrong to be because of having the parameter included. Would it be better have my developer set these to the default url ( and/or adjust it in Google Webmaster Tools?

    • Kristina Azarenko Kristina Azarenko says:

      Hi Jeff,
      It seems I’ve already answered that question above. You can go for the canonicals first. Once Google notices them, adjust the parameters in GWT.

  25. Umesh Tandel says:

    Hi Kristina,

    I need to get products which are having same values in their attributes.

    For example,
    If any product attribute (color, size, manufacturer) values matches with another product attribute values, then need those products to be added in product collection.


    • Ksenia Dobreva Ksenia Dobreva says:

      Hi Umesh,

      thanks for the question. Looks like you need to ask a Magento developer to do this task for you.

  26. Ricardo says:

    Thank you for this post. I though I was going to experience a duplicate content hell with magento when I noticed google crawling all those url versions. You saved me. I used robots.txt to remove all unwanted url paths that were already indexed. You did not mention anything about index.php duplicates where all you website could get duplicated trough the index.php path. I removed the index.php with robots.txt. Is it best to redirect it?

    • Kristina Azarenko Kristina Azarenko says:

      Hi Ricardo,
      Yes, you are right, sometimes homepages can be accessed with index.php as well, so it’s better to set a 301 redirect pointing to the version without it.

  27. chris says:

    I have 1000 pages on my sites. I changed the magento suffix from “- myshop” to “myshop .com” now google has indexed 2000 pages. A few week later I changed again the suffix of my store to “| mishop” and google indexed again 1000 pages. Now I have 3000 pages indexed instead of 1000. Google ignores completely that it is the same page and that only the title changed. This search engine is completely STUPID.

    • Ksenia Dobreva Ksenia Dobreva says:

      Hey Chris,

      thanks for sharing your experience.

      However, I’m not sure that changing the suffix can cause the duplicate issue. These things are not connected at all. Are you completely sure that your shop does not create duplicate pages with various titles?

      • chris says:

        You’re probably right (I was tired) however magento is generating duplicate content even with all SEO rules like someone posted earlier. canonical links, link rel and NOINDEX on any filters/searchs doesn’t fix the problem. I ‘m working on it for months, saw lots of comments in magento forums since 2014 and nobody cannot precisely explain why. I tried another CMS like prestashop (without mods) and I do not have any duplicate issue with google. For startups and very small businesses who rely on natural indexing, magento is a nightmare if you don’t have enough backlinks.

        • Ksenia Dobreva Ksenia Dobreva says:

          Hey Chris,

          if your installation still generates duplicate content and settings/extensions aren’t helping, maybe it can be a good idea to ask a developer to have a look at what exactly causes duplicates and to fix it manually?

          • chris says:

            I’m a developper myself (Since 1992) and PHP since 2000 😉
            Can you provide just one Magento site used in production without duplicates confirmed by google with – I checked a dozen of magento sites and all of them are full of duplicates in Google. Thanks for the help.

            • chris says:

              (doesn’t mean I’m good)

            • Ksenia Dobreva Ksenia Dobreva says:


              you can check our site – it’s on Magento.

              • chris says:

                I mean a classic ecommerce magento www shop with at least 500 products, filters, colors, ajax etc… for google says 2400 indexed pages (1900 without your blog) for just 124 products, does it sound right?

                • Ksenia Dobreva Ksenia Dobreva says:


                  I understand what you mean, but the number of products doesn’t mean anything here. If the settings are right, there won’t be duplicate issues even for a large store, and if the settings are wrong, there will be duplicate issues for a small store as well.
                  As for our site, if you look through the indexed pages, excluding blog and support articles, here’s what you’ll see: the products themselves, CMS pages, categories, knowledge base articles, user guides and demos for each extension. According to Google Webmaster tools, we have around 500 pages indexed by Google (this is a more exact data if compared to what the SERP tells you). If you search for any product and limit the search by our site, you’ll see we have absolutely no duplicates.
                  I know that you’re frustrated by what’s happening by your site, but it’s difficult to say what’s happening without seeing the site itself. Are you using any SEO extensions? Have you looked through Google Webmaster Tools?

      • chris says:

        Yes, titles are unique.

  28. Simply says:

    The author needs to brush up his knowledge about use of canonical tags on paginated pages. Paginated pages are not the case of content duplication. More info. on Official Google Webmaster Blog here.

    • Kristina Azarenko Kristina Azarenko says:


      Thanks for your comment. You are absolutely right about paginated results and using rel=canonical (though paginated URLs ARE partial duplicates as they generally have the same navigation, sidebars, footer, meta data and similar content).
      Using rel=next/prev is also mentioned in the article as a way to indicate paginated results. Although this is a preferable way of handling such URLs, it’s not always possible to implement it, and using canonicals in this case is possible if to choose between them and no changes at all.

  29. Irina Plantier says:

    Howdy! I simply want to give a huge thumbs up for the nice info you will have here on this post. I will be coming back to your weblog for extra soon.

  30. Parvesh Sareen says:

    You are a life saviour.. This article is gonna my all time favorite.
    I am working on a large ecommerce website with millions of pages. And duplicate content is the major issue, I am trying to overcome.
    I am sure using some of your advice, may surely help to deal with duplicate content to a large extent.

    Thanks Heaps..I really appreciate..!!

Leave a Reply

Your email address will not be published. Required fields are marked *