The Shopify robots.txt file enables you to give commands to search engine crawlers such as Google. These instructions will tell them what resources are most relevant to your search. The file will also tell web crawlers when not to crawl certain resources, such as login or admin pages. What’s more, users of Shopify can also change the instructions to get a more accurate and useful result.
The robots.txt is found in the root folder of a user’s primary domain. Previously, it wasn’t possible to edit this file. As of June 16, 2021, however, this option is now available to Shopify users. Tobi Lutke, the CEO of Shopify, confirmed this fact by tweeting about it on the aforementioned date.
If you’re interested in editing your Shopify robots.txt file, here are the steps you’ll need to follow:
Before you follow these steps though, keep in mind that these kinds of customizations are unsupported. This means that Shopify support will not be able to help in troubleshooting any problems.
If you want to hide your Shopify pages from Google, simply change the instructions regarding this. You can do this in the following way:
At some point, you might also want to hide the Shopify search template from search engines like Google. Follow the steps below to accomplish this:
If you’re looking to hide one specific page on Shopify from Google web crawlers, here’s how to go about it:
The page handle in Shopify is the URL slug, which is the net of the domain. It’s basically the title of the object in lowercase letters. If there are any spaces in the titles, these are represented by hyphens. The same goes for any special characters such as a comma, semicolon, etc.
The Shopify Developers page has more information about Object Handle documentation. So, you can also go there for further clarification.
When you want several pages to stay hidden from search engine web crawlers, especially Google’s, here’s what you need to do:
If your aim is to hide dynamic Shopify pages from search engines, try these steps:
Disclaimer: Keep in mind that the methods we’ve discussed above do not involve editing the Shopify robots.txt. They’re aimed at adding a ‘robots’ meta tag to the pages. Even after you’ve made these edits, Google web crawlers might still crawl them. If you want to guide such pages from things like Googlebot, we’ll discuss the solution later on. This will also replace any need for editing Shopify robots.txt.
You should carry out the following steps for hiding Shopify pages from the web crawlers like Googlebot:
If you need help with getting started with Shopify SEO for your e-commerce site, set up a free consultation with Shout digital today. We provide PPC, social media, content marketing, SEO in Sydney, SEO in Melbourne, and other cities for both the eCommerce and service websites.
As Lead SEO at Shout Digital, Nandeep Kandimalla is at the forefront to create an SEO strategy to improve organic site visibility and lead conversion.
With over 9+ years of experience in SEO and a Master’s degree in Information Systems (Security) from RMIT Melbourne, Nandeep provides creative SEO strategies based on analytical data of past & future trends.
Connect with him through LinkedIn or visit Shoutdigital.com.au for a 20 min SEO consultation on how to improve organic growth.
We’ll see you on January 7 at 11am AEST
A confirmation has been sent to your email address.