Configuring URL Parameters in Webmaster Tools

TL;DR

Configure URL parameters in Webmaster Tools by first marking parameters such as session, affiliate, and tracking IDs as “does not change content,” then identifying parameters that sort, narrow, or otherwise change page content and choosing Googlebot’s preferred crawling behavior. Incorrect settings can drop pages from Google’s index, while careful configuration reduces duplicate crawling, saves bandwidth, and focuses attention on unique content. Read on for the key rules and risks.

Transcript

Maile Ohye: Hi. I'm Maile Ohye. I'm a member of Google's webmaster support team. And I'd like to help you better understand how to use the URL parameters feature in webmaster tools. URL parameters is a fairly advanced feature. So some of this information might be more complex than one would expect. Before you further watch this video, please chec... Read More

Key Insights

• URL parameters are an advanced feature in Webmaster Tools that help Google crawl websites more efficiently by providing hints on which URLs to prioritize.
• Improper configuration of URL parameters can lead to pages being dropped from search indexes, so it's crucial to understand their impact.
• Googlebot aims to crawl unique and useful content, reducing the crawling of duplicate URLs to save bandwidth and improve content focus.
• Parameters that do not change page content, such as session IDs or tracking IDs, can be marked accordingly to streamline crawling.
• Parameters that alter page content, like sort or filter options, require careful configuration to ensure important content remains accessible.
• The 'sort' parameter can be complex; if it’s optional or consistent across the site, specific crawling rules can be set to optimize efficiency.
• The 'narrows' parameter filters content; if it leads to less useful pages, setting it to 'Crawl No URLs' might be beneficial, but requires verification.
• For 'specifies' and 'paginates' parameters, it's generally best to allow crawling of every URL to ensure all content is indexed.

Questions & Answers

Q: How do you configure URL parameters in Webmaster Tools?

Start by identifying parameters that do not affect page content, such as session, affiliate, or tracking IDs, and mark them as “does not change content.” Next, identify parameters that change, reorder, or narrow content, select their effect, and specify how you prefer Googlebot to crawl the resulting URLs.

Q: What is the purpose of the URL parameters feature in Webmaster Tools?

The feature gives Google helpful hints for crawling a site more efficiently. It can reduce duplicate crawling, save bandwidth, and help Google focus on unique content rather than repeatedly crawling the same information.

Q: What are the risks of configuring URL parameters incorrectly?

Incorrect configuration can cause pages from a site to be dropped from Google’s index and stop appearing in search. Google therefore recommends avoiding the tool unless adjustments are necessary, especially because it is an advanced feature.

Q: When should you leave URL parameters unconfigured?

Leave them unconfigured if Webmaster Tools says Googlebot is not experiencing coverage problems and no adjustments are necessary. When unsure, remember that incorrect parameter settings can prevent pages from being crawled and indexed.

Q: Which URLs are eligible for the URL parameters feature?

Eligible URLs use key-value or name-value pairs, such as “category equals office” or multiple pairs joined by an ampersand. Parameter order does not matter, but URLs using plus signs, directory structures, or custom encoding instead of name-value pairs are ineligible.

Q: How should parameters that do not change page content be handled?

Parameters such as session IDs, affiliate IDs, and tracking IDs should be marked as “does not change content.” Webmaster Tools then uses one representative URL as the setting, and Googlebot acts accordingly.

Q: Does the URL parameters feature remove URLs or block crawling?

No; its purpose is to improve crawling efficiency, not to remove URLs or provide an explicit crawl block. The transcript directs users to URL Removals for removal and identifies robots.txt disallow or meta noindex as stronger directives.

Q: Can page-level markup be used with URL parameter settings?

Yes, Google can still consider rel="canonical", rel="next" and rel="prev", and rel="hreflang" alongside URL parameter settings. The page must remain crawlable, meaning it cannot be blocked by robots.txt or configured not to be crawled through URL parameters.

Summary & Key Takeaways

• Maile Ohye from Google's webmaster support team explains the use of URL parameters in Webmaster Tools to enhance crawling efficiency. She emphasizes the importance of correct configuration to avoid dropping pages from search indexes and advises on handling parameters that do or do not affect page content.

• The video outlines different types of parameters such as 'sort', 'narrows', 'specifies', 'translates', and 'paginates', and provides guidance on how to configure them. It highlights scenarios where Googlebot's crawling behavior can be tailored to improve efficiency and reduce redundancy.

• Ohye advises webmasters to start by identifying parameters that do not change page content and then proceed to configure those that do. She stresses the importance of letting Googlebot decide when in doubt and provides best practices for organizing translated content in sub-directories.