Optimizing Your Configuration Settings in Google Webmaster Tools

Simple stuff... but important!

As we walk through this series on Google Webmaster Tools there will be a few areas that won’t take too much explanation and there are some other areas that we’ll spend a little bit more time in.

This blog post, for many of you, will be one in which you might simply blaze through and be quickly able to say “Done!”

These are important steps to work through so don’t skip them if you haven’t ever really taken a look at them though! And heck, it could be a good review for you too – might as well, right?

So, let’s just get in it, shall we?

1. Sitemaps

I’ve covered the importance of Sitemaps previously here, here, and here so this will be a review for some of you but the point is that you should want to have one, especially if you want your blog to be indexed and found more effectively by search engines and Google!

A sitemap also helps Google and other search engines find content and pages that they might not otherwise find directly, especially if you’ve got a complicated blog with tons of links, pages, and posts.

Adding/submitting a sitemap is simply something you should do and it’s easy to add one to Google Webmaster Tools:

Easy! Just put it here...!

The challenge is creating one, right? I’ve outlined how to create a simple sitemap right here, and here! Just copy and paste the link into your Google Webmaster Tools when you’re ready to go!

2. Crawler Access

Setting your crawler access enables you to “hide” content at a particular link, remove links that are old and/or expired, and even to create a new robots.txt file.

I’ve provided some direction already about Robots.txt file so you’ll want to review that really quickly but you probably won’t have to do much here – in fact, most people get away with not ever having a robots.txt file and doing just fine.

As you can see here’s my existing file:

Nothing fancy.

One typical complaint or issue that people have has been that Google still provides links in Google SERPs (Search Engine Results Pages) despite being explicit in their robots.txt file. Matt Cutts has a simply explanation and clarification for that here:

[tentblogger-youtube KBdEwpRQRD0]

Got it?

Finally, if you do need to get rid of an old URL you have the options available to you in this configuration setting as well:

Be gone!

It’s worth noting that you should review the removal requirements via Google if you need this tool.

Many of you may want to look into this a bit more, especially if you have long-standing blogs that has content that no longer applies or that you’ve gone through an extensive rebranding process and would like those links to no longer bump more relevant and recent content. People who really architect well can see some substantial bumps in traffic if the changes are warranted and reasonable.

3. Sitelinks

What are Sitelinks? According to Google:

Sitelinks are links to other pages on your site that may appear under the search result for your site. Google generates these links automatically, and not all sites have sitelinks.

So, if they happen “automagically” then how do we increase the chance of them occurring (which is a good thing)? I wrote a big post on increasing the chance of getting sitelinks!

Go review that. Once you have them you can begin to play around with the specifics. Most people won’t have these for a bit of time so don’t worry, but occasionally you should Google yourself and see if they appear.

And then guess what? Go celebrate!

4. Change of Address

If you’re headed to greener pastures and are changing your site domain and address you’ll use this tool to help Google recognize the change and then index the page and site faster. Most of you can just skip right over this but if you do end up re-branding or changing things entirely you’ll want to make sure this is on top of your list of priorities!

Here are some of their suggestions that you should implement:

  • If you’re changing your domain because of a rebrand or redesign of your site, you might want to think about doing this in two phases: first, move your site; and secondly, launch your redesign. This manages the amount of change your users see at any stage in the process, and can make the process seem smoother. Limiting the number of changes at any one time can also make it easier to troubleshoot unexpected behavior.
  • Use a 301 Redirect to permanently redirect all pages on your old site to your new site. This tells search engines and users that your site has permanently moved. We recommend that you move and redirect a section or directory first, and then test to make sure that your redirects are working correctly before moving all your content.
  • Don’t do a single redirect directing all traffic from your old site to your new home page. This will avoid 404 errors, but it’s not a good user experience. It’s more work, but a page-to-page redirect will help preserve your site’s ranking in Google while providing a consistent and transparent experience for your users. If there won’t be a 1:1 match between pages on your old site and your new site (recommended), try to make sure that every page on your old site is at least redirected to a new page with similar content.
  • Check both external and internal links to pages on your site. Ideally, you should contact the webmaster of each site that links to yours and ask them to update the links to point to the page on your new domain. However, if this isn’t practical, make sure that all pages with incoming links are redirected to your new site. You should also check internal links within your old site, and update them to point to your new domain. Once your content is in place on your new server, use a link checker like Xenu to make sure you don’t have broken legacy links on your site. This is especially important if your original content included absolute links (likewww.example.com/cooking/recipes/chocolatecake.html) instead of relative links (like../recipes/chocolatecake.html).
  • To prevent confusion, it’s best to retain control of your old site domain for at least 180 days.
  • Use the Change of Address tool in Webmaster Tools to notify Google of your site’s move. (Note: To use the Change of Address tool, you must be a verified owner of both the new and the old sites.)
  • Add your new site to your Webmaster Tools account, and verify your ownership of it.
  • We recommend that you create and submit a Sitemap listing the URLs on your new site. Submitting a Sitemap is a way to tell Google about pages on your new site that we might not otherwise discover.

Got it? Good!

5. Settings

Here are some important settings that you’ll want to at least take a look at before you go further. For example you can set your Geographic target to help Google provide results in a specific geographical location, preferably the main source of your audience and language.

What does this do? It helps determine how your site appears in search results and can even improve search results for those queries. Note that you can only do this for neutral top-level domains such as .com and .org. If you have any “funky” country-specific domains then it’ll be automatically associated with that particular country.

A common question is whether having a non-neutral domain will hurt you if your core audience is english-speaking or in the United States. The answer is no. Just Google “8bit” and you’ll find my startup company at the top of the list:

Oh yeah.

Oh wait a second – your domain is 8bit.io…!!! Yes, it’s not a .com or a .org but it still ranks pretty darn high. It’ll dip below the top on occasion but that doesn’t bother me one bit.

In terms of the Preferred Domain setting this is up to you but the point is to choose one! This is one of the strategies to avoid duplicate content. Most people will be rejected at the door at see something like this initially:

Fail!

All you have to do is add the one domain that you haven’t added or verified so that you have both:

Twins?

And then you can set your preference:

Win!

Done and done!

Finally, in terms of Crawl Rate I always go big, especially if you don’t have any issues with server speed or load:

Why not, right?

But make sure that you make your server faster, optimize your WordPress blog for speed, upgrade any hosting/hardware that you’ve got and just make your blog scream speed. This will help too.

It’s on a 90-day cycle so you’ll have to go back after three months to keep the crawl rate high. Just put it on the calendar and set it!

6. URL Parameters

This one might be the most complex to understand (at first) but if you can wrap your head around it and you spend some time optimizing you’ll find some really great results!

Telling Googlebot about each parameter’s purpose helps them crawl your site more efficiently saving you bandwidth and boosting the overall number of unique pages Google can crawl. This is only a good thing!

But you must first identify them and then deal with them. One of the most obvious (and easiest) is the multiple instances of your blog’s comments: “replytocom=xxxx”:

Deal with this quickly!

As you can see I have all these new “instances” or pages that could be indexed by Google every time someone new comments and could be considered duplicate content or even dilute your results. Google attempts to “pick” the right one and all we’re doing is telling them the right one so that they don’t have to risk getting it wrong!

What I choose to do is to have “one representative URL” instead of a new URL per comment. This means that Google can optimize their search returns and that my core content is not diluted by all the comments. Some would argue that they think the comments help increase page results and ranking and they make a decent and valid case but I haven’t found this to be necessarily true.

I wrote my content to be found by people and be optimized – I love the comments but they may not be guaranteed to be “on point” and focused as the content that was originally created.

Does this make sense? Good! You can do this for a lot of parameters but start with just one for now.

Next time we’ll be diving into some neat things about queries and your site as seen by search!

[This is part of the Bloggers Guide to Google Webmaster Tools.]

  • http://www.savetheartist.net Chase

    The URL Parameters won was entirely new for me. Fixed now so thanks!

    • http://john.do John Saddington

      sure thing chase!

  • sandeep kumar

    Great effort John….

    • http://john.do John Saddington

      thanks sandeep!

  • http://www.veight.de Stefan

    When adding both domains (with and without “www.”) is it necessary to change options on both profiles?
    for example I only worked without the www. profile, now I’ve verfied the www. address :X

    Thank you!

    • http://john.do John Saddington

      i do not believe so!

  • http://aushiker.com Andrew Priest

    Thanks John for the tips. I have had a little fun playing with my settings. Will be interesting to see the results.

    • http://john.do John Saddington

      can only be better results, right?

  • http://ziolowyzakatek.com.pl Atria

    John, as always, great post!
    I stick to your blog for more than few months and I’m never dissapointed by your advice and encouraging do move!

    Of course I change settings in Webmaster tools and I’m waiting for more!

    Thanks.

    • http://john.do John Saddington

      cool! glad you can “hang” in there!

  • http://CaptivatingCappadocia.com Duke Dillard

    This series is great, John. Can you explain the parameters a bit more, please. How did you know about the replytocom parameter? Where did you find it? I am using Standard. Are there others? How would I know? Thank you.

    • http://john.do John Saddington

      duke,

      that was shown in google webmasters explicitly.

  • http://www.austintenantadvisors.com/ Nathan K Smith

    Hi John, Do I leave both sites…………. the www version and non-www version in Google webmaster tools or do I eventually delete the non-www version?

    Right now when I am in Google webmaster tools home page I see both urls.

    Also what settings do you suggest for Blocked URLS? I currently have:
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

    • http://john.do/ John Saddington

      i keep both.

      • http://www.austintenantadvisors.com/ Nathan K Smith

        Thanks John.

        What other robots txt settings do you suggest for Blocked URLS? I have a wordpress website and my settings in GWT currently are:

        User-agent: *
        Disallow: /wp-admin/
        Disallow: /wp-includes/

  • http://click-for-money.com nestorvibes

    Very nice work done here, thanks for this info. I’ve changed the crawl rate without consulting anyone, but it seems I’ve done the right thing afterall. Thanks

  • http://www.deviltools.net/ prudhvi raj

    very usefull post