Intro to Meta Elements: SEO Spider Test and Search Engine Crawler Tool

[This is part of the The Blogger’s Essential Guide to Search Engine Optimization Series.]

Throughout this Blog Series on SEO we’re going to take a look at some of the most important elements of your blog’s information and data, especially the information that the search engines care about.

As I mentioned in the practical introduction to SEO the more you know about search engines the more adequately you’ll be able to leverage and manage them well for your advantage!

As a result I’ve created a new tool that’ll help you visually see a number of the elements that a search engine sees – this way you can start thinking like a Search Engine a tad bit more.

Take a look here: SEO Spider Test & Crawl Tool.

It’s pretty easy to use and once you kick in your blog URL and the anti-bot code then you should see the system spit out something like this:

It might seem like a bit of gibberish but this information is what the Search Engines see when they see your blog and website! Sure, there are some other elements that particular search engines review as well that are related to their particular algorithm but from a pure content-perspective what you see is literally what you get.

Meta Elements: The What and The Why!

Technically the term for these pieces of information meta elements but don’t worry so much about the technical terms – your goal is to be able to both recognize them and understand their importance.

I could stare at this all day!

Meta elements provide the much-needed and helpful information for search engines to categorize them and catalogue their data for future retrieval for your much-needed visitors! This information is directly inserted into the resulting HTML document that is created (via WordPress or another Content Management System or publishing application) as it’s interpreted into the web and browser layer. In other words, you don’t see this information (or you don’t see it the same as a SE does).

The point is that these elements are a huge focus for you and I as it relates to search engine optimization (SEO) and how a SE ranks the sites compared to the many competitors that sit out there ready to gobble up your attention. For a long time up until the mid to late 90’s most SEs relied very heavily on this meta data to not only classify but to rank. Obviously the more you knew or the more you knew how to take advantage of this was a boon to your organization and website as those that knew how to manage and leverage their native meta data on their website saw much higher returns of traffic.

And an entire industry was born: Search Engine Optimizers (that’s what they were called and still are called today), professionals, consultants, and even full-blown businesses rose overnight charging tens of thousands of dollars to help other organizations compete in the growing landscape. Fascinating, right? Some of these organizations did it “right” (we call this White Hat SEO, or legal SEO) and some of these organizations played a bit “dirty” (we call this Black Hat SEO, or illegitimate SEO).

Meta elements today play a slightly different role on the ranking of your site now and their bottom-line utility has wavered a bit – does this mean that you can skip this post and move on from here without caring? No. Not a chance. This is where every blogger that cares about SEO needs to start and covering your bases from the foundation is the most important place to start!

Here’s what you need to ultimately remember: SEO for your blog is about everything that is on it and coming to it. The things on your blog are all the elements you might expect:

  1. Quantity and quality of content
  2. Text (spelling and grammar count)
  3. Images
  4. Media
  5. Links (working and broken)
  6. Meta Data
  7. Server Information
  8. Code, Scripts, other 3rd party elements, and technical precision of the work done (pays to get it done right!)
  9. Speed
  10. Language
  11. Uniqueness
  12. Redundancy
  13. Geography
  14. Advertising revenue yield
  15. Freshness
  16. And more depending on the specific SE algorithms

And some of the elements that are coming to it are as follows:

  1. Incoming links
  2. Volume of searches
  3. Consistency of returns
  4. Viewer traffic
  5. Time within the website directly
  6. Page views
  7. Visits and revisits
  8. Click throughs
  9. And other elements and factors depending on the specific SE algorithms

We’ll spend much more time in some of these areas in the next few blog posts in this series but this will give you a good and comprehensive overview that there is not only a lot to consider but a lot of opportunity to optimize and take your blog to some serious new heights!

Many of you may never have even considered or even thought about some of these elements – don’t worry! There is a very slim chance that you’re doing anything that’s critically and catastrophically reducing your search engine results but we’ll want to make sure in any case, right?

Meta Elements Represented in the Spider Tool:

So, back to the tool that I’ve released – here’s a breakdown of the elements that are listed via the tool, what they mean, and why they are generally important!

1. URL

This is simply your blog address as understood by the SEs. You’ll notice that it’ll take both http:// and www. or without.

2. Title

This is the actual Title of your site and what the SEs see. It’s important that this relevant to your site content and focus! You might include your “tagline” or catch phrase here or another mission statement-like saying. Relevancy is what is important!

3. Description

This is the description attribute that is support by most major search engines such as Yahoo and Bing. Google will fall back on this tag when information about your blog is requested.

Essentially this element should provide a concise and accurate explanation of what the content of the web page (or your blog) is all about. You have the opportunity here to craft something meaningful, memorable, and contextually relevant to your readers and the SEs, especially if they are unable to come up with it themselves.

Sometimes this information is automatically shown on the search engine returns and queries but sometimes not – the problem is when it doesn’t and it says something completely wrong (or something that changes so often as to confuse the user and the search engine).

Take for example the description of my blog when you Google “TentBlogger”:

Ah, what a great meta description!

My meta description is simple, to the point, and also keyword-rich as it relates to the things that my blog is all about:

Blogging Tips, Tricks, Tools and Practical Teaching covering SEO, WordPress, and Making Money with Your Blog!

And this is what SEO Spider and Crawler Test shows as well:

Not a coincidence! I made it that way!

See how my description is shown here? That’s what the search engine sees because that’s what I’ve asked it to see and that’s what the resulting user gets to see as well.

Simple management and curation of your blog meta information is vitally important, wouldn’t you say?

Finally, one neat thing you might notice is that my meta description has keywords related to my significant pages listed in Google as well. Coincidence? No way – I did that on purpose.

4. Keywords

The keywords attribute has been one of the most popular meta elements that most SEOs have considered to be on the top of the list. However, times have changed (as the algorithms of the SEs have changed) and this is no longer the dominant factor in search engine ranking. This is due to the fact that people tried to “Black Hat” their execution and create spam sites that took advantage of the SE’s systems.

Thus, search engines began to drop the support for this meta information and there is no general consensus as to how important it is in the major search engine systems. In fact, it’s been stated explicitly by some of the more important luminaries of SEO, like Matt Cutts of Google, that Google itself is no longer taking keywords into a strong account in their ranking formula.

Yahoo! and other search engines still do though and so the point of this is that you should still consider it, especially when it’s not too difficult to manage or maintain. So do it!

Here’s what my keywords are as produced by the Spider Tool:

More the merrier?

Here’s what is in in text form:

blog tips, blogging, seo, wordpress, wordpress themes, wordpress plugins, problogger, blog, blogging tips, tools, apps, making money

The natural question is how many keywords you should actually put in there and it varies. I’ve seen that you should keep it light (less than 5 or 6) and then I’ve seen some relatively bogus accounts on doing 30+. I’m pretty comfortable going up to 15 keywords myself (although I currently have just 12).

5. Size

This is the size of the page that’s being selected to test in bytes. You might see a different value in different calculators but this is generally important for one reason and one reason alone: The larger your size the longer it takes to download your information and the longer it takes the longer it takes your readers and search engines to get your content.

Slow performance will impact your ranking negatively! You don’t want a heavy blog and you don’t want to lose ranking and opportunity just because you really really really want that 10mb image and 30mb song to play in the background.

Of course there are tons of ways to reduce the overall page weight and size of your blog, including these strategies:

  • Throw out any page elements that aren’t essential.
  • Clean up your design and CSS.
  • Get rid of any heavy page scripts, frames, or 3rd party elements.
  • Compress and optimize your images and media.
  • Clean up your code, HTML, and interior elements.
  • And much much more…

We’ll cover more of these in a future post in this series very soon but you’ll want to keep these in the back of your mind.

6. Texts

The text area just simply shows all the text that a search engine sees when it indexes and crawls your blog. It should be all your content that you can see directly on your site. Nothing too fancy here.

But, you might want to consider the following meta information that can prove to be quite valuable like the number of words on your homepage (front page of your blog) and distinction.

7. Number of Words

There are a number of competing theories on the perfect number of words on a page that search engines take into consideration in their ranking system. My research and experience has shown that a good page that ranks well has 500 to 3,000 words on it which could be from about 2,000 to 20,000 characters.

Almost 1,500 exactly!

This is a good number to strive for and with a blog there are some distinct ways of managing this to optimize search engine returns, especially like something as simple as the More Tag in WordPress.

As you can see by my results I try to hit right in the middle (1,500 words on the homepage) as often as I can. You can be sure that I do this after I publish a post every single time to make sure I’m in this “sweet spot.”

Heck, it’s really easy for you to do this too now with the Spider and Crawl Tool – every time you hit the publish button come back to the tool and see how many words you’ve got on your homepage. If you’re hitting a bit too high then use that More Tag to reduce the number of words on the main page.

Easy, right? And make sure that your sidebar content isn’t taking up all that valuable text!

8. Number of Distinct Words

The number of distinct words is also important as this is directly related to what we call keyword density. I’m going to spend much more time on this in another blog post but generally this will show you the relative frequency of the word in the text on your homepage.

This is quite easy to calculate and isn’t rocket science by any means: If you have 100 words and you use the same word 5 times then the frequency is 5 or 5% keyword density.

The element that you should be concerned with is having the right mix and frequency – too little and SEs don’t pay attention and too many they will consider it keyword spamming and you’ll be penalized.

As I mentioned before I’ll go into this in much greater depth soon.

9. Keywords

This next section shows you some of the keywords that are currently on your homepage as well as the relative keyword density. We’ll take this information and use it more much later but it will behoove you to take a look and walk through some of your keywords as understood by search engines.

10. Links

This shows you what links the search engines see when they crawl your homepage. These provide valuable information to not only the sites that are directly linked but also in your own blog as well. The better quality links that are relevant and contextual the better you rank.

Also, the number of links that you have play a role as well. It has well been understood that you should have no more than 100 links per indexable page. As you can see the number of links that I have is currently 64, well within that region. You should optimally shoot for this as well.

So yummy.

11. HTML Source Code

Finally, if you should ever need to see the actual HTML code of your blog and site it’s provided there as well. We’ll dig into this section more as well but this is what the search engine interprets when it’s crawling and spidering your site.

Well that’s that! There is a general overview of not only the tool but the resulting meta information that you’re going to want to care about.

Let me know if you have any questions and please feel free to share the new Spider and Crawl Tool with others! Thanks so much guys!

By the way, I love building apps for you all – makes my heart smile.

[This is part of the The Blogger’s Essential Guide to Search Engine Optimization Series.]