10 Search Engine Limitations that Impact You and Your Blog

NS-5 : From the movie iRobot

[This is part of the The Blogger’s Essential Guide to Search Engine Optimization Series.]

As we continue to move forward through this series it’s worth mentioning not only the awesome things that search engines do for your blog but also the limitations that they have so that we can respond and react appropriately.

And this makes sense since search engine optimization is about making your blog the best that it can be for both the human and the robot (but not at the cost of the human, of course!) and if we’re to do this in the very best way it behooves us to know what we can optimize due to the technical nature of the search engine’s software.

Some of these elements will require you to make specific choices about not only your content but also how your users engage with your content – in other words, you’ll find that SEO is about finding the perfect compromise between the robot and your community.

And the more you know about these limitations and compromises the better you are at leveraging them to the fullest. Shall we begin?

Ever see one of these...?

Search Engines Are Not Perfect

One of the most satisfying things about search engine software is that they still aren’t perfect – there’s something still missing and I feel that’s healthy since it represents the inability to completely mimic that of the human being (at least for now). Like the NS-5 administrative robot in the movie iRobot it was good but not perfect.

With this in mind let’s remember that all search engines operate similarly – that is, they are pieces of software that crawl through your content indexing as they go along into databases that are larger than you could possibly dream of.

At the end of the day it’s just data and mining it, extracting it, and providing it back to the user is a significant modern-marvel; but it’s not without a few blemishes or hiccups or limitations. You see, the search engines can’t mine everything – in fact, there are certain pieces of code, design, and formats that are akin to massive brick wall that the spider can’t crawl or dig through.

Here is a list of the top 10 things that modern spiders, crawlers, and search engines can’t do with your blog (or for your blog) content:

  1. Flash – Search engines still have a bit of trouble indexing flash. Sure, Google and Adobe have released updates in the last few years (via services like Flex) that allow increased indexing of flash-based content but the challenge can still exist. The conservative blogger would opt for non-flash media when appropriate for better and more robust indexing opportunities.
  2. Frames – Using frames is just bad form – so don’t do it! In fact, some search engines completely ignore it altogether. If you have a few elements that sport frames then you’re missing out. It’s worth the effort to bring those natively to the surface for better indexing.
  3. Poor Link Structure – There’s nothing that a search engine can do to recover from a poorly architected website or blog. Guess what, that’s your fault! The challenge though is when you have the more “creative” blog designs that might look good but do terrible when trying to be indexed via a SE. Do you go with the clever and unique design or do you go with a more functional approach to your design structure?
  4. Forms – Search engines can’t fill out forms for you and so any content that is available after the form is completely lost. This is too bad since a lot of us have valuable content that might require a query (or two) to be accessed. What does a blogger do then? They’d have to come up with another way to present the information without a form request.
  5. Java Applets – These are interactive applications on web pages that you probably engage with every single day. They do include some HTML but they are run on the end-user’s computer and not via the web browser. Thus they can’t be opened, operated, or really touched by search engines. Sure, they make the web more cool and interesting but at the compromise and cost of not being able to help your SEO efforts.
  6. Plug-in Content – This is very similar to java applets as listed above.
  7. Audio, Video – Search engines are increasingly optimizing their software to index audio and video better but the fact remains that there are still elements in each format and medium that search engines cannot index. Our hope is that as the search technology increases these formats will get even more valuable (and they are already very valuable) but it’s something to consider in your creation of content as a blogger.
  8. Code – Any text that is not in an HTML format in an easy-to-understand way (we call this the ability to be parsed, or interpreted) is inherently invisible to any and all search engines. This is a general catch-all for such things as flash and swf files, images, video, audio, plug-in content, and more. The quality of your code matters and if it can’t be simplified to an HTML-level then you’re missing out.
  9. Internationalization, Language, Grammar – Many of you have probably not thought about the challenges of ranking and search as it relates to other languages and forms of grammar but it’s definitely something to consider. For example, different spelling of words impacts your SEO and these subtleties, while you may deem as unimportant, can be the difference between a first page ranking on a SERP to one on page nine. The difference is staggering.
  10. Hidden, Private Content – Finally, if you have content that is behind a pay-wall or subscription or simply private then search engines can’t get to it. If you’re blog community and user experience is based on the “private” model then you can kiss a lot of optimization and organic traffic goodbye. What will you do?

There are other factors but this will be enough for you to feel confident moving forward.

The Human Factor

Finally, one of the more “touchy-feely” elements of search and the limitations therein is the idea around quality. You see, there’s still the “Hey I’m Human” factor because at the end of the day a search engine is just that – a search engine. It cannot tell you what is more important or less important based on the “quality” of writing, the “beauty” of your prose, or the “visually stunning” landscape in which the content sits.

It parses the content and see words, calculates density, makes intelligent calculations based on clicks, pageviews, social proof, links, and more, but it can’t tell you if it’s staring at a Picasso-quality piece of work and my 5-month year olds vomit. Only humans can do that (and good search engine marketers and strategists).

"Panda Slapped" by Google? Oh what fun!

Now, we are getting closer to a world and a reality where Google and the other search engines are taking into account elements of quality, value, trust, and overall experience of the site (and this is a good thing!)

The most recent update (affectionately called “Panda“) made these changes known to the world and a few sites rejoiced while many sites got punched in the stomach. Elements of design, usability, content quality, usage metrics, and social proof became even more important in calculating SERPs.

What This Means for You and Your Blog

Ultimately I want to keep this series practical and it feels right now pretty heady if you ask me (although some of you might be thoroughly enjoying this). There are a few things that you’ll want to consider as we move forward:

  • Stay Updated on Search Changes – Were you aware of “Panda?” No? Yikes! How would you have know about this unless you had read this blog post?! The pressure is on – but don’t feel discouraged. You see, the point is this: The world of search is one that’s constantly changing with new elements being deemed as more important and new limitations being developed that will change the way you think about your content, especially if you want to stay and remain as competitive as you possibly can.
  • Know the Limitations – Become fairly aware of the listed limitations for search engines and think practically how this applies to your blog. For example, if we know that java applets are terrible for search and you have a bunch on your blog then you probably should do something about that, right?
  • Understand (and Respect) the Human Factor – Elements of design, trust, social proof, and usability are all more important that ever. Practically this relates to your WordPress Theme (and how to choose the right one) and even the plugins you use for your blog.
  • Don’t Worry Too Much – You’re not a search engine marketer, professional SEOr (Search Engine Optimizer) – you’re just a blogger. Your job is to know of the general principles (like those shared in the SEO Series for Bloggers) and stay abreast of the major updates that will impact your blog.

At the end of the day you will be responsible for knowing these things and the more you know the more effective you will become. You can bet that I think about these things fairly consistently (heck, it’s my job to think about them, right?) and I promise that if you begin to inherit this mindset as well you’ll begin to execute better than your competitors and when you do that you’ll be handsomely rewarded with increased traffic, more visibility, and higher financial returns.

I want to be on that side of the fence and I want you to be there with me.

[This is part of the The Blogger’s Essential Guide to Search Engine Optimization Series.]