Where business and technology create a winning customer experience.

If you care about how your site is found in organic search, you must spend some of your time thinking about search engine optimization (SEO). In the olden days (2005), certain kinds of content had no shot of showing up in the search index (and thus, could never be found). But in recent years, more and more dynamic content is showing up in Google’s search index, as Google makes its spider smarter and smarter.  So, now there’s nothing to worry about with dynamic content, right? Not quite.

I don’t want to downplay the amazing strides that have been made by the Googlebot. Google has worked tirelessly with Adobe to make Flash content indexable. If it is a Flash video, there isn’t much text to index, but many Flash experiences are full of text and Google can index a lot more of it than ever before.

Huntsman spider.

Image via Wikipedia

Similarly, dynamic content generated from databases is indexed better than it once was, so it is less important to hide dynamic URLs than in years past.

And then there was the tweet heard round the SEO world in November, when Google’s Matt Cutts confirmed that Facebook comments are now being indexed. That might sound like a small thing, but SEO gurus know that it is one more step in Google’s road to conquering a very difficult problem: understanding everything a developer can do with JavaScript.  Just as a browser contains a JavaScript interpreter to render pages correctly, now Google’s spider contains some of that ability. Already, some are wondering how to take advantage of the new smarter Googlebot.

But it’s not smart to count on any of this dynamic content being indexed, for a few reasons:

  • Better ain’t necessarily good. Sure it works better than it did, but if it omits any of your content, you’re losing something. By using tried-and-true techniques that avoid dynamic content, all of your content gets indexed, which still seems like the way to go.
  • Google ain’t the only search engine. Sure, it’s nice that the Googlebot is getting so smart, but Bing runs 30% of U.S. searches and many other search engines grab market share around the world. Why hide your content from them?
  • The negative effects can be bigger than you think. When the spider fails to identify dynamic content, you might lose a lot more than a few words on a page. If that content contains links, the spider might not see whole pages on your site, and whatever pages THEY link to.

So, I’m a technical guy, and I really love to see the spiders getting smarter. It would be great if any Web page that can be rendered properly in a browser could be crawled and indexed by all search engines. It would make SEO a lot simpler and would allow us to concentrate on content rather than technical mumbo-jumbo.

But we’re not there yet. So, make sure that you know what the spiders see (all of them) before you employ lots of dynamic content techniques.

Enhanced by Zemanta

ico-rssLike this post?
Sign up for our emails here.

Mike Moran

About Mike Moran

Mike Moran has a unique blend of marketing and technology skills that he applies to raise return on investment for large marketing programs. Mike is a former IBM Distinguished Engineer and a senior strategist at Converseon, Revealed Context, and SoloSegment. Mike is the author of three books on digital marketing, an instructor at Rutgers Business School and a Senior Fellow at the Society for New Communications Research. He is also a Certified Speaking Professional.

Tweet about this on TwitterShare on FacebookShare on Google+Pin on PinterestShare on LinkedInEmail this to someone
6 replies to this post
  1. With the right CMS, dynamic content isn’t a big fear. Good coders are able to make sure Google can crawl your page effectively, and you can still rank well for relevant keywords.

    Unfortunately, most web designers don’t put as much effort into the technical side of things.

    Good post and we look forward to more.

  2. Do search engines index Disqus content?…

    Disqus comments can be indexed. This either requires special server side setup or relying on Google only which can index dynamically loaded content. The surest way of getting Disqus indexed is rendering comments server side (e.g. using the Disqus / Wor…

Leave a Reply


Are you confused and frustrated by the rapid pace of change in digital marketing? Do you find yourself struggling to get ahead...