PJ Hyett


© 2004-2011

Why SEO is Important

While I was at CNET, traffic was growing steadily on our sites, but we had a rude awakening one particular day.

The “Oh Shit” moment

I remember the morning when my boss said to me:

We’re having a conference call in a few minutes with CNET’s SEO dude, we lost half of our uniques starting last Thursday, why don’t you join in.

The first thing I did is wonder what the hell I did wrong, because in the back of my mind, this problem was inevitably engineering’s fault.

The realization that Google owns you

Chowhound was getting 85% of its traffic from Google. It has around 500k topics, averaging 4.5 posts per topic, each an opportunity to become indexed by Google.

This particular week, Google decided to purge its index of stale data, and in doing so, eliminated the majority of the pages Chowhound had indexed, thus slashing our uniques in half. An interesting side-note was that our pageviews didn’t drop nearly as much, because the core users that were still visiting the site had a much higher pageview per visit ratio than the Google visitors.

What we did wrong

The solution was actually quite simple. On every board, we originally used ‘Next’ and ‘Previous’ links that would paginate you to the corresponding page of topics. Straightforward, but horrible for Google’s bots. They’d click a few of the Next links and say the hell with this and move on.

To give you an idea, the Manhattan board alone has over 800 pages of topics, and when Google is only looking at the first three pages, you have a massive problem.

How we fixed it

Simply put, I wrote the will_paginate plugin. Instead of just having ‘Next’ and ‘Previous’, it also displays a window of page numbers that allows bots the ability to navigate hundreds more pages within one click. Now we’re getting somewhere.

Eureka!

As the traffic began to come back in a serious way, well beyond where it was before the dip, the notion of SEO became quite clear to me. Given a site with tons of great content, instead of building more features, you should be trying to leverage what you already have.

This also makes a lot of sense from a financial standpoint. How long would it take your team to build a new feature for 10k more uniques a day, how long would it take them to add meta tags and a sitemap for 10k more uniques a day. There’s no comparison.

The Checklist

SEO isn’t rocket science (unless you’re trying to push the envelope), you just have to be aware of a few guidelines.

Page-specific

  • Unique title tag
  • Unique meta description (Yahoo is the only one using meta keywords anymore)
  • One h1 tag (the text used in your h2-h4 tags is still very important, though)

Site-wide

  • Allow only quality outbound links (eg. rel="nofollow" on all user-generated links)
  • Block duplicate content from being indexed with robots.txt
  • Structure your CSS so the content can go first
  • Consistent navigation that allows bots to get to any page on your site within 2 or 3 clicks

Piece of cake. This discussion can go far deeper than these points, but if you do the stuff above, you’re 90% of the way there. The last 10% is the most time-consuming, so I leave that to you to decide if it’s worth your time.

I know a lot of people think of SEO as a dirty word, and that it conjures up negatives connotations of link-baiting and spam, but that really couldn’t be further from the truth. I’m just going to start calling it FT…Free Traffic.

Speaking of Which

Err Free does more than just Rails, we’d be happy to apply what we’ve learned about SEO to your site. Who doesn’t love Free Traffic?