J. Cornelius

During a session at SXSW on Web Standards and SEO (transcribed here), the conversation revolved around how search engines treat semantically correct, valid pages versus the spaghetti mess of code that makes up the bulk of the web. Tim Mayer from Yahoo! said that they do look at valid markup as a 'signal of quality', but can't give extra weight to pages using web standards yet because there are not enough of them. I say this is a bit like the chicken and the egg.

Let's be honest, search engines are a driving force in how we design our sites. This is becoming more and more true each day. There are a plethora of SEO experts, SEO services, and volumes of writing about how to best design your pages for search engines. All of these people dedicating their lives (and a significant bit of money) to the practice of designing for search engines. Now I'm not saying this is a bad thing, it just puts the focus on the wrong place.

We should design for people, not machines

We hear the standards community preaching this every day and they are right. People come to our websites and people buy our products and services; not search engines. If I had a nickel for every time a search engine spider visited my site I could retire... quite comfortably. Fact is, the only way to build a successful website is to build it for people. It's fairly obvious to me when I visit a site that has been SEO'ed. It usually has a poor design, redundant text with no real message, and navigation clumped together in an incoherent mess. Yuck.

Can Web Standards Fix the Mess

Actually no. The mess is created by bad designers and developers. People who really shouldn't be making websites. I equate it to a brick mason that never checks to make sure things are square and level. After a short time the walls start to bend, weave, lean, and curve in all kinds of odd ways. Who would live in a lopsided house built by an unskilled craftsman? Not me; not you; but as the web goes, apparently most everyone.

Now the upside is that using Web Standards actually forces better design. Just think about making the brick mason use a square and level; and have a look at the blueprints to make sure his work is correct. Now we have a house you can live in. The more people use accepted and agreed standards, the more the quality of the work improves. Not a new concept in home construction, but still in its infancy on the web.

So what about the Search Engines

Ah yes, thanks for reminding me. Since the browsers aren't going to force people to use them, and authoring tools can't force people to use them, who can? That's right .... search engines.

By even slightly favoring sites that use web standards, valid semantic markup, and accessibility guidelines it will force developers to use them.

The SEO community is watching the search engines like a hawk to pick up on any trend that might help them and their clients to get better rankings. It would only take a minor shift in the algorithms to start the ball rolling. It actually starts with the engines de-mystifying a super-tiny bit of the way they rank sites. They could just say "we like sites with valid markup". That would push SEOs everywhere to start making sure pages validated. Then we can move towards symantec markup, on to accessibility, then to web utopia.

What's in it for the Search Engines

Having a bit of programming experience with a well known search engine reporting service I can say with certainty that parsing mangled markup is a pain in the ass. I can only imagine what the teams at Google and Yahoo! must deal with. Most of the pages we see are more than 60% markup and the rest is content. If pages were valid, it would clean up a lot of the code that spiders sites looking for content. It would also make it much easier to detect what page is truly about. Not to mention all the benefits for the people that make and maintain websites that come with less markup.

Eventually this momentum would grow and reach critical mass, then the engines could start subtracting rank from the sites that didn't validate, or had spaghetti markup. Then it would be forced upon developers to make standards compliant sites, or risk not ever seeing traffic from a search engine.

Why don't they do it?

There are 1000's of companies (probably more) out there with spaghetti websites that would jump up and down kicking and screaming if they had to spend another x-million dollars on their website just because Google, Yahoo!, and MSN "like sites with valid markup". I'm sure there are some backstage phone calls in the mix somewhere. Outside of the conspiracy/corruption theory I don't know. Perhaps Mr. Cutts can answer that one.

What can we do?

Not much beyond what we do already. I've seen multiple examples of sites that see search rankings go up once they fully embrace web standards. The best course of action is to continue to build sites for people first, with valid, semantically correct markup. There is already some obvious overlap in standards based design and search rank, let's just try to get the snowball rolling faster.

Tim Mayer said in the SXSW session, "We are trying to lift the quality of the search engine results. Using web standards could be a concise way of doing that." Let's just hope it evolves that way.