Two days ago, Google unveiled Google Instant — the company’s newest solution to making googling even faster. The feature presents search results which appear even as you’re still typing (for background, watch the super-creepy, super-enthusiastic, almost Onion-ish video above). 

While that solution seems obvious and not particularly elegant — Is it just us who find our eyeballs spinning in their sockets using this thing? — the Google team apparently played with all kinds of different UI solutions to the problem. 

Google describes the challenge as being to present that mass of information whizzing by your eyes in a way that’s “relevant not distracting,” so the solutions thus had to do with grouping information in clever ways. For example, here’s one prototype that trying to group results from similar searches only after you typed: 

 

And here’s a version with the suggested search results to the left of the toolbar: 

 

Obviously, none of these were going to work — the first prototype above was bound to show you irrelevant searches; and the second simply increased the number of places on the screen you had to look to perform a search. A little design know-how would have made that obvious, but Google instead tested these ideas out. 

We ran through a sequence of prototypes, usability studies (testing with people from the community), dogfooding (testing with Google employees) and search experiments (testing with a small percentage of Google users). 

If you have any type of design background, it’s probably funny to you that Google frequently mentions “design,” but doesn’t mention any “designers” involved — the Google design process seems to simply be creating a bunch of fairly obvious alternatives, and testing the hell out of them. 

Before Google Instant, probably the most infamous example of Google’s design-by-testing approach was the “41 Blues” — Google’s engineers apparently couldn’t decide on two shades of blue for showing search results, so they tested 41 of them to see which attracted the most clicks. (They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.) 

You might imagine such a robotic approach to design would be frustrating for anyone with a design background. And you’d be right — Douglas Bowman, a prominent UI designer and now the creative director at Twitter, left Google, apparently due to day-to-day inanity of trying to actually design a user experience there: 

I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such minuscule design decisions. There are more exciting design problems in this world to tackle. 

He expands: 

When I joined Google as its first visual designer, the company was already seven years old. Seven years is a long time to run a company without a classically trained designer. Google had plenty of designers on staff then, but most of them had backgrounds in CS or HCI. And none of them were in high-up, respected leadership positions. Without a person at (or near) the helm who thoroughly understands the principles and elements of Design, a company eventually runs out of reasons for design decisions. With every new design decision, critics cry foul. Without conviction, doubt creeps in. Instincts fail. “Is this the right move?” When a company is filled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative effects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions. 

Sounds about right, given all we know about Google. 

But back to Google Instant. What’s baffling about the whole thing is that Google’s “solution” to providing instant results still seems so primitive and ugly. In the name of shaving a second off of a user’s search, is it really worth it to make them go through the pain of scanning five to seven different results pages as they type? 

A second might matter tremendously to an engineer — and indeed, Google is happy to point out that those seconds, over a year, add up to hours of time saved. But that’s kind of a silly way of thinking about it. After all, it’s not like you get all those hours at the end of the year as a dividend. If you saved a second while googling, what are you going to actually do with that second? Take another sip of coffee? 

The chief mandate of design thinking is empathy — and I’m pretty sure Google’s engineers didn’t have too much empathy for all those over the age of 28 who don’t find it all that useful to have their eyes assaulted by information they weren’t looking for in the first place. 

Which brings me to my last point. Testing can only tell you so much — and it often only reveals that people only like things that are similar to what they’ve had before. But brilliant design solutions convert people over time, because they’re both subtle and ground breaking. 

For example, imagine if Google, in presenting its progressive, instant results stripped all the text away from below the hyperlinks. What if the details only filled in as your typing began to slow? Might that have decreased the visual confusion? Could they at least have worked on a way of reduce the sheer pixels of information you have to look at, while also providing you with more content? I’m pretty sure that testers would have complained about “missing something.” But over time, I’ll bet that the reduced strain would have emerged as a boon. 

Testing can, at best, prevent massive mistakes. But it can also give you a blinkered perception of reality — and that’s just as dangerous. Remember that New Coke was vastly preferred over regular Coke in blind taste tests — the New Coke disaster happened because marketers never tested the deep real-world associations people had with the brand. Coke is bitter and acrid, and that’s what makes it a “Coke.” 

The exact wrong lesson to take from that is that you should test even more, with greater detail. The right lesson is that testing artificially limits the worldview of the people doing the testing. Market research can’t tell you whether the “problem” you’re trying to solve is even the right one to be addressing. It can’t tell you that the entire project you’re working on was a bad idea to begin with. 

Google is the perfect example of this. As they’ve added new products, has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? Have either of them become a pleasure to look at or play with? No. But why not? Every new Apple product seems to make progress on that — and I’m not even talking about hardware. Just look at the music listings in new, redesigned iTunes. 

Then again, I’ll bet the testers (and engineers) still prefer their Google as it is now, simply because it’s what they’ve always known.For more about Bowman’s departure from Google, click here.

For more about the infamous 41 blues, click here.[Top image: A Chinese factory photographed by Andreas Gursky]