Stompernet faculty member & "Chief Scientest" (sic) Andy Edmonds pointed out a very useful resource the other day:

Seven Pitfalls to Avoid when Running Controlled Experiments on the Web is a great white paper by Thomas Crook, Brian Frasca, Ronny Kohavi, Roger Longbotham from Microsoft. Check out the site for the MSFT Experimentation Platform while you're at it. Cool stuff.

For those of you who don't know Andy, he used to work at Microsoft, on the Live Search team. He does research, actual work, tool development, and training for us at Stompernet, on analytics, usability, testing, and conversion. He is smarter than everyone I have ever met, combined. Andy's "Always Be Testing" blog is required reading for anyone working in any of the above areas. Seriously.

Andy's "Scrutinizer" tool is also one of the coolest free things any web designer, conversion/usability professional, or web entrepreneur could ever wish for. It lets you see your website the way your visitors do, by simulating human vision. This allows you to spot subtle weaknesses in design, usability, and conversion that would otherwise require expensive eye-tracking studies to understand.