How Do Really You Know?

When you write a weekly column like this, you're bound to get an assortment of reactive mail. Sometimes it's flattering. Sometimes it's threatening. And sometimes, it's even on topic.

Last week, a Mr. Richard Feder from Fort Lee, New Jersey, zapped in this question: "I'm wondering if you could tackle the topic of doing demographic surveys on your site. How does one do it right? What are the pitfalls to avoid? How do you know if the answers are truthful?"

Well, Mr. Feder, I certainly can tackle that topic. In fact, I'd be willing to wrestle it to the ground and pummel it for you. And what better way to begin the pummeling than by pounding out Rule #1, the Big Kahuna of all research rules:

Thou shalt keep it neutral at all times. You see, the biggest goof do-it-yourselfers make is tipping their hand when soliciting data from respondents. The way you ask your questions is even more important than the veracity with which your respondents answer them, because people with fake names have nothing to lose. On the other hand, anyone with any name will be led astray if you ask the question in a leading or influential manner. After all, which of these questions do you suppose would be answered more truthfully:

1. My neck is (A) Small (B) Medium (C) Large (D) Extra Large

2. The collar size I wear most often is (A) >15 (B) 15 (C) 16 (D) 17+

Okay, so neck sizes aren't terribly stimulating. But if you're in the rag business, this is can't-put-it-down kind of stuff. One look at question #1 ought to tell you that it's way too personal, because if focuses on the respondent's self-image. Chances are -- and I know I'll get mail on this -- male respondents will edge toward the Extra Large option, whereas the fairer sex will tend to gravitate toward the more diminutive. When you shift the same information over to the SHIRT, however, you're shifting the respondent's attention away from him or herself, and on to the garment.

Sneaky, eh? But it sure does net you the information you're after, in a much more reliable way.

Okay, so now that your survey is squeaky clean from bias, how do you know your answers are truthful? Easy! MORE TRICKERY! Oh, come on. You do this every day to find out if that guy or girl you ran into at the library has a significant other. You can be just as clever when it comes to verifying the demographics of your respondent audience. It mostly depends on how you bait your victim.

1. MAKE IT SHORT: Keep it really simple and quick. After waiting for your 154K graphic to load, the last thing anyone wants to do is spend more time at your site doing things they don't want to do. Four, five questions tops.

2. DON'T ASK FOR REDUNDANT INFORMATION: Don't waste your questions on things like Name and E-Mail Address. People regard that as very personal information. If you have a Submit form, it can trap the user's name and e-mail anyway (provided their browser's configured that way). If they don't fall for that, don't worry, because of #3.

3. KEEP YOUR EYE ON THE BALL: If you want e-mail addresses, ask for those. But if you want information, don't worry if you don't get e-mail addresses. Many people will give you better information if they can stay anonymous.

4. BRIBES WORK BEST FOR E-MAIL ADDRESSES: Hey, if you're offering a free something or other, people are going to tell you where to send it. To verify, make sure your delivery process is a two-step process, requiring them to send you a confirmation of their order.

5. MAKE THEM ONE OF THE CROWD: Nobody likes to feel they're the only guinea pig in the lab. By placing a Guestbook on the site -- one with a public comment section -- you can lull your respondent out of their garden-variety internet suspicions by allowing them to view previous respondents' comments. The more friendly your site appears, the more likely your prospects are to let their guards down.

6. MAKE IT FUN: Come on. It's the least you can do for them helping you out. If you come off too serious, they'll think you're fronting for the CIA.

7. SUBMIT/E-MAIL THE QUESTIONS: Once you've got them answering the questions, make it simple to get them out of their faces. Click a Submit button and send it on its way to some safe harbor where you can analyze the heck out it.

So Mr. Feder, if you're still wondering whether anyone can get 100% truthful answers from a survey, the 100% truthful answer is no. But don't feel bad -- even CNN, CBS, Newsweek and TIME magazine polls carry a margin of +/- 3%.

And least that's what they admit to. Who knows if they're telling the truth?

Rob Frankel