Survey designers beware…
Posted by bbc on 27 Jul 2007 | Tagged as: general
Even to those of us with an overdeveloped desire to be helpful and assist others with research, etc (e.g. especially people trained as librarians), the insistence of some companies on requiring a “readership” survey before they let you link on through to an online article is simply stupid. What do they use that data for – and how reliable do they think it is? Does a newspaper on the west coast really think that a 5-year-old from Florida is reading the online paper with avid interest? Or do they – like so many other people with surveys – throw out the data that doesn’t interest them or doesn’t match up with what they’re trying to prove?
With only a slight increase in effort I could fill up their survey with data that would seem reasonable – that is would list zip codes from their surrounding area of interest and would list ages that are readers they want to attract. How could they possibly know that isn’t happening? If a normally law-abiding, rule-enforcing person like me is annoyed enough by the pop-up that simply delays my getting anywhere to deliberately enter incorrect information, what about the people who always answer in ways that are misleading if not downright false? There’s no way to check – and surveys by their very nature don’t reveal very much about the actual users. Good survey design is hard and getting the right people to take it and give real answers is even harder. Purporting to be a “news” institution and deliberately blocking the easy access to that news is counterproductive.
The New York Times of course requires me to register and only allows access to certain content unless you’re a paying customer. I find that much less objectionable – they have a clear policy and it’s easy to know what you can get and what you can’t. The “pop-up” folks on the other hand purport to have a free site but annoy you constantly with those idiotic readership survey questions about age and zip code. The New York Times doesn’t care much about that – or if they do they ask it once and you’re done. If you allow the cookie to be saved on your computer you may never have to even enter your password again (in practice of course you should make sure you know your password because you’ll need it to access NYT on another computer or in case your hard drive goes south as so many do).
Other news sites follow a similar model of registration once – or actual paid subscription once – but that’s it. And CNN lets you into almost everything all the time. I admit I don’t go there very often but every time I have, it’s been relatively easy to use and not annoying in that nagging way.
So beware of “surveys” and of web designers who tell you they could just put in a little survey and you’d get useful information and it would hardly be an intrusion at all. Given the increasing overload of information, communication and attention demands, any unexpected set of questions is annoying and having them pop up more than once in a relationship is enough to cause some of us to start making the results more creative than your designer expected them to be. So the person who gets stuck doing data analysis will either be very impressed by your readers in Australia or will decide that the data is somehow corrupt and toss it out…
My desk was behind this column.