Neither approach is optimal.  They don’t look great to your primary customer — the survey taker — perhaps reducing your response rate. What’s more, you may be limiting your options for effective survey logic.

Every question on a new page

The survey taker has to check the “Next” button too many times, with each click giving an opportunity to think about quitting.  Each new page requires additional information to be downloaded from the survey host, causing extra time delay.  If the survey taker is using dialup, or your survey uses lots of unique graphics, the additional delay is likely to be noticeable, but in any case you create an unnecessary risk of looking stupid.

One reason for surveys being created like this is is a hangover from early days of online surveying when limitations were common, and as a result surveyors may think it is a best practice.  Another possibility is leaving a default set in the online survey design tool for placing each question on a new page.  But, rather than just programming without thinking, try to put yourself in the mind of the survey taker, and consider how they might react to the page breaks.

Most surveys have enough short questions that can be easily combined to reduce the page count by 20% or more.

It is generally easy to save clicks at the end of the survey, by combining demographic questions, and this is a great way of reducing fatigue and early termination.  However, try hard to make improvements at the beginning also, to minimize annoyances before the survey taker is fully engaged.  If you have several screening questions there should be opportunities to combine questions early on.

Be careful that combining pages doesn’t cause problems with survey logic.  Inexpensive survey tools often require a new page to use skip patterns.  Even if you are using a tool with the flexibility of showing or hiding questions based on responses earlier in a page this usually requires more complex programming.

Everything on one long page

People who create surveys on a single long page seem to be under the impression that they are doing the survey taker a favor, as their invitations generally promote a single page as if that means the survey is short.  Surveys programmed like this tend to look daunting, with little thought given to engaging with the survey taker.  There might be issues for low bandwidth users (although generally these surveys are text heavy with few graphics, so the page loading time shouldn’t be much of an issue).

Single page surveys rarely use any logic, even when it would be helpful.  As described above it may more difficult to use logic on a single page.  I often recommend that survey creators build a document on paper for review before starting programming, but single page surveys often look like they started with a questionnaire that could have been administered on paper (even down to “if you answered X to question 2, please answer question 3“), but that misses the benefits of surveying online.  One benefit of surveying online that isn’t always well understood is being able to pause in the middle of a survey and return to it later.  This feature is helpful when you are sending complex surveys to busy people who might be interrupted, but it only works for pages that have been previously submitted.

One of the most extreme examples of overloading questions on pages I’ve seen recently printed out as 9 sheets of paper!  It also included numerous other errors of questionnaire design, but I’ll save them for other posts.

In the case of long pages, consider splitting up the questions to keep just a few logical questions together.  For some reason, these long page surveys are usually (overly) verbose so it may be best to just use one question per page, or, more productively, reviews by other people to distill the questionnaire to the most important elements with clear and concise wording.

To finish on a positive note, one of the best online surveys I’ve seen recently was a long page survey from the Dutch Gardens company.  There were two pages of questions, one with 9 questions and the second with 6, plus a half-page of demographics.  The survey looked similar to a paper questionnaire in being quite dense, but it didn’t look overwhelming because it made effective use of layout and varied question types to keep the interest level high.  None of the questions were mandatory, refreshing in itself.  And the survey was created with SurveyMonkey — it just goes to show what a low-end tool is capable of.  This structure was possible because the survey was designed without needing logic.

I hope that you’ll get some useful ideas from this post to build surveys with page structure that helps increase the rapport with your survey takers.

Idiosyncratically,
Mike Pritchard

To understand this, think of what happens when you are asked to choose a question by a telephone interviewer.  When the list of choices are presented for a single choice question, you might be think of the first option as more of a fit, or perhaps the last option is top-of-mind.   The problem is even more acute when the person answering the survey has to comment on each of several attributes, for example when rating how well a company is doing for time taken to answer the phone, courtesy, quality of the answer, etc.   As survey creators, we don’t know exactly how the survey taker will react to the order, so the easiest way is to eliminate the potential for problems by presenting the options in a random order.  Telephone surveys with reasonable sample sizes are almost always administered with question options randomized for this reason, using CATI systems (computer assisted telephone interviewing).

When we create a survey for online delivery, a similar problem exists.  It could be argued that the survey taker can generally see all of the options so why is a random order needed?  But the fact is that we can’t predict how survey takers will react to the order of the options.  Perhaps they give more weight to the option nearest the question, or perhaps to the one at the bottom.  If they are filling out a long matrix or battery of ratings, perhaps they will change their scheme as they move down the screen.  They might be thinking something like “too many highly rated, that doesn’t seem to fit how I feel overall, so I’ll change, but I don’t want to redo the ones I already did”.    Often there could be an effect from one option being next to another that might be minimized by separating them, which randomizing will do (randomly).   The results from these options being next to each other would likely be very different: