Exploring online research methods - Incorporating TRI-ORM

Design of online questionnaires 1: Appearance

[Skip instructions]

[i] Click on the headings to open them. They will open on this page. Open the following link for further information about these headings if required.

Your browser does not support these headings. To ensure that the contents remain accessible, they have been automatically opened so that all the information on the page is displayed.

However, to take advantage of the headings and to ensure that the layout and design of this site are displayed correctly, you are recommended to upgrade to a current version of one of the following standards compliant browsers:

There are references to sources and further reading within the text. You can view the full reference by clicking on the name to open a 'pop-up window'. You can then add comments to these references and include them in a personal references list.

Ongoing instructions are provided, but if you would like to read more information on how to do this before you begin, or if you experience problems, select this link for instructions on how to use the personal references list

Instructions:

  1. Select the references to see full bibliographic details in a pop-up window.
  2. NB. If you use pop-up window blocking software, you will need to deactivate it for pop-ups on this site to use the reference list. Alternatively, all full references can be seen by navigating to the 'References' page.
  3. If you would like to add a comment to attach to your record of the reference, write in the text box.
  4. Select 'add to list' to add the reference and comment to your list.
  5. You can view your references at any time, by selecting one of the 'Show references list' links. This will open your list in a pop-up window.
  6. NB. Each module has a different reference list. If you are navigating between modules, any references collected will be saved to different lists. To view the list for a particular module, select any 'Show references list' link within that module.
  7. If you leave this page, your list will be saved and will be available for you to refer to again if you return.
    (This will not work if you have disabled cookies in your browser settings)
  8. NB. Comments will not be saved if you navigate away from the page. You should copy all comments before you leave if you would like to save them.
  9. Use of the references list is JavaScript dependent. If JavaScript is disabled, it will be necessary to open the 'References' page to view the full references.

Glossary links are also included within the text. If a word appears as a link, clicking on this link will show the definition of the word in a 'pop-up window'. Select the following link for information about these glossary links if required.

  1. Select the links see the definitions in a pop-up window.
  2. NB. If you use pop-up window blocking software, you will need to deactivate it for pop-ups on this site to use the glossary links. Alternatively, all glossary definitions can be seen on the 'Glossary' page in the 'Resources' section.
  3. Use of the glossary links is JavaScript dependent. If JavaScript is disabled, it will be necessary to open the 'Glossary' page to view the definitions. Opening this page in a new window may allow you to refer more easily to the definitions while you navigate the site.

Open/close headingDesign features of online questionnaires

Design issues are an extremely important consideration for online questionnaires because of the highly visual nature of the web environment, and the variety of technical skills of survey respondents. An inappropriately designed questionnaire can lead directly to increases in measurement error (Vehovar and Lozar Manfreda, 2008). The massive range of purposes of questionnaires and diversity of the populations to be studied mean that there is no single design approach that is appropriate for all online questionnaires. However, Best and Krueger (2008) point to two pairs of key interrelated issues. They stress the importance of uniformity (how consistently the questionnaire displays on different computers, balanced with consideration of usability (ease of use), and they also focus on the security (ensuring the questionnaire is only accessible by members of the sample and avoiding access by the same person more than once), balanced with anonymity (avoiding an impression that tracking or security considerations are compromising anonymity).

Some general design principles are noted in this section. This page will focus on issues primarily related to appearance, and the second page will consider those issues that are more concerned with content.

 

Close heading CLOSE

Open/close headingConsistency

The online questionnaire must be consistently designed throughout and it must also be consistent in appearance when viewed using different computer hardware and software packages as visual design has been found to have a direct impact on how respondents answer questions (Smyth et. al, 2006). It is important to avoid possible variations in the appearance owing to differences between the respondents' computer and the researchers' hardware or software. To achieve this, consideration of file-size and download time is important, along with the need to check that the questionnaire works at a range of screen settings.

It is common practice to design for the most common screen size (no bigger than 800 x 600), but it is important to check that the questionnaire does not display badly at larger (or smaller) sizes. It is important to use relative rather than absolute sizing where possible, and to avoid the use of images or other fixed size elements (such as tables with a fixed width) without checking their suitability on various screen sizes. If these elements are an essential part of the research question, then the possible impact on the validity of results is clear.

The following links show an extract from a questionnaire when viewed at three common screen-sizes. They are designed to show the dangers of failing to take this issue into account when developing the questionnaire.

[i] Select the links to open the questionnaire in a window set to the different sizes shown in pixels. Notice the effect that viewing the questionnaire would have for users of screen with these sizes. Each link will open in a new window, which you should close to return to this page.

1024 x 768 | 800 x 600 | 640 x 480

 

It is also good practice to test the questionnaire on as many different web browsers as possible prior to launch. This can be done by downloading archived versions of major browsers. It is also often a good idea to ask friends and colleagues who may use different browsers or have different connection speeds to view the questionnaire and feed back before deploying it. As shown in the 'multi-media stimuli' section in 'Design of online questionnaires 2: Content', the use of images or multi-media can have dramatic effects on file size and can lead to huge variation in download time depending on the speed of connection.

Remember also that users may want to increase text size or change colour options according to accessibility needs or preference. Where this will not affect the validity of results, it is therefore good practice to use Cascading Style Sheets which can be easily replaced or manipulated by the user if necessary. For optimum accessibility, this can be combined with measures such as avoiding absolute text sizes and setting a range of font options.

See the 'Key design issues' section of the 'Technical guide' module for further details about these issues.

 

Close heading CLOSE

Open/close headingColour

Colour can be used to enhance the questionnaire style and ease navigation. But technical variance in computers can result in colour variation in the received questionnaire and can increase download times. In order to minimize variation in the colour scheme, Best and Krueger (2004) suggest adopting the 216 colour web-safe palette that eliminates the 40 variable colours on an 8-bit system. Additionally, yellow and blue should be used and red, green, purple and orange avoided (Rigden 1999). Contrast between text and background should be maximized. Also, colours can convey cultural values. For example, green is associated with safety in the USA, criminality in France and youth and the future in Japan (Best and Krueger 2004). Overall, colour should be used with care and for specific purpose.

Open/close headingLearning activity: Colour

The following example uses web-safe colours to ensure consistency. However, the example aims to show the potential effects of colour variance where they are not used. The colour scheme may appear to be suitable on the designer's machine, but may render differently on other computer screens.

[?] Consider the use of colour in the example and make note in the text box. When you have finished, select 'check answer' to compare your notes with a suggested answer.

 

Look at the following words, and give each a score according to how positive you feel about it, by selecting the radio buttons underneath.

Aluminium Tin
Positive image of a row of 7 radio buttons Negative
Positive image of a row of 7 radio buttons Negative
Silver Magnesium
Positive image of a row of 7 radio buttons Negative
Positive image of a row of 7 radio buttons Negative
Copper Brass
Positive image of a row of 7 radio buttons Negative
Positive image of a row of 7 radio buttons Negative

Check Answer

 

Close heading CLOSE

 

Close heading CLOSE

Open/close headingText appearance and layout choices

Text appearance is based on font, size and decoration. The choice can affect transmission times, screen configurations and perceived length of questionnaire. Times New Roman and Arial are the preferred font style for high levels of legibility and readability (Bernard and Mills 2000). Verdana is also a good choice. The most popular font size are 12 and 14-point (Bernard et al. 2002) but this does vary with sub-group, with 14+ point being preferred amongst the elderly and children (Best and Krueger 2004). Generally font decoration (italics, bold etc) should be used sparingly as it can reduce reading speed and accuracy and they should also have consistent meaning throughout the survey (Dillman and Smyth, 2007). Underlining can cause confusion with hypertext links, and the use of unusual special characters should be avoided as these can be displayed inconsistently (Best and Krueger, 2008). It is also important to add adequate 'white space' for ease of reading and processing. This is a key factor in creating a suitable layout for the questionnaire. Questions should be easy to distinguish and should be consistently formatted and laid-out. Choices given should be equally spaced and positioned consistently to avoid measurement error.

Open/close headingLearning activity: Text and layout

The following two examples, show the effect that text and layout choices can have on readability and accessibility. In each case you are asked to comment and suggest improvements.

[i] The examples will open in a new window, which you should close to return to this page.

Example 1
Example 2

 

Close heading CLOSE

 

Close heading CLOSE

 

 

 

 

 

 

 

 

 

  © 2004-2010  All rights reserved    |    Maintained by ReStore    |    About this website    |    Disclaimer    |    Copyright    |    Citation policy    |    Contact us