Exploring online research methods - Incorporating TRI-ORM

Frequently-asked questions

[Skip instructions]

[i] Click on the headings to open them. They will open on this page. Open the following link for further information about these headings if required.

Your browser does not support these headings. To ensure that the contents remain accessible, they have been automatically opened so that all the information on the page is displayed.

However, to take advantage of the headings and to ensure that the layout and design of this site are displayed correctly, you are recommended to upgrade to a current version of one of the following standards compliant browsers:

Glossary links are included within this page. If a word appears as a link, clicking on this link will show the definition of the word in a 'pop-up window'. Select the following link for information about these glossary links if required.

  1. Select the links see the definitions in a pop-up window.
  2. NB. If you use pop-up window blocking software, you will need to deactivate it for pop-ups on this site to use the glossary links. Alternatively, all glossary definitions can be seen on the 'Glossary' page in the 'Resources' section.
  3. Use of the glossary links is JavaScript dependent. If JavaScript is disabled, it will be necessary to open the 'Glossary' page to view the definitions. Opening this page in a new window may allow you to refer more easily to the definitions while you navigate the site.

 

Open/close headingHave many researchers used online questionnaires successfully? Can I see what they did?

Yes, many people are now using online questionnaires. There are a number of general social science and psychology-related sites which advertise online questionnaires and experiments:

Some key online questionnaire and research methods sources can be found at:

There are, of course, many more.

 

Close heading CLOSE

 

[Open/close heading]Is it possible to obtain a broad representative sample from internet research?

Given the increase in the number of internet users, many writers are increasingly optimistic about the ability of internet studies to reach a representative sample of the population for quantitative research. However, where such a sample is required, the researcher is still likely to face major problems. Attempts to create an accurate sampling frame are hampered by the fact that email listings for broad cross-sections of the population are difficult to come by. Use of the internet remains unevenly distributed both socially and geographically. People from poorer countries and from particular groups within richer countries (such as those from lower social classes or the elderly) are less likely to have access (Denscombe 2003).

Generally, online questionnaires are best used in situations where a particular group is targeted through non-probability sampling or self-selection (see 'Sampling' section).

 

Close heading CLOSE

 

Open/close headingAre my response rates likely to be higher or lower if I use an online questionnaire rather than a paper-based one? Are the results likely to be of equal quality?

Research suggests that online and postal questionnaires return roughly similar results in terms of response rates. Though research is limited, the studies that have been undertaken also suggest that the quality of response is roughly equivalent. (Denscombe 2003)

 

Close heading CLOSE

 

[Open/close heading]How can I maximise my response rate?

The following have been reported to increase response rates:

  • Send introductory letter outlining project and estimated time needed to complete the questionnaire;
  • Include an institutionally sanctioned website to validate researchers' identity;
  • Provide clear instructions on how to complete the questionnaire;
  • Request personal information at the start of the questionnaire rather than the end;
  • Use simple questionnaire format and avoid unnecessary graphics;
  • Avoid grid questions, open-ended questions and requests for email addresses;
  • Design survey so it takes approximately 10 minutes to complete;
  • Do not include more than 15 questions;
  • Send one or two follow up reminders;
  • Include 'social presence' or missing data messages to reduce item non-response;
  • Emphasise confidentiality.

 

Close heading CLOSE

 

[Open/close heading]How long should I give people to fill in an online questionnaire after I've sent it out?

Research suggests that if people are going to complete a web-survey they will do so in the first few hours or days of receiving it. However, it has also been found that increasing response rates can be achieved by follow-up reminders, a single reminder can double the number of respondents while it is suggested that four repeated contacts yields the highest response rate. The type of internet connection and hardware and software used in accessing the internet will also impact on how quickly the questionnaire is completed. But generally speaking your response rates will tail off after the first few days and you are unlikely to get many responses from your initial request after four weeks but of course, this varies with the target population and type of research project.

 

Close heading CLOSE

 

[Open/close heading]What kind of design issues do I need to consider when creating a questionnaire for online use?

Design issues are an extremely important consideration for online questionnaires because of the highly visual nature of the web environment, and the variations in technical skills of survey respondents. The massive range of purposes of questionnaires and diversity of the populations to be studied mean that there is no single design approach that is appropriate for all online questionnaires. Some general good practice guidelines are noted below.

  • Ensure consistency of appearance with different browsers and computers;
  • Include a welcome screen;
  • Provide clear and precise instruction on how to complete and submit the questionnaire;
  • Keep the questionnaire as short as possible - 10 minutes maximum;
  • Include a progress indicator to reduce drop-outs;
  • Use colour sparingly - yellow and blue most effective;
  • Use 12 or 14 point Times New Roman or Arial font;
  • Use a variety of question types appropriate to the research question and type of respondent;
  • Only use multi-media stimuli when essential to the research project;
  • Pilot and do usability testing.

See the 'Design of online questionnaires 1: Appearance' and the 'Design of online questionnaires 2: Content' sections.

 

Close heading CLOSE

 

[Open/close heading]How can I create boxes in a Word document/web page that people can easily type responses into?

Forms fields can be added to web pages by adding appropriate HTML '<input>' tags or inserting them using WYSIWYG (What You See Is What You Get) editors such as Macromedia Dreamweaver or Microsoft FrontPage. A full guide to creating forms in web pages is available in the 'Technical guide' module, along with links to further information in the resources section.

It is also easy to create questionnaires within Word documents that can be sent as attachments. Participants will, however, need to have a suitable version of the software installed on their machine and it should be remembered that the use of attachments may seriously reduce response rates. Attachments may expose respondents to more computer viruses and the online researcher must ensure that they never forward any viruses with their emails or attachments. Indeed Hewson et al. (2003, 117) note that a researcher using attachments can become a 'global pariah' and it is best to refrain from using attachments altogether and rely on text-based messaging.

However, if a researcher chooses to proceed with this method, it is possible to create word documents ready-made for questionnaires. By inserting form elements from the 'forms' toolbar into a template and then protecting it, the word document is locked so that users are only able to add information to the appropriate form fields and cannot change any other sections.

A step-by-step guide to doing this is available on Microsoft's TechNet site:

[External Link - opens in a new window] http://office.microsoft.com/en-us/word/HA100307461033.aspx

and further information can be found via the University of Essex's Computing Service pages:

[External Link - opens in a new window] http://www2.essex.ac.uk/cs/documentation/ use/word6/howto/forms.html

Other resources are also easy to obtain through a simple Google search.

 

Close heading CLOSE

 

Open/close headingDoes surveying just have to be quantitative? Can I include 'discuss' questions?

Questionnaires are generally based on a quantitative methodology but that does not preclude the use of more open discursive type questions. However, you will need to think carefully about how you will analyse such responses so you do not fall into the trap of simplistic description. The following site from the School of Library, Archival and Information Studies at the University of British Colombia is a useful starting point for thinking about qualitative analysis.
[External Link - opens in a new window] http://www.slais.ubc.ca/resources/ research_methods/qualitat.htm

 

Close heading CLOSE

 

[Open/close heading]Is there any problem in blending together data that was collected online and onsite?

In theory, no. Indeed, it is likely that in the future we will see an increase in the use of 'mixed method triangulation' with onsite and online methods both used to interrogate and verify the intersections between real and virtual infrastructures, enabling research to take place across a variety of online/offline domains. However, practical issues may emerge relating to compatibility of sampling in the two domains, transference of online and onsite questionnaire design and data format comparability. This remains a very interesting area of methodological enquiry which requires further investigation.

 

Close heading CLOSE

 

Open/close headingWhat kind of ethical issues do I need to consider?

As with any research, there are a wide range of ethical issues that should be considered when carrying out research via online questionnaires. Examples include issues of privacy, consent and confidentiality. It is also essential to maximise data security in collection and storage. See the 'Ethics' module of this training package for further information.

 

Close heading CLOSE

 

  © 2004-2010  All rights reserved    |    Maintained by ReStore    |    About this website    |    Disclaimer    |    Copyright    |    Citation policy    |    Contact us