Exploring online research methods - Incorporating TRI-ORM

Types of online questionnaires

[Skip instructions]

[i] Click on the headings to open them. They will open on this page. Open the following link for further information about these headings if required.

Your browser does not support these headings. To ensure that the contents remain accessible, they have been automatically opened so that all the information on the page is displayed.

However, to take advantage of the headings and to ensure that the layout and design of this site are displayed correctly, you are recommended to upgrade to a current version of one of the following standards compliant browsers:

Open/close headingWeb-based questionnaire

A questionnaire is designed as a web-page and either hosted on a web-site or as a link from an email message.

Advantages

  • Generally provides a far superior questionnaire interface to email surveys (can include icons, colors, and graphics).
  • Possible to make questionnaire more user-friendly and attractive than email-based questionnaires, encouraging higher response rates.
  • Can be included on a dedicated website which can be used as a platform to provide more information about the research project and the researchers.
  • All web pages can include links to the affiliated institution, to give the project credibility and ensure the participants can verify the authenticity of the research.
  • Can be designed to be simple and quick to fill in and include a variety of question types including tick box yes/no questions, ranking attitudinal questions, open-ended responses, multiple select questions, sliding scales and drop down lists.
  • Questions can be randomly ordered and tailored to suit the profile of particular respondents.
  • Responses can be fed automatically into a spreadsheet or data base, increasing speed and accuracy of data collection.
  • Respondents may provide more candid responses as there is a greater sense of anonymity.

Disadvantages

  • Design does require some degree of technical expertise so it may be necessary to employ a web designer, increasing costs.
  • Relies on respondents coming to the webpage so issues of recruitment are crucial, but this can be overcome by linking to relevant websites (with permission) or sending solicited emails with hypertext link.

Open/close headingExample

Geographies of eBay (Colin Williams, Oli Mould, Tim Vorley: University of Leicester)

A link to the survey was sent to specific eBay users through the 'contact user' function. Participants involved in different sales categories were contacted according to the sampling frame which established the proportion of items in different sales categories. Completed questionnaires requested respondents contact details which were used for verification of identity and subsequent contact.

[i] Click on the image to see a larger version with a description. This link will open in a new window which you should close to return to this page.

screenshot of web-based questionnaire. Screenshot of a web-based questionnaire

 

Close heading CLOSE

 

Close heading CLOSE

Open/close headingE-mail questionnaire

The questions are submitted as part of the email itself.

Advantages

  • Sent directly to respondent ensuring delivery to recipient.
  • Requires little preparation, so low cost.
  • Easy to design and answer.
  • Easy for respondent to return via email 'reply' button.
  • Few technical skills required.

Disadvantages

  • Questionnaire design usually simplistic.
  • Not attractive owing to limited design features.
  • Has to be quite short or get very low response rates.
  • Graphics and embedded objects can not be included.
  • Results must be hand-entered into a data base which increases time, costs. and data entry error.
  • Valid email addresses required for sampling purposes.
  • Anonymity of respondent may be jeopardized as email address returned with questionnaire and so may respond in a more socially desirable manner.

Open/close headingExample

Folk perceptions of belief (Claire Hewson, University of Bolton)

Sampling was done by posting a brief message to a number of newsgroups. This provided a short introduction and asked for people who might be interested in taking part in a study looking at folk perceptions of belief to send an email to the researcher requesting either more information, or the study questionnaire/materials.

This questionnaire consisted of two stories, each followed by a question.

Questionnaire:

 

Subject: Questionnaire

 

Thanks for taking part in this experiment. Before proceeding could you please indicate below your general academic background (i.e. what subjects you have studied):

This questionnaire consists of two stories, each followed by a question.

For each story you are to read it through to the end and then answer the question which follows. You may now proceed.


Below is a short story. Please read this story and then answer the question at the end.

Story 1.

Question 1.


Below is a short story. Please read this story and then answer the question at the end.

This is a story about two identical twins, Sam and Jim. Sam and Jim grow up to have a very close relationship, each is very fond and proud of his brother. Because they are so close, Sam and Jim spend a lot of time together, and, in fact, tend to adopt very similar mannerisms and interests. However, in order to try and maintain some degree of individuality the brothers dress quite differently, Sam being happiest in a suit and Jim preferring to wear jeans. Being very friendly and outgoing young men, Sam and Jim have many good friends. One such friend is Sara. Sara knows both Sam and Jim very well, and though they look identical and mimic each others mannerisms, she doesn't have any difficulty telling them apart, because of their differing styles of dress.

However, one sunny morning Sara looks out of her window and notices Sam walking past, looking rather smart as usual, and thinks to herself "Ah, there goes Sam walking past my window". In fact, she is mistaken since it is not Sam but Jim, dressed up for a job interview. Meanwhile, Sam, as his usual smart self, happens to be walking past his cousin Jane's window, and she also looks out and thinks "Ah, there goes Sam walking past my window".

Question: Do Sara and Jane have the same or different beliefs when they say "Ah, there goes Sam walking past my window"? (If you wish you may give a reason for your answer).

When participants returned their responses, they were sent an email
which thanked them for taking part, provided brief information about the
study, and asked them to get in touch if they had further questions. Here is a typical example:

Thank you for taking part in this study. Below is some information about the nature of this research.

To put things most simply, the purpose of this experiment is to try and find out something about people's everyday commonsense concept of belief. (The reason why I asked about your academic background was to assess whether this is likely to have affected your commonsense intuitions on the matter).

What I am trying to find out is the kind of factors that influence whether people will characterise two beliefs as being the same or different. So, for example, if two people both make a statement such as "there goes Sam walking past my window" are people likely to conclude that they have the same belief. Will this conclusion be modified depending on whether each person REALLY IS seeing Sam, or whether one of them is actually seeing Jim (having mistaken him for Sam.)

The answers you gave to the questions should provide some information about your intuitions on these matters. My intention is to compare the intuitions that people really do have with what philosophers have claimed peoples' intuitions are likely to be. (The specific area of philosophy that I am looking at comes under the heading the 'Folk Psychology Debate'.)

I hope you found the experiment interesting. Please do contact me if you have any questions or would like further information about this study.

 

Close heading CLOSE

 

Close heading CLOSE

Open/close headingQuestionnaire attached to an email

A questionnaire is sent as an attachment to an email.

Advantages

  • Can use word-processing or spreadsheets so simple to produce.
  • Can be more attractive than email questionnaire as more design features.
  • Can increase response rates as more visually appealing.
  • Few technical skills required.
  • Reduced length of email required and so increases participation.

Disadvantages

  • Reply may be complicated as respondent has to open, complete and save attachment and then reattach to email and return.
  • Respondent must have software capable of reading attachment.
  • Attachments may expose respondents to more computer viruses and the online researcher must ensure that they never forward any viruses with their emails or attachments. Indeed Hewson et al. (2003, 117) note that a researcher using attachments can become a 'global pariah' and it is best to refrain from using attachments all together and stick to text-based messaging.
  • Results may have to be hand-entered into a data base which increases time, costs and data entry error.
  • Valid email addresses required for sample.

 

Open/close headingExample

Employer perspectives survey (Cyndy Hawkins, University of Leicester)

The following is an example of a Microsoft word form which makes use of Check boxes. This document was sent to participants as an email attachment and hard copies were also distributed to provide a choice of methods of returning it.

[i] Click on the image to see the word document (224KB). This link will open in a new window which you should close to return to this page.

screenshot of a Word questionnaire sent as an attachment.
Screenshot of a word questionnaire sent as an attachment

 

Close heading CLOSE

 

Close heading CLOSE

 

 

 

 

 

 

 

 

 

 

 

 

 

  © 2004-2010  All rights reserved    |    Maintained by ReStore    |    About this website    |    Disclaimer    |    Copyright    |    Citation policy    |    Contact us