Exploring online research methods - Incorporating TRI-ORM

Project overview

[Skip instructions]

[i] Click on the headings to open them. They will open on this page. Open the following link for further information about these headings if required.

Your browser does not support these headings. To ensure that the contents remain accessible, they have been automatically opened so that all the information on the page is displayed.

However, to take advantage of the headings and to ensure that the layout and design of this site are displayed correctly, you are recommended to upgrade to a current version of one of the following standards compliant browsers:

Glossary links are included within this page. If a word appears as a link, clicking on this link will show the definition of the word in a 'pop-up window'. Select the following link for information about these glossary links if required.

  1. Select the links see the definitions in a pop-up window.
  2. NB. If you use pop-up window blocking software, you will need to deactivate it for pop-ups on this site to use the glossary links. Alternatively, all glossary definitions can be seen on the 'Glossary' page in the 'Resources' section.
  3. Use of the glossary links is JavaScript dependent. If JavaScript is disabled, it will be necessary to open the 'Glossary' page to view the definitions. Opening this page in a new window may allow you to refer more easily to the definitions while you navigate the site.

Open/close headingBackground

Cyberparents logo showing the image of a child and parent hand in hand
The 'Cyberparents' logo

 

The interest of the team in Online Research Methods sprang from the [External Link - opens in a new window]'Cyberparents' research which two of the principal investigators, Clare Madge and Henrietta O' Connor carried out in 1998. This research aimed to examine how, why and in what ways new parents use the internet as an information source about parenting and as a form of social support.

The researchers employed online questionnaires and synchronous online interviews, and they found that whenever they presented the results of their research at conferences and seminars, interest was expressed in using these online methods.

Photo of the project's three original principal investigators in a team meeting
The original project team

 

 

 

Based on this interest, they decided to put in the bid to the ESRC to disseminate training in online methods. Jane Wellens was asked to join the project team to provide advice in the pedagogic issues surrounding online learning and coordinate some of the training and dissemination events. Tristram Hooley was then brought in to cover this role when Jane went on maternity leave

 

 

 

 

Close heading CLOSE

Open/close headingAims

The main aims of the project were as follows:

  1. To produce and evaluate a high-quality online portal to provide training in online research methods;
  2. To act as a self-supporting online resource to enhance understanding of both the theoretical and practical aspects of online research methods including online questionnaires, and virtual synchronous and asynchronous interviews;
  3. To draw on a wide range of successful good practice case studies, cover associated ethical issues of online research, and provide important resource links and technical guidance.

 

Close heading CLOSE

Open/close headingContext

Logo for the Research Methods Programme.  Shows the following text in large type:  Research Methods - an ESRC research programme

 

The project was part of phase two of the ESRC Research Methods Programme. The focus of this phase was on training activities that would have a direct impact on the methodological skill base of social science research in the UK. Through consultation, a number of key points were identified, including:

  1. The need for ongoing training for researchers at all levels;
  2. The need for a close link between training and substantive research questions;
  3. The need for training through practice, potentially incorporating a range of non-traditional types of training.

The priorities identified were:

  1. Supporting 'trainers' to undertake training which they can cascade down to those they train;
  2. Encouraging training materials to be made available as online learning resources;
  3. Encouraging training in combining different methods;
  4. Supporting projects aiming to develop innovative training approaches to research design.

Further details about the consultation process and the priorities for Phase Two can be seen in the Phase Two specification paper. The Phase Two commissioning report also provides further information about the Research Methods Programme in general, including details of Phase One and the background to Phase Two.

The project was one of nine successful applications from a total of sixteen, and was the only successful project specifically focusing on online methods.

The full details of the competition can be seen in the commissioning report, and a complete list of projects funded under both Phase One and Phase Two can be seen on the ESRC website.

ESRC Research Methods Programme
[External Link - opens in a new window] http://www.ccsr.ac.uk/methods/

Phase Two specification paper
[External Link - opens in a new window] http://www.esrcsocietytoday.ac.uk/ ESRCInfoCentre/about/ How_we_do_it/ theDirectorates/RTD/ ResearchResources/ ResearchMethodsProgramme/ index.aspx

Phase Two commissioning report
[External Link - opens in a new window] http://www.esrcsocietytoday.ac.uk/ ESRCInfoCentre/ opportunities/ Commissioning_updates/ index45.aspx

List of Phase One and Phase Two projects
[External Link - opens in a new window] http://www.ncrm.ac.uk/research/phase1.php

The Research Methods Programme website also has a series of briefing papers from projects it has funded:

[External Link - opens in a new window] http://www.ccsr.ac.uk/methods/publications/

 

Close heading CLOSE

Open/close headingRationale

The project sprang from the recognition of a great interest in online research methods and of the need for a practical training package focusing specifically on the potentials and problems of online methods. The following key points summarise the rationale as outlined at the beginning of the project in 2004:

Open/close headingA note of caution

The growth and impact of the internet in recent years has meant that the use of online research methods has proven to be an increasingly alluring option for social scientists. Though there is little doubt that the popularity of these methods has increased, there has still been little empirical study of their effectiveness. In producing this package, the project team aim to raise awareness of some of the key questions raised by online research, and to stress the general need for caution and for more appraisal of its value.

 

Close heading CLOSE

Open/close headingA cross-disciplinary approach

There has been some variety across different disciplines in the extent to which online methods have taken hold, and in the level of awareness of the theoretical, practical, and technical issues involved. The package aims to make training in these methods widely available across disciplines, and to highlight the potentials and problems that they bring.

 

Close heading CLOSE

Open/close headingA training need

At the outset of the project, although training in research methods was available to UK researchers through a wide range of ESRC recognised programmes, there was little specific emphasis on online methods. A number of specialist text books had been published in an attempt to meet the demand for training, but there remained a lack of easily available hands-on facilities.

 

Close heading CLOSE

Open/close headingRecognising a clear demand

The demand for such facilities was clear to the principal investigators from the interest aroused by their use of online methods in their 'Cyberparents' research. This was expressed at presentations and through the response to publication of the research in Geography, Sociology and Market Research. The commissioning of contributions to books on online methods and the incorporation of the 'Cyberparents' research as a teaching resource at a number of academic institutions further emphasised the interdisciplinary interest in and relevance of the methods used.

 

Close heading CLOSE

Open/close headingThe benefits of an online approach

The team felt that an online learning resource would be the ideal means of meeting this demand, offering users choice over how and when to access the package, along with the flexibility to explore different content areas according to need, and prior knowledge and experience. By producing the first complete Online Research Methods training package for online delivery, the team hope to make training in current best-practice available within the UK and for the wider world beyond.

 

Close heading CLOSE

 

Close heading CLOSE

Open/close headingTraining and dissemination

The plan for training and dissemination was outlined in 2004 as follows:

A range of training and dissemination activities will take place to ensure that the training package will enhance current training programmes for the research community and contribute to the body of research in online methods and online learning.

These will include both face-to-face and online training and dissemination, as follows:

Open/close headingFace-to-face training

  1. Face-to-face training for academic staff, specifically targeted at staff supervising research students. This will consist of workshops promoted through organisations such as the National Centre for Research Methods, the Learning and Teaching Support Network (LTSN-GEES), the Geography Discipline Network (GDN), and the M1/M69 Academic Development Network of Universities Training Programme.
  2. Face-to-face training for postgraduates, comprising of input at the ESRC-accredited Wessex Consortium Conference for national Geography postgraduate training and through the University of Leicester ESRC-accredited Research Training Programme for Social Scientists which is followed by all social science postgraduates across the University of Leicester.

 

Close heading CLOSE

Open/close headingOnline training and dissemination

The website will be designed for self-study use online. To maximise awareness of the site among target groups, it will be widely promoted through organisations and websites such as:
  • The Learning and Teaching Support Network (LTSN)
  • The Geography Discipline Network (GDN)
  • The Royal Geographical Society (RGS)
  • The ESRC National Centre for Research Methods (NCRM)
  • The Social Science Information Gateway (SOSIG)

 

Close heading CLOSE

Further details of the training and dissemination activities can be seen in the 'Training and dissemination' section of this area of the site.

 

Close heading CLOSE

Open/close headingEvaluation

From the outset, a programme of evaluation was established to ensure ongoing feedback could be obtained and used to inform the design and development process. The evaluation process is documented in the 'Evaluation' section of this area of the site, where you can find the initial timeline for evaluation activities, and a description of each stage. Demonstrating best practice, the programme aimed to ensure the training package would meet the needs of different user communities as effectively as possible through the use of the following:

Open/close headingHeuristic evaluation

Heuristic evaluation is a method developed by Nielsen and Molich (1990) for finding usability problems in the design of a computer interface. It is undertaken by evaluators who are experienced in interface design or human-computer interaction. These evaluators are presented with a scenario of use, and each evaluator inspects the interface and produces a written log of their findings. They allocate each problem a severity rating from 0-4.

This evaluation was carried out by evaluation consultant Julia Meek of Birmingham University with a team of evaluators. Julia has a wide range of experience of evaluating learning technologies in Higher Education, and she has developed the 'Evaluation Lifecycle Toolkit' which is an integrated package designed to place evaluation at the heart of the development of learning technology, building evaluation activities into every stage of the process.

 

Close heading CLOSE

Open/close headingUsability Studies

The expert evaluation was supplemented by usability studies to allow the development team to observe and evaluate the use of the facility by potential users and to elicit feedback. These studies took the form of cognitive walkthroughs which provide a framework for structured observation of user behaviour. The user interacts with the site and is asked to 'think aloud', describing what they see, what they are doing and reasons for any choices made. The aim is to identify aspects of a site that are difficult to use or understand. Recommendations from these studies were incorporated into the design of the package.

 

Close heading CLOSE

Open/close headingTrialling and user studies

At different stages of the development, users were asked to evaluate the effectiveness of the site. They were observed using the training materials and asked to provide feedback on their experience via questionnaire and focus group to assess the effectiveness of the module materials and learning activities. This process informed changes to the organisation and content where required.

 

Close heading CLOSE

Open/close headingContent Evaluation

Recognised subject experts undertook a detailed evaluation of the content of the training package once a draft of all the modules was completed. This was supplemented by further content-focused user studies eliciting feedback from target users through questionnaire and focus group activities.

 

Close heading CLOSE

 

Close heading CLOSE

 

  © 2004-2010  All rights reserved    |    Maintained by ReStore    |    About this website    |    Disclaimer    |    Copyright    |    Citation policy    |    Contact us