top

This exemplar is based on data from the Edinburgh Study of Youth Transitions and Crime. This is a longitudinal survey that has collected data on young people through their six years of secondary education (ages 12 to 17). A cohort of pupils beginning their secondary school education in Edinburgh in autumn 1998 was the target population. Only a few schools and a small proportion of eligible pupils were withdrawn by their parents. Further information about the project can be found at http://www.law.ed.ac.uk/cls/esytc. The data for the earlier sweeps of the study can be accessed at the ESRC data archive, and the later ones will be there in time.

To illustrate multiple imputations for longitudinal data we are using data collected at all six sweeps. Each year the participants completed a detailed questionnaire. For the first 4 years almost all the participants were in school and the response rate was excellent. At years 5 and 6 it was slightly worse as a proportion of young people left school and moved away from home.

The population who were eligible for the study changed over time as families moved in and out of the city. For the longitudinal analysis we considered the 4328 participants who were eligible to participate in year 4. The response rate at each year is shown in Figure 6.1. The non-responders in years 1 and 2 were largely those who moved into study schools after this. From year 4 onwards the decline is largely due to the worse response rate from those who had left school and had to be followed up elsewhere. 74% of participants responded at all 6 sweeps.

In this exemplar we will be looking at how the rates of self-reported delinquency change over the 6 years. The questionnaire contained a series of questions like the following example:-

• In the last year did you steal something from a shop or a store?
  If ‘yes’ how many times?

There were between 15 and 18 such questions, depending on the sweep of the questionnaire. Three measures of delinquency were derived from each set of questions

1. Prevalence 1 if a yes response to any question, else 0.
2. Variety Number of questions answered out of the total.
3. Volume – Sum of the number of times for all questions answered yes, with the answer 'more than 10 times' coded as 11.

Being missing on any question lead to a missing score. This was most pronounced at sweep 6 where some of the questions were no longer relevant to those who had left school (e.g. truancy). Figure 6.2 shows how the mean values of these measures changed over time for the observed subjects.

We can see that the delinquency measures peak at around secondary years 3 and 4 (S3 and S4) and decline after this. But perhaps some of this decline relates to losses to follow up? If the more delinquent failed to respond in later sweeps the mean score for responders would be increased.

For every longitudinal survey we need to consider what we mean by our ‘eligible sample’ since some respondents may not be asked to participate in every wave of the survey. This eligible sample may sometimes vary depending on how many sweeps of the survey we wish to look at together. For this survey it was decided to consider the eligible sample as those who were pupils in Edinburgh schools at S4, and to exclude any pupils who had moved away in earlier years. Table 1 shows the response patterns for all 4328 pupils in the eligible sample. An R represents a response at each wave. The pupils in the first row of Table 6.1 responded at all sweeps, the second row missed the last sweep, and so on. Although a large proportion of response patterns (the first 6 groups) are nested there are many others, so a purely nested analysis would not work.

In addition to those who did not fill in a questionnaire at some sweeps (unit non-response) we will have additional missing data from individual questions that were not answered ( item non response ). For the questions on delinquency that we investigate the unit non-response averaged around 5%, with a smaller proportion missing in the older age groups for most questions.

We will illustrate how different ways of imputing the non-respondents' values affect the patterns of delinquency over time.

top

Different types of imputation are discussed in the theory part of this site: Imputation section 6.2. We will be illustrating only model- based imputation here. Two main approaches have been developed. Both of them involve making some quite strong assumptions about the data.

1. Assume a multivariate normal distribution. This is the implemented by Schafer and available in some packages and discussed in his text book. See theory section 6.
2. Predict each variable in your data set sequentially from all the others. Methods using this approach are known as chained methods.

Both of these methods are proper imputations and both can be run as multiple imputations. Methods to combine the imputations can then be used that allow the standard errors of estimates to be increased to allow for the missing data.

The scores mentioned above were derived from a large number of individual questions about offending behaviour. We have a choice of either

• Imputing the total scores at each sweep
• Imputing the original questions and recalculating scores (102 variables)

The second of these will have the advantage of obtaining scores from people who have answered some, but not all, of the questions that make up the score.

As well as the choices between methods and approaches, we need to decide which other variables in the data set to use to predict what values the missing data should take. We also need to decide how the imputation of each variable is to be handled, what model to use and which other variables will be used to impute it.

top

top

If we just use the 6 values of the delinquency prevalence we can use a repeated measures analysis of variance that will adjust correctly for the missing data without any complicated imputation being required, but it makes the assumption of multivariate normality. Only some repeated measures programs will do this correctly (see below section on software).

The programs for this exemplar carry out procedures to deal with the missing data in different ways. These include:-

1. Using all available data at each sweep
2. Using only those pupils who responded at all 6 sweeps (complete cases)
3. Carrying out a repeated measures analysis
4. Imputing prevalence at other sweeps using chained methods with logistic regressions
5. Imputing prevalence at other sweeps assuming a normal distribution

Figure 6.3 illustrates the results from these analyses for the 1/0 variable measuring delinquency prevalence over time. Note the expanded scale here compared to Figure 6.1.

All the imputation methods give results that are fairly close to one another and are only differ from those for all the available data at the sixth sweep, and then by only a modest amount. Using only complete cases (listwise deletion) gives much lower rates. This might have been expected since those who have completed the survey on every occasion might be expected to be a more law-abiding group. The complete data from each sweep and the imputed data agree well. The drop in delinquency at S5 and S6 seems, from this analysis, to be mainly genuine and not just the result of sample attrition.

We can also use the post imputation procedures to see how the imputation procedure has affected the standard errors of the estimates, always assuming that the biases have been corrected adequately by the model fitted. Table 6.2 compares the estimated prevalence at sweep 6 from 4 imputations and gives the combined estimate. The combined estimate has a mean that is the mean of the 4 imputations, but its standard error has been increased by the variation between the imputations. The combined imputed estimate gives a lower standard error than the original data, indicating that as well as reducing bias, the imputation procedure has recovered information for the missing cases.

This all looks very reassuring. The complete case analysis showed that the data were not missing completely at random. However the pattern of imputed data at each wave did not suggest that it would have been very different from an analysis that used all the data available from each sweep.. Sweep 6 is most strongly affected by the missing data but differences are small. It was very reassuring that all the packages were in reasonable agreement, and that different methodologies gave very close results.

top

top

To fit the imputation model to the 102 individual questions, each with 8 possible response categories, proved challenging. It required many weeks of work to investigate all the possible approaches and to reject those with obvious problems. See especially the SAS file that documents problems. Some of the practical problems encountered are detailed in the theory page on imputation (Section 6.6) and problems with individual packages are documented in the code for each one. See especially the SAS file that documents problems with chained equations ex6DETprob.htm.

Finally two different acceptable chained equations unit non-response analyses were achieved by two different packages (Stata and SAS). In the Stata analysis each question was modelled as an ordered categorical variable. To include all of the questions in every model together would have produced models that were too big and would fail. One solution could be to write 102 equations giving details of which other variables were to be used to impute each one. An alternative option to divide the questions into subsets of related questions was used since it was easier to program. The SAS IVEWARE model used a COUNT variable to model the responses to each question.

The NORM analyses also gave some problems and had to be run in two halves because of resource problems. Scanning the results for individual questions after NORM, it was clear that a large proportion of the missing data had been imputed as having done an offence just once, compared to the observed data. These results were not extreme, however, and it is not clear they would have been an obvious error had the results from other methods not been there to compare. The NORM procedure had to be run without imposing bounds on the range of values. When this was attempted the program failed because the predictive distribution repeatedly gave a value outside the range.

Looking at the detailed questions makes one realise the problems of imputation. One of the offending questions relates to "skiving school". School leavers, who accounted for many of the non-responders, could hardly have answered this in any way. But the other questions are all relevant. It was clear on inspecting the data that the answers to many individual questions tracked very strongly from one period to another and the imputed data preserved this pattern.

top

Figure 6.5 shows the mean prevalence, by gender, for all respondents over all 6 sweeps. We can see that our original suspicion that those who were not followed up at the later sweeps is now confirmed by the results of the imputation, especially those that impute from the detailed questions. Differences between methods are less pronounced than differences between the variables selected for the imputation. Compared to the smaller imputation, the prevalence at sweeps 5 and 6 were higher for all the imputation methods.

Figure 6.5 Prevalence over all 6 sweeps imputed by different methods

Figure 6.6 shows the same data for variety of offending . Differences here are smaller except for imputing with a normal distribution from the detailed questions. This was due to the high proportion of imputed values that imputed to 'one time' rather than zero.

Figure 6.6 Variety over all 6 sweeps imputed by different methods

Figure 6.7 shows the mean prevalence, by gender, for all respondents over all 6 sweeps. Here all methods are in reasonable agreement with the possible exception of imputing the scores from a normal distribution that tends to give higher values, especially for girls. There is a modest increase in volume seen at sweep 6, compared to the original data.

Figure 6.7 Volume over all 6 sweeps imputed by different methods

top

All t he analyses that imputed from the scores gave similar results for the coefficients, as did all the analyses that imputed from the detailed questions. This included the NORM method in each case. The values in table 6.5 are from the IVEWARE imputations.

All the three analyses agree that girls offend less than boys, that deprivation is associated with more offending and that those attending independent schools offend less.

The analysis of data imputed from the total scores differ only slightly from that from the observed data. But data imputed from the totals gives a much stronger effect for deprivation and a somewhat stronger effect for independent schools.

The numbers in the other two school sectors are smaller and so the estimates are more volatile. All three analyses agree that a non-behavioural special school is associated with less offending. The effect of having been at a school for pupils with behaviour problems shows up only in the analysis that imputes from the individual questions and is only marginally significant. This may well be because of a high percentage of missing data for this group. It would be possible to check this in the data file. I have not had time, but the data are here on the site if you want to.

If time had allowed more modelling analyses would have been added to this site. To allow you to test this yourselves we provide the data sets with the imputed data (10 imputations) in SAS, Stata and SPSS format. The SAS data sets have the imputation number identified by the variable _IMPUTATION_.

top

top

This was a simple longitudinal survey. All secondary schools in Edinburgh were approached. Although it was clustered by school, this was a natural clustering rather than one induced by the survey design. We have not used the information on clustering in the analyses presented here, although this might be possible with some of the software discussed.

The longitudinal data set has been produced by linking the data from all sweeps of this survey (S1 to S6). The data from sweeps 1 to 4 of the survey are available from the UK data archive. Most of the data used in this exemplar comes from the young people's questionnaires which can be viewed at the survey web site. The data archive documentation explains the codes for the variables used from the questionnaires at the first 4 sweeps, and the conventions continue in an obvious way at the final two sweeps. Details as to how data for use in this exemplar have been extracted from the larger data files is available here.

The questions that contribute to the prevalence, volume and variety scores for delinquency differed slightly for each of the sweeps. They are listed in the codes for the data sets with individual questions (below).

top

top

This data set includes 102 questions each giving the number of times each participant reported having done each of the items 0= not done 1 to 5 represent the number of times, 6= 5 to 10 times 7=10 or more times.