1. Sample Stratification

Proportionate Stratified Sampling
Disproportionate Sampling
 Introduction Sample stratification involves two steps: (a) divide the population of sampling units into population sub-groups, called strata (b) select a separate sample per strata If the same sampling fraction is used in each stratum this is termed ‘proportionate stratified sample’; if the sample fraction is not the same in each stratum this is termed ‘disproportionate sampling’. More commonly the latter would be described as ‘over-sampling of one or more sub-groups’. Proportionate stratified sampling almost always leads to an increase in survey precision (relative to a design with no stratification), although the increase will often be modest, depending upon the nature of the stratifiers. Disproportionate sampling sometimes increases precision and sometimes reduces precision. Surveys using disproportionate sampling have to utilise survey weights more about weighting if they are to give unbiased cross-strata estimates. As well as using stratification in the sample design, survey statisticians sometimes use ‘ post-stratification ’ once the data has been collected. 'Post-stratification' is a weighting method that adjusts for any differences between the survey data and the population in terms of a few key population variables (often age and sex). The aim is to reduce any bias in the survey due to sampling error and/or non-response effects. More detail is given below.

Proportionate stratified sampling

1.1. What it is

In a proportionate stratified sample, the population of sampling units are divided into sub-groups, or strata, and the sample is selected separately in each stratum. For the sampling to be proportionate, the sampling fraction (or interval) must be identical in each stratum.

Suppose a sample of 100 students is to be selected from a school with 2000
students, so that the sampling fraction to be used is 1 in 20. If, before drawing the sample, the school roll is divided by age and sex, and a separate sample is drawn per age and sex stratum, then if the sampling fraction of 1 in 20 is used in each stratum the sample would be a proportionate stratified sample.

This is illustrated in the Table below where we can see that in real examples the sampling fraction will vary a little between different strata. This is because we can’t get a fraction of a pupil.

 age/sex    group school roll 1 20th number selected actual sampling fraction equivalent to 1 in under 6 M 169 8.45 8 0.0473 21.1 under 6 F 147 7.35 7 0.0476 21.0 7-9 M 194 9.7 10 0.0515 19.4 7-9 F 213 10.65 11 0.0516 19.4 10 + M 177 8.85 9 0.0508 19.7 10 + F 198 9.9 10 0.0510 19.8 TOTAL 1098 54.9 55 0.0501 20.0
Table 1.1 Sampling Fractions Proportionate allocation is used for two reasons: (i) to reduce standard error for survey estimates; (ii) to ensure that sample sizes for strata are of their expected size. For example, almost all large-scale GB surveys that use the PAF as a sampling frame use samples stratified by region, and within region, by a measure of relative area deprivation. The first stratifier (region) is used to ensure that the selected sample is correctly proportioned by region. (A national sample that, just by chance, happened to under or over-represent some of the regions would be considered by many as 'unrepresentative'. The second stratifier (area deprivation) is used to ensure that the selected sample is correctly proportioned by area type. In practice, many survey statisticians would argue that of the two, only the second stratifier is strictly necessary, and that the regional stratifier is largely cosmetic for the purpose of getting national estimates. This is because area deprivation is strongly correlated with many of the outcome measures social surveys collect. So ensuring that the sample has the correct area deprivation profile means there will be less sampling variance in the estimates and standard errors are almost bound to be smaller than would be the case with an unstratified sample. Put another way, if the area deprivation profile of the sample is controlled, the risk of selecting an unrepresentative sample by chance is reduced. Region, in contrast, tends to be only weakly associated with social survey outcome measures, so stratification by region does not reduce sampling variance by very much. In other words, even if the regional profile of the sample is controlled, the risk of selecting an unrepresentative - sample by chance does not significantly reduce. The reason for stratification by region is usually not to improve precision of national estimates, but to get better estimates for regional analyses. Click here for a discussion of why this is so. Relative to taking a completely unstratified sample, taking a proportionate sample is either a good thing, in that it reduced standard errors, or a neutral thing, in that standard errors don’t change. Proportionate stratification can never increase standard errors. The reasoning is as follows: - total sampling variance can be decomposed into two components: within-strata variation and between-strata variation (the split between the two depending on how the strata are defined); - with proportionate stratification the between-strata variance becomes zero. So, proportionate stratification is most efficient when the stratifiers that are used split the total variance in a way that maximises the between-strata variance. 1.5 Proportionate stratification in practice (explicit and implicit stratification) In proportionate stratification a distinction is made between ‘explicit’ and ‘implicit’ stratification. ‘Explicit stratification’ is where the population of sampling units is explicitly divided into strata and a separate sample selected per stratum. ‘Implicit stratum’ is where the population of sampling units is sorted by some characteristic(s) and then the sample is selected from the sorted list using a fixed sampling interval and a random start. For example, a population of adults might be sorted by sex, and then, within sex by date of birth. Suppose every nth person is then selected from the population by taking a random start between 1 and n and then every nth person after that, working down the list. This sample would then be described as a proportionate stratified sample with explicit stratification by sex and implicit stratification by date of birth. Note that for explicit stratification only categorical stratifying variables can be used (or continuous variables that have been grouped into categories). Implicit stratification, in contrast, which only involves sorting a population rather than grouping. It can be used for continuous variables as well as those with a large number of classes some of which can be rare in certain strata. Large-scale surveys often use a combination of explicit and implicit stratification. The sampling frame will firstly be grouped into a number of explicit strata, and within each of these the sampling frame will be sorted by a continuous variable or one with many classes. See exemplar 2 for an example of this.

 Software packages that calculate standard errors for complex surveys usually only allow for explicit stratification. The way around this for a survey that uses implicit stratification is to: (a) Keep the sample of PSUs in the same order as it was selected in. (b) Put achieved cases into pairs, working down the list (i.e. the first two achieved cases working down the list are the first pair, the third and fourth achieved are the second pair, etc.). (c) If there are an uneven number of achieved cases then put the last three achieved cases together to give a triplet. (d) Treat each pair/triplet as if they were selected from the same explicit stratum. So there will be half as many explicit strata as there are achieved cases. This ‘trick’ needs some care when calculating standard errors for sub-groups, since the approach only works if there are two achieved cases per ‘pair’. For a sub-group this can easily drop to one. One option would be to re-pair the sample for each sub-group, but this is too onerous in practice.Survey packages vary in how they can handle this which is an example of the lonely PSU problem. Surveys within this type of implicit stratiifcation may be better analysed by replication methods, but rather few packages can handle this.

 For large-scale government sponsored surveys it is common practice to spread fieldwork over a period, often of a year. In these cases the sample for a whole year is selected at one point in time (usually using a combination of implicit and explicit stratification) and then the primary sampling units are systematically allocated to the 12 months of the year. The allocation is done in such a way that, within each month, the original stratification is maintained. With this design the decision on how to deal with the stratification in estimating standard errors is not so straightforward. If the pairing follows the sample stratification then, in all pairs, the two primary sampling units will be from different months of the survey. This means that the ‘within-stratum between-psu’ estimated component of variance will incorporate both a genuine between-psu element plus a between-month element (which will often be a seasonal effect). This latter component tends to over-estimate the standard errors for estimates. To avoid this one approach is to treat the sample for each month as an independent sample and treat the sample within each month as a stratified sample. This in effect means that the original sample is resorted, firstly by month, and then within month, by the original order. The pairs are then constructed from this new list.

 Disproportionate sampling 1.8 What it is In a disproportionate stratified sample, the population of sampling units are divided into sub-groups, or strata, and a sample selected separately per stratum. Crucially, the sampling fraction is not the same within all strata: some strata are over-sampled relative to others. Suppose a sample of 50 white students and 50 non-white students is to be selected from a school with 2000 students, of whom 100 are non-white. To achieve this the school roll would need to be divided into two strata: white and non-white, and separate samples selected per strata. The sampling fraction to be applied in the white stratum would be 1 in 38; the sampling fraction to be applied in the non-white stratum would be 1 in 2. Disporportionate sampling is more common in surveys of organisations. Larger organisations will tend to be more variable, for example in their labour force numbers. More precise estimates could then be obtained from samples with an over representation of larger organisations. 1.10 Why disproportionate stratification is used Disproportionate stratification is used for two purposes: A. To give larger than proportionate sample sizes in one or more sub-groups so that separate analyses by sub-group will be possible; and, far more rarely B. To increase the precision of key survey estimates. Disproportionate stratification will only reduce standard errors (relative to a proportionate stratified sample) if the population standard deviation for the variable of interest is higher than average within the over-sampled strata. (In practice, standard errors will be minimised if the sampling fraction used per stratum is proportional to the population standard deviation within the stratum). For surveys that want to estimate the porportion of certain types of respondent this means reducing the sampling fraction when the types of interest is uncommon. The fact that most surveys collect data on a wide range of variables means that disproportionate stratified sampling to reduce standard errors is very rarely used in household surveys – since the optimal sample design for one variable is unlikely to be optimal for others. Furthermore, the population standard deviations are often not known at the design stage. It is more common in other types of survyes, such as those of farms or businesses where some groups are much more variable than others. To obtain unbiased estimates for a disproportionate stratified sample, the survey estimates have to be weighted. This is achieved within most software packages by defining a weight variable that gives a weight per case. The cases are then ‘weighted by’ this weight variable in the analysis. The calculation of the weight is fairly straightforward: it is simply the inverse of the sampling fraction used in the stratum that the case belongs to. So, in a stratum where the sampling fraction is 1 in 10 all cases would get a weight of 10; and in a stratum where the sampling fraction is 1 in 22 all cases would get a weight of 22. In practice the weights applied to a particular survey may be more complex than this if, for instance, within strata not all cases are selected with equal probability, of if non-response weights have been included.

 1.12 The effect of disproportionate stratification on standard errors. As you might expect, within strata that have higher sampling fractions estimates are more precise than if a smaller ample had been drawn as part of a proportionate sampling scheme. Similarly the precision will be worse within strata with lower sampling fractions. The effect of disproportionate stratification on overall estimates for the population will depend on a variety of factors. It depends on the relative sampling fractions and on how the variability of the survey responses differs in the different strata. This is discussed in detail in the section on weighting. When you are reading the weighting section remember that an over sampled stratum needs to be given a lower weight in calculating an overall total, mean or percentage to get an unbiased estimate. Similarly strata with low sampling fractions need to be given large weights.

peas project 2004/2005/2006.