/****************************************************************************************************************************************
Imputation SAS code for Exemplar 6
THESE ARE TRIALS THAT DID NOT WORK
The output from the program showed this in all cases,
but it needed to be checked through carefully.
In most, but not all, cases there was a warning message in
the log file for some of the equations. ****************************************************************************************************************************************/
libname ex6 'C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\data' ;
/*------------------------------------------------------------------
Now the analysis of detailed questions using the IVEWARE macros
details of what the commands mean are in the user guide on the IVE
site.
--------------------------------------------------------------------*/
title 'Imputation of detailed questions with IVEWARE';
/*-------------------------------------------------------------------
Notes on IVEWARE trials on detailed questions that did not work

1 (link to code) Defining the variables as CATEGORICAL with 8 classes (0 to 7)
was too big a program to be be able to include all variables.
Restricting to a maximum of 10 predictors seemed to work OK
despite some warning messages until I noticed that a few sparse questions
had imputed to give almost all the missing data to the highest category.

2. (link to code)Then tried dividing them into subgroups. But this could have just the same
problems especially for sparse variables. Even this seemingly small model is
really very complex since each of the categorical
variables generates 7 dummy variables.

3 (link to code)Defining variables as MIXED (0 plus regression for those with events)
with no bounds gave many huge values + or -ve so all either 0 or 7 when truncated

4 (link to code)MIXED variables with <=7 gave either all imputed at 7 or all at zero (once rounded etc)

5(link to code) MIXED with both bounds on gave almost everything predicted to zeros but produced
lots of collinearity errors

6 (link to code)Restricting MIXED to 10 predictors was disastrous and seemed to stop the
bounds from working

7 (link to code)Defining a COUNT variable with no limits pulling the highest values back to 7
gave rather too many in the highest category originally imputed as some very high values
, though it seemed not bad.

8 (link to code)Counts with limits <=7 seemed better will try this a bit more. This is in the
program EX6DET.SAS
The code for each of these efforts is given below
-----------------------------------------------------------------------*/
back to top
/*-------------------------------------------------------------------
Attempt 1
This took almost a day, so don't try it unless you are about to go away
for a bit.
And it only worked afer various preliminary investigations that
set the stepwise procedures to have a max of 10 variables and to
stop the iterations at 10, which is a bit dodgy. It does give warnings.
Otherwise it would have taken months
-----------------------------------------------------------------------*/


%impute(name=ivesetup3,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Attempt one all categorical;
datain EX6.ex6DETAIL;
dataout imp1 all ;
default categorical;
transfer caseid ;
maxpred 10;
maxlogi 10;
iterations 5;
multiples 5;
seed 1768;
run ;
back to top
/*----------------------------------------------------------------
Attempt 2
Smaller categorical model
This one fails too in a similar way to 1 and takes less time.
------------------------------------------------------------------*/
data temp;
set ex6.ex6det;
keep caseid yayars02 ybyars02 ycyars02 ydyars02 yeyars02 yfyars02 gender ;
run;
%impute(name=ivesetup3,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Multiple imputation detailed questions (subsets) categorical;
datain temp;
dataout imp2 all ;
default categorical;
maxlogi 50;
iterations 5;
multiples 1;
transfer caseid ;
print coef;
run ;
back to top
/*----------------------------------------------------------------
Attempt 3
Mixed (zeros and normal when non-zero).
------------------------------------------------------------------*/
%impute(name=ivesetup3,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Attempt 3 mixed zeros and normal;
datain EX6.ex6DETAIL;
dataout imp3 all ;
default mixed;
transfer caseid ;
iterations 5;
multiples 1;
seed 1768;
run ;
/*----------------------------------------------------------------
Following code is needed to bring them back in range and round them.
------------------------------------------------------------------*/
data imp3; set imp3; * need to sort out data after;
array all _numeric_;
gender=100+gender; * so that this 1/2 var is not affected;
do over all;
all=round(all);
if all<0 then all=0;
if all>7 and all<100 then all=7;
gender=gender-100;
end;
run;
/*----------------------------------------------------------------
To check these first include the checking macro code which
you should save to your computer
*****************************************************************/
%include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas";
%checkimp(imp3,ex6.ex6det);
back to top
/*----------------------------------------------------------------
Attempt 4
Mixed (zeros and normal when non-zero) with upper bound.
------------------------------------------------------------------*/
%impute(name=ivesetup3,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Multiple imputation from detailed questions;
datain ex6.ex6det;
dataout imp4 all ;
default mixed;
bounds YAYARS02(<=7) YAYBOP02(<=7)YAYBUS02(<=7) YAYCBK02(<=7) YAYGRF02(<=7) YAYHBK02(<=7) YAYHIT02(<=7) YAYHOM02(<=7) YAYJRD02(<=7)
YAYROB02(<=7) YAYSCL02(<=7) YAYSHP02(<=7) YAYSKV02(<=7) YAYVND02(<=7) YAYWEP02(<=7) YBYARS02(<=7) YBYBOP02(<=7) YBYBUS02(<=7) YBYCBK02(<=7)
YBYGRF02(<=7) YBYHBK02(<=7) YBYHIT02(<=7) YBYHOM02(<=7) YBYJRD02(<=7) YBYPET02(<=7) YBYROB02(<=7) YBYSCL02(<=7)
YBYSHP02(<=7) YBYSKV02(<=7) YBYVND02(<=7) YBYWEP02(<=7) YCYARS02(<=7) YCYBOP02(<=7) YCYBUS02(<=7) YCYCBK02(<=7)
YCYDRG02(<=7) YCYGRF02(<=7) YCYHBK02(<=7) YCYHIT02(<=7) YCYHOM02(<=7) YCYJRD02(<=7) YCYPET02(<=7) YCYRAB02(<=7) YCYROB02(<=7)
YCYSCL02(<=7) YCYSHP02(<=7) YCYSKV02(<=7) YCYVND02(<=7) YCYWEP02(<=7) YDYARS02(<=7) YDYBOP02(<=7) YDYBUS02(<=7) YDYCBK02(<=7) YDYDRG02(<=7) YDYGRF02(<=7)
YDYHBK02(<=7) YDYHIT02(<=7) YDYHOM02(<=7) YDYJRD02(<=7) YDYPET02(<=7) YDYRAB02(<=7) YDYROB02(<=7)
YDYSCL02(<=7) YDYSHP02(<=7) YDYSKV02(<=7) YDYVND02(<=7) YDYWEP02(<=7) YEYARS02(<=7) YEYBOP02(<=7) YEYBUS02(<=7)
YEYCBK02(<=7) YEYDRG02(<=7) YEYGRF02(<=7) YEYHBK02(<=7) YEYHIT02(<=7) YEYJRD02(<=7) YEYPET02(<=7) YEYRAB02(<=7)
YEYROB02(<=7) YEYRST02(<=7) YEYSHP02(<=7) YEYSKV02(<=7) YEYVND02(<=7)
YEYWEP02(<=7) YFYARS02(<=7) YFYBFT02(<=7) YFYBOP02(<=7) YFYCBK02(<=7) YFYDRG02(<=7) YFYFRD02(<=7)
YFYHBK02(<=7) YFYHIT02(<=7) YFYJRD02(<=7) YFYPET02(<=7) YFYRAB02(<=7) YFYROB02(<=7) YFYRST02(<=7) YFYRST22(<=7)
YFYSHP02(<=7) YFYSKV02(<=7) YFYVND02(<=7) YFYWEP02(<=7);
iterations 5;
multiples 1;
transfer caseid ;
run ;
/*----------------------------------------------------------------
Following code is needed to bring them back in range and round them.
------------------------------------------------------------------*/
data imp4; set imp4; * need to sort out data after;
array all _numeric_;
gender=100+gender; * so that this 1/2 var is not affected;
do over all;
all=round(all);
if all<0 then all=0;
if all>7 and all<100 then all=7;
gender=gender-100;
end;
run;
/*----------------------------------------------------------------
To check these first include the checking macro code which
you should save to your computer
*****************************************************************/
%include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas";
%checkimp(imp4,ex6.ex6det);
back to top
/*----------------------------------------------------------------
Attempt 5
Mixed (zeros and normal when non-zero) with both bounds.
------------------------------------------------------------------*/
 
%impute(name=ivesetup3,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Multiple imputation from detailed questions;
datain ex6.ex6det;
dataout imp5 all ;
default mixed;
bounds YAYARS02(>0,<=7) YAYBOP02(>0,<=7)YAYBUS02(>0,<=7) YAYCBK02(>0,<=7) YAYGRF02(>0,<=7) YAYHBK02(>0,<=7) YAYHIT02(>0,<=7) YAYHOM02(>0,<=7) YAYJRD02(>0,<=7)
YAYROB02(>0,<=7) YAYSCL02(>0,<=7) YAYSHP02(>0,<=7) YAYSKV02(>0,<=7) YAYVND02(>0,<=7) YAYWEP02(>0,<=7) YBYARS02(>0,<=7) YBYBOP02(>0,<=7) YBYBUS02(>0,<=7) YBYCBK02(>0,<=7)
YBYGRF02(>0,<=7) YBYHBK02(>0,<=7) YBYHIT02(>0,<=7) YBYHOM02(>0,<=7) YBYJRD02(>0,<=7) YBYPET02(>0,<=7) YBYROB02(>0,<=7) YBYSCL02(>0,<=7)
YBYSHP02(>0,<=7) YBYSKV02(>0,<=7) YBYVND02(>0,<=7) YBYWEP02(>0,<=7) YCYARS02(>0,<=7) YCYBOP02(>0,<=7) YCYBUS02(>0,<=7) YCYCBK02(>0,<=7)
YCYDRG02(>0,<=7) YCYGRF02(>0,<=7) YCYHBK02(>0,<=7) YCYHIT02(>0,<=7) YCYHOM02(>0,<=7) YCYJRD02(>0,<=7) YCYPET02(>0,<=7) YCYRAB02(>0,<=7) YCYROB02(>0,<=7)
YCYSCL02(>0,<=7) YCYSHP02(>0,<=7) YCYSKV02(>0,<=7) YCYVND02(>0,<=7) YCYWEP02(>0,<=7) YDYARS02(>0,<=7) YDYBOP02(>0,<=7) YDYBUS02(>0,<=7) YDYCBK02(>0,<=7) YDYDRG02(<=7) YDYGRF02(<=7)
YDYHBK02(>0,<=7) YDYHIT02(>0,<=7) YDYHOM02(>0,<=7) YDYJRD02(>0,<=7) YDYPET02(>0,<=7) YDYRAB02(>0,<=7) YDYROB02(>0,<=7)
YDYSCL02(>0,<=7) YDYSHP02(>0,<=7) YDYSKV02(>0,<=7) YDYVND02(>0,<=7) YDYWEP02(>0,<=7) YEYARS02(>0,<=7) YEYBOP02(>0,<=7) YEYBUS02(>0,<=7)
YEYCBK02(>0,<=7) YEYDRG02(>0,<=7) YEYGRF02(>0,<=7) YEYHBK02(>0,<=7) YEYHIT02(>0,<=7) YEYJRD02(>0,<=7) YEYPET02(>0,<=7) YEYRAB02(>0,<=7)
YEYROB02(>0,<=7) YEYRST02(>0,<=7) YEYSHP02(>0,<=7) YEYSKV02(>0,<=7) YEYVND02(>0,<=7)
YEYWEP02(>0,<=7) YFYARS02(>0,<=7) YFYBFT02(>0,<=7) YFYBOP02(>0,<=7) YFYCBK02(>0,<=7) YFYDRG02(>0,<=7) YFYFRD02(>0,<=7)
YFYHBK02(>0,<=7) YFYHIT02(>0,<=7) YFYJRD02(>0,<=7) YFYPET02(>0,<=7) YFYRAB02(>0,<=7) YFYROB02(>0,<=7) YFYRST02(>0,<=7) YFYRST22(>0,<=7)
YFYSHP02(>0,<=7) YFYSKV02(>0,<=7) YFYVND02(>0,<=7) YFYWEP02(>0,<=7);
iterations 5;
multiples 1;
transfer caseid ;
run ;
/*----------------------------------------------------------------
Following code is needed to round them.
------------------------------------------------------------------*/
data imp5; set imp5; * need to sort out data after;
array all _numeric_;
gender=100+gender; * so that this 1/2 var is not affected;
do over all;
all=round(all);
end;
gender=gender-100;
run;
/*----------------------------------------------------------------
To check these first include the checking macro code which
you should save to your computer
*****************************************************************/
%include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas";
%checkimp(imp5,ex6.ex6det);
back to top
/*----------------------------------------------------------------
Attempt 6
Mixed (zeros and normal when non-zero) with upper and lower bound
and limit on number of predictors.
------------------------------------------------------------------*/
 
%impute(name=ivesetup3,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Multiple imputation from detailed questions;
datain ex6.ex6det;
dataout imp6 all ;
default mixed;
maxpred 10.
bounds YAYARS02(>0,<=7) YAYBOP02(>0,<=7)YAYBUS02(>0,<=7) YAYCBK02(>0,<=7) YAYGRF02(>0,<=7) YAYHBK02(>0,<=7) YAYHIT02(>0,<=7) YAYHOM02(>0,<=7) YAYJRD02(>0,<=7)
YAYROB02(>0,<=7) YAYSCL02(>0,<=7) YAYSHP02(>0,<=7) YAYSKV02(>0,<=7) YAYVND02(>0,<=7) YAYWEP02(>0,<=7) YBYARS02(>0,<=7) YBYBOP02(>0,<=7) YBYBUS02(>0,<=7) YBYCBK02(>0,<=7)
YBYGRF02(>0,<=7) YBYHBK02(>0,<=7) YBYHIT02(>0,<=7) YBYHOM02(>0,<=7) YBYJRD02(>0,<=7) YBYPET02(>0,<=7) YBYROB02(>0,<=7) YBYSCL02(>0,<=7)
YBYSHP02(>0,<=7) YBYSKV02(>0,<=7) YBYVND02(>0,<=7) YBYWEP02(>0,<=7) YCYARS02(>0,<=7) YCYBOP02(>0,<=7) YCYBUS02(>0,<=7) YCYCBK02(>0,<=7)
YCYDRG02(>0,<=7) YCYGRF02(>0,<=7) YCYHBK02(>0,<=7) YCYHIT02(>0,<=7) YCYHOM02(>0,<=7) YCYJRD02(>0,<=7) YCYPET02(>0,<=7) YCYRAB02(>0,<=7) YCYROB02(>0,<=7)
YCYSCL02(>0,<=7) YCYSHP02(>0,<=7) YCYSKV02(>0,<=7) YCYVND02(>0,<=7) YCYWEP02(>0,<=7) YDYARS02(>0,<=7) YDYBOP02(>0,<=7) YDYBUS02(>0,<=7) YDYCBK02(>0,<=7) YDYDRG02(<=7) YDYGRF02(<=7)
YDYHBK02(>0,<=7) YDYHIT02(>0,<=7) YDYHOM02(>0,<=7) YDYJRD02(>0,<=7) YDYPET02(>0,<=7) YDYRAB02(>0,<=7) YDYROB02(>0,<=7)
YDYSCL02(>0,<=7) YDYSHP02(>0,<=7) YDYSKV02(>0,<=7) YDYVND02(>0,<=7) YDYWEP02(>0,<=7) YEYARS02(>0,<=7) YEYBOP02(>0,<=7) YEYBUS02(>0,<=7)
YEYCBK02(>0,<=7) YEYDRG02(>0,<=7) YEYGRF02(>0,<=7) YEYHBK02(>0,<=7) YEYHIT02(>0,<=7) YEYJRD02(>0,<=7) YEYPET02(>0,<=7) YEYRAB02(>0,<=7)
YEYROB02(>0,<=7) YEYRST02(>0,<=7) YEYSHP02(>0,<=7) YEYSKV02(>0,<=7) YEYVND02(>0,<=7)
YEYWEP02(>0,<=7) YFYARS02(>0,<=7) YFYBFT02(>0,<=7) YFYBOP02(>0,<=7) YFYCBK02(>0,<=7) YFYDRG02(>0,<=7) YFYFRD02(>0,<=7)
YFYHBK02(>0,<=7) YFYHIT02(>0,<=7) YFYJRD02(>0,<=7) YFYPET02(>0,<=7) YFYRAB02(>0,<=7) YFYROB02(>0,<=7) YFYRST02(>0,<=7) YFYRST22(>0,<=7)
YFYSHP02(>0,<=7) YFYSKV02(>0,<=7) YFYVND02(>0,<=7) YFYWEP02(>0,<=7);
iterations 5;
multiples 1;
transfer caseid ;
run ;
/*----------------------------------------------------------------
No checking done as so obviously wrong from IVEWARE output
------------------------------------------------------------------*/
back to top
/*----------------------------------------------------------------
Attempt 7
Count variable.
------------------------------------------------------------------*/
%impute(name=ivesetup3,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Attempt 7 all countsl;
datain EX6.ex6DETAIL;
dataout imp7 all ;
default count;
transfer caseid ;
iterations 5;
multiples 1;
seed 1768;
run ;
/*----------------------------------------------------------------
Following code is needed to bring them back in range.
Need to change code for gender to avoid it being afected
------------------------------------------------------------------*/
data imp7; set imp7; * need to sort out data after;
array all _numeric_;
gender=-gender; * so that this 1/2 var is not affected;
caseid=-caseid;
do over all;
if all>7 then all=7;
end;
gender=-gender;
caseid=-caseid;
run;
/*----------------------------------------------------------------
To check these first include the checking macro code which
you should save to your computer
*****************************************************************/
%include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas";
%checkimp(imp7,ex6.ex6det);
back to top
/*----------------------------------------------------
Attempt 8 count with bounds
longer run is in the program ex6det.sas
-------------------------------------------------------*/
%impute(name=ivesetupdet,
dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code
,setup=new);
title Multiple imputation prevalence only;
datain ex6.ex6det;
dataout imp8 all ;
default count;
bounds YAYARS02(<=7) YAYBOP02(<=7)YAYBUS02(<=7) YAYCBK02(<=7) YAYGRF02(<=7) YAYHBK02(<=7) YAYHIT02(<=7) YAYHOM02(<=7) YAYJRD02(<=7)
YAYROB02(<=7) YAYSCL02(<=7) YAYSHP02(<=7) YAYSKV02(<=7) YAYVND02(<=7) YAYWEP02(<=7) YBYARS02(<=7) YBYBOP02(<=7) YBYBUS02(<=7) YBYCBK02(<=7)
YBYGRF02(<=7) YBYHBK02(<=7) YBYHIT02(<=7) YBYHOM02(<=7) YBYJRD02(<=7) YBYPET02(<=7) YBYROB02(<=7) YBYSCL02(<=7)
YBYSHP02(<=7) YBYSKV02(<=7) YBYVND02(<=7) YBYWEP02(<=7) YCYARS02(<=7) YCYBOP02(<=7) YCYBUS02(<=7) YCYCBK02(<=7)
YCYDRG02(<=7) YCYGRF02(<=7) YCYHBK02(<=7) YCYHIT02(<=7) YCYHOM02(<=7) YCYJRD02(<=7) YCYPET02(<=7) YCYRAB02(<=7) YCYROB02(<=7)
YCYSCL02(<=7) YCYSHP02(<=7) YCYSKV02(<=7) YCYVND02(<=7) YCYWEP02(<=7) YDYARS02(<=7) YDYBOP02(<=7) YDYBUS02(<=7) YDYCBK02(<=7) YDYDRG02(<=7) YDYGRF02(<=7)
YDYHBK02(<=7) YDYHIT02(<=7) YDYHOM02(<=7) YDYJRD02(<=7) YDYPET02(<=7) YDYRAB02(<=7) YDYROB02(<=7)
YDYSCL02(<=7) YDYSHP02(<=7) YDYSKV02(<=7) YDYVND02(<=7) YDYWEP02(<=7) YEYARS02(<=7) YEYBOP02(<=7) YEYBUS02(<=7)
YEYCBK02(<=7) YEYDRG02(<=7) YEYGRF02(<=7) YEYHBK02(<=7) YEYHIT02(<=7) YEYJRD02(<=7) YEYPET02(<=7) YEYRAB02(<=7)
YEYROB02(<=7) YEYRST02(<=7) YEYSHP02(<=7) YEYSKV02(<=7) YEYVND02(<=7)
YEYWEP02(<=7) YFYARS02(<=7) YFYBFT02(<=7) YFYBOP02(<=7) YFYCBK02(<=7) YFYDRG02(<=7) YFYFRD02(<=7)
YFYHBK02(<=7) YFYHIT02(<=7) YFYJRD02(<=7) YFYPET02(<=7) YFYRAB02(<=7) YFYROB02(<=7) YFYRST02(<=7) YFYRST22(<=7)
YFYSHP02(<=7) YFYSKV02(<=7) YFYVND02(<=7) YFYWEP02(<=7);
iterations 5;
multiples 1;
transfer caseid ;
run ;
%checkimp(imp8,ex6.ex6det);