/**************************************************************************************************************************************** Imputation SAS code for Exemplar 6 THESE ARE TRIALS THAT DID NOT WORK The output from the program showed this in all cases, but it needed to be checked through carefully. ALSO to run the IVEWARE software you need to close all advanced editor windows and use only the old program editor CHANGE the libname to shere your data sets are stored proc contents data=ex6.ex6DETAIL short position;run; ****************************************************************************************************************************************/ libname ex6 'C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\data' ; /*------------------------------------------------------------------ Now the analysis of detailed questions using the IVEWARE macros details of what the commands mean are in the user guide on the IVE site. --------------------------------------------------------------------*/ title 'Imputation of detailed questions with IVEWARE'; /*------------------------------------------------------------------- Notes on IVEWARE trials on detailed questions that did not work 1 Defining the variables as CATEGORICAL with 8 classes (0 to 7) was too big a program to be be able to include all variables. Restricting to a maximum of 10 predictors seemed to work OK despite some warning messages until I noticed that some sparse questions had imputed to give almost all the missing data to the highest category. 2. Then tried dividing them into subgroups. But this could have just the same problems especially for sparse variables. 3 Defining variables as MIXED (0 plus regression for those with events) with no bounds gave many huge values + or -ve so all either 0 or 7 when truncated 4 MIXED variables with <=7 gave either all imputed at 7 or all at zero (once rounded etc) 5 MIXED with both bounds on gave almost everything predicted to zeros but produced lotd of collinearity errors 6 Restricting MIXED to 10 predictors was disastrous and seemed to stop the bounds from working 7 Defining a COUNT variable with no limits pulling the highest values back to 7 gave rather too many in the highest category originally imputed as some very high values , though it seemed not bad. 8 Counts with limits <=7 seemed better will try this a bit more. This is in the program EX6DET.SAS The code for each of these efforts is given below -----------------------------------------------------------------------*/ /*------------------------------------------------------------------- Attempt 1 This took almost a day, so don't try it unless you are about to go away for a bit. And it only worked afer various preliminary investigations that set the stepwise procedures to have a max of 10 variables and to stop the iterations at 10, which is a bit dodgy. It does give warnings. Otherwise it would have taken months -----------------------------------------------------------------------*/ %impute(name=ivesetup3, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Attempt one all categorical; datain EX6.ex6DETAIL; dataout imp1 all ; default categorical; transfer caseid ; maxpred 10; maxlogi 10; iterations 5; multiples 5; seed 1768; run ; /*---------------------------------------------------------------- Attempt 2 Smaller categorical model This one fails too in a similar way to 1 and takes less time. ------------------------------------------------------------------*/ data temp; set ex6.ex6det; keep caseid yayars02 ybyars02 ycyars02 ydyars02 yeyars02 yfyars02 gender ; run; %impute(name=ivesetup3, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Multiple imputation detailed questions (subsets) categorical; datain temp; dataout imp2 all ; default categorical; maxlogi 50; iterations 5; multiples 1; transfer caseid ; print coef; run ; /*---------------------------------------------------------------- Attempt 3 Mixed (zeros and normal when non-zero). ------------------------------------------------------------------*/ %impute(name=ivesetup3, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Attempt 3 mixed zeros and normal; datain EX6.ex6DETAIL; dataout imp3 all ; default mixed; transfer caseid ; iterations 5; multiples 1; seed 1768; run ; /*---------------------------------------------------------------- Following code is needed to bring them back in range and round them. ------------------------------------------------------------------*/ data imp3; set imp3; * need to sort out data after; array all _numeric_; gender=100+gender; * so that this 1/2 var is not affected; do over all; all=round(all); if all<0 then all=0; if all>7 and all<100 then all=7; gender=gender-100; end; run; /*---------------------------------------------------------------- To check these first include the checking macro code which you should save to your computer *****************************************************************/ %include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas"; %checkimp(imp3,ex6.ex6det); /*---------------------------------------------------------------- Attempt 4 Mixed (zeros and normal when non-zero) with upper bound. ------------------------------------------------------------------*/ %impute(name=ivesetup3, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Multiple imputation from detailed questions; datain ex6.ex6det; dataout imp4 all ; default mixed; bounds YAYARS02(<=7) YAYBOP02(<=7)YAYBUS02(<=7) YAYCBK02(<=7) YAYGRF02(<=7) YAYHBK02(<=7) YAYHIT02(<=7) YAYHOM02(<=7) YAYJRD02(<=7) YAYROB02(<=7) YAYSCL02(<=7) YAYSHP02(<=7) YAYSKV02(<=7) YAYVND02(<=7) YAYWEP02(<=7) YBYARS02(<=7) YBYBOP02(<=7) YBYBUS02(<=7) YBYCBK02(<=7) YBYGRF02(<=7) YBYHBK02(<=7) YBYHIT02(<=7) YBYHOM02(<=7) YBYJRD02(<=7) YBYPET02(<=7) YBYROB02(<=7) YBYSCL02(<=7) YBYSHP02(<=7) YBYSKV02(<=7) YBYVND02(<=7) YBYWEP02(<=7) YCYARS02(<=7) YCYBOP02(<=7) YCYBUS02(<=7) YCYCBK02(<=7) YCYDRG02(<=7) YCYGRF02(<=7) YCYHBK02(<=7) YCYHIT02(<=7) YCYHOM02(<=7) YCYJRD02(<=7) YCYPET02(<=7) YCYRAB02(<=7) YCYROB02(<=7) YCYSCL02(<=7) YCYSHP02(<=7) YCYSKV02(<=7) YCYVND02(<=7) YCYWEP02(<=7) YDYARS02(<=7) YDYBOP02(<=7) YDYBUS02(<=7) YDYCBK02(<=7) YDYDRG02(<=7) YDYGRF02(<=7) YDYHBK02(<=7) YDYHIT02(<=7) YDYHOM02(<=7) YDYJRD02(<=7) YDYPET02(<=7) YDYRAB02(<=7) YDYROB02(<=7) YDYSCL02(<=7) YDYSHP02(<=7) YDYSKV02(<=7) YDYVND02(<=7) YDYWEP02(<=7) YEYARS02(<=7) YEYBOP02(<=7) YEYBUS02(<=7) YEYCBK02(<=7) YEYDRG02(<=7) YEYGRF02(<=7) YEYHBK02(<=7) YEYHIT02(<=7) YEYJRD02(<=7) YEYPET02(<=7) YEYRAB02(<=7) YEYROB02(<=7) YEYRST02(<=7) YEYSHP02(<=7) YEYSKV02(<=7) YEYVND02(<=7) YEYWEP02(<=7) YFYARS02(<=7) YFYBFT02(<=7) YFYBOP02(<=7) YFYCBK02(<=7) YFYDRG02(<=7) YFYFRD02(<=7) YFYHBK02(<=7) YFYHIT02(<=7) YFYJRD02(<=7) YFYPET02(<=7) YFYRAB02(<=7) YFYROB02(<=7) YFYRST02(<=7) YFYRST22(<=7) YFYSHP02(<=7) YFYSKV02(<=7) YFYVND02(<=7) YFYWEP02(<=7); iterations 5; multiples 1; transfer caseid ; run ; /*---------------------------------------------------------------- Following code is needed to bring them back in range and round them. ------------------------------------------------------------------*/ data imp4; set imp4; * need to sort out data after; array all _numeric_; gender=100+gender; * so that this 1/2 var is not affected; do over all; all=round(all); if all<0 then all=0; if all>7 and all<100 then all=7; gender=gender-100; end; run; /*---------------------------------------------------------------- To check these first include the checking macro code which you should save to your computer *****************************************************************/ %include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas"; %checkimp(imp4,ex6.ex6det); /*---------------------------------------------------------------- Attempt 5 Mixed (zeros and normal when non-zero) with both bounds. ------------------------------------------------------------------*/ %impute(name=ivesetup3, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Multiple imputation from detailed questions; datain ex6.ex6det; dataout imp5 all ; default mixed; bounds YAYARS02(>0,<=7) YAYBOP02(>0,<=7)YAYBUS02(>0,<=7) YAYCBK02(>0,<=7) YAYGRF02(>0,<=7) YAYHBK02(>0,<=7) YAYHIT02(>0,<=7) YAYHOM02(>0,<=7) YAYJRD02(>0,<=7) YAYROB02(>0,<=7) YAYSCL02(>0,<=7) YAYSHP02(>0,<=7) YAYSKV02(>0,<=7) YAYVND02(>0,<=7) YAYWEP02(>0,<=7) YBYARS02(>0,<=7) YBYBOP02(>0,<=7) YBYBUS02(>0,<=7) YBYCBK02(>0,<=7) YBYGRF02(>0,<=7) YBYHBK02(>0,<=7) YBYHIT02(>0,<=7) YBYHOM02(>0,<=7) YBYJRD02(>0,<=7) YBYPET02(>0,<=7) YBYROB02(>0,<=7) YBYSCL02(>0,<=7) YBYSHP02(>0,<=7) YBYSKV02(>0,<=7) YBYVND02(>0,<=7) YBYWEP02(>0,<=7) YCYARS02(>0,<=7) YCYBOP02(>0,<=7) YCYBUS02(>0,<=7) YCYCBK02(>0,<=7) YCYDRG02(>0,<=7) YCYGRF02(>0,<=7) YCYHBK02(>0,<=7) YCYHIT02(>0,<=7) YCYHOM02(>0,<=7) YCYJRD02(>0,<=7) YCYPET02(>0,<=7) YCYRAB02(>0,<=7) YCYROB02(>0,<=7) YCYSCL02(>0,<=7) YCYSHP02(>0,<=7) YCYSKV02(>0,<=7) YCYVND02(>0,<=7) YCYWEP02(>0,<=7) YDYARS02(>0,<=7) YDYBOP02(>0,<=7) YDYBUS02(>0,<=7) YDYCBK02(>0,<=7) YDYDRG02(<=7) YDYGRF02(<=7) YDYHBK02(>0,<=7) YDYHIT02(>0,<=7) YDYHOM02(>0,<=7) YDYJRD02(>0,<=7) YDYPET02(>0,<=7) YDYRAB02(>0,<=7) YDYROB02(>0,<=7) YDYSCL02(>0,<=7) YDYSHP02(>0,<=7) YDYSKV02(>0,<=7) YDYVND02(>0,<=7) YDYWEP02(>0,<=7) YEYARS02(>0,<=7) YEYBOP02(>0,<=7) YEYBUS02(>0,<=7) YEYCBK02(>0,<=7) YEYDRG02(>0,<=7) YEYGRF02(>0,<=7) YEYHBK02(>0,<=7) YEYHIT02(>0,<=7) YEYJRD02(>0,<=7) YEYPET02(>0,<=7) YEYRAB02(>0,<=7) YEYROB02(>0,<=7) YEYRST02(>0,<=7) YEYSHP02(>0,<=7) YEYSKV02(>0,<=7) YEYVND02(>0,<=7) YEYWEP02(>0,<=7) YFYARS02(>0,<=7) YFYBFT02(>0,<=7) YFYBOP02(>0,<=7) YFYCBK02(>0,<=7) YFYDRG02(>0,<=7) YFYFRD02(>0,<=7) YFYHBK02(>0,<=7) YFYHIT02(>0,<=7) YFYJRD02(>0,<=7) YFYPET02(>0,<=7) YFYRAB02(>0,<=7) YFYROB02(>0,<=7) YFYRST02(>0,<=7) YFYRST22(>0,<=7) YFYSHP02(>0,<=7) YFYSKV02(>0,<=7) YFYVND02(>0,<=7) YFYWEP02(>0,<=7); iterations 5; multiples 1; transfer caseid ; run ; /*---------------------------------------------------------------- Following code is needed to round them. ------------------------------------------------------------------*/ data imp5; set imp5; * need to sort out data after; array all _numeric_; gender=100+gender; * so that this 1/2 var is not affected; do over all; all=round(all); end; gender=gender-100; run; /*---------------------------------------------------------------- To check these first include the checking macro code which you should save to your computer *****************************************************************/ %include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas"; %checkimp(imp5,ex6.ex6det); /*---------------------------------------------------------------- Attempt 6 Mixed (zeros and normal when non-zero) with upper and lower bound and limit on number of predictors. ------------------------------------------------------------------*/ %impute(name=ivesetup3, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Multiple imputation from detailed questions; datain ex6.ex6det; dataout imp6 all ; default mixed; maxpred 10. bounds YAYARS02(>0,<=7) YAYBOP02(>0,<=7)YAYBUS02(>0,<=7) YAYCBK02(>0,<=7) YAYGRF02(>0,<=7) YAYHBK02(>0,<=7) YAYHIT02(>0,<=7) YAYHOM02(>0,<=7) YAYJRD02(>0,<=7) YAYROB02(>0,<=7) YAYSCL02(>0,<=7) YAYSHP02(>0,<=7) YAYSKV02(>0,<=7) YAYVND02(>0,<=7) YAYWEP02(>0,<=7) YBYARS02(>0,<=7) YBYBOP02(>0,<=7) YBYBUS02(>0,<=7) YBYCBK02(>0,<=7) YBYGRF02(>0,<=7) YBYHBK02(>0,<=7) YBYHIT02(>0,<=7) YBYHOM02(>0,<=7) YBYJRD02(>0,<=7) YBYPET02(>0,<=7) YBYROB02(>0,<=7) YBYSCL02(>0,<=7) YBYSHP02(>0,<=7) YBYSKV02(>0,<=7) YBYVND02(>0,<=7) YBYWEP02(>0,<=7) YCYARS02(>0,<=7) YCYBOP02(>0,<=7) YCYBUS02(>0,<=7) YCYCBK02(>0,<=7) YCYDRG02(>0,<=7) YCYGRF02(>0,<=7) YCYHBK02(>0,<=7) YCYHIT02(>0,<=7) YCYHOM02(>0,<=7) YCYJRD02(>0,<=7) YCYPET02(>0,<=7) YCYRAB02(>0,<=7) YCYROB02(>0,<=7) YCYSCL02(>0,<=7) YCYSHP02(>0,<=7) YCYSKV02(>0,<=7) YCYVND02(>0,<=7) YCYWEP02(>0,<=7) YDYARS02(>0,<=7) YDYBOP02(>0,<=7) YDYBUS02(>0,<=7) YDYCBK02(>0,<=7) YDYDRG02(<=7) YDYGRF02(<=7) YDYHBK02(>0,<=7) YDYHIT02(>0,<=7) YDYHOM02(>0,<=7) YDYJRD02(>0,<=7) YDYPET02(>0,<=7) YDYRAB02(>0,<=7) YDYROB02(>0,<=7) YDYSCL02(>0,<=7) YDYSHP02(>0,<=7) YDYSKV02(>0,<=7) YDYVND02(>0,<=7) YDYWEP02(>0,<=7) YEYARS02(>0,<=7) YEYBOP02(>0,<=7) YEYBUS02(>0,<=7) YEYCBK02(>0,<=7) YEYDRG02(>0,<=7) YEYGRF02(>0,<=7) YEYHBK02(>0,<=7) YEYHIT02(>0,<=7) YEYJRD02(>0,<=7) YEYPET02(>0,<=7) YEYRAB02(>0,<=7) YEYROB02(>0,<=7) YEYRST02(>0,<=7) YEYSHP02(>0,<=7) YEYSKV02(>0,<=7) YEYVND02(>0,<=7) YEYWEP02(>0,<=7) YFYARS02(>0,<=7) YFYBFT02(>0,<=7) YFYBOP02(>0,<=7) YFYCBK02(>0,<=7) YFYDRG02(>0,<=7) YFYFRD02(>0,<=7) YFYHBK02(>0,<=7) YFYHIT02(>0,<=7) YFYJRD02(>0,<=7) YFYPET02(>0,<=7) YFYRAB02(>0,<=7) YFYROB02(>0,<=7) YFYRST02(>0,<=7) YFYRST22(>0,<=7) YFYSHP02(>0,<=7) YFYSKV02(>0,<=7) YFYVND02(>0,<=7) YFYWEP02(>0,<=7); iterations 5; multiples 1; transfer caseid ; run ; /*---------------------------------------------------------------- No checking done as so obviously wrong from IVEWARE output ------------------------------------------------------------------*/ /*---------------------------------------------------------------- Attempt 7 Count variable. ------------------------------------------------------------------*/ %impute(name=ivesetup3, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Attempt 7 all countsl; datain EX6.ex6DETAIL; dataout imp7 all ; default count; transfer caseid ; iterations 5; multiples 1; seed 1768; run ; /*---------------------------------------------------------------- Following code is needed to bring them back in range. Need to change code for gender to avoid it being afected ------------------------------------------------------------------*/ data imp7; set imp7; * need to sort out data after; array all _numeric_; gender=-gender; * so that this 1/2 var is not affected; caseid=-caseid; do over all; if all>7 then all=7; end; gender=-gender; caseid=-caseid; run; /*---------------------------------------------------------------- To check these first include the checking macro code which you should save to your computer *****************************************************************/ %include "C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code\checkimp_macro.sas"; %checkimp(imp7,ex6.ex6det); /*---------------------------------------------------- Attempt 8 count with bounds longer run is in the program ex6det.sas -------------------------------------------------------*/ %impute(name=ivesetupdet, dir=C:\Documents and Settings\gillian raab\My Documents\aprojects\peaslaptop\ex6datafiles\program_code ,setup=new); title Multiple imputation prevalence only; datain ex6.ex6det; dataout imp8 all ; default count; bounds YAYARS02(<=7) YAYBOP02(<=7)YAYBUS02(<=7) YAYCBK02(<=7) YAYGRF02(<=7) YAYHBK02(<=7) YAYHIT02(<=7) YAYHOM02(<=7) YAYJRD02(<=7) YAYROB02(<=7) YAYSCL02(<=7) YAYSHP02(<=7) YAYSKV02(<=7) YAYVND02(<=7) YAYWEP02(<=7) YBYARS02(<=7) YBYBOP02(<=7) YBYBUS02(<=7) YBYCBK02(<=7) YBYGRF02(<=7) YBYHBK02(<=7) YBYHIT02(<=7) YBYHOM02(<=7) YBYJRD02(<=7) YBYPET02(<=7) YBYROB02(<=7) YBYSCL02(<=7) YBYSHP02(<=7) YBYSKV02(<=7) YBYVND02(<=7) YBYWEP02(<=7) YCYARS02(<=7) YCYBOP02(<=7) YCYBUS02(<=7) YCYCBK02(<=7) YCYDRG02(<=7) YCYGRF02(<=7) YCYHBK02(<=7) YCYHIT02(<=7) YCYHOM02(<=7) YCYJRD02(<=7) YCYPET02(<=7) YCYRAB02(<=7) YCYROB02(<=7) YCYSCL02(<=7) YCYSHP02(<=7) YCYSKV02(<=7) YCYVND02(<=7) YCYWEP02(<=7) YDYARS02(<=7) YDYBOP02(<=7) YDYBUS02(<=7) YDYCBK02(<=7) YDYDRG02(<=7) YDYGRF02(<=7) YDYHBK02(<=7) YDYHIT02(<=7) YDYHOM02(<=7) YDYJRD02(<=7) YDYPET02(<=7) YDYRAB02(<=7) YDYROB02(<=7) YDYSCL02(<=7) YDYSHP02(<=7) YDYSKV02(<=7) YDYVND02(<=7) YDYWEP02(<=7) YEYARS02(<=7) YEYBOP02(<=7) YEYBUS02(<=7) YEYCBK02(<=7) YEYDRG02(<=7) YEYGRF02(<=7) YEYHBK02(<=7) YEYHIT02(<=7) YEYJRD02(<=7) YEYPET02(<=7) YEYRAB02(<=7) YEYROB02(<=7) YEYRST02(<=7) YEYSHP02(<=7) YEYSKV02(<=7) YEYVND02(<=7) YEYWEP02(<=7) YFYARS02(<=7) YFYBFT02(<=7) YFYBOP02(<=7) YFYCBK02(<=7) YFYDRG02(<=7) YFYFRD02(<=7) YFYHBK02(<=7) YFYHIT02(<=7) YFYJRD02(<=7) YFYPET02(<=7) YFYRAB02(<=7) YFYROB02(<=7) YFYRST02(<=7) YFYRST22(<=7) YFYSHP02(<=7) YFYSKV02(<=7) YFYVND02(<=7) YFYWEP02(<=7); iterations 5; multiples 1; transfer caseid ; run ; %checkimp(imp8,ex6.ex6det);