The following object(s) are masked from 'lager (position 3)': brand, X1, X2, X3, X4, X5, X6, X7, Y The following object(s) are masked from 'lager (position 5)': brand, X1, X2, X3, X4, X5, X6, X7, Y > lager brand X1 X2 X3 X4 X5 X6 X7 Y 1 A 15.85 0.26 14.43 0.93 36.58 0.13 0.006 60.06 2 B 13.75 0.32 10.36 0.84 32.56 0.12 0.014 53.88 3 C 18.17 0.41 17.85 1.28 38.73 0.06 0.007 57.44 4 D 17.47 0.69 21.56 1.97 49.94 0.18 0.042 43.50 5 E 17.98 0.89 7.09 1.14 29.77 0.07 0.008 41.44 6 F 18.44 0.62 31.28 0.88 69.36 0.09 0.010 48.38 7 G 16.51 0.39 14.06 0.73 32.32 0.08 0.016 40.94 8 H 14.66 0.27 15.99 0.74 44.76 0.10 0.023 47.38 9 I 27.68 0.32 20.87 3.02 42.93 0.16 0.035 64.13 10 J 13.05 1.07 25.95 0.58 50.31 0.12 0.024 54.00 11 K 11.39 0.61 15.92 0.47 32.75 0.07 0.012 56.06 12 L 22.69 0.77 20.96 1.47 52.62 0.16 0.029 49.25 13 M 12.63 0.42 16.62 0.53 32.52 0.08 0.013 46.13 14 N 13.49 0.71 21.83 0.86 49.20 0.11 0.015 49.06 15 O 15.94 0.65 18.41 1.13 36.40 0.13 0.023 50.25 16 P 13.04 0.32 19.45 0.61 43.38 0.06 0.003 46.94 17 Q 11.14 0.28 15.73 0.38 40.86 0.04 0.001 61.75 18 R 8.29 0.29 12.99 0.44 32.65 0.08 0.006 57.44 19 S 15.45 0.28 11.43 0.98 28.27 0.06 0.013 42.81 20 T 44.10 0.65 34.46 5.71 83.88 0.19 0.019 60.06 21 U 19.02 0.24 11.66 0.45 36.50 0.10 0.025 49.56 22 V 16.61 0.59 19.51 0.91 43.26 0.11 0.021 48.50 23 W 25.21 0.41 20.46 2.47 47.39 0.12 0.021 56.00 > install.packages("leaps") # Must have "set mirror" in R Warning: package ‘leaps’ is in use and will not be installed > library(leaps) > > allpossreg <- regsubsets(Y ~ X1+X2+X3+X4+X5+X6+X7,nbest=6,data=lager) > aprout <- summary(allpossreg) > > with(aprout,round(cbind(which,rsq,adjr2,cp,bic),3)) (Intercept) X1 X2 X3 X4 X5 X6 X7 rsq adjr2 cp bic 1 1 0 0 0 1 0 0 0 0.119 0.078 2.597 3.345 1 1 1 0 0 0 0 0 0 0.073 0.029 3.732 4.524 1 1 0 0 1 0 0 0 0 0.062 0.018 3.997 4.791 1 1 0 1 0 0 0 0 0 0.050 0.005 4.289 5.081 1 1 0 0 0 0 1 0 0 0.048 0.002 4.354 5.145 1 1 0 0 0 0 0 1 0 0.039 -0.007 4.566 5.352 2 1 0 0 0 0 0 1 1 0.206 0.127 2.472 4.099 2 1 0 1 0 1 0 0 0 0.197 0.117 2.695 4.360 2 1 0 1 1 0 0 0 0 0.194 0.113 2.774 4.453 2 1 0 0 0 1 0 0 1 0.188 0.107 2.917 4.619 2 1 0 1 0 0 1 0 0 0.155 0.070 3.732 5.541 2 1 1 0 0 1 0 0 0 0.151 0.067 3.812 5.630 3 1 0 1 0 0 0 1 1 0.282 0.169 2.612 4.925 3 1 0 1 0 1 0 0 1 0.236 0.115 3.738 6.350 3 1 0 1 1 1 0 0 0 0.234 0.113 3.779 6.401 3 1 0 0 0 1 0 1 1 0.233 0.112 3.809 6.438 3 1 1 1 0 1 0 0 0 0.229 0.107 3.908 6.558 3 1 1 0 0 0 0 1 1 0.214 0.090 4.272 6.996 4 1 0 1 1 0 0 1 1 0.319 0.168 3.693 6.828 4 1 0 1 0 1 0 1 1 0.302 0.146 4.129 7.421 4 1 0 1 0 0 1 1 1 0.287 0.129 4.482 7.890 4 1 1 1 0 0 0 1 1 0.287 0.128 4.499 7.912 4 1 0 1 1 1 0 0 1 0.271 0.109 4.870 8.396 4 1 1 1 1 1 0 0 0 0.271 0.109 4.889 8.420 5 1 0 1 1 0 1 1 1 0.366 0.179 4.556 8.341 5 1 0 1 1 1 0 1 1 0.325 0.126 5.559 9.778 5 1 1 1 0 1 0 1 1 0.320 0.120 5.677 9.941 5 1 1 1 1 0 0 1 1 0.319 0.119 5.693 9.963 5 1 0 1 1 1 1 0 1 0.308 0.105 5.961 10.331 5 1 1 1 1 1 0 0 1 0.302 0.096 6.126 10.553 6 1 0 1 1 1 1 1 1 0.386 0.156 6.053 10.720 6 1 1 1 1 0 1 1 1 0.377 0.143 6.285 11.072 6 1 1 1 1 1 0 1 1 0.347 0.102 7.022 12.155 6 1 1 1 0 1 1 1 1 0.323 0.069 7.615 12.992 6 1 1 1 1 1 1 0 1 0.320 0.065 7.678 13.078 6 1 1 0 1 1 1 1 1 0.298 0.035 8.206 13.795 7 1 1 1 1 1 1 1 1 0.388 0.103 8.000 13.775 > > > # These Stepwise Methods are based on Model Criteria, not individual regression coefficients > # direction="both" begins like backward and works down > # Criteria: k=2 uses AIC (default) k=log(length(y)) uses BIC > > reg.full <- lm(Y ~ X1+X2+X3+X4+X5+X6+X7) > reg.null <- lm(Y ~ 1) > > summary(reg.full) Call: lm(formula = Y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7) Residuals: Min 1Q Median 3Q Max -8.9722 -2.6885 -0.4587 3.5230 10.5647 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 53.2318 7.9678 6.681 7.34e-06 *** X1 -0.1490 0.6501 -0.229 0.822 X2 -10.0862 6.7913 -1.485 0.158 X3 0.7951 0.6256 1.271 0.223 X4 2.1770 4.0779 0.534 0.601 X5 -0.3536 0.3499 -1.011 0.328 X6 98.4047 75.9735 1.295 0.215 X7 -383.5664 235.0144 -1.632 0.123 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 6.349 on 15 degrees of freedom Multiple R-squared: 0.3884, Adjusted R-squared: 0.103 F-statistic: 1.361 on 7 and 15 DF, p-value: 0.2905 > > forward.reg <- step(reg.null,direction="forward",scope=list(upper=reg.full,lower=reg.null)) Start: AIC=88.5 Y ~ 1 Df Sum of Sq RSS AIC + X4 1 118.072 870.48 87.572 988.56 88.497 + X1 1 72.318 916.24 88.750 + X3 1 61.628 926.93 89.017 + X2 1 49.852 938.71 89.307 + X5 1 47.238 941.32 89.371 + X6 1 38.705 949.85 89.579 + X7 1 12.762 975.80 90.198 Step: AIC=87.57 Y ~ X4 Df Sum of Sq RSS AIC + X2 1 76.671 793.81 87.451 870.48 87.572 + X7 1 67.702 802.78 87.710 + X1 1 31.633 838.85 88.720 + X3 1 3.632 866.85 89.476 + X6 1 3.161 867.32 89.488 + X5 1 0.316 870.17 89.563 Step: AIC=87.45 Y ~ X4 + X2 Df Sum of Sq RSS AIC 793.81 87.451 + X7 1 38.566 755.25 88.306 + X3 1 36.897 756.92 88.356 + X1 1 31.708 762.11 88.514 + X5 1 7.900 785.91 89.221 + X6 1 0.883 792.93 89.426 > > summary(forward.reg) Call: lm(formula = Y ~ X4 + X2) Residuals: Min 1Q Median 3Q Max -10.3444 -4.5607 -0.0758 5.5401 10.3339 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 52.879 3.349 15.792 9.29e-13 *** X4 2.206 1.155 1.911 0.0705 . X2 -8.218 5.913 -1.390 0.1798 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 6.3 on 20 degrees of freedom Multiple R-squared: 0.197, Adjusted R-squared: 0.1167 F-statistic: 2.453 on 2 and 20 DF, p-value: 0.1115 > > backward.reg <- step(reg.full,direction="backward",k=log(length(Y))) Start: AIC=100.27 Y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 Df Sum of Sq RSS AIC - X1 1 2.118 606.71 97.217 - X4 1 11.487 616.08 97.570 - X5 1 41.174 645.77 98.652 - X3 1 65.097 669.69 99.489 - X6 1 67.621 672.22 99.575 604.59 100.272 - X2 1 88.904 693.50 100.292 - X7 1 107.366 711.96 100.897 Step: AIC=97.22 Y ~ X2 + X3 + X4 + X5 + X6 + X7 Df Sum of Sq RSS AIC - X4 1 20.274 626.99 94.838 - X5 1 60.703 667.42 96.275 - X6 1 76.938 683.65 96.828 - X3 1 83.591 690.30 97.051 - X2 1 87.952 694.66 97.195 606.71 97.217 - X7 1 125.104 731.82 98.394 Step: AIC=94.84 Y ~ X2 + X3 + X5 + X6 + X7 Df Sum of Sq RSS AIC - X5 1 45.835 672.82 93.325 - X3 1 77.629 704.62 94.387 626.99 94.838 - X2 1 111.808 738.79 95.477 - X7 1 144.368 771.35 96.469 - X6 1 152.121 779.11 96.699 Step: AIC=93.33 Y ~ X2 + X3 + X6 + X7 Df Sum of Sq RSS AIC - X3 1 37.055 709.88 91.423 672.82 93.325 - X2 1 104.035 776.86 93.496 - X7 1 107.393 780.22 93.596 - X6 1 109.503 782.33 93.658 Step: AIC=91.42 Y ~ X2 + X6 + X7 Df Sum of Sq RSS AIC - X2 1 74.976 784.85 90.597 709.88 91.423 - X7 1 148.696 858.57 92.661 - X6 1 226.241 936.12 94.650 Step: AIC=90.6 Y ~ X6 + X7 Df Sum of Sq RSS AIC 784.85 90.597 - X7 1 165.00 949.85 91.850 - X6 1 190.94 975.80 92.469 > > summary(backward.reg) Call: lm(formula = Y ~ X6 + X7) Residuals: Min 1Q Median 3Q Max -9.7330 -4.1652 -0.9828 4.0525 13.9520 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 46.577 3.791 12.286 8.96e-11 *** X6 112.868 51.168 2.206 0.0393 * X7 -413.080 201.452 -2.051 0.0537 . --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 6.264 on 20 degrees of freedom Multiple R-squared: 0.2061, Adjusted R-squared: 0.1267 F-statistic: 2.595 on 2 and 20 DF, p-value: 0.09951 > > stepwise.reg <- step(reg.full,direction="both") Start: AIC=91.19 Y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 Df Sum of Sq RSS AIC - X1 1 2.118 606.71 89.269 - X4 1 11.487 616.08 89.621 - X5 1 41.174 645.77 90.704 604.59 91.188 - X3 1 65.097 669.69 91.540 - X6 1 67.621 672.22 91.627 - X2 1 88.904 693.50 92.344 - X7 1 107.366 711.96 92.948 Step: AIC=89.27 Y ~ X2 + X3 + X4 + X5 + X6 + X7 Df Sum of Sq RSS AIC - X4 1 20.274 626.99 88.025 606.71 89.269 - X5 1 60.703 667.42 89.462 - X6 1 76.938 683.65 90.015 - X3 1 83.591 690.30 90.238 - X2 1 87.952 694.66 90.382 + X1 1 2.118 604.59 91.188 - X7 1 125.104 731.82 91.581 Step: AIC=88.02 Y ~ X2 + X3 + X5 + X6 + X7 Df Sum of Sq RSS AIC - X5 1 45.835 672.82 87.648 626.99 88.025 - X3 1 77.629 704.62 88.710 + X4 1 20.274 606.71 89.269 + X1 1 10.906 616.08 89.621 - X2 1 111.808 738.79 89.799 - X7 1 144.368 771.35 90.791 - X6 1 152.121 779.11 91.021 Step: AIC=87.65 Y ~ X2 + X3 + X6 + X7 Df Sum of Sq RSS AIC - X3 1 37.055 709.88 86.881 672.82 87.648 + X5 1 45.835 626.99 88.025 - X2 1 104.035 776.86 88.955 - X7 1 107.393 780.22 89.054 - X6 1 109.503 782.33 89.116 + X4 1 5.406 667.42 89.462 + X1 1 0.007 672.81 89.647 Step: AIC=86.88 Y ~ X2 + X6 + X7 Df Sum of Sq RSS AIC 709.88 86.881 - X2 1 74.976 784.85 87.190 + X3 1 37.055 672.82 87.648 + X4 1 19.465 690.41 88.241 + X5 1 5.261 704.62 88.710 + X1 1 4.575 705.30 88.732 - X7 1 148.696 858.57 89.255 - X6 1 226.241 936.12 91.244 > > summary(stepwise.reg) Call: lm(formula = Y ~ X2 + X6 + X7) Residuals: Min 1Q Median 3Q Max -8.669 -3.695 -1.177 3.298 11.435 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 49.255 4.154 11.856 3.17e-10 *** X2 -8.506 6.004 -1.417 0.1728 X6 124.512 50.599 2.461 0.0236 * X7 -393.145 197.069 -1.995 0.0606 . --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 6.112 on 19 degrees of freedom Multiple R-squared: 0.2819, Adjusted R-squared: 0.1685 F-statistic: 2.486 on 3 and 19 DF, p-value: 0.09168 > > dev.off() Error in dev.off() : cannot shut down device 1 (the null device) >