------------------------------------------------------------------------------- name: log: /Users/carlosmendez/Documents/GitHub/starter-academic-v501/content > /post/stata_sc/analysis.log log type: text opened on: 27 Apr 2026, 13:29:52 . . di _newline(2) . di "============================================" ============================================ . di " Synthetic Control Method (SCM) Tutorial" Synthetic Control Method (SCM) Tutorial . di " Abadie, Diamond & Hainmueller (2010)" Abadie, Diamond & Hainmueller (2010) . di " $S_DATE $S_TIME" 27 Apr 2026 13:29:52 . di "============================================" ============================================ . . * Apple Silicon compatibility check . di _newline . di "Machine type:" Machine type: . display c(machine_type) Macintosh (Intel 64-bit) . di _newline . di "NOTE: If you are using a Mac with Apple Silicon," NOTE: If you are using a Mac with Apple Silicon, . di "the synth optimization plugin requires Rosetta 2." the synth optimization plugin requires Rosetta 2. . di "Right-click Stata > Get Info > Open Using Rosetta." Right-click Stata > Get Info > Open Using Rosetta. . . . *=================================================== . * PART 1: DATA AND BASELINE SCM . * Dataset: smoking_sc.dta (39 states x 31 years) . *=================================================== . . . *--------------------------------------------------- . * Section 1: Load and explore the dataset . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 1: DATA LOADING & EXPLORATION" SECTION 1: DATA LOADING & EXPLORATION . di "========================================" ======================================== . . use "https://github.com/quarcs-lab/data-open/raw/master/isds/smoking_sc.dta", > clear (Tobacco Sales in 39 US States) . . * Inspect variable labels and storage types . describe Contains data from https://github.com/quarcs-lab/data-open/raw/master/isds/smok > ing_sc.dta Observations: 1,209 Tobacco Sales in 39 US States Variables: 7 23 Mar 2007 21:59 ------------------------------------------------------------------------------- Variable Storage Display Value name type format label Variable label ------------------------------------------------------------------------------- state long %14.0g state state no year float %9.0g year cigsale float %9.0g cigarette sale per capita (in packs) lnincome float %9.0g log state per capita gdp beer float %9.0g beer consumption per capita age15to24 float %9.0g percent of state population aged 15-24 years retprice float %9.0g retail price of cigarettes ------------------------------------------------------------------------------- Sorted by: state year . . * Check means, SD, min, max . summarize Variable | Obs Mean Std. dev. Min Max -------------+--------------------------------------------------------- state | 1,209 20 11.25929 1 39 year | 1,209 1985 8.947973 1970 2000 cigsale | 1,209 118.8932 32.7674 40.7 296.2 lnincome | 1,014 9.861634 .1706769 9.397449 10.48662 beer | 546 23.4304 4.22319 2.5 40.4 -------------+--------------------------------------------------------- age15to24 | 819 .175472 .0151589 .1294482 .2036753 retprice | 1,209 108.3419 64.38199 27.3 351.2 . . * Show first few observations . list in 1/6 +------------------------------------------------------------------+ | state year cigsale lnincome beer age15~24 retprice | |------------------------------------------------------------------| 1. | Alabama 1970 89.8 . . .1788618 39.6 | 2. | Alabama 1971 95.4 . . .1799278 42.7 | 3. | Alabama 1972 101.1 9.498476 . .1809939 42.3 | 4. | Alabama 1973 102.9 9.550107 . .1820599 42.1 | 5. | Alabama 1974 108.2 9.537163 . .183126 43.1 | |------------------------------------------------------------------| 6. | Alabama 1975 111.7 9.540031 . .1841921 46.6 | +------------------------------------------------------------------+ . . * Declare panel structure: state = unit, year = time . xtset state year Panel variable: state (strongly balanced) Time variable: year, 1970 to 2000 Delta: 1 unit . . * Panel summary: within/between variance . xtsum Variable | Mean Std. dev. Min Max | Observations -----------------+--------------------------------------------+---------------- state overall | 20 11.25929 1 39 | N = 1209 between | 11.40175 1 39 | n = 39 within | 0 20 20 | T = 31 | | year overall | 1985 8.947973 1970 2000 | N = 1209 between | 0 1985 1985 | n = 39 within | 8.947973 1970 2000 | T = 31 | | cigsale overall | 118.8932 32.7674 40.7 296.2 | N = 1209 between | 26.47924 63.83548 213.0645 | n = 39 within | 19.74754 49.5287 202.0287 | T = 31 | | lnincome overall | 9.861634 .1706769 9.397449 10.48662 | N = 1014 between | .1304321 9.584268 10.20371 | n = 39 within | .1119729 9.588297 10.14454 | T = 26 | | beer overall | 23.4304 4.22319 2.5 40.4 | N = 546 between | 4.012627 13.72143 36.7 | n = 39 within | 1.45541 2.22326 31.60897 | T = 14 | | age15~24 overall | .175472 .0151589 .1294482 .2036753 | N = 819 between | .0071531 .1593386 .1927357 | n = 39 within | .0134118 .1326057 .1989868 | T = 21 | | retprice overall | 108.3419 64.38199 27.3 351.2 | N = 1209 between | 8.751969 88.31936 127.3258 | n = 39 within | 63.79926 23.21613 339.6194 | T = 31 . . * Identify California's state code . label list state: 1 Alabama 2 Arkansas 3 California 4 Colorado 5 Connecticut 6 Delaware 7 Georgia 8 Idaho 9 Illinois 10 Indiana 11 Iowa 12 Kansas 13 Kentucky 14 Louisiana 15 Maine 16 Minnesota 17 Mississippi 18 Missouri 19 Montana 20 Nebraska 21 Nevada 22 New Hampshire 23 New Mexico 24 North Carolina 25 North Dakota 26 Ohio 27 Oklahoma 28 Pennsylvania 29 Rhode Island 30 South Carolina 31 South Dakota 32 Tennessee 33 Texas 34 Utah 35 Vermont 36 Virginia 37 West Virginia 38 Wisconsin 39 Wyoming . . di _newline . di "Panel: 39 states x 31 years (1970-2000) = 1,209 observations" Panel: 39 states x 31 years (1970-2000) = 1,209 observations . di "Treatment unit: California (state == 3)" Treatment unit: California (state == 3) . di "Treatment period: 1989 (Proposition 99 went into effect)" Treatment period: 1989 (Proposition 99 went into effect) . di "Donor pool: 38 control states" Donor pool: 38 control states . di _newline . di "Variables:" Variables: . di " cigsale - Cigarette sales per capita (packs)" cigsale - Cigarette sales per capita (packs) . di " lnincome - Log personal income per capita" lnincome - Log personal income per capita . di " age15to24 - % population aged 15-24" age15to24 - % population aged 15-24 . di " retprice - Average retail cigarette price" retprice - Average retail cigarette price . di " beer - Beer consumption per capita" beer - Beer consumption per capita . . . *--------------------------------------------------- . * Section 2: Visualize raw trends . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 2: RAW TRENDS" SECTION 2: RAW TRENDS . di " California vs. Donor Pool Average" California vs. Donor Pool Average . di "========================================" ======================================== . . preserve . . * Create California indicator . gen california = (state == 3) . . * Collapse to year x group means . collapse (mean) cigsale, by(year california) . . twoway (connected cigsale year if california==1, /// > msymbol(O) mcolor("106 155 204") lcolor("106 155 204") /// > lwidth(medthick)) /// > (connected cigsale year if california==0, /// > msymbol(T) mcolor("128 128 128") lcolor("128 128 128") /// > lwidth(medium) lpattern(dash)), /// > xline(1989, lcolor("217 119 87") lpattern(dash) lwidth(medium)) /// > ytitle("Cigarette Sales (packs per capita)") xtitle("Year") /// > legend(order(1 "California" 2 "Donor Pool Average") /// > position(6)) /// > title("Cigarette Sales: California vs. Donor Pool") /// > note("Source: Abadie, Diamond & Hainmueller (2010)." /// > "Vertical line = Proposition 99 (1989).") /// > graphregion(color(white)) plotregion(color(white)) /// > name(raw_trends, replace) . . graph export "stata_sc_raw_trends.png", replace width(2400) (file stata_sc_raw_trends.png not found) file stata_sc_raw_trends.png written in PNG format . . restore . . di "Figure saved: stata_sc_raw_trends.png" Figure saved: stata_sc_raw_trends.png . di _newline . di "Before 1989, California's cigarette sales broadly tracked" Before 1989, California's cigarette sales broadly tracked . di "the donor pool average. After Proposition 99, California's" the donor pool average. After Proposition 99, California's . di "sales diverge sharply downward, suggesting a treatment effect." sales diverge sharply downward, suggesting a treatment effect. . di "However, a simple average is not the best comparison --" However, a simple average is not the best comparison -- . di "the SCM constructs a weighted combination of control states" the SCM constructs a weighted combination of control states . di "that better matches California's pre-treatment trajectory." that better matches California's pre-treatment trajectory. . . . *--------------------------------------------------- . * Section 3: Baseline synthetic control estimate . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 3: BASELINE SCM ESTIMATE" SECTION 3: BASELINE SCM ESTIMATE . di "========================================" ======================================== . . di _newline . di "Command: synth2 with nested optimization and allopt" Command: synth2 with nested optimization and allopt . di _newline . di "Predictors:" Predictors: . di " lnincome - Log income (economic demand factor)" lnincome - Log income (economic demand factor) . di " age15to24 - Young population share (demographic)" age15to24 - Young population share (demographic) . di " retprice - Retail price (price elasticity)" retprice - Retail price (price elasticity) . di " beer - Beer consumption (complementary goods)" beer - Beer consumption (complementary goods) . di " cigsale(1988), cigsale(1980), cigsale(1975)" cigsale(1988), cigsale(1980), cigsale(1975) . di " - Pre-treatment cigarette sales at key years" - Pre-treatment cigarette sales at key years . di _newline . di "Options:" Options: . di " trunit(3) - Treated unit: California (state==3)" trunit(3) - Treated unit: California (state==3) . di " trperiod(1989) - Treatment onset: January 1989" trperiod(1989) - Treatment onset: January 1989 . di " xperiod(1980(1)1988) - Predictor averaging: 1980-1988" xperiod(1980(1)1988) - Predictor averaging: 1980-1988 . di " nested - Nested optimization for better weights" nested - Nested optimization for better weights . di " allopt - Multiple starting values for robustness" allopt - Multiple starting values for robustness . di _newline . di "Running baseline SCM... (this may take a few minutes)" Running baseline SCM... (this may take a few minutes) . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1988) cigsale(1980) c > igsale(1975), trunit(3) trperiod(1989) xperiod(1980(1)1988) nested allopt sav > egraph(stata_sc, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 1.7556 > 7 Number of Covariates = 7 R-squared = 0.9743 > 4 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0000 10.0766 9.8588 -2.16% 9.8292 -2.45 > % age15to24 | 0.5459 0.1735 0.1735 -0.01% 0.1725 -0.59 > % retprice | 0.0174 89.4222 89.4108 -0.01% 87.2661 -2.41 > % beer | 0.0031 24.2800 24.2278 -0.21% 23.6553 -2.57 > % cigsale(1988) | 0.0049 90.1000 91.6677 1.74% 113.8237 26.33 > % cigsale(1980) | 0.0066 120.2000 120.5017 0.25% 138.0895 14.88 > % cigsale(1975) | 0.4221 127.1000 127.1112 0.01% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3340 Nevada | 0.2350 Montana | 0.2020 Colorado | 0.1610 Connecticut | 0.0680 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Nebraska NewHampshire NewMexico NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 89.9945 -7.5945 1990 | 77.8000 87.5039 -9.7039 1991 | 68.7000 82.1751 -13.4751 1992 | 67.5000 81.6075 -14.1075 1993 | 63.4000 81.1897 -17.7897 1994 | 58.6000 80.7295 -22.1295 1995 | 56.4000 78.5023 -22.1023 1996 | 54.5000 77.4827 -22.9827 1997 | 53.8000 77.7123 -23.9123 1998 | 52.3000 74.3976 -22.0976 1999 | 47.2000 73.5711 -26.3711 2000 | 41.6000 67.3550 -25.7550 ------+---------------------------------------------------- Mean | 60.3500 79.3518 -19.0018 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -19.0018. (file stata_sc_bias.gph not found) file stata_sc_bias.gph saved (file stata_sc_weight_vars.gph not found) file stata_sc_weight_vars.gph saved (file stata_sc_weight_unit.gph not found) file stata_sc_weight_unit.gph saved (file stata_sc_pred.gph not found) file stata_sc_pred.gph saved (file stata_sc_eff.gph not found) file stata_sc_eff.gph saved Finished. . . * Display stored results . ereturn list scalars: e(N) = 1209 e(T) = 31 e(T0) = 19 e(T1) = 12 e(K) = 7 e(J) = 39 e(rmse) = 1.755672369538799 e(r2) = .9743364199596987 e(att) = -19.0017671585083 macros: e(panelvar) : "state" e(timevar) : "year" e(varlist) : "cigsale lnincome age15to24 retprice beer cigsale(.." e(depvar) : "cigsale" e(indepvars) : "lnincome age15to24 retprice beer cigsale(1988) ci.." e(unit_all) : "Alabama Arkansas California Colorado Connecticut .." e(unit_tr) : "California" e(unit_ctrl) : "Alabama Arkansas Colorado Connecticut Delaware Ge.." e(time_all) : "1970 1971 1972 1973 1974 1975 1976 1977 1978 1979.." e(time_tr) : "1989" e(time_pre) : "1970 1971 1972 1973 1974 1975 1976 1977 1978 1979.." e(time_post) : "1989 1990 1991 1992 1993 1994 1995 1996 1997 1998.." e(graph) : " bias weight_vars weight_unit pred eff" matrices: e(V_wt) : 7 x 7 e(U_wt) : 5 x 1 e(bal) : 7 x 6 . . * Save matrices before graph operations clear e() . matrix X_balance = e(bal) . matrix W_weights = e(U_wt) . . * Convert .gph files to .png (batch mode compatible) . foreach g in pred eff bias weight_unit weight_vars { 2. capture confirm file "stata_sc_`g'.gph" 3. if _rc == 0 { 4. graph use "stata_sc_`g'.gph" 5. graph export "stata_sc_`g'.png", replace width(2400) 6. erase "stata_sc_`g'.gph" 7. di "Figure saved: stata_sc_`g'.png" 8. } 9. } (file stata_sc_pred.png not found) file stata_sc_pred.png written in PNG format Figure saved: stata_sc_pred.png (file stata_sc_eff.png not found) file stata_sc_eff.png written in PNG format Figure saved: stata_sc_eff.png (file stata_sc_bias.png not found) file stata_sc_bias.png written in PNG format Figure saved: stata_sc_bias.png (file stata_sc_weight_unit.png not found) file stata_sc_weight_unit.png written in PNG format Figure saved: stata_sc_weight_unit.png (file stata_sc_weight_vars.png not found) file stata_sc_weight_vars.png written in PNG format Figure saved: stata_sc_weight_vars.png . . . *--------------------------------------------------- . * Section 4: Interpret predictor balance and weights . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 4: PREDICTOR BALANCE & WEIGHTS" SECTION 4: PREDICTOR BALANCE & WEIGHTS . di "========================================" ======================================== . . * Predictor balance: California vs. Synthetic California . di _newline . di "Predictor Balance (California vs. Synthetic California):" Predictor Balance (California vs. Synthetic California): . di "-------------------------------------------------------" ------------------------------------------------------- . matrix list X_balance X_balance[7,6] Vweight Treated ValueSynth~l BiasSynthe~l lnincome .00004916 10.076559 9.8587684 -2.161355 age15to24 .54587094 .17353238 .17352193 -.00602001 retprice .01741005 89.422223 89.4108 -.01277453 beer .0031354 24.28 24.2278 -.21499293 cigsale(1988) .00490342 90.099998 91.667698 1.7399554 cigsale(1980) .00655685 120.2 120.5017 .25100113 cigsale(1975) .42207419 127.1 127.1112 .00881275 ValueAvera~l BiasAverag~l lnincome 9.8291968 -2.4548245 age15to24 .17251011 -.5890933 retprice 87.266082 -2.411192 beer 23.655263 -2.5730524 cigsale(1988) 113.82368 26.330395 cigsale(1980) 138.08947 14.883093 cigsale(1975) 136.93158 7.7353113 . . di _newline . di "The predictor balance table compares California's actual" The predictor balance table compares California's actual . di "predictor values with those of the synthetic California." predictor values with those of the synthetic California. . di "Close values indicate a good pre-treatment match." Close values indicate a good pre-treatment match. . di "The SCM optimizes predictor weights (V matrix) to" The SCM optimizes predictor weights (V matrix) to . di "minimize this discrepancy." minimize this discrepancy. . . * Unit weights: which states form synthetic California . di _newline . di "Unit Weights (Donor Pool Contributions):" Unit Weights (Donor Pool Contributions): . di "-----------------------------------------" ----------------------------------------- . matrix list W_weights W_weights[5,1] Weight Utah .334 Nevada .235 Montana .202 Colorado .161 Connecticut .068 . . di _newline . di "The unit weights show which control states contribute" The unit weights show which control states contribute . di "to the synthetic California. Most states receive zero" to the synthetic California. Most states receive zero . di "or near-zero weight. The key contributors are states" or near-zero weight. The key contributors are states . di "with similar pre-treatment cigarette sales trajectories." with similar pre-treatment cigarette sales trajectories. . di "Weights are non-negative and sum to 1." Weights are non-negative and sum to 1. . . . *--------------------------------------------------- . * Section 5: Interpret treatment effects . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 5: TREATMENT EFFECTS" SECTION 5: TREATMENT EFFECTS . di "========================================" ======================================== . . di _newline . di "Treatment Effect Interpretation:" Treatment Effect Interpretation: . di "================================" ================================ . di _newline . di "The 'pred' graph shows California's actual cigarette" The 'pred' graph shows California's actual cigarette . di "sales vs. the synthetic California's predicted sales." sales vs. the synthetic California's predicted sales. . di "Pre-1989: the two lines closely track each other," Pre-1989: the two lines closely track each other, . di "indicating a good pre-treatment fit." indicating a good pre-treatment fit. . di _newline . di "Post-1989: California's actual sales fall sharply below" Post-1989: California's actual sales fall sharply below . di "the synthetic control, indicating that Proposition 99" the synthetic control, indicating that Proposition 99 . di "reduced cigarette consumption." reduced cigarette consumption. . di _newline . di "The 'eff' graph shows the gap (treatment effect) over" The 'eff' graph shows the gap (treatment effect) over . di "time. The negative gap widens through the 1990s," time. The negative gap widens through the 1990s, . di "suggesting the policy's effect grew stronger over time." suggesting the policy's effect grew stronger over time. . di _newline . di "Estimand: ATT (Average Treatment Effect on Treated)" Estimand: ATT (Average Treatment Effect on Treated) . di "The estimated effect is on California specifically --" The estimated effect is on California specifically -- . di "not a general population parameter." not a general population parameter. . . . *=================================================== . * PART 2: INFERENCE AND ROBUSTNESS . * Placebo tests and leave-one-out analysis . *=================================================== . . . *--------------------------------------------------- . * Section 6: In-space placebo test . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 6: IN-SPACE PLACEBO TEST" SECTION 6: IN-SPACE PLACEBO TEST . di "========================================" ======================================== . . di _newline . di "In-space placebo test: apply the SCM to each control" In-space placebo test: apply the SCM to each control . di "state as if IT were the treated unit. If California's" state as if IT were the treated unit. If California's . di "estimated effect is unusually large compared to these" estimated effect is unusually large compared to these . di "placebo effects, we have evidence of a real treatment" placebo effects, we have evidence of a real treatment . di "effect rather than a statistical artifact." effect rather than a statistical artifact. . di _newline . di "Options:" Options: . di " placebo(unit) - Run SCM for each control state" placebo(unit) - Run SCM for each control state . di " cut(2) - Keep states with pre-MSPE <= 2x CA's" cut(2) - Keep states with pre-MSPE <= 2x CA's . di " (filters out poor-fit placebos)" (filters out poor-fit placebos) . di " sigf(6) - 6 significant figures for convergence" sigf(6) - 6 significant figures for convergence . di " (allopt dropped to save computation time)" (allopt dropped to save computation time) . di _newline . di "Running in-space placebo test... (this takes several minutes)" Running in-space placebo test... (this takes several minutes) . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1988) cigsale(1980) c > igsale(1975), trunit(3) trperiod(1989) xperiod(1980(1)1988) nested placebo(un > it cut(2)) sigf(6) savegraph(stata_sc, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 1.7795 > 5 Number of Covariates = 7 R-squared = 0.9741 > 1 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0001 10.0766 9.8527 -2.22% 9.8292 -2.45 > % age15to24 | 0.0020 0.1735 0.1737 0.07% 0.1725 -0.59 > % retprice | 0.0147 89.4222 89.3766 -0.05% 87.2661 -2.41 > % beer | 0.0083 24.2800 24.2236 -0.23% 23.6553 -2.57 > % cigsale(1988) | 0.0142 90.1000 91.3866 1.43% 113.8237 26.33 > % cigsale(1980) | 0.1923 120.2000 120.2357 0.03% 138.0895 14.88 > % cigsale(1975) | 0.7683 127.1000 127.0999 -0.00% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3450 Nevada | 0.2410 Montana | 0.2070 Colorado | 0.1480 Connecticut | 0.0590 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Nebraska NewHampshire NewMexico NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 89.8201 -7.4201 1990 | 77.8000 87.3789 -9.5789 1991 | 68.7000 81.9182 -13.2182 1992 | 67.5000 81.4061 -13.9061 1993 | 63.4000 81.0228 -17.6228 1994 | 58.6000 80.5678 -21.9678 1995 | 56.4000 78.3083 -21.9083 1996 | 54.5000 77.3429 -22.8429 1997 | 53.8000 77.6174 -23.8174 1998 | 52.3000 74.1877 -21.8877 1999 | 47.2000 73.3950 -26.1950 2000 | 41.6000 67.1478 -25.5478 ------+---------------------------------------------------- Mean | 60.3500 79.1761 -18.8261 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -18.8261. Implementing placebo test using fake treatment unit Alabama...Arkansas...Colora > do...Connecticut...Delaware...Georgia...Idaho...Illinois...Indiana...Iowa...K > ansas...Kentucky...Louisiana...Maine...Minnesota...Mississippi...Missouri...M > ontana...Nebraska...Nevada...NewHampshire...NewMexico...NorthCarolina...North > Dakota...Ohio...Oklahoma...Pennsylvania...RhodeIsland...SouthCarolina...South > Dakota...Tennessee...Texas...Utah...Vermont...Virginia...WestVirginia...Wisco > nsin...Wyoming... In-space placebo test results using fake treatment units: ------------------------------------------------------------------------------- Unit | Pre MSPE Post MSPE Post/Pre MSPE Pre MSPE of Fake Unit/ | Pre MSPE of Treated Unit ---------------+--------------------------------------------------------------- California | 3.1668 391.2533 123.5490 1.0000 Alabama | 5.4170 8.2108 1.5157 1.7106 Arkansas | 4.5587 28.8239 6.3228 1.4395 Colorado | 17.6103 68.6736 3.8996 5.5609 Connecticut | 20.6396 118.9189 5.7617 6.5175 Delaware | 30.3949 499.0694 16.4195 9.5980 Georgia | 1.4610 116.8893 80.0074 0.4613 Idaho | 5.8142 39.1830 6.7392 1.8360 Illinois | 4.3146 89.0552 20.6406 1.3624 Indiana | 14.4145 469.4150 32.5654 4.5518 Iowa | 14.6527 31.5816 2.1553 4.6270 Kansas | 14.1121 9.0349 0.6402 4.4563 Kentucky | 431.7229 1475.7975 3.4184 136.3284 Louisiana | 2.0183 94.1070 46.6279 0.6373 Maine | 8.6989 115.5412 13.2822 2.7469 Minnesota | 14.1736 52.5217 3.7056 4.4757 Mississippi | 4.0894 37.2754 9.1151 1.2913 Missouri | 1.2009 85.1794 70.9308 0.3792 Montana | 5.2861 54.8978 10.3853 1.6692 Nebraska | 4.8287 36.5597 7.5713 1.5248 Nevada | 40.6500 83.4186 2.0521 12.8364 NewHampshire | 3436.5980 134.9018 0.0393 1085.2007 NewMexico | 5.0577 63.7459 12.6036 1.5971 NorthCarolina | 90.2241 67.3144 0.7461 28.4907 NorthDakota | 8.0725 72.3200 8.9588 2.5491 Ohio | 2.9585 12.3535 4.1757 0.9342 Oklahoma | 5.7128 267.8078 46.8786 1.8040 Pennsylvania | 2.7029 6.7073 2.4815 0.8535 RhodeIsland | 87.9904 242.6697 2.7579 27.7854 SouthCarolina | 2.1997 41.2941 18.7727 0.6946 SouthDakota | 7.5688 32.2367 4.2592 2.3901 Tennessee | 5.2043 123.3097 23.6938 1.6434 Texas | 4.6691 239.8559 51.3707 1.4744 Utah | 593.7643 223.2758 0.3760 187.4975 Vermont | 15.3860 116.8473 7.5944 4.8585 Virginia | 2.7825 219.8136 78.9994 0.8786 WestVirginia | 8.1492 242.1734 29.7175 2.5733 Wisconsin | 2.8950 75.2425 25.9901 0.9142 Wyoming | 83.7717 31.8266 0.3799 26.4532 ------------------------------------------------------------------------------- Note: (1) Using all control units, the probability of obtaining a post/pretreatment MSPE ratio as large as California's is 0.0256. (2) Excluding control units with pretreatment MSPE 2 times larger than the treated unit, the probability of obtaining a post/pretreatment MSPE ratio as large as California's is 0.0500. (3) The pointwise p-values below are computed by excluding control units with pretreatment MSPE 2 times larger than the treated unit. (4) There are total 19 units with pretreatment MSPE 2 times larger than the treated unit, including Colorado Connecticut Delaware Indiana Iowa Kansas Kentucky Maine Minnesota Nevada NewHampshire NorthCarolina NorthDakota RhodeIsland SouthDakota Utah Vermont WestVirginia Wyoming. In-space placebo test results using fake treatment units (continued, cutoff = 2 > ): ---------------------------------------------------------------- Time | Treatment Effect p-value of Treatment Effect | Two-sided Right-sided Left-sided ------+--------------------------------------------------------- 1989 | -7.4201 0.0500 1.0000 0.0500 1990 | -9.5789 0.1000 0.9500 0.1000 1991 | -13.2182 0.1500 0.9000 0.1500 1992 | -13.9061 0.1000 0.9500 0.1000 1993 | -17.6228 0.0500 1.0000 0.0500 1994 | -21.9678 0.0500 1.0000 0.0500 1995 | -21.9083 0.0500 1.0000 0.0500 1996 | -22.8429 0.0500 1.0000 0.0500 1997 | -23.8174 0.0500 1.0000 0.0500 1998 | -21.8877 0.1000 0.9500 0.1000 1999 | -26.1950 0.0500 1.0000 0.0500 2000 | -25.5478 0.0500 1.0000 0.0500 ---------------------------------------------------------------- Note: (1) The two-sided p-value of the treatment effect for a particular period is defined as the frequency that the absolute values of the placebo effects are greater than or equal to the absolute value of treatment effect. (2) The right-sided (left-sided) p-value of the treatment effect for a particular period is defined as the frequency that the placebo effects are greater (smaller) than or equal to the treatment effect. (3) If the estimated treatment effect is positive, then the right-sided p-value is recommended; whereas the left-sided p-value is recommended if the estimated treatment effect is negative. (file stata_sc_bias.gph not found) file stata_sc_bias.gph saved (file stata_sc_weight_vars.gph not found) file stata_sc_weight_vars.gph saved (file stata_sc_weight_unit.gph not found) file stata_sc_weight_unit.gph saved (file stata_sc_pred.gph not found) file stata_sc_pred.gph saved (file stata_sc_eff.gph not found) file stata_sc_eff.gph saved (file stata_sc_eff_pboUnit.gph not found) file stata_sc_eff_pboUnit.gph saved (file stata_sc_ratio_pboUnit.gph not found) file stata_sc_ratio_pboUnit.gph saved (file stata_sc_pvalTwo_pboUnit.gph not found) file stata_sc_pvalTwo_pboUnit.gph saved (file stata_sc_pvalRight_pboUnit.gph not found) file stata_sc_pvalRight_pboUnit.gph saved (file stata_sc_pvalLeft_pboUnit.gph not found) file stata_sc_pvalLeft_pboUnit.gph saved Finished. . . * Convert placebo .gph files to .png . foreach g in eff_pboUnit ratio_pboUnit pvalTwo_pboUnit pvalRight_pboUnit pval > Left_pboUnit { 2. capture confirm file "stata_sc_`g'.gph" 3. if _rc == 0 { 4. graph use "stata_sc_`g'.gph" 5. graph export "stata_sc_`g'.png", replace width(2400) 6. erase "stata_sc_`g'.gph" 7. di "Figure saved: stata_sc_`g'.png" 8. } 9. } (file stata_sc_eff_pboUnit.png not found) file stata_sc_eff_pboUnit.png written in PNG format Figure saved: stata_sc_eff_pboUnit.png (file stata_sc_ratio_pboUnit.png not found) file stata_sc_ratio_pboUnit.png written in PNG format Figure saved: stata_sc_ratio_pboUnit.png (file stata_sc_pvalTwo_pboUnit.png not found) file stata_sc_pvalTwo_pboUnit.png written in PNG format Figure saved: stata_sc_pvalTwo_pboUnit.png (file stata_sc_pvalRight_pboUnit.png not found) file stata_sc_pvalRight_pboUnit.png written in PNG format Figure saved: stata_sc_pvalRight_pboUnit.png (file stata_sc_pvalLeft_pboUnit.png not found) file stata_sc_pvalLeft_pboUnit.png written in PNG format Figure saved: stata_sc_pvalLeft_pboUnit.png . . di _newline . di "In-Space Placebo Results:" In-Space Placebo Results: . di "=========================" ========================= . di _newline . di "eff_pboUnit: Spaghetti plot of treatment effects for all" eff_pboUnit: Spaghetti plot of treatment effects for all . di " states. California's line (bold) should stand out as an" states. California's line (bold) should stand out as an . di " outlier among the grey placebo lines." outlier among the grey placebo lines. . di _newline . di "ratio_pboUnit: Ranks states by post/pre MSPE ratio." ratio_pboUnit: Ranks states by post/pre MSPE ratio. . di " A high ratio means the post-treatment gap is large" A high ratio means the post-treatment gap is large . di " relative to pre-treatment fit. California should rank" relative to pre-treatment fit. California should rank . di " at or near the top." at or near the top. . di _newline . di "pvalTwo/Right/Left: Fisher exact p-values over time." pvalTwo/Right/Left: Fisher exact p-values over time. . di " Two-sided: tests for any effect (positive or negative)" Two-sided: tests for any effect (positive or negative) . di " Right-tail: tests for a negative effect on sales" Right-tail: tests for a negative effect on sales . di " Left-tail: tests for a positive effect on sales" Left-tail: tests for a positive effect on sales . di " If California's p-value is below 0.05, the effect is" If California's p-value is below 0.05, the effect is . di " statistically significant at the 5% level." statistically significant at the 5% level. . . . *--------------------------------------------------- . * Section 7: In-time placebo test . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 7: IN-TIME PLACEBO TEST" SECTION 7: IN-TIME PLACEBO TEST . di "========================================" ======================================== . . di _newline . di "In-time placebo test: pretend the treatment happened" In-time placebo test: pretend the treatment happened . di "in 1985 (4 years before the actual intervention)." in 1985 (4 years before the actual intervention). . di "If the model is valid, there should be NO significant" If the model is valid, there should be NO significant . di "effect at this fake treatment date." effect at this fake treatment date. . di _newline . di "Key changes from baseline:" Key changes from baseline: . di " - Dropped cigsale(1988) from predictors" - Dropped cigsale(1988) from predictors . di " (would be post-fake-treatment)" (would be post-fake-treatment) . di " - xperiod(1980(1)1984) instead of 1980(1)1988" - xperiod(1980(1)1984) instead of 1980(1)1988 . di " (predictor averaging ends before fake treatment)" (predictor averaging ends before fake treatment) . di " - placebo(period(1985)) specifies fake treatment year" - placebo(period(1985)) specifies fake treatment year . di _newline . di "Running in-time placebo test..." Running in-time placebo test... . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1980) cigsale(1975), > trunit(3) trperiod(1989) xperiod(1980(1)1984) nested placebo(period(1985)) sa > vegraph(stata_sc, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 2.2053 > 0 Number of Covariates = 6 R-squared = 0.9525 > 3 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0000 10.0372 9.8639 -1.73% 9.7892 -2.47 > % age15to24 | 0.0000 0.1815 0.1825 0.55% 0.1814 -0.06 > % retprice | 0.0384 76.2200 76.1523 -0.09% 71.8353 -5.75 > % beer | 0.0000 25.0000 23.0089 -7.96% 23.6947 -5.22 > % cigsale(1980) | 0.9597 120.2000 120.0846 -0.10% 138.0895 14.88 > % cigsale(1975) | 0.0019 127.1000 126.8324 -0.21% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3600 Nevada | 0.2880 Connecticut | 0.1990 Colorado | 0.1020 NewMexico | 0.0500 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Montana Nebraska NewHampshire NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 93.0322 -10.6322 1990 | 77.8000 89.4297 -11.6297 1991 | 68.7000 82.4727 -13.7727 1992 | 67.5000 80.6731 -13.1731 1993 | 63.4000 79.5929 -16.1929 1994 | 58.6000 78.1272 -19.5272 1995 | 56.4000 75.6207 -19.2207 1996 | 54.5000 74.8372 -20.3372 1997 | 53.8000 74.5395 -20.7395 1998 | 52.3000 71.1561 -18.8561 1999 | 47.2000 71.4380 -24.2380 2000 | 41.6000 65.8382 -24.2382 ------+---------------------------------------------------- Mean | 60.3500 78.0631 -17.7131 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -17.7131. Implementing placebo test using fake treatment time 1985... In-time placebo test results using fake treatment time 1985: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1985 | 102.8000 106.1262 -3.3262 1986 | 99.7000 103.2850 -3.5850 1987 | 97.5000 106.1524 -8.6524 1988 | 90.1000 98.4873 -8.3873 1989 | 82.4000 96.5237 -14.1237 1990 | 77.8000 91.9127 -14.1127 1991 | 68.7000 83.7156 -15.0156 1992 | 67.5000 81.4730 -13.9730 1993 | 63.4000 79.7911 -16.3911 1994 | 58.6000 77.9078 -19.3078 1995 | 56.4000 76.2193 -19.8193 1996 | 54.5000 75.2010 -20.7010 1997 | 53.8000 75.1958 -21.3958 1998 | 52.3000 71.9437 -19.6437 1999 | 47.2000 72.2260 -25.0260 2000 | 41.6000 67.1861 -25.5861 ------+---------------------------------------------------- Mean | 69.6437 85.2092 -15.5654 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -15.5654. file stata_sc_bias.gph saved file stata_sc_weight_vars.gph saved file stata_sc_weight_unit.gph saved file stata_sc_pred.gph saved file stata_sc_eff.gph saved (file stata_sc_pred_pboTime1985.gph not found) file stata_sc_pred_pboTime1985.gph saved (file stata_sc_eff_pboTime1985.gph not found) file stata_sc_eff_pboTime1985.gph saved Finished. . . * Convert in-time placebo .gph files to .png . foreach g in pred_pboTime1985 eff_pboTime1985 { 2. capture confirm file "stata_sc_`g'.gph" 3. if _rc == 0 { 4. graph use "stata_sc_`g'.gph" 5. graph export "stata_sc_`g'.png", replace width(2400) 6. erase "stata_sc_`g'.gph" 7. di "Figure saved: stata_sc_`g'.png" 8. } 9. } (file stata_sc_pred_pboTime1985.png not found) file stata_sc_pred_pboTime1985.png written in PNG format Figure saved: stata_sc_pred_pboTime1985.png (file stata_sc_eff_pboTime1985.png not found) file stata_sc_eff_pboTime1985.png written in PNG format Figure saved: stata_sc_eff_pboTime1985.png . . di _newline . di "In-Time Placebo Results:" In-Time Placebo Results: . di "========================" ======================== . di _newline . di "pred_pboTime1985: Shows California vs. Synthetic California" pred_pboTime1985: Shows California vs. Synthetic California . di " with the fake 1985 treatment date. The two lines should" with the fake 1985 treatment date. The two lines should . di " remain close between 1985 and 1989, confirming no spurious" remain close between 1985 and 1989, confirming no spurious . di " pre-treatment effect." pre-treatment effect. . di _newline . di "eff_pboTime1985: Shows the gap (effect) over time with the" eff_pboTime1985: Shows the gap (effect) over time with the . di " fake treatment. Between 1985 and 1989, the gap should be" fake treatment. Between 1985 and 1989, the gap should be . di " near zero. After 1989 (the real treatment), the gap should" near zero. After 1989 (the real treatment), the gap should . di " widen, consistent with the baseline results." widen, consistent with the baseline results. . di _newline . di "If a large effect appears at the fake treatment date, it" If a large effect appears at the fake treatment date, it . di "would suggest the model is overfitting or that unobserved" would suggest the model is overfitting or that unobserved . di "confounders are driving the results." confounders are driving the results. . . . *--------------------------------------------------- . * Section 8: Leave-one-out robustness check . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 8: LEAVE-ONE-OUT ROBUSTNESS" SECTION 8: LEAVE-ONE-OUT ROBUSTNESS . di "========================================" ======================================== . . di _newline . di "Leave-one-out (LOO) test: re-estimate the SCM" Leave-one-out (LOO) test: re-estimate the SCM . di "removing one donor state at a time. If the results" removing one donor state at a time. If the results . di "are sensitive to any single state, it raises concerns" are sensitive to any single state, it raises concerns . di "about the robustness of the synthetic control." about the robustness of the synthetic control. . di _newline . di "Options:" Options: . di " loo - Enable LOO iterations" loo - Enable LOO iterations . di " frame(california) - Store results in Stata frame" frame(california) - Store results in Stata frame . di " savegraph(california, replace) - Save individual .gph graphs" savegraph(california, replace) - Save individual .gph graphs . di _newline . di "Running leave-one-out test... (this takes several minutes)" Running leave-one-out test... (this takes several minutes) . . synth2 cigsale lnincome age15to24 retprice beer cigsale(1988) cigsale(1980) c > igsale(1975), trunit(3) trperiod(1989) xperiod(1980(1)1988) nested loo frame( > california) savegraph(california, replace) Fitting results in the pretreatment periods: ------------------------------------------------------------------------------- > - Treated Unit : California Treatment Time : 198 > 9 ------------------------------------------------------------------------------- > - Number of Control Units = 38 Root Mean Squared Error = 1.7832 > 9 Number of Covariates = 7 R-squared = 0.9736 > 5 ------------------------------------------------------------------------------- > - Covariate balance in the pretreatment periods: ------------------------------------------------------------------------------- > - Covariate | V.weight Treated Synthetic Control Average Control > | Value Bias Value Bias > ---------------+--------------------------------------------------------------- > - lnincome | 0.0002 10.0766 9.8509 -2.24% 9.8292 -2.45 > % age15to24 | 0.0124 0.1735 0.1736 0.03% 0.1725 -0.59 > % retprice | 0.0113 89.4222 89.3331 -0.10% 87.2661 -2.41 > % beer | 0.0416 24.2800 24.2554 -0.10% 23.6553 -2.57 > % cigsale(1988) | 0.0231 90.1000 91.2783 1.31% 113.8237 26.33 > % cigsale(1980) | 0.4378 120.2000 120.1872 -0.01% 138.0895 14.88 > % cigsale(1975) | 0.4735 127.1000 126.9907 -0.09% 136.9316 7.74 > % ------------------------------------------------------------------------------- > - Note: "V.weight" is the optimal covariate weight in the diagonal of V matrix. "Synthetic Control" is the weighted average of donor units with optimal weights. "Average Control" is the simple average of all control units with equal weights. Optimal Unit Weights: -------------------------- Unit | U.weight -------------+------------ Utah | 0.3420 Nevada | 0.2380 Montana | 0.2170 Colorado | 0.1450 Connecticut | 0.0580 -------------------------- Note: The unit Alabama Arkansas Delaware Georgia Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Minnesota Mississippi Missouri Nebraska NewHampshire NewMexico NorthCarolina NorthDakota Ohio Oklahoma Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Vermont Virginia WestVirginia Wisconsin Wyoming in the donor pool get a weight of 0. Prediction results in the posttreatment periods: ----------------------------------------------------------- Time | Actual Outcome Synthetic Outcome Treatment Effect ------+---------------------------------------------------- 1989 | 82.4000 89.7304 -7.3304 1990 | 77.8000 87.3001 -9.5001 1991 | 68.7000 81.8829 -13.1829 1992 | 67.5000 81.4287 -13.9287 1993 | 63.4000 81.0450 -17.6450 1994 | 58.6000 80.6229 -22.0229 1995 | 56.4000 78.4191 -22.0191 1996 | 54.5000 77.4316 -22.9316 1997 | 53.8000 77.7288 -23.9288 1998 | 52.3000 74.3255 -22.0255 1999 | 47.2000 73.4654 -26.2654 2000 | 41.6000 67.2107 -25.6107 ------+---------------------------------------------------- Mean | 60.3500 79.2159 -18.8659 ----------------------------------------------------------- Note: The average treatment effect over the posttreatment period is -18.8659. Implementing leave-one-out robustness test that excludes one control unit with > a nonzero weight Utah...Nevada...Montana...Colorado...Connecticut... Leave-one-out robustness test results in the posttreatment period: ------------------------------------------------------- Time | Outcome Synthetic Outcome (LOO) | Actual Synthetic Min Max ------+------------------------------------------------ 1989 | 82.4000 89.7304 88.3892 92.3509 1990 | 77.8000 87.3001 83.5373 89.2205 1991 | 68.7000 81.8829 80.8905 82.4889 1992 | 67.5000 81.4287 80.6239 81.8815 1993 | 63.4000 81.0450 79.7801 82.0592 1994 | 58.6000 80.6229 78.6141 83.3112 1995 | 56.4000 78.4191 75.9901 81.3864 1996 | 54.5000 77.4316 75.0801 80.5833 1997 | 53.8000 77.7288 71.7877 84.4150 1998 | 52.3000 74.3255 71.1668 79.0314 1999 | 47.2000 73.4654 71.5421 77.5396 2000 | 41.6000 67.2107 65.0850 69.9503 ------------------------------------------------------- Note: The last two columns report the minimum and maximum synthetic outcomes when one control unit with a nonzero weight is excluded at a time. ------------------------------------------------------- Time | Treatment Effect Treatment Effect (LOO) | Min Max ------+------------------------------------------------ 1989 | -7.3304 -9.9509 -5.9892 1990 | -9.5001 -11.4205 -5.7373 1991 | -13.1829 -13.7889 -12.1905 1992 | -13.9287 -14.3815 -13.1239 1993 | -17.6450 -18.6592 -16.3801 1994 | -22.0229 -24.7112 -20.0141 1995 | -22.0191 -24.9864 -19.5901 1996 | -22.9316 -26.0833 -20.5801 1997 | -23.9288 -30.6150 -17.9877 1998 | -22.0255 -26.7314 -18.8668 1999 | -26.2654 -30.3396 -24.3421 2000 | -25.6107 -28.3503 -23.4850 ------------------------------------------------------- Note: The last two columns report the minimum and maximum treatment effects when one control unit with a nonzero weight is excluded at a time. (file california_bias.gph not found) file california_bias.gph saved (file california_weight_vars.gph not found) file california_weight_vars.gph saved (file california_weight_unit.gph not found) file california_weight_unit.gph saved (file california_pred.gph not found) file california_pred.gph saved (file california_eff.gph not found) file california_eff.gph saved (file california_pred_loo.gph not found) file california_pred_loo.gph saved (file california_eff_loo.gph not found) file california_eff_loo.gph saved Finished. . . * Combine all LOO graphs into a single display . graph combine `e(graph)', cols(2) altshrink /// > title("Leave-One-Out Robustness: Synthetic California") /// > note("Each panel excludes one donor state.") /// > graphregion(color(white)) /// > name(loo_combined, replace) . . graph export "stata_sc_loo_combined.png", replace width(2400) (file stata_sc_loo_combined.png not found) file stata_sc_loo_combined.png written in PNG format . di "Figure saved: stata_sc_loo_combined.png" Figure saved: stata_sc_loo_combined.png . . di _newline . di "Leave-One-Out Results:" Leave-One-Out Results: . di "======================" ====================== . di _newline . di "The combined graph shows predicted vs. actual cigarette" The combined graph shows predicted vs. actual cigarette . di "sales for California when each donor state is excluded" sales for California when each donor state is excluded . di "from the pool one at a time." from the pool one at a time. . di _newline . di "If the treatment effect estimate remains similar across" If the treatment effect estimate remains similar across . di "all LOO iterations, the results are robust. Substantial" all LOO iterations, the results are robust. Substantial . di "changes when a specific state is removed would indicate" changes when a specific state is removed would indicate . di "over-reliance on that state in the synthetic control." over-reliance on that state in the synthetic control. . . . *--------------------------------------------------- . * Section 9: Inspect leave-one-out frame . *--------------------------------------------------- . . di _newline(2) . di "========================================" ======================================== . di " SECTION 9: LOO FRAME INSPECTION" SECTION 9: LOO FRAME INSPECTION . di "========================================" ======================================== . . di _newline . di "The LOO results are stored in a Stata frame named" The LOO results are stored in a Stata frame named . di "'california'. Frames allow multiple datasets in memory" 'california'. Frames allow multiple datasets in memory . di "(available in Stata 16+)." (available in Stata 16+). . . frame change california . describe Contains data Observations: 1,209 Tobacco Sales in 39 US States Variables: 23 ------------------------------------------------------------------------------- Variable Storage Display Value name type format label Variable label ------------------------------------------------------------------------------- state long %14.0g state state no year float %9.0g year cigsale float %9.0g cigarette sale per capita (in packs) age15to24 float %9.0g percent of state population aged 15-24 years beer float %9.0g beer consumption per capita lnincome float %9.0g log state per capita gdp retprice float %9.0g retail price of cigarettes pred·cigsale float %9.0g prediction of cigsale tr·cigsale float %9.0g treatment effect on cigsale pred·cigsale·~h float %9.0g prediction of cigsale (Utah excluded) generated by 'robustness test' tr·cigsale·rm~h float %9.0g treatment effect on cigsale (Utah excluded) generated by 'robustness test' pred·cigsale~da float %9.0g prediction of cigsale (Nevada excluded) generated by 'robustness test' tr·cigsale·r~da float %9.0g treatment effect on cigsale (Nevada excluded) generated by 'robustness test' pred·cigsale~na float %9.0g prediction of cigsale (Montana excluded) generated by 'robustness test' tr·cigsale·r~na float %9.0g treatment effect on cigsale (Montana excluded) generated by 'robustness test' pred·cigsale·~o float %9.0g prediction of cigsale (Colorado excluded) generated by 'robustness test' tr·cigsale·rm~o float %9.0g treatment effect on cigsale (Colorado excluded) generated by 'robustness test' pred·cigsale·~t float %9.0g prediction of cigsale (Connecticut excluded) generated by 'robustness test' tr·cigsale·rm~t float %9.0g treatment effect on cigsale (Connecticut excluded) generated by 'robustness test pred·cigsale·~n float %9.0g min prediction of cigsale generated by 'robustness test' pred·cigsale·~x float %9.0g max prediction of cigsale generated by 'robustness test' tr·cigsale·lo~n float %9.0g min treatment effect on cigsale generated by 'robustness test' tr·cigsale·lo~x float %9.0g max treatment effect on cigsale generated by 'robustness test' ------------------------------------------------------------------------------- Sorted by: Note: Dataset has changed since last saved. . frame change default . . di _newline . di "The 'california' frame contains variables for each LOO" The 'california' frame contains variables for each LOO . di "iteration: predicted values, treatment effects, and the" iteration: predicted values, treatment effects, and the . di "identity of the excluded state. Researchers can use this" identity of the excluded state. Researchers can use this . di "frame for further custom analysis." frame for further custom analysis. . . . *--------------------------------------------------- . * Section 10: Closing summary . *--------------------------------------------------- . . di _newline(2) . di "============================================" ============================================ . di " ANALYSIS COMPLETE" ANALYSIS COMPLETE . di "============================================" ============================================ . di _newline . di " Estimand: ATT (Average Treatment Effect on Treated)" Estimand: ATT (Average Treatment Effect on Treated) . di " Treated unit: California (state==3)" Treated unit: California (state==3) . di " Treatment: Proposition 99 (effective January 1989)" Treatment: Proposition 99 (effective January 1989) . di _newline . di " Key Findings:" Key Findings: . di " 1. The SCM constructs a weighted combination of control" 1. The SCM constructs a weighted combination of control . di " states that closely matches California pre-1989." states that closely matches California pre-1989. . di " 2. After 1989, actual California cigarette sales fall" 2. After 1989, actual California cigarette sales fall . di " well below the synthetic control, indicating a" well below the synthetic control, indicating a . di " substantial reduction in cigarette consumption." substantial reduction in cigarette consumption. . di " 3. The in-space placebo test shows California's effect" 3. The in-space placebo test shows California's effect . di " is an outlier among control states, supporting" is an outlier among control states, supporting . di " statistical significance." statistical significance. . di " 4. The in-time placebo test confirms no spurious effect" 4. The in-time placebo test confirms no spurious effect . di " at the fake 1985 treatment date." at the fake 1985 treatment date. . di " 5. Leave-one-out analysis shows robust results across" 5. Leave-one-out analysis shows robust results across . di " different donor pool compositions." different donor pool compositions. . di _newline . di " Figures:" Figures: . di " stata_sc_raw_trends.png" stata_sc_raw_trends.png . di " stata_sc_pred.png" stata_sc_pred.png . di " stata_sc_eff.png" stata_sc_eff.png . di " stata_sc_bias.png" stata_sc_bias.png . di " stata_sc_weight_unit.png" stata_sc_weight_unit.png . di " stata_sc_weight_vars.png" stata_sc_weight_vars.png . di " stata_sc_eff_pboUnit.png" stata_sc_eff_pboUnit.png . di " stata_sc_ratio_pboUnit.png" stata_sc_ratio_pboUnit.png . di " stata_sc_pvalTwo_pboUnit.png" stata_sc_pvalTwo_pboUnit.png . di " stata_sc_pvalRight_pboUnit.png" stata_sc_pvalRight_pboUnit.png . di " stata_sc_pvalLeft_pboUnit.png" stata_sc_pvalLeft_pboUnit.png . di " stata_sc_pred_pboTime1985.png" stata_sc_pred_pboTime1985.png . di " stata_sc_eff_pboTime1985.png" stata_sc_eff_pboTime1985.png . di " stata_sc_loo_combined.png" stata_sc_loo_combined.png . di _newline . di " Reference:" Reference: . di " Abadie, A., Diamond, A. & Hainmueller, J. (2010)." Abadie, A., Diamond, A. & Hainmueller, J. (2010). . di " Synthetic control methods for comparative case" Synthetic control methods for comparative case . di " studies: Estimating the effect of California's" studies: Estimating the effect of California's . di " tobacco control program. JASA, 105(490), 493-505." tobacco control program. JASA, 105(490), 493-505. . di "============================================" ============================================ . di _newline . di "=== Script completed successfully ===" === Script completed successfully === . . log close name: log: /Users/carlosmendez/Documents/GitHub/starter-academic-v501/content > /post/stata_sc/analysis.log log type: text closed on: 27 Apr 2026, 13:45:25 -------------------------------------------------------------------------------