*! Help file for diffplot.ado by Mead Over version 3.11 12Apr2024

help for diffplotVersion 3.11 12Apr2024

Title

diffplot -- Plot the results of a difference-in-differences
evaluation of causal impact and optionally export tables of the results

Syntax

diffplot
[
using filename
]
[if] [in]
,
period(varname)
treated(varname)
[
optional_options
]

Options

Required options
period(pname) Time period dummy. Must equal zero
before the treatment occurs and one after
the treatment has occurred.
The character string pname cannot
be embedded in the name gname.

treated(gname) Treatment group dummy. Must equal zero for the control
group and one for the group receiving the treatment.
The character string gname cannot
be embedded in the name pname.

Terminology: Synonyms in common use for referring to the "treatment" or "treated"
group include "enrolled", "intervention" and "experimental".
Synonyms for "control" group include "untreated", "comparison" and,
in biostatistics, "placebo".
Synonyms for "Before" include "Baseline" and "Initial"
and for "After" include "Endline" and "Final".
The term "counterfactual" outcome used here is a synonym
for Rubin's "potential" outcome.

Table export options
using filename Specifies the file to which optional tables are
written by the companion program frmttable,
which the user must separately install as part of the outreg package
here. By default, the exported file is formatted
for MS Word, but the user can specify export to a TeX file. Suboptions to exert finer control
over the exported tables are documented below.

Plot options

The primary purpose of diffplot is to construct the classic
visualization of the results of a difference-in-differences (DID)
analysis based on a preceding regression estimated by a standard
Stata estimation routine or by Juan Villa's community contributed program
diff. The following options control the production of that graph.

margat(atspec) Specifies that the DID estimates be evaluated
at specified values of covariates. See help for
atspec.

expression(exp) Transforms the DID outcomes and their
confidence intervals and test statistics to an alternative metric.
For example, if the dependent variable in the preceding statistical
analysis was the natural log of y, the option
expression(exp(predict(xb))) would cause the graph
to display outcomes in the metric of the antilog of ln(y).
See expression

allci Display the confidence intervals on all six estimated
DID levels. The default is to display no confidence intervals.
This option is an alternative to cfactci or tci.

cfactci Display only the confidence interval on the estimated
counterfactual and only after the intervention or treatment.
This option is an alternative to allci or tci.

tci Display only the confidence interval on the estimated
level of the treatment group and only after the intervention.
This option is an alternative to either cfactci or tci.

level(#) Set confidence level which determines the width
of displayed confidence intervals; default is level(95)
or the -level()- option specified in the preceding execution of diff.

title(string) Title for the graph and also for either the
difftable or tradtable tables, if they are requested.

subtitle(passthru) Subtitle for the graph. Not used on the tables.

ytitle(passthru) Title for the y-axis of the graph.

ylabel(passthru) Labels for the y-axis of the graph.
(Labels for the x-axis are controlled by diffplot and thus not user-selectable.)

xtitle(passthru) Title for the x-axis of the graph.

notext By default, diffplot labels all six plotted points
as well as the difference-in-differences estimate. Specifying the option
notext suppresses all of these text annotations.

name(string) Name of the diffplot memory graph.
Use sub-option replace to replace a memory graph with the same name.

saving(filename) Name of the diffplot graph file saved to disk.
Use sub-option replace to replace a saved graph file with the same name.

replace Allows diffplot to overwrite any previous
saved MS Word or TeX files generated by the options difftable
and tradtable.

beflbl(text) Label designating
the estimated level of the outcome when the variable specified in the period
option equals zero. Used in the diffplot graph and,
if difftable or tradtable is selected,
in the generated tables of results. Default: "Before".

aftlbl(text) Label designating
the estimated level of the outcome when the variable specified in the period
option equals one. Used in the diffplot graph and,
if difftable or tradtable is selected,
in the generated tables of results. Default: "After".

conlbl(text) Label designating
the estimated level of the outcome when the variable specified in the treated
option equals zero. Used in the diffplot graph and,
if difftable or tradtable is selected,
in the generated tables of results. Default: "Control".

trtlbl(text) Label designating
the estimated level of the outcome when variable specified in the treated
option equals one. Used in the diffplot graph and,
if difftable or tradtable is selected,
in the generated tables of results. Default: "Treated".

cfactlbl(text) Label designating
the estimated level of the counterfactual or potential outcome
in the absence of the treatment.
Used in the diffplot graph and,
if difftable or tradtable is selected,
in the generated tables of results. Default: "Counterfactual".

options specific to difftable
difftable Construct a single-column table summarizing the statistical
results of the DID analysis as displayed in the diffplot graph.
This table is modeled on that produced by Juan Villa's program
diff, with addtional options.

cfact In addition to the four rows of the table that present
the estimated outcome for both the control and treatment
group both before and after the treatment, this option
adds a fifth row presenting the estimated counterfactual
outcome after the treatment.

options specific to tradtable
tradtable Construct a two-by-two table summarizing the statistical
results of the DID analysis as displayed in the diffplot graph.
This table is modeled on those used in the classical exposition of
the DID method as in Gertler (2016), Angrist and Pischke(2009) and
Wikipedia (2020).

stats(subopts) Controls which statistics, if any, are to be placed
below the entries in the nine cells of the results tables.
The subopts may include any of
se, t, p and N, in any order,
separated by spaces or commas.
If specified, N and se are displayed in every cell of the table.
The other statistics, if specified, are displayed only in the table's
margins, as applied to one of the calculated differences.

Less frequently used or advanced options
margins(varlist) Argument for the margins command
executed by diffplot. Because the default is i.period#i.treated
usually works, this option is rarely needed.

reg4margins(name) When processing the DID results of the diff program,
diffplot renames the rows and columns of diff's returned e(b) and e(V)
regression results using factor-variable syntax so that they can be
interpreted by Stata's margins command.
As an aid to understanding diffplot's calculations and for possible
use debugging its interpretaion of diff results,
this option reg4margins stores the modified diff regression
results using Stata's estimates store command.
This option is only useful if the DID results processed by diffplot
were estimated by diff rather than by a Stata estimation command
using factor variable syntax for the interaction of
the period and treated variables.

difftxtleft On the diffplot graph, place the text labeling
the DID estimate and its p-value on the left, instead of on the right.
This option can help to highlight the DID result and
distinguish it from the other labeled outcome estimates.
With the notext option specified, this option is ineffective.

beftxtleft On the diffplot graph, place the text labeling
the "Before" estimates of both the control and the treatment groups
on the left, instead of on the right.
With the notext option specified, this option is ineffective.

estrestore(estname) If the results of a recent DID estimation are not
current in Stata's memory, this option invokes Stata's
estimates restore so that diffplot
can proceed. Cannot be used to restore the diff results.

estuse(estfilename) If the results of a recent DID estimation are not
current in Stata's memory, this option invokes Stata's
estimates use to retrieve estimates saved to disk
by estimates save.
Cannot be used to restore the diff results.

ciopts(options) Modify the look of the lines and caps of
the confidence intervals. These options apply to all selected confidence intervals.
See Stata's help file for ciopts.

textsize(textsizestyle) Controls the size of the text which labels
the diffplot graph. Default is vsmall.

legtitle(text) Specify a title for the graph's legend.
Default is no title.

legpos(opts) Allows the user to suppress the legend or specify its
location.
Accepts the options position(clockposstyle),
ring(ringposstyle), xoffset(#), yoffset(#) and off.
The default options are: pos(3) ring(1).

legopts(opts) Enables user control of legend
contents other than
pos(clockposstyle), ring(ringposstyle) and off.
Rarely helpful or needed.

keywidth(size) The default size specification of *0.8
usually assures that diffplot's default labels for the legend keys
fit within the legend box. User can experiment with other size
specifications.

verbose Displays intermediate output, including the output of the
margins and marginsplot commands.

frmtopts(opts) Options passed through to frmttable,
which produces the results tables. Neither this option nor any of
its suboptions listed below is needed to produce the diffplot graph
or the results tables generated by the options tradtable or difftable.

Formatting options for exported tables
using filename Specifies the file to which tables are
exported by the companion program frmttable,
which the user must separately install as part of the outreg package
here.

For the exported tables, the user may override some of diffplot's
defaults by specifying appropriate frmttable options as subopts within
the diffplot option frmtopts(subopts).
The following frmttable options are among the most useful.

tex By default, difftable or tradtable
writes its table to a file in MS Word format.
With the option tex, the file is instead written to a TeX file.

addtable Add the newly created
difftable or tradtable
to the bottom of the existing Word or Tex file named in the using option.
By default, diffplot replaces an existing file of the same name.

sfmt(fmtgrid) Numerical format for
all statistics in the requested traditional or diff-styled table.
Alternatives are f, g and e. Default is f.

colwidth(numlist) Change
column widths. Default is frmttable's best guess.

basefont(fontlist) Change
the base font for all text. Default is Arial 11.

nobrket Suppress the application
of brackets to statistics such as se, t, p and N in the tradtable output.

Forbidden frmttable options

The following frmttable options are pre-empted by diffplot
and thus NOT available through the option frmtopts(opts).
Specifying any of these options in the diffplot command line generates an error.
Furthermore, except options explicitly mentioned under
"frmttable options" above,
most of the other frmttable options,
will distort the generated tables in one way or another.
For more details on compatible and incompatible frmttable options,
click here

statmat(matrix) matrix of statistics for body of table

substat(#) number of substatistics
to place below first statistics.
diffplot with option difftable pre-empts this frmttable
option with its stats(subopts) option.

doubles(matrix) matrix indicating double statistics

eq_merge merge multi-equation statistics into multiple columns

noblankrows drop blank rows in table

findconst put _cons in separate section of table

sdec(numgrid) Decimal places for all statistics.

Description

The challenge of evaluating the impact of a "treatment" (or "intervention" or
"experiment") on a group of individuals (people or other subjects) is to
estimate what would have happened to that same group of individuals
without treatment.
However, since this group in fact received the treatment,
it is impossible to actually observe what
would have happened to them without treatment.
This unobservable situation, in which those who actually received the treatment did not receive it,
is called the "counterfactual" state of the world
and the outcome realized in this hypothetical state of the world is called
the "counterfactual" outcome, the "potential" outcome - or just the "counterfactual".
Alternative statistical methods of impact evaluation are essentially
techniques to estimate the expected counterfactual outcome and to test whether
the mean of the observed outcomes for treated individuals differs from
the expected counterfactual outcome.

The method of difference-in-differences (DID) is one of the most intuitive
approaches to estimating the expected counterfactual outcome for the treated
individuals. The DID approach starts with the definition of
a "treatment" or "intervention" which occurs between
two points in time. We refer to those two points in time as
"Before" and "After". At the "Before" time and the "After" time,
we measure an outcome for each individual receiving the treatment.
We also define a "comparison" or "control" group of individuals
who are as similar as possible to the treatment group,
but did not receive the treatment.
We measure the same outcome for each individual in the control
group at the same two points in time, Before and After.

The idea is to compare the average observed change in the
treated group to the average change that can be observed in a "comparable"
group (or "control" group") that did not receive the treatment.

Table 1. Traditional format for presenting DID results

| After Before | Difference
------------+------------------------+----------------
Treatment | B A | (B-A)
| | [p-value]
| |
Control | D C | (D–C)
| | [p-value]
------------|------------------------+----------------
| | DID=(B-A)–(D-C)
| | [p-value]*
Difference | (B-D) (A-C) | DID=(B-D)–(A-C)
| [p-value] [p-value] | [p-value]*

* The focus for hypothesis testing is the statistical significance of the DID.

The above table presents the calculation of the DID
in the format of the traditional DID table.
The letters A, B, C and D represent the average outcomes for individuals
in each of the four cells of the body of the table. The notation (B-A)
denotes the difference obtained by subtracting B from C.
(See for example Gertler et al., 2016, Table 7.1, p.133.)

The DID estimator of the impact of the intervention can be computed either
by differencing the two differences in the column at the right margin of the table
or by differencing the two differences in the row at the bottom margin of the table.

The formula for computing the DID, (B-A) - (D-C),
can be rearranged as B - [A + (D-C)].
The term in brackets of this alternative expression for the DID,
[A + (D-C)], is an estimate of the counterfactual or potential outcome.
In words, the estimated counterfactual outcome is the level that
the control group would have attained, if it had started at
A instead of at C, assuming that, in the absence of the intervention
(i.e. in the counterfactual state-of-the-world), the outcomes of the intervention
group would have changed at the same rate as the outcomes of the control group.

The last mentioned assumption, that in the counterfactual situation
the intervention group's outcomes would have changed at the same rate
as the control group, is referred to as the assumption of "parallel trends".
Parallel trends is one of several assumptions which must hold
for the DID method to provide a consistent
estimate of the impact of the intervention.

This program, diffplot, is designed as a postestimation command
following a DID estimation by any Stata estimation command
which expresses the interaction of the period and treated
variables using factor-variable syntax and
leaves in memory Stata's standard e(b), e(V) and e(sample)
results. Estimation commands which are potentially compatible with diffplot
include regress, xtreg and also the community contributed
DID estimator diff by Juan Villa.
In addition to facilitating DID analysis of the mean outcome,
diff allows DID analysis of a quantile,
adjustment for covariates, propensity score matching
and bootstrapped standard errors. diffplot can plot the results
of all of these diff analysis options.

diffplot is essentially a wrapper for Stata's margins
and marginsplot commands, which perform the work of computing and graphing
the DID results and their confidence intervals.

Discussion

The abstract of Card and Krueger's classic 1994
DID study in labor economics reads:

"On April 1, 1992 New Jersey's minimum wage increased from $4.25 to $5. 05 per
hour. To evaluate the impact of the law we surveyed 410 fast food restaurants
in New Jersey and Pennsylvania before and after the rise in the minimum.
Comparisons of the changes in wages, employment, and prices at stores in New
Jersey relative to stores in Pennsylvania (where the minimum wage remained
fixed at $4.25 per hour) yield simple estimates of the effect of the higher
minimum wage. Our empirical findings challenge the prediction that a rise in
the minimum reduces employment. Relative to stores in Pennsylvania, fast food
restaurants in New Jersey increased employment by 13 percent."
(Source: Abstract for Card & Kreuger, 1994.)

As background for the demonstration of diffplot in the
examples below,
we first use the original Card & Krueger data to show how Stata
can compute the DID estimate of impact from first principles.

. use cardkrueger1994edit.dta, clear
(Annotated version of Card & Krueger 1994. Type "notes".)

.
. * Describe the data
. describe

Contains data from cardkrueger1994edit.dta
Observations: 801 Dataset from Card&Krueger (1994)
Variables: 10 4 Aug 2023 14:21
(_dta has notes)

Variable Storage Display Value
name type format label Variable label

Contains data from cardkrueger1994edit.dta
Observations: 820 Annotated version of Card & Krueger 1994. Type "notes".
Variables: 11 14 Aug 2023 08:58
(_dta has notes)

Variable Storage Display Value
name type format label Variable label

id2 int %8.0g * Unique store id

period byte %8.0g period * Time period

treated byte %8.0g treated New Jersey = 1; Pennsylvania = 0

fte float %9.0g * Outcome: Full-time employees

bk byte %8.0g Burger King == 1

kfc byte %8.0g Kentuky Fried Chiken == 1

roys byte %8.0g Roy Rogers == 1

wendys byte %8.0g Wendy's == 1

chain byte %8.0g chain Polytomous franchise identifier

id int %8.0g * Store ID

t byte %8.0g * Feb. 1992 = 0; Nov. 1992 = 1

* indicated variables have notes

Sorted by: id2 period

In 19 restaurants, the outcome variable, -fte-, is not observed.

. summarize

Variable | Obs Mean Std. dev. Min Max

+

id2 | 820 268.4585 475.7312 1 9407
period | 820 .5 .5003052 0 1
treated | 820 .8073171 .3946469 0 1
fte | 801 17.59457 9.022517 0 80
bk | 820 .4170732 .4933761 0 1

+

kfc | 820 .195122 .3965364 0 1
roys | 820 .2414634 .4282318 0 1
wendys | 820 .1463415 .3536639 0 1
chain | 820 2.117073 1.109819 1 4
id | 820 246.5073 148.1413 1 522

+

t | 820 .5 .5003052 0 1

Drop the observations in which the outcome is not observed:

. drop if missing(fte)
(19 observations deleted)

The two indicator variables in the data are treated and period.
treated distinguishes the control from the treated units
and period distinguishes between observations before and after the treatment.

A tabulation gives a count of the observations in each
of the four cells of the classic DID study design:

. tab treated period

New Jersey |
= 1; |
Pennsylvan | Time period
ia = 0 | Feb 1992 Nov 1992 | Total

+
+

PA | 78 77 | 155
NJ | 326 320 | 646

+
+

Total | 404 397 | 801

Note that each of the four restaurant chains in the sample is
is represented on both sides of the PA-NJ border.

. tab chain treated

| New Jersey = 1;
Franchise | Pennsylvania = 0
brand | PA NJ | Total

+
+

BK | 69 265 | 334
KFC | 24 136 | 160
Roy's | 34 160 | 194
Wendy's | 28 85 | 113

+
+

Total | 155 646 | 801

The simplest version of a DID estimate can be computed by using the
tabulate command to estimate the mean in each group
at each period:

. tab treated period, sum(fte) mean

Means of Full-time employment

New Jersey |
= 1; |
Pennsylvan | Time period
ia = 0 | Feb 1992 Nov 1992 | Total

+
+

PA | 19.948718 17.542208 | 18.753226
NJ | 17.065184 17.572656 | 17.316563

+
+

Total | 17.621906 17.566751 | 17.594569

For hypothesis testing, it is convenient to estimate these four cell means
with an ordinary least squares regression.
Using Stata's ibn factor variable notation together with the option noconstant,
regress produces the same four cell means as well as standard errors and
t-tests of the hypotheses that each cell mean is zero.
(See https://www.stata.com/support/faqs/statistics/keep-all-levels-of-variable/)

. regress fte ibn.period#ibn.treated, noconstant

Source | SS df MS Number of obs = 801

+
F(4, 797) = 766.42
Model | 248488.667 4 62122.1667 Prob > F = 0.0000
Residual | 64600.6458 797 81.0547626 R-squared = 0.7937

+
Adj R-squared = 0.7926
Total | 313089.313 801 390.873049 Root MSE = 9.003

+

--------

fte | Coefficient Std. err. t P>|t| [95% con f. interval]

+

--------

period#treated |
Feb 1992#PA | 19.94872 1.019394 19.57 0.000 17.9477 21.94973
Feb 1992#NJ | 17.06518 .4986324 34.22 0.000 16.0864 18.04397
Nov 1992#PA | 17.54221 1.025992 17.10 0.000 15.52824 19.55617
Nov 1992#NJ | 17.57266 .5032853 34.92 0.000 16.58473 18.56058

+

--------

The theory of supply and demand posits that,
after New Jersey implemented the minimum wage,
restaurants in that state would shed employees more than
similar restaurants in the control group across the border in Pennsylvania.
Among the Pennsylvania restaurants, the average number
of full-time employees (FTEs)
declined from 19.949 to 17.542.
If the theory were correct, FTEs should decline even more
in New Jersey, where the minimum wage was implemented.

However, in contrast to the Pennsylvania restaurants,
employment in the New Jersey restaurants slightly increased,
contradicting the null hypothesis of no difference in the change
between the states.

A straightforward way to test whether the contradiction
of the null hypothesis that is observed in the descriptive statistics
is statistically significant is to use Stata's lincom command
with the coefficients of the above regression.

First, use regress with the option coeflegend to display
the coefficient names
by which lincom can refer to the coefficients:

. regress, coeflegend

Source | SS df MS Number of obs = 801

+
F(4, 797) = 766.42
Model | 248488.667 4 62122.1667 Prob > F = 0.0000
Residual | 64600.6458 797 81.0547626 R-squared = 0.7937

+
Adj R-squared = 0.7926
Total | 313089.313 801 390.873049 Root MSE = 9.003

+

--------

fte | Coefficient Legend

+

--------

period#treated |
Feb 1992#PA | 19.94872 _b[0bn.period#0bn.treated]
Feb 1992#NJ | 17.06518 _b[0bn.period#1.treated]
Nov 1992#PA | 17.54221 _b[1.period#0bn.treated]
Nov 1992#NJ | 17.57266 _b[1.period#1.treated]

+

--------

Then use these coefficient names to compute the double-difference and its standard error:

. lincom (_b[1.period#1.treated] - _b[0bn.period#1.treated]) - (_b[1.period#0bn.treated] - _b[0bn.period#0bn.treated])

( 1) 0bn.period#0bn.treated - 0bn.period#1.treated - 1.period#0bn.treated + 1.period#1.treated = 0

+

--------

fte | Coefficient Std. err. t P>|t| [95% con f. interval]

+

--------

(1) | 2.913982 1.610513 1.81 0.071 -.2473667 6.075331

+

--------

With the expression(exp) option:, the margins command produces the same calculations.
(In this case, margins's warnings can be ignored.)

.margins, expression( (_b[1.period#1.treated] - _b[0bn.period#1.treated]) - (_b[1.period#0bn.treated] - _b[0bn.period#0bn.treated]) )

warning: option expression() does not contain option predict() or xb().

warning: prediction constant over observations.

Predictive margins Number of obs = 801
Model VCE: OLS

Expression: (_b[1.period#1.treated] - _b[0bn.period#1.treated]) - (_b[1.period#0bn.treated] - _b[0bn.period#0bn.treated])

+

--------

| Delta-method
| Margin std. err. z P>|z| [95% con f. interval]

+

--------

_cons | 2.913982 1.610513 1.81 0.070 -.2425658 6.070531

+

--------

Alternatively, we can estimate the DID directly as the estimated
coefficient on an interaction term between the group and period indicators.
Using factor variable syntax, we can add the interaction term like this:

. regress fte i.treated##i.period

Source | SS df MS Number of obs = 801

+
F(3, 797) = 2.15
Model | 524.003099 3 174.6677 Prob > F = 0.0919
Residual | 64600.6458 797 81.0547626 R-squared = 0.0080

+
Adj R-squared = 0.0043
Total | 65124.6489 800 81.4058111 Root MSE = 9.003

+

--------

fte | Coefficient Std. err. t P>|t| [95% con f. interval]

+

--------

treated |
NJ | -2.883534 1.134812 -2.54 0.011 -5.111107 -.6559608
|
period |
Nov 1992 | -2.40651 1.446314 -1.66 0.097 -5.245544 .4325237
|
treated#period |
NJ#Nov 1992 | 2.913982 1.610513 1.81 0.071 -.2473667 6.075331
|
_cons | 19.94872 1.019394 19.57 0.000 17.9477 21.94973

+

--------

The above parameterization is the most common approach to DID impact evaluation
seen in the literature. A number of different community-contributed programs
can be used to produce the same simple DID estimate or variations that address
common sources of bias or inconsistency issues that may threaten such a simple estimate.
One such community-contributed program,
Juan Villa's diff, was the inspiration for diffplot.
It's simplest application produces the same numerical results we computed above.

. diff fte, treated(treated) period(period)

DIFFERENCE-IN-DIFFERENCES ESTIMATION RESULTS

Number of observations in the DIFF-IN-DIFF: 801

Before After
Control: 78 77 155
Treated: 326 320 646
404 397

Outcome var. | fte | S. Err. | |t| | P>|t|

+
+
+
+

Before | | | |
Control | 19.949 | | |
Treated | 17.065 | | |
Diff (T-C) | -2.884 | 1.135 | -2.54 | 0.011**
After | | | |
Control | 17.542 | | |
Treated | 17.573 | | |
Diff (T-C) | 0.030 | 1.143 | 0.03 | 0.979
| | | |
Diff-in-Diff | 2.914 | 1.611 | 1.81 | 0.071*

R-square: 0.01
* Means and Standard Errors are estimated by linear regression
**Inference: *** p<0.01; ** p<0.05; * p<0.1

Again
the DID estimate is that the impact of the minimum wage was an increase of 2.914
full-time-equivalent workers rather than a recution in FTEs.
The program diff computes the p-value of this difference as 0.071,
so the null is rejected at the 10% significance level.

Examples

Our first example shows how diffplot visualizes the results of the
Card & Krueger DID estimate of the impact of the minimum wage as discussed
above. Their ordinary-least-squares estimate
of the impact was:

. regress fte i.treated##i.period

<see the above discussion for the regress output>

Now we use diffplot after regress to display a graph
of its results along with the confidence intervals.

. diffplot, period(period) treated(treated) ///
title("DID after -regress- with group-period interaction") ///
saving(__diffplot_ex_0, replace)

Click on the following link to view the graph produced by diffplot:

. graph use __diffplot_ex_0

diffplot displays time on the horizontal axis.
The estimated mean outcome of each group is tracked
by a solid line connecting the observations before and after
the intervention. In this application, the estimated means
of the two groups are coincidentally almost identical
after the intervention. As a result, the post-intervention labels
of the two groups are superimposed.
(To reposition a label, use the graph editor.)

The dashed line displays the counterfactual change in employment
in the New Jersey restaurants. That is, in the absence of the
passage and implementation of the minimum wage law in New Jersey
the assumption of parallel trends in the control and treatment groups
implies that employment would have declined in the New Jersey
restaurants by the same amount that it was observed to decline
in the Pennsylvania restaurants - to 14.7 FTEs.

The double-headed arrow spans the difference between the observed
employment of 17.573 in New Jersey and the estimated counterfactual
employment of 14.7 in those same restaurants in the absence
of the minimum wage laws. The difference is 2.91 with a p-value of 0.0708.
So without controlling for any covariates and using a two-sided hypothesis test,
we reject the null hypothesis at a significance level of 10%.

The following graph shows that diffplot also works after diff
and demonstrates a few diffplot options.

. The options cfactci and {level(90)} show
that the actual employment level in New Jersey
after the intervention was larger than the upper bound
of the counterfactual's 90% confidence interval.
. The option beftxtleft improves legibility by positioning
the labels for the earlier period on the left margin of the plot.
. The option tradtable has no effect on the plot.
Its purpose is to compute and display the traditional DID
results table as an alternative to the table produced by diff.

. diff fte, treated(treated) period(period)

<see the above discussion for the diff output>

. diffplot, period(period) treated(treated) ///
level(90) cfactci beftxtleft tradtable ///
title("DID after -diff-") ///
saving(__diffplot_ex_1, replace)

diffplot determines that results from Juan Villa's diff are currently in memory

Producing the two-by-two traditional DID table using the program frmttable

DID after -diff-

After Before
Nov_1992 Feb_1992 Difference

NJ 17.573 17.065 0.507
PA 17.542 19.949 -2.407*
Difference 0.030 -2.884** 2.914*

* p < .1; ** p < .05; *** p < .01

diffplot's option tradtable displays the above table
with the same information as diff's table, but with
a more traditional DID layout.
Click on the following link to view the graph produced by diffplot after diff:

. graph use __diffplot_ex_1

To control for possible franchise-specific effects,
we include a dummy variable for three of the four chains,
so the omitted dummy identifies the reference level.
Thus estimated levels in the DID table and the graph
are for the omitted restaurant type: Burger King.
The estimated DID impact is unaffected by this choice.

. diff fte, treated(treated) period(period) cov(kfc roys wendys)

DIFFERENCE-IN-DIFFERENCES ESTIMATION RESULTS

Number of observations in the DIFF-IN-DIFF: 801

Before After
Control: 78 77 155
Treated: 326 320 646
404 397

Outcome var. | fte | S. Err. | |t| | P>|t|

+
+
+
+

Before | | | |
Control | 22.078 | | |
Treated | 19.754 | | |
Diff (T-C) | -2.324 | 1.031 | -2.25 | 0.024**
After | | | |
Control | 19.675 | | |
Treated | 20.286 | | |
Diff (T-C) | 0.611 | 1.037 | 0.59 | 0.556
| | | |
Diff-in-Diff | 2.935 | 1.460 | 2.01 | 0.045**

R-square: 0.19
* Means and Standard Errors are estimated by linear regression
**Inference: *** p<0.01; ** p<0.05; * p<0.1

. diffplot, period(period) treated(treated) ///
level(95) cfactci beftxtleft tradtable ///
title("DID after -diff- with franchise fixed effects") ///
ytitle("Estimated full-time employment" "in Burger King restaurants") ///
saving(__diffplot_ex_2, replace)

diffplot determines that results from Juan Villa's diff are currently in memory

Producing the two-by-two traditional DID table using the program frmttable

DID after -diff- with franchise fixed effects

After Before
Nov_1992 Feb_1992 Difference

NJ 17.696 17.163 0.532
PA 17.084 19.487 -2.403*
Difference 0.611 -2.324** 2.935**

* p < .1; ** p < .05; *** p < .01

So after controlling for franchise fixed effects,
the estimated impact on FTEs per restaurant rises slightly
and is more statistically significant.
Click on the following link to view the graph produced by diffplot:

. graph use __diffplot_ex_2

Now show that diffplot produces the same result after regress with covariates:

. regress fte i.treated##i.period kfc roys wendys

Source | SS df MS Number of obs = 801

+
F(6, 794) = 30.61
Model | 12233.2833 6 2038.88054 Prob > F = 0.0000
Residual | 52891.3656 794 66.6138106 R-squared = 0.1878

+
Adj R-squared = 0.1817
Total | 65124.6489 800 81.4058111 Root MSE = 8.1617

+

--------

fte | Coefficient Std. err. t P>|t| [95% con f. interval]

+

--------

treated |
NJ | -2.323906 1.030692 -2.25 0.024 -4.347109 -.300703
|
period |
Nov 1992 | -2.402678 1.311311 -1.83 0.067 -4.976724 .1713688
|
treated#period |
NJ#Nov 1992 | 2.93502 1.460147 2.01 0.045 .0688142 5.801225
|
kfc | -10.12174 .7858127 -12.88 0.000 -11.66425 -8.57922
roys | -1.813925 .7371392 -2.46 0.014 -3.260897 -.3669533
wendys | -.9168795 .8888416 -1.03 0.303 -2.661637 .8278775
_cons | 22.07757 .9825978 22.47 0.000 20.14878 24.00637

+

--------

. diffplot, period(period) treated(treated) ///
level(95) cfactci beftxtleft tradtable ///
title("DID after -regress- with franchise fixed effects") ///
ytitle("Estimated full-time employment" "in Burger King restaurants") ///
saving(__diffplot_ex_3, replace)

diffplot determines that results from a regression containing the interaction term i.treated##i.period are currently in memory

Producing the two-by-two traditional DID table using the program frmttable

DID after -regress- with franchise fixed effects

After Before
Nov_1992 Feb_1992 Difference

NJ 17.696 17.163 0.532
PA 17.084 19.487 -2.403*
Difference 0.611 -2.324** 2.935**

* p < .1; ** p < .05; *** p < .01

Click below to view the graph produced by diffplot:

. graph use __diffplot_ex_3

With the option kernel, the diff program uses the variables specified
in the option cov(varlist) to match treatment and control units on a propensity score.
Adding the option support forces diff to drop 6 observations without common support.

. diff fte, t(treated) p(period) cov(bk kfc roys) kernel id(id2) support report

KERNEL PROPENSITY SCORE MATCHING DIFFERENCE-IN-DIFFERENCES

Estimation on common support

Report - Propensity score estimation with probit command

Atention: _pscore is estimated at baseline

Iteration 0: log likelihood = -198.21978
Iteration 1: log likelihood = -196.7657
Iteration 2: log likelihood = -196.7636

Probit regression Number of obs = 404
LR chi2(3) = 2.91
Prob > chi2 = 0.4053
Log likelihood = -196.7636 Pseudo R2 = 0.0073

+

treated | Coefficient Std. err. z P>|z| [95% conf. interval]

+

bk | .1812529 .2090916 0.87 0.386 -.2285591 .5910649
kfc | .3888298 .246799 1.58 0.115 -.0948873 .8725469
roys | .2997977 .2318227 1.29 0.196 -.1545664 .7541618
_cons | .6476036 .1777446 3.64 0.000 .2992305 .9959767

+

Matching iterations...

..............................................................................................
..............................................................................................
..............................................................................................

DIFFERENCE-IN-DIFFERENCES ESTIMATION RESULTS

Number of observations in the DIFF-IN-DIFF: 795

Before After
Control: 78 76 154
Treated: 326 315 641
404 391

Outcome var. | fte | S. Err. | |t| | P>|t|

+
+
+
+

Before | | | |
Control | 20.040 | | |
Treated | 17.065 | | |
Diff (T-C) | -2.975 | 0.943 | -3.16 | 0.002***
After | | | |
Control | 17.449 | | |
Treated | 17.499 | | |
Diff (T-C) | 0.050 | 0.955 | 0.03 | 0.958
| | | |
Diff-in-Diff | 3.026 | 1.342 | 2.25 | 0.024**

R-square: 0.02
* Means and Standard Errors are estimated by linear regression
**Inference: *** p<0.01; ** p<0.05; * p<0.1

Given diff's default options for kernel type and bandwidth,
six restaurants are determined to lack common support.
Now use diffplot to display the DID impact estimate.
With the option stats(N), the traditional DID table displays
the number of observations in each cell.

. diffplot, period(period) treated(treated) ///
level(95) cfactci beftxtleft tradtable stats(N ) ///
title("DID after -diff- with propensity score matching") ///
saving(__diffplot_ex_4, replace)

diffplot determines that results from Juan Villa's diff are currently in memory
(6 missing values generated)

Producing the two-by-two traditional DID table using the program frmttable

DID after -diff- with propensity score matching

After Before
Nov_1992 Feb_1992 Difference

Treated 17.499 17.065 0.434
NJ (315) (326) (641)
Control 17.449 20.040 -2.592***
PA (76) (78) (154)
Difference 0.050 -2.975*** 3.026**
(391) (404) (795)

* p < .1; ** p < .05; *** p < .01
N displayed below estimated differences

Click on the following link to view the graph produced by diffplot:

. graph use __diffplot_ex_4

Dependencies

Using diffplot after executing Juan Villa's diff requires the user
to have already installed the following programs:

. diff (Villa, 2016)

. estout (Jann, 2005, 2023)

. erepost (Jann, 2015)

Options tradtable and difftable require Luke Gallup's program:

. frmttable (Gallup, 2012)

References

Abadie, A., G. W. Imbens (March, 2016)
"Matching on the estimated propensity score",
Econometrica (March, 2016), Vol. 84, Issue 2
NBER (2009)

Abadie, A. (2005) "Semiparametric Difference-in-Differences Estimators".
Review of Economic Studies 72: 1--19

Angrist, J.D., J-S. Pischke (2009)
Mostly Harmless Econometrics,
Princeton University Press

Baker, A., D.F. Larcker, C.C.Y. Wang (March 29, 2021)
"How Much Should We Trust Staggered Difference-In-Differences Estimates?",
Blog accessed 16April2021.

Baker, A., D.F. Larcker, C.C.Y. Wang (March 29, 2021)
"How Much Should We Trust Staggered Difference-In-Differences Estimates?",
European Corporate Governance Institute – Finance Working Paper No. 736/2021,
Rock Center for Corporate Governance at Stanford University
Working Paper No. 246, Available at SSRN

Card, D., Krueger, A. (Sep, 1994)
"Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania".
The American Economic Review, Vol. 84, No. 4, pp. 772-793.

Gallup, Luke (2012)
A programmer's command to build formatted statistical tables,
Stata Journal, Vol.12, No. 4, pp. 655-673

Gertler, P, S. Martinez, P. Premand, L.B. Rawlings, C.M.J. Vermeersch,
Impact Evaluation in Practice, Second edition. (2016) Inter-American
Development Bank and World Bank, Free download
here.

Hegland, Thomas A. "wooldid: Estimation of Difference-in-Differences Treatment Effects with Staggered Treatment Onset Using Heterogeneity-Robust Two-Way Fixed Effects Regressions".
Statistical Software Components s459238 (2023)

Houngbedji, K. (2016).
"Abadie's Semiparametric Difference-in-differences Estimator [absdid]".
The Stata Journal, 16(2), 482–490.

Jann, Ben (2005)
Making regression tables from stored estimates,
Stata Journal, Vol. 5, No. 3, pp. 288-308

Jann, Ben (2023)
estout: Making regression tables in Stata,
Jann's website

Schechter, and others,
"Diff-in-diff approach when implementation of policy is `staggered' (2018-)
Discussion topic on Statalist.
Also using see the topics
found
by searching on the three search terms "diff", "in" and "diff" (without the hyphens).

Social Sciences Computing Center (2015)
"Propensity Score Matching in Stata using teffects (2018-)
Blog from University of Wisconsin-Madison.

Villa, J. M. (2016)
"diff: Simplifying the estimation of difference-in-differences treatment effects",
The Stata Journal. 16, Number 1, pp. 52–71.

Wikipedia, "Difference-in-Differences"
Accessed 25Oct, 2020

Wing, C., K. Simon, R.A. Bello-Gomez (2018)
"Designing Difference in Difference Studies: Best Practices for Public Health Policy Research",
Annual Review of Public Health, 2018.39:453-469.
Princeton University Press