*! ver. 2.61 11Mar2024 by Mead Over, Center for Global Development: Section on programming under construction

help for Mead's FavoritesVersion 2.61 11Mar2024

Introductions to Stata

The best way to learn Stata is to take Stata's own web-based course
NC101.
Stata has a compendium of other resources on their web site
here

For beginners, some of the best web resources are the following tutorials:

From Harvard here.

From Princeton here.

From the London School of economics
here.

From UCLA
here.

From Wolfgang Ludwig-Mayerhofer
here.

From Germán Rodríguez
here.

From Boston College
"Introduction to Stata"

From the Univeristy of Pennsylvania
here.

From Stanford
here

Once you have started writing and de-bugging DO files, you might find useful the utility
do2screen.

Chris Baum of Boston College has published a manual
on Stata programming available from Amazon or from Stata's online bookstore
here.
In addition to his introductory
157-slide set,
he has also posted
more advanced pedagogical PPTs on Stata programming listed below:

Introduction to Programming in Stata

Maximum Likelihood Estimation and Nonlinear Least Squares

Stata's Matrix Programming Language, Mata

Monte Carlo Simulation in Stata

Dynamic Panel Data Estimators in Stata

ADO-file programming in Stata (2015)

Replicability: Documenting and logging your results

One of the strengths of the Stata programming package is "replicability",
also sometimes known as "reproducibility". An essential component of
a strategy to assure reproducibility is the log file.
I have observed that many programmers write beautifully clear DO files
to document the process they used to arrive at research results,
but omit to make or publish the log file that would record how
Stata processed that DO file at the time it was executed.

Perhaps users would be more likely to make and distribute log files if log files
were easier to read. The utility lineno2 adds line numbers
to a Stata log file, whether it is in smcl or log format.
A PDF option is a shortcut to a PDF version of the SMCL file with line numbers.
With line nmbers added, researchers on a Zoom call together can more easily
find the code or results to discuss.

For a more general discussion of the importance of reproducibility, see this Stata blog by
Enrique Pinzon.

More recently the Harvard Data Science Review has published an open article by Lars Vilhuber on
"Reproducibility and Replicability in Economics" available
here
and in PowerPoint
here.

Stata gives an overview of its new and old capabilities for assuring reproducibility and automated reporting:
here.

Many user-contributed Stata programs are designed to help users beautify Stata output and results.
A useful tool for adding HTML or LaTex markup to a log file is log2markup,
by Niels Henrik Bruun. Also see the links in this help file under Table Making Utilities
and Workflow organization, reproducibility and code dependency.

If you are using the replace command to change selected values of a variable,
perhaps correcting errors in the data or specifying a specific value for a specific observation,
the danger exists that you will later forget exactly what the value used to be
and how you have changed it. The command repshow is an enhancement of the replace command
which will document in your results window, and thus in your log file, the value of
observations before and after they are replaced.

If you are in too much of a rush to label variables you are creating, the
ancient user-written command genl will do it for you, thus helping
you document your logic. Or try the more recent and versatile labgen2,
which is part of the download package labutil2. Fernando Rios-Avila
includes his own version of the same program as fgen in his 2021
utility f_able.

Transformations, functions & expressions

Suppose you would like to explore various transformations of variables
x, y and z and the relationships among these transformed
variables. The obvious approach is to use Stata's generate
command to generate the various transformations and then explore the
transformed variables using various graphics or descriptive statistic
commands. The result is a dataset littered with all of these transformed
variables. An alternative is to use the clever program(s) written by
Jeroen Weesie entitled expr and exprcmd.

Beginning in Stata 11, Stata's factor variables provide another alternative
which works for powers of a variable and for interaction variables. Thus
instead of generating x^2, x^3 or x*y, one can instead specify them as
c.x#c.x, c.x#c.x#c.x or as c.x#c.y. By using two #'s instead of one like this:
c.x##c.y, one can now specify the individual x and y variables as well as the product
between them.

See fvvarlist for documentation and examples. For programing purposes,
the commmand fvrevar converts a factor variable expression into a
variable list that can be used by commands that do not otherwise support factor variables or
time-series-operated variables. This
FAQ
from Stata Corp. shows how to use
factor variable syntax in your own programs.

An important advantage of using Stata's factor variables to represent non-linear functions
of variables on the right-hand-side of estimation commands is that Stata's margins
is able to compute the proper impacts or derivatives of a right-hand-side variable
that appears in more than one factor-variable term. So to regress y
on a quadratic function of x and then estimate it's derivative at its mean value,
one need only type:

regress y c.x##c.x
margins , dydx(x)

However factor variables are unable to represent transformations
other than simple polynomials. To compute margins of variables transformed
in ways which cannot be represented by factor variables, see Fernando Rios's
f_able which is documented in his 2021 SJ article.
"Estimation of marginal effects for models with alternative variable transformations".
His f_able program makes clever use of Stata's _ms_dydx_parse
command to augment the estimated coefficient and variance-covariance matrices
returned by the estimation command so that they can be used by margins.

Stata's built in functions are described in functions. But the menu of
Stata's built-in transformations is substantially expanded by a large set
of "egen" commands. The user community has added to Stata's built-in
egen commands. Perhaps the two largest packages of user-written egen
commands are egenodd and egenmore, by Nick Cox and others and updated in 2019.
Since few "egen" commands have a replace option,
the user-written ereplace may come in handy.

Those working with DHS data (from the Demographic and Health Survey) will appreciate
Ian Timaeus' utility,
//raw.githubusercontent.com/bugbunny/convertCMC/master")"':convertCMC,
which converts so-called CMC (Century Month Calendar) dates to Stata encoded dates.
An "egen" command to compute poverty measures is
//fmwww.bc.edu/RePEc/bocode/e")"':egen_inequal.
Accelerated versions of some egen commands are available in the packages
ftools and gtools.
See Kit Baum's PPT on
ADO-file programming in Stata
for an introduction into writing your own egen utility.

While Stata's estimation commands are brilliant at allowing the user to
specify weights, Stata's approach to the task of constructing a weighted
average is to use the command ameans, which approaches the task
as an estimation problem, designed to produce a scalar result.
Alternatively, for simple weighted means, one can use collapse,
but that approach has the disadvantage of replacing the data in memory with the collapsed data.
To construct a variable defined as the weighted or unweighted arithmetic,
geometric or harmonic mean by a grouping variable, it is often more convenient
to use the new egen command, wmean, by Gueorgui I. Kolev.
See the Statalist discussions
here
and
here.

Economists frequently analyze highly skewed data. Variables that are typically positively skewed
(having distributions with long right tails) include those measuring expenditure, consumption, production, firm size, wealth, etc..
The analysis of skewed variables takes a different path depending on whether the variable in question
has any zero or negative variables.

Suppose the positively skewed variable is strictly positive, (i.e. has no negative or zero values).
For this case, the decades old solution is to transform the variable(s) using a logarithmic transformation.
A good discussion on the web is
here.

The Stata web site has a good introduction to how to proceed with the transformation
and, more importantly, how to interpret the results.
In the spotlight: Interpreting models for log-transformed outcomes.
However, the website is incorrect in describing their approach as appropriate for non-negative variables.
It is only reliable for strictly positive variables.
Or in situations where discarding the observations with zero values is defensible.

For variables that are non-negative and positively skewed,
using the logarithmic transform is problematic, because the logarithm of zero is undefined.
Thus for non-negative positively skewed variables the choice to use a log-transform
requires either discarding the observations with zeros from the sample,
or explicitly modeling the zeros using some kind of selection or two-part model.
(Stata's official and community contributed programs give many options,
including heckman, heckprobit, heckpoisson, heckoprobit, twopm, cmp, etc. .)

Another approach to non-negative positively skewed variables
which might be definsible in some situations is to simply add a positive constant
to a non-negative variable before performing the log transformation.
For example, my program superscatter has an option log10plusone which adds a one to variables
before taking their logarithm and graphing them
so that the user can see the contribution of the zero values to the marginal distributions.
A more robust, approach is to use a concave transformation that, unlike the logarithm, does accommodate zeros.
Two possibilities are the square root and the hyperbolic tangent tanh().
superscatter offers the option sqrt which applies the sqrt() transformation
as an alternative to the options log10plusone and log10plusone.
(None of these options disturbs the user's data. They only affect the graph.)

Austin Nichols'
presentation
at a 2010 Stata conference proposes less ad hoc solutions
based on the idea that the statistical model should attempt to capture the
data generating process (DGP). He discusses the use of Poisson regression
which does not reuqire that the non-negative dependent variable be transformed.

If the variable is positively skewed, but has negative and zero values, as might be true for example of household wealth,
a sinusoidal transformation like logit() might help reduce heteroskedasticity and improve the model.

Missing data

See the October, 2008 FAQ from UCLA Academic Technology Services entitled
"How can I see the number of missing values & patterns in my data file?".
Also see Cox's missings and missingplot.

For panel data, the following command may be useful for finding "holes"
in the data and filling them with imputed values: findholes

The command regmsng allows the user to set a tolerance, nmiss(#),
for missing values for the right-hand-side variables in an ordinary linear regression.
For example, with nmiss(2), variables with 1 or 2 missing values are retained,
while those with 3 or more missing values are dropped.
The missing values of retained variables are imputed with the user's choice of
several imputation techniques. The standard errors are not adjusted for the
imputaton.

More rigorous approaches to the imputation of missing values are available in
hotdeck, ice and mim. This 2007 FAQ on imputing values
in a panel data set provides an introduction to mim
http://www.ats.ucla.edu/stat/stata/faq/mi_longitudinal.htm

Beginning with Stata 11, Stata is providing a suite of programs dedicated to multiple
imputation of missing values. See mi

Comparing data files and variables

Like Stata's compare, compare2 reports the differences and similarities
between two variables with different names located in the same dataset.
However, unlike compare, compare2 also returns stored results in Stata's return space
for subsequent use by the programmer. With the added reldif option,
compare2 presents the summary statistics of the relative difference
between the two variables as computed by the Stata function reldif.
See help for reldif. For ease of use, compare2 has a companion dialog.

STATA's cf command is a powerful tool for comparing
variables in a "master" data set in memory to
identically named variables in a saved data set on disk.
But cf fails when the two data sets have
different numbers of observations
or when the only difference between two data sets is the way they are sorted.
cf2 is a wrapper for Stata's cf which first sorts
the two datasets and then compares identically named variables
on only those observations that match according to the sorting variables.
For ease of use, cf2 has a companion dialog.

The commands cf and cf2 report mismatches between variables
with the same name located in different data sets.
When the variables being compared are numerical and are both in the current data set,
compare or compare2 provides a more complete analysis of differences.
To obtain the more detailed comparison in the style of compare
for variables with the same names located in different datasets,
try the program compuse which also pops up a scatter plot of one version
of each variable against the version in the other dataset.

Both cf2 and compuse have the required option sortvars(varlist)
to specify the variable or variables which
uniquely sort the two compared datasets.
Stata refers to such a set of variables as an "ID" variable,
while others refer to them as "key" variables.
Prior to executing either cf2 or compuse, the user should confirm
that proposed sort variables do indeed uniquely identify the observations in both data
sets. Stata's isid serves this purpose for both the master and, with the
using option, for the -using- dataset. Also see dta_equal
and the community contributed commands assertky and findunique.

Matrix utilities

Two kinds of matrices are available in Stata. From time immemorial, Stata has its own
matrices and matrix utilities. Since version 11 of Stata, the program
includes a high-powered, super fast, compiled, matrix-based programming language called
Mata.

While a Stata matrix can be read by a Mata program and vice-versa, the two types of matrices
are quite distinct. For example, the elements of a Stata matrix can only be numeric,
while a Mata matrix can consist either entirely of numeric or entirely of string elements.
A second distinction is that the rows and columns of a Stata matrix have
alphanumeric (i.e. string) names and one can reference an element of a Stata matrix
by those row and column names.
(The matrices produced by Stata's estimation packages have row and column names called "matrix stripes".
Manipulation of matrix stripes is facilitated by a set of undocumented Stata commands
that begin with the prefix _ms. See help undocumented.)

On the other hand, the rows and columns of a Mata matrix
only have numeric indices, not alphanumeric names and therefore must be referenced
by those numeric row and column indices. For a work-around see this
post
by Jeff Pitblado on Statalist.

For Stata matrices, members of the Stata community have contributed programs that
complement the official Stata matrix utilities. A few of these collections are:
Jerome Weesie's matfunc and varfunc collections,
Nick Cox's collections from STB-50 and from STB-56,
Paul Millar's matsort program, Neils Bruun's matrixtools
and the utilities matout4 and Chris Baum and Bill Gould's
matin4 for saving a Stata matrix to file and then reading it back into Stata.

While matrixtools works with Stata matrices
using a Mata library, Ben Jann's moremata adds additional utilities
for manipulating Mata matrices.

In Stata it is sometimes useful to use a matrix as a lookup table.
See Cox's
Matrices as look-up tables
for a general discussion and tips.
To assist with extracting an item or multiple items from either a Stata or a Mata matrix,
try my matrix lookup programs mlu and mluwild.

Table making utilities

Beginning with version 17, Stata has augmented their existing special purpose
table making commands such as estimates table
with an impressive set of table-making capabilities described
here.
For those without access to Stata 17, the pre-existing range of free community-contributed
programs described below provides wide functionality at zero cost.

Also consider Stata's tabdisp and table commands. The former
is for making simple tables of individual data values, where no computations are required.
tabdisp has the power to format information into "supercolumns" composed of
individual columns or "superrows" of individual rows. table is a newer command which
inherits the "superrow" and "supercolumn" capability, but also performs computations. Scroll
down to the bottom of table's help file for a link to a video showing how it works.

Most table-making commnads in Stata display results for categorical variables,
naming the rows and columns with STATA's value labels. Utilities for
documenting and extracting these category labels are discussed
below.

STATA's tabstat command is useful and flexible. It's two most glaring deficiencies
are its unattractive column labels (which consist only of the variable name or statistic name)
and the fact that its format option can produce column headings that are not
aligned with the columns of statistics. My hack of STATA's tabstat, tabstat2,
addresses these two problems. First, it produces more attractive output when the format
option is selected by aligning the columns and the variable names. And tabstat2 also
has two additional options, noheader and describe which optionally suppress the standard
heading and add a listing of the variable names keyed to their variable labels. Sorry, no help file yet.

Ian Watson has contributed a powerful general purpose table-making command called
tabout, which can be installed from SSC using the command:
ssc install tabout, replace. In October, 2022 a newer version
is available in beta from his web site
here.

To create tables of estimation results, the basic procedure has been to
collect the estimates using the estimates store command
and then display them in a table using estimates table.
In version 17, Stata has introduced the new command etable,
which makes Stata's estimates table command essentially obsolete.
It seems to me that etable also renders obsolete all
the community contributed programs in this class, including those I discuss
in the remaining paragraphs in this section.
Stata's Jeff Pitlabo provided an easy-to-follow introduction to etable in a
PowerPoint
delivered at the mid-2021 Stata Users Group Meeting in Europe.

For years multiple community members have created programs that provide much more user-control
of the structure, content and formatting of tables of estimation results
than were available in estimates table.
One of the first such commands is outreg by Luke Gallup
STB 46, Nov. 1998, pp. 28.
In 2012, Gallup completely rewrote the program in Mata, now available
//www.stata-journal.com/software/sj12-4")"':here.
Using its companion utility program frmttable,
outreg produces tables for either MS Word or Tex, but not in Excel format.
For that purpose, available user-written commands include
xml_tab by Misha Lokshin, of the World Bank,
outreg2 by Roy Wada and esttab by Ben Jann,
who gives examples on the Excel page of his
web site.

By invoking esttab's using myfilename.rtf option, the user can
export a table to MS Word. The MS Word page of Jann's extensive companion
web site
suggests how the user can further modify esttab's RTF output with
RTF formatting commands.
Note that, although both esttab and estout program have using filename options,
only esttab will produce a properly formatted RTF, CSV or TeX file.
To customize the font or page orientation of the RTF file produced by esttab,
Ben Jann recommends (personal communication)
using the substitute() option
to add approriate commands to the RTF code at the start of the document.
Since esttab starts its RTF documents with the code string "\deflang1033\plain\fs24",
this is a good place to add additional formatting instructions.
An example of a substitution that accommodates a wide table in landscape orientation on US-style A4 printer paper
is to use estout's option, modelwidth(#), where # might be 8 or even smaller, together with
the option substitute() specified as follows:

. esttab using text.rtf, substitute("\deflang1033\plain\fs24" "\landscape\paperw15840\paperh12240\deflang1033\plain\fs24")

To print onto European-style A5 paper, the substitute() option would instead be:

. esttab using text.rtf, substitute("\deflang1033\plain\fs24" "\landscape\paperw16834\paperh11909\deflang1033\plain\fs24")

To also reduce the font sizes used in the RTF document, change
the font size twice for both table contants and table notes.
For the content, change the font size from 12-point to 10-point by substituting "fs20" for "fs24".
Then change the font size in the notes by substituting "fs16" for "fs20" as follows:

. esttab using text.rtf, substitute("\deflang1033\plain\fs24" "\landscape\paperw15840\paperh12240\deflang1033\plain\fs20" "\pard\ql\fs20" "\pard\ql\fs16" )

To also change the margins, add to the landscape specification as follows:

. esttab using text.rtf, substitute("\deflang1033\plain\fs24" "\landscape\paperw15840\paperh12240\margl1440\margr1440\margt1800\margb1800\deflang1033\plain\fs20" )

While properly formatted tables of coefficients are the primary means of communicating
esimated regression coefficients, plotting those coefficients and their confidence
intervals helps your audience quickly grasp your results. Ben Jann's command coefplot
is an extraoridinaarliy useful program to plot coefficients and their confidence intervals.
This 2017 PPT by Dawn Koffman at Princeton's Office of Population Research gives a superb
introduction to using -coefplot-.
Another way to access and manipulate regression coefficients is available in Roger Newson's parmest
program, described below.

A standard component of many journal articles is a table of descriptive statistics.
Such a table can usefully include not only the descriptive statistics on each independent variable,
but also the bivariate correlation coefficient between each independent variable and the dependent
variable. The program bivariate makes a table of correlation coefficients and variance
inflation factors. Adding the tabstat option to bivariate directs it to
also produce a table of means and standard deviations produced by Stata's tabstat
command on the observations that would be included in the multiple regression.

For an interesting approach to visualizing a table with a Stata graph, see the discussion of
tabplot below.

Version 15 of Stata introduced official utilities to export directly to MS Word or MS Excel.
See Stata's documentation on putdocx and putpdf.
Other recently contributed, user-written table-making commands include sumtable,
table1, table1_mc, partchart.
A recent ambitious entry in this category is asdoc by Attaullah Shah.
At this writing (18Aug2023), asdoc competes neck-and-neck with Ben Jann's
estout and Roy Wada's outreg2. To check on this competition,
type ssc whatshot, n(15) at Stata's command prompt.
While Jann's programs export in the non-proprietary RTF and CSV formats,
which are compatible respectively with MS Word and MS Excel, Shah's asdoc
exports directly to MS Word's .doc format, but does not export easily to Excel.
Shah has now released a commercial aftermarket version of his program called asdocx
which is sold on a dedicated
web site.

Graphing hints and favorites

The overall look of a Stata graph is controlled by the "scheme" the user employs.
See schemes for an introduction and scheme_option for the syntax
to alter the scheme of any graph command. From inside your Stata,
type graph query, schemes to see a list of the schemes installed on your
instance of Stata.

The prolific Stata program contributor, Ben Jann, has written a powerful
commmand to facilitate the user's customization of their own personal
(or institutional) Stata graphics style. This program, called grstyle
is documented in two Stata Journal articles
here (gated until 2021) and
here (gated until 2022) and
explained in a Powerpoint presentation posted
here
(ungated). He has also published companion programs palettes,
colorpalette, symbolpalette and line palette.
His Stata Journal article entitled
Color palettes for Stata graphics
(gated until 2022) is the best available guide to user accessible portfolios of palettes.

In his article, "Color palettes for Stata grphics", Jann reviews many different schemes.
The scheme lean from the Stata Journal conserves on unnecessary "ink".
Two user-written branded color schemes with interesting color palettes are the
MRC scheme and the TFL scheme.
Color schemes popular at CGD include Stata's built-in scheme,
s1rcolor, which uses a black background and
s2clr_on_white, produced here at CGD.
As its name implies, the s2clr_on_white scheme
replaces the light-blue backgrounds with white backgrounds. Install
s2clr_on_white from CGD's Stata repository
here.
New abstemious schemes to consider are: here.

Stata's graph command is powerful but a user often struggles to create
a graph with the desired appearance and labeling. It's often helpful to construct
an initial version of the desired graph interactively using the dropdown dialogue
box under the graphs menu. When a dialogue box is "submit"ted, the code to generate the
same result is typed into Stata's results window. By creating a do file,
and saving this code into that file, the user has the basics of the desired graph.
Then by elaborating on the saved code, saving and executing the DO file, and then
altering it again, the user can achieve virtually any desired graph.

The box-and-whisker graph constructed by graph_box is a powerful descriptive tool.
However, STATA does not make it easy to learn the numeric values of the "adjacent values"
at which the command constructs the "whiskers". The command sumadj is the answer
and also offers a log scale option.

If the range of a variable you are graphing is in the millions or billions,
the graph will be ugly unless you rescale the variable. Of course you can -gen- a new
variable for your graph. An alternative is to use the rescale command,
which creates temporary versions of your variables which have been rescaled
by a power of ten. Instead of rescaling the variables, one can simply rescale
the variable labels. Utilities to facilitate the construction of "nice" axis labels
include nicelabels and mylabel.
Nick Cox's mylabels can add the prefix "$" or the suffix "%" to the axis labels.
To label the axis of a log-scaled variable consider the program niceloglabels.

Jeroem Weesie's utility graphf assembles the predicted score from
the variables in a regression which all vary together, such as x, x^2 and x^3
or the various variables of which a spline is composed. My version graphf2
additionally returns the functions used to construct the score so that the user
can construct it him/herself. Stata's official margins command and the accompanying
marginsplot command subsume some of this functionality for non-linear functions
expressed in Stata's factor_variables syntax.

There are two possible approaches to constructing a graph with multiple panels.
The most efficient way is to use a single graph command, adding the by option.
But not all graphs will accept a by option. And sometimes, even for commands
that have a by option, a multiple-step alternative is easier:
execute and save a separate graph command for each panel and then assemble them into a
single multi-panel image using the command graph combine.

A multi-panel graph is more legible if all panels share the same legend.
When the multiple panels are constructed with a single graph command and
the by option, controlling the placement and contents of the legend is tricky,
requiring that some aspects of the legend be specified inside the by option
and other aspects outside the by option. See the help file discussion
here.

When a multi-panel graph is constructed by using graph combine to assemble
several saved graphs into a single image, a useful utility for defining
a single legend is the utility grc1leg written by Vince Wiggins of Stata Corp,
which can be obtained
here.
But also see my own improved version, grc1leg2.

To graph an arbitrary function in STATA, without necessarily having data, try the following command:

twoway (function y=log(x)*sin(x)) (function z=x*cos(x))

See twoway_function or page 49ff of
Chris Baum's 2011 Power Point

Isoclines and level sets of various sorts, such as isoquants, production possibility frontiers,
isocost curves or temperature or altitude contours, can be plotted
in recent versions of Stata using twoway contour.
A contour mapping program that works in version 10.2 of Stata is
plotmatrix written by Adrian Mander. Get his most recent version from
plotmatrix.
Sergiy Radyakin of the World Bank has also written a contour plotting program called
matrixplot which is described in this Powerpoint presentation
"Implementing custom graphics in Stata"
and demonstrated in the figures and videos available
in a zip file: dc09_radyakin.zip.
Unfortunately Sergiy never distributed the command itself.
A different approach is available with my isoplot.

A nice utility for constructing histograms with an overlaid normal distribution is historaj.
My utility
hist_overlay
overlays one histogram with a differently shaded second histogram so that both
histograms can be distinguished. (Overlaying more than two histograms will rarely be useful.)

To display all, or a subset of, the graphs in memory or in the current folder, try displaygph.

Stata's example of how to use their graph combine command is a
scatter plot enhanced by the addition of marginal distributions to the x- and
y axes. It's gorgeous to look at but tedious to program. My utility
superscatter automates this process and provides additional options
and enhancements. Find it
here.

Suppose you need to make a graph that compares two rankings of the same set of
objects. Nick Cox's hints in his 2009 SJ article
"Paired, parallel, or profile plots for changes, correlations, and other comparisons"
are just what you need.
My program
rankplot
offers an enhanced implementation of one of his algorithms.

The visualization known as a "chord plot" seems to be gaining popularity.
Prefiguring this interest was a
discussion
on the Statlist back in 2014. In the last 2014 comment on that thread,
Joe Canner promised a command called
polar
which is now available at SSC. Soon thereafter a discussion on
Stack Exchange
suggests that the chord diagram is a member of a
class of non-linear flow diagrams called "Kriskograms"
and asks whether there are linear equivalents to chord diagrams.
On the latter question, a user cites the 2009 SJ article by Nick Cox
op cit.
(See also rankplot.)
Asjad Naqvi on the platform
Medium
presents a series of Stata tutorials. I particularly like his
tutorial
on using Stata's graph engine with polar coordinates.

Stata has facilitated adding special characters to graph text such as titles,
marker symbols, etc. For example, the "degree sign" can be inserted in a graph legend
as "45{&degree} line" and appears like this "45° line" and the symbol
denoting "function of" can be inserted like this "{&function}{it:(x)}"
and appears like this "(x)". Many symbols are available including
all the small and capital Greek letters and letters from non-Roman alphabets.
Nick Cox's tip from 2004 is still useful and can be found
here (ungated).
More recently, Stata has added a help file explicitly for this purpose. See
help graph_text. For background, see help smcl.

When working with two categorical variables, a scatter plot does not accurately
convey their joint distribution. scatter's option jitter can help.
A more useful solution is Nick Cox's program
tabplot which is available from the Stata journal repository
here
and from SSC here.
Also see his 2004 article on graphing categorical data
"Speaking Stata: Graphing categorical and compositional data".

The "box-and-whisker" plot is a useful non-parametric graph of a distribution.
See graph box for details and for the definitions of the key elements
of the graph including the "adjacent values". Unfortunately, Stata does not
include a command to compute the numerical values of the adjacent values.
I have written the command sumadj to add those computations to the
usual output of stata's summarize, detail command. With the
graph option, the command will also draw the box-and-whisker plot
and label the adjacent values. To depict a highly skewed distribution,
such as those typical of expenditure data,
add the option ylog which rescales the
y-axis to a log scale, an option that is not available in current version of
Stata's graph box command.
(A more general approach to depicting non-symmetric distributions is embodied
in Ben Jann's command robbox (for "robust box").)
Other utilities for calculating "adjacent values" include Nick Cox's
(help adjacent) and the sub-commands adjl(varname) and adju(varname)
in the package of additional egen commands egenmore.

Stata graphs can be animated! See Chuck Huber's 2014 blog
"How to create animated graphics using Stata".
and Di Liu's blog
"How to create animated graphics to illustrate spatial spillover effects".
Here's a 2019 blog on creating a GIF file from Stata graphs by Laura Whiting.
"Visualisation - Creating a Moving Stata Graphic".

Many publishers insist on reconstructing an author's graphs from the original data. They ask the author to provide the data.
For simple graphs this is not a problem. But some Stata graphics commands such as twoway qfit create their own "data".
In this and other situations it is useful to be able to export the data that resides in a Stata gph file file to a spreadsheet program.
Two programs that perform some of the work towards this end are my own gph2xl and Ben Daniels' gph2csv.
gph2csv can be installed from GitHub by clicking
here.

Survival analysis

See Stephen Jenkin's course material at:
https://www.iser.essex.ac.uk/resources/survival-analysis-with-stata

In addition to the extensive set of training materials listed when you search survival,
see Coviello et al's 2015 SJ article,
"Estimating net survival using a life-table approach",
which documents the command stnet.
The authors have updated their command several times, most recently in 2020.
Also check out Hills et al's 2014 article
"strel2: A command for estimating excess hazard and relative survival in large population-based studies".
Their command,
//www.stata-journal.com/software/sj14-1")"':strel2,
can be downloaded from Stata Journal's software site,
but does not weem to have been updated since 2014.

Installing CGD programs in STATA

Since May, 2009, it is possible for anyone with a copy of Stata
anywhere in the world to install some CGD programs directly
from CGD's server. Click here

Or in the command window of Stata, type:

view net from "http://digital.cgdev.org/doc/stata"

or in the viewer just type:

net from "http://digital.cgdev.org/doc/stata"

and follow directions. The advantage of installing utilities using STATA's net approach
is that you can automatically look for updates to all your installed utilities
by just typing ado update. (In Stata 15 or earlier, adoupdate.)

Programs or packages on CGD's external site can be found by typing:

findit program_name

or

search program_name

Stata help on maintaining this system can be found at: usersite

Directory tools by Ulrich Kohler and Nick Winter

Dirtools is a package of directory navigation tools for use inside Stata.
To install, type -ssc install dirtools-.
Once installed, type dirtools for an index to the different programs.
Some people may prefer to navigate using Nick Winter's package -fastcd-,
which can be installed by typing -ssc install fastcd-.
The package -dirtools- is intended to be compatible and to build on the package
-fastcd-, but in my experience the two packages are not compatible
in more recent versions of Stata. Choose one or the other.

Sample size & power calculation for simple and complex sample designs

Beginning with version 13, Stata has an entire manual
called PSS, which is devoted to computing
statistical power and designing samples.

In addition several users have written sample design and power calculation programs
for more complex situations. For example, nstage is a user-written suite
of programs for multi-arm, multi-stage sample designs. Originally written in 2009,
Patrick Royston and coauthors have posted an updated version which you can
download
here

A new user-written menu-driven package for cluster designs is clustersampsi,
which has been updated in 2014 on Stata's site. Install the updated version
here

Stata has also made it easy for users to write their own special purpose power commands
as is documented in the PSS manual and in this
help file: power_userwritten.

The old command powerreg and the 2011 command sampsi_reg extend Stata's sampsi analysis to
regression models. The 2014 commands by Aberson go further see
Aberson (2014), while those by
Batistatou, Roberts and Roberts (2014)
handle cluster designs.

Computing and plotting predicted values from a nonlinear regression

Stata 11's new factor variables are described under the heading "Transformations & Expressions" above.
Using the factor variable notation allows one to perform a regression on a cubic polynomial as follows:

sysuse auto, clear

gen kprice = price/1000

lab var kprice "Price of car in thousands of dollars"

regress mpg c.kprice##c.kprice##c.kprice i.foreign i.rep78 turn trunk headroom

where mpg is defined as the car's mileage rate in miles-per-gallon and kprice is the selling price.

Now suppose we want to compute and graph the fitted value of mpg as a function of kprice,
holding constant the other right-hand-side variables at their means (or at some other user-selected values).
Knowing that price varies in the data from 3 to 15 thousand dollars,
we can use Stata 11's margins command to compute the values of the predicted value
at each integer value of kprice from 3 to 15.

margins, at(kprice=(3/15)) vsquish post

Then following the advice of Jeff Pitblado of StataCorp (see http://www.stata.com/news/statanews.25.3.pdf ),
we can use Roger Newson's parmest command to generate a new data set
of the predicted values.

parmest, norestore evec(at) rename(ev_1 kprice)

(N.B.: By adding the evec(at) rename(ev_1 kprice) options, one can
avoid re-generating the kprice variable.) Now we can graph the
predicted values against -kprice- like this:

twoway rarea max95 min95 kprice, pstyle(ci) || line estimate kprice

While the parmest command remains useful as a general tool to extract coefficients estimates,
standard errors and confidence intervals, Ben Jann's program coefplot,
discussed in greater detail above, is more powerful if one's objective is to
plot the coefficients themselves.

Patrick Royston has since contributed to the Stata Journal
issue 13:3
an elegant command which automates the above process and
has the advantage of leaving the data intact. His command
marginscontplot2,
which is the 2018 update of his orignal command
marginscontplot,
can be used to produce the same graph
as the above code using parmest by immedately following
the above regression command with the commmands:

sysuse auto, clear

gen kprice = price/1000

lab var kprice "Price of car in thousands of dollars"

regress mpg c.kprice##c.kprice##c.kprice i.foreign i.rep78 turn trunk headroom

marginscontplot2 kprice, ci

Additional documentation on the use of marginscontplot
and marginscontplot2, especially in combination with mkspline,
is available from Richard Williams (2021)
here.

A different and sometimes more useful approach to estimating the predictd values from estimation results
with non-linear terms in the right-hand-side variables is presented in
Rios-Avila's f_able command which is discussed in greater detail
above under the section of this help file on
Transformations, functions & expressions.

Working with categorical variables & value labels

Some variables are measured continuously, such as age GDP or temperature. Continuous
variables are natural to Stata. Stata calls all such variables "numeric".
Stata's basic statistical commmands like summarize or regress
work naturally with numeric variables.

Other variables are categorical, such as sex or country. Categorical variables
are most naturally represented by text such as "Male"/"Female" or
"Denmark"/"Chile"/"Botswana"/"Vietnam"/... .
For non-numeric variables, Stata has a type of variable called a
"string variable". If you attempt to apply a numerical statistical operation
such as summarize or regress to a string variable, Stata will
complain that you have "no observations".

While Stata can do many things with string variables, sometimes it needs a numerical
variable in order to proceed. And sometimes naturally categorical variables like "country"
or "religion" are stored as integers in a survey data repository. In these cases,
it is essential to know how to convert a string variable to a numerical code
and then to attach a "value label" to each numerical code so generated.
Furthermore, when importing data from another Statistical package,
it's often necessary or desirable to rationalize the value labels so that
Stata can best use and display them.

Stata has a set of utilities to address these issues. See for example, Stata's
help for encode, decode, labelbook,
_labels2names, _strip_labels and _restore_labels.
To manipulate value labels in more sophisticated ways,
see the user-written packages of utilities labutil,
labutil2, labeldup and labelrename.
New in SJ 19(4) is Dan Klein's elabel package.
The latest version is available on the SSC site
here.

When the data contain two variables that identify the same units,
one of which is numeric and the other string,
the command labmask is particularly useful. As an example, consider the
International Standard for country codes,
which provides a three-digit numeric code for each country.
From this source, one can create two Stata variables,
one of which is the "short country name" and the other a three-digit ISO-certified numeric code.
Suppose you name the Stata variables cname and ccode.
When Stata insists on a numeric code for the country,
one could use Stata's encode command to create a numeric version of
the string variable, cname. But the numeric variable so created would be different
from the official three-digit ISO code. (For example, encode would assign
Afghanistan the numeric code "1", while the official iso code for
Afghanistan is actually "8".)
labmask is a simple soultion to this problem.
To prevent confusion from having
two numeric codes for the same country, one would instead use the command:

labmask ccode, values(cname) lblname(cname)

which creates a set of value labels called cname and attaches them as value labels
to the numeric variable ccode. (Hat tip to Julian Duggan.)

Sometimes one needs to map ranges of a numeric variable into values of a categorical variable.
A shortcut for this procedure is to use Stata's egen
with the sub-command cut(). UCLA's Statistical Consulting service
has a helpful tutorial on using egen newvar = cut(...)
here.
If you are using an old version of Stata into which Stata has not yet
added the cut() sub-command, you can install the add-on
//www.stata.com/stb/stb50")"':here.
Caveat emptor on such an old community contributed add-on.

Sometimes one's data on multiple variables arrives in a long string.
To parse such a string, extracting usable individual string or numerical values,
Stata's string functions are key. Several of the egen functions
discussed above manipulate strings
also see the moss comand for finding multiple occurrences of substrings
and the FAQ on how to parse regular expressons
here.
For an example of this issue and proposals for its resolution, see
Statalist here.

Interpreting models with limited or categorical dependendent variables

The statistics text by Long and Freese (2014) presents a range of useful techniques for estimating and,
especially, interpreting the results of regression models for which the dependent
variable is either a dummy (i.e. binary) variable or an ordered or unordered
categorical (i.e. polytomous) variable. The 2010 Stata Journal article by Jann & Long (2010)
shows how the user can display spost13 results using Ben Jann's estout
package.

References:

J. Scott Long & Jeremy Freese. 2014. Regression Models for Categorical Dependent Variables Using Stata, 3rd Edition. College Station, TX: Stata Press

B. Jann & J.S. Long, Tabulating SPost results using estout and esttab, Stata Journal, Volume 10 Number 1: pp. 46-60.

As of the summer of 2017, the updated
program can be installed
here.

Workflow organization, reproducibility and code dependency

Managing the workflow for a Stata project has several objectives. Good managmeent practices
help the Stata coder produce code that is fully documented, can be understood by others
and is free of errors. When the analytical product is published, good practice is
to provide a supplementary package containing the code and data which allow others
to replicate the numerical results. A useful source of suggestions
for workflow organization is
Long, J. Scott (2009) The workflow of data analysis usine Stata.
Stata Press.
Also see the World Bank DIME project's
"https://dimewiki.worldbank.org/Ietoolkit:workflow management toolkit.

One of Stata's strengths is the availability on the web of hundreds (thousands?)
of community-contributed programs, including many programs suggested in this help file,
which are not part of official Stata.
These unofficial programs required for program execution are called
"dependencies".
Community programmers update their programs, sometimes in ways that
change the results or break previously working code.
Sometimes community-contributed programs disappear from the web.
Thus, in order to assure that a future analyst can replicate one's results,
one must not only document these dependencies, but also
include with the replication package the utilized version of all
community-contributed programs.

Diana Goldemberg's SSC program dependencies enables the user
to "freeze" (her word) a set of user-contributed
Stata ADO/MATA/MLIB files in order to package them with the distributed
replication package and thus facilitate future replication of their results.
I like the fact that it allows the future replicator to temporarily modify their
adopath as they replicate.
Then, when the replication work has completed, the replicator can choose
to delete the replication-specific collection of ADO/Mata/MLIB programs
and return their personal adopath to its former state.

For well-maintained community-contributed programs that one deems
likely to persist into the future, it may not be necessary
to copy the package's code into one's replication package.
It might be sufficient simply to include in one's code a check on the version number
of community-contributed programs. This topic has been discussed in nuanced detail on Statalist by
Daniel Klein (2019)
and on this thread started by
Sergio Correia (2021).

Three useful community-contributed programs that help implement
version control, manage dependencies and assemble a replication package
for distribution are Daniel Klein's rqrs, Sergio Correia's require
and Daniel Bela's zippkg.
To help a user manage a project's directory tree,
Correia recommends the contributed utility -here-,
which can be installed from GitHub by clicking
//raw.githubusercontent.com/korenmiklos/here/master/")"':here.

Stochastic Frontier (SFA) and Data envelopment analysis (DEA)

For statistical approaches to efficiency analysis, Stata has two Stata-supported
stochastic frontier commands, frontier and xtfrontier.
In addition there is a new user-written command with extended capabilities called sfcross.
For stochastic frontier models with endogenous right-hand-side variables, see sfkk.
For confidence intervals of the technical efficiency estimates see frontier_teci.

Since the Data Envelpment Analysis (DEA) approach to computing the technical efficiency
of "Decision Making Units" originated in the linear programming community
rather than the statistical community, DEA has only in the last few years benefited from
community contributed Stata programs. A building-block of recent commands is
teradial which is part of a package documented in
Badunenko and Mozharovskyi (2016). "Nonparametric frontier analysis using Stata"
Stata Journal, Vol. 16:3, pp. 550-89.
Also see their
PowerPoint presentation
to the 2016 Stata Users Group meeting in Germany.
Newer contributions that build on Badunenko's work with Mozharovskyi include
Badunenko and Tauchmann's simarwilson comand and
Whang, Du and Zhang's teddf and gtfpch commands which
accommodate not only desirable outputs, but undesirable ones like pollution.
Also see Tauchmann's earlier command orderalpha and the antecedent
of all community contributed DEA programs in Stata,
Ji and Lee's dea.

Simulation tools

Bill Gould presented a technique for constructing a multivariate normal dataset from scratch in order to match
a desired correlation matrix. His FAQ on this is here: http://www.stata.com/support/faqs/stat/mvnorm.html}

However, his approach is no longer necessary thanks to the new Stata command: drawnorm. Do not confuse this
command with corr2data which is not usually suitable for use in simulation studies.

Spatial data analysis in Stata

Descriptive analysis and cartography.
Beginning with version 15 of Stata, spatial analysis is now a formal, supported
part of Stata. If you don't have Stata 15, you can preview Stata 15's spatial
capabilities here.
From inside Stata 15, see spatial for a quick introduction and overview
with links to the new-in-version-15 Stata SP manual.

Pisati's 2012 presentation
gives a good introduction to using his shp2dta which is now superceded
by Stata's own command spshape2dta.

Some useful independent web sites are those by Friedrich Huebler
and an anonymous Filipino woman
who gives the code for producing a map of the Philippines.

To install an older suite of geospatial mapping and regression tools for Stata,
including those mentioned above, click on "install spatial" on this
menu.

Inferential analysis and geospatial econometrics.
Geospatial statistical analysis requires a weight matrix, which in general
is of dimension N x N, where N is the number of individuals or spatiallly distinct
objects in the analysis. Stata's ability to construct, manipulate, store
and manage large weight matrices took a giant leap forward with the
publication in 2013 of spmat, which has now been incorporated in
formal Stata as spmatrix.

Causal Treatment effects

The World Bank's "DIME Project" maintains a set of Stata tools for impact evaluation
on GitHub which can be described and installed
here.
For an introduction to the DIME approach to causal analysis and a link to its GitHub repository go
here.
For more links to discussions if impact evaluation methods, see the landing page for
DIME blogs.

Begining with version 16, Stata has an entire manual devoted to the estimation of
treatment effects. In versions 16 and 17, the manual is entitled Treatment Effects.
In version 18, the manual is renamed Causal Inference.
Because Stata makes its manuals freely available on the web,
even those with earlier versions of Stata can benefit from the version 18
manuals' exposition of statistical methods for impact evaluation.
As a supplement to, or even a replacement for, standard textbooks
on IE methods, I recommend the following extracts from
Stata's version 18
Causal Inference and Treatment-Effects Estimation Reference Manual (2023).

o Introduction to causal inference and treatment-effects estimation (15 pp.)

o Introduction to treatment effects for observational data (13 pp.)

o Advanced introduction to treatment effects for observational data (13 pp.)

Propensity score matching: The teffects psmatch command
is a particularly useful addition because, unlike the community contributed psmatch2,
it provides unbiased estimates of the standard errors of estimated treatment effects.
For a review and comparison of the two programs, see
Propensity Score Matching in Stata using teffects.
Note also this trenchant critique of PS matching by King and Nielsen.
King and Nielsen (2019).

Difference-in-differences:
Versions 17 and 18 expand Stata's official repertoire
of DID estimators with didregress, hdidregress
and the panel data DID estimators, and .
Useful complementary DID commands and utilities contributed by the community include
various programs for automating difference-in-differences (or "dif-in-dif")
estimates of treatment effects. These include
Juan Villa's diff,
Mora and Reggio's didq,
Chaisemartin et al's fuzzydid,
Cerulli and Ventura's tvdiff,
Fernando Rios-Avila et al.'s drdiff and
csdiff,
and others. A related contribution by Niels Bruun which also visualizes a treatment effect is his
emc program
described in his SJ 19(3) article.

Regression discontinuity or RDD: Cattaneo and coauthors have published a series of useful community-contributed
commands to execute and interpret regression discontinuity analyses.
The updated version of the package -rdrobust- is available
here.
The package was first distributed with the 2017 Stata Journal article:
here.

Synthetic controls or Regression controls or the Matrix completion method: Methods and Stata programs for using synthetic controls and regression discontinuity are presented at the
2021 NBER Methods lecture by Alberto Abadie, Matias Cattaneo and Rocio Titiunik.
The lecture is posted
here.
To see a list of all the programs by Matias Cattaneo and Rocio Titiunik et. al.
that are related to methods for causal inference, search with the following Stata command:

search Cattaneo, a

or the command:

search Titiunik, a.

Daniel Pailanir has ported to Stata/Mata from R the methods for synthetic Difference-in-Differences
introduced Arkhangelsky et al (2021). While his program is available on SSC
//fmwww.bc.edu/repec/bocode/s"')"':here,
he advises that the latest version be downloaded from his
GitHub repository with the
github
command like this:

github search sdid

Instead of using the Stata github command, you can instead use Stata's
net command to access and then install sdid directly from GitHub like this:

//raw.github.com/Daniel-Pailanir/sdid/master"'"':net from `"https://raw.github.com/Daniel-Pailanir/sdid/master"'

Wrapper programs that facilitate the synthetic control approach to impact evaluation include:

//raw.github.com/bquistorff/synth_runner/master"'"':synth_runner from GitHub

//www.stata-journal.com/software/sj20-4")"':npsynth same version v16 as on SSC, but with more ancillary files.

Citations on synthetic control include:

Abadie, A., Diamond, A., and J. Hainmueller. 2015.
Comparative Politics and the Synthetic Control Method,
American Journal of Political Science, Vol. 59, No. 2, pp. 495-510.

Abadie, A., Diamond, A., and J. Hainmueller. 2010.
Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program.
Journal of the American Statistical Association 105(490): 493-505.

Abadie, A. and Gardeazabal, J. 2003.
Economic Costs of Conflict:A Case Study of the Basque Country.
American Economic Review 93(1): 113-132.

Arkhangelsky, Dmitry, Susan Athey, David A. Hirshberg, Guido W. Imbens, and Stefan Wager.
Synthetic Difference in Differences,
American Economic Review, December 2021.

Galiani, S., B. Quistorff.
The synth_runner package: Utilities to automate synthetic control estimation using synth,
Stata Journal, 17:44, pp. 834-849

Mediation analysis

A useful set of tools for parcelling out an observed association among several
channels by which it might be mediated is provided by the user written program
medsens. A 2013 presentation describing the technique can be found
here.

But Michael Clemens suggests that the command b1x2 written by
Jonah Gelbach is a superior approach. Gelbach's exposition of this command is
here.

Decomposition Analysis (Blinder-Oaxaca & Shapley values)

An important tool for analyzing the policy implications of various right-hand-side
variables is called
Blinder-Oaxaca decomposition analysis.
The authoritative overview of decomposition techniques is Fortin, Lemieux and Firpo's Chapter 1 from volume 4a of the Handbook of Labor Economics:
[Link to the gated reference]
[Link to the working paper version downloadable from NBER]

The 2008 Stata journal
article
on the Blinder-Oaxaca decomposition is written by Ben, Jann,
one of Stata's most productive and cleverest community contributors.
(See the SJ editors' description of his contributions when they awarded him the
Stata Journal Editors' Prize 2017.)
Jann's 2008
article
documents a Stata program Jann has written called “oaxaca”.
While the technique was originally developeded for a linear regression,
Yun (2004) has extended the technique to non-linear models.
(Jann's article gives the reference.)
Following Yun (2004), Jann has included options to perform the decomposition
for logit or probit models. Instead of installing “oaxaca” from the
Stata Journal site, I suggest installing it from SSC,
because Jann has updated the program through 2011 on SSC.
The Stata command for installing from inside Stata is:
ssc install oaxaca.
From the installation site, you can also retrieve the auxiliary files in order to replicate
the examples in the help file and in the SJ article.
Note that, unlike the method of dominance analysis described below,
Blinder-Oaxaca decomposition typically assumes that the right-hand-side (rhs) variables in the regression model
are fixed, perhaps because the theory motivating the regression mandates their inclusion.

Sinning, Hahn and Bauer (2008)
provide
an alternative Stata implementation of the Blinder-Oaxaca decomposition method,
extended to apply to any non-linear regression model, including the logit model.
The first author, Mathias Sinning, presented their nldecompose Stata
program at Stata Users Group meeting in 2007. Here’s the Powerpoint presentation
version of their 2008 SJ
article.
The installation command from inside Stata for the updated May 2009 version of nldecompose is:
view net sj 9-2 st0152_1.
This command is also accompanied by auxiliary files.

The program xtoaxaca is a panel data version of oaxaca, which accomodates fixed-effects.

An alternative type of decomposition has the objective
of partitioning the R^2 of the full regression into a portion attributable to each rhs variable.
If we consider only the full regression including all the rhs variables, this is an impossible task,
because each rhs variable's contribution to the R^2 depends on which of the other variables is included in the regression.
(The only exception would be if all the rhs variables are mutually orthoganal,
in which case partition of the R^2 is unambiguous.)

A way to slice this Gordian knot is to borrow from game theory the concept of the "Shapley value". Stata programs
that define an explanatory variable's contribution to the R^2 as the average of its contributions to R^2
in all possible regressions are based on the concept of the Shapley value.
Such programs have been contributed to the Stata community by Joseph Luchman.
They are: domin, domme, mixdom and mvdom.
Also see domdiag. However, it is hard to understand how knowing the Shapley value
(or "dominance") of a rhs variable in a given sample of data informs policy.
For example, if the same explanatory variables were studied in a controlled experiment,
the correlations among them would be entirely different as would the dominance statistics.
Thus, those interested in drawing inferences about the relative importance
of explanatory variables for policy design, would typically apply Stata's
margins command instead of presenting dominance statistics.

For other applications of the Shapley value concept in Stata programs, try search shapley

Plausible exogeneity

In a 2018 Stata Journal paper entitled
"Practical considerations for questionable IVs",
Damian Clarke and Benjamin Matta
review the classical assumptions to support the claim that a variable is a valid instrument
and then review the literature that asks how badly an IV regression would be biased
if an instrumental variable it uses is not "perfectly exogenous". They also contribute two Stata programs
plausexog and imperfectiv which implement the procedures suggested originally
by Art Kraay
(2008)
and subsequently developed in
Conley, Hansen & Rossi (2012)
Clarke and Matta have continued to update their programs on the SSC site,
so one should install them from SSC instead of from the Stata Journal site
associated with their 2018 article. Here are the links for installation from SSC:
//fmwww.bc.edu/RePEc/bocode/p") force"':plausexog
//fmwww.bc.edu/RePEc/bocode/i") force"':imperfectiv.
The SSC sites include the data used in the 2018 article, but not the DO files.
To download the DO files to your current working directory, click
//www.stata-journal.com/software/sj18-3") force"':here.

Chow test for pooling data

A test of whether the estimated coefficients in a regression model are
significantly different from one subset of observations to another is called a Chow test.
Failure to reject the null hypothesis of no difference between the slope parameters
across groups justifies pooling the data to estimate a single regression equation
and supports the validity of the typically higher t-statistics from the pooled regression.

On the other hand, rejection of the null hypothesis can be interpreted as supporting
the alternative hypothesis that slope parameters differ before and after
a point in time or from one subset of observations to another.
See the
Wikipedia (accessed 13Dec2023)
entry for graphical examples of situations in which the null
hypothesis would be rejected.

Many if not most introductory econometrics texts include a presentation of the Chow test.
However, despite the popularity of the Chow test in applied econometrics and statistics,
Stata contains no official command dedicated to the performance of a Chow test.

But Chow tests are easy to do in Stata.
Bill Gould, one of Stata's founders, has contributed several clear FAQ expositions
of how to do a Chow test in Stata, which can be found by typing
search Chow, faq at the Stata command line.
He explains that different versions of the Chow test allow the estimated regressions
for different time-periods or groups to have different constant terms or different
disturbance term variances.
There are also at least two community contributed commands which I have not tried:
chowtest and chowreg. Before using them,
I would want to benchmark them against the results in Gould's FAQs.
Another Stata FAQ
by Stata's Isabel Canette
shows how to perform a Chow test using Stata's gsem command.

Perhaps the easiest way to do a Chow test in Stata is not mentioned in Gould's FAQs
and does not appear in response to the command search Chow.
Stata's official command suest was present in version 15 and possibly earlier.
Example 2 in suest's help file shows how to construct
a Chow test which allows the variance of the disturbance term to vary across the two groups,
but imposes the maintained assumption that the cross-equation covariances of the coefficients are all zero.
The example uses simulated data from Stata's website.

Example 2: Do coefficients vary between groups? ("Chow test")

. webuse income, clear
. regress inc edu exp if male
. estimates store Male

. regress inc edu exp if !male
. estimates store Female

. suest Male Female
. estimates store suest_M_F // -suest- results can be stored
. test [Male_mean = Female_mean]

The above code tests the null hypothesis that the slope coefficients of edu
and exp are identical for males and females.
It fails to reject. A stricter version of the Chow test null hypothesis also
requires that the constant terms be equal across groups. As long as the
suest test results are still in memory, this stricter test
can be computed by adding the option constant.

. estimates restore suest_M_F
. test [Male_mean = Female_mean], constant

Note that the Chi^2 test statistic is larger, but the p-value does not decline.
At 0.39, it is almost eight times larger than would be required
to reject the null at the 0.05 significance level.
(With real-world data on male and female income, even if the slope coefficients
of education and experience were similar, statistically indistinguishable
constant terms would be unlikely - because, even after controlling
for all observables, women have been paid less than men.)

An alternative approach to the slopes_and_constant Chow test explained by Gould's FAQs
is to compare the two separately estimated regressions with a pooled regression
that uses factor (i.e. dummy) variables like this:

. regress inc i.male##c.edu i.male##c.exp
. estimates store Pooled

A test of the null hypothesis that gender does not affect the slope or intercept coefficients
is a test of whether the coefficient of 1.male and all the coefficients interacted with 1.male
are jointly equal to zero. My program testallwith is a wrapper for Stata's testparm
which facilitates this test, like this:

. estimates restore Pooled
. testallwith i.male##c.edu i.male##c.exp, joint(male)

The p-value of this test constructed by testallwith is 0.38,
is almost identical to the 0.39 p-value using suest.

Although the fastest way to do a Chow test using suest will likely
work well in most situations, it has a flaw.
Because it simply assembles the estimates from the separate regressions for males
and females, the suest Chow test must impose the maintained hypothesis
that the estimated coefficient of a variable in the Male equation is uncorrelated
with the estimated coefficients of all the variables in the Female equation.
The Pooled result does not impose such a restriction. To see this, compare
the estimated variance-covariance matrix from suest:

. estimates restore suest_M_F
. matrix list e(V) // Note the block of off-diagonal zeros

with the estimated variance-covariance matrix from the pooled regression:

. estimates restore Pooled
. matrix list e(V) // Non-zero values off the diagonal

Thus, when these estimated coefficients are actually correlated across equations,
the -Pooled- regression approach to the Chow test is the better approach.

An even stricter Chow test uses a null hypothesis that asserts that,
not only are the slope and intercept parameters identical between men and women,
but the variance of the disturbance term is also identical.
(This assumption is also imposed in the -Pooled- regression.)
Restoring the suest results to memory, we can compute this strictest test like this.

. estimates restore suest_M_F
. test [Male_mean = Female_mean], constant
. test _b[Male_lnvar:_cons] = _b[Female_lnvar:_cons], accum

The Chi^2 test statistic is slightly larger, but the p-value increases.
So in this artificial data, there is really no empirical support
for estimating a separate regression for men and women.

Data Access Utilities

To check the version of Stata that can read a given Stata dta file,
use the Stata utility dtaversion.

New-in-2018, the program sdmxuse facilitates
downloading data from a variety of international organizations into a Stata formatted
data file. Institutions include not only the World Bank, but also the IMF,
the OECD, the IMF, the United Nations, Eurostat and the European Central Bank.
The article can be accessed
here.
However, some users have had some difficulties using sdmxuse.
Look for comments on Statalist.

The 2016 program wbopendata is a more carefully and completely documented
utility which is expressly designed to download World Bank data. An abstract
describing the program is available on the Ideas RePEC site
here.
An older program with the same objective is wdireshape.

The World Inequality Database provides data that can be converted to Stata
formatted dta files. The website is here.
John Parman
provides a useful tutorial on accessing this data
here.
He advises installing a 2020 utility named wid
written by Thomas Blanchet.
Also see Blanchet's GitHub site on inequality here.

Programming an ADO file in Stata (Under construction)

A PowerPoint to introduce ADO-file programming can be found here:
Introduction to Programming in Stata.

The following blogs by Stata's David Drukker give
clear and complete presentations on how to make a user-friendly Stata ADO file:

https://blog.stata.com/2016/01/15/programming-an-estimation-command-in-stata-a-map-to-posted-entries/

Or search for all blog entries on programming here:

https://blog.stata.com/category/programming/

A useful utility for use with Stata's syntax command is the undocumented
opts_exclusive. Documentation for several other "undocumented" commands
is available by typing help undocumented.

The benefits from writing one's own Stata ado files are multiplied exponentially by using macros.
There are many resources for learning how to use Stata's macros. Among them is entry #3
in David Drukker's blog

https://blog.stata.com/2015/11/03/programming-an-estimation-command-in-stata-global-macros-versus-local-macros/

Macros work together with lists, both of which are managed with the help of quotation marks.
One important application of macros is their use in "loops" as described in Nick Cox's
"Speaking Stata: Loops, again and again", Stata Journal, 2020

Often it is useful to assemble or manipulate a list of character strings.
These might be strings without spaces which follow Stata's
naming conventions, such as the names of variables, matrices, scalars or macros.
Or they might be the names of e.g. countries, some of which have spaces.
A further complication arises if some of the character strings contain single, double or compound quotes.
Stata includes a set of utilities for manipulating lists.
Useful discussions on Statalist include the following two threads started by
Belinda Foster
and by
Jasmine Davidson.
The latter thread contains the comment by William Lisowski that:
"[The Stata command] -macro list macname- is the only reliable way of seeing exactly what is stored in a macro."
To refer to a local macname after -macro list-, prefix it by an underscore character.
Otherwise -macro list- assumes it is a global macro.
Post
#5 by Bjarte Aagnes
is helpful in demonstrating how to construct a list of strings all of which are enclosed in quotes. As shown in that post,
the line in the following code that defines the local macro -newvlist- does not work,
because the double quotes are stripped from the first element of the macro -newvlist-.
But the line of code defining -newvlist2- does work, because of the compund double quotes.
The command -macro list _newvlist2-
shows that the first element of the list is -"price"- as it should be to work properly
in the function inlist().

sysuse auto, clear
foreach v of varlist price length mpg turn make {
local newvlist `newvlist' `comma' "`v'"
local newvlist2 `"`newvlist2' `comma' "`v'""'
local comma ,
}
macro list _newvlist
cap noi di inlist("mpg",`newvlist')

macro list _newvlist2
di inlist("mpg",`newvlist2')

Other repositories of user written programs

For a clickable list of all user sites that Stata can locate with its webcrawler,
consult "SJ and other community-contributed commands" under
the pull-down Help dialogue at the top of the Stata Results window or
click here

To find all programs contributed by current or former CGD-affiliated Stata contributors,
from inside Stata type search CGD. Or
click here

Ben Jann is a major contributior of user-written programs, some of which are referenced
in this help file.
(See the SJ editors' description of his contributions when they awarded him the
Stata Journal Editors' Prize 2017.)
His site can be reached at
http://repec.sowi.unibe.ch/stata/index.html by clicking
here
Stata's own help facility identifies many of Ben Jann's programs,
and gives links to the Stata Journal articles documenting many of those programs,
if you type:
search jann, a.

J. Scott Long's two books presenting his Stata programs are
"The Workflow of Data Analysis Using Stata" and
"Regression mdoels for categorical dependent variables".
To download any of his programs from these two books, open Stata's viewer and type:
net from "http://www.indiana.edu/~jslsoc/stata".

Roger Newson has moved to
King's College London.
His Stata repository is
here.

Fernando Rios-Avila
has several GitHub repositories, including one devoted to extensions of Stata's visualization tools
here.
He usually posts his programs on Boston College's SSC site.

Researchers at the World Bank use Stata for their own research and also for
collaboration with researchers in client countries. The World Bank's
Poverty Analysis Toolkit and other useful Stata programs are described
here (Accessed 19Aug2019).
Available tools include
ADePT
which can be downloaded here
here.
Stata impact evaluation resources from World Bank authors are on
GitHub. Also see the book
"Impact Evaluation in Practice".
One of the World Bank contributors is Misha Lokshin.
Look for commands he has written or co-authored here Lokshin.
For an introduction to the DIME approach to causal analysis and a link to its GitHub repository go
here.
A former DIME contributor who has moved to Georgetown University is Ben Daniels.
Daniels' GitHub site
is a repository for many useful Stata programs, some of which he has not yet posted to SSC.
The easiest way to install a subset of his packages is from his
//raw.githubusercontent.com/bbdaniels/stata/main""':Stata compatible download site.

Thanks to Haghish, E. F. (2020) The many Stata packages on GitHub can now be more easily downloaded and installed.
See his journal article:

Haghish, E. F. (2020). Developing, maintaining, and hosting Stata statistical software on GitHub.
The Stata Journal, 20(4), 931-951.

or his GitHub site for details. Or install his program
github here.

If you know the name of a Stata ADO file available on GitHub,
finding and installing it is simplified
by the command github search. For example, the program btable by
Alan Haynes
cannot be found with Stata's search command. But by typing:

github search btable

we see the following display.

Repository Username Install Description

btable CTU-Bern Install Stata package for baseline tables
132k updated on 2022-10-10
Fork:2 Star:0 Lang:Stata

Clicking on CTU-Bern under Username takes one to the GitHub user site.
Clicking on Install adds the program to one's own Stata installation.

If a GitHub Stata site has a -stata.toc- file which lists Stata -pkg- files,
it can be accessed from inside Stata with a net from command.
For example Keith Kranker's archived adofile site on GitHub is most easily available
like this:

//raw.githubusercontent.com/kkranker/kk-adofiles/master""'

Brian Quistorff's two GitHub Stata repositories are accessible from Stata by the following two lines of code.
(The programs themselves are located in the alphabetical subdirectories.)

//raw.githubusercontent.com/bquistorff/Stata-modules/master""'

//raw.githubusercontent.com/bquistorff/stata-reproducible/master""'

Thomas Grund's suite of commands for network analysis are available here:

//raw.githubusercontent.com/ThomasGrund/nwcommands/master""'

Author

Contact Mead Over at:
MOver@CGDev.Org with problems or suggestions.