Dynamic Panel Data Models: Employment Persistence

Downloads

Each dataset is available as a labeled Stata .dta and its source file.

⇩ Download all data (ZIP)stata_codebook.do

Run stata_codebook.do in Stata once to attach long-form per-variable notes to the .dta files.

Load directly in code

Every file loads straight from GitHub (raw URLs). Swap the file name to load any dataset.

Stata

* Stata 14+ : `use` reads an https URL directly
global BASE "https://raw.githubusercontent.com/cmg777/starter-academic-v501/master/content/post/python_dynamic_panel/data/"
use "${BASE}abdata.dta", clear
describe
notes

Python

!pip install -q pyreadstat
import pandas as pd
BASE = "https://raw.githubusercontent.com/cmg777/starter-academic-v501/master/content/post/python_dynamic_panel/data/"
df = pd.read_stata(BASE + "abdata.dta")

# load every dataset at once
files = ["abdata", "data_prepared"]
data = {f: pd.read_stata(BASE + f + ".dta") for f in files}

# pyreadstat (richest metadata) reads LOCAL files -> download first
import pyreadstat, urllib.request
urllib.request.urlretrieve(BASE + "abdata.dta", "abdata.dta")
df, meta = pyreadstat.read_dta("abdata.dta")

Copy and paste this snippet in Google Colab app. https://colab.research.google.com/notebooks/empty.ipynb

R

# R : haven::read_dta auto-downloads an https URL
library(haven)
BASE <- "https://raw.githubusercontent.com/cmg777/starter-academic-v501/master/content/post/python_dynamic_panel/data/"
df <- read_dta(paste0(BASE, "abdata.dta"))

Overview & sources

Companion data for a hands-on Python tutorial that estimates how persistent firm-level employment is — the autoregressive coefficient ρ of a dynamic labor-demand equation — using the canonical Arellano and Bond (1991) panel of 140 UK manufacturing firms observed annually over 1976–1984 (1,031 firm-years, unbalanced). The tutorial walks the full estimator ladder: pooled OLS (biased up by the omitted firm effect) and fixed effects (Nickell bias, biased down) via pyfixest, then Anderson–Hsiao IV, Arellano–Bond difference GMM and Blundell–Bond system GMM via pydynpd, with the AR(2), Hansen, and instrument-collapse diagnostics that separate the one defensible estimate (system GMM, ρ̂ = 0.927) from four seductive wrong ones. This dataset is the classic dynamic-panel teaching dataset, used by Arellano & Bond, Blundell & Bond (1998), and Roodman (2009).

Data sources

Cite this data

Please cite this dataset as follows.

APA

Mendez, C. (2026). Dynamic Panel Data Models in Python: From Nickell Bias to System GMM [Data set]. https://carlos-mendez.org/post/python_dynamic_panel/

Arellano, M., & Bond, S. (1991). Some Tests of Specification for Panel Data: Monte Carlo Evidence and an Application to Employment Equations. Review of Economic Studies, 58(2), 277–297.

BibTeX

@misc{mendez2026pythondynamicpanel,
  author       = {Mendez, Carlos},
  title        = {Dynamic Panel Data Models in Python: From Nickell Bias to System GMM},
  year         = {2026},
  howpublished = {\url{https://carlos-mendez.org/post/python_dynamic_panel/}},
  note         = {Data set}
}

@article{arellano1991some,
  author  = {Arellano, Manuel and Bond, Stephen},
  title   = {Some Tests of Specification for Panel Data: Monte Carlo Evidence and an Application to Employment Equations},
  journal = {Review of Economic Studies},
  volume  = {58}, number = {2}, pages = {277--297}, year = {1991},
  doi     = {10.2307/2297968}
}

Variable explorer search & filter all 20 variables

Type to filter by name or label, or use the chips to filter by type. Each row shows a mini distribution. Click a header to sort.

Cross-file variable index

Which file each variable appears in (● = present).

Construction & formulas

The model is a dynamic labor-demand equation for log employment n, with current and lagged log real wages w and log capital k, a firm fixed effect α_i, year effects δ_t, and an idiosyncratic shock ε_it:

• Levels → logs: n = log(emp), w = log(wage), k = log(cap), ys = log(indoutpt) — the estimation variables are the logged columns; the level columns are kept for reference.
• Dynamic model: n_it = ρ·n_i,t-1 + β1·w_it + β2·w_i,t-1 + β3·k_it + β4·k_i,t-1 + α_i + δ_t + ε_it — ρ (the coefficient on n_lag1 / L1.n) is the persistence parameter of interest.
• One-period lag (per firm): v_lag1 = v_i,t-1 for v ∈ {n, w, k}; n_lag2 = n_i,t-2 (the Anderson–Hsiao instrument).
• First difference (per firm): d_v = v_it − v_i,t-1 for v ∈ {n, w, k}; d_v_lag1 = d_v_i,t-1. Differencing eliminates α_i exactly.

All lags and differences are computed within firm id after sorting by [id, year], so no firm inherits a lag from another firm. The first observed year of every firm therefore has missing lags/differences.

The datasets

Switch datasets with the tabs. Each shows the full variable dictionary plus a sortable statistics table with mini distributions and data coverage.

Expand all datasets expand to search (Ctrl/⌘+F) or print across all datasets

Known limitations & caveats

• Teaching dataset. This is a methods showcase on 1970s–80s UK manufacturing, not a current estimate of employment dynamics; the value is in the workflow, not the era-specific numbers.
• Each lag burns data. With T as small as 7–9 per firm, every lag/difference costs each firm its first observed year(s): the panel falls from 1,031 rows to 891 (one lag), to 751 (GMM), to 611 (the two-lag replication spec). The lag/difference columns in data_prepared are missing for those dropped firm-years.
• Unbalanced panel. 103 firms appear for 7 years, 23 for 8, and 14 for all 9; estimators handle this natively but observation counts vary by specification.
• Variance is lopsided. The between-firm SD of log employment (1.339) is about seven times the within-firm SD (0.195), so the unobserved firm level dominates — which is exactly why naive estimators fail and why dynamic-panel GMM is needed.
• Headline caveat. The system-GMM 95% CI [0.773, 1.081] includes the unit root, so 'employment is stationary' is not a defensible claim — only the point estimate and its lower bound are.