Juan Antolín-Díaz, MIT Sloan

Paper: Attention-based Vector Autoregressions
Abstract: We introduce attention-based vector autoregressions, a new class of nonlinear time series models in which the conditional mean and variance are modeled through the attention mechanism, the core of the Transformer architecture in language modeling. Attention leads to state-dependent, time-varying dynamics, that are deterministic functions of past data. The framework nests constant-parameter linear VARs and approximates Dynamic Factor Models, but provides dimensionality reduction with a natural economic interpretation. We develop Bayesian priors and estimation methods suitable for time series and sample sizes available in macroeconomics and finance, and characterize the model’s theoretical properties. An empirical application to forecasting U.S. inflation after the pandemic demonstrates gains in point and density forecast accuracy relative to standard benchmarks.