Markov Regime Switching

Fit a Markov regime switching model to every asset: the mean and volatility of its returns depend on a regime that is not observed and switches from one period to the next with fixed probabilities (Hamilton, 1989). With two regimes the model typically separates calm markets (a higher mean and a low volatility) from turbulent ones (a lower mean and a high volatility), and it says which regime the market is most likely in today, how persistent each regime is and how long it lasts on average.

The regimes are ordered from the lowest to the highest volatility, so Regime 1 is the calmest. With switching_variance=False all regimes share one volatility, and they are ordered from the lowest to the highest mean instead, so Regime 1 is the one with the lowest mean. The expected duration of a regime is 1 / (1 - p), with p the probability of staying in it, in periods. The share is the part of the periods in which the regime was the most likely one.

The likelihood of a mixture of regimes has local optima, so the model is fitted from its default starting values and from the best of 20 random perturbations of them, and the fit with the highest likelihood is kept. The random perturbations come from NumPy’s global random state, so when competing optima are close, repeated calls can differ slightly; call np.random.seed beforehand for results that can be reproduced.

The formula is as follows:

\[\text{yₜ} = \text{mu\_Sₜ} + \text{sigma\_Sₜ} \cdot \text{epsilonₜ},\;\; \text{with yₜ the column series (Return by default)},\;\; \text{epsilonₜ standard normal and Sₜ ∈ \{1},\;\; \text{...},\;\; \text{K\} the regime},\;\; K = \text{regimes} (2 \;\text{by default}) \text{;} \text{sigma\_Sₜ} = \sigma \;\text{for every regime when}\; \text{switching\_variance} = \text{False}\] \[P(\text{Sₜ} = j | \text{Sₜ₋₁} = i) = \text{pᵢⱼ},\;\; \text{a first-order Markov chain with constant transition probabilities}\]

μ, σ and pᵢⱼ are fitted by Maximum Likelihood with the Hamilton filter (statsmodels MarkovRegression), keeping the best of the default start and 20 random perturbations

\[\text{Smoothed probabilities}\; P (\text{Sₜ} = j | \text{y₁..y}_{T}) \text{per}\; \text{date;} \;\text{Expected Duration} = 1 / (1 - \text{pⱼⱼ}) \text{periods;} \;\text{Share} = \text{the fraction of dates where regime}\; j \;\text{has the highest smoothed}\; \text{probability;} \;\text{Current Regime} = \text{that regime on the last date},\;\; \text{with its smoothed probability}\]

For more information, see the following paper:

Hamilton, J.D. (1989). “A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle.” Econometrica, 57(2), 357-384.

Also known as: Markov switching model, hidden Markov model, regime detection, bull and bear regimes.

Markov Regime Switching in Python or via the MCP

Install the Finance Toolkit with:

pip install financetoolkit -U

Then call get_markov_regime_switching as shown below.

from financetoolkit import Toolkit

toolkit = Toolkit(
    ["AAPL", "MSFT"],
    api_key="FINANCIAL_MODELING_PREP_KEY",
    start_date="2015-01-01",
    end_date="2025-12-31",
)

toolkit.econometrics.get_markov_regime_switching(period="weekly")

Or, after setting up the Finance Toolkit MCP, ask:

Which calm and turbulent regimes did the weekly returns of Apple and Microsoft go through from 2015 to 2025, with their mean, volatility and expected duration?

Which returns:

	AAPL	MSFT
Regime 1 Mean	0.009	0.0065
Regime 1 Volatility	0.0236	0.0217
Regime 1 Probability of Staying	0.917	0.9528
Regime 1 Expected Duration	12.0479	21.194
Regime 1 Share	0.5436	0.6411
Regime 2 Mean	0.0006	0.0021
Regime 2 Volatility	0.0488	0.0455
Regime 2 Probability of Staying	0.91	0.9209
Regime 2 Expected Duration	11.1105	12.6346
Regime 2 Share	0.4564	0.3589
Current Regime	1	1
Current Regime Probability	0.8877	0.8087

Using Your Own Data

get_markov_regime_switching runs on any data, not only prices: ratios, economic indicators, yield curves or your own dataset. Pass a DataFrame, a Series or a dictionary of them to Econometrics as data, with the dates in the rows or the columns, and set changes to how its changes are calculated: "percentage", "log", "difference" for rates and ratios, or None when the data are changes already. For example, for the monthly unemployment rates of the United States and Germany:

from financetoolkit import Economics, Econometrics

economics = Economics(start_date="2000-01-01")

unemployment = economics.get_unemployment_rate(
    countries=["United States", "Germany"], period="monthly"
)

econometrics = Econometrics(data=unemployment, changes="difference")

econometrics.get_markov_regime_switching()

Parameters

get_markov_regime_switching accepts the following parameters.

Parameter	Type	Description	Example
`period`	`str`	The data frequency (daily, weekly, monthly, quarterly, or yearly). Defaults to “daily”.	`"weekly"`
`column`	`str`	The historical data column to model. Defaults to “Return”.	`"Return"`
`regimes`	`int`	The number of regimes. Defaults to 2.	`2`
`switching_variance`	`bool`	Whether the volatility switches with the regime as well as the mean. Defaults to True.	`True`
`probabilities`	`bool`	Whether to return the smoothed probability of every regime per date instead of the summary. Defaults to False.	`False`
`include_benchmark`	`bool`	Whether to include “Benchmark” among the assets. Defaults to False.	`True`
`rounding`	`int \| None`	The number of decimals to round the results to. Defaults to None.	`2`

Markov Regime Switching in Python or via the MCP

Using Your Own Data

Parameters

Related Regimes & Structural Breaks