Compare the forecast accuracy of two volatility forecasting methods against realized (squared return) Variance, per asset, via the Diebold-Mariano (1995) test.

Rather than comparing two different assets, this compares two different ways of forecasting the same asset’s next-day Variance – e.g. the simpler exponentially weighted (EWMA, see Risk.get_ewma_volatility) approach against a plain rolling-window Standard Deviation – to determine whether one is significantly more accurate than the other for a given asset.

For more information about the method, see the following paper:

  • Diebold, F.X., & Mariano, R.S. (1995). “Comparing Predictive Accuracy.” Journal of Business & Economic Statistics, 13(3), 253-263.

Also known as: DM test, forecast comparison test.

No programming experience? With the Finance Toolkit MCP server, AI assistants such as Claude and ChatGPT can calculate the Diebold-Mariano Test for you. Just ask in plain English.

Calculate the Diebold-Mariano Test in Python

The Diebold-Mariano Test is available in the Econometrics module of the open-source Finance Toolkit. Install it with:

pip install financetoolkit -U

Then call get_diebold_mariano_test as shown below.

from financetoolkit import Toolkit

toolkit = Toolkit(["AAPL", "MSFT"], api_key="FINANCIAL_MODELING_PREP_KEY")

toolkit.econometrics.get_diebold_mariano_test(method_a="ewma", method_b="rolling")

Which returns:

  AAPL MSFT
Diebold-Mariano Statistic -2.4324 -2.8338
P-Value 0.0152 0.0047
Mean Loss Differential -0.0000 -0.0000
Observations 735 735

Parameters

get_diebold_mariano_test accepts the following parameters:

  • period (str, optional): The data frequency for returns (daily, weekly, quarterly, or yearly). Defaults to “daily”, since window_size is expressed in return observations of this frequency.
  • method_a (str, optional): The first volatility forecasting method, one of “ewma” or “rolling”. Defaults to “ewma”.
  • method_b (str, optional): The second (competing) volatility forecasting method, one of “ewma” or “rolling”. Defaults to “rolling”.
  • window_size (int, optional): The rolling window size used by the “rolling” method. Defaults to 22 (approximately one trading month).
  • lambda_ (float, optional): The decay factor used by the “ewma” method. Defaults to 0.94.
  • loss (str, optional): The loss function to compare forecast errors with, one of “squared” or “absolute”. Defaults to “squared”.
  • include_benchmark (bool, optional): Whether to include “Benchmark” among the assets tested. Defaults to False.
  • rounding (int | None, optional): The number of decimals to round the results to. Defaults to None.
Root Mean Squared Error (RMSE) Mean Absolute Error (MAE) Out-of-Sample Validation

The Econometrics module page introduces the module, and the sidebar lists all of its functions.

Share