Skip to main content
Blog home
Newsletter
News
Video
Podcast
Reading list
Wright Brief
Sign In

What is Hidden Markov Model?

A plain-language explanation of Hidden Markov Model, with a worked example, common pitfalls and the questions people ask most.

Listen to this article
Audio · ~2 min
Add as a preferred source on Google

A hidden Markov model, or HMM, assumes that a system moves between a small number of unobserved states, such as calm and turbulent markets, and that what we observe, like returns, depends on the current state. The model estimates which state the market is in.

The building blocks

  • Hidden states: for example, state 1 is a calm, rising market and state 2 a volatile, falling one.
  • Transition probabilities: the chance of moving from one state to another each period.
  • Emission distributions: the pattern of returns, such as the mean and volatility, in each state.

An illustrative example

From \ ToState 1 (calm)State 2 (turbulent)
State 10.950.05
State 20.100.90

If the chance of staying in state 1 each week is 0.95, the expected stay is 1 / (1 - 0.95) = 20 weeks. For state 2 it is 1 / (1 - 0.90) = 10 weeks. The probabilities are made up for illustration.

Fitting and using it

The parameters are estimated from the data, commonly with the Baum-Welch (expectation-maximisation) algorithm. Two outputs matter:

  • Filtered probabilities use data up to each date only, so they can be used in real time.
  • Smoothed probabilities use the whole sample and look much better in hindsight, but they contain look-ahead bias if used for trading decisions in a test.

Cautions

  • States are unstable. Estimates can change when new data arrive, and states can swap labels.
  • Number of states. Each extra state gives fewer observations to learn from, which invites overfitting.
  • Lag. Regime changes are usually recognised only after they have started.

HMMs are one approach to the problem described under regime detection models and market regimes.

Frequently asked questions

What is a hidden Markov model?

It is a statistical model in which an unobserved state moves between a few possibilities over time according to fixed probabilities, and the observed data depend on the current state.

How are hidden Markov models used in finance?

They are used to infer market regimes, such as calm and turbulent periods, from returns and other data, and to estimate how likely a switch is.

What is the difference between filtered and smoothed probabilities?

Filtered probabilities use only data up to the current date and can be used live. Smoothed probabilities use the whole sample and are not available at the time, so using them in a backtest leaks the future.

Can a hidden Markov model predict the next regime?

It gives probabilities of moving between states, but regime changes are hard to anticipate and are usually recognised with a delay.

Further reading

  1. Hamilton, J. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57(2).
  2. Rabiner, L. (1989). A tutorial on hidden Markov models. Proceedings of the IEEE 77(2).
Browse the full Quant Glossary →

Disclaimer: Investment in securities market are subject to market risks. Read all the related documents carefully before investing. Registration granted by SEBI, membership of a SEBI recognized supervisory body (if any) and certification from NISM in no way guarantee performance of the intermediary or provide any assurance of returns to investors.

The content in these posts/articles is for informational and educational purposes only and should not be construed as professional financial advice and nor to be construed as an offer to buy/sell or the solicitation of an offer to buy/sell any security or financial products. Users must make their own investment decisions based on their specific investment objective and financial position and using such independent advisors as they believe necessary.

Wryght Research & Capital Pvt (Brand name: Wright Research) is a SEBI Registered Portfolio Manager Reg No: INP000007979 (Validity: Apr 03, 2023 – Perpetual) and a SEBI Registered Research Analyst No: INH000017295 (Validity: Jul 03, 2024 – Perpetual), with its registered office at 103, Shagun Vatika Prag Narayan Road, Lucknow, UP, 226001 India and CIN: U67100UP2019PTC123244. Past performance may or may not be sustained in future. Performance provided there in is not verified by SEBI. Investment in securities is subject to market and other risks, and there is no assurance or guarantee that the objectives of any of the strategies of the Portfolio Management Services will be achieved. Registration granted by SEBI, enlistment as RA with Exchange and certification from National Institute of Securities Markets (NISM) in no way guarantee performance of the intermediary or provide any assurance of returns to investors. Please read the Disclosure document carefully before investing. Securities quoted are for illustration only and are not recommendatory. Charts shown are for illustration only. For more information and disclosures, visit our disclosures page here.

Found this useful? Share it.
Wright PMS · Portfolio Management Service

Put this research to work

The same 300+ factor research engine behind this article — applied to a professionally managed portfolio, end to end.

300+
Factors tracked
2L+
Investors
₹1,200+ Cr
Invested
SEBI
Registered PMS