PSRC Markov Regime
- Sandra Wakefield
- Jul 25
- 12 min read
1. Overview & Purpose
The PSRC Markov Regime Confidence Filter is a companion module to the PSRC Daily Bias Dashboard. It answers a question the base dashboard does not: given the current market regime, is a bias signal statistically likely to persist, or is it occurring in a regime where persistence has historically been unreliable?
Access the Markov Regime Whitepaper
Release approved by Sally Fong and Joseph Plazo
The base dashboard scores bias from three structural factors — Prior Day Value Area position, liquidity sweep + absorption, and VWAP/CVD momentum — and gates zone activation on a single confidence measure: overnight range as a percentage of 20-day ADR. That measure answers "was there enough information built overnight," not "is the broader regime the kind that rewards this setup." The Markov filter closes that gap.
This manual documents the module's internal logic, the institutional design principles it follows, its full input reference, and the operating practices required to run it safely in a live environment. It is written for the desk operating the tool, not as marketing collateral — limitations are stated plainly where they exist.
2. Core Logic
2.1 Why a Discrete Markov Chain, Not a Fitted HMM
The obvious "institutional" answer to regime detection is a Gaussian Hidden Markov Model (HMM), fit via Expectation-Maximization (Baum-Welch), with hidden states inferred from the joint likelihood of observed returns. That is the correct model for offline research. It is the wrong model to run bar-by-bar inside a charting platform, for a specific reason:
• EM fitting is iterative and requires the full sample (or a rolling re-fit window) each time it runs — it is not an incremental, single-pass update.
• Running it live, on every bar, either means re-fitting constantly (prohibitive compute cost in a charting environment) or fitting once and freezing parameters (which silently goes stale as the market's regime statistics drift).
• A model that must look at the whole sample to produce today's state, then gets "frozen" and quietly used going forward, is a subtle form of the same look-ahead problem this framework is otherwise strict about avoiding.
The filter instead uses a first-order discrete-state Markov chain: states are assigned by a deterministic rule (Section 2.2) rather than inferred as latent variables, and the transition matrix is estimated by simple incremental counting over a rolling window (Section 2.3).
This is a lower-variance, fully causal, O(1)-per-bar estimator. It gives up the ability to infer regimes the classification rule doesn't already encode — the trade made explicitly, not hidden.
2.2 State Classification — Kaufman's Efficiency Ratio
Each completed daily bar is classified into one of three states using Kaufman's Efficiency Ratio (ER), a standard measure of trend efficiency:
ER = | Close(t) − Close(t − N) | ÷ Σ | Close(i) − Close(i−1) | for i = t−N+1 … t N = erLen (default 10 trading days) ER ∈ [0, 1] — 1.0 = a perfectly efficient, one-directional move 0.0 = maximum churn, net-zero displacement
ER is the ratio of net directional displacement to the total path length traveled to get there. A market that moves from 100 to 110 in a straight line has ER ≈ 1; a market that oscillates between 95 and 105 for ten days before closing at 110 has a much lower ER despite an identical net move. This distinguishes a trend day from a chop day using price action alone — no volume or external series required.
Classification rule applied to each confirmed daily close:
• ER ≥ erThresh and net direction positive → BULL
• ER ≥ erThresh and net direction negative → BEAR
• ER < erThresh (regardless of direction) → CHOP
This classification is fully deterministic and reproducible from price data alone — critical for the transition matrix in Section 2.3 to be built on a consistent, auditable sequence.
2.3 Transition Matrix Construction
The filter maintains a 3×3 matrix of transition counts — one row per current-day state, one column per next-day state — updated incrementally as each new day is confirmed:
→ Next: BEAR Next: CHOP Next: BULL Current: BEAR n₀₀ n₀₁ n₀₂ Current: CHOP n₁₀ n₁₁ n₁₂ Current: BULL n₂₀ n₂₁ n₂₂
The matrix is maintained over a rolling window (default 90 trading days) using a sliding-window count — not a full re-scan. On each new confirmed day, the transition from the prior state to the new state increments by 1; when the window's oldest day rolls off, its corresponding transition is decremented by 1. This keeps the estimator current with recent regime behavior (a stale 2019 transition matrix is of limited use in judging a 2026 regime) while remaining O(1) per update rather than O(window length).
From the current state's row, the model reads off:
• P(persist) — probability the current regime repeats tomorrow (the diagonal entry, normalized by the row total)
• P(next = BULL) / P(next = BEAR) / P(next = CHOP) — full one-step-ahead distribution conditioned on today's state
This is a standard empirical (frequentist) Markov chain estimator — the maximum-likelihood estimate of each transition probability given the observed counts in the window.
2.4 Confidence Gate Derivation
The gate the filter exposes to downstream systems, markovConfidenceOK, requires two conditions simultaneously:
1. Sample adequacy — the current state's row must contain at least minSamples observed transitions (default 12). A persistence probability computed from four historical transitions is a coin flip dressed up as a statistic; the gate refuses to answer when the sample is too thin, rather than reporting a number that implies false precision.
2. Persistence strength — the diagonal (self-transition) probability must clear persistThr (default 55%). Below this, the current regime is no more likely to continue than a range of weaker alternatives, and treating it as a stable regime would be circular reasoning: assuming persistence in order to justify trading persistence.
Both conditions failing independently produce distinct HUD states ("LOW SAMPLE" vs. "WEAK") rather than a single opaque "NO" — the operator should know which condition is failing, since the appropriate response differs (widen the lookback window vs. wait for a genuinely more persistent regime).
2.5 Timeframe-Independence Architecture
All daily-bar data is sourced via request.security_lower_tf(symbol, "D", …) rather than reading the chart's own bars. This guarantees every completed daily bar is processed exactly once, regardless of what timeframe the chart is displaying:
Chart Timeframe | Behavior |
Below 1D (1m – 12H) | Array returns 0 elements on most bars, 1 element on the bar where a calendar day rolls over. |
Exactly 1D | Array returns 1 element per bar — equivalent to reading `close` directly. |
Above 1D (1W, 1M) | Array returns multiple elements per bar (up to ~7 on weekly, ~31 on monthly) — every interior day is still processed. |
A naive approach — detecting day-rollover from the chart's own bar timestamps — fails silently on weekly/monthly charts: it re-evaluates once per chart bar and skips interior days, corrupting the transition sequence without throwing any error. This is the specific defect corrected in the current build.
REPAINT DISCIPLINE A transition-matrix increment is a permanent counter, not a value that self-corrects like a plot. On the currently-forming chart bar, the final element of a request.security_lower_tf() array can be an incomplete daily bar. The filter withholds that element until barstate.isconfirmed is true, using a watermark timestamp to guarantee each day is counted exactly once — never early, never twice. |
3. Why This Is an Institutional-Grade Approach
"Institutional-grade" is not a marketing label here — it refers to a specific set of engineering and statistical disciplines that separate a tool suitable for capital allocation from a retail indicator that merely looks correct on a backtest. The filter is built to each of the following standards:
3.1 Non-Repainting by Construction, Not by Convention
Every reference to historical data uses either a confirmed-bar offset or lookahead=barmerge.lookahead_off. The realtime hold-back guard (Section 2.5) closes the one remaining repaint vector inherent to request.security_lower_tf() on higher timeframes. This matters because a regime filter that quietly repaints will show excellent performance in a backtest and materially different, worse performance live — the single most common reason retail indicators fail to survive contact with live capital.
3.2 Sample-Size Discipline Before Confidence Is Reported
Institutional risk frameworks distinguish between "we have no signal" and "we have a signal we do not yet trust." Reporting a persistence probability computed from an inadequate sample as though it were reliable is a statistical malpractice that shows up in retail tools constantly — a percentage is displayed because the arithmetic is possible, not because the estimate is meaningful. The minSamples gate enforces this distinction explicitly rather than leaving it to the operator's judgment on a bar-by-bar basis.
3.3 Probabilistic Regime Read, Not a Binary Switch
The filter does not output "bull market / bear market" as a hard binary. It outputs a full one-step-ahead probability distribution over three states, conditioned on the current state. This is closer to how a multi-strategy allocator actually treats regime information — as a continuously graded input to position sizing and strategy selection — than a binary flag that a strategy either obeys or ignores.
3.4 Designed as a Layer in a Confluence Stack, Not a Standalone Signal
The filter produces no entries, exits, or standalone trade signals. It is explicitly built to sit inside PSRC's existing Tier 2 protection layer (Daily Bias Alignment) as an additional AND-condition on the existing confOK gate — consistent with the desk's broader philosophy that no single factor should authorize a trade, and that Bias Timeframe, Confirmation Timeframe, and Entry Timeframe each carry their own veto.
3.5 Rolling-Window Estimation, Not a Static Historical Average
The transition matrix is intentionally windowed (default 90 days) rather than accumulated over the full instrument history. Regime statistics from five years ago are treated as decreasingly relevant, not as ground truth — a discipline standard in adaptive, regime-aware allocation frameworks and one many static indicators omit entirely.
3.6 Comparative Summary
Property | Naive Threshold Filter | PSRC Markov Filter |
Regime determination | Fixed rule (e.g. price vs. moving average) | Empirically estimated transition probabilities, re-estimated on a rolling window |
Output type | Binary flag | Full probability distribution over next-state outcomes |
Sample-size awareness | None — always reports a state | Explicit gate; reports "insufficient sample" rather than a false-precision number |
Timeframe behavior | Typically tied to chart resolution | Sourced from daily bars regardless of chart timeframe |
Repaint exposure | Frequently unaudited | Explicitly guarded with a confirmed-bar watermark |
Adaptivity | Static rule, no memory of recent regime behavior | Rolling-window matrix decays stale transitions automatically |
4. Input Reference
All inputs are grouped in the indicator settings exactly as listed below. Defaults are tuned for a liquid daily-driven instrument (major FX, index futures); see Section 5 for adjustment guidance by asset class.
4.1 Regime Classification
Input | Default | Effect |
Efficiency Ratio Lookback (erLen) | 10 days | Window over which the ER trend-efficiency measure is computed. Shorter = more reactive, more prone to reclassifying on noise. Longer = smoother, slower to recognize genuine regime change. |
ER Threshold (erThresh) | 0.35 | Minimum efficiency required to classify a day as trending (BULL/BEAR) rather than CHOP. Higher = stricter trend qualification, more days fall into CHOP. |
4.2 Markov Chain
Input | Default | Effect |
Transition Matrix Rolling Window (lookbackN) | 90 days | Number of trailing daily transitions used to estimate the matrix. Shorter windows adapt faster to structural change but with fewer samples per state; longer windows are more stable but slower to reflect a genuine regime shift. |
Min Samples Per State (minSamples) | 12 | Minimum row-total transitions required before the persistence probability for that state is trusted. Below this, the gate reports LOW SAMPLE rather than a number. |
Min Persistence Probability (persistThr) | 55% | Diagonal (self-transition) probability required for the confidence gate to pass. This is the single highest-leverage tuning input — see Section 5.3. |
4.3 Display
Input | Default | Effect |
Show HUD | On | Displays the regime dashboard table (state, sample size, probabilities, gate status). |
Color Background by Regime | On | Shades the pane background by current state — green (BULL), red (BEAR), gray (CHOP) — for at-a-glance regime awareness. |
HUD Position | Top Right | Placement of the dashboard table on the pane. |
5. Best Practices
5.1 Sizing the Rolling Window to the Instrument
The default 90-day window assumes a liquid instrument with reasonably stationary regime behavior over a quarter. Adjust as follows:
• Higher-volatility or narrative-driven instruments (single-name equities around earnings, altcoins): shorten the window (45–60 days) — regime character changes faster and a 90-day matrix will lag genuine shifts.
• Slower macro instruments (major FX majors, broad indices): the default 90 days, or lengthen to 120–150, is generally more stable and less prone to noise-driven regime flips.
• Never shorten the window below roughly 3× minSamples per expected state — a window that can't accumulate an adequate sample for its own gate defeats the point of the gate.
5.2 Calibrating minSamples — Don't Set It for Convenience
It is tempting to lower minSamples when the gate frequently reports LOW SAMPLE. Resist this. A low sample count reporting LOW SAMPLE is the filter doing its job — it means the current regime state genuinely hasn't recurred often enough in the window to estimate its persistence reliably. The correct responses, in order of preference, are: (1) widen lookbackN, (2) accept that some regimes are legitimately rare and trade smaller/without this confirmation layer when they occur, (3) only as a last resort, lower minSamples with the explicit understanding that the resulting probability carries wider real uncertainty than displayed.
5.3 Calibrating persistThr Through Walk-Forward Review, Not Curve-Fitting
persistThr is the input most likely to be tuned toward a specific backtest's historical performance — precisely the input most vulnerable to curve-fitting. Two practices reduce this risk:
1. Set it from first principles first: 55% is chosen because it is the minimum probability at which persistence is more likely than the combined probability of the other two states, given a 3-state system. Treat this as the baseline, not a parameter to be swept for the best-looking equity curve.
2. If adjusting, validate walk-forward — evaluate the setting on a period after the one used to select it, not the same period. A threshold that only works in-sample is not a regime filter; it is a fitted parameter wearing one.
5.4 Wiring Into the PSRC Daily Bias Dashboard
The intended integration point is PSRC's existing confOK gate inside the Buy/Sell Confluence Zone Engine. Copy the block marked MARKOV REGIME ENGINE — COPY BLOCK in the filter source into the PSRC dashboard script, then modify the confidence gate as follows:
// Before:confOK = not requireOKConf or confLabel != "LOW" // After — requires BOTH overnight-range confidence AND regime persistence:confOK = not requireOKConf or (confLabel != "LOW" and markovConfidenceOK)
For a stricter variant that additionally requires the persisting regime to match the direction of the PSRC bias score (not merely persist, regardless of direction), gate on markovBullishGate / markovBearishGate instead:
rawActiveBias = (score >= minAbsScore and confOK and markovBullishGate) ? 1 : (score <= -minAbsScore and confOK and markovBearishGate) ? -1 : 0
This second variant will materially reduce trade frequency — it rejects any setup occurring during a CHOP regime or a persisting regime of the opposite direction. Test both variants independently before choosing; the correct choice depends on the desk's tolerance for frequency versus the strictness of confirmation required.
5.5 Alerting Workflow
Four alert conditions are exposed: a unified regime-shift alert, directional (BULL/BEAR) shift alerts, and a confidence-gate-opened alert restricted to shifts where the sample and persistence conditions both pass. For a desk running discretionary confirmation on top of the automated dashboard, subscribe to the gate-opened alert rather than the raw shift alert — a raw regime shift with an inadequate sample is not, by itself, actionable information.
5.6 Multi-Instrument Deployment
Each instance of the indicator maintains its own independent state history and transition matrix, scoped to the chart it is applied to. When deploying across a watchlist, do not assume a threshold calibrated on one instrument transfers cleanly to another with different volatility character — recalibrate erThresh and persistThr per instrument class (Section 5.1 applies equally here) rather than applying one global default across dissimilar assets.
Access the Markov Regime Whitepaper
6. Known Limitations
Stated directly, not buried in a disclaimer footer:
• Three-state ER classification is a coarse proxy for regime, not a fitted emission model. It will misclassify genuinely transitional or whipsaw days — this is an inherent property of a deterministic classification rule, not a bug to be patched.
• Session boundary mismatch: request.security_lower_tf("D", …) uses the instrument's native exchange daily session, not the NY-00:00 anchor used elsewhere in the PSRC dashboard. Immaterial to a 10-day-plus ER window; would need a custom session reconstruction to fully align, which has not been built as the added complexity is not justified for this component.
• A first-order Markov chain assumes transition probability depends only on the current state, not on how long the market has already been in it. Real regimes often have duration-dependent behavior (a trend three months old behaves differently than one three weeks old) that this model does not capture.
• Regime state count (3) is a modeling choice, not a discovered truth. It has not been benchmarked against 2-state or 4-state alternatives for every instrument class; the desk should treat this as a reasonable default, not a proven optimum.
• As with any regime model, the filter is a lagging confirmation tool — it requires several observations of a new pattern before the transition matrix reflects it. It is not, and should not be used as, a leading indicator of regime change.
7. Glossary
Term | Definition |
Efficiency Ratio (ER) | Kaufman's measure of trend efficiency: net directional displacement divided by total path length over a lookback window. |
Transition Matrix | A table of empirically estimated probabilities of moving from each regime state to each other state over one time step. |
Persistence Probability | The diagonal entry of the transition matrix row for the current state — the probability the current regime repeats. |
Confidence Gate | The composite boolean (markovConfidenceOK) requiring both adequate sample size and persistence probability above threshold. |
Non-repainting | A calculation that, once displayed on a historical bar, will never change value on subsequent bars or reloads. |
Look-ahead bias | The (avoided) use of information not yet available at the time a historical calculation is displayed. |
Walk-forward validation | Testing a parameter on data after the period used to select it, to detect overfitting to the selection period. |
8. Change Log
Version | Date | Change |
1.0 | 2026-07-25 | Initial release. Discrete 3-state Markov chain regime classifier with rolling transition matrix, sample-size and persistence-threshold confidence gate. Daily-bar-native via request.security_lower_tf() for full timeframe independence, with confirmed-bar repaint guard. |
Access the Markov Regime Whitepaper
Comments