factrix.metrics.oos_decay

Out-of-sample (OOS) persistence analysis for any time-indexed series.

This tool is agnostic to what the series represents — it only knows about IS/OOS splits on a time-indexed numeric sequence.

Notes

Pipeline. Time-series only, IS/OOS window split on a 1-D series; descriptive decay diagnostic (no formal H_0). oos_decay is the single-split primitive; oos_decay_splits is the robustness workflow that runs it over a pre-declared set of split fractions.

Input. DataFrame with date, value (IC series, CAAR series, spread series), one row per period.

Output. MetricResult with value = survival ratio + sign-flip / status detail in metadata.

factrix.metrics.oos_decay.oos_decay

oos_decay(series: DataFrame, value_col: str = 'value', is_ratio: float = 0.7, survival_threshold: float = 0.5, *, expected_warnings: tuple[str, ...] = ()) -> MetricResult

Single-split out-of-sample (OOS) survival ratio with sign-flip detection.

Splits the sorted series at is_ratio (IS = first is_ratio * n rows, OOS = remainder), computes |mean_OOS| / |mean_IS| (the survival ratio), and checks for an IS/OOS sign flip.

Parameters:

Name	Type	Description	Default
series	DataFrame	DataFrame with date and value_col, carrying exactly one row per period on its own distinct-date grid (what every producer in the DAG emits). Row order is irrelevant — the series is sorted by date here.	required
value_col	str	Numeric column to evaluate. Null, NaN and ±inf observations are dropped and the drop is recorded in metadata; the periods that remain are the split's grid.	'value'
is_ratio	float	Fraction of the retained periods allocated to IS (default 0.7). Must lie strictly inside (0, 1).	0.7
survival_threshold	float	Minimum survival ratio for status="PASS" (default 0.5). It is the share of the in-sample mean magnitude the factor must retain out of sample, so its domain is the finite half-open interval (0, 1].	0.5

Raises:

Type	Description
UserInputError	is_ratio is not strictly inside (0, 1), or survival_threshold is not inside (0, 1] (bool, NaN, ±inf and non-numeric included). Raised at construction, before any data work.
UserInputError	series carries more than one row for some date. Raised at call time, before the split.

Returns:

Type	Description
MetricResult	MetricResult with:
MetricResult	value: survival ratio |mean_oos| / |mean_is|. NaN on a
MetricResult	short-circuit, and NaN when |mean_is| ~ 0: the ratio is an
MetricResult	undefined 0/0 there, not a survival of zero. Reporting 0.0
MetricResult	VETOED a factor with a large out-of-sample mean as "fully
MetricResult	decayed" when the honest reading is "in-sample carried no signal
MetricResult	to decay from". That case raises
MetricResult	WarningCode.DEGENERATE_VARIANCE and keeps status="VETOED"
MetricResult	— the gate must not read "cannot assess" as "passed".
MetricResult	stat: None — descriptive only (no hypothesis test
MetricResult	attached; a t-stat at MIN_OOS_PERIODS_HARD = 5 would have power
MetricResult	~ 0 and would invite mis-reading the diagnostic as a
MetricResult	significance test)
MetricResult	metadata: sign_flipped (bool) status ("PASS" | "VETOED") is_ratio (float) mean_is (float) mean_oos (float) survival_threshold (float) reason (str, short-circuit only): "insufficient_oos_periods"

Notes

One call is one split. A regime change landing near the cut point can reverse this gate, so for a fraction-robust read use :func:oos_decay_splits, which runs this same primitive over a set of fractions the caller declares up front, purges the overlapping in-sample tail, and reports one aggregate verdict plus every individual split. A caller-side loop over is_ratio does neither: the overlapping-horizon leakage at each boundary stays unpurged, and the rule that turns the per-split ratios into one verdict is never declared.

Descriptive only — no p_value is emitted.

Split validity. is_ratio must be strictly inside (0, 1) and is validated up front: is_ratio=1.0 used to produce an empty OOS slice whose polars mean() is None, and float(None) then raised a bare TypeError from deep inside the metric. Beyond that, the min_periods floor bounds the series length, not the split, so an extreme ratio can still leave one side with fewer than two observations; that short-circuits with the usual reason="insufficient_oos_periods" rather than reporting a survival ratio computed from a single point.

Input contract. The split is a count of periods on the series' own distinct-date grid, taken positionally after sorting by date, so the series must carry one row per period. A duplicated date has no defined position in that sort and can put the same chronological period on both sides of the boundary; it is rejected rather than aggregated under a guessed rule (see :class:UserInputError above). Non-finite observations are dropped first, so n_obs and the split index both count the periods that actually reached the means.

Threshold domain. survival_threshold is validated as a finite fraction inside (0, 1]. Outside that range the knob stops gating and starts forcing: <= 0 PASSes every series a ratio exists for, float("nan") VETOES every one of them, and True silently reads as 1.0. A threshold above 1 would demand out-of-sample amplification rather than survival — read value directly for that question.

References

• McLean-Pontiff (2016): post-publication returns ~58% lower than in-sample, with ~32% of that drop attributable to publication itself (the remaining ~26% is the pure out-of-sample decay).
• Lopez-de-Prado (2018): CPCV for robust train/test split.

Examples:

Survival on a per-date information coefficient (IC) series from :func:~factrix.metrics.ic.compute_ic:

>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics.ic import compute_ic
>>> from factrix.metrics.oos_decay import oos_decay
>>> panel = compute_forward_return(
...     fx.datasets.make_cs_panel(n_assets=80, n_dates=240, rng=0),
...     forward_periods=5,
... )
>>> series = compute_ic(panel)["factor"].rename({"ic": "value"}).select("date", "value")
>>> result = oos_decay(series)
>>> result.name == ""
True

factrix.metrics.oos_decay.oos_decay_splits

oos_decay_splits(series: DataFrame, value_col: str = 'value', split_fractions: Sequence[float] = DEFAULT_SPLIT_FRACTIONS, survival_threshold: float = 0.5, forward_periods: int = DEFAULT_FORWARD_PERIODS, *, expected_warnings: tuple[str, ...] = ()) -> MetricResult

Purged out-of-sample (OOS) survival over a pre-declared set of splits.

Robustness validation of :func:oos_decay, not a second search over it. A single cut point can reverse the single-split gate when a regime change lands near it; this runs the same primitive at every fraction in a set the caller declares before seeing any result, and reports one aggregate verdict plus every individual split.

Parameters:

Name	Type	Description	Default
series	DataFrame	DataFrame with date and value_col, one row per period on its own distinct-date grid — the same contract :func:oos_decay enforces.	required
value_col	str	Numeric column to evaluate.	'value'
split_fractions	Sequence[float]	The declared cut points, each strictly inside (0, 1) and distinct (default (0.6, 0.7, 0.8)). Order does not matter: the set is canonicalised ascending, and the aggregate is order-invariant.	DEFAULT_SPLIT_FRACTIONS
survival_threshold	float	Minimum aggregate survival ratio for status="PASS" (default 0.5), in (0, 1]. Passed through unchanged to each split, so the per-split statuses read against the same bar.	0.5
forward_periods	int	Length in periods of the forward-return window the series values were built from (default :data:~factrix._types.DEFAULT_FORWARD_PERIODS). This is the purge gap: the last forward_periods periods of each in-sample window are dropped, so no in-sample observation's return window reaches into the out-of-sample side.	DEFAULT_FORWARD_PERIODS
expected_warnings	tuple[str, ...]	Declared warning codes, passed to each split.	()

Raises:

Type	Description
UserInputError	split_fractions is empty, carries a duplicate, or carries a value outside (0, 1); survival_threshold is outside (0, 1]; forward_periods is not an integer >= 1; or series carries more than one row for some date.

Returns:

Type	Description
MetricResult	MetricResult with:
MetricResult	value: the median survival ratio across the declared
MetricResult	splits. NaN when any declared split could not be assessed — the
MetricResult	aggregate is defined over the set that was declared, not over
MetricResult	whichever subset happened to compute, and a silently narrowed
MetricResult	sample would read as a verdict.
MetricResult	n_obs: periods that entered the sweep (after the non-finite
MetricResult	drop), on the "periods" axis.
MetricResult	stat: None — descriptive, like the primitive. No
MetricResult	p_value is emitted, so there is no multiplicity to correct.
MetricResult	metadata: status ("PASS" | "VETOED") splits: one record per declared fraction, ascending, each with split_fraction, n_is, n_oos, purge_periods, survival, sign_flipped, status, and (where the primitive emitted them) mean_is, mean_oos, reason, signal_status split_fractions (tuple), n_splits, n_assessable, n_sign_flips aggregate ("median"), sign_flip_policy ("any_flip_vetoes") forward_periods, purge_periods, survival_threshold reason (str, withheld aggregate only): "unassessable_splits"

Notes

Aggregate rule. status="PASS" requires all three of: every declared split assessable, no split sign-flipped, and the median survival ratio at or above survival_threshold. Direction and magnitude are aggregated differently on purpose — a sign flip says the factor predicts the wrong way over some tail, which a majority vote would launder, while a magnitude below the bar at one cut point is exactly the arbitrariness the median is here to absorb. The median is fixed rather than exposed as a knob: choosing an aggregate after seeing the per-split ratios is the search this workflow closes off.

value is still reported when the gate vetoes on a sign flip — it is the ratio that ran, and the veto is recorded in status and n_sign_flips rather than by corrupting the number.

Purge. forward_periods periods are dropped off the end of each in-sample window, counted on the series' own distinct-date grid after the non-finite drop (never calendar time) — the same grid the split index is taken on. Dropping observations only removes periods from that grid, so the resulting gap spans at least forward_periods periods of the producing panel's grid, and more wherever the band it covers had missing values: on a 104-period series with 4 non-finite observations inside the band, forward_periods=5 leaves a gap of 10 panel periods. Purging wider than the horizon is safe — it only removes in-sample observations — where purging narrower would leave the leakage the gap exists for. A value stamped at period t built from a forward_periods-period forward return is realised over (t, t + forward_periods], so without the gap the last in-sample observations are partly realised inside the out-of-sample window — the boundary leakage Lopez-de-Prado (2018) purges. The gap comes off the in-sample side only; the out-of-sample window is what is being validated and is never shortened for it. A series with no forward window (a contemporaneous spread) still passes forward_periods=1, which drops one period — conservative, and cheaper than a knob that turns the guard off.

Multiplicity. This is robustness validation, not an additional model search, and the distinction is what keeps it honest. The split set is declared up front, every split is reported, the aggregate is fixed, and no p_value is emitted anywhere — so there is nothing to correct and no best split to select. Re-running with different fraction sets until one passes converts it into an uncorrected search; the pre-declaration is the only thing preventing that, and the library cannot enforce it for you.

References

• Lopez-de-Prado (2018): purging and embargoing overlapping train/test windows; CPCV.
• McLean-Pontiff (2016): post-publication decay the survival_threshold default is calibrated against.

Examples:

>>> import factrix as fx
>>> from factrix.metrics.ic import compute_ic
>>> from factrix.metrics.oos_decay import oos_decay_splits
>>> from factrix.preprocess import compute_forward_return
>>> panel = compute_forward_return(
...     fx.datasets.make_cs_panel(n_assets=80, n_dates=240, rng=0),
...     forward_periods=5,
... )
>>> series = compute_ic(panel)["factor"].select("date", "ic")
>>> result = oos_decay_splits(series, value_col="ic", forward_periods=5)
>>> result.metadata["split_fractions"]
(0.6, 0.7, 0.8)
>>> len(result.metadata["splits"])
3

---

Descriptive only — no formal \(H_0\)

oos_decay emits a survival ratio + sign-flip detail; no p_value is attached and stat is None. A \(t\)-test at the MIN_OOS_PERIODS_HARD floor would have power \(\approx 0\) and would invite mis-reading the diagnostic as a significance test. Callers routing this output into Benjamini-Hochberg-Yekutieli (BHY) / gate logic must read status ("PASS" / "VETOED") and sign_flipped, not a probability.

Use cases

• Persistence read on a factor-return series

  ---

  oos_decay is a standalone series diagnostic — input is a 1-D (date, value) series, typically information coefficient (IC) from compute_ic, spread from compute_spread_series, or any other factor-mimicking-portfolio return series. Reports \(|\mathrm{mean}_{\text{OOS}}| / |\mathrm{mean}_{\text{IS}}|\) on a single is_ratio split.

• One row per period, finite values only

  ---

  The split is a count of periods on the series' own distinct-date grid, taken positionally after a sort by date. A duplicated date has no defined position in that sort, so it could put the same chronological period on both sides of the boundary; it raises UserInputError rather than being aggregated under a rule the caller never chose. Null / NaN / \(\pm\infty\) observations are dropped first and recorded in metadata, so n_obs and the split index count the same periods.

• survival_threshold is a retention fraction

  ---

  Its domain is the finite half-open interval \((0, 1]\), validated at construction. Outside it the knob stops gating and starts forcing: \(\le 0\) passes every series a ratio exists for, float("nan") vetoes every one of them, and True reads as 1.0. A threshold above 1 would demand out-of-sample amplification rather than survival — read value directly for that question.

• Sign-flip veto

  ---

  Any split with opposite-signed IS and out-of-sample (OOS) means flips sign_flipped = True and forces status = "VETOED" — IC sign-flip OOS means the factor predicts the wrong direction, not just a weaker one. McLean & Pontiff (2016) report average OOS decay around 32 %; factrix's default survival_threshold = 0.5 sits inside that window.

• Sweep the split fraction with oos_decay_splits

  ---

  One oos_decay call is one is_ratio, and a regime change landing near that cut point can reverse the gate. oos_decay_splits runs the same primitive over a set of fractions the caller declares up front (default (0.6, 0.7, 0.8)), purges the overlapping in-sample tail, and reports one aggregate verdict plus every individual split.

Robustness sweep — oos_decay_splits

Robustness validation, not another model search

The split set is declared before the sweep runs, every split is reported, the aggregate rule is fixed, and no \(p\)-value is emitted anywhere — so there is nothing to correct for and no best split to select. Re-running with different fraction sets until one passes turns it back into an uncorrected search. The pre-declaration is the only thing preventing that, and the library cannot enforce it for you.

Aggregate rule. status = "PASS" requires all three of:

1. every declared split assessable (a split that could not be computed withholds the aggregate: value is NaN, reason = "unassessable_splits" — "cannot assess" must not read as "passed");
2. no split sign-flipped;
3. the median survival ratio at or above survival_threshold.

Direction and magnitude are aggregated differently on purpose. A sign flip says the factor predicts the wrong way over some contiguous tail, which the single-split primitive already treats as a hard veto — a majority vote over splits would launder it. A magnitude below the bar at one cut point is exactly the arbitrariness the median is there to absorb (50 % breakdown point, and order-invariant, so the declaration order cannot move the verdict). The median is fixed rather than exposed as a knob: choosing an aggregate after seeing the per-split ratios is the search this closes off. When the gate vetoes on a sign flip, value still carries the median that ran — the veto lives in status and n_sign_flips, not in a corrupted number.

Purge gap. A value stamped at period \(t\) built from a forward_periods-period forward return is realised over \((t,\, t + \texttt{forward\_periods}]\), so the last in-sample observations are partly realised inside the out-of-sample window. oos_decay_splits drops forward_periods periods off the end of each in-sample window — counted on the series' own distinct-date grid after the non-finite drop, never calendar time — which is the purge of Lopez de Prado (2018). Because that drop only removes periods, the resulting gap spans at least forward_periods periods of the producing panel's grid (a 104-period series with 4 non-finite observations inside the band gives a 10-panel-period gap at forward_periods=5). Purging wider than the horizon is safe; purging narrower would leave the leakage the gap exists for. The gap comes off the in-sample side only: the out-of-sample window is the thing being validated and is never shortened to protect the window it is validated against.

Illustrative

from factrix.metrics import oos_decay_splits

out = oos_decay_splits(
    ic_df, value_col="ic", split_fractions=(0.6, 0.7, 0.8), forward_periods=5,
)
print(out.value, out.metadata["status"])           # median ratio, aggregate gate
for split in out.metadata["splits"]:               # provenance, ascending
    print(split["split_fraction"], split["n_is"], split["n_oos"],
          split["survival"], split["sign_flipped"], split["status"])

Choosing a function

Goal	Function
Single-split OOS survival + sign-flip gate on a (date, value) series	oos_decay
Purged survival across a pre-declared set of splits, with provenance	oos_decay_splits

Worked example — IC series fed into oos_decay

compute_ic → oos_decay

import factrix as fx
from factrix.metrics.ic import compute_ic
from factrix.metrics.oos_decay import oos_decay
from factrix.preprocess import compute_forward_return

raw   = fx.datasets.make_cs_panel(
    n_assets=100, n_dates=1000, ic_target=0.08, rng=2024,
)
panel = compute_forward_return(raw, forward_periods=5)

# The series diagnostic consumes (date, value); the value column on
# the compute_ic output is named ``ic``.
ic_df = compute_ic(panel)["factor"]
out   = oos_decay(ic_df, value_col="ic")
print(out.value, out.metadata["status"], out.metadata["sign_flipped"])
# 0.93   PASS   False   (approximate)
print(out.metadata["is_ratio"],
      out.metadata["mean_is"], out.metadata["mean_oos"])
# 0.7   0.0771   0.0719   (approximate)

# One call is one split. For a fraction-robust read, declare the split
# set up front and let oos_decay_splits purge and aggregate it.
from factrix.metrics import oos_decay_splits

swept = oos_decay_splits(ic_df, value_col="ic", forward_periods=5)
print(swept.value, swept.metadata["status"], swept.metadata["n_sign_flips"])

See also

• compute_ic / compute_spread_series

  ---

  Canonical producers of the (date, value) series this diagnostic consumes.

  api/metrics/ic →

• positive_rate / trend

  ---

  Sibling series diagnostics on the same input shape — sign significance and slope detection. Pair with oos when both in-sample magnitude and out-of-sample persistence matter.

  api/metrics/positive_rate →

• by_slice

  ---

  Per-slice survival summaries (regime / universe / sector).

  api/by-slice →

• Metric applicability reference

  ---

  When this metric applies and the sample-size guards that gate it (MIN_OOS_PERIODS_HARD * 2 floor; per-split MIN_OOS_PERIODS_HARD on each side).

  reference/metric-applicability →

• Series diagnostics landing

  ---

  Adjacent axis-agnostic series diagnostics.

  api/metrics/series-tools →