Input contracts: evaluate consumes an evaluation panel with the four-column floor documented in Data schema. evaluate_horizons consumes a raw canonical panel with price and factor columns, then computes forward_return internally for each horizon. Scalar post-processing helpers such as breakeven_cost and net_spread are direct-call utilities, not evaluate() metrics.

factrix.evaluate

evaluate(data: DataInput, *, price_data: DataInput | None = None, metrics: dict[str, MetricBase], factor_cols: list[str], forward_periods: int | None = None, overlap_periods: int | None = None, strict: bool = True, expected_warnings: tuple[str, ...] = ()) -> dict[str, EvaluationResult]

Evaluate one or more factors against forward returns through the DAG executor.

Closed-set DAG dispatch — every spec referenced by another spec's requires is auto-pulled into the executor, batched stage-1 producers run once across the whole factor batch (IC's compute_ic etc.), and per-factor consumers run once per factor.

Parameters:

Name	Type	Description	Default
data	DataInput	Long-format data satisfying the four-column floor (date, asset_id, <factor_col>, forward_return). The three fixed-name columns (date, asset_id, forward_return) are validated eagerly; forward_return must already be attached via :func:factrix.preprocess.compute_forward_return. price is optional — consumed by event-study metrics which short-circuit to NaN when it is absent.	required
price_data	DataInput | None	Optional complete price panel containing exactly alignable date, asset_id, and price columns. Use the raw panel here when event path metrics must reach prices that :func:factrix.preprocess.compute_forward_return dropped from data (its forward-return tail, or dates outside a coarser evaluation grid). data remains the sole owner of events and the forward-return sample; extra columns on price_data are ignored. Key dtypes must match data exactly.	None
metrics	dict[str, MetricBase]	dict[str, Metric] mapping a caller-chosen label to a metric instance from :mod:factrix.metrics (e.g. {"ic_5d": ic(), "spread": quantile_spread(n_groups=5)}). Results key by these labels. Passing the bare class (ic rather than ic()), a str or a :class:MetricSpec is rejected with a targeted error. One metric class may run under several labels with different config — e.g. {"spread5": quantile_spread(n_groups=5), "spread10": quantile_spread(n_groups=10)} — via by-value DAG dedup; shared upstream producers are computed once per distinct config. forward_periods is not a metric knob: every metric runs at the data's single return horizon. To compare horizons, use :func:evaluate_horizons on the clean raw panel. Scalar-input helpers (for example breakeven_cost / net_spread) are post-processing utilities, not panel-evaluation metrics; compute their upstream diagnostics first and call them directly.	required
factor_cols	list[str]	Names of factor columns on data. List-only — single str is rejected. Non-empty, no duplicates, every name must exist on data.	required
forward_periods	int | None	The data's return horizon — the forward_periods forward_return was built with, in periods of the price grid. Normally omitted — :func:factrix.preprocess.compute_forward_return stamps it on the panel and it is read from there. Pass it only to declare the horizon for a self-attached forward_return column that carries no stamp; a value disagreeing with the stamp is rejected. Stamped on every :class:EvaluationResult as forward_periods; it names the hypothesis and never changes with the evaluation grid.	None
overlap_periods	int | None	The overlap of adjacent observations on the evaluation grid — the quantity inference consumes (HAC bandwidth and effective df, non-overlapping stride, stride-scaled floors). Normally omitted — compute_forward_return stamps it too: equal to forward_periods on the full grid, derived on a coarser grid passed via dates=. For an unstamped self-attached panel it defaults to forward_periods; pass it only when the panel was built on a coarser grid than the horizon. A value disagreeing with the stamp is rejected. Stamped on every :class:EvaluationResult as overlap_periods (bookkeeping, not identity) and injected into each metric (surfaced in result.metrics[label].metadata["overlap_periods"]).	None
strict	bool	When True (default), raise if a metric is incompatible with the factor cell (scope / density / structure) or if a metric that fits the data could not produce a value — a data shortage (insufficient_*) or a missing input / config (no_*). A not_applicable* type-routing verdict (the metric's type does not fit this factor, e.g. a continuous-magnitude metric on a discrete ±k signal) is not a strict failure even under True: it surfaces as a NaN MetricResult with is_applicable=False and metadata["reason"] while the applicable metrics in the same call still return — so a mixed battery is not aborted by one inapplicable metric. When strict=False, every such case (including data shortages and cell mismatches) is kept as a NaN output with attached warnings instead of raising. Config-time / construct-time failures always raise.	True
expected_warnings	tuple[str, ...]	:class:~factrix.WarningCode values (e.g. ("few_assets",)) declaring warning regimes that are the study's design, not an accident — a single-asset or pairs study has no cross-section by construction, so its every run trips few_assets. Declared codes are marked, never dropped: each matching :class:Warning record is kept with expected=True (read the alert view via result.unexpected_warnings), so nothing leaves the audit trail — the declaration says "this regime is expected", not "pretend it is absent". Inference is untouched (the method that ran stays readable in metadata["method"]); only the human-facing channels go quiet — the per-run UserWarning echoes stop and repr emphasis moves to unexpected warnings. Unknown codes are rejected (typo guard). Default () — behavior is completely unchanged unless declared. Not a per-metric knob: constructing a metric with expected_warnings= is rejected; the study-level declaration here applies to every metric in the call.	()

Returns:

Type	Description
dict[str, EvaluationResult]	dict[str, EvaluationResult] keyed by factor column name, in
dict[str, EvaluationResult]	factor_cols insertion order. Each value carries the full
dict[str, EvaluationResult]	per-metric outputs, panel structural stats (n_periods,
dict[str, EvaluationResult]	n_pairs), and warnings for that factor.

Raises:

Type

Description

UserInputError

metrics not a dict[str, Metric] of instances; factor_cols empty / single str / contains duplicates / names a reserved stamp column / references a column not on data; data missing a baseline column; a metric requires a producer absent from the registry; under strict=True, a metric inapplicable to the data; data carries no horizon stamp and none is declared via forward_periods, or the declared forward_periods disagrees with the stamp.

Examples:

Single-factor IC + IC information ratio (IR):

>>> import factrix as fx
>>> from factrix.metrics import ic, ic_ir
>>> raw = fx.datasets.make_cs_panel(n_assets=15, n_dates=80)
>>> data = fx.preprocess.compute_forward_return(raw, forward_periods=5)
>>> results = fx.evaluate(
...     data,
...     metrics={"ic": ic(), "ic_ir": ic_ir()},
...     factor_cols=["factor"],
...     forward_periods=5,
... )
>>> "ic" in results["factor"].metrics
True
>>> "ic_ir" in results["factor"].metrics
True

---

factrix.evaluate_horizons

evaluate_horizons(data: DataInput, *, metrics: dict[str, MetricBase], factor_cols: list[str], forward_periods: list[int], dates: Series | Iterable[object] | None = None, strict: bool = True, expected_warnings: tuple[str, ...] = ()) -> list[EvaluationResult]

Sweep evaluate across several return horizons of one raw panel.

A thin composition over the existing primitives — for each horizon it rebuilds the panel with :func:factrix.preprocess.compute_forward_return and runs a single :func:evaluate, then flattens the per-factor results into one list. No new type is introduced and the single-horizon contract of evaluate is untouched: every inner run still evaluates one panel at one stamped horizon.

The horizon must be rebuilt from the raw panel for each value — compute_forward_return is not idempotent (it drops the last forward_periods + 1 rows per asset and stamps the horizon), so a horizon cannot be re-derived from an already-attached panel. This wrapper exists to make that rebuild-per-horizon loop hard to get wrong.

The raw panel is also forwarded to each inner :func:evaluate call as price_data. Event offsets and MFE/MAE windows therefore retain the complete price grid even though each horizon's forward-return panel has a shorter tail (and may use a coarser evaluation grid via dates=).

Identity of a swept result is the composite (factor, forward_periods), not a unique scalar factor key — so the return is a flat list[EvaluationResult] (the native shape of the aggregation layer), not the factor-keyed dict that evaluate returns at a fixed horizon. factor and forward_periods are existing native attributes of :class:EvaluationResult; the list feeds straight into :func:compare and into :func:factrix.multi_factor.bhy. Pool all horizons when selection may choose across them; use expand_over=('forward_periods',) only for predeclared horizon-specific screens that are selected and reported separately.

Parameters:

Name	Type	Description	Default
data	DataInput	A raw panel carrying date, asset_id, price and the factor columns, without a forward_return column — it is rebuilt per horizon. Passing an already-attached panel is rejected by compute_forward_return. Only forward-return is computed; winsorize / abnormal-return are out of scope, build those panels and call :func:evaluate per horizon by hand.	required
metrics	dict[str, MetricBase]	Same contract as :func:evaluate — a dict[str, Metric] of metric instances. Applied identically at every horizon.	required
factor_cols	list[str]	Same contract as :func:evaluate. Every column is evaluated at every horizon; the flat result has one entry per (factor, horizon).	required
forward_periods	list[int]	The horizons to sweep, as a non-empty list[int] of distinct positive panel-period counts (e.g. [5, 20, 60]). Duplicates are rejected — they would yield a duplicate (factor, forward_periods) identity that compare / bhy reject downstream.	required
dates	Series | Iterable[object] | None	Optional evaluation grid, forwarded unchanged to each inner :func:factrix.preprocess.compute_forward_return call. Every horizon is then evaluated on the same caller-chosen dates and each result carries its own derived overlap_periods; the (factor, forward_periods) identity is unaffected.	None
strict	bool	Forwarded unchanged to each inner :func:evaluate.	True
expected_warnings	tuple[str, ...]	Forwarded unchanged to each inner :func:evaluate — the declaration is a property of the study, so it applies identically at every horizon.	()

Returns:

Type	Description
list[EvaluationResult]	Flat list[EvaluationResult] grouped by horizon (outer) then
list[EvaluationResult]	factor_cols order (inner). Each entry carries its own stamped
list[EvaluationResult]	forward_periods.

Raises:

Type	Description
UserInputError	forward_periods is not a non-empty list[int] of distinct positive values; plus any error raised by the inner :func:evaluate / compute_forward_return (e.g. data already carries forward_return).

Notes

Comparability across horizons is a scale alignment, not a free lunch: compute_forward_return divides by forward_periods so rank-IC is directly comparable across horizons, but signed-return-mean metrics carry a compounding bias that grows with forward_periods (see :func:factrix.preprocess.compute_forward_return Notes). Treat a cross-horizon sweep of signed-mean metrics as descriptive.

Examples:

>>> import factrix as fx
>>> from factrix.metrics import ic
>>> raw = fx.datasets.make_cs_panel(n_assets=20, n_dates=300)
>>> results = fx.evaluate_horizons(
...     raw,
...     metrics={"ic": ic()},
...     factor_cols=["factor"],
...     forward_periods=[5, 10, 20],
... )
>>> [r.forward_periods for r in results]
[5, 10, 20]
>>> board = fx.compare(results, metrics=["ic"])  # one row per horizon
>>> board.height
3

---

Full price data for event paths

compute_forward_return returns the evaluation panel: it removes rows whose forward return cannot be formed and, with dates=, keeps only the chosen evaluation dates. Those rows are the correct sample for return metrics, but they are not a complete price path. A later event offset or MFE/MAE window can need prices from the removed tail or from dates between evaluations.

Keep the three grids separate by passing the original panel as price_data:

import factrix as fx

raw = fx.datasets.make_event_panel(n_assets=50, n_dates=400, rng=7)
panel = fx.preprocess.compute_forward_return(raw, forward_periods=5)

result = fx.evaluate(
    panel,
    price_data=raw,
    metrics={
        "path": fx.metrics.event_around_return(offsets=[-1, 6, 12, 24]),
        "excursion": fx.metrics.mfe_mae(),
    },
    factor_cols=["factor"],
    strict=False,
)["factor"]

Input/grid	Owns	Never contributes to
price_data price grid	Complete per-asset prices used by offsets, excursion windows, and the event baseline	Forward-return samples, sample floors, or factor routing
data evaluation grid	Eligible event dates and the rows selected by dates=	Prices outside the returned panel
Finite data.forward_return rows	Return estimates and their effective sample	Extending an event price path

price_data must contain unique (date, asset_id) keys plus price; its key dtypes must exactly match data. Extra columns are ignored, so the event and factor authority cannot silently move to the side panel. If price_data is omitted, event paths continue to use data.price.

Offsets and windows are counted on the grid that supplies them

offsets= and window= are counts of periods on the price grid the path walk reads. Without price_data that grid is data's own distinct dates; with price_data it is the price panel's. When the two grids differ, the same argument therefore spans a different amount of the sample, and the excursion or offset it measures changes with it.

Measured on a panel whose evaluation grid keeps every tenth period of the price grid, mfe_mae(window=3):

Grid the walk read	Median MFE
data evaluation grid (no price_data)	0.1272
price_data price grid	0.0120

Three evaluation periods reach ten times further than three price periods, so the coarser grid reports the larger excursion. Neither number is wrong; they answer different questions. State the grid you mean, and scale window= and offsets= to it.

Behaviour change (pre-1.0)

This is not an additive API. evaluate_horizons owns the raw panel and now forwards it as price_data on every call, so an existing evaluate_horizons run that includes event_around_return or mfe_mae changes value: event paths gain the tail and the between-evaluation periods that were previously unreachable, and offsets and windows are re-based onto the raw grid as above. Return-only metrics are unaffected.

Migrate by deciding which grid each event-path argument counts on:

• evaluate_horizons callers: re-read offsets= and window= against the raw panel's grid. Where the evaluation grid was the intended unit, pass the already-preprocessed panel to evaluate instead.
• evaluate callers who preprocess returns themselves: retain raw and add price_data=raw to get the complete paths; no change is needed for return-only metrics.

forward_periods= and overlap_periods=

Both are properties of the data, normally read from the stamps compute_forward_return leaves on the panel, and never per-metric knobs. They answer different questions:

Argument	Question	Read from	Declare it when
forward_periods	Over how many periods of the price grid was the return measured? The hypothesis; joins the (factor, forward_periods, *params) identity.	_forward_periods stamp	The panel carries a self-attached forward_return with no stamp.
overlap_periods	How many adjacent evaluation observations share future periods? What inference consumes.	_overlap_periods stamp (equal to the horizon on the full grid; derived from dates= on a coarser one)	The unstamped panel sits on a coarser evaluation grid than its horizon. Otherwise it defaults to forward_periods.

A value that disagrees with its stamp raises UserInputError rather than silently overriding the data. Both surface on the result — forward_periods as identity, overlap_periods as bookkeeping — and overlap_periods is injected into every metric as metadata["overlap_periods"]. See Evaluating on a coarser grid for why the two are kept apart.

Two more period counts the tradability metrics declare

On a full evaluation grid all four counts below are the same number, which is why one name used to carry all of them. They separate as soon as the evaluation grid is coarser than the return horizon.

Quantity	Question	Unit	Declared on
forward_periods	Economic horizon: over how many periods was the return measured?	Underlying period grid	compute_forward_return; stamped
overlap_periods	Inference: how many adjacent evaluation observations share future periods?	Evaluation-grid observations	Derived and stamped; injected, never a metric knob
rebalance_lag	Turnover: how far apart are the rankings / memberships compared?	Evaluation-grid observations	rank_turnover(), notional_turnover()
holding_periods	Cost amortisation: how many return periods pass between paying trading cost?	Underlying period grid	breakeven_cost(), net_spread()

rebalance_lag defaults to the injected overlap_periods — horizon-aligned turnover, the historical behaviour. Pass rebalance_lag=1 when the evaluation grid is the rebalance schedule. holding_periods has no default tie to the stamp: gross_spread is per underlying return period (compute_forward_return divides by forward_periods), so the cost must be amortised over underlying periods too. Substituting the derived evaluation-grid overlap there is a unit error, not a conservative choice — see the tradability migration note.

Use cases

• Single-factor significance

  ---

  One panel of factor data → one result carrying the mainstream metric's p_value and the cell-specific statistics.

• Batch screening with false discovery rate (FDR)

  ---

  Loop evaluate over candidate signal columns and feed the resulting EvaluationResult list to bhy for false-discovery-rate control. See Multi-factor FDR.

• Cross-cell apples-to-apples

  ---

  Compare information coefficient (IC) rank-ordering against Fama-MacBeth λ on the same panel, or individual-asset factors against broadcast macro factors. Return shape is identical across cells.

• TIMESERIES dispatch

  ---

  At n_assets == 1 there is no cross-section, so any DENSE metric whose cell is PANEL — Individual × Continuous (ic, fm_beta) and Common × Continuous (common_beta, common_quantile, common_asymmetry) — raises IncompatibleAxisError (or NaN + structure_mismatch under strict=False). Single-asset data runs through the same entry point with predictive_beta for dense predictive-regression slopes, sparse metrics whose cell wildcard allows TIMESERIES, and panel-input wildcard metrics such as directional_hit_rate. Two-column diagnostics (positive_rate, oos_decay, ic_trend) are standalone (date, value) tools; in evaluate() they layer on panel IC series, not raw single-asset dense panels.

Worked example — single-factor smoke test

Synthetic panel → evaluate → read value + p_value

Full runnable example complementing the doctest snippets in Examples above with realistic console output.

import factrix as fx
from factrix.metrics import ic, quantile_spread

# 1. Create dummy panel data
raw = fx.datasets.make_cs_panel(n_assets=15, n_dates=80)
data = fx.preprocess.compute_forward_return(raw, forward_periods=5)

# 2. Run evaluation
results = fx.evaluate(
    data,
    metrics={"ic": ic(), "spread": quantile_spread(n_groups=5)},
    factor_cols=["factor"],
    forward_periods=5,
)

# 3. Retrieve and inspect results
res = results["factor"]
print(f"Factor: {res.factor}")
print(f"Cell: {res.cell}")
print(f"Plan: \n{res.plan}")

# Access metrics result group
ic_res = res.metrics["ic"]
print(f"IC Value: {ic_res.value:.4f}")
print(f"IC p-value: {ic_res.p_value:.4f}")

Sensitivity grids

For exploratory grids across asset counts, horizons, or factor families, run with strict=False and stack each result's long-form table. The table carries is_applicable and reason, so the grid can keep running while still making failed metric/input combinations visible.

import polars as pl

results = fx.evaluate(
    data,
    metrics={"ic": ic(), "spread": quantile_spread(n_groups=2)},
    factor_cols=["factor"],
    forward_periods=5,
    strict=False,
)

status = pl.concat([r.to_frame() for r in results.values()])
failed = status.filter(~pl.col("is_applicable"))

Evaluating under different cell contexts

Metric behaviors are defined by instantiating metric classes directly. The DAG executor handles dispatch automatically depending on the cell registered by the metric.

Illustrative

import factrix as fx
from factrix.metrics import ic, caar, common_beta

# 1. Individual × Continuous (e.g. Information Coefficient)
results_ic = fx.evaluate(
    data,
    metrics={"ic": ic()},
    factor_cols=["factor"],
    forward_periods=5
)

# 2. Individual × Sparse (e.g. Event Study CAAR, requires a 'price' column)
results_caar = fx.evaluate(
    data_with_price,
    metrics={"caar": caar()},
    factor_cols=["event_factor"],
    forward_periods=5
)

# 3. Common × Continuous (e.g. Time-Series Beta)
results_common_beta = fx.evaluate(
    data,
    metrics={"common_beta": common_beta()},
    factor_cols=["macro_factor"],
    forward_periods=5
)

Per-cell required / optional columns and the DataStructure (PANEL vs TIMESERIES) derivation are automatically resolved at dispatch time.

Next steps

• Multi-factor FDR

  ---

  Wires evaluate into the multi-factor FDR pipeline: pass candidate results to BHY; choose between bhy / partial_conjunction / bhy_hierarchical; mixed-cell batches.

  Read the guide →

• Data schema

  ---

  New to the fixed-horizon input contract? Start here for the evaluation panel floor (date, asset_id, factor, forward_return), dtype semantics, and optional columns that activate extra metrics.

  Read the schema →

See also

• Common-factor regression conventions

  ---

  The n_assets == 1 dispatch rules and SE conventions for the per-asset time-series stage.

  reference/ts-mode-conventions →

• Panel vs timeseries sample guard

  ---

  Per-metric, per-axis sample floors on the effective sample, and the InsufficientSampleError recovery path (.axis / .actual / .required).

  guides/panel-timeseries →