Input contracts: evaluate consumes an evaluation panel with the four-column floor documented in Data schema. evaluate_horizons consumes a raw canonical panel with price and factor columns, then computes forward_return internally for each horizon. Scalar post-processing helpers such as breakeven_cost and net_spread are direct-call utilities, not evaluate() metrics.

factrix.evaluate ¶

evaluate(data: DataInput, *, metrics: dict[str, MetricBase], factor_cols: list[str], forward_periods: int | None = None, strict: bool = True, expected_warnings: tuple[str, ...] = ()) -> dict[str, EvaluationResult]

Evaluate one or more factors against forward returns through the DAG executor.

Closed-set DAG dispatch — every spec referenced by another spec's requires is auto-pulled into the executor, batched stage-1 producers run once across the whole factor batch (IC's compute_ic etc.), and per-factor consumers run once per factor.

Parameters:

Name	Type	Description	Default
`data`	`DataInput`	Long-format data satisfying the four-column floor `(date, asset_id, <factor_col>, forward_return)`. The three fixed-name columns `(date, asset_id, forward_return)` are validated eagerly; `forward_return` must already be attached via :func:`factrix.preprocess.compute_forward_return`. `price` is optional — consumed by event-study metrics which short-circuit to NaN when it is absent.	required
`metrics`	`dict[str, MetricBase]`	`dict[str, Metric]` mapping a caller-chosen label to a metric instance from :mod:`factrix.metrics` (e.g. `{"ic_5d": ic(), "spread": quantile_spread(n_groups=5)}`). Results key by these labels. Passing the bare class (`ic` rather than `ic()`), a `str` or a :class:`MetricSpec` is rejected with a targeted error. One metric class may run under several labels with different config — e.g. `{"spread5": quantile_spread(n_groups=5), "spread10": quantile_spread(n_groups=10)}` — via by-value DAG dedup; shared upstream producers are computed once per distinct config. `forward_periods` is not a metric knob: every metric runs at the data's single overlap horizon. To compare horizons, use :func:`evaluate_horizons` on the clean raw panel. Scalar-input helpers (for example `breakeven_cost` / `net_spread`) are post-processing utilities, not panel-evaluation metrics; compute their upstream diagnostics first and call them directly.	required
`factor_cols`	`list[str]`	Names of factor columns on `data`. List-only — single `str` is rejected. Non-empty, no duplicates, every name must exist on `data`.	required
`forward_periods`	`int \| None`	The data's overlap horizon (rows of the time axis). Normally omitted — :func:`factrix.preprocess.compute_forward_return` stamps it on the panel and it is read from there. Pass it only to declare the horizon for a self-attached `forward_return` column that carries no stamp; a value disagreeing with the stamp is rejected. Stamped on every :class:`EvaluationResult` as `forward_periods` and injected into each metric (surfaced in `result.metrics[label].metadata["forward_periods"]`).	`None`
`strict`	`bool`	When `True` (default), raise if a metric is incompatible with the factor cell (scope / density / structure) or if a metric that fits the data could not produce a value — a data shortage (`insufficient_`) or a missing input / config (`no_`). A `not_applicable` type-routing verdict (the metric's type* does not fit this factor, e.g. a continuous-magnitude metric on a discrete ±k signal) is not a strict failure even under `True`: it surfaces as a NaN `MetricResult` with `is_applicable=False` and `metadata["reason"]` while the applicable metrics in the same call still return — so a mixed battery is not aborted by one inapplicable metric. When `strict=False`, every such case (including data shortages and cell mismatches) is kept as a NaN output with attached warnings instead of raising. Config-time / construct-time failures always raise.	`True`
`expected_warnings`	`tuple[str, ...]`	:class:`~factrix.WarningCode` values (e.g. `("few_assets",)`) declaring warning regimes that are the study's design, not an accident — a single-asset or pairs study has no cross-section by construction, so its every run trips `few_assets`. Declared codes are marked, never dropped: each matching :class:`Warning` record is kept with `expected=True` (read the alert view via `result.unexpected_warnings`), so nothing leaves the audit trail — the declaration says "this regime is expected", not "pretend it is absent". Inference is untouched (e.g. the small-cross-section block-bootstrap switch still fires and stays readable in `metadata["method"]`); only the human-facing channels go quiet — the per-run `UserWarning` echoes stop and repr emphasis moves to unexpected warnings. Unknown codes are rejected (typo guard). Default `()` — behavior is completely unchanged unless declared. Not a per-metric knob: constructing a metric with `expected_warnings=` is rejected; the study-level declaration here applies to every metric in the call.	`()`

Returns:

Type	Description
`dict[str, EvaluationResult]`	`dict[str, EvaluationResult]` keyed by factor column name, in
`dict[str, EvaluationResult]`	`factor_cols` insertion order. Each value carries the full
`dict[str, EvaluationResult]`	per-metric outputs, panel structural stats (`n_periods`,
`dict[str, EvaluationResult]`	`n_pairs`), and warnings for that factor.

Raises:

Type Description

UserInputError

metrics not a dict[str, Metric] of instances; factor_cols empty / single str / contains duplicates / references a column not on data; data missing a baseline column; a metric requires a producer absent from the registry; under strict=True, a metric inapplicable to the data; data carries no horizon stamp and none is declared via forward_periods, or the declared forward_periods disagrees with the stamp.

Examples:

Single-factor IC + IC information ratio (IR):

>>> import factrix as fx
>>> from factrix.metrics import ic, ic_ir
>>> raw = fx.datasets.make_cs_panel(n_assets=15, n_dates=80)
>>> data = fx.preprocess.compute_forward_return(raw, forward_periods=5)
>>> results = fx.evaluate(
...     data,
...     metrics={"ic": ic(), "ic_ir": ic_ir()},
...     factor_cols=["factor"],
...     forward_periods=5,
... )
>>> "ic" in results["factor"].metrics
True
>>> "ic_ir" in results["factor"].metrics
True

factrix.evaluate_horizons ¶

evaluate_horizons(data: DataInput, *, metrics: dict[str, MetricBase], factor_cols: list[str], forward_periods: list[int], strict: bool = True, expected_warnings: tuple[str, ...] = ()) -> list[EvaluationResult]

Sweep evaluate across several overlap horizons of one raw panel.

A thin composition over the existing primitives — for each horizon it rebuilds the panel with :func:factrix.preprocess.compute_forward_return and runs a single :func:evaluate, then flattens the per-factor results into one list. No new type is introduced and the single-horizon contract of evaluate is untouched: every inner run still evaluates one panel at one stamped horizon.

The horizon must be rebuilt from the raw panel for each value — compute_forward_return is not idempotent (it drops the last forward_periods + 1 rows per asset and stamps the horizon), so a horizon cannot be re-derived from an already-attached panel. This wrapper exists to make that rebuild-per-horizon loop hard to get wrong.

Identity of a swept result is the composite (factor, forward_periods), not a unique scalar factor key — so the return is a flat list[EvaluationResult] (the native shape of the aggregation layer), not the factor-keyed dict that evaluate returns at a fixed horizon. factor and forward_periods are existing native attributes of :class:EvaluationResult; the list feeds straight into :func:compare and into :func:factrix.multi_factor.bhy. Pool all horizons when selection may choose across them; use expand_over=('forward_periods',) only for predeclared horizon-specific screens that are selected and reported separately.

Parameters:

Name	Type	Description	Default
`data`	`DataInput`	A raw panel carrying `date`, `asset_id`, `price` and the factor columns, without a `forward_return` column — it is rebuilt per horizon. Passing an already-attached panel is rejected by `compute_forward_return`. Only forward-return is computed; winsorize / abnormal-return are out of scope, build those panels and call :func:`evaluate` per horizon by hand.	required
`metrics`	`dict[str, MetricBase]`	Same contract as :func:`evaluate` — a `dict[str, Metric]` of metric instances. Applied identically at every horizon.	required
`factor_cols`	`list[str]`	Same contract as :func:`evaluate`. Every column is evaluated at every horizon; the flat result has one entry per `(factor, horizon)`.	required
`forward_periods`	`list[int]`	The horizons to sweep, as a non-empty `list[int]` of distinct positive row counts (e.g. `[5, 20, 60]`). Duplicates are rejected — they would yield a duplicate `(factor, forward_periods)` identity that `compare` / `bhy` reject downstream.	required
`strict`	`bool`	Forwarded unchanged to each inner :func:`evaluate`.	`True`
`expected_warnings`	`tuple[str, ...]`	Forwarded unchanged to each inner :func:`evaluate` — the declaration is a property of the study, so it applies identically at every horizon.	`()`

Returns:

Type	Description
`list[EvaluationResult]`	Flat `list[EvaluationResult]` grouped by horizon (outer) then
`list[EvaluationResult]`	`factor_cols` order (inner). Each entry carries its own stamped
`list[EvaluationResult]`	`forward_periods`.

Raises:

Type	Description
`UserInputError`	`forward_periods` is not a non-empty `list[int]` of distinct positive values; plus any error raised by the inner :func:`evaluate` / `compute_forward_return` (e.g. `data` already carries `forward_return`).

Notes

Comparability across horizons is a scale alignment, not a free lunch: compute_forward_return divides by forward_periods so rank-IC is directly comparable across horizons, but signed-return-mean metrics carry a compounding bias that grows with forward_periods (see :func:factrix.preprocess.compute_forward_return Notes). Treat a cross-horizon sweep of signed-mean metrics as descriptive.

Examples:

>>> import factrix as fx
>>> from factrix.metrics import ic
>>> raw = fx.datasets.make_cs_panel(n_assets=20, n_dates=300)
>>> results = fx.evaluate_horizons(
...     raw,
...     metrics={"ic": ic()},
...     factor_cols=["factor"],
...     forward_periods=[5, 10, 20],
... )
>>> [r.forward_periods for r in results]
[5, 10, 20]
>>> board = fx.compare(results, metrics=["ic"])  # one row per horizon
>>> board.height
3

Use cases¶

Single-factor significance

One panel of factor data → one result carrying the mainstream metric's p_value and the cell-specific statistics.
Batch screening with false discovery rate (FDR)

Loop evaluate over candidate signal columns and feed the resulting EvaluationResult list to bhy for false-discovery-rate control. See Multi-factor FDR.
Cross-cell apples-to-apples

Compare information coefficient (IC) rank-ordering against Fama-MacBeth λ on the same panel, or individual-asset factors against broadcast macro factors. Return shape is identical across cells.
TIMESERIES dispatch

At n_assets == 1 there is no cross-section, so any DENSE metric whose cell is PANEL — Individual × Continuous (ic, fm_beta) and Common × Continuous (common_beta, common_quantile, common_asymmetry) — raises IncompatibleAxisError (or NaN + structure_mismatch under strict=False). Single-asset data runs through the same entry point with predictive_beta for dense predictive-regression slopes, sparse metrics whose cell wildcard allows TIMESERIES, and panel-input wildcard metrics such as directional_hit_rate. Two-column diagnostics (positive_rate, oos_decay, ic_trend) are standalone (date, value) tools; in evaluate() they layer on panel IC series, not raw single-asset dense panels.

Worked example — single-factor smoke test¶

Synthetic panel → evaluate → read value + p_value

Full runnable example complementing the doctest snippets in Examples above with realistic console output.

import factrix as fx
from factrix.metrics import ic, quantile_spread

# 1. Create dummy panel data
raw = fx.datasets.make_cs_panel(n_assets=15, n_dates=80)
data = fx.preprocess.compute_forward_return(raw, forward_periods=5)

# 2. Run evaluation
results = fx.evaluate(
    data,
    metrics={"ic": ic(), "spread": quantile_spread(n_groups=5)},
    factor_cols=["factor"],
    forward_periods=5,
)

# 3. Retrieve and inspect results
res = results["factor"]
print(f"Factor: {res.factor}")
print(f"Cell: {res.cell}")
print(f"Plan: \n{res.plan}")

# Access metrics result group
ic_res = res.metrics["ic"]
print(f"IC Value: {ic_res.value:.4f}")
print(f"IC p-value: {ic_res.p_value:.4f}")

Sensitivity grids¶

For exploratory grids across asset counts, horizons, or factor families, run with strict=False and stack each result's long-form table. The table carries is_applicable and reason, so the grid can keep running while still making failed metric/input combinations visible.

import polars as pl

results = fx.evaluate(
    data,
    metrics={"ic": ic(), "spread": quantile_spread(n_groups=2)},
    factor_cols=["factor"],
    forward_periods=5,
    strict=False,
)

status = pl.concat([r.to_frame() for r in results.values()])
failed = status.filter(~pl.col("is_applicable"))

Evaluating under different cell contexts¶

Metric behaviors are defined by instantiating metric classes directly. The DAG executor handles dispatch automatically depending on the cell registered by the metric.

import factrix as fx
from factrix.metrics import ic, caar, common_beta

# 1. Individual × Continuous (e.g. Information Coefficient)
results_ic = fx.evaluate(
    data,
    metrics={"ic": ic()},
    factor_cols=["factor"],
    forward_periods=5
)

# 2. Individual × Sparse (e.g. Event Study CAAR, requires a 'price' column)
results_caar = fx.evaluate(
    data_with_price,
    metrics={"caar": caar()},
    factor_cols=["event_factor"],
    forward_periods=5
)

# 3. Common × Continuous (e.g. Time-Series Beta)
results_common_beta = fx.evaluate(
    data,
    metrics={"common_beta": common_beta()},
    factor_cols=["macro_factor"],
    forward_periods=5
)

Per-cell required / optional columns and the DataStructure (PANEL vs TIMESERIES) derivation are automatically resolved at dispatch time.

Next steps¶

Multi-factor FDR

Wires evaluate into the multi-factor FDR pipeline: pass candidate results to BHY; choose between bhy / partial_conjunction / bhy_hierarchical; mixed-cell batches.

Read the guide →
Data schema

New to the fixed-horizon input contract? Start here for the evaluation panel floor (date, asset_id, factor, forward_return), dtype semantics, and optional columns that activate extra metrics.

Read the schema →