factrix.by_slice

by_slice(metric: Callable[..., MetricOutput], df: DataFrame, *, label: str, **kwargs: Any) -> SliceResult

Apply metric to each value-partition of df keyed by label.

Universe overlap (superset / multi-membership / hierarchical / sliding window / cross-product) is composed by the caller — see the API page for reference patterns. by_slice does no cross-slice statistical inference; for paired comparison see :func:factrix.slice_pairwise_test / :func:factrix.slice_joint_test.

Parameters:

Name	Type	Description	Default
metric	Callable[..., MetricOutput]	Callable returning a MetricOutput (e.g. :func:factrix.metrics.ic, :func:factrix.metrics.caar).	required
df	DataFrame	Metric's primary DataFrame; must already contain label as a column. If you have a separate labels DataFrame, join it in upstream — :func:by_slice deliberately does not ingest a separate labels argument.	required
label	str	Column name in df whose distinct values define the slices. Must already exist in df.columns; for cross-product slicing (e.g. market × sector) compose a single composite column upstream (pl.concat_str([...]).alias("...")).	required
**kwargs	Any	Forwarded unchanged to metric on every per-slice call.	{}

Returns:

Type	Description
SliceResult	class:SliceResult — Mapping[str, MetricOutput] (so every
SliceResult	dict-shaped consumer keeps working) plus a
SliceResult	meth:SliceResult.to_frame long-form renderer. Keys are
SliceResult	stringified (an Int64 decile column yields "1".."10");
SliceResult	iteration order matches polars
SliceResult	partition_by(as_dict=True). No cross-slice statistical
SliceResult	inference — see API page.

Raises:

Type	Description
TypeError	df is not a polars DataFrame.
ValueError	label not in df.columns, or df is empty.

Examples:

Per-year information coefficient (IC) on a synthetic cross-sectional panel — attach the slice label to the metric's primary DataFrame upstream, then dispatch one metric call per slice value:

>>> import polars as pl
>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics import ic, compute_ic
>>> raw = fx.datasets.make_cs_panel(n_assets=100, n_dates=500)
>>> panel = compute_forward_return(raw, forward_periods=5)
>>> ic_df = compute_ic(panel).with_columns(
...     pl.col("date").dt.year().alias("year")
... )
>>> per_year = fx.by_slice(ic, ic_df, label="year")

factrix.SliceResult

SliceResult(data: Mapping[str, MetricOutput])

Bases: Mapping[str, MetricOutput]

Mapping of slice label → :class:MetricOutput with a frame renderer.

Returned by :func:factrix.slicing.by_slice. Iteration order matches the upstream polars.DataFrame.partition_by order (insertion-order of distinct values, not lexicographic) — call .sort("slice") on the rendered frame if a stable order is needed downstream.

The container deliberately exposes only the universal projection (name, value, stat, p_value). For metric-specific metadata (tie_ratio, shanken_correction, ...), build the DataFrame directly from [(k, m.metadata) for k, m in result.items()].

Warning

p_value in the rendered frame is the per-slice marginal p-value computed by the metric on that slice alone — not adjusted for the K parallel tests across slices. Filtering df.filter(pl.col("p_value") < 0.05) across K=10 sectors inflates the family-wise error rate (FWER); under H0 you expect ≈ 0.4 "significant" slices by chance. For cross-slice inference with FWER / false discovery rate (FDR) control, use :func:factrix.slice_pairwise_test (Holm / Romano-Wolf / Bonferroni) or :func:factrix.slice_joint_test (omnibus χ²) instead. The container is for exploration; the inference functions are for claims.

Examples:

>>> import polars as pl
>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics import ic, compute_ic
>>> raw = fx.datasets.make_cs_panel(n_assets=40, n_dates=240)
>>> panel = compute_forward_return(raw, forward_periods=5)
>>> ic_df = compute_ic(panel).with_columns(
...     pl.col("date").dt.year().alias("year")
... )
>>> per_year = fx.by_slice(ic, ic_df, label="year")
>>> isinstance(per_year, fx.SliceResult)
True
>>> len(per_year) >= 1
True

to_frame

to_frame(*, slice_col: str = 'slice') -> DataFrame

Render slices as a long-form pl.DataFrame.

Schema (always, in order): slice_col, name, value, stat, p_value. stat and p_value are None when the underlying MetricOutput does not carry them (descriptive metric, short-circuit failure path, etc.).

Parameters:

Name	Type	Description	Default
slice_col	str	Output column name for the slice label. Default "slice"; override when the caller needs to avoid a clash with an existing identity column they intend to join in upstream.	'slice'

Returns:

Type	Description
DataFrame	pl.DataFrame with one row per slice. Row order follows
DataFrame	iteration order of self.

Examples:

>>> import polars as pl
>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics import ic, compute_ic
>>> raw = fx.datasets.make_cs_panel(n_assets=40, n_dates=240)
>>> panel = compute_forward_return(raw, forward_periods=5)
>>> ic_df = compute_ic(panel).with_columns(
...     pl.col("date").dt.year().alias("year")
... )
>>> per_year = fx.by_slice(ic, ic_df, label="year")
>>> df = per_year.to_frame()
>>> set(df.columns) >= {"slice", "name", "value"}
True

Override the slice column name to avoid collision:

>>> df2 = per_year.to_frame(slice_col="year")
>>> "year" in df2.columns
True

Axis-agnostic research dispatcher. Slices any metric's date-keyed input by a column already present in the DataFrame and runs the metric per slice. Returns a SliceResult — a Mapping[str, MetricOutput] with a .to_frame() long-form renderer.

The axis name does not bake into the API — market, sector, regime, market-cap tier, ADV bucket all share the same dispatcher.

Argument contract

The first argument is the metric callable (e.g. ic, caar, fama_macbeth); the second is the metric's primary date-keyed DataFrame with the slicing column already present; label names that column; remaining keyword args (forward_periods=..., etc.) forward unchanged on every per-slice call. See the docstring Examples block above for the canonical call shape.

SliceResult is a Mapping[str, MetricOutput]: result["<slice-key>"].value for dict-style access, result.to_frame() for the long-form pl.DataFrame.

Why "label is a column name"

by_slice does not accept a separate labels DataFrame. Users typically already carry the partition key (market / sector / regime) on the panel or can join it once upstream — splitting that join into the dispatcher's signature is redundant and forces a fixed (date, label) shape that does not generalise across axes (sector labels are by-asset, not by-date).

If label is not in df.columns, by_slice raises ValueError — compose the column upstream with df.with_columns(...) or df.join(...).

What it does not do

by_slice performs no cross-slice statistical inference. It returns the per-slice outputs and stops. Per-slice t-stats / SE in each MetricOutput are computed on that slice alone (different N, different autocorrelation structure per slice) and are not directly comparable — max(out.values(), key=lambda m: m.tstat) is not a defensible cross-regime selection rule. A generic cross-slice test (Benjamini-Hochberg-Yekutieli (BHY) adjustment, Sharpe-diff Wald, paired-difference Newey-West (NW), etc.) cannot be applied honestly across the metric matrix — the appropriate test depends on the metric family. For metrics that expose a per_date_series capability (ic, fama_macbeth, hit_rate), slice_pairwise_test / slice_joint_test provide cross-slice contrasts with joint-heteroskedasticity-and-autocorrelation-consistent (HAC) or block-bootstrap inference.

Universe overlap reference patterns

by_slice only partitions on a single column's distinct values. Any overlapping-universe scenario — same row needs to count toward multiple slices — is composed with three lines of polars upstream. The shared idiom: filter + with_columns(label=...) per target slice, then pl.concat.

1. Superset (subset and full set side-by-side)

market is "TWSE" / "OTC"; you want three slices: 上市, 上櫃, 全市場 (every row also belongs to 全市場).

import polars as pl

mapping = {"TWSE": "上市", "OTC": "上櫃"}
expanded = pl.concat([
    df.with_columns(pl.col("market").replace(mapping).alias("uni")),
    df.with_columns(pl.lit("全市場").alias("uni")),
])
by_slice(ic, expanded, label="uni")

2. Multi-membership (one stock in multiple indices)

Each index membership is a separate boolean column; the same row may have several True values.

expanded = pl.concat([
    df.filter(pl.col("in_sp500")).with_columns(pl.lit("SP500").alias("uni")),
    df.filter(pl.col("in_nasdaq100")).with_columns(pl.lit("N100").alias("uni")),
])
by_slice(ic, expanded, label="uni")

3. Hierarchical nesting (Top-10 ⊂ Top-50 ⊂ LargeCap ⊂ All)

Nested by market-cap rank; each tier contains every smaller tier.

tiers = [(10, "Top10"), (50, "Top50"), (200, "LargeCap")]
expanded = pl.concat([
    *[
        df.filter(pl.col("market_cap_rank") <= cutoff)
          .with_columns(pl.lit(name).alias("tier"))
        for cutoff, name in tiers
    ],
    df.with_columns(pl.lit("All").alias("tier")),
])
by_slice(ic, expanded, label="tier")

4. Sliding window (overlapping ADV buckets)

Adjacent deciles overlap to smooth boundary noise:

windows = [(0, 30), (10, 40), (20, 50), (30, 60),
           (40, 70), (50, 80), (60, 90), (70, 100)]
expanded = pl.concat([
    df.filter((pl.col("adv_pct") >= lo) & (pl.col("adv_pct") < hi))
      .with_columns(pl.lit(f"W[{lo},{hi})").alias("adv_win"))
    for lo, hi in windows
])
by_slice(ic, expanded, label="adv_win")

5. Cross-product / multi-axis (universe × sector)

Not an overlap problem — label only takes a single column. Compose a composite column with pl.concat_str:

df = df.with_columns(
    pl.concat_str(["market", "sector"], separator="-").alias("uni_sec")
)
by_slice(ic, df, label="uni_sec")
# keys: "TWSE-Tech", "TWSE-Finance", "OTC-Tech", ...

The API does not accept label: list[str]: single-column vs multi-column semantics would diverge on output dict key type (str vs tuple), breaking the dict[str, MetricOutput] convention downstream.

General template

Cases 1–4 share one idiom — per-target-slice filter + with_columns(label=...), then pl.concat. Overlapping rows are duplicated naturally by the concat.

expanded = pl.concat([
    df.filter(expr).with_columns(pl.lit(name).alias("group"))
    for name, expr in user_definitions.items()
])
by_slice(metric, expanded, label="group")

SliceResult

by_slice returns a SliceResult, a Mapping[str, MetricOutput] subclass — every dict-shaped consumer (for k, v in result.items(), result["bull"], len(result)) keeps working unchanged. The added value is .to_frame(), which flattens per-slice MetricOutput rows into a fixed-schema long-form pl.DataFrame for plotting, leaderboards, and Notebook rendering.

result = by_slice(ic, ic_df, label="regime")
result.to_frame().sort("slice")           # plot-ready, lexicographic order
# shape: (2, 5)
# ┌────────┬──────┬───────┬───────┬──────────┐
# │ slice  │ name │ value │ stat  │ p_value  │
# │ ---    │ ---  │ ---   │ ---   │ ---      │
# │ str    │ str  │ f64   │ f64   │ f64      │
# ╞════════╪══════╪═══════╪═══════╪══════════╡
# │ bear   │ ic   │ -0.02 │ -0.41 │ 0.683    │
# │ bull   │ ic   │ 0.07  │ 2.31  │ 0.024    │
# └────────┴──────┴───────┴───────┴──────────┘

# leaderboard: rank slices by t-stat magnitude
result.to_frame().sort(pl.col("stat").abs(), descending=True)

p_value is per-slice, not cross-slice-adjusted

Each row's p_value tests that slice alone against its own null (e.g. ic mean = 0). Filtering df.filter(pl.col("p_value") < 0.05) across K parallel slices inflates the family-wise error rate (FWER) — under H0, K=10 sectors yields ≈ 0.4 expected "significant" slices by pure chance. The container is for exploration; for inference claims with FWER / false discovery rate (FDR) control, use slice_pairwise_test (Holm / Romano-Wolf / Bonferroni) or slice_joint_test (omnibus χ²).

Schema

to_frame() always returns the same five columns in the same order:

Column	Source	None when
slice	mapping key (rename via slice_col=)	never
name	MetricOutput.name	never
value	MetricOutput.value	never
stat	MetricOutput.stat	descriptive metric / short-circuit failure
p_value	metadata["p_value"]	descriptive metric / short-circuit failure

stat and p_value semantics follow the underlying metric (stat may be a t, z, F, or χ² — see metadata["stat_type"]; p_value may be one- or two-sided per the metric's null hypothesis). When concatenating frames from multiple metrics (pl.concat([by_slice(ic, ...).to_frame(), by_slice(hit_rate, ...).to_frame()])), the stat column mixes statistic types and is not directly comparable across rows of different name.

Row order matches iteration order of the result, which matches the upstream polars.DataFrame.partition_by order (insertion order of distinct values, not lexicographic). For plotting and leaderboards, sort explicitly downstream (.sort("slice") for deterministic axis order, .sort("value") / .sort("stat") for ranking). The example block above shows the plot-ready idiom.

Why a fixed schema instead of cols=...?

Quant exploration overwhelmingly wants the same five columns — slice, metric name, effect size, test statistic, p-value. A configurable cols= parameter would have to choose between (a) a fixed lookup set that excludes p-value (which lives in metadata) or (b) per-slice metadata key discovery, whose candidate set drifts across metrics and even across success vs. short-circuit paths within one metric. Fixed schema avoids both failure modes; for metric-specific metadata (tie_ratio, shanken_correction, ...), build the frame directly:

pl.DataFrame(
    [{"slice": k, **m.metadata} for k, m in result.items()]
)