factrix.multi_factor.partial_conjunction ¶

partial_conjunction(results: list[EvaluationResult], *, metrics: list[str], min_pass: int, expand_over: tuple[str, ...], n_conditions: int | None = None, family_size: int | Mapping[tuple[Any, ...], int] | None = None, q: float = 0.05, inactive_policy: InactivePolicy = 'count') -> dict[str, PartialConjunctionResult]

Partial-conjunction screening: identities significant in min_pass of m conditions, one screen per metric.

For "factor X is significant in universes A and B" style claims, naive intersection-of-survivors does not preserve FDR (Benjamini-Bogomolov 2014). The partial conjunction test (Benjamini-Heller 2008) combines an identity's per-condition p-values into a single PC p-value, then runs BHY across identities.

PC formula (Bonferroni-style): for k = min_pass, p_PC = (m - k + 1) * p_((k)) capped at 1. k = 1 reduces to m * min(p) (union semantics) and is forbidden.

min_pass must be declared before the p-values are seen. (m - k + 1) * p_((k)) is not monotone in k: raising the bar from "2 of 4" to "3 of 4" can lower the PC p-value and turn a non-survivor into a survivor. Picking k after looking at the p-values is therefore a selection effect the outer BHY step-up does not correct — the k-of-m claim is part of the hypothesis, not a knob to tune.

Inactive conditions (a data-shortage short-circuit or a degenerate_variance result) are governed by inactive_policy, exactly as in :func:bhy. Under the default "count" they stay in m at an inert p = 1; under "exclude" submitted inactive conditions leave m. An identity whose remaining computed plus declared-unsubmitted conditions number fewer than min_pass stays in the audit output with NaN and never enters the outer BHY family.

Parameters:

Name	Type	Description	Default
`results`	`list[EvaluationResult]`	:class:`EvaluationResult` records. The aggregation identity is the hypothesis identifier minus the `expand_over` components; results sharing an identity form that identity's conditions and must differ on at least one `expand_over` key. Any other swept knob (`params` key or `forward_periods` outside `expand_over`) keeps identities apart instead of inflating `m`.	required
`metrics`	`list[str]`	`list[str]` — one PC screen runs per metric; return dict keyed by `label`.	required
`min_pass`	`int`	`k` in "k of m". Must be `>= 2`.	required
`expand_over`	`tuple[str, ...]`	Non-empty tuple of `params` keys (or `"forward_periods"`) defining the condition axis.	required
`n_conditions`	`int \| None`	Strict condition-count declaration. `None` lets `m` be inferred per identity; an `int` requires every identity's declared `family_size` to have exactly that many conditions.	`None`
`family_size`	`int \| Mapping[tuple[Any, ...], int] \| None`	Complete condition-family size per identity. A scalar applies to every identity; a mapping declares heterogeneous sizes keyed like the returned `family_size`. Declared-but-unsubmitted conditions enter k-of-m as inert non-rejections. `None` uses the submitted condition count.	`None`
`q`	`float`	Nominal FDR target for the BHY step-up over PC p-values. Must satisfy `0 < q < 1`.	`0.05`
`inactive_policy`	`InactivePolicy`	`"count"` (default) keeps an inactive condition in the `k`-of-`m` denominator at an inert `p = 1`; `"exclude"` drops it from `m` under an independent or pre-specified activity filter. See :data:`InactivePolicy`.	`'count'`

Returns:

Type	Description
`dict[str, PartialConjunctionResult]`	`dict[str, PartialConjunctionResult]` keyed by
`dict[str, PartialConjunctionResult]`	`label`.

Raises:

Type	Description
`UserInputError`	`min_pass < 2`; `expand_over` empty; `inactive_policy` not `'count'` / `'exclude'`; `n_conditions < min_pass`; condition-count mismatch; identity with fewer than `min_pass` conditions; any `_resolve_family` invariant failure.

Contract-bearing screening for the "factor X is significant in \(k\) of \(m\) conditions" claim. Replaces the notebook idiom set(survivors_a) & set(survivors_b), which does not preserve false discovery rate (FDR) (Benjamini & Bogomolov 2014), with the partial conjunction test of Benjamini & Heller (2008).

Illustrative

import dataclasses

import factrix as fx
from factrix.metrics import ic

# "Momentum is significant in BOTH large-cap AND small-cap universes"
# evaluate() returns dict[str, EvaluationResult]; pull the single result
# and stamp the universe label onto it with dataclasses.replace so
# expand_over can split the family on params["universe_id"].
def stamp(panel, factor_col, **params):
    res = fx.evaluate(panel, metrics={"ic": ic()}, factor_cols=[factor_col])[factor_col]
    return dataclasses.replace(res, params=params)

results = [
    stamp(panel_large, "mom", universe_id="large_cap"),
    stamp(panel_small, "mom", universe_id="small_cap"),
    # ... + value, quality, etc. one result per (factor, universe) cell
]
survivors = fx.multi_factor.partial_conjunction(
    results,
    metrics=["ic"],
    min_pass=2,
    n_conditions=2,
    expand_over=("universe_id",),
    q=0.05,
)

Versus `bhy(expand_over=...)` — same data, different question¶

Both functions accept expand_over=, but the survivor unit and the question answered differ. This is the single most common source of confusion; pick the row that matches your claim.

Function	Survivor unit	Question	Example claim
`bhy(results, expand_over=("universe_id",))`	`(factor, universe)` pair	"Where is this factor significant?"	"Momentum is significant in `large_cap`; value is significant in `small_cap`"
`partial_conjunction(results, min_pass=2, expand_over=("universe_id",))`	`factor` identity	"Which factors are significant across all conditions?"	"Momentum is significant across both universes"

In other words: bhy treats each universe as its own hypothesis and expands the family; partial_conjunction treats each universe as a condition the factor must pass jointly and aggregates back to one hypothesis per factor.

When not to reach for `partial_conjunction`¶

Real intent	Reach for	Why
"At least any condition is significant"	`bhy(results, expand_over=(...))`	`min_pass=1` is union semantics — FDR inflates to ~2q. `partial_conjunction` raises rather than implement this.
Rank candidates (no FDR control)	`compare`	`compare` is a view, not a filter.
Sensitivity to estimator / sample choice	a dedicated robustness sweep	Conditions there are methods, not data slices.
Cross-slice metric difference (descriptive)	`by_slice`	Returns per-slice metric values; no inference.
Cross-slice metric difference (inferential, slice-pairs)	`slice_pairwise_test` / `slice_joint_test`	Tests whether the slices' metric series differ, not whether the factor is jointly significant.

Strict vs lenient mode¶

n_conditions is the contract knob.

DataStructure	When	Behavior
Strict (`n_conditions=int`)	Paper-grade; you know the design (e.g. exactly 2 universes, exactly 4 horizons)	Identity with any condition count other than `n_conditions` raises. Data gaps surface fail-loud.
Lenient (`n_conditions=None`)	EDA / prototyping; condition count varies by identity	`m` inferred per identity from the data; only requires `m >= min_pass`.

Both modes count the conditions you submitted. Placeholder conditions are then excluded from m before the PC combination (see below), so a strictly declared design can still leave an identity with too few real conditions to screen.

Illustrative

# Strict: 2 universes required for every factor; missing one raises.
fx.multi_factor.partial_conjunction(
    results, min_pass=2, n_conditions=2, expand_over=("universe_id",)
)

# Lenient: "at least 3 of however many horizons each factor has".
fx.multi_factor.partial_conjunction(
    results, min_pass=3, expand_over=("fwd_period",)
)

How the math works¶

Per identity, the \(m\) per-condition \(p\)-values are reduced to a single PC \(p\)-value (Bonferroni-style, BH2008):

\[ p_{\text{PC}}^{(k/m)} = \min\bigl(1,\; (m - k + 1) \cdot p_{(k)}\bigr) \]

where \(p_{(k)}\) is the \(k\)-th smallest of the \(m\) \(p\)-values and \(k = \texttt{min\_pass}\). Two corner cases worth knowing:

\(k = m\) (full conjunction) → \(p_{\text{PC}} = \max(p)\). Reject only when even the worst condition is significant.
\(k = 1\) (union) → \(p_{\text{PC}} = m \cdot \min(p)\). Bonferroni-corrected minimum. Forbidden here — the surface raises with a pointer to bhy(expand_over=...), where the family-level FDR inflation is explicit rather than hidden in a "robust across" claim.

Declare `min_pass` before you look at the p-values¶

\(p_{\text{PC}}\) is not monotone in \(k\): the multiplier \(m - k + 1\) falls as the order statistic \(p_{(k)}\) rises, so demanding more passing conditions can lower the PC \(p\)-value. For \(p = [0.01, 0.02, 0.9]\), \(k = 2\) gives \(0.04\) and \(k = 3\) gives \(0.9\); for \(p = [0.01, 0.4, 0.5]\) the ordering reverses (\(0.8\) against \(0.5\)). Which direction helps depends on the data.

Choosing \(k\) after seeing the \(p\)-values is therefore a selection effect the outer BHY step-up does not correct — it is the same shopping this function exists to prevent, one level up. The "\(k\) of \(m\)" claim is part of the hypothesis: fix it with the design.

Inactive conditions and \(m\)¶

A condition whose metric never ran a test (insufficient_* short-circuit or degenerate_variance) is governed by inactive_policy, the same declaration every other verb takes — see inactive candidates. Under the default "count" it stays in the k-of-m denominator at an inert \(p = 1\) (it can never be one of the \(k\) passes, so it can only make the claim harder). Under "exclude" it leaves \(m\), and an identity left with fewer than min_pass conditions cannot support the claim: it stays in entries for audit with pc_p / adj_p of NaN and never enters the outer BHY family.

The PC \(p\)-values are then fed to a standard Benjamini-Hochberg-Yekutieli (BHY) step-up across identities, controlling group-level FDR ≤ q. The harmonic dependence correction \(c(m) = \sum 1/i\) is applied because PC \(p\)-values across identities are not generally positive regression dependence on a subset (PRDS) — sharing underlying panels makes the joint distribution unknown, so the conservative choice is the default.

BH2008 also presents a Simes-style PC combiner, which is less conservative under PRDS. factrix ships only the Bonferroni-style — it is uniformly valid without dependence assumptions; a Simes path may be added later if a use case demands it.

Result output¶

partial_conjunction returns a PartialConjunctionResult per metric — the same _FdrResultBase shape as bhy's BhyResult (entries / survivors / adj_p / q / family_size), plus PC-specific fields:

Field	Meaning
`entries`	One representative result per tested identity (first result of that identity, input order) — every identity, not just survivors
`adj_p_all`	BHY-adjusted PC \(p\)-value, aligned with `entries`; identity survives iff `adj_p_all <= q`
`pc_p_all`	Raw PC \(p\)-value (pre-BHY), aligned with `entries`
`survivors` / `adj_p`	Surviving subset and its adjusted p-value (derived from `adj_p_all <= q`)
`min_pass`	The \(k\) you passed
`family_size`	Keyed by the identity tuple — `factor`, then `forward_periods` and `params` items not named by `expand_over` → the k-of-m denominator \(m\) for that identity, including declared-unsubmitted conditions
`family`	Declared / computed / inactive / unsubmitted / adjusted condition counts and the `inactive_policy` used
`n_passed_uncorr_all`	Per-identity count of family conditions with raw \(p \le q\) (an inert condition enters at \(p = 1\) and never counts as a pass) — the same `<=` rejection rule every screen uses — aligned with `entries`. Descriptive — flags borderline (`n_passed_uncorr_all == min_pass`) and data-gap cases at a glance. Cutoff is your `q`, so the count moves with `q` — using it to override `adj_p` survivor selection is the anti-shopping failure mode this function exists to prevent.

to_frame() gives a factor | forward_periods | <sorted non-condition params> | adj_p | survived DataFrame over every tested identity, eliminated ones included. Keys named by expand_over are omitted because they are the conditions already combined into the row; all other swept knobs remain part of its identity. When forward_periods is itself the condition axis, it is omitted for the same reason instead of exposing one arbitrary representative horizon.

factrix.multi_factor.PartialConjunctionResult `dataclass` ¶

PartialConjunctionResult(metric_name: str, entries: list[EvaluationResult], adj_p_all: ndarray, q: float, family_size: Mapping[tuple[Any, ...], int], family: FamilyAccounting, pc_p_all: ndarray, expand_over: tuple[str, ...], min_pass: int, n_passed_uncorr_all: ndarray)

Bases: _FdrResultBase

Per-identity partial-conjunction survivors for one metric.

Shares metric_name / entries / adj_p_all / q / family_size with :class:_FdrResultBase. entries is one representative :class:EvaluationResult per identity; adj_p_all is the BHY-adjusted PC p-value (NaN for an identity left with fewer than min_pass conditions, which never enters the family); family_size is m in the k-of-m claim per identity — every declared condition under inactive_policy="count", the computed ones only under "exclude" — keyed by the identifier with the expand_over components stripped (factor, then forward_periods and params items not named by expand_over).

Attributes:

Name	Type	Description
`pc_p_all`	`ndarray`	Raw PC p-value aligned with `entries`.
`expand_over`	`tuple[str, ...]`	`params` keys defining the condition axis.
`min_pass`	`int`	`k` in the `k` of `m` partial conjunction test.
`n_passed_uncorr_all`	`ndarray`	Per-identity count of family conditions whose raw p-value is at or below `q` — the same `<=` rejection rule every other screen uses (descriptive — not used in inference), aligned with `entries`. An inactive condition enters at `p = 1` and so never counts as a pass.

to_frame ¶

to_frame() -> DataFrame

Return one row per identity after collapsing the condition axis.

expand_over parameters are deliberately absent: their values are conditions combined into the row, not part of the resulting hypothesis identity.

Validation summary¶

Trigger	Outcome
`min_pass < 2`	`UserInputError`. `min_pass == 1` additionally points at `bhy(expand_over=...)`.
`expand_over` empty / `None`	`UserInputError` — the function is undefined without a condition axis.
`expand_over` names an identity field (`factor` / `forward_periods`)	`UserInputError` (anti-shopping defense — same as `bhy`).
`n_conditions < min_pass`	`UserInputError` (unsatisfiable).
Strict mode: identity's condition count \(\neq\) `n_conditions`	`UserInputError` — surfaces missing-universe / missing-horizon data gaps.
Identity with submitted condition count \(<\) `min_pass` (lenient)	`UserInputError`.
Identity with fewer than `min_pass` real conditions (the rest placeholders)	No error — audited with `NaN`, excluded from the outer BHY family.
Duplicate `(identity, expand_over_values)` partition key	`UserInputError` (family-resolution invariant).

References¶

[BH2008] Benjamini, Y. & Heller, R. (2008). Screening for partial conjunction hypotheses. Biometrics, 64(4), 1215–1222.
[BB2014] Benjamini, Y. & Bogomolov, M. (2014). Selective inference on multiple families of hypotheses. JRSS-B, 76(1).
[HY2014] Heller, R. & Yekutieli, D. (2014). Replicability analysis for genome-wide association studies. AOAS, 8(1).