factrix.metrics.event_quality

Per-event quality descriptive statistics for event signals.

All metrics operate on the signed_car (return x sign(factor)) of individual events. They describe the quality and shape of per-event outcomes — distinct from significance testing (caar.py) and path analysis (mfe_mae.py).

Metrics

event_hit_rate — fraction of correct-direction events (binomial test) event_ic — density strength → return correlation (Spearman) signal_density — average time gap between events profit_factor — sum(gains) / sum(losses) event_skewness — skewness of signed_car distribution

Notes

Pipeline. Per-event scalar (hit / information coefficient (IC) / skew / density) computed on signed_car, then cross-event aggregation; generalised sign inference for hit rate, rank-based inference for IC, and descriptive summaries elsewhere.

Windows are counted in panel periods. The estimation window behind each event's abnormal return is measured on the panel's distinct-date grid, not on the asset's own rows, so it spans estimation_window grid periods for every name; an asset's missing periods count as missing observations inside it, and a ragged panel raises ragged_period_grid.

factrix.metrics.event_quality.event_hit_rate

event_hit_rate(data: DataFrame, *, factor_col: str = 'factor', return_col: str = 'forward_return', overlap_periods: int = DEFAULT_FORWARD_PERIODS, estimation_window: int = 60, expected_warnings: tuple[str, ...] = ()) -> MetricResult

Fraction of events with return in expected direction.

The static event floor (MIN_EVENTS_HARD) gates the hit-rate binomial test on the count of non-zero (event) observations that survive the event-axis spacing pass; the warn floor scales with the overlap_periods parameter (the spacing stride), so the threshold is declared as a resolver (a callable sample_threshold) rather than a constant.

Parameters:

Name	Type	Description	Default
data	DataFrame	Panel with event density and either a supplied abnormal_return or the raw return_col used to estimate it.	required

Returns:

Type	Description
MetricResult	MetricResult with value=hit rate. stat is the hit count for the
MetricResult	exact test, a z statistic when same-period clustering is adjusted,
MetricResult	and None when the null variance is zero.

Notes

hits = sum_i 1{signed_car_i > 0}, rate = hits / N. The two-sided generalised sign test uses the Cowan (1992) null described below. It is exact (scipy.stats.binomtest) when no material same-period dependence is detected, with stat equal to the hit count. When the clustering adjustment applies, stat is the deflated normal z instead. A boundary null probability of zero or one has zero Bernoulli variance: the hit-rate point estimate is kept, while stat, p_value and alternative are withheld under the standard degenerate_variance contract.

Events whose selected return source or factor_col is non-finite are dropped before counting (see :func:_finite_events) and reported as metadata["n_events_dropped_non_finite"]. Previously such an event failed signed_car > 0 and was scored as a miss, which both depressed the rate and inflated N.

What the null says. \(H_0: p = p_0\), the frequency with which a hit happens anyway — the generalised sign test of Cowan (1992) — not \(H_0: p = 0.5\). The unsigned rate \(p_{\uparrow} = P(AR > 0)\) is estimated on the non-event rows (Cowan's estimation-window proportion; metadata["sign_base_rate_up"]). A hit is signed, so a positive-factor event hits with probability \(p_{\uparrow}\) under the null and a negative-factor event with \(1 - p_{\uparrow}\); Cowan's test is one-directional and never needed the distinction, but a mixed-sign trigger does. The null tested is the mixture over the tested events, \(p_0 = w_{+}\,p_{\uparrow} + w_{-}\,(1 - p_{\uparrow})\) with \(w_{\pm}\) the share of events on each side, reported as metadata["sign_base_rate"] and in h0. Measured on a right-skewed iid null (20 assets, h = 5, \(p_{\uparrow} \approx 0.4\)): with every event short-signed the unsigned \(p_{\uparrow}\) rejected 100% of draws and the mixture 4.7%; all-long events are the same under both. Two distinct things would otherwise be read as skill: drift, which the abnormal-return model removes, and skew, which it does not — mean-adjusting sets \(E[AR] = 0\), but a right-skewed return has a negative median and is positive less than half the time, so a 0.5 null scores that asymmetry as a signal. With fewer than _MIN_BASE_RATE_ROWS usable non-event rows there is nothing to estimate from and the symmetric 0.5 is used, recorded as sign_base_rate_source="assumed_symmetric".

Independence. The exact binomial treats the \(N\) events as independent trials, which is false for two events on one asset inside one forward-return window: they share bars. Events are therefore strided per asset first (Brown-Warner (1985) non-overlap sampling on the event axis, the same treatment caar applies), and EVENT_WINDOW_OVERLAP reports what that removed. What the stride cannot address is same-period dependence across names. For that the hit indicator's within-period intraclass correlation is estimated and, when the Kish design effect it implies is material (scale below KP_MATERIAL_SCALE), the standardised hit count is deflated and the p-value read off the normal instead of the exact binomial — stat_type switches to "z" and EVENT_CLUSTERING_ADJUSTED records the switch. Below that materiality floor, and on a single asset, the exact test stands: an earlier version left it at \(\hat r = 0\) on every multi-asset panel. Read clustering_hhi alongside, and prefer bmp_z or corrado_rank when the shared shock is the object of study.

Conservative when the abnormal-return model is contaminated. The mean-adjusted abnormal return subtracts a window that, on a dense trigger or a long horizon, is mostly other events' realised returns; the per-event abnormal returns are then negatively correlated and the hit count under-rejects — 1.7% at \(h = 5\) and \(h = 21\) on 20 assets, 0.7% on a single asset at \(h = 21\) (nominal 5%; 6.0% at \(h = 1\)). metadata["estimation_window_event_share"] measures it and ESTIMATION_WINDOW_CONTAMINATED fires above ESTIMATION_WINDOW_EVENT_SHARE_WARN; see :func:~factrix.metrics._helpers._attach_abnormal_return for why neither window choice nor the market-adjusted branch removes it.

The selected abnormal-return series must be sign-symmetric around zero — signed_car = abnormal_return * sign(factor_col) — so an always-positive magnitude target (realised volatility, turnover) collapses sign(signed_car) to sign(factor_col) and silently turns the hit rate into a count of positive-factor events, with no error raised. Use :func:~factrix.metrics.ic.ic or :func:~factrix.metrics.monotonicity.monotonicity for those targets instead.

Examples:

>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics.event_quality import event_hit_rate
>>> panel = compute_forward_return(
...     fx.datasets.make_event_panel(n_assets=50, n_dates=400, rng=0),
...     forward_periods=5,
... )
>>> result = event_hit_rate(panel)
>>> result.name == ""
True

factrix.metrics.event_quality.event_ic

event_ic(data: DataFrame, *, factor_col: str = 'factor', return_col: str = 'forward_return', overlap_periods: int = DEFAULT_FORWARD_PERIODS, estimation_window: int = 60, expected_warnings: tuple[str, ...] = ()) -> MetricResult

Spearman correlation between factor value and realised forward return.

The static event floor (sample_threshold=SampleThreshold(min_events=MIN_EVENTS_HARD)) gates the rank correlation on the count of non-zero (event) observations.

Spearman correlation between |factor| and signed_car (return × sign(factor)), computed only on event rows.

Unlike standard information coefficient (IC) (full cross-section per date), this measures whether density magnitude predicts return magnitude among triggered events. Direction is already accounted for via sign().

Only meaningful when density values have magnitude variance (not all ±1). The metric returns a not-applicable result when variance is absent.

Parameters:

Name	Type	Description	Default
data	DataFrame	Panel with event density and forward return.	required

Returns:

Type	Description
MetricResult	MetricResult with value=Spearman rho, stat=z from Fisher transform.

Notes

rho = Spearman(|factor|, signed_car) over event rows; Fisher z-transform z = atanh(rho) * sqrt(N - 3) against H0: rho = 0. Direction is already absorbed into signed_car so this isolates the magnitude-of-density → magnitude-of-return link.

factrix short-circuits "not_applicable_discrete_signal" when |factor| lacks variance (e.g. {0, ±1} events): event-IC is undefined without magnitude variation, distinct from "too few events".

Events with a non-finite selected return source or factor_col are dropped before the correlation (scipy.stats.spearmanr would otherwise return NaN for both ρ and p, and a NaN p raises out of MetricResult); the count is reported as metadata["n_events_dropped_non_finite"].

Examples:

>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics.event_quality import event_ic
>>> panel = compute_forward_return(
...     fx.datasets.make_event_panel(n_assets=50, n_dates=400, rng=0),
...     forward_periods=5,
... )
>>> result = event_ic(panel)
>>> result.name == ""
True

factrix.metrics.event_quality.profit_factor

profit_factor(data: DataFrame, *, factor_col: str = 'factor', return_col: str = 'forward_return', overlap_periods: int = DEFAULT_FORWARD_PERIODS, estimation_window: int = 60, expected_warnings: tuple[str, ...] = ()) -> MetricResult

Profit factor = sum(gains) / sum(|losses|) across events.

The static event floor (sample_threshold=SampleThreshold(min_events=MIN_EVENTS_HARD)) gates the ratio on the count of non-zero (event) observations.

Per-event aggregate — no strategy assumptions. A profit factor > 1 means gross gains exceed gross losses across all events.

Parameters:

Name	Type	Description	Default
data	DataFrame	Panel with event density and forward return.	required
expected_warnings	tuple[str, ...]	Warning codes the caller declares; a declared code is still recorded, the UserWarning echo is silenced.	()

Returns:

Type	Description
MetricResult	MetricResult with value=profit_factor.

Notes

PF = sum(signed_car_i * 1{signed_car_i > 0}) / |sum(signed_car_i * 1{signed_car_i < 0})|. Descriptive only; no formal H0 (the ratio's sampling distribution lacks a clean closed-form null without distributional assumptions). PF > 1 means gross gains exceed gross losses across all events; the metric ignores per-event variance.

When gains are positive and losses are zero, factrix returns inf: the gross gain/loss ratio is unbounded, not zero. When both gains and losses are zero, the ratio is undefined and the metric returns NaN with metadata["profit_factor_status"].

Events with a non-finite selected return source or factor_col are dropped up front and counted in metadata["n_events_dropped_non_finite"]. They were already absent from both sums (NaN > 0 and NaN < 0 are both False), but used to remain inside n_obs / n_events — so n_wins + n_losses silently failed to reconcile with the advertised sample size.

Examples:

>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics.event_quality import profit_factor
>>> panel = compute_forward_return(
...     fx.datasets.make_event_panel(n_assets=50, n_dates=400, rng=0),
...     forward_periods=5,
... )
>>> result = profit_factor(panel)
>>> result.name == ""
True

factrix.metrics.event_quality.event_skewness

event_skewness(data: DataFrame, *, factor_col: str = 'factor', return_col: str = 'forward_return', overlap_periods: int = DEFAULT_FORWARD_PERIODS, estimation_window: int = 60, expected_warnings: tuple[str, ...] = ()) -> MetricResult

Skewness of signed event return distribution.

The static event floor (MIN_EVENTS_HARD) gates the descriptive skewness on the count of non-zero (event) observations that survive the event-axis spacing pass; the warn floor scales with the overlap_periods parameter (the spacing stride), so the threshold is declared as a resolver (a callable sample_threshold) rather than a constant.

Positive skew = occasional large gains, frequent small losses (desirable for event strategies). Uses scipy's Fisher skewness (bias-corrected).

Descriptive only: no p-value, no test statistic.

Parameters:

Name	Type	Description	Default
data	DataFrame	Panel with event density and forward return.	required

Returns:

Type	Description
MetricResult	MetricResult with value=skewness, stat=None and p_value=None.

Notes

skew = m_3 / m_2^(3/2) (Fisher, bias-corrected via scipy.stats.skew(bias=False)).

factrix used to publish D'Agostino's skew-test z and its p beside the point estimate whenever n_events >= 20. That test is withdrawn. It has no calibrated pooled form here for two independent reasons, and neither is repaired by the same-period clustering deflation the sibling event tests apply (event_hit_rate, event_ic) — routing the z through the design effect of the per-event standardised cubed deviation ((x_i - x̄)/s)^3, whose mean is the skewness.

First, non-normal signed CARs, with no clustering needed. D'Agostino's test assumes the sample is normal under the null and derives Var(g_1) = 6/n from that; it is the fourth moment that breaks it. On a true null event panel averaging 1.55 events per event period — near enough no clustering — the sampled signed CARs carry excess kurtosis (measured +0.63 at 252 periods, +0.95 at 504), the null z comes out with SD 1.31 and 1.50 against a nominal 1.0, and the test rejects 19.0% and 23.3% of the time at a nominal 5%. Deflation barely moves those: 17.7% and 22.7%. The within-period correlation of the cubed-deviation score is near zero on this null by construction — signing by sign(factor) enters a shared period shock with an asset-random sign — so the deflator has nothing to grip.

Second, same-period shocks with sign-aligned events. When every event on a period shares one factor sign, the period's shock no longer cancels: 40 assets firing together on 20 shared periods, one common shock per period, rejects 30.3% at a nominal 5% with near-zero excess kurtosis (−0.09) — a pure dependence failure, not a kurtosis one. Here the deflator does grip (ICC 0.30, mean scale 0.286) but does not restore size: it drives the rejection rate to 0.0%, trading a 6x over-rejection for a test with no power at all.

A deflator tuned to fix either failure worsens the other, so factrix publishes no test rather than one calibrated against the null it happens to be measured on. Take value as the descriptive shape of the event return distribution and test direction with event_hit_rate / event_ic / bmp_z, which are calibrated on both. The full size table is in docs/reference/inference-calibration.md.

Events with a non-finite selected return source or factor_col are dropped before the moments are taken and counted in metadata["n_events_dropped_non_finite"]. A single NaN used to make skew NaN, and a NaN p_value raised ValueError out of MetricResult — the metric crashed rather than degrading.

Examples:

>>> import factrix as fx
>>> from factrix.preprocess import compute_forward_return
>>> from factrix.metrics.event_quality import event_skewness
>>> panel = compute_forward_return(
...     fx.datasets.make_event_panel(n_assets=50, n_dates=400, rng=0),
...     forward_periods=5,
... )
>>> result = event_skewness(panel)
>>> result.p_value is None and result.stat is None
True

factrix.metrics.event_quality.signal_density

signal_density(data: DataFrame, *, factor_col: str = 'factor', expected_warnings: tuple[str, ...] = ()) -> MetricResult

Average bars per event (inverse frequency).

No event-axis sample_threshold is declared (sample_threshold=SampleThreshold()) because the in-body floor is a degeneracy guard (>= 2 events overall, then >= 2 per asset) needed to define the per-asset bars-per-event ratio, not the statistical MIN_EVENTS_HARD floor the other event metrics gate on; like other math-degeneracy guards it stays in the body.

Answers: "how frequently does this density fire?"

Computed per-asset as total_bars / n_events (inverse event frequency), then averaged across every asset that fires at least once. This is not the mean of actual inter-event gaps: bars-per-event depends only on counts, so clustered events and evenly-spaced events yield the same value. See clustering_hhi for event-date concentration.

Low density (large gaps) means the density is selective; high density (small gaps) means the density fires often — capacity is higher but independence may be weaker.

Parameters:

Name	Type	Description	Default
data	DataFrame	Panel with date, asset_id, factor.	required
expected_warnings	tuple[str, ...]	Warning codes the caller declares; a declared code is still recorded, the UserWarning echo is silenced.	()

Returns:

Type	Description
MetricResult	MetricResult with value = mean bars-per-event across assets.

Notes

Per asset i: bars_per_event_i = total_bars_i / n_events_i; the headline is the cross-asset mean of this ratio. This is an inverse-frequency measure, not the mean of inter-event gaps: clustered and evenly-spaced events at the same total count map to the same value.

factrix exposes clustering_hhi for event-date concentration; pair the two when independence assumptions matter.

Examples:

>>> import factrix as fx
>>> from factrix.metrics.event_quality import signal_density
>>> panel = fx.datasets.make_event_panel(n_assets=50, n_dates=400, rng=0)
>>> result = signal_density(panel)
>>> result.name == ""
True

---

Event-study contracts

These metrics use the sign-only form \(\text{signed\_car} = \text{abnormal\_return} \times \text{sign}(\text{factor})\) — distinct from caar's magnitude-weighted \(\text{abnormal\_return} \times \text{factor}\). See the abnormal-return table for the full per-metric contract and the confounded-event note for the event-axis spacing and same-period dependence rules.

Use cases

• Directional accuracy

  ---

  Fraction of events whose signed_car is positive. The Cowan (1992) generalised sign null is estimated from non-event abnormal returns. stat is the hit count for the exact test and a z statistic when a material same-period clustering adjustment applies. A zero-variance boundary null keeps the hit rate but withholds the test fields.

• Magnitude → magnitude

  ---

  Among triggered events, does the signal's |factor| co-move with the realised signed_car? Spearman rank correlation with Fisher-\(z\) inference, using the Fieller-Hartley-Pearson Spearman standard error \(1.06/\sqrt{n-3}\) rather than the Pearson \(1/\sqrt{n-3}\), and deflated for same-period clustering of the per-event rank score (see EVENT_CLUSTERING_ADJUSTED). Auto-skips on \(\{0, \pm 1\}\) inputs where |factor| has no variance.

• Gain / loss ratio and shape

  ---

  profit_factor reports \(\sum\text{gains} / |\sum\text{losses}|\) as a descriptive gross ratio. If gains are positive and losses are zero, the ratio is unbounded (value = inf, profit_factor_status = "unbounded_no_losses"); if both are zero, the ratio is undefined (value = NaN). event_skewness reports the Fisher- corrected skewness of the signed_car distribution, descriptively: p_value and stat are always None because no pooled test of that third moment is calibrated here (see Inference calibration and limitations). Useful for screening fat-right-tail vs symmetric event payoffs.

• Firing frequency

  ---

  signal_density reports mean bars-per-event per asset (inverse frequency). Pair with clustering_hhi when independence assumptions matter — bars-per-event ignores temporal clustering.

Choosing a function

Goal	Function
Directional-accuracy binomial test	event_hit_rate
Magnitude-of-signal → magnitude-of-return rank correlation	event_ic
Gross gain / loss ratio (descriptive only)	profit_factor
Tail asymmetry of signed_car (descriptive only)	event_skewness
Inverse firing frequency (bars per event)	signal_density

Worked example — directional accuracy + tail shape

event_hit_rate + event_skewness on a synthetic event panel

import factrix as fx
from factrix.metrics.event_quality import (
    event_hit_rate, event_skewness, profit_factor,
)
from factrix.preprocess import compute_forward_return

raw   = fx.datasets.make_event_panel(
    n_assets=200, n_dates=500, event_rate=0.02,
    event_magnitude_jitter=0.5, post_event_drift_bps=40.0, rng=2024,
)
panel = compute_forward_return(raw, forward_periods=5)

hit = event_hit_rate(panel)
assert 0.0 <= hit.value <= 1.0
assert hit.n_obs > 0
assert hit.stat is not None and hit.p_value is not None

sk = event_skewness(panel)
assert sk.stat is None and sk.p_value is None

pf = profit_factor(panel)
assert pf.metadata["n_wins"] + pf.metadata["n_losses"] <= pf.n_obs

See also

• caar / bmp_z

  ---

  Mean-CAAR significance and BMP variance-robust \(z\) on the same event sample.

  api/metrics/caar →

• clustering_hhi

  ---

  Event-date concentration index — read alongside signal_density when independence matters.

  api/metrics/clustering →

• by_slice

  ---

  Per-slice event-quality summaries (regime / universe / sector).

  api/by-slice →

• Statistical methods

  ---

  Binomial test branches, Fisher-\(z\) Spearman, D'Agostino skew test.

  reference/statistical-methods →

• Metric applicability reference

  ---

  Sample-size guards and signed_car contracts for the sign-only family.

  reference/metric-applicability →

• Individual × Sparse landing

  ---

  Adjacent event-study metrics in the same cell.

  api/metrics/individual-sparse →