Guide — Repertoire diversity

Hill numbers (q = 0, 1, 2) with coverage standardisation and bootstrap CI.

Why Hill numbers

Repertoire-size estimates are notoriously sample-size dependent. Hill numbers form a one-parameter family — q controls how strongly rare types are weighted — and the coverage-standardised variants let you compare two corpora at the same effective sampling depth, not the same raw count.

Asymptotic estimators

from decipher import hill_numbers, coverage_estimator

counts = [...]   # type counts (e.g. from collections.Counter on stream.symbols)

D0 = hill_numbers(counts, q=0)   # richness (Chao1)
D1 = hill_numbers(counts, q=1)   # Shannon effective number (Chao-Shen)
D2 = hill_numbers(counts, q=2)   # inverse Simpson (unbiased)
C  = coverage_estimator(counts)  # Good-Turing sample coverage in [0, 1]

Coverage-standardised comparison

Pick a target coverage (e.g. 0.95) below the lower of two corpora's asymptotic coverage estimates, and ask each corpus for its diversity at that coverage:

from decipher import estimate_d_at_coverage, hill_bootstrap_ci

D_a = estimate_d_at_coverage(counts_a, q=1, coverage=0.95)
D_b = estimate_d_at_coverage(counts_b, q=1, coverage=0.95)

ci_a = hill_bootstrap_ci(counts_a, q=1, coverage=0.95, n_boot=1000, alpha=0.05)
ci_b = hill_bootstrap_ci(counts_b, q=1, coverage=0.95, n_boot=1000, alpha=0.05)

The estimator does the right thing whether the target coverage requires interpolation (sub-sample) or extrapolation — it is the iNEXT recipe implemented in pure NumPy.