decipher.infotheory

Information-theoretic primitives over categorical streams. All entropy estimates use the Miller-Madow correction by default.

function decipher.infotheory.shannon_entropy

def shannon_entropy(
    symbols,
    alphabet_size: int,
    correction: str = "miller-madow",   # or "none"
) -> float

Plug-in Shannon entropy H(X) in bits from a symbol sequence. The Miller-Madow correction adds (K-1)/(2·N·ln 2) bits, where K is the number of observed distinct symbols and N is the sequence length.

function decipher.infotheory.shannon_entropy_from_counts

def shannon_entropy_from_counts(
    counts,
    correction: str = "miller-madow",
) -> float

Same, but starting from a count vector. The alphabet size is len(counts); N = sum(counts).

function decipher.infotheory.joint_symbols

def joint_symbols(
    s_a: np.ndarray, K_a: int,
    s_b: np.ndarray, K_b: int,
) -> tuple[np.ndarray, int]

Encode parallel (s_a[i], s_b[i]) pairs as a single joint symbol id: joint[i] = s_a[i] * K_b + s_b[i]. Returns (joint_ids, K_joint) where K_joint = K_a * K_b.

function decipher.infotheory.joint_entropy_from_streams

def joint_entropy_from_streams(
    s_a: np.ndarray, K_a: int,
    s_b: np.ndarray, K_b: int,
    correction: str = "miller-madow",
) -> float

H(A,B) in bits via joint-symbol encoding.

function decipher.infotheory.mutual_information_bits

def mutual_information_bits(
    s_a: np.ndarray, K_a: int,
    s_b: np.ndarray, K_b: int,
    correction: str = "miller-madow",
) -> dict

Plug-in MI(A;B) = H(A) + H(B) − H(A,B), all bias-corrected. Returns a dict with keys: H_a, H_b, H_joint, MI, H_a_given_b, H_b_given_a, NMI.

NMI uses the arithmetic-mean normalisation 2·MI / (H(A) + H(B)) (matches sklearn's arithmetic_mean).

function decipher.infotheory.permutation_null_mi

def permutation_null_mi(
    s_a: np.ndarray, K_a: int,
    s_b: np.ndarray, K_b: int,
    n_permutations: int = 500,
    rng_seed: int = 42,
    correction: str = "miller-madow",
    return_samples: bool = False,
) -> dict

Null MI distribution under independence (shuffles s_b). Bias correction is applied identically to observed and shuffled streams, so the null mean captures the plug-in MI bias at the given (N, K_a, K_b): bias_corrected_MI = observed_MI − null_mean.

Returns a dict with mean, std, q025, q975, max, n_permutations, and (optionally) samples.

dataclass decipher.infotheory.PartitionAlignment

@dataclass
class PartitionAlignment:
    n: int
    K_a: int
    K_b: int
    nmi: float
    ari: float
    mi_bits: float
    H_a: float
    H_b: float
    H_joint: float
    H_a_given_b: float
    H_b_given_a: float
    compactness_a: float
    compactness_b: float
    delta_compactness: float
    conditional_asymmetry: float

Result of compare_partitions. Combines bias-corrected information-theoretic metrics with the adjusted Rand index, plus compactness measures that detect when one partition is a refinement of the other.

function decipher.infotheory.compare_partitions

def compare_partitions(
    labels_a: np.ndarray,
    K_a: int,
    labels_b: np.ndarray,
    K_b: int,
    correction: str = "miller-madow",
) -> PartitionAlignment

Compare two label partitions over the same items.

function decipher.infotheory.context_dependency

def context_dependency(
    labels: np.ndarray,
    K: int,
    context: np.ndarray,
    K_context: int,
    correction: str = "miller-madow",
) -> dict

Tests whether a categorical partition is dependent on an external context label. Returns chi-squared, MI, and per-context entropies.