decipher.ssl

Frozen self-supervised audio encoder (AVES wav2vec2) with mean and mean-std pooling.

class decipher.ssl.AVESEncoder

class AVESEncoder:
    def __init__(
        self,
        weights_path: Path | str,
        config_path: Path | str,
        device: str = "cpu",
    )

    def embed_clip(
        self,
        samples: np.ndarray,
        layer: int = -1,
    ) -> np.ndarray
        # → (n_frames, 768) frame embeddings

    def embed_batch(
        self,
        batch_samples: np.ndarray,
        layer: int = -1,
    ) -> np.ndarray
        # → (batch, n_frames, 768)

Wraps the AVES wav2vec2 model from Hagiwara 2023. Loads pretrained weights and a torchaudio config; runs frozen (no gradient).

Inputs must be at 16 kHz. Use prepare_for_ssl to resample and center-pad/crop your units before embedding.

function decipher.ssl.pool_mean

def pool_mean(frame_embeddings: np.ndarray) -> np.ndarray

Mean over the time axis. (n_frames, D) → (D,).

function decipher.ssl.pool_mean_std

def pool_mean_std(frame_embeddings: np.ndarray) -> np.ndarray

Concatenate the per-dimension mean and standard deviation along the time axis. (n_frames, D) → (2D,).