Signal Complexity and Similarity

Initialize NeuroAnalyzer
using NeuroAnalyzer
import CairoMakie
# Quarto sets a default figure size (672 × 480 px), which overrides the sizes of
# NeuroAnalyzer figures: remove it
delete!(CairoMakie.Makie.CURRENT_DEFAULT_THEME, :resolution)
using DataFrames
NeuroAnalyzer.progress_bar = false

# preprocessed recording, EEG channels only, 10-second epochs
eeg = load_neuro("files/eeg/eeg.hdf")
eeg = keep_channel(eeg; ch = "eeg")
e10 = epoch(eeg; ep_len = 10)

Signal complexity describes how irregular or unpredictable a signal is over time. High complexity reflects rich, non-linear dynamics with many interacting components; low complexity indicates regular (e.g. periodic) or simple activity. Note that complexity measures do not separate “interesting” complexity from noise: white noise has maximal entropy.

Common measures of EEG complexity:

  1. Entropy (e.g. Shannon entropy, sample entropy): unpredictability of the signal; lower values indicate more regular activity (e.g. during seizures, deep sleep or anesthesia).
  2. Fractal dimension (e.g. Higuchi): self-similarity of the signal across time scales; higher values indicate greater irregularity.
  3. Hurst exponent: long-range temporal dependence (memory) of the signal.
  4. Lempel-Ziv complexity and compression ratio: the rate at which new patterns emerge in the signal (how well it can be compressed).
  5. Multiscale entropy: entropy across multiple time scales, capturing both fast and slow dynamics.

Signal similarity describes how alike two signals or segments are, e.g. when comparing brain states, subjects or conditions. Common measures are the correlation coefficient (see covariance and correlation), the Euclidean distance, dynamic time warping (alignment of signals that are shifted or vary in speed), cross-approximate entropy and phase synchronization measures (see the functional connectivity tutorials).

Applications:

  • Complexity: detecting pathological states (e.g. epilepsy, dementia), monitoring the depth of anesthesia, sleep stages and cognitive load.
  • Similarity: comparing EEG patterns across subjects or conditions, template matching.

Entropy and Negentropy

Entropy

Entropy is a measure of randomness or uncertainty of a signal; in information theory, it quantifies the average amount of information carried by each sample. For EEG, entropy describes the complexity or predictability of brain activity.

Types of entropy used in EEG analysis:

  • Shannon entropy: the average unpredictability of the values of the signal:

\[ H(X) = -\sum_{i} p(x_i) \log p(x_i) \]

where:
\(p(x_i)\) is the probability of observing the value \(x_i\).

  • Sample entropy (SampEn): the negative logarithm of the probability that sequences (patterns) of samples that are similar remain similar when extended by one sample; lower values indicate more regular, self-similar signals.
  • Approximate entropy (ApEn): similar to sample entropy, but it counts self-matches, which biases it towards regularity, especially in short signals.
  • Multiscale entropy: sample entropy calculated for progressively coarse-grained signals (multiple time scales).

entropy() calculates several entropy descriptors of each channel and epoch; the output is a named tuple of matrices (channels, epochs):

Field Descriptor
ent entropy of the amplitude histogram in bits (Freedman-Diaconis binning)
shent Shannon entropy (Wavelets.coefentropy())
leent log energy entropy (Wavelets.coefentropy())
sent sample entropy (ComplexityMeasures.jl)
nsent normalized sample entropy (ComplexityMeasures.jl)
dent differential entropy

The histogram-based entropy is calculated from the bins of the amplitude histogram as \(p = n / \sum n\) and \(H = -\sum p \log_2(p)\), where \(n\) are the bin counts.

Entropy descriptors of the first epoch:

en = entropy(e10; ch = "eeg")
DataFrame(channel = labels(e10),
          ent = en.ent[:, 1],
          shent = en.shent[:, 1],
          sent = en.sent[:, 1],
          nsent = en.nsent[:, 1],
          dent = en.dent[:, 1])
19×6 DataFrame
Row channel ent shent sent nsent dent
String Float64 Float64 Float64 Float64 Float64
1 Fp1 4.24009 6.93321 1.68794 0.112526 3.26591
2 Fp2 3.95028 7.06126 1.99486 0.132987 2.97821
3 F3 4.27511 7.00602 1.70076 0.113381 3.30144
4 F4 3.6983 7.0525 1.81586 0.121054 2.71435
5 C3 3.91185 7.02042 1.5052 0.100344 3.9389
6 C4 4.57081 6.94907 1.38467 0.0923085 3.59674
7 P3 4.28108 7.02493 1.41824 0.0945464 4.30687
8 P4 4.27382 7.02964 1.40163 0.0934395 4.30332
9 O1 4.31074 7.01744 1.46556 0.0977013 4.34744
10 O2 4.44053 7.0578 1.36613 0.0910727 4.47145
11 F7 4.40198 6.98066 1.80062 0.120038 3.43036
12 F8 3.63792 7.03341 1.41864 0.094573 3.65799
13 T3 4.00894 7.01327 1.5343 0.102283 4.03781
14 T4 3.99903 7.00057 1.33151 0.0887651 4.02646
15 T5 4.19589 6.98653 1.5326 0.102171 4.22335
16 T6 4.24882 7.00496 1.31909 0.0879365 4.27862
17 Fz 3.73033 7.10044 1.44281 0.0961847 2.75055
18 Cz 3.83868 6.95539 1.35993 0.0906594 3.86146
19 Pz 4.24495 6.98853 1.35406 0.0902681 4.27507

Sample entropy of all channels in the first epoch:

plot_bar(en.sent[:, 1];
         glabels = labels(e10),
         xlabel = "Channels",
         ylabel = "Sample entropy",
         title = "Sample entropy, epoch 1")

The amplitude-based descriptors (ent, shent, leent, dent) depend on the amplitude distribution (and thus on the variance) of the signal, while the sample entropy describes its temporal regularity. The lateral frontal channels (Fp1, Fp2, F3, F4, F7) have the highest sample entropy (they are the least regular), while the posterior channels, dominated by the slow posterior rhythm, have lower values.

Negentropy

Negentropy is a measure of non-Gaussianity: how much the distribution of a signal deviates from a Gaussian distribution with the same variance. Among all distributions with a given variance, the Gaussian has the highest entropy, so negentropy is always ≥ 0 and equals 0 for a Gaussian signal:

\[ J(Y) = H(Y_{Gaussian}) - H(Y) \]

where:
\(H(Y)\) is the differential entropy of \(Y\),
\(Y_{Gaussian}\) is a Gaussian random variable with the same variance as \(Y\).

  • High negentropy: the signal is far from Gaussian (structured, e.g. containing large transients, or multi-modal).
  • Low negentropy: the signal is close to Gaussian (e.g. a mixture of many independent sources, or noise).

Negentropy is a contrast function in independent component analysis (ICA): by the central limit theorem, mixtures of independent sources are closer to Gaussian than the sources themselves, so ICA finds components that maximize non-Gaussianity. Non-Gaussian components are more likely to represent meaningful sources (brain activity or artifacts such as eye blinks).

negentropy() calculates \(J = 0.5 \log_2(2 \pi e \sigma^2) - H\) (in bits), where \(H\) is the differential entropy (default, type = :diff); by default the signal is demeaned (demean = true) and normalized by its total energy (norm = true):

ne = negentropy(e10; ch = "eeg")
DataFrame(channel = labels(e10),
          negentropy = ne[:, 1])
19×2 DataFrame
Row channel negentropy
String Float64
1 Fp1 0.0105422
2 Fp2 -0.0161941
3 F3 -0.00422798
4 F4 -0.0140381
5 C3 -0.0092447
6 C4 0.00376175
7 P3 -0.0085111
8 P4 -0.00605389
9 O1 -0.0093919
10 O2 -0.0153249
11 F7 0.00303669
12 F8 -0.0104553
13 T3 -0.00672451
14 T4 0.000847541
15 T5 -0.00301494
16 T6 -0.00437277
17 Fz -0.0155774
18 Cz 0.0108948
19 Pz 0.00122859

The ongoing EEG is close to Gaussian, so its negentropy is close to 0; small negative values are estimation errors of the differential entropy.

Tip: type = :shannon and type = :sample use the Shannon or sample entropy instead of the differential entropy; these are not on the same scale as the Gaussian entropy, so the result is not a negentropy in the strict sense and is not ≥ 0.

Entropy vs. Negentropy

Feature Entropy Negentropy
Definition Measure of randomness Measure of non-Gaussianity
High value High randomness, unpredictability High structure, non-randomness
Use in EEG Assesses signal complexity Separates independent sources
Application Brain states, pathologies ICA, artifact detection

Higuchi Fractal Dimension

The Higuchi fractal dimension (HFD; Higuchi, 1988) measures the complexity and self-similarity of a time series directly from the signal: it estimates how the length of the curve changes with the time scale (lag \(k\)) at which it is measured. Unlike other fractal dimension methods, it does not require phase-space reconstruction, so it is computationally efficient and suitable for short and noisy signals.

Interpretation:

  • higher HFD (closer to 2): a more complex, irregular signal (noise-like)
  • lower HFD (closer to 1): a more regular, smooth signal

hfd() returns a matrix (channels, epochs):

h = hfd(e10; ch = ["C3", "C4", "O1", "O2"])
DataFrame(channel = ["C3", "C4", "O1", "O2"],
          hfd = h[:, 1])
4×2 DataFrame
Row channel hfd
String Float64
1 C3 1.77873
2 C4 1.84247
3 O1 1.75379
4 O2 1.72497

Generalised Hurst Exponents

The Hurst exponent \(H\) measures the long-term memory of a time series: how quickly its auto-correlation decreases as the lag between pairs of values increases. It quantifies the tendency of the signal either to regress to the mean or to persist in one direction:

  • \(0.5 < H \leq 1\): long-term positive auto-correlation (persistence); a high value tends to be followed by another high value, and the auto-correlation decays slowly, following a power law.
  • \(H = 0.5\): short memory; auto-correlations decay exponentially to zero (e.g. a random walk has independent increments).
  • \(0 \leq H < 0.5\): anti-persistence; a high value tends to be followed by a low value and vice versa, and this switching lasts a long time.

The generalised Hurst exponents (GHEs) \(H(q)\) describe the scaling of the \(q\)-th moment of the absolute increments of the signal with the lag \(\tau\); \(H(q)\) that varies with \(q\) indicates multifractal scaling.

ghexp() estimates the exponents over the lags tau_range (in samples) for the moments q_range; without q_range the standard Hurst exponent (\(q = 2\)) is returned. The output shape is (channels, q, 2, epochs); the third dimension holds the exponent and its standard deviation:

q = 0.5:0.5:2.0
ghe = ghexp(e10;
            ch = ["C3", "C4"],
            tau_range = 1:20,
            q_range = q)
DataFrame(q = collect(q),
          C3 = ghe[1, :, 1, 1],
          C4 = ghe[2, :, 1, 1])
4×3 DataFrame
Row q C3 C4
Float64 Float64 Float64
1 0.5 0.32238 0.380696
2 1.0 0.323695 0.376568
3 1.5 0.324261 0.371256
4 2.0 0.323736 0.364628

The exponents are below 0.5 (anti-persistence) and nearly constant across \(q\) (no multifractality) at these short lags (up to 20 samples, 78 ms).

For details, see the Hurst.jl documentation.

Summed Similarity

Summed similarity comes from models of memory and categorization (e.g. the generalized context model): the similarity of two items decreases exponentially with the distance between them in a feature space:

\[ s = e^{-\theta \times d} \]

where:
\(d\) is the Euclidean distance between the two signals,
\(\theta\) is the decay parameter (sensitivity).

Summing the similarities of an item to all items stored in memory gives its summed similarity (its familiarity).

sumsim() calculates the similarity of each pair of signals (channels ch1 of obj1 and ch2 of obj2, epochs ep1 and ep2); values are in the range \([0, 1]\), and higher values indicate greater similarity. The distance between two 10-second EEG segments is large (hundreds of μV), so \(\theta\) must be small for the similarity not to be 0:

ss = sumsim(e10,
            e10;
            ch1 = "O1",
            ch2 = "O2",
            ep1 = 1,
            ep2 = 1,
            theta = 0.001)
println("O1 vs O2: ", round(ss[1]; digits = 3))
ss = sumsim(e10,
            e10;
            ch1 = "O1",
            ch2 = "Fp1",
            ep1 = 1,
            ep2 = 1,
            theta = 0.001)
println("O1 vs Fp1: ", round(ss[1]; digits = 3))
O1 vs O2: 0.874
O1 vs Fp1: 0.787

Dirichlet Energy

Dirichlet energy measures the roughness of a discrete signal as the sum of squared differences between consecutive samples:

\[ E(s) = \sum_{i} (s_{i + 1} - s_i)^2 \]

  • A smooth, slowly varying signal has a low Dirichlet energy; it is 0 for a constant signal.
  • A rough signal (high-frequency activity, noise, sharp transitions such as artifacts or muscle activity) has a high Dirichlet energy.

The energy depends on both the amplitude and the frequency content of the signal (for a sinusoid it increases with the square of the amplitude and approximately with the square of the frequency). It can be used to detect segments with high-frequency artifacts, or as a smoothness penalty in signal denoising.

dirinrg() returns a matrix (channels, epochs):

de = dirinrg(e10; ch = ["Fp1", "O1"])
println("Fp1, epochs 1-3: ", round.(de[1, 1:3]; digits = 1))
println("O1, epochs 1-3:  ", round.(de[2, 1:3]; digits = 1))
Fp1, epochs 1-3: [6613.5, 5503.3, 5873.4]
O1, epochs 1-3:  [16832.2, 14828.0, 14784.4]

Zip Ratio

The zip ratio is a simple marker of signal complexity: the ratio of the size of the compressed data (zip -9) to the size of the uncompressed data (exported as CSV). Regular, predictable signals are compressed better, so lower values indicate lower complexity. zipratio() requires the zip command to be available in the system.

zr = zipratio(e10)
println("Zip ratio: ", round(zr; digits = 3))
Zip ratio: 0.471