Covariance and Correlation

Initialize NeuroAnalyzer
using NeuroAnalyzer
import CairoMakie
# Quarto sets a default figure size (672 × 480 px), which overrides the sizes of
# NeuroAnalyzer figures: remove it
delete!(CairoMakie.Makie.CURRENT_DEFAULT_THEME, :resolution)
using Statistics
NeuroAnalyzer.progress_bar = false

# preprocessed recording, EEG channels only, 10-second epochs
eeg = load_neuro("files/eeg/eeg.hdf")
eeg = keep_channel(eeg; ch = "eeg")
e10 = epoch(eeg; ep_len = 10)
# alpha band (8-13 Hz)
e10_alpha = NeuroAnalyzer.filter(e10;
                                 ch = "eeg",
                                 fprototype = :butterworth,
                                 ftype = :bp,
                                 cutoff = (8, 13),
                                 order = 4)
# raw (unfiltered) recording, 60-second segment
eeg_raw = import_edf("files/eeg/eeg.edf")
eeg_raw = crop(eeg_raw; seg = (100, 160))

In EEG analysis, covariance and correlation describe the relationships between signals recorded by different electrodes (or between a signal and its own delayed copy).

Applications in EEG analysis:

  • Connectivity analysis: covariance and correlation between channels are simple measures of functional connectivity; note that in the scalp EEG a high correlation of neighboring electrodes largely reflects volume conduction (the same source recorded by many electrodes).
  • Feature extraction: covariance matrices are features for classification of brain states (e.g. in brain-computer interfaces) and the basis of spatial filters (PCA, ICA, CSP, GED).
  • Artifact detection: artifacts common to many channels (e.g. eye blinks, line noise) increase the correlation between channels.
  • Rhythmicity: auto-covariance and auto-correlation reveal periodic activity in a single channel.

Covariance

Variance measures the spread of a single variable around its mean.

Covariance measures the degree to which two paired variables vary together, i.e. whether they tend to increase and decrease in tandem.

Corrected (sample) covariance:

\[ cov(x, y) = \frac{1}{n - 1} \sum_{i = 1}^{n} (x_i - \bar{x})(y_i - \bar{y}) \]

Uncorrected (population) covariance:

\[ cov(x, y) = \frac{1}{n} \sum_{i = 1}^{n} (x_i - \bar{x})(y_i - \bar{y}) \]

x = [1.0, 2.0, 3.0, 4.0]
y = [1.5, 2.0, 3.5, 4.0]
println("Corrected covariance:   ", cov(x, y; corrected = true))
println("Uncorrected covariance: ", cov(x, y; corrected = false))
Corrected covariance:   1.5
Uncorrected covariance: 1.125

Properties of covariance:

  • Units: covariance has the units of both variables (e.g. cm × kg for height and weight, μV² for two EEG channels), so its magnitude depends on the scale of the data: it indicates the direction of a relationship, but not its strength.
  • Mean-centering: the means are subtracted before the products are summed; when covariance is computed by matrix multiplication (see below), the data must be mean-centered first.
  • Symmetry: \(cov(x, y) = cov(y, x)\), so covariance alone cannot be used to infer causality or directionality.
  • Sign: positive when the variables tend to increase together, negative when one tends to increase as the other decreases, and zero when there is no linear relationship.

Covariance Matrix

The relationships between many variables are summarized by the variance-covariance matrix (covariance matrix), with the variances on the main diagonal and the pairwise covariances off the diagonal:

x y z
x var(x) cov(x, y) cov(x, z)
y cov(x, y) var(y) cov(y, z)
z cov(x, z) cov(y, z) var(z)

Interpretation:

  • Diagonal terms (variances): the power of each channel; large values indicate high variability.
  • Off-diagonal terms (covariances): large magnitudes indicate strong linear relationships between pairs of channels, and therefore redundancy (the two channels carry overlapping information).

For a multichannel signal \(X\) (channels × samples) with the mean of each channel subtracted, the covariance matrix (channels × channels) is:

\[ C = \frac{1}{n - 1} X X^T \]

# 3 channels × 100 samples
s = randn(3, 100)
# subtract the mean of each channel
s = s .- mean(s; dims = 2)
c = (s * s') ./ (size(s, 2) - 1)
# the same using Statistics.cov() (variables in columns)
println("Equal to cov(s'): ", c ≈ cov(s'))
Equal to cov(s'): true

Covariance is a foundation of many multichannel analyses, including principal component analysis (PCA), independent component analysis (ICA), source imaging of M/EEG data and least-squares fitting. Interpolating the variances (the diagonal) over the electrode locations gives a topographic map of the signal power, and seeded covariance maps show the covariance between a selected (seed) electrode and all other electrodes.

covm() calculates the covariance matrix of the selected channels in each epoch; the output shape is (channels, channels, epochs):

cm = covm(e10; ch = "eeg")
size(cm)
(19, 19, 100)

Plotting the matrix of the first epoch:

plot_matrix(cm[:, :, 1];
            xlabels = labels(e10),
            ylabels = labels(e10),
            title = "Covariance matrix, epoch 1",
            cb_title = "[μV²]")

The posterior (parietal, occipital and posterior temporal) channels have the largest variances and covariances: they share the dominant posterior rhythm, which in this recording is slow (about 5–6 Hz).

Correlation

Correlation is covariance normalized by the standard deviations of both variables, so that it is in the range \([-1, 1]\) and does not depend on the units or scale of the data:

\[ cor(x, y) = \frac{cov(x, y)}{s_x \times s_y} \]

Similarly, a correlation matrix is a covariance matrix normalized by the standard deviations of the variables; its diagonal is always 1.

Two correlation coefficients are commonly used:

  • Pearson correlation: the strength of a linear relationship; the standard choice when linearity and normality can be assumed.
  • Spearman correlation: the strength of a monotonic relationship; the values are first converted to ranks (e.g. \([0.1, 1, 10, 100]\) becomes \([1, 2, 3, 4]\)). It makes no assumption about linearity or distribution, so it is more appropriate for non-Gaussian data such as power spectra, firing rates or pixel values.

Geometrically, the Pearson correlation is the cosine of the angle between the two mean-centered vectors (cosine similarity of mean-centered data).

# 3 channels × 100 samples
s = randn(3, 100)
# channel × channel correlation matrix (variables in columns)
cor(s')
3×3 Matrix{Float64}:
 1.0       0.11746  0.131131
 0.11746   1.0      0.10249
 0.131131  0.10249  1.0

corm() calculates the correlation matrix of the selected channels in each epoch; the output shape is (channels, channels, epochs):

cm = corm(e10; ch = "eeg")
plot_matrix(cm[:, :, 1];
            xlabels = labels(e10),
            ylabels = labels(e10),
            title = "Correlation matrix, epoch 1")

Neighboring electrodes are strongly correlated (e.g. O1–O2, P3–Pz, T5–O1), mostly due to volume conduction and the common reference.

Auto- and Cross- Forms

Covariance and correlation are also calculated as a function of the lag \(L\) (the time shift of one signal relative to the other):

  • auto- forms compare a signal with a shifted copy of itself,
  • cross- forms compare a signal with a shifted copy of another signal.

The cross-covariance of two zero-mean signals \(x\) and \(y\) at lag \(L\) is:

\[ C_{xy}(L) = \frac{1}{N} \sum_{n = 0}^{N - 1 - L} x(n) \times y(n + L) \]

where:
\(N\) is the number of samples,
\(L\) is the lag (in samples).

The auto-covariance is the special case \(y = x\); at lag \(L = 0\) it equals the variance of the signal:

\[ C_{xx}(0) = \frac{1}{N} \sum_{n = 0}^{N - 1} x(n)^2 = \sigma_x^2 \]

The correlation forms are normalized by the standard deviations, which constrains them to the range \([-1, 1]\):

\[ R_{xy}(L) = \frac{C_{xy}(L)}{\sigma_x \times \sigma_y} \]

\[ R_{xx}(L) = \frac{C_{xx}(L)}{C_{xx}(0)} = \frac{C_{xx}(L)}{\sigma_x^2} \]

so that the auto-correlation at lag 0 is always 1.

Biased and unbiased estimators: the biased estimator divides the sum by \(N\) at every lag (as above), although only \(N - L\) pairs of samples contribute to the sum at lag \(L\); the estimate therefore decreases linearly towards large lags. It has a lower variance and is guaranteed to be positive semi-definite, which many methods (e.g. autoregressive modeling) require. The unbiased estimator divides the sum by \(N - L\); it corrects the decrease, but the estimates at large lags are based on fewer pairs of samples and are much noisier. Estimates at lags approaching the signal length are unreliable regardless of the estimator, so the lags should be much shorter than the signal.

Auto-covariance

Auto-covariance describes how well the signal at time \(n\) predicts the signal at a later (or earlier) time, and so detects rhythmic or periodic activity.

Algorithm:

  1. subtract the mean from the signal (demean = true, default)
  2. multiply each sample \(x(n)\) by the sample \(x(n + L)\)
  3. sum the products over all \(n\) and divide by \(N\) (biased, default) or \(N - L\) (unbiased, biased = false)

At lag \(L = 0\) the auto-covariance reaches its maximum: the signal is perfectly aligned with itself. For a periodic signal:

  • positive peaks occur at lags of full cycles of the dominant frequency, e.g. every 1/50 = 0.02 s for 50 Hz line noise, where the shifted signal realigns with itself;
  • negative peaks occur at lags of half-cycles, e.g. 0.01 s, 0.03 s, … for 50 Hz noise, where the shifted signal is in anti-phase with the original.

acov() calculates the auto-covariance of each channel and epoch for lags -l:l (in samples); the output is a named tuple: autocov (channels, 2l + 1, epochs) and lags (in seconds). The raw recording contains strong line noise:

ac = acov(eeg_raw;
          ch = "Fp2",
          l = 20,
          biased = false)
plot_xac(ac.autocov[1, :, 1],
         ac.lags;
         ylabel = "Auto-covariance [μV²]",
         title = "Auto-covariance, Fp2")

The auto-covariance shows positive peaks every 0.02 s and negative peaks in between, produced by the dominant 50 Hz line noise.

axc2frq() detects the peaks of an auto-/cross-covariance or correlation and converts the intervals between them into frequencies:

ac = acor(eeg_raw;
          ch = "Fp2",
          l = 10)
frq = axc2frq(ac.autocor[1, :, 1], ac.lags)
println("Dominant frequency: ", frq, " Hz")
Dominant frequency: [50.0] Hz

This can be verified in the power spectrum:

plot_psd(eeg_raw; ch = "Fp2")

Auto-correlation

Auto-correlation is the auto-covariance normalized to 1 at lag 0, i.e. the Pearson correlation between a signal and its delayed copy as a function of the lag.

  • For periodic signals, the auto-correlation oscillates at the frequency of the dominant rhythm: the lag of its first positive peak is the length of one cycle.
  • For random signals (white noise), the auto-correlation is near zero at all non-zero lags.

Auto-correlation therefore answers two questions:

  1. Is the signal periodic?: look for oscillations in the auto-correlation function; a value at lag 1 significantly different from zero indicates temporal structure in the data.
  2. What is the period?: the lag of the first positive peak is the length of one cycle; the auto-correlation over many lags also characterizes the dependence structure, e.g. for choosing a time series model.

Wiener-Khinchin theorem: the auto-correlation function and the power spectral density are a Fourier transform pair.

acor() calculates the auto-correlation for lags -l:l (in samples); the output is a named tuple: autocor (channels, 2l + 1, epochs) and lags (in seconds). The auto-correlation of O1 over ±0.5 s:

ac = acor(e10;
          ch = "O1",
          l = 128)
plot_xac(ac.autocor[1, :, 1],
         ac.lags;
         ylabel = "Auto-correlation",
         title = "Auto-correlation, O1, epoch 1")

The auto-correlation falls to near zero within about 50 ms and shows only weak, quickly damped oscillations: the broadband EEG is dominated by aperiodic (1/f) activity, and its rhythms are neither strong nor sustained enough to dominate the auto-correlation. For a strongly rhythmic signal (such as the line noise above) the oscillations persist over many cycles.

Cross-covariance

Cross-covariance extends auto-covariance to two different signals: it measures their similarity as a function of the lag of one signal relative to the other. It detects shared periodic activity and estimates time delays between signals.

xcov() calculates the cross-covariance between channels ch1 of obj1 and ch2 of obj2 (which may be the same object) for lags -l:l, where l is in seconds; the output is a named tuple: xc (channel pairs, lags, epochs) and lags (in seconds). Cross-covariance of the alpha activity of O1 and O2:

xc = xcov(e10_alpha,
          e10_alpha;
          ch1 = "O1",
          ch2 = "O2",
          l = 0.5)
plot_xac(xc.xc[1, :, 1],
         xc.lags;
         ylabel = "Cross-covariance [μV²]",
         title = "Cross-covariance of alpha, O1 vs O2, epoch 1")

Cross-correlation

Cross-correlation is the cross-covariance normalized by the standard deviations of both signals, so that it is in the range \([-1, 1]\). Computationally it is a sliding dot product: at each lag, the dot product of the two signals is calculated, and it peaks at the lag at which the signals are most aligned.

xc = xcor(e10_alpha,
          e10_alpha;
          ch1 = "O1",
          ch2 = "O2",
          l = 0.5)
plot_xac(xc.xc[1, :, 1],
         xc.lags;
         ylabel = "Cross-correlation",
         title = "Cross-correlation of alpha, O1 vs O2, epoch 1")

The lag of the maximum cross-correlation is the delay between the two signals (a positive lag means that ch2 lags behind ch1):

r = xc.xc[1, :, 1]
idx = argmax(r)
println("Maximum correlation: ", round(r[idx]; digits = 2))
println("At lag:              ", round(xc.lags[idx] * 1000; digits = 1), " ms")
Maximum correlation: 0.89
At lag:              0.0 ms

The alpha activity of O1 and O2 is strongly correlated at zero lag. A peak at a non-zero lag may indicate that the activity of one region precedes the other; however, zero-lag correlation of scalp EEG channels is dominated by volume conduction, so cross-correlation is not a reliable measure of connectivity between brain regions (see the functional connectivity tutorials for measures that are insensitive to zero-lag coupling).