predicting human brain resting states using transformers

this are some notes i put together while reading predicting human brain resting states using transformers

abstract:

the human brain is complex and a highly dynamic system. our current understanding of its functional mechanisms is still very limited.

we can use functional magnetic resonance imaging (fMRI) to analyze blood oxygen level-dependent (BOLD) changes, reflecting neural activity to infer brain states and dynamics.

the paper explores the possibility of the use of transformers to predict human brain resting states based on the large-scale high-quality fMRI data from the human connectome project (HCP). the HCP is a research initiative aimed at mapping the neural connections of the human brain.

intro:

what is fMRI?

it is a widely used non-invasive and in-vivo technique to observe whole brain dynamics spatially at the meso-scale and temporally at the second scale.

the meso-scale refers to an intermediate level of detail in spatial resolution, capturing clusters of neurons or specific functional areas. not down to individual neurons or synapses. it's a middle ground between very fine (microscopic) and very broad (macroscopic) levels of detail.

the second scale refers to the temporal resolution of fMRI, meaning that it can capture changes in brain activity over periods of seconds. while fMRI cannot provide millisecond-level precision like some other methods (e.g., EEG), it is well-suited for observing slower dynamics, such as blood flow changes related to neural activity.

why predict brain states?

reduce scan time: fMRI scans can be lengthy and challenging for some individuals, such as those with movement disorders or young children. if we can predict future brain states accurately, it may be able to shorten the scan time without losing important information.
early detection of disorders: some brain disorders, like epilepsy, have distinct patterns of brain activity that can be detected with fMRI. by predicting future brain states, it may be able to identify these patterns earlier and provide timely interventions.

why use transformers?

given the ability of transformers to find long-distance relationships between data tokens (in our case, brain states) grounded in correlations and with links to graph theory, self-attention-based architectures could have the ability to learn patterns from sequential brain activity and predict the upcoming brain states.

methods:

data preprocessing:

the dataset was preprocessed to remove noise and unwanted signals, and the brain activity signals were smoothed to reduce random noise and make the signals clearer.
the data was filtered to keep only the brain activity signals we were interested in, which change within a specific speed range.
the data was adjusted so the average value was zero, and the range of variation was consistent across the dataset.
the brain was divided into 379 regions, and the activity for each region was measured and averaged over time to create a single "snapshot" of brain activity for all regions at each moment.

model architecture:

the model used a transformer architecture, which is a type of neural network that excels at processing sequential data.
the transformer processes time-series data as a sequence of small pieces, called tokens, over a specific time window. it uses a method called "self-attention" to find connections between these tokens, but since it doesn’t automatically know the order, they added information about the order using a mathematical pattern (sine and cosine functions).
the model had two main parts: an encoder and a decoder. the encoder analyzed the input data, and the decoder predicted the next brain state based on the encoder’s output.

training:

the task of predicting brain states was set up as an autoregression task, meaning the model predicts the next brain state based on the previous ones.
to measure how good or bad the model’s predictions were, they used mean squared error (MSE). this calculates the average difference between the predicted values and the actual values, with bigger penalties for larger errors.
to improve the model during training, they used an algorithm called the adam optimizer, which adjusts the model’s settings step by step to make it more accurate.
the optimizer started with a small learning rate of 0.0001 (10⁻⁴), which controls how big each adjustment step was, to ensure smooth and stable training.

evaluation:

the model's ability to predict a longer sequence of brain states was tested by starting with a small amount of real fMRI data (30 time points).
using the 30 real data points, the model predicted the next point. this prediction was added to the sequence, and the input window was shifted to include it for the next prediction.
this process was repeated until a full sequence of synthetic data (1150 time points) was created, matching the length of the real data (1200 time points).
compared the predicted sequence with the actual data using:
MSE: to measure how close the predictions were to the actual data.
spearman’s correlation: to check if the predicted and actual brain states followed similar patterns, even if they weren't identical.

results and discussion:

results provide strong evidence that the transformer model effectively learned and leveraged the temporal dependencies present in the fMRI time series.

overall, results of the time series prediction show that future brain states of around 5.04s can be accurately predicted using a short fMRI sequence of only 21.6s.

conclusion:

the results demonstrated that this method could actually learn the temporal dependencies of brain states over time and accurately predict approximately 5.04s based on 21.6s of fMRI data.

this shows the possibility of using a short fMRI segment for future brain state prediction.

Back to home