Neuroscience |

Watershed by Flooding: Applied Data Structures

Fri, 01 May 2020 11:12:32 -0700

I’m applying some methods developed in this paper for testing purposes in my own thesis research. Specifically, I have some float-valued data, $F$, that varies along the cortical surface of the brain. Visually, I can see that there are areas where these scalar maps change abruptly. I want to identify these boundaries – eventually, I’ll segment out the regions I’m interested in.

Computing the Gradient Map

The authors use some conventional brain imaging software to compute the gradient of their data. The domain of this data is a triangulated mesh, described by the graph $G = (V, E)$, where $V$ are vertices in Euclidean space and $E$ the edges between these vertices. In short, for a vertex, $v_{i}$, we first need to compute the gradient vector of the scalar field at $v_{i}$. We “unfold” the 3D positions of adjacent vertices onto the tangent plane of $v_{i}$ – which we can do by orthogonally projecting the adjacent vertices onto the affine subspace at $v_{i}$ and then weighting appropriately. We then regress the graph signal (our scalar field) onto these unfolded positions. The $L_{2}$ norm of this vector is the gradient at $v_{i}$.

For each vertex, we have a normal vector to the surface $N$, its spatial 3D coordinates $v_{i} = (x, y, z)$, and a list of its adjacent vertices. We can compute the orthogonal projector onto the affine subspace spanned by $N$, $P_{N}$, and its orthogonal complement, $Q_{N}$, as:

$$\begin{align} P_{N} &= N(N^{T}N)^{-1}N^{T} \\ Q_{N} &= I - P_{N} \end{align}$$

For any vertex, $v_{j}$, we can compute the orthogonal projection onto the affine subspace spanned by $Q_{N}$ as:

$$\begin{align} q(v_{j}) = Q_{N}(v_{j} - v_{i}) + v_{i} \end{align}$$

We generate the vectors

$$ S_{i} = f(i) - \begin{bmatrix} f(v_{1}) \\ f(v_{2}) \\ \vdots \\ f(v_{j}) \end{bmatrix} \in \mathbb{R}^{j} \;\;\; R_{i} = \begin{bmatrix} q(v_{1}) \\ q(v_{2}) \\ \vdots \\ q(v_{j}) \end{bmatrix} \in \mathbb{R}^{j \times 3} $$

where $S_{i}$ is the difference between the scalar value at our vertex $f(v_{i})$ and the vector of adjacent $j$ scalar field values, and $R_{i}$ is the matrix of $j$ orthogonally projected adjacent vertex coordinates. Then we perform least squares regression to solve for $\beta$:

$$\begin{align} S_{i} = R_{i}\beta \end{align}$$

where $\beta \in \mathbb{R}^{3}$, which indicates how much each coordinate axis contributes to variation in the scalar field at $v_{i}$. The gradient value at vertex $v_{i} = \left || \beta \right||_{2}$.

Watershed By Flooding Algorithm

The new scalar field of $L_{2}$ norms is our gradient field, which describes how “quickly” our original data changes at each vertex. We can now apply the Watershed Algorithm to segment our mesh data. In brief, the watershed algorithm treats the gradient field as a topographic map: low-elevation areas (areas with a small gradient) are “water basins”. If we imagine water flooding this map from the bottom up, basins at low elevation will flood first, while areas at higher elevations will fill last. When water basins meet, the water has reached a “boundary” (or ridgeline, if we’re using the topographic map idea).

I’ve implemented an algorithm variant called “Priority Flooding”, using Python’s heapq priority queue data type class. The priority queue is an application of the binary heap data structure – as nodes are added to the heap, the branching process determines where to put nodes (left or right of a current node), based on some value – in the case of the priority queue, this value is the “priority”. We utilize the priority queue because it gives us a principled way to iterate over unlabeled vertices, and, with some auxiliary data structures, is guaranteed to converge. The algorithm proceeds as follows:

Identify local minima, and assign each minimum a unique label
Add directly adjacent vertices of local minima to priority queue (lower gradient -> higher priority)
While the queue is not empty, get the highest-priority item
- If vertices adjacent to this vertex have only one label, assign this vertex to that label
- Else assign this vertex as a boundary vertex
- Add unlabeled adjacent vertices to the queue

import numpy as np
from queue import PriorityQueue

class PriorityFlood(object):

 """
 Class to segment a scalar field using the Priority Flooding
 watershed algorithm.
 """

 def __init__(self):

 self.pq = PriorityQueue()

 def fit(self, gradient, A, M):

 """
 Fitting procedure for watershed algorithm.

 Parameters:
 - - - - -
 gradient: float, array
 map of gradient values of scalar field
 A: dict of lists
 adjacency list of data domain
 M: list
 local minima in scalar field used to seed algorithm
 """

 # initialize empty label vector
 labels = np.zeros((gradient.shape))
 labels[:] = np.nan

 # keep track of items in queue
 in_queue = np.zeros((gradient.shape)).astype(np.bool)

 # assign local minima unique labels
 # add their adjacent vertices to heap
 for i, loc_min in enumerate(M):

 # label local minima
 labels[loc_min] = i+1

 # add neighbors of local minima to p-queue
 for nidx in A[loc_min]:
 in_queue[nidx] = True
 self.pq.put((gradient[nidx], nidx))

 # iterate over p-queue items
 # items assigned to a label or
 # assigned as a boundary
 while not self.pq.empty():

 # get highest priority item
 # mark as not in p-queue
 [gr, idx] = self.pq.get()
 in_queue[idx] = False

 # get item neighbors and their labels
 i_neighbors = np.asarray(A[idx])
 i_nlabels = labels[i_neighbors]

 # get labels of adjacent vertices that are not nan
 nans = np.isnan(i_nlabels)
 unique_labels = np.unique(i_nlabels[~nans])

 # if more than one unique label
 # assign current vertex as border vertex
 if len(unique_labels) > 1:
 continue

 # otherwise assign to water basin
 else:

 # basin assignment
 labels[idx] = unique_labels

 # identify neighbors without labels that arent in the p-queue
 gidx = np.where((nans) & (~in_queue[i_neighbors]))[0]

 # add these to p-queue
 for nidx in i_neighbors[gidx]:
 if not in_queue[nidx]:
 in_queue[nidx] = True
 self.pq.put((gradient[nidx], nidx))

 self.labels_ = labels

Example of PriorityFlooding algorithm, applied to gradient of inferiorparietal region. Shown on inflated and flattened cortical surfaces.

One caveat that came up is that the PriorityQueue class does not check for duplicates – that is, two vertices might share an adjacent vertex, and this vertex might have already been added to the heap. In this case, we would unnecessarily view the same vertices many times. To alleviate this, we create a boolean Numpy array, in_queue, that stores whether an item is already in the queue.

Mahalanobis Distances of Brain Connectivity

Fri, 07 Dec 2018 05:12:32 -0700

For one of the projects I’m working on, I have an array of multivariate data relating to brain connectivity patterns. Briefly, each brain is represented as a surface mesh, which we represent as a graph $G = (V,E)$, where $V$ is a set of $n$ vertices, and $E$ is the set of edges between vertices.

Additionally, for each vertex $v \in V$, we also have an associated scalar label, which we’ll denote $l(v)$, that identifies what region of the cortex each vertex belongs to, the set of regions which we define as $L = {1, 2, … k}$. And finally, for each vertex $v \in V$, we also have a multivariate feature vector $r(v) \in \mathbb{R}^{1 \times k}$, that describes the strength of connectivity between it, and every region $l \in L$.

Example of cortical map, and array of connectivity features.

I’m interested in examining how “close” the connectivity samples of one region, $l_{j}$, are to another region, $l_{k}$. In the univariate case, one way to compare a scalar sample to a distribution is to use the $t$-statistic, which measures how many standard deviations away from the mean a given sample is:

$$\begin{align} t_{s} = \frac{\bar{x} - \mu}{\frac{s}{\sqrt{n}}} \end{align}$$

where $\mu$ is the population mean, and $s$ is the sample standard deviation. If we square this, we get:

$$\begin{align} t^{2} = \frac{(\bar{x} - \mu)^{2}}{\frac{s^{2}}{n}} = \frac{n (\bar{x} - \mu)^{2}}{S^{2}} \sim F(1,n) \end{align}$$

We know the last part is true, because the numerator and denominator are independent $\chi^{2}$ distributed random variables. However, I’m not working with univariate data – I have multivariate data. The multivariate generalization of the $t$-statistic is the Mahalanobis Distance:

$$\begin{align} d &= \sqrt{(\bar{x} - \mu)\Sigma^{-1}(\bar{x}-\mu)^{T}} \end{align}$$

where the squared Mahalanobis Distance is:

$$\begin{align} d^{2} &= (\bar{x} - \mu)\Sigma^{-1}(\bar{x}-\mu)^{T} \end{align}$$

where $\Sigma^{-1}$ is the inverse covariance matrix. If our $X$’s were initially distributed with a multivariate normal distribution, $N_{p}(\mu,\Sigma)$ (assuming $\Sigma$ is non-degenerate i.e. positive definite), the squared Mahalanobis distance, $d^{2}$ has a $\chi^{2}_{p}$ distribution. We show this below.

We know that $(X-\mu)$ is distributed $N_{p}(0,\Sigma)$. We also know that, since $\Sigma$ is symmetric and real, that we can compute the eigendecomposition of $\Sigma$ as:

$$\begin{align} \Sigma = U \Lambda U^{T} \end{align}$$

and consequently, because $U$ is an orthogonal matrix, and because $\Lambda$ is diagonal, we know that $\Sigma^{-1}$ is:

$$\begin{align} \Sigma^{-1} &= (U \Lambda U^{T})^{-1} \\ &= U \Lambda^{-1} U^{T} \\ &= (U \Lambda^{\frac{-1}{2}}) (U \Lambda^{\frac{-1}{2}})^{T} \\ &= R R^{T} \end{align}$$

Therefore, we know that $R^{T}(X-\mu) \sim N_{p}(0,I_{p})$:

$$\begin{align} X &\sim N_{p}(\mu,\Sigma) \\ (X-\mu) = Y &\sim N_{p}(0,\Sigma)\\ R^{T}Y = Z &\sim N_{p}(0, R^{T} \Sigma R) \\ &\sim N_{p}(0, \Lambda^{\frac{-1}{2}} U^{T} (U \Lambda U^{T}) U \Lambda^{\frac{-1}{2}}) \\ &\sim N_{p}(0, \Lambda^{\frac{-1}{2}} I_{p} \Lambda I_{p} \Lambda^{\frac{-1}{2}}) \\ &\sim N_{p}(0,I_{p}) \end{align}$$

so that we have

$$\begin{align} &= (X-\mu)\Sigma^{-1}(X-\mu)^{T} \\ &= (X-\mu)RR^{T}(X-\mu)^{T} \\ &= Z^{T}Z \end{align}$$

the sum of $p$ squared standard Normal random variables, which is the definition of a $\chi_{p}^{2}$ distribution with $p$ degrees of freedom. So, given that we start with a $MVN$ random variable, the squared Mahalanobis distance is $\chi^{2}_{p}$ distributed. Because the sample mean and sample covariance are consistent estimators of the population mean and population covariance parameters, we can use these estimates in our computation of the Mahalanobis distance.

Also, of particular importance is the fact that the Mahalanobis distance is not symmetric. That is to say, if we define the Mahalanobis distance as:

$$\begin{align} M(A, B) = \sqrt{(A - \mu(B))\Sigma(B)^{-1}(A-\mu(B))^{T}} \end{align}$$

then $M(A,B) \neq M(B,A)$, clearly. Because the parameter estimates are not guaranteed to be the same, it’s straightforward to see why this is the case.

Now, back to the task at hand. For a specified target region, $l_{T}$, with a set of vertices, $V_{T} = {v \; : \; l(v) \; = \; l_{T}, \; \forall \; v \in V}$, each with their own distinct connectivity fingerprints, I want to explore which areas of the cortex have connectivity fingerprints that are different from or similar to $l_{T}$’s features, in distribution. I can do this by using the Mahalanobis Distance. And based on the analysis I showed above, we know that the data-generating process of these distances is related to the $\chi_{p}^{2}$ distribution.

First, I’ll estimate the covariance matrix, $\Sigma_{T}$, of our target region, $l_{T}$, using the Ledoit-Wolf estimator (the shrunken covariance estimate has been shown to be a more reliable estimate of the population covariance), and mean connectivity fingerprint, $\mu_{T}$. Then, I’ll compute $d^{2} = M^{2}(A,A)$ for every $\{v: v \in V_{T}\}$. The empirical distribution of these distances should follow a $\chi_{p}^{2}$ distribution. If we wanted to do hypothesis testing, we would use this distribution as our null distribution.

Next, in order to assess whether this intra-regional similarity is actually informative, I’ll also compute the similarity of $l_{T}$ to every other region, $\{ l_{k} \; : \; \forall \; k \in L \setminus \{T\} \}$ – that is, I’ll compute $M^{2}(A, B) \; \forall \; B \in L \setminus T$. If the connectivity samples of our region of interest are as similar to one another as they are to other regions, then $d^{2}$ doesn’t really offer us any discriminating information – I don’t expect this to be the case, but we need to verify this.

Then, as a confirmation step to ensure that our empirical data actually follows the theoretical $\chi_{p}^{2}$ distribution, I’ll compute the location and scale Maximum Likelihood(MLE) parameter estimates of our $d^{2}$ distribution, keeping the d.o.f. (i.e. $p$) fixed.

See below for Python code and figures…

Step 1: Compute Parameter Estimates

%matplotlib inline
import matplotlib.pyplot as plt

from matplotlib import rc
rc('text', usetex=True)

import numpy as np
from scipy.spatial.distance import cdist
from scipy.stats import chi2, probplot

from sklearn import covariance

# lab_map is a dictionary, mapping label values to sample indices
# our region of interest has a label of 8
LT = 8

# get indices for region LT, and rest of brain
lt_indices = lab_map[LT]
rb_indices = np.concatenate([lab_map[k] for k in lab_map.keys() if k != LT])

data_lt = conn[lt_indices, :]
data_rb = conn[rb_indices, :]

# fit covariance and precision matrices
# Shrinkage factor = 0.2
cov_lt = covariance.ShrunkCovariance(assume_centered=False, shrinkage=0.2)
cov_lt.fit(data_lt)
P = cov_lt.precision_

Next, compute the Mahalanobis Distances:

# LT to LT Mahalanobis Distance
dist_lt = cdist(data_lt, data_lt.mean(0)[None,:], metric='mahalanobis', VI=P)
dist_lt2 = dist_lt**2

# fit covariance estimate for every region in cortical map
EVs = {l: covariance.ShrunkCovariance(assume_centered=False,
 shrinkage=0.2) for l in labels}

for l in lab_map.keys():
 EVs[l].fit(conn[lab_map[l],:])

# compute d^2 from LT to every cortical region
# save distances in dictionary
lt_to_brain = {}.fromkeys(labels)
for l in lab_map.keys():

 temp_data = conn[label_map[l], :]
 temp_mu = temp_data.mean(0)[None, :]

 temp_mh = cdist(data_lt, temp_mu, metric='mahalanobis', VI=EVs[l].precision_)
 temp_mh2 = temp_mh**2

 lt_to_brain[l] = temp_mh2

# plot distributions seperate (scales differ)
fig = plt.subplots(2,1, figsize=(12,12))
plt.subplot(2,1,1)
plt.hist(lt_to_brain[LT], 50, density=True, color='blue',
 label='Region-to-Self', alpha=0.7)

plt.subplot(2,1,2)
for l in labels:
 if l != LT:
 plt.hist(lt_to_brain[l], 50, density=True, linewidth=2,
 alpha=0.4, histtype='step')

Empirical distributions of within-region (top) and between-region (bottom) $d^{2}$ values. Each line is the distribution of the distance of samples in our ROI to a whole region.

As expected, the distribution of $d^{2}$, the distance of samples in our region of interest, $l_{T}$, to distributions computed from other regions, is (considerably) larger and much more variable, while the profile of points within $l_{T}$ looks to have much smaller variance – this is good! This means that we have high intra-regional similarity when compared to inter-regional similarities. This fits what’s known in neuroscience as the “cortical field hypothesis”.

Step 2: Distributional QC-Check

Because we know that our data should follow a $\chi^{2}_{p}$ distribution, we can fit the MLE estimate of our location and scale parameters, while keeping the $df$ parameter fixed.

p = data_lt.shape[1]
mle_chi2_theory = chi2.fit(dist_lt2, fdf=p)

xr = np.linspace(data_lt.min(), data_lt.max(), 1000)
pdf_chi2_theory(xr, *mle_chi2_theory)

fig = plt.subplot(1,2,2,figsize=(18, 6))

# plot theoretical vs empirical null distributon
plt.subplot(1,2,1)
plt.hist(data_lt, density=True, color='blue', alpha=0.6,
 label = 'Empirical')
plt.plot(xr, pdf_chi2_theory, color='red',
 label = '$\chi^{2}_{p}')

# plot QQ plot of empirical distribution
plt.subplot(1,2,2)
probplot(D2.squeeze(), sparams=mle_chi2_theory, dist=chi2, plot=plt);

Density and QQ plot of null distribution.

From looking at the QQ plot, we see that the empirical density fits the theoretical density pretty well, but there is some evidence that the empirical density has heavier tails. The heavier tail of the upper quantile could probability be explained by acknowledging that our starting cortical map is not perfect (in fact there is no “gold-standard” cortical map). Cortical regions do not have discrete cutoffs, although there are reasonably steep gradients in connectivity. If we were to include samples that were considerably far away from the the rest of the samples, this would result in inflated densities of higher $d^{2}$ values.

Likewise, we also made the distributional assumption that our connectivity vectors were multivariate normal – this might not be true – in which case our assumption that $d^{2}$ follows a $\chi^{2}_{p}$ would also not hold.

Finally, let’s have a look at some brains! Below, is the region we used as our target – the connectivity profiles from vertices in this region were used to compute our mean vector and covariance matrix – we compared the rest of the brain to this region.

Region of interest.

Estimated squared Mahalanobis distances, overlaid on cortical surface.

Here, larger $d^{2}$ values are in red, and smaller $d^{2}$ are in black. Interestingly, we do see pretty large variance of $d^{2}$ spread across the cortex – however the values are smoothly varying, but there do exists sharp boundaries. We kind of expected this – some regions, though geodesically far away, should have similar connectivity profiles if they’re connected to the same regions of the cortex. However, the regions with connectivity profiles most different than our target region are not only contiguous (they’re not noisy), but follow known anatomical boundaries, as shown by the overlaid boundary map.

This is interesting stuff – I’d originally intended on just learning more about the Mahalanobis Distance as a measure, and exploring its distributional properties – but now that I see these results, I think it’s definitely worth exploring further!

Image Transformations With OpenCV

Sat, 01 Sep 2018 17:12:32 -0700

I’ve been toying around with openCV for generating MRI images with synethetic motion injected into them. I’d never used this library before, so I tested a couple examples. Below I detail a few tools that I found interesting, and that can quickly be used to generate image transformations.

# import necessary libraries
import matplotlib.pyplot as plt
import nibabel as nb
import cv2

# load image file
image_file = './data/T1w_restore_brain.nii.gz'
img_obj = nb.load(image_file)
img = img_obj.get_data()

# reorient so Anterior-Posterior axis corresponds to dim(0)
img = np.fliplr(img)
img = np.swapaxes(img, 0, 2)

# get single image slice and rescale
data = img[130, :, :]
data = (data-data.min())/data.max()
plt.imshow(data)

For any linear transformations with cv2, we can use the cv2.warpAffine method, which takes in the original image, some transformation matrix, and the size of the output image.

Let’s start with translations. The matrix will translate the image 10 pixels to the right (width), and 0 pixels down (height).

# Use the identity rotation matrix
# Third column specifies translation in corresponding direction
translation = np.array([[1, 0, 20],
 [0, 1, 0]])

translated = cv2.warpAffine(data, translation, data.T.shape)
plt.imshow(translated)

Now, in order to rotate the image, we can use cv2.getRotationMatrix2D. We’ll rotate our image by 45$^{\circ}$ .

# get shape of input image
rows, cols = data.shape

# specify angle of rotation around central pixel
M = cv2.getRotationMatrix2D((cols/2,rows/2), 45, 1)
rotated = cv2.warpAffine(data, M, (cols, rows))
plt.imshow(rotated)

Here are a few examples of randomly translating +/- 1, 5, or 9 voxels in the X and Y directions, and randomly rotating by 1, 5, or 9 degrees:

# get shape of input image
rows, cols = data.shape
# specify range of rotations and translations
txfn = [1, 5, 9]

for rt in txfn:

 # generate rotation matrix
 # randonly rotate to left or right
 M = cv2.getRotationMatrix2D(
 (cols/2, rows/2),
 np.random.choice([-1, 1], 1)[0]*rt, 1)

 # apply rotation matrix
 rotated = cv2.warpAffine(data, M, data.T.shape)

 # generate translation matrix
 # randomly translate to left or right
 T = np.array(
 [[1,0,np.random.choice([-1, 1], 1)[0]*rt],
 [0, 1,np.random.choice([-1, 1], 1)[0]*rt]]).astype(np.float32)

 # apply translation matrix
 translated = cv2.warpAffine(data, T, data.T.shape)

 # compose rotated and translated images
 movement = (rotated + translated)/2
 # compute difference between input and transformed
 difference = data-movement
 res = difference.reshape(np.product(difference.shape))

 fig,[ax1,ax2,ax3] = plt.subplots(1,3,figsize=(15,5))
 ax1.imshow(movement, cmap='gray')
 ax1.set_title('Composed Random Rotation and Translation \n Magnitude = {:}'.format(rt), fontsize=15)
 ax2.imshow(D-rotated, cmap='gray')
 ax2.set_title('Difference Map'.format(rt), fontsize=15)
 ax3.hist(res[res!=0],100,density=True)
 ax3.set_title('Difference Density', fontsize=15)
 plt.tight_layout()
 plt.show()

While this approach of generating synthetic motion into MRI images is a poor model of how motion actually occurs during an MRI scan, there are a few things I learned here. For example, if you define a measure of image similarity, like mutual information, entropy, or correlation ratio as a cost function, we can see how we can use warpAffine to find the optimal transformation matrix between two images.

I was hoping to use openCV to generate and apply 3d affine transformations to volumetric MRI data. One approach to doing this is to iteratively apply rotations and transformations along each axis – however, openCV will interpolate the data after each transformation, resulting in a greater loss of signal than I am willing to compromise on. It doesn’t seem like openCV has ability to apply 3d affine transformations to volumetric data in a single interpolation step.

A more realistic approach to generating synthetic motion artifacts that would more accurately parallell the noise-generating process, is to compute the Fast Fourier Transform of my 3d volume, and then apply phase-shifts to the k-space signal – this will also manifest as motion after applying the inverse FFT.

After doing a bit more digging through the openCV API, it seems there’s a lot of cool material for exploration – these applications specifically caught my eye and would be fun to include in projects:

video analysis for motion tracking
object recognition for detecting faces
openCV Android for app development

But alas – the search continues!

Two Weeks of Open Science: A Rekindled Flame

Fri, 17 Aug 2018 01:12:33 -0700

I recently attended Neurohackademy 2018, hosted by the University of Washington’s eScience Institute, and organized by Dr. Ariel Rokem and Dr. Tal Yarkoni.

This was a 2-week long event, beginning with a series of daily lectures, and ending with a fast-paced, high-intensity scramble to put together a beta (but working) version of some project coupling neuroimaging with software development. The lectures varied in topic, from how to test academic code and how to organize open source projects, to machine learning and algorithms for low-dimensional representations of neural recordings, to neuroethics (full lecture list here).

Many of the lecturers are scientists and developers who I’ve looked up to for years – a few have even been my intellectual, and now post-Neurohackademy, philosophical role models. While the lectures were enlightening in their own right, there was a level of intimacy during these 2 weeks that’s been unmatched during grad school so far. Rarely do young researchers like myself get to pick the brains of and engage in scientific banter with scientists whose papers they read, or whose updates they follow, in such a fluid and collaborative setting. I feel a little weird being so enthusiastic about it (specifically because I know some of them might read this), but (and I think I speak for all of us who participated) it was a richly rewarding experience.

The #NH18 participants varied in status from graduate students, to post-docs, to industry members, and traveled from all around the world to Seattle, but each one of us was, in one way or another, involved with neuroscience research. As one of my new friends put it on Twitter: “Spent yesterday in a room full of relative strangers who were collaborating, mentoring, & supporting. Devoid of egos or tribalism. Feels like what science (& society) should be.” The sense of community was strong, positivity was plentiful, and people supported one another – without regard for experience or any sense of return-on-investment. We had an established Git person, Python person, data viz person, fMRI person, C++ person, etc. – if you had a question or ran into an issue, with high probability there was someone who could and would help you out. Everyone was genuinely excited to learn, to share, to create, to bond, and especially, ~~to neuro/computer/data-science~~.

— to science —

(origin: probably Newton)
verb. To perform scientific research, almost always in a smooth or cool way.

I worked on a project directly related to my research, but that I’d only previously written some messy, non-shareable scripts for. Conveniently, my co-NeuroHacker Michael Notter had a similar idea and we hit the ground running with some of our colleagues. The project, titled parcellation_fragmenter, provides a means for fragmenting the brain cortex into a predefined number of regions, or a set of regions each of the same size. These regions can be anatomically constrained, or arbitrarily spread across the cortex. Our goal was to use this tool to speed up statistical tests, like SearchLight FDR, or as a feature extraction method for down-stream machine learning applications. I’m currently using this tool to examine how cortical network resolution impacts pairwise regional network properties.

Here is something pretty! It was created with the new parcellation fragmenter (https://t.co/9VLCpr336Y), developed by Kristian Eschenburg, @kako_toro, Amanda Sidwell & me during the #NHW18. Thank's to #nilearn & #nibabel creating this toolbox was straightforward and a lot of fun! pic.twitter.com/LdqTCMSyrJ
— Michael Notter (@miyka_el) August 10, 2018

My most important takeaway from our project was learning how to collaboratively write and develop software with a group of people. Not only did each of our team members have unique ideas about how to approach our specific problem, we also each had different ways of thinking about how to write software in general. Clear communication, open-mindedness, and understanding on all of our parts were integral to seeing this development through. Overall, the project was a success!

Here are a few things I learned, that I’m going to incorporate into my own work (and hopefully convince people in my lab to do the same):

Unit-test my code using pytest and nose
Incorporate continuous integration (I’ve already made use of TravisCI!)
Learn web-dev, and specifically, JavaScript (to use D3, and develop interactive posters and publications)
Contribute to issues / create pull-requests on GitHub repos that I use or find interesting
Pre-register my papers and submit to open-source journals

This event was what I’d hoped graduate school would be like all along. While idealistic and naive to some degree, I still think it can be. The open-source model is shifting how research is performed – the act of doing research is evolving in such a way that it is no longer tethered to specific institutions or labs, and given tools like Docker and AWS, you can almost perfectly recreate specific computing environments needed to perform the work. With the rise of open-source datasets, especially due to researchers willingly distributing their data and code, collaborative environments like that fostered by Neurohackademy (even if digital), and the ability to replicate workflows, results, and analyses, are becoming more and more feasible. It only takes a few proponents of the open-source model to give the idea momentum.

If this is the future, the future is looking good.

Exploring Neurological Dynamical Systems: Part 2

Thu, 24 May 2018 11:30:43 -0700

In my previous post on dynamic mode decomposition, I discussed the foundations of DMD as a means for linearizing a dynamical system¹²³. In this post, I want to look at a way in which we can use rank-updates to incorporate new information into the spectral decomposition of our linear operator, $A$, in the event that we are generating online measurements from our dynamical system⁴ – see the citation below if you want a more-detailed overview of this topic along with open source code for testing this method.

Recall that we are given an initial data matrix

$$\begin{align} X = \begin{bmatrix} x_{n_{1},m_{1}} & x_{n_{1},m_{2}} & x_{n_{1},m_{3}} & … \\ x_{n_{2},m_{1}} & x_{n_{1},m_{2}} & x_{n_{2},m_{3}} & … \\ x_{n_{3},m_{1}} & x_{n_{1},m_{2}} & x_{n_{3},m_{3}} & … \\ … & … & … & … \\ \end{bmatrix} \in R^{n \times m} \end{align}$$

which we can split into two matrices, shifted one unit in time apart:

$$\begin{align} X^{\ast} &= \begin{bmatrix} \vert & \vert & \dots & \vert \\ \vec{x}_1 & \vec{x}_2 & \dots & \vec{x}_{m-1} \\ \vert & \vert & \dots & \vert \\ \end{bmatrix} \in R^{n \times (m-1)} \\ Y &= \begin{bmatrix} \vert & \vert & \dots & \vert \\ \vec{x}_2 & \vec{x}_3 & \dots & \vec{x}_{m} \\ \vert & \vert & \dots & \vert \\ \end{bmatrix} \in R^{n \times (m-1)} \end{align}$$

and we are interested in solving for the linear operator $A$, such that

$$\begin{align} Y = AX^{\ast} \end{align}$$

For simplicity, since we are no longer using the full matrix, I’ll just refer to $X^{\ast}$ as $X$. In the previous post, we made the constraint that $n > m$, and that rank($X$) $\leq m < n$. Here, however, we’ll reverse this assumption, such that $m > n$, and that rank($X$) $\leq m < n$, such that $XX^{T}$ is invertible, so by multiplying both sides by $X^{T}$ we have

$$\begin{align} AXX^{T} &= YX^{T} \\ A &= YX^{T}(XX^{T})^{-1} \\ A &= QP_{x} \end{align}$$

where $Q = YX^{T}$ and $P_{x} = (XX^{T})^{-1}$. Now, let’s say you observe some new data $x_{m+1}, y_{m+1}$, and you want to incorporate this new data into your $A$ matrix. As in the previous post on rank-one updates, we saw that directly computing the inverse could potentially be costly, so we want to refrain from doing that if possible. Instead, we’ll use the Sherman-Morrison-Woodbury theorem again to incorporate our new $x_{m+1}$ sample into our inverse matrix, just as before:

$$\begin{align} (X_{m+1}X^{T}{m+1})^{-1} = P{x} + \frac{P_{x}x_{m+1}x_{m+1}^{T}P_{x}}{1 + x_{m+1}^{T}P_{x}x_{m+1}} \end{align}$$

Likewise, since we’re appending new data to our $Y$ and $X$ matrices, we also have

$$\begin{align} Y_{m+1} = \begin{bmatrix} Y & y_{m+1} \end{bmatrix} \\ X_{m+1} = \begin{bmatrix} \\ X & x_{m+1} \end{bmatrix} \\ \end{align}$$

such that

$$\begin{align} Y_{m+1} X_{m+1}^{T} &= YX^{T} + y_{m+1}x_{m+1}^{T} \\ &= Q + y_{m+1}x_{m+1}^{T} \end{align}$$

which is simply the sum of our original matrix $Q$, plus a rank-one matrix. The authors go on to describe some pretty cool “local” DMD schemes, by incorporating weights, as well as binary thresholds, that are time-dependent into the computation of the linear operator, $A$.

P.J. Schmid. Dynamic mode decomposition of numerical and experimental data. Journal of Fluid Mechanics 656.1. 2010. ↩︎
Tu et al. On Dynamic Mode Decomposition: Theory And Applications ↩︎
Kunert-Graf et al. Extracting Reproducible Time-Resolved Resting State Networks Using Dynamic Mode Decomposition. Front. Comput. Neurosci. 2019. ↩︎
Zhang et al. Online dynamic mode decomposition for time-varying systems. 2017. ↩︎

Exploring Neurological Dynamical Systems: Part 1

Tue, 22 May 2018 14:02:52 -0700

In the next two posts, I want to talk briefly about an algorithm called Dynamic Mode Decomposition (DMD). DMD is a spatiotemporal modal decomposition technique that can be used to identify spatial patterns in a signal (modes), along with the time course of these spatial patterns (dynamics). As such, the algorithm assumes that the input data has both a spatial and a temporal component. We are interested in modeling how the system evolves over time.

If you’d like to find more information about DMD, Peter Schmid¹ and Jonathan Tu² have written excellent expositions on the topic. Likewise, if you’d like to follow along with the code for the following analysis, see my repo. For a more in-depth analysis that applies DMD to brain activity in the resting brain, see a recent publication by my colleagues and me³, along with the code used for our analysis.

The DMD Algorithm

Let’s assume that you’ve taken $n$ measurements from specific points in space for $m$ time points, where for now we assume that $m\lt n$. For now, we’ll assume that the sampling frequency, $\omega$, is stable across the entire experiment. We define our entire data matrix as

We are interested in solving for the matrix, $A \in R^{n \times n}$, such that

$$\begin{align} x_{t+1} = A x_{t} \; \; \forall \; \; t = 1,2,…m-1 \end{align}$$

Given our full data matrix $X$, we can define two matrices $X^{*}$ and $Y$ such that

$$\begin{align} X^{*} &= \begin{bmatrix} \vert & \vert & \dots & \vert \\ \vec{x}_{1} & \vec{x}_{2} & \dots & \vec{x}_{m-1} \\ \vert & \vert & \dots & \vert \\ \end{bmatrix} \in R^{n \times (m-1)} \\ Y &= \begin{bmatrix} \vert & \vert & \dots & \vert \\ \vec{x}_2 & \vec{x}_3 & \dots & \vec{x}_{m} \\ \vert & \vert & \dots & \vert \\ \end{bmatrix} \in R^{n \times (m-1)} \end{align}$$

so that we can write

$$\begin{align} Y = AX^{\ast} \end{align}$$

If $n$ is small, this is relatively easy to compute – however, if $n$ is large, as is the case when modeling temporal dynamics in resting-state MRI, it would be computationally inefficient to compute A directly. To alleviate this, we can make use of the Singular Value Decomposition (SVD) of our predictor matrix $X^{\ast}$. We define the SVD of $X^{\ast}$ as

$$\begin{align} X^{\ast} = U \Sigma V^{T} \ \end{align}$$

as well as the Moore-Penrose pseudo-inverse of $X^{\ast} = X^{\dagger}$ as

$$\begin{align} X^{\dagger} = V \Sigma^{-1} U^{T} \\ \end{align}$$

such that we can write

$$\begin{align} YX^{\dagger} = YV \Sigma^{-1} U^{T} = A X^{\ast}X^{\dagger} = A \\ \end{align}$$

Additionally, if we assume that $rank(X^{\ast}) = r \leq m$, then we can use the truncated SVD such that

$$\begin{align} U & \in R^{n \times r} \\ V^{T} & \in R^{r \times m} \\ \end{align}$$

and

$$\begin{align} \Sigma = \begin{bmatrix} \sigma_{1} & 0 & 0 & … \\ 0 & \sigma_{2} & 0 & … \\ 0 & 0 & \ddots & … \\ \vdots & \vdots & \vdots & \sigma_{r} \\ \end{bmatrix} \in R^{r \times r} \end{align}$$

As it stands now, we still compute an $A \in R^{n \times n}$ matrix. However, because we have a potentially low-rank system, we can apply a Similarity Transformation to $A$ in order to reduce its dimensionality, without changing its spectrum. Using our spatial singular vectors $U$, we define

$$\begin{align} \tilde{A} &= U^{T} A U \\ &= U^{T} (YV \Sigma^{-1} U^{T}) U \\ &= U^{T} Y V \Sigma^{-1} \\ \end{align}$$

where $\tilde{A} \in R^{r \times r}$. If we consider the above SVD, we see that $U$ is the matrix of left singular vectors, an orthogonal basis that spans $C(X^{\ast})$, which is an r-dimensional subspace of $R^{n}$. Thus, the similarity transform represents a mapping $f(A) = U^{T} A U : R^{n} \rightarrow R^{r}$. We now have a reduced-dimensional representation of our linear operator, from which we can compute the spatial modes and dynamic behavior of each mode. First, however, because of the notion of variance captured by the singular values of our original predictor matrix, we weight $\tilde{A}$ by the singular values as

$$\begin{align} \hat{A} = \Sigma^{\frac{1}{2}} \tilde{A} \Sigma^{\frac{1}{2}} \\ \end{align}$$

such that our computed spatial modes have been weighted by the amount they contribute to our measured signal. We can now compute the eigendecomposition of $\hat{A}$ as

$$\begin{align} \hat{A} W = W \Lambda \\ \end{align}$$

where the eigenvectors $W$ are the reduced-dimension representations of our spatial modes, and the eigenvalues $\Lambda$ capture the dynamic behavior of our spatial modes. Because our original data matrix $X^{\ast}$ had spatial dimension $n$ and our eigenvectors have dimension $r$, we need to up-project our eigenvectors $W$ to compute the final spatial modes, via

$$\begin{align} \Phi = Y V \Sigma^{\frac{-1}{2}}W \end{align}$$

From the SVD of our prediction matrix $X^\ast=U \Sigma V^{T}$, the matrix $V \in R^{m \times r}$ is the matrix of right singular vectors, an orthogonal basis spanning the space of $X^{\ast T}$ (i.e. $r$ basis vectors spanning the space of the measured time courses). Thus, we see that $H = (V \Sigma^{\frac{-1}{2}})W$ represents a linear combination of the temporal basis vectors (a mapping from $R^{r} \rightarrow R^{m}$) for each eigenvector $w_{i}$ of $W$, weighted by the corresponding singular value $\sigma_{i}^{\frac{-1}{2}}$ (that acts to normalize the spatial mode amplitudes). Finally, we see that $\Phi = X^{\ast}H$ computes how much of each temporal basis vector is present in the measured time course at each point in space.

Because we are modeling a dynamical system, we can compute the continuous time dynamics of our system using our spatial modes and eigenvalues as

$$\begin{align} \vec{x}(t) \approx \sum_{i=1}^{r} b_{i}\exp^{((\gamma_{i} + 2i\pi f_{i})\cdot t)} \vec{\phi}_{i} \end{align}$$

where $\gamma_{i}$ is a growth-decay constant and $f_{i}$ is the frequency of oscillation of the spatial mode $\phi_{i}$. We can compute these two constants as

$$\begin{align} \gamma_{i} &= \frac{\text{real}(\text{ln}(\lambda_{i}))}{\Delta t} \\ f_{i} &= \frac{\text{imag}(\text{ln}(\lambda_{i}))}{2\pi \Delta t} \end{align}$$

So, we can see that DMD linearizes our measured time series, by fitting what can be analogized to a “global” regression. That is, instead of computing how a single time point predicts the next time point, which could readily be solved using the simple Normal equations, DMD computes how a matrix of time points predicts another matrix of time points that is shifted one unit of time into the future. To this extent, DMD minimizes the Frobenius norm of

$$\begin{align} \min \limits_{A} \lVert Y - AX^{\ast} \rVert^{2}_{F} \\ \end{align}$$

However, rather than explicitly computing the matrix $A$, DMD computes the eigenvectors and eigenvalues of $A$, by utilizing the Singular Value Decomposition, along with a Similarity Transformation, in order to generate a reduced-dimensional representation of $A$.

This spectral decomposition of our linear operator is of particular importance, because it sheds light on the fact that DMD models the temporal dynamics of our system using a Fourier basis. Each spatial mode is represented by a particular Fourier frequency along with a growth-decay constant that determines the future behavior of our spatial mode. Additionally, the Fourier basis also determines what sorts of time series can be modeled using DMD – time series that are expected to have sinusoidal behavior will be more reliably modeled using DMD, whereas signals that show abrupt spike patterns might be more difficult to model.

P.J. Schmid. Dynamic mode decomposition of numerical and experimental data. Journal of Fluid Mechanics 656.1. 2010. ↩︎
Tu et al. On Dynamic Mode Decomposition: Theory And Applications ↩︎
Kunert-Graf et al. Extracting Reproducible Time-Resolved Resting State Networks Using Dynamic Mode Decomposition. Front. Comput. Neurosci. 2019. ↩︎