Abstract
Large-scale in vivo neuroimaging datasets offer new possibilities for reliable, well-powered measures of interregional structural differences and biomarkers of pathological changes in a wide variety of neurological and psychiatric diseases. However, it has been impossible to determine the cortical layer or neurobiological processes causing these changes. We developed artificial neural networks to segment cortical and laminar surfaces in the BigBrain, a 3D histological model of the human brain, to test whether gradients of MRI thickness in sensory and motor processing cortices were present in a histological atlas of cortical thickness, and which cortical layers were contributing to these gradients. Identifying common gradients of cortical organisation enables the formation of mechanistic links between microstructural, macrostructural and functional cortical parameters. In our fully segmented layered model of the cerebral isocortex, we found that histological thickness corroborated MRI thickness gradients in sensory cortices but was the inverse from those reported in fronto-motor cortices, with layers III, V and VI being the primary drivers of these thickness gradients. Our laminar atlas creates a link between single-neuron morphological changes, mesoscale cortical layers and macroscale cortical thickness, and our findings suggest that across multiple measurement domains, the primary motor cortex should not be indiscriminately grouped with primary sensory areas.
Introduction
The cerebral isocortex has six layers that vary depending on cortical area and local morphology. Previous work has described relationships between cytoarchitecture, cortical morphology and functional organisation (Broca, 1861; Fischl et al., 2008; Wagstyl et al., 2015), nevertheless the precise nature of this relationship is unclear. The goal of the current study was to determine whether gradients of increasing cortical thickness in sensory cortices, first described using in vivo MRI, were present under histological measurement and, if so which cortical layers contribute to these gradients. Furthermore we aimed to test whether such cortical and laminar thickness gradients were present in the fronto-motor cortices and whether they followed the same primary-to-association progression measured in sensory areas. If they diverge, this raises questions for the placement of motor areas in gradient and hierarchical models of cortical organisation(Huntenburg et al., 2017b).
Sensory processing hierarchies describe the concept that the cerebral cortex is organised with gradually changing structural and functional properties from primary sensory areas, to secondary sensory areas and ultimately higher-order association areas. Multiple measurement modalities converge on similarly ordered patterns including single-neuronal morphological(Elston and Rosa, 1998) and electrophysiological characteristics(Elston and Fujita, 2014), laminar connectivity patterns of projecting cortical neurons (Felleman and Van Essen, 1991; Goulas et al., 2018), laminar differentiation(Barbas, 1986), MRI cortical thickness(Wagstyl et al., 2015), MRI myelination(Huntenburg et al., 2017a), receptor densities (Palomero-Gallagher and Zilles, 2017) and temporal dynamics(Murray et al., 2014). Topographically, hierarchies are organised such that progressively higher cortical areas are located with increasing geodesic distance across the cortical surface from their primary areas (Margulies et al., 2016; Wagstyl et al., 2015). Ordering cortical areas along these gradients provides a framework for understanding the relationships between interareal structural and functional characteristics.
Cortical thickness is a widely used marker of in vivo and ex vivo cortical structure(Wagstyl and Lerch, 2018). Early histological studies noted marked interareal thickness differences on post mortem histological sections(Brodmann, 1909; von Economo and Koskinas, 1925), which have since been replicated (Fischl and Dale, 2000; Lerch and Evans, 2005) and extended using in vivo MRI (Wagstyl et al., 2015) and alterations in this pattern may be seen in neuropsychiatric illness (Wagstyl et al., 2016). MRI cortical thickness demonstrated patterns of cortical thickness relating to functional and structural hierarchical organisation across sensory cortices of both macaque and humans (Wagstyl et al., 2015). A similar gradient of increasing MRI cortical thickness was reported from primary motor cortex to anterior frontal cortex (Thiebaut de Schotten et al., 2016). However, while classical studies of cortical histology observed that primary sensory regions were thinner than their surrounding secondary sensory cortices the inverse is true for the motor cortex which is especially thick (Brodmann, 1909; von Economo and Koskinas, 1925). Furthermore thickness gradients identified in MRI extended far beyond secondary areas into association cortical areas. Thus it remains unclear whether thickness gradients found in MRI are artefactual, driven by gradient differences in cortical myelination causing systematic cortical reconstruction errors (Glasser and Van Essen, 2011; Huntenburg et al., 2017a), or truly represent the underlying histology. Moreover the isocortex has six histologically defined layers, and it is unclear to what extent changes in cortical thickness are driven by different layers.
However carrying out analyses of histological thickness gradients poses several methodological challenges. First, most histology is carried out in 2D, with associated measurement artefacts due to oblique slicing (Wagstyl et al., 2018a). Second manual measurement is associated with observer dependent variability, estimated to be up to 0.5 mm (von Economo and Koskinas, 1925). Third many histological atlases have a small number of sample points. These factors hinder the ability to detect potentially subtle, cross-cortical changes in overall and laminar thicknesses. BigBrain offers a unique dataset to resolve histological cortical layers in 3D, thereby providing a concrete link between microscale patterns of structure and in vivo markers.
We therefore set out to automate segmentation of cortical layers in 3D in order to explore patterns of cortical and laminar thickness across sensory and fronto-motor cortical hierarchies. To do this we used a convolutional neural network to segment profiles of histological intensity sampled between the pial and white matter. Training profiles were generated from examples of manually segmented layers on cortical regions from 2D histological sections of the BigBrain. The trained network was used to segment intensity profiles derived obliquely through the 3D histological volume and generate mesh segmentations of six cortical layers. These surfaces were used to calculate cortical and laminar thicknesses. Geodesic surface distance from primary visual, auditory, somatosensory and motor areas were calculated and used as a marker of hierarchical progression. Cortical and laminar thickness gradients were calculated for each system.
Methods
Volumetric data preparation
BigBrain is a 20×20×20μm (henceforth described as 20μm) resolution volumetric reconstruction of a histologically processed post mortem human brain (male, aged 65), (available for download at https://bigbrain.loris.ca). In order to run computations on this amount of data, the BigBrain was stored into 125 individual blocks, corresponding to five subdivisions in the x, y and z directions, with overlap. The overlap of blocks was calculated to be sufficient such that each single cortical column could be located in a single block, enabling extraction of complete intensity profiles between pairs of vertices at the edge of blocks without intensity values being altered by boundary effects when the data were smoothed. Blocks were smoothed anisotropically, predominantly along the direction tangential to the cortical surface, to maximise interlaminar intensity differences while minimising the effects of intralaminar intensity variations caused by artefacts, blood vessels and individual neuronal arrangement(Kimia and Siddiqi, 1996; Wagstyl et al., 2018a). The degree of anisotropic smoothing is determined by repeatedly applying the diffusive smoothing algorithm. The optimal level of smoothing was previously determined and gave an effective maximum FWHM of 0.163mm. For subsequent analyses both the raw 20μm and anisotropically smoothed blocks were used.
Lower resolution volumes were extracted by subsampling raw BigBrain 20μm volume at 40, 100, 200, 400 and 1000μm. Anisotropically smoothed volumes were also generated at each of these resolutions.
Profile extraction
Pial and white surfaces originally extracted using a tissue classification of 200μm were taken as starting surfaces(Lewis et al., 2014). Prior to intensity profile extraction, in order to minimise cortical streamlines intersecting, to improve their approximation of columnar trajectories and to ensure profiles captured the full extent of the cortex the following steps were taken:
1) A mid-surface was generated that was closer to the pial surface in sulci and closer to the white surface in gyri, weighting the distance vector by the cortical curvature. Thus the mid surface was closer to the surface with the higher curvature.
2) This mid-surface was upsampled to 655362 vertices to increase the resolution of the surface.
3) For each vertex, the vector between nearest pial and white coordinates was calculated.
4) To minimise self-intersections, the vector components were smoothed across the mid-surface with a FWHM of 3mm (Supplementary figure 1).
5) Profiles were calculated along these vectors from the midsurface, extending the profiles 0.5mm further than the minimum distance in the pial or white direction, to ensure the resultant intensity profile captured the full extent of the cortex.
The resultant profiles were less oblique and more likely line up with the cortical columns (Supplementary figure 1).
Extended intensity profiles were then created for each pair of vertices by sampling voxels from the raw and anisotropically smoothed BigBrain volumes, at each available resolution.
Training data
Manual segmentations of 6 cortical layers were created on 51 regions of the cortex, distributed across 13 of the original histological BigBrain sections at 5μm (Figure 2). Segmentations were inspected by histological experts. This resolution is sufficient to distinguish cell body distribution patterns used to delineate specific layers. Averaged across all training examples, layer classes contributed to profiles as follows: supra-pial: 14.6%, layer I: 7.5%, layer II: 5.6%, layer III: 20.8%, layer IV: 5.5%, layer V: 14.8%, layer VI: 17.8%, white matter: 13.4%. These represent an approximate relative thickness of each layer.
Manual segmentations were then co-registered to the full aligned BigBrain space. The manually drawn layers were used to create corresponding pial and white surfaces and these cortical limits were extended into the white matter and beyond the pial surface between 0.25mm and 0.75mm so as to match the variability of cortical extent in the test profile dataset. Training profiles were created sampling raw, smoothed and labelled data, generating thousands of profiles per sample. Each pixel in the labelled data had a class value of 0-7, where pixels superficial to the pial surface were set to 0, followed by layers numbered 1-6, and white matter was classed as 7. This 1D profile based approach greatly expanded the training dataset from 51 labelled 2D samples to 564,041 profiles.
Neural network
A 1D convolutional network for image segmentation was created with the following structure. The network was created using stacked identical blocks. Each block contained a batch normalisation layer to normalise feature distributions between training batches, a rectify non-linearity layer used to model neurons which can have a graded positive activation but no negative response(Nair and Hinton, 2010), and a convolutional layer(Lecun et al., 1998). There was a final convolutional layer with filter size 1 and 8 feature maps, one for each tissue class. The cost function was median class-frequency weighted cross-entropy. Class-frequency weighting was added to weigh errors according to the thickness of the layers so that incorrectly classified pixels in thinner layers were more heavily weighted than errors in incorrectly classified thicker layers(Eigen and Fergus, 2014). Raw and smoothed profiles were considered as two input channels. The network was iteratively trained until the accuracy did not improve for 50 epochs (all training profiles are seen once per epoch). At this point the previous best model was saved and used for subsequent testing on the full dataset. When testing the network, a soft maximum was then applied to detect the most likely layer class for each pixel. The output was a vector of predicted layers for each of the 200 pixels, for each vertex on the cortical mesh. For each vertex, a measure of confidence was calculated from these predictions. Per pixel confidence is the difference between the prediction value for the highest predicted class and the value of the 2nd highest predicted class. Per class/layer confidence is the mean confidence for all pixels in that class/layer. The per-vertex summary measure is the mean across all pixels in the profile. These measures give an indication of the relative confidence regional and laminar classifications.
Hyperparameter optimisation and cross-validation
There is no consensus method for finding optimum parameters for a neural network. Here a set of 50 experiments with random hyperparameters was carried out to explore their impact on training accuracy. Learning rate, convolutional filter size, number of layers (blocks), weight decay and batch size were all varied. In summary, the final network was initialised with 6 layers, filter size = 49, learning rate = 0.0005, weight decay = 0.001, where the learning rate determines the amount weights are updated on each iteration and weight decay determines the rate at which weights decrease each iteration which helps prevent overfitting.
For network cross validation, the manually labelled areas were subdivided into 10 equally sized random subsets or folds. Initially 2 folds were removed from the dataset during training and network weights were optimised for segmenting samples on one of these folds. This trained network was then used to predict layers on the final previously unseen test fold with which the accuracy was calculated. This process was repeated 10 times to generate an estimate of the network’s ability to segment novel cortical regions. The same process was carried out using profiles extracted at all available resolutions.
For generating BigBrain layer segmentations, the network was trained on the full training dataset and tested on all intensity profiles.
Shrinkage estimate
Shrinkage due to fixation was estimated using two methods. The first was calculated based on the estimated fresh volume of BigBrain, based on the original fresh weight, and the volume after histological processing. This gave a volume-based (3) shrinkage factor of 1.931, which corresponds to a length-based (1D) shrinkage factor of 1.245
The second estimate was given by coregistering the BigBrain volume to a MRI volumetric template based on a group of older subjects (ADNI template). The transformation matrix gave linear scale factors of 1.122, 1.212 and 1.439 in the x,y and z directions, with a mean of 1.257. The convergence of these measures of shrinkage suggests subsequent thickness and length estimates can be adequately corrected for comparison to in vivo measures by scaling up values by 25%.
Surface reconstruction: post-processing 1D profiles
1D classified profiles were transformed into mesh layer boundary reconstructions as follows. Transitions between predicted layers were located for each profile and the coordinates of these transitions became vertex locations for the new layer meshes. For the small number of vertices where the network failed (less than 1%), vertex locations were interpolated from the neighbouring vertices. Surface indices were smoothed 0.5mm FWHM across the cortical surface and 20 iterations of geometric mesh smoothing was applied to the output surface (Taubin, 1995). This removed non-biologically high frequency changes in surface curvature, most commonly due to minor, local mis-registrations of consecutive 2D coronal sections.
Cortical thickness, layer thickness
Cortical thickness was calculated between pial and white cortical surfaces and laminar thicknesses were calculated between adjacent pairs of cortical surfaces.
Masking
Manual masks were created to remove medial wall, archicortex, including areas of the cingulate and entorhinal cortex which do not have 6 layers, and large cuts in the anterior temporal cortex from subsequent analyses.
Gradients
Surface labels for the primary visual, somatosensory, auditory and motor areas were manually delineated on each hemisphere using morphological markers and histological characteristics. Geodesic distance across the cortex from each of these labels was calculated to each of the vertices within a larger masked area containing associated cortical regions (Funck et al., 2017; Glasser et al., 2016; Wagstyl et al., 2015).
Results
Neural network training
In the cross-validation, average per-pixel accuracy on the test fold was 83% +/-2% prior to post processing suggesting the network was able to learn generalisable layer-specific features and transfer these to novel cortical areas. The predictions of the model trained on the full dataset were used to create a 3D segmentation of the cortical layers in both hemispheres (Figure 1).
Confidence results
Layer confidence maps, given by the difference between the first and second predicted classes for each pixel, give an approximation of the reliability of laminar segmentations for the cortex where ground truth manual segmentations have not been carried out (Supplementary figure 2). Throughout the cortex, the network has high confidence for supra-pial and white matter classes. Layers I, II, III, V and VI also exhibit relatively consistent confidence maps, whereas layer IV demonstrates a high variability of confidence, which is high in the occipital cortex but low anteriorly. This pattern matches with visual observations that layer IV is relatively prominent in the occipital cortex, but more difficult to identify in frontal lobe areas, and absent in primary motor cortex.
Resolution results
Downsampling BigBrain to decrease the resolution from 20 microns down to 1000 microns, progressively decreased the accuracy of the network on the test folds from 0.85 to 0.4 (Supplementary figure 3). However at 100um, the approximate upper limit for current high-resolution structural MRI, profiles had sufficient detail to maintain an accuracy of 0.76.
Visual inspection of 3D layers of the cortex
On visual inspection, the automatically identified cortical layers closely follow bands of intensity within the BigBrain (Figure 1) and continue to follow the same features beyond the limits of training examples (Figure 2).
In the original surfaces, the white surface was placed at the maximum intensity gradient between gray matter and white matter, while the new white surface is based on the network trained on manual placement of the white boundary, which is defined by the presence of cortical neurons. On closer inspection, the maximum gradient appears to be at the border between sublayers VIa and VIb where the change in neuronal density is sharper than at the boundary between white matter and layer VI (Supplementary figure 4).
A second feature apparent on visual inspection is that the layers do not follow a single set of rules - they vary between cortical areas. This is most readily apparent at the V1-V2 boundary where layer IV is dramatically different (Figure 2). Layer IV is particularly thick in V1 and has multiple sublayers creating extra peaks and troughs in the intensity profiles whereas in V2 it is much thinner and no longer differentiated into sublayers. The transition from a thick layer IV to a thin layer IV occurs precisely at the boundary between these two regions suggesting the network is internally learning certain areal properties also.
Visual comparison of total and layer thickness maps
On visual inspection maps of BigBrain cortical thickness are consistent with classical atlases of histological thickness reported by von Economo and Koskinas (Figure 3). In particular the motor cortex is the thickest part of the cortex with values over 3.8mm uncorrected (over 4.5mm when adjusted for shrinkage). The thickness of the motor cortex is often underestimated in MRI thickness measurement(Zilles and Amunts, 2015), probably due to the high degree of intracortical myelination which affects the grey-white contrast, causing the white matter surface to be too close to the grey surface, such that cortical thickness is underestimated (Glasser and Van Essen, 2011; Wagstyl et al., 2015). Clearly visible in both BigBrain and von Economo is the thin occipital cortex.
On comparison with the available layer maps for layers I, III, V and VI from the von Economo atlas, there are several similarities but also meaningful differences (Figure 4). Layer III is thick in the precentral areas, and particularly thin in the primary visual cortex. Layers V and VI are thick in frontal and cingulate cortices, but also thin in the occipital cortex. Of interest is the clear boundary exhibited by layer IV at the boundary between V1 and V2 in the occipital cortex. This change in thickness is abrupt enough to generate an automated anatomical label for V1 (Figures 2 and 4).
Cortical gradients
Cortical thickness was positively correlated with geodesic distance in visual (right, R=0.38, p=0, left, R=0.38, p=0) and somatosensory (right, R=0.25, p=0, left, R=0.25, p=0) cortices, but only very weakly so in the auditory cortex (right, R=0.08, p=0, left, R=0.03, p=0) (Figure 5A-C). By contrast in the motor cortex, thickness was negatively correlated with geodesic distance (right, R=-0.21, p=0, left, R=-0.36, p=0) (Figure 5D). This is consistent with MRI thickness findings in sensory gradients but contradictory for the fronto-motor gradient.
Cortical layers did not contribute equally to the total thickness gradient in the visual cortex and somatosensory (Figure 6A). Layers III and V had the largest contributions to the total thickness gradient, followed by layer VI and then II. A similar but less pronounced pattern was seen within the auditory cortex.
In the motor cortex the inverse was true, with decreases in layers III, V and VI. Layer IV also exhibited a decrease in thickness, however there was low network confidence for layer IV in these areas and thus this particular finding is inconclusive.
Thus, consistent with MRI thickness gradients, somatosensory and visual areas exhibited positive histological thickness gradients primarily driven by layers III, V and VI. By contrast the fronto-motor areas exhibited an inverse gradient, peaking in the motor cortex and driven by the same layers (Figure 6B).
Open data availability
All code used for the segmentation and analysis of cortical layers will be made available at https://github.com/kwagstyl and BigBrain volumes, surfaces and intensity profiles are available to download at https://bigbrain.loris.ca/.
Discussion
We automatically segmented the six histological layers of the cerebral cortex in the BigBrain and used these to test for gradients of cortical and laminar thickness within sensory and motor processing hierarchies. Consistent with previous findings using in vivo MRI, sensory hierarchies exhibited an increasing cortical thickness gradient from primary sensory to higher order areas. In visual, somatosensory and auditory cortices, these gradients were primarily driven by layers III, V, VI. By contrast, the fronto-motor cortices exhibited a decreasing cortical thickness gradient away from the primary motor cortex towards higher order frontal areas, which was driven by decreases in these layers. These findings highlight the utility of the BigBrain for linking micro- and macro-scale patterns of cortical organisation.
Gradients of thickness are large scale markers of systematic changes in the cortical microcircuit. The volume of the cortex is 80-90% neuropil (Braitenberg and Priv Doz Dr, 1991; Chklovskii et al., 2002), of which 60% is axons and dendrites and the remainder is synaptic boutons, spines and glia. As neuronal density decreases with increasing cortical thickness (Collins et al., 2010; la Fougère et al., 2011), and most of the volume of the cortex is neuropil, increased thickness is most likely to mark increased intracortical connectivity (Wagstyl and Lerch, 2018). At a laminar level, the strongest contributors to the overall thickness gradients were layers III, V and VI (Figure 6). Cell-morphological studies in macaques have shown that the cell size and dendritic arborisation of layer III and V pyramidal neurons increases across the visual pathway (Elston and Rosa, 2000, 1998; Scholtens et al., 2014). Similarly afferent axonal patch sizes scale with pyramidal neuronal arborisation (Lund et al., 1993). Increasing dendritic arborisation, axonal field size and numbers of synapses all contribute to an increase in the volume of laminar neuropil and are therefore a likely driver of the laminar and overall thickness gradients measured here. Layer VI also increased in thickness in sensory pathways. However, while neurons located in these layers might exhibit increases their associated neuropil, the measured thickness change may in part be due to the extended arborisations of III/V pyramidal neurons forming connections within these layers. Histological gradients of layer thickness provide us with a meso-scale link between in vivo patterns of MRI cortical thickness and microstructural, neuronal-scale changes in the cortical microcircuit.
In contrast to in vivo studies of fronto-motor functional, myelin and MRI cortical thickness organisation, which place the primary motor cortex at the same level as primary sensory areas(Huntenburg et al., 2017a; Margulies et al., 2016; Thiebaut de Schotten et al., 2016), we found that total and laminar fronto-motor thickness gradients were the inverse of those measured in sensory cortices. Histologically the motor cortex was especially thick and the thickness decreased with geodesic distance from the primary motor cortex; with layers III, V and VI following a similar inverse pattern. This finding is consistent with reported trends in other histological properties such as laminar structural type (von Economo and Koskinas, 1925) and neuronal density (Collins et al., 2010), as well as the observation that the motor cortex has large, pyramidal neurons with extensive dendritic arborisation (Barbas and García-Cabezas, 2015; Cajal, 1899). Functionally, these structural differences might be considered in terms of narrow, specific columnar receptive fields for accurate sensory perception (Song et al., 2015), and wider receptive fields (Schellekens et al., 2018) for the coordination of multiple muscle groups (Mountcastle, 1997) in precise motor control. Thus there is a coherent group of cortical microcircuit properties which diverge from patterns of cortical myelination and fMRI-derived gradients, locating the motor cortex at the peak of a gradient of increasing cortical thickness, layer III, V and VI thickness and pyramidal neuronal arborisation, with primary sensory areas at the opposite extreme.
Atlas of cortical layers
The layers we have generated to test gradient based hypotheses have applications beyond the scope of this study. Surface based models of layer structure also create a framework for translating between microstructural modalities and surface-based neuroimaging. Furthermore current approaches to measuring laminar structure and function in vivo rely on prior models of the cortical layers - for example signal-source simulation in MEG (Troebinger et al., 2014) or for sampling laminar BOLD signal in fMRI (Muckli et al., 2015). The whole-brain histological models for areal layer depth provided here, combined with a good understanding of how the layers vary with local cortical morphology(Waehnert et al., 2014; Wagstyl et al., 2018a, 2018b) will aid such anatomical models.
Limitations
It is important to acknowledge that the gradients of laminar thickness measured may be affected by problems in the BigBrain dataset. The first limitation is that the post mortem brain was damaged during extraction and mounting. In some areas this resulted in minor shears. This problem was addressed to some extent through the utilisation of non-linear registration techniques. Nevertheless, some shifts in cortical tissue between consecutive sections are present and will affect the accuracy of layer reconstructions. In other areas the cortex has been torn. Spatial smoothing and the large total number of sample points make it unlikely that these errors are affecting results. A second limitation is that, without manually segmenting the entire cortex, it is not possible to know exactly how accurate layer measurements are in each region. However, in the 51 manually segmented regions, the automated laminar segmentations are exceptionally accurate and therefore it is unlikely that these errors are driving the large-scale trends of cytoarchitecture.
Total cortical thickness and thicknesses for each of the six isocortical layers were measured in the BigBrain to explore the histological drivers of MRI-based thickness gradients. Overall the pattern of thickness in the BigBrain was consistent with histological atlases of cortical thickness. In the visual and somatosensory cortices an increasing gradient of histological cortical thickness was identified and found to be primarily driven by layers III, V and VI. In the fronto-motor cortex the inverse pattern was found. These findings provide a link between patterns of microstructural change and morphology measurable through MRI and emphasise the importance of testing MRI-based anatomical findings against histological techniques. The laminar atlases provide an invaluable tool for comparison of histological and macroscale patterns of cortical organisation.