Abstract
Complex behavior requires fast changes of functional connectivity in large-scale cortical networks. Here, we report on the dynamics of functional coupling across visual, auditory and parietal areas during a lateralized detection task in the ferret. We hypothesized that fluctuations in coupling, indicative of dynamic variations in the network state, should be predictive of the animals’ performance. Analysis of power for hit and miss trials revealed significant differences around stimulus- and response-onset. In contrast, phase coupling already differed between hits and misses before stimulus onset, indicating fluctuations in large-scale network connectivity. In particular, higher phase coupling of visual and auditory regions to parietal cortex was predictive of task performance. Furthermore, we observed that long-range coupling became more predominant during the task period compared to the pre-stimulus baseline. Taken together, these results suggest that fluctuations in the network state, particular with respect to long-range connectivity, are critical determinants of the animals’ behavior.
Introduction
The brain continuously integrates information from different sensory systems enhancing its ability to detect noisy signals, to construct coherent percepts and guide appropriate actions (Stein and Meredith, 1993; Alais et al., 2010; Green and Angelaki, 2010; van Atteveldt et al., 2014). Such processes require the flexible orchestration of distributed neuronal populations within and across brain areas (Singer and Gray, 1995; Engel et al., 2001; Fries, 2005; Senkowski et al., 2008; Siegel et al., 2012). It has been proposed that coordination of distributed neural signals relies on several modes of intrinsically generated coupling which can involve synchronization of neural phase, correlation of amplitude envelopes, or phase-amplitude interactions (Raichle, 2010; Canolty and Knight, 2010; Siegel et al., 2012; Engel et al., 2013). Numerous studies have suggested that dynamic neuronal coupling operates in multiple frequency bands, which seem to be involved in mediating different cognitive functions (Fries, 2005; Buzsaki, 2006; Engel and Fries, 2010; Jensen and Mazaheri, 2010) or provide different channels for long-range communication (Bastos et al., 2015; Michalareas et al., 2016).
While substantial advances have been made in recent years in the analysis of dynamic functional connectivity in data recorded with non-invasive methods such as EEG or MEG (Brookes et al., 2011a; Siegel et al., 2012; Engel et al., 2013), a more mechanistic understanding requires the study of multi-site communication by direct recordings from the neural substrate. Approaches for invasive measurements of dynamic neural interactions are well established at the level of microcircuits (Coulter et al., 2011; Bastos et al., 2012; Feldmeyer et al., 2013), but addressing large-scale functional connectivity in relation to task performance or cognitive processing has remained a challenge. Invasive studies that addressed functional connectivity between distant population in behaving animals typically have recorded simultaneously only from relatively small numbers of sites. This holds, e.g., for studies of the coupling between early and higher-order visual cortex, or visual and prefrontal cortex subserving attentional selection (Womelsdorf and Fries, 2007; Gregoriou et al., 2009; Miller and Buschman, 2013) or of coupling between frontal and parietal-occipital regions during working memory (Liebe et al., 2012; Salazar et al., 2012).
However, to study the relation between large-scale network dynamics and behavior multi-site implants are required that permit simultaneous recordings from extended sets of distributed brain areas. Such recordings can be carried out using electrocorticographic (ECoG) arrays which have, beyond their clinical application, gained in importance in the study of functional network connectivity in recent years (Crone et al., 2006; Bosman et al., 2012; Keller et al., 2014; Fukushima et al., 2015; Lewis et al., 2015).
Here, we have employed an ECoG approach to study large-scale functional connectivity, as reflected in phase coupling of neural oscillations, underlying spatial multisensory processing. Studies on the relation between phase coupling and multisensory interaction are rare in humans (Keil et al., 2014; Giordano et al., 2017). In animal studies, the potential role of functional connectivity involved in multisensory interactions has so far been investigated only by multielectrode recordings from the same cortical areas (Lakatos et al., 2007; Kayser et al., 2008) or by simultaneous recordings from at most two different regions (Ghazanfar et al., 2008). We have now investigated dynamics during multisensory processing in a large-scale cortical network involving visual, auditory, somatosensory and parietal areas using 64-channel ECoG recordings in behaving ferrets.
Ferrets were trained in a 2-alternative-forced-choice paradigm to detect brief auditory and visual stimuli presented either left or right from the midline. We hypothesized that fluctuations in coupling, indicative of dynamic variations in the network state, should be predictive of the animals’ performance. Separate analysis of power for hit and miss trials revealed significant differences around stimulus and response onset but not for the baseline period before stimulus onset. In contrast, phase coupling already differed between hits and misses at the baseline, suggesting that fluctuations in network state are an important factor that determines task performance. Analysis of the topography of connectivity differences between hits and misses suggested specific patterns. In particular, higher phase coupling of visual and auditory regions with parietal cortex was predictive of task performance. Furthermore, we observed that long-range coupling became more predominant during the task period compared to the pre-stimulus baseline. Our results suggest that fluctuations in the networks state, particular with respect to long-range connectivity, are critical for task performance during multisensory processing.
Results
To investigate neural responses and task-related fluctuations of functional connectivity, we recorded local field potentials (LFPs) via 64-channel ECoG arrays chronically implanted in four female ferrets performing an audio-visual detection task (Hollensteiner et al., 2015). Initially, the animals were handled and accustomed to the experimental setup. Subsequently, they were trained in the detection task (Fig. 1) to determine unimodal and bimodal psychophysical thresholds, which were quantified by logistic fits to the response statistics (Fig. 2A). The ECoG was implanted after successful training and LFP recordings during execution of the acquired behaviors were carried out.
Behavioral data
The task involved four different types of trials. Stimulus presentation could be unimodal visual (V), unimodal auditory (A) or a spatiotemporally congruent bimodal audio-visual stimulus combination. The bimodal combination either consisted of an auditory stimulus of variable amplitude accompanied by a visual stimulus of constant contrast (Av) or, conversely, a visual stimulus of varying contrast accompanied by an auditory stimulus of constant amplitude (Va). The intensity value of the constant stimulus from the respective second modality was set at a level of 75% detection accuracy as determined during unimodal stimulation. Figure 2 shows an example for the psychometric functions fitted to the behavioral data for the four conditions. For further analysis, data were taken only from trials with stimulus intensities at threshold to allow comparisons across conditions and animals (Fig. 2A,B).
To investigate reaction time (RT) difference between unimodal and bimodal trial types, a paired sample t-test between was run between Av and A, as well as between Va and V trials, respectively. Responses to bimodal stimulation were, on average, faster than those to unimodal stimulation (p<0.05). To evaluate whether this effect was driven by the audio or the visual condition, we computed a repeated one-way analysis of variance (rm-ANOVA), with sensory modality as a main factor. This revealed a main effect of condition (F(3,12) = 5.3, p<0.05), but post hoc t-tests (Bonferroni corrected) only revealed significant differences between the A trials and all other conditions. The population averages of RTs for Av, Va and V were not significantly different (Fig. 2B). Nevertheless, the population RT mean of V (0.21 sec) was slightly higher compared to Va (0.19 sec) and Av (0.19 sec). The significant difference between the RTs in the A and the Av condition suggest that there was a multisensory interaction effect leading to improved performance for the bimodal stimulation. However, our data do not suggest a substantial improvement of visual detection by a concurrent auditory stimulus.
A major aim of our analysis of the electrophysiological data was to test the relation of power and functional connectivity to task performance. To this end, we contrasted hit and miss trials. In the hit trial group, we included all trials with stimulus intensities around 75 % threshold and stimulation contralateral to the implanted ECoG (265±14 hit trials per animal, mean±SEM). To match the number of hit trials we used all miss trials (236±45 miss trials per animal) throughout all sessions (15±2 sessions per animal). Miss trials were defined as trials with sensory stimulation but without behavioral response, i.e., the ferret maintained the centered head position throughout the response window. The number of false responses, i.e., orientation of the ferret to the side contralateral to the sensory stimulation, were too few and highly variable across animals (104±58 false responses per animal). Therefore, these trials were not included in analysis of the electrophysiological data.
LFP power reflects stimulus processing and response preparation
We hypothesized that spectral characteristics of LFPs immediately before task execution might predict the animal’s performance. To assess how LFP spectral characteristics evolve during hit, miss and bi- and unimodal stimulation, we computed the power across in analysis time windows aligned to trial-, stimulus- and response-onset for each animal individually (Fig. 3). Results of all animals were pooled after correction for different trial numbers. Power differences between hit and miss trials were analysed in the theta (4-8 Hz), alpha (8-16 Hz), beta (18-24 Hz), gamma (30-80 Hz) and high-gamma (80-160 Hz) frequency band across the analysis time windows (Tab. 1). In addition, one-way ANOVAs were calculated on power values within each frequency band with condition as the main factor to examine differences related to stimulus type (Suppl. Fig. 1; Suppl. Tab. 1).
Figure 3 shows the grand average spectra and time-frequency representation of power changes during each analysis time window (baseline, stimulus and response onset) for hit and miss trials. During the baseline only the high-gamma frequency-band (80-160 Hz) showed differences with higher power for miss trials (p<0.05; FDR corrected). Comparison of the average spectra in the stimulus onset analysis window showed significant reductions for hit and miss trials in the gamma frequency-range (30-160 Hz) compared to baseline (Fig. 3C, Tab. 1). However, hit trials showed significantly higher power compared to miss trials after stimulus onset in both gamma (p<0.001) and high-gamma (p<0.001) frequency bands (Fig. 3B). Around response onset, hit trials consistently showed significantly higher power compared to miss trials in the theta-(p<0.01), alpha-(p<0.01), gamma-(p<0.001) and high-gamma band (p<0.001). Statistical comparison across analysis time windows within hit- and miss-trials revealed significant de- and increase in the gamma frequency range compared to baseline and stimulus onset, respectively (Tab. 1; all p-values, within and across windows of interest, FDR corrected).
Supplementary Figure 2 shows the topographies of the respective spectral differences between hit and miss trials. In the baseline window, power topographies showed high spatial uniformity across all frequency bands. Around stimulus onset, notable differences occurred mainly in the beta band where, in hit trials, higher beta power occurred in parietal areas. Interestingly, highest regional differences occurred in the response window. Occipital regions exhibited higher power across all frequency bands during response onset in miss trials compared to hit trials. In contrast, auditory and parietal areas showed increased power in lower frequency bands (theta and alpha) for hit trials. An increase of gamma band power occurred in the response window for hit trials in regions around the lateral sulcus (Suppl. Fig. 2).
To investigate stimulus-type dependent effects within hit-trials a one-way ANOVA between the four conditions (Av, Va, A and V; Suppl. Fig. 1; Suppl. Tab. 1) was calculated. It showed no main effect in the factor condition across frequency bands during baseline, stimulus or response onset. This suggests that power changes related to stimulus processing and response preparation did not depend on the modality of the presented stimulus or on crossmodal interactions.
Functional connectivity predicts performance
To determine whether functional connectivity predicts the animals’ performance we computed the PLV between all pairs of ECoG channels for the different frequency bands. PLV analysis was performed for the same time windows, aligned to baseline, stimulus and response onset, respectively, as the power analysis reported above. Figure 4 displays the population average PLV spectra for all hit and miss trials as well as the difference between hits and misses and the relative spectral changes in the stimulus and response onset window relative to baseline. Paired t-tests were applied between hit and miss trials within and across all time windows for each frequency band. Within each analysis window, significantly higher PLV was observed for hits compared to miss trials in all frequency bands, with exception of the theta band during response onset (p<0.05, FDR corrected) (Fig. 4A).
When compared to the baseline time window, PLV was significanctly reduced in the stimulus and response onset windows in nearly all frequency bands. Exceptions were in the theta frequency band for miss trials around stimulus as well as response onset. From stimulus to response onset, there was an increase in alpha PLV and a decrease of gamma PLV for hit trials, as well as a significant decrease in high-gamma PLV for both trial groups (Fig. 4A and Tab. 2).
A one-way ANOVA within analysis time windows and frequency bands with Condition as the main factor revealed differences in PLV between stimulation conditions (Av, Va, A and V; Suppl. Fig. 3). During the baseline period there was no significant effects in the main factor condition in the theta frequency band (F(3, 44) = 2.46, p>0.05). However, starting with the alpha (F(3, 76) = 4.98, p=0.003), beta (F(3, 60) = 3.71, p=0.016), gamma (F(3, 412) = 31.89, p<0.001) and high-gamma (F(3, 652) = 56.99, p<0.001) frequency bands, there was a main effect in Conditions. This effect persisted also for the windows around stimulus onset (theta: F(3, 44) = 1.23, p>0.05; alpha: F(3, 76) = 3.74, p=0.015; beta: F(3, 60) = 3.5, p=0.021; gamma: F(3, 412) = 26.12, p<0.001; high-gamma: F(3, 652) = 55.67, p<0.001) and response onset (theta: F(3, 44) = 2.31, p>0.05; alpha: F(3, 76) = 3.63, p=0.017; beta: F(3, 60) = 3.67, p=0.017; gamma: F(3, 412) = 25.12, p<0.001; high-gamma: F(3, 652) = 51.24, p<0.001). Furthermore, post hoc t-test confirmed a constant pattern of significant differences between conditions (Suppl. Tab. 2-4). Interestingly, for hit trials connectivity was higher in the A and Av conditions than in the V condition, and there was a trend for connectivity in bimodal Va trials to be higher than in unimodal V trials (Suppl. Fig. 3).
In summary, our analysis of functional connectivity revealed significant differences between hits and misses already in the baseline period. The differences between hit and miss trials in PLV occurred over a broad range of frequencies across all analysis time windows. Importantly, this is in contrast to the power differences between hits and misses, which were not observed before stimulus onset and spectrally more confined. This suggests that the observed power changes predominantly reflect differences in local computations within sensory systems, whereas the changes observed in PLV indicate fluctuations in network state involving multiple frequency bands that relate to the animals’ performance.
Large-scale coupling shows specific changes during the task
To examine functional connectivity in relation to the topography of cortical areas, we assigned the data of each ECoG contact to a distinct region based on the functional map of cortical areas from Bizley et al. (2007) (see Methods) and constructed functional connectivity matrices accordingly (Fig. 5). Cortical areas were grouped in three functional systems, comprising visual (areas 17, 18, 19, 20, 21), auditory (areas A1, AAF, ADF, PPF and PSF), and parietal areas (SSY, PPc, PPr, S2). Differences between hit and miss trials were expressed using the sensitivity index (d’).
In line with the grand average PLV results, the connectivity matrices generally display positive d’ values and, thus, higher functional connectivity for hit compare to miss trials. (Fig. 5). This effect was most prominent during the baseline window and decreased during the stimulus window. For the alpha, gamma and high-gamma band, the PLV difference between hit and miss trials increased again in the response window. Generally, connections between early sensory cortices and parietal cortex displayed the highest differences between hit and miss trials, in particular during the baseline window (Fig. 5; Suppl. Fig. 4).
The topography of functional connectivity changes was further analyzed by contrasting between-system and within-system interactions (Fig. 6; Suppl. Fig. 4). For hit trials, interactions between the three functional systems were stronger than functional connections within the respective system. Regarding the within-system connectivity, the auditory system showed the lowest d’. In the period from baseline to stimulus-onset, the strength of d’ decreased across all frequency bands. With response onset the spectral profile of d’ became more heterogeneous. In the theta-band d’ decreased further, whereas it remained unaltered in the beta-band, and increased in the alpha-, gamma- and high- gamma band.
For statistical analysis, we averaged PLV d’ values and derived the difference of d’ values between long-range (between-system) and short-range (within-system) connectivity and calculated statistics across analysis time windows for each frequency band (Fig. 6; for statistics across systems and within analysis time windows and frequency see Suppl. Fig. 4 and Suppl. Tab. 5; for modality specific statistics see Suppl. Fig. 5). To reveal dynamic effects within frequency bands of interest a one-way ANOVA with time window as the main factor was computed. There was no main effect in the theta band (F(2, 6) = 4.39, p>0.05). The ANOVA exposed significant effects for the main factor time window (alpha (F(2, 12) = 17.17, p<0.001; beta (F(2, 9) = 14.43, p=0.002; gamma (F(2, 75) = 3.71, p<0.001; high-gamma (F(2, 120) = 17.94, p<0.001). Post hoc t-tests revealed significant differences between baseline and stimulus-onset for all frequency bands with the main effect (alpha: p=0.011; beta: p=0.002; gamma: p<0.001; high-gamma: p<0.001), between stimulus and response onset for the gamma frequency range (gamma: p=0.009; high-gamma: p<0.001) and between baseline and response-onset in the alpha-(p<0.001) and beta-(p=0.006) frequency bands (Bonferroni corrected).
Taken together, comparison of functional connectivity between and within cortical systems across the different analysis time windows revealed that between-compared to within-system connectivity was stronger during the task period compared to the pre-stimulus baseline. In hit trials, phase coupling was higher between systems in the gamma frequency range during stimulus onset compare to baseline and response onset. Furthermore, low frequency (alpha and beta) coupling between systems increased around response onset in hit trials compared to miss trials. These results suggest that fluctuations in the networks state, particular with respect to long-range connectivity, are related to task performance.
Discussion
This study has aimed at investigating the cortical network dynamics underlying performance in a multisensory detection task. The main focus of our analysis was on the relation between functional connectivity and task performance. We have used an ECoG approach to record a large-scale network comprising visual, auditory, somatosensory and parietal areas in behaving animals over extended time periods. We hypothesized that fluctuations in power and coupling, indicative of dynamic variations in the network state, should be predictive of the animals’ performance. In line with this hypothesis, our data show that pre-trial baseline connectivity predicts the animals’ performance, suggesting that fluctuations in coupling across the network lead to variability in behavior. In contrast, local signal power in the pre-trial baseline does not predict hits and misses. Analysis of the topography of connectivity differences between hits and misses suggested specific patterns. In particular, higher phase coupling of visual and auditory regions to parietal cortex was predictive of task performance. Furthermore, we analyzed the ratio between long- and short-range functional connectivity across the different task phases. We observed that long-range coupling became more predominant during the task period compared to the pre-stimulus baseline and changed its spectral profile over the course of the trials.
Investigating large-scale network dynamics
A large number of studies have shown that large-scale functional connectivity is highly relevant for cognition and behavior (Engel et al., 2001; Varela et al., 2001; Fries, 2005; Engel and Fries, 2010; Siegel et al., 2012). In particular, phase coupling has been suggested to serve the routing of information through cortical networks and to promote selective communication between distant brain areas (Engel et al., 2001; Fries, 2005; Womelsdorf et al., 2007). In the human brain, studies of large-scale network dynamics have remained challenging, although substantial advances have been made with non-invasive methods such as EEG or MEG (Brookes et al., 2011a; Siegel et al., 2012; Engel et al., 2013; O’Neill et al., 2017). In animal studies, a substantial number of investigations have addressed functional connectivity between distant populations, but these have typically been restricted to recordings from a relatively small number of sites (Buschman and Miller, 2007; Gregoriou et al., 2009; Salazar et al., 2012). However, in order to capture large-scale network dynamics and fluctuations of functional connectivity across multiple regions, implants are required that permit simultaneous recordings from extended sets of brain areas.
To this end, ECoG arrays have gained importance in recent years (Crone et al., 2006; Keller et al., 2014; Fukushima et al., 2015; Lewis et al., 2015). Beyond their clinical application, ECoG arrays are ideally suited for the study of functional connectivity in networks underlying cognitive and sensorimotor functions. In patients, phase coupling across brain areas has been investigated using ECoGs in a variety of perceptual and cognitive tasks, providing supportive evidence for the notion that long-range synchronization of oscillatory signals has a functional role (Canolty and Knight, 2010; Fox et al., 2018). In monkeys, ECoG recordings have been applied to study phase coupling between early and higher-order visual areas serving attentional selection (Bosman et al., 2012; Bastos et al., 2015). In rodents, ECoG approaches have been employed to investigate patterns of synchrony in visual cortex (Toda et al., 2018). We have recently used ECoG implants to investigate phase coupling during spontaneously occurring state changes in the ferret (Stitt et al., 2017).
For studies chronically implanted animals, ECoG arrays are advantageous because of the long-term stability of the implant and the recorded signals (Rubehn et al., 2009; Bosman et al., 2012; Bastos et al., 2015; Stitt et al., 2017). Thus, they permit repetitive and long-term tracking of brain activity and the investigation of spatiotemporal neural patterns under a broad variety of experimental conditions. Another advantage is the higher signal-to-noise ratio of the recorded local field potential signals. Furthermore, it has been shown that high-resolution ECoG arrays with sufficiently small contacts also permit the recording of spikes from the cortical surface (Khodagholy et al., 2016; Bockhorst et al., 2018).
Despite the fact that the ECoG array in this study covered only the posterior half of the cortical hemisphere, it enabled us to record from an extended network of cortical areas involving visual, auditory, somatosensory and parietal areas, allowing to investigate large-scale network dynamics during trained behavior over a large number of recording sessions. In addition to the advantages mentioned above, a key feature in the present study was the possibility to simultaneously monitor several different functional systems. Since our micro-ECoG array also allowed recordings from multiple areas within the same cortical system, this approach enabled a systematic comparison of within-system and between-system functional connectivity as a function of the animals’ behavior.
Relation of large-scale connectivity to task performance
A key goal of this study was to investigate the relation between phase coupling and performance of the animal in the detection task, with the aim to provide further evidence for the functional relevance of synchrony across cortical populations. A number of different approaches have been used to capture the relation between large-scale connectivity and task performance. In humans, phase coupling has been increasingly studied using EEG or MEG in recent years (Fell and Axmacher, 2011; Siegel et al., 2012; O’Neill et al., 2017), enabled by progress in source localization techniques and in connectivity methods that allow to remove spurious coupling resulting from volume conduction (Nolte et al., 2004; Gross et al., 2001; Brookes et al., 2011a; Hipp et al., 2012). Using these approaches, coherence of different brain areas has been observed during cognitive tasks involving perceptual selection (Hipp et al 2011), attentional selection (Siegel et al., 2008; Michalareas et al., 2016), working memory (Brookes et al., 2011b) or speech processing (Giraud and Poeppel, 2012; Giordano et al., 2017).
Animal investigations have elucidated the relation between large-scale connectivity and task performance on different spatial and temporal scales. Early studies performig multi-site recordings in chronically implanted animals demonstrated a relation of neural synchrony to sensorimotor processing (Bernasconi et al., 2000; Roelfsema et al., 1997). Long-range phase coupling in various frequency ranges has been shown to be related to attentional selection and working memory processes in the monkey (Gregoriou et al., 2009; Salazar et al., 2012; Liebe et al., 2012; Bosman et al., 2012; Saalmann et al., 2012; Bastos et al., 2015). Importantly, recordings in monkeys suggest that feedforward and feedback signaling between cortical areas are mediated by distinct frequency channels, i.e., gamma coupling is likely to subserve feedforward whereas lower-frequency coupling seems to be involved in feedback signaling (Buschman and Miller, 2007; Bastos et al., 2015).
Our ECoG approach enabled us to investigate dynamic large-scale connectivity between functional cortical systems during a spatial detection task. Our data show that phase coupling, in contrast to local power, is already different between hit and miss trials in the pre-trial baseline, suggesting that fluctuations in network state may be an imported factor that determines task performance. The data indicate that stronger phase coupling in the network may lead to higher efficiency of stimulus processing and response preparation. This is in line with earlier studies that have demonstrated that increased synchrony enhances communication across brain areas, e.g., in the context of attention (Fries, 2005; Bosman et al., 2012). Enhanced functional connectivity seems important to link task-relevant modalities to areas involved in selection of responses (Siegel et al., 2012; Miller and Buschman, 2013). Indeed, we find that stronger phase coupling of early sensory to parietal areas was predictive of task performance (Suppl. Fig. 4).
Interestingly, no clear frequency specificity was apparent in performance-predictive phase coupling as observed in previous studies. We did not observe the typical signatures of vigilance- or attention-related fluctuations characterized by an increase in higher frequency components and a decrease in low frequenciy bands (Hanslmayr et al., 2007; Siegel et al., 2008; Bosman et al., 2012). Importantly, we also did not observe classical signatures of arousal changes in local signal power, since we did not see spectral differences between hit and miss trials in the pre-stimulus baseline in the recorded cortical areas.
Although the average connectivity differences between hits and misses had a rather broad spectral profile, a closer analysis of the contrast between long-range and local connections revealed an interesting pattern (Fig. 6). The comparison of functional connectivity between and within cortical systems across the different analysis time windows revealed that long-range compared to short-range connectivity was stronger during the task period compared to the pre-stimulus baseline. In hit trials, between-system phase coupling in the gamma frequency range was higher during stimulus onset compared to baseline and response onset. In contrast, in low frequency (alpha and beta) ranges between-system coupling increased around response onset in hit trials compared to miss trials.
Although we did not employ directed connectivity measures such as Granger causality to distinguish between bottom-up and top-down information flows (Bastos et al., 2015), these data suggest the possibility that, during stimulus processing, higher frequency interactions predominate in successful trials, which might reflect stronger bottom-up signaling. This could relate to attentional gating, allowing information to pass through the network to higher areas. In contrast, the predominance of low-frequency interactions in the epoch around reponse onset might indicate stronger top-down information flow (Bastos et al., 2015). Alternatively, in that phase of the trial, parts of the network could already return to a default state dominated by low frequency oscillations. Analysis of directed connectivity would be required to further test these hypotheses. Averaged across areas, spectral differences between connections with dominant bottom-up and top-down information flows might average out and, thus, yield the broad spectral profile of connectivity changes observed in the present analysis.
Functional connectivity and multisensory processing
As discussed above, dynamic functional coupling likely constitutes a mechanism for integration of distributed neural signals. This holds, in particular, for perceptual integration within individual sensory modalities (Singer and Gray, 1995; Singer, 1999; Engel et al., 2001; Fries, 2005; Arnal et al., 2015; Ploner et al., 2017). It has been suggested that similar mechanisms might operate for the integration of information across different sensory systems. Thus, multisensory interactions might involve dynamic coupling of oscillatory signals arising in different cortical systems (Senkowski et al., 2008; Talsma et al., 2010). In the human brain, the vast majority of studies on neural oscillations and crossmodal processing have focused on local power changes, and only few investigations have addressed the relation between multisensory processing and functional coupling (Keil et al., 2014; Giordano et al., 2017). Using a data-driven approach for analysis of functional coupling in the human EEG, we could recently demonstrate that coherence in networks involving parietal and sensorimotor areas predicts performance in a visuotactile matching task (Wang et al., 2018).
A number of studies in monkeys have investigated functional connectivity during multisensory tasks. Thus, simultaneous recordings from auditory cortex and superior temporal sulcus revealed increased coherence during congruent auditory and visual stimulation (Maier et al., 2008). Another study showed that interactions mediated by coherent gamma band oscillations between these areas are involved in crossmodal integration of information related to faces and voices (Ghazanfar et al., 2008). An interesting mechanism that might also contribute to multisensory integration has been suggested by studies demonstrating that phase-resetting of oscillatory activity in sensory areas can occur under the influence of stimuli from a different sensory modality (Lakatos et al., 2007; Kayser et al., 2008; Schroeder and Lakatos, 2009).
While these in-vivo studies investigated multisensory interactions only by recordings from the same cortical areas or by simultaneous recordings from at most two different regions, our study has addressed multisensory networks at a larger scale involving visual, auditory and parietal cortical regions. While our data clearly demonstrate a relation of functional connectivity to task performance, we did not obtain evidence for stimulus-specific changes of coupling between visual and auditory areas. Behaviorally, we observed audiovisual interaction effects reflected in a shortening of reaction times for bimodal compared to unimodal stimulus detection. However, there were no strong connectivity difference related to these multisensory interaction effects. Rather, our findings show broadband coupling between visual, auditory and parietal areas, which gets stronger before hits compared to instances where the animal fails to detect the stimulus. Interestingly, connectivity in auditory hit trials was higher than in visual hit trials and, in the visual conditions, there was a trend for higher connectivity in bimodal compared to unimodal trials (Suppl. Fig. 3).
The absence of profound connectivity effects related to multisensory interaction may relate to the nature of the task used in our study, which did not require integration of features across modalities, but only the rapid detection of highly transient stimuli. Nonetheless, our connectivity data support the notion that significant functional coupling can occur already between early sensory areas, suggesting that multisensory integration can already occur at early processing stages and does not solely rely on binding of information at higher processing levels (Bizley et al., 2007; Lakatos et al., 2007; Senkowski et al., 2008). This is in line with anatomical data suggesting that direct projections from primary auditory to visual cortices can enable functional coupling supporting early multisensory interactions (Bizley et al., 2015).
Network state and behavioral variability
A key result of our study is that phase coupling differed between hit and miss trials already in ongoing activity before stimulus onset. Interestingly, local power in the pre-trial baseline did not differ between hits and misses. Our data suggest that fluctuations in the network state occur which lead to variability in the animals’ behavior, and that these state changes are primarily reflected in shifts of long-range connectivity, rather than changes in the dynamics of local populations.
Variability in ongoing activity has been addressed in numerous studies which showed that pre-stimulus activity can influence sensory processing and behavior. Early studies assumed that ongoing neural activity corresponds to noise resulting from random signal fluctuations without any functional relevance (Zohary et al., 1994; Shadlen et al., 1996). However, this view is defeated by evidence showing that ongoing activity carries information and can shape the processing of stimuli (Arieli et al., 1996; Galindo-Leon et al., 2019). Abundant evidence is available that oscillatory temporal patterning of ongoing activity is important for sensory and cognitive processes (Busch et al., 2009; Keil et al., 2014). One relevant mechanism is that the ongoing neuronal oscillations can amplify or diminish local responses depending on whether inputs arrive during phases of high or low excitability (Fries et al., 2001; Engel et al., 2001; Lakatos et al., 2007). Such fluctuations of ongoing activity do not occur only locally, but are strongly synchronized across spatially distributed neuronal populations (Steriade et al., 1996; Stitt et al., 2017; Fischer et al., 2018).
Phase coupling of oscillations in pre-stimulus epochs has been shown both in animal and human studies to predict perception and performance in cognitive tasks. For instance, studies in monkey visual cortex indicate that fluctuations in gamma-band phase coupling modulates the speed at which animals can detect a behaviourally relevant stimulus change (Womelsdorf et al., 2006). EEG studies in humans provide convergent evidence that pre-stimulus fluctuations in phase coupling can modulate target detection (Hanslmayr et al., 2007). Furthermore, intrinsic fluctuations of phase coupling is associated with fluctuations in perceptual states in ambiguous stimulus settings. Fluctuations in beta-band or gamma-band phase coupling have been shown to predict the perceptual state in ambiguous sensory paradigms (Rose and Büchel, 2005; Hipp et al., 2011) and decision making in near-threshold stimulation regimes (Donner et al., 2009). Our current results corroborate and extend this evidence by showing that, at perceptual threshold, detection of the lateralized stimuli by the animals is biased by phase coupling in the pre-stimulus interval. A novel element in our approach is the ability to simultaneously monitor numerous cortical areas, enabling us to quantify within-system and between-system interactions and to characterize these in a spectrally resolved manner. Our analysis showed that, in particular, long-range connectivity between different functional systems was related to successful stimulus detection. This is in line with the hypothesis that long-range phase coupling may serve efficient transmission of task-relevant information in sensorimotor networks (Engel et al., 2001; Fries, 2005; Womelsdorf et al., 2007).
Our results also support the view that characterization of brain state changes strongly benefits from inclusion of connectivity analyses. The observation that changes in network state are reflected, in particular, in fluctuations of large-scale connectivity is in line with results of other recent studies. Supportive evidence has been obtained in studies on the human brain (He et al., 2008; Supp et al., 2011). Using the same ECoG recording approach for the study of ongoing activity, we have observed that functional connectivity shows state-dependent reconfiguration which can also involve shifts in the ratio between short- and long-range interactions (Stitt et al., 2017; Fischer et al., 2018). These findings raise the question of possible mechanisms that might modulate large-scale functional connectivity in a state- and task-dependent manner. Possible candidates are changes in the output of ascending neuromodulatory systems (Harris and Thiele, 2011). Phase coupling has long been known to be influenced by neuromodulators (Steriade et al., 1993). For instance, activation of cholinergic brain stem nuclei has been shown to enhance gamma-band phase coupling in cortical networks (Munk et al., 1996). Neuropharmacological evidence suggests, furthermore, that noradrenergic brain stem inputs can modulate large-scale functional connectivity in cortex (van den Brink et al., 2018).
Conclusion
Our study has demonstrated functional coupling across visual, auditory and parietal areas during a lateralized detection task in the ferret. Analysis of power for hit and miss trials revealed significant differences around stimulus and response onset. In contrast, phase coupling already differed between hits and misses at baseline, suggesting fluctuations in large-scale network connectivity. In particular, higher phase coupling of visual and auditory regions to parietal cortex was predictive of task performance. We observed that long-range coupling became more predominant during the task period compared to the pre-stimulus baseline. Taken together, these results suggest that fluctuations in the network state, particular with respect to long-range connectivity, are critical determinants of the animals’ behavior. Future studies might address the relation to the underlying structural connectivity and the mechanisms that give rise to the observed variability of phase coupling. Furthermore, analyses of directed coupling might elucidate the interplay of bottom-up and top-down interactions during network states predictive of task performance.
Materials and Methods
Animals
Data were recorded in four adult female ferrets (Mustela putorius furo) trained on a two-alternative forced-choice task involving lateralized detection of visual and auditory stimuli that we have established previously (Hollensteiner et al., 2015). All experiments were approved by the independent Hamburg state authority for animal welfare (BUG-Hamburg) and were performed in accordance with the guidelines of the German Animal Protection Law.
Sensory stimulation
All experiments were carried in a dark sound-attenuated chamber (Acoustair, Moerkapelle, Netherlands). Visual and auditory stimuli were generated with the Psychophysics Toolbox (Brainard, 1997). Visual stimuli were presented on an LCD monitor (Samsung SyncMaster 2233, frame rate 100 Hz) placed 20cm in front of the animal. Auditory stimuli were generated digitally with a sample rate of 96 kHz and delivered through two Beyerdynamic T1 speakers. Auditory stimuli consisted of white noise bursts of 100 ms duration and were presented with intensities between 2 to 62 dB SPL. Visual stimuli consisted of circular gratings (22.5 °, 0.2 cycles / °, 5 Hz) with Michelson contrast (Cm) between 0 and 0.38. A static random noise pattern located in the center of the screen was presented to indicate trial onset by a decrease in contrast. Multisensory stimuli consisted of both stimulation types presented simultaneously.
Training
Initially, ferrets were handled and accustomed to the experimental setup (Fig. 1A). Subsequently, they were trained in the spatial detection task, and unimodal as well as crossmodal psychophysical thresholds were estimated. In the task, we used a crossmodal approach that bears on established unimodal training paradigms (e.g. Nodal et al., 2008). Animals were restricted from access to water for a period of 8 hours before the measurements, and conditioned by using water rewards during the task. In the first phase of the study, the animals were trained to detect unimodal auditory and visual stimuli presented in a randomized fashion. Auditory and visual unimodal detection thresholds were determined using 20 different stimulus amplitudes (auditory: 2-62 dB SPL; visual: 0-0.38 Cm) in a 1 down / 3 up staircase procedure (Kaernbach, 1991). Next, unimodal and bimodal thresholds were assessed in a combined approach, using the previously determined unimodal thresholds to adjust the test parameters (for details of the procedure see Hollensteiner et al., 2015). Subsequently, the ferrets were accustomed to electrophysiological recordings during the detection task with a reduced set of stimulus amplitudes (eight per modality), adjusted to the individual psychometric functions to acquire a higher number of trials in the performance range of interest around 75% accuracy.
Detection task
The trial schedule of the detection task is shown in Figure 1B and 3A. To initialize a trial, the ferret had to maintain a central head position to break the central light-barrier for 500 ms. This caused the static random noise pattern in the center of the screen to decrease in contrast informing the animal that the window for stimulus presentation (from 0 to 1000 ms after trial onset) had started. During this interval, the animal had to further maintain its central head position. A stimulus was presented for 100 ms on either the left or the right side. Stimulus presentation could be unimodal visual (‘V’), unimodal auditory (‘A’) or a temporally congruent bimodal audio-visual stimulus combination. This combination either consisted of a variable auditory stimulus accompanied by a visual stimulus of constant contrast (‘Av’) or, conversely, a visual stimulus of varying contrast supported by an auditory stimulus of constant amplitude (‘Va’). The intensity value of the accompanying second stimulus with constant amplitude was set at a level of 75% accuracy. After stimulus offset, the animal had to respond within 700 ms by moving its head to the stimulated side; otherwise, the trial was considered as a miss (no response). If the response was correct the animal received a water reward (∼80 µl) from a spout at the stimulus position and could immediately start the next trial. If the response occurred prematurely (before stimulus onset or within 100 ms after stimulus onset), was incorrect (wrong side) or omitted (no response), the trial was immediately terminated, followed by a 2000 ms interval during which no new trial start could be initialized.
Implantation and alignment of ECoG array
Micromachining technology (Rubehn et al., 2009) was used to design and implement an ECoG array that matched the anatomy of the ferret brain (Fig. 7). Each thin-film (10µm) polymide-foil ECoG contained 64 platinum electrodes with a diameter of 250 µm, each, arranged in a hexagonal formation at an inter-electrode distance of 1.5 mm.
The surgical procedure for the implantation of the ECoG array started with an initial injection of anesthesia (ketamine 15 mg/kg and medetomidine 0.08 mg/kg) and additional ketamine (20 mg/kg) was supplied during surgery. The animal was freely breathing throughout the implantation. Physiological parameters such as the electrocardiogram (ECG) and rectal temperature were monitored throughout the surgery. All surgical procedures were performed under sterile conditions. After the operating field was prepared, a craniotomy was performed using a saline-cooled ultrasonic microsaw (Mectron) in order to expose the posterior half of the left cerebral hemisphere. The dura was carefully removed and the ECoG array was gently placed on the surface of the cortex such that it covered occipital, temporal and parietal areas (Fig. 7D). The dura was then folded back over the ECoG array and an absorbable artificial dura was placed above the ECoG, covering the trepanation to full extent. The excised piece of bone was fixed back in place with titanium plates and screws and subsequently the gaps were filled with fast set absorbable bone putty. Finally, the ECoG’s connector was placed on top of the skull and fixed in place with titanium screws and dental acrylic. After the surgery, the animals received analgesics (carprofen, 4mg/kg) and antibiotics (enrofloxacin, 5 mg/kg) for at least 7 days. To align electrode positions across animals, the ECoG array placement and the cortical parcellation introduced by Bizley et al. (2007) was utilized (Fig. 7B). For each animal, the exact ECoG array position over the posterior cortex was photographically documented during surgery. The locations of all 64 ECoG electrodes were then projected onto a scaled illustration of a ferret model brain. Each electrode was then mapped to the underlying cortical area.
Electrophysiological recordings
Local field potentials (LFPs) were digitized at 1.4 kHz and sampled simultaneously with a 64-channel AlphaLab SnR recording system (Alpha Omega Engineering, Israel). The high pass filter was set at 0.1 Hz and the low pass filter at 357 Hz.
Data analysis and preprocessing
To reveal dynamic neurophysiological processes along the trial time line, we analyzed spectral power and functional connectivity in the LFPs recorded during the task. To ensure comparability across animals and modalities, only trials with an accuracy of 75±10 % were considered in all electrophysiological analyses. Furthermore, this selection of trials ensured better comparability between all four modalities because the value that was fixed during the bimodal stimulation was set at a level of 75 % accuracy for all stimulation amplitudes. To assess a sufficient amount of right side trials (i.e., trials with stimulus presentation contralateral to the implanted hemisphere) per modality in the 75 % accuracy range, trials collected on different days were pooled; see Hollensteiner et al. (2015) for non-stationarity effects across sessions.
All offline data analysis was performed using custom scripts in Matlab (The Mathworks Inc, MA). The hardware-filtered LFP signals were band-pass filtered with a phase-preserving fourth-order Butterworth filter between 2 and 200 Hz. Next, band-stop Butterworth filters at 49-51, 99-101 and 149-151 Hz were applied to notch the line noise. Following, the continuous LFP signals were cut into epochs aligned to trial, stimulus and response onset. In each of these analysis time windows data was cut from 500 ms pre- to 500 ms post-onset, with 500 ms real data padding on each side to prevent edge artifacts in frequency space. Afterwards, we applied independent component analysis (Hyvärinen and Oja, 2000) to the concatenated analysis window data in order to detect ECG, muscle and eye blink artifacts. Subsequently, we re-referenced the LFP time series of each ECoG contact with the LFP time series of its right neighbor. This processing step created 55 virtual electrodes from the 64 recorded contacts. The virtual electrode position was derived from the difference of both real electrode positions. Subsequently, the oscillatory signal components of unimodal and bimodal trials where analyzed using spectral decomposition.
Spectral power analysis
Channel-wise spectral power was computed by taking the square of the absolute value of time frequency estimates. Spectra were computed for all three analysis time windows using a Hanning window approach (2-200 Hz, 2 Hz steps, 500 ms window; for the LFP spectrogram the window was shifted from –500 ms to 500 ms around the window of interest) in 1 ms steps.
Functional connectivity
To estimate phase synchronization between ECoG signals, we calculated the PLV (Lachaux et al., 1999) across all frequencies (2-200 Hz). In general, the instantaneous phase θ was extracted from the analytic signals that were produced by the Fourier transform of the convolution of the ECoG time series with the Hanning window. The PLV between channels A and B at carrier frequency f is defined as follows:
At first, PLV was computed between all pairs of ECoG electrodes. To contrast PLV between modalities, we first computed the global mean PLV for the same time windows selected in the spectral power analysis. Subsequently, the PLV between anatomical regions, as defined in Figure 7B, was calculated by averaging across all virtual electrodes overlaying the same area. To normalize and compare PLV values from hit and miss trials we computed the sensitivity index (d’), which is defined by: where µ denotes the mean and s the standard deviation for hit and miss trials, respectively.
Competing Interests
They are no competing interests.
Author Contributions
K.J.H, F.P. and A.K.E. conceived and designed experiments. G.E. wrote animal ethics permission. K.J.H and F.P. surgically implanted arrays. K.J.H. performed experiments. K.J.H., F.P., E.G.L. and G.N. provided analyzing tools. K.J.H. analyzed data. K.J.H. and A.K.E. wrote the paper. All authors edited and approved the manuscript.
Acknowledgements
We would like to thank Dorrit Bystron for her assistance throughout experiments. This research was supported by funding from the DFG (SFB 936/A2, SPP 1665/EN533/13-1, SPP 2041/EN533/15-1, A.K.E.; SFB936/Z3, G.N.).