Abstract
Social functioning involves learning about the social networks in which we live and interact; knowing not just our friends, but also who is friends with our friends. Here we utilized a novel incidental learning paradigm and representational similarity analysis (RSA), a functional MRI multivariate pattern analysis technique, to examine the relationship between learning social networks and the brain’s response to the faces within the networks. We found that accuracy of learning face pair relationships through observation is correlated with neural similarity patterns to those pairs in the left temporoparietal junction (TPJ), the left fusiform gyrus, and the subcallosal ventromedial prefrontal cortex (vmPFC), all areas previously implicated in social cognition. This model was also significant in portions of the cerebellum and thalamus. Our findings indicate that these areas of the brain not only process knowledge and understanding of others, but also support learning relations between individuals in groups.
The Neural Representational Space of Social Memory
Social relationships guide and support nearly all human behavior. Not only do we form strong lifelong family bonds, we also interact with others in work, education, and leisure settings and are able to create strong lasting non-kin relationships. Humans are not unique in this capacity; it has long been known that other primates create large social networks, but there is also emerging evidence that non-primate species such as elephants, hyenas, horses, dolphins, and whales engage in complex social and cultural bonding (Cantor and Whitehead, 2013; Seyfarth and Cheney, 2012). It is likely that the social propensity of humans has evolutionary origins, as the neurobiology of social cognition has been remarkably conserved across vertebrate species (Seyfarth and Cheney, 2015; Weitekamp and Hofmann, 2014). In particular, human and non-human primate cognitive functioning is well suited to promote social engagement through several important brain areas (Dunbar and Schultz, 2007). The posterior superior temporal sulcus (pSTS), temporoparietal junction (TPJ), medial prefrontal cortex (mPFC), and amygdala serve to make up a network that is particularly tuned to sociality, and underpin processes critical for interacting with others such as social perception, action understanding, and mentalizing (Yang, Rosenblau, Keifer, and Pelphrey, 2015). However, the full extent of the brain’s involvement in human sociality is still a question of great interest.
The ability to perceive and understand social signals yields great benefits when creating and maintaining relationships with others. Several of the aforementioned areas have been demonstrated to have specific roles in forming relationships. Dynamically changing relationship ties and tie strength has been linked to activity in the bilateral pSTS and TPJ (Bault, Pelloux, Fahrenfort, Ridderinkhof, and van Winden, 2015). The left TPJ is selectively modulated by vasopressin, a neuropeptide linked to a number of complex social behaviors, during social recognition (Zink et al., 2011) and lesions to this area create specific deficits in social reasoning (Samson, Apperly, Chiavarino, and Humphreys, 2004). In addition, the TPJ, dorsomedial PFC, and ventrolateral PFC have been shown to be engaged when recalling different facets of socially relevant knowledge (Satpute, Badre, and Ochsner, 2014). It is also likely that the reward network, in particular the subcortical striatum and the ventromedial PFC, plays a role in the formation of social ties and social networks; this may be especially important when the individual encoding the network is embedded within it, as social reward has been shown to be a strong driving factor in making decisions and pursuing certain social outcomes (Fareri and Delgado, 2014). This is supported by research demonstrating that both the social cognition network and the affective valuation network track the popularity of real-world social network members (Zerubavel, Bearman, Weber, and Ochsner, 2015), the vmPFC shows increased activation when thinking about friends compared to kin (Wlodarski and Dunbar, 2016), and the subgenual cingulate cortex is involved in tracking individual differences in perceptions of cohesiveness in kin groups (Rüsch et al., 2014).
Due to our highly social world and our ability to form relationships with numerous others, we are required to hold many simultaneous relationships of both acquaintances and close friends in our memory. Through nothing more than observation, we must be able to discern who is friends with whom and how close they are due to the frequency with which they interact. Individuals tend to report group and relationship averages or norms more accurately than individual interactions, but more experienced observers show more accurate recall, especially when group structure is transitive (Freeman and Romney, 1987; Freeman, 1992; Kumbasar, Romney, and Batchelder, 1994). Furthermore, while human beings as a whole are very social, there are individual differences in how well people can encode and recall social network patterns (Brewer, 2000; Casciaro, 1998; Freeman, Romney, and Freeman, 1987). Some individuals have more social skills than others, which could in part be influenced by how well they learn from the social world around them (Heerey, 2013; Milch-Reich, Campbell, Pelham, Connelly, and Geva, 1999). Sociality is also linked to both neural and behavioral measures of social perception (Baron-Cohen et al., 1999; Bickart, et al., 2011; Dziura and Thompson, 2014; Kanai et al., 2011). There are also differences in the ability to perceive and remember non-social patterns, but evidence suggests that learning, remembering, and storing social information might be distinct from traditional learning and memory systems (Albers, 2012; Okuyama, Kitamura, Roy, Itohara, and Tonegawa, 2016; Meyer and Lieberman, 2012; Meyer, Taylor, and Lieberman, 2015; Tendler and Wagner, 2015). Neural populations that code social hierarchy information are found in social, nonsocial, and reward areas (Chiao, 2010; Kumaran, Melo, and Duzel, 2012; Zink et al., 2008). It has been suggested that humans use cognitive heuristics such as triadic closure in order to remember social ties (De Soto, 1960; Freeman, 1992; Brashears, 2013; Brashears and Quintaine, 2015), but it is not yet understood what underlying neural processes are involved in the recall of complex social networks.
In this study, we examined the learning and neural representation of novel social networks, using fMRI and representational similarity analysis. We examined the learning of not only social network ties but tie strength, and determined the neural representational space related to learning and memory of social network structures. This study was conducted using a novel paired association task among connected members of the networks, where the frequency of paired presentation indicated a stronger relationship. The patterns of voxel fMRI responses to each face in the network were then assessed and compared to the accuracy of tie strength recall among the faces using functional magnetic resonance imaging (fMRI) and multivariate pattern analysis. Our results show that people learn relationship patterns between previously unknown faces based on the frequency of co-occurrence of face pairings, and that the accuracy of the recall of relationship tie strength was significantly correlated with the neural pattern similarity between faces in TPJ, fusiform gyrus, and subgenual vmPFC.
Results
Behavioral task
Participants became familiar with the structure of two six-person social networks by viewing two faces presented simultaneously (Figure 1). A paired set of faces represented a connection within the network, with the frequency of pairing indicating relationship strength. Each trial consisted of a face pair accompanied by an unrelated personality question, and participants were asked to make a comparison between the faces and decide which person better fit the characteristic. For recall testing, participants were told that the frequency of paired presentation represented relationship strength. They were then presented with face pairs again and asked to rate the pair’s relationship on a scale of 0 (do not know each other) to 4 (know each other very well). See Methods for a more detailed explanation of the learning task.
Analysis of social network recall data was conducted in Microsoft Excel (version 2016) and R Version 3.3.2 (https://www.r-project.org/). Subjects correctly identified relationship ties significantly greater than chance across both networks (t(21) = 8.08, p < 0.0000001). Table 1 shows the average hit rate, false alarm rate, sensitivity (d’), and the correlation between true and reported perceived strength for ties and relationship strength across subjects. Paired sample two-tailed t-tests revealed no significant differences between recall measures for the two networks. There were also no significant age or gender effects for any of the measures. When averaged together across subjects, group perceived relationship strength was highly correlated with the true network structure (r = 0.896, p < 0.00001). In order to assess whether our behavioral task was comparable to previous forms of social network learning and recall, we calculated performance measures used by Brashears (2013). Accuracy refers to the number of ties correctly recalled divided by the number of total ties reported, coverage refers to the number of ties correctly recalled divided by the total tie number in the network, and performance refers to the product of accuracy and coverage. T-tests revealed no significant differences between accuracy or performance measures in our task and those of Brashears (accuracy: t(21) = 0.98, p = 0.34; performance: t(21) = 0.58, p = 0.56), and we actually saw an increase in coverage (t(21) = 3.58, p < 0.01), although our networks were smaller, so participants did not need to remember as many ties.
When exploring network recall, it is important to not only look at the correctly identified ties, but also at the pattern of mistakes made. Specifically, we wanted to see whether there are systematic biases that could be predicted by the level of relationship strength of the friend pairs. We assessed recall by relationship strength by looking at the relative direction of the errors made (i.e. how much subjects overestimated or underestimated the strength of the connection). A linear mixed effects regression model (fixed effect = strength; random effects = subject, residual) revealed that relationship strength affected recall error compared to a null model (χ2(1) = 226.9, p < 2.2e-16). This pattern shows that overall, weak ties were reported to be stronger than they actually were whereas strong ties were reported to be less strong (Figure 2). This reflects a general tendency to assume a mid-level relationship between observed people when the relationship is not explicitly known or is unable to be recalled. This central tendency effect seems to be robust, as it was also observed in a separate subject sample (N = 23) with a larger learned social network and a larger possible range of relationship strengths to choose from (0-6) (χ2(1) = 362.84, p < 2.2e-16; see Supplementary Methods and Supplementary Figure 1). In order to be able to compare recall errors to the neural patterns in response to each face in the network, we converted the relative error for each subject to absolute error, which gives a measure of distance from the true network structure, regardless of the direction of that error. The absolute error measure for each subject for each network was then used as a dissimilarity model for RSA to elucidate what neural patterns underlie these errors.
fMRI Results
During fMRI scanning, participants viewed the original faces from the social network behavioral session, as well as 12 novel faces and were asked to press a button when they saw a face repeated to guarantee attention. We first conducted a GLM comparing the 12 familiar faces from the two networks to unfamiliar control faces. Supplementary Figure 2 shows that an area of the left fusiform gyrus was more active when viewing unfamiliar faces, whereas the posterior cingulate gyrus/precuneus was more active when viewing familiar faces (p < 0.05, FWE-corrected with threshold-free cluster enhancement within an anatomical mask composed of areas previously shown to be relevant for face perception and memory; see Supplementary Table 1 for cluster information). While perception for different categories of faces is highly dependent on task demands, our findings are consistent with some previous literature examining recognition of familiar faces (Natu and O’Toole, 2011). The fusiform gyrus has been shown to activate significantly less to famous faces than to strangers in the left hemisphere (Gobbini, Leibenluft, Santiago, and Haxby, 2004), and the posterior cingulate/precuneus area is consistently activated more to personally familiar faces when compared to strangers (Gobbini et al., 2004; Pierce et al., 2004; Sugiura et al., 2001). There were no univariate differences between responses to faces across the two networks.
Representational Similarity Analysis
To examine whether information related to social network recall is represented in the brain, we carried out RSA searchlight analysis using accuracy as measured by each subject’s absolute distance from each true network structure and the 1-correlation distance between the neural response to each face viewed in the scanner. An association between these two measures would indicate that the more accurately a subject perceives the true relationship tie strength between a pair of faces, the more similar their neural pattern response is to those two faces. Neural pattern similarity in the left TPJ, the left fusiform gyrus, the subcallosal cingulate cortex, the cerebellum, the left thalamus, and a small portion of the left lateral occipital lobe was significantly correlated with the recall accuracy model, suggesting that neural populations within these areas are important for accurate perception of social relationship strength (Figure 2). Table 2 reports MNI coordinates, cluster size, and peak voxel activity of results.
We also conducted RSA searchlights using three other dissimilarity matrix models: true structure, recalled structure as measured by behavioral judgments, and the group average of those behavioral judgments (Figure 3). Neural pattern similarity that reflects network structure would indicate that the brain carries information about the true relationship between individuals, regardless of whether people recall those relationships accurately. Neural pattern similarity that reflects behavioral recall would indicate that the brain carries information about an individual’s perception of relationships, regardless of how accurate those perceptions are. This perceived structure at the group average level can show general trends in how relationships are viewed by groups. However, no other models reached significance in the whole-brain searchlight analysis. Finally, we conducted RSA correlation calculations within an anatomical mask created from previously defined areas involved in social cognition, as well as from functionally defined regions of interest. These results are described in the Supplementary Information.
Discussion
In this study, we assessed the degree to which participants learned friend connections and relationship strength within a social network through observation of pairs, and examined whether their neural representational space reflects patterns of recall accuracy, assessed by the distance between true individual social network relationships and the subject’s behavioral judgments of those relationships. We found a central tendency effect in that when mistakes were made, they were more likely in overestimating weak ties and underestimating strong ties. We also found that a number of brain areas, including the TPJ, subcallosal vmPFC, fusiform gyrus, cerebellum, and thalamus underlie patterns of accurate recall.
The TPJ and vmPFC in particular have been consistently linked with complex facets of social knowledge. The TPJ is involved in social recognition and the formation and strengthening of connections (Bault, et al., 2015; Zink et al., 2011). The vmPFC distinguishes friends from family and tracks levels of group cohesiveness (Wlodarski and Dunbar, 2016; Rüsch et al., 2014). Our findings are in line with previous literature showing the importance of these areas in forming and maintaining social relationships. They further indicate that these areas are not only important in the knowledge and understanding of other individuals, but they also support learning relations between individuals in groups. The fusiform gyrus is also heavily involved in social perception, particularly in response to face stimuli (Kanwisher, McDermott, and Chun, 1997). While early models of face perception suggested a strict feed-forward mechanism for distinguishing, identifying, and gaining socially-relevant information from faces, recent proposals indicate a more interactive process between different neural regions when engaging in higher-order social face perception (Atkinson and Adolphs, 2011). Our data indicates that patches of the fusiform gyrus do not simply perceive and distinguish facial features (from each other as well as non-face stimuli), but are also involved in learning more socially abstract relationships between faces.
A large meta-analysis of fMRI studies has revealed that areas of the cerebellum are activated in several features of social cognition, with increases in activity occurring with increasing social abstraction levels in the cognitive tasks (Overwalle, Baetens, Marien, and Vandekerckhove, 2014). The authors suggest this cerebellar activity is due to a general increase in cognitive task demands, in line with the theory of the cerebellum as a cognitive process modulator (Andreasen and Pierson, 2008). Our finding that the cerebellum is involved in accurate knowledge of abstract learned relationships between others is consistent with this. Furthermore, we found that the thalamus is also involved in this process. The thalamus has a large number of connections to other areas of the brain, and has been shown to have specific emotional and socially-relevant associations (Christoffel et al., 2015; Feng et al., 2016; Ioannidis et al., 2013). It also has high functional connectivity to the hippocampus (Stein et al., 2000), and may be a critical link in the formation of episodic memories, regardless of the sociality of those memories (Aggleton et al., 2010).
While most of our subjects did pass the learning threshold to continue on to the fMRI portion of the study, several did not learn the network relationships well enough. And even within those who accurately reported ties, there were individual differences between ability to recall relationship strength (measured by the correlation between the true structure and the reported structure of the networks). Further experiments could explore this type of task explicitly, as prior social network learning studies informed participants that they would be tested on connections (Brashears, 2013; De Soto, 1960). The lack of significant findings in the three other cognitive models within the whole brain analysis (true structure, individual recalled structure, and group average recall) could also be due to participants not learning the connections to a high enough degree. It is possible that since the task was difficult and there were individual differences in performance, the true structure hadn’t been embedded enough in every subject’s memory to yield sufficient power. Increased time spent viewing or more explicit learning of social network connections could further examine this question.
The way in which people learn and remember social connections between individuals in groups has a huge impact on everyday life. Human beings are not only able to perceive and understand the social signals of other individuals, but we can also perceive and understand information about connections or relationships between ourselves and others, as well as social relationships with which we are not involved. Our results show that this is in large part driven by areas of the brain previously shown to be involved in other forms of social cognition, so it is possible that it is a uniquely social ability. The accuracy with which we perceive and remember subtle connections and relationships seen in our surroundings helps us move more freely and easily in our highly social world.
Methods
Participants
22 healthy individuals (10 females; age range = 18-34; mean age = 23; ethnicity = 64% White, 18% Hispanic/Latino, 18% Asian) participated in a 1.5 hour learning session immediately followed by a 1.5 hour fMRI scanning session. Behavioral data from a total of 31 individuals was collected, but seven subjects did not meet the learning criteria from the behavioral task, one subject was unable to be scanned, and one subject’s fMRI data was incomplete. All participants were right handed (self-reported) with normal or corrected-to-normal vision. Participants provided written informed consent in accordance with the Declaration of Helsinki and the Human Subjects Review Board at George Mason University and were compensated for their time.
Stimuli and Task
Stimuli
Task stimuli consisted of 24 faces of varying ethnicities, equally divided by gender. Faces were all in color and facial expressions were all smiling. These stimuli were downloaded from the Park Aging Mind Laboratory Database at UT Dallas (Minear & Park, 2004) and were chosen to be as realistic to a college campus as possible, ensuring the perception of real people who might interact and be friends with each other.
Task
Participants completed a two-alternative forced choice task to become familiar with the structure of two six-person social networks (See Figure 1). Pairs of faces represented connections within each network, with the frequency of pairing indicating relationship strength. Each network had an equal number of male and female faces of varying ethnicities. Each trial consisted of a face pair presented for 4 seconds accompanied by a question, and participants were asked to make a comparison between the faces and decide which person better fit the question. Questions consisted of behavioral and personality characteristics taken from various personality surveys included in the International Personality Item Pool (http://ipip.ori.org/). Half of the questions asked which person was more likely to exhibit a characteristic, and half asked which person was less likely (example: “Who is more likely to be easily intimidated?”). Participants completed 720 trials in total (360 per network), with the weakest network connections being presented a total of 20 times and the strongest a total of 80 times.
After completing the paired face viewing portion, participants were explicitly tested on their knowledge of the network connections. They were told that the faces represented college students living in a dorm together, the faces that they saw paired together previously represented friend connections, and the more often they were presented together, the closer in friendship the pair was. They were then presented with all possible within-network face pairs twice and asked to rate their relationship on a scale of 0 (do not know each other) to 4 (know each other very well). This explicit testing period was included to ensure that participants learned the structure of the networks to an appropriate level before being scanned. Participants who were within 2 standard deviations of pilot data (hit rate = 0.85, SD = 0.14; false alarm rate = 0.35, SD = 0.15) were included in further analysis. Both parts of the behavioral task (learning and recall) were presented to the participant using PsychoPy version 1.842 software (http://www.psychopy.org/).
The fMRI task stimuli included the same 12 faces from the behavioral task as well as 12 novel faces as a control. Faces were presented one at a time for one second on a black background with a 4.5 second inter-stimulus interval (black screen with a white fixation dot), and participants completed a 1-back task to ensure they were attentive. The task consisted of four runs of 9.6 minutes each, resulting in each face being presented a total of 16 times (not counting repeats, which were included in analysis as a separate regressor). Following the face task, participants underwent an unrelated dynamic localizer session. Localizer stimuli consisted of 18 second blocks each of faces, body parts, outdoor scenes, moving objects, and scrambled objects. The fMRI experiment was presented to the participant using Neurobehavioral Systems Presentation version 16.3 (https://www.neurobs.com).
fMRI data acquisition, preprocessing, and analysis
fMRI data were collected with a Siemens Allegra 3T scanner and a standard, one channel quadrature birdcage head coil at the Department of Psychology at George Mason University. Visual stimuli were displayed on a rear projection screen and viewed by participants on a head coil-mounted mirror. Blood oxygenation level dependent (BOLD) data were acquired using gradient-echo, echoplanar imaging scans: 40 axial slices (3-mm slice thickness), repetition time (TR)/echo time (TE) = 2350/30 ms, flip angle = 70, 64 x 64 matrix, field of view = 192 mm. 245 volumes were collected in each run. At the end of the fMRI scanning session, one T1 whole-head anatomical structural scan was acquired using a three-dimensional, magnetization-prepared, rapid-acquisition gradient echo (MPRAGE) pulse sequence. The following parameters were used for these scans: 160 1-mm slices, 256 x 256 matrix, field of view = 260 mm, TR/TE = 2300/3.37 ms. Functional data were analyzed using FSL (version 5.0.8) fMRI Expert Analysis Tool (fsl.fmrib.ox.ac.uk) and Matlab (version R2012a) software (http://www.mathworks.com). Preprocessing included brain extraction, high-pass filtering at 96 s, slice-timing correction, motion correction, and smoothing with a 6 mm FWHM kernel. Runs with > 1 mm of motion were run through the BrainWavelet Despiking program in Matlab (Patel et al., 2014). For first-level analysis, linear regression was conducted at each voxel, using generalized least squares with a voxel-wise, temporally and spatially regularized autocorrelation model, drift fit with Gaussian-weighted running line smoother. For second-level analysis, linear regression was conducted at each voxel, using ordinary least squares.
Regions of Interest (ROI) and Mask Creation
Localizer data preprocessing steps were identical except the functional data was registered only to each subject’s specific structural image. Face-selective regions of interest (ROIs) were created from subtracting the combined object, scrambled object, and scene conditions from the face condition. These regions included bilateral posterior STS and fusiform face area (FFA). Activity was thresholded at Z > 3.7 (p < 0.0001) for most ROIs, although this threshold was relaxed to Z > 3 (p < 0.001) in one subject, Z > 2.3 (p < 0.01) in four subjects, and Z > 1.65 (p < 0.05) in three subjects because of lower overall BOLD activity. These masks were projected back into native functional space for further analysis. Finally, two anatomical masks of areas involved in social processing were created from the Harvard-Oxford Cortical Structural Atlas in FSL: A mask of brain areas implicated in memory for faces (encompassing the bilateral pSTS, extrastriate body area (EBA), ventral temporal/fusiform gyrus, precuneus/posterior cingulate cortex (PC/PCC), and hippocampus), and a mask of brain areas involved in social cognition (encompassing the bilateral pSTS, TPJ, amygdala, and mPFC extending dorsally to the anterior cingulate cortex (ACC) and ventrally to the subcallosal cortex).
Univariate Analysis
Each subject’s functional data was registered to his or her anatomical scan and then registered to the MNI standard template. The regressors used in the generalized linear modeling (GLM) analysis were Network 1 v. rest, Network 2 v. rest, Control v. rest, and Response Trials v. rest. Contrasts used were Network 1 v. Control, Network 2 v. Control, Both Networks v. Control, Control v. Both Networks, Network 1 v. Network 2, and Network 2 v. Network 1. Group nonparametric 1-sample (conditions v. rest) and 2-sample (condition A v. condition B) t-tests (5000 permutations) including threshold-free cluster enhancement and variance smoothing of 8 mm were conducted with fslrandomise within a mask created from anatomically-defined regions selective for face processing and memory.
Representational Similarity Analysis
Representational similarity analysis (RSA) is a form of multivariate pattern analysis that compares the distance between stimuli in neural representational space (Kriegeskorte, 2008), and correlates this neural information with external patterns of information. In this way it can be utilized to assess different models or patterns of cognition above and beyond univariate analysis, or even more traditional multivariate pattern classification techniques (Haxby, Connolly, and Guntupalli, 2014). For initial analysis of task data, no registration to structural or functional data was carried out, and the smoothing kernel used was 4 mm FWHM. All other preprocessing parameters mirrored the univariate whole-brain analysis. The GLM included separate regressors for each of the 24 faces and repeats. Resulting z-statistics were grouped by network for further analysis. Four separate dissimilarity matrices (DMs) were created for each network (for examples, see Figure 3): true network structure, perception of network structure (taken from each subject’s behavioral recall data after learning the networks), group average of perceived structure (where each face pair’s perceived strength was averaged across subjects), and recall accuracy (measured by calculating the absolute distance between the true strength of each face pair and the average strength of the pair reported in the recall phase). The CoSMoMVPA toolbox in Matlab was used for RSA calculations (Oosterhof, Connolly, and Haxby, 2016).
Whole-Brain Analysis
Separate whole-brain searchlights using Spearman correlations (size = 50 voxels) were conducted on the average z-statistics for the faces within each network for each DM, and the ensuing correlation maps were averaged across networks within subjects and then transformed to standard space for group analysis. Group nonparametric 1-sample t-tests (5000 permutations) including threshold-free cluster enhancement and variance smoothing of 8 mm were conducted with fslrandomise. Resulting t-statistic maps were visualized in the MNI volume as well as transformed to the PALS-B12 standard atlas in Caret (http://www.nitrc.org/projects/caret/) for surface data visualization (Van Essen, 2005).
Regions of Interest
RSA was carried out within each localizer-defined ROI and the resulting correlations within each region were averaged across subjects. RSA searchlights using the same parameters and same DMs were also conducted within the anatomically-defined social brain mask (transformed to each subject’s individual native space), and the ensuing correlation maps were averaged across networks within subjects and then transformed to standard space for group analysis. Group nonparametric 1-sample t-tests (5000 permutations) including threshold-free cluster enhancement and variance smoothing of 8 mm were conducted with fslrandomise.
Author Contributions
S. L. Dziura designed and conducted the experiment, analyzed the data, and wrote the paper. J.C. Thompson provided the initial experiment idea, advised on data analysis, and provided critical paper edits.
Competing Interests
Neither author has any financial or non-financial competing interests.