Abstract
Introduction: It is unclear if sexual orientation is a biological trait that has neurofunctional footprints. With deep learning, the power to classify biological datasets without an a priori selection of features has increased by magnitudes. The aim of this study was to correctly classify resting-state electroencephalogram (EEG) data from males with different sexual orientation using deep learning and to explore techniques to identify the learned distinguishing features. Methods: Three cohorts (homosexual men, heterosexual men, and a mixed sex cohort), one pretrained network on sex classification, and one newly trained network for sexual orientation classification were used to classify sex. Further, Grad-CAM methodology and source localization were used to identify the spatiotemporal patterns that were used for differentiation by the networks. Results: Using a pretrained network for classification of males and females, no differences existed between classification of homosexual and heterosexual males. The newly trained network was able, however, to correctly classify the cohorts with a total accuracy of 83%. The retrograde activation using Grad-CAM technology yielded distinctive functional EEG patterns in the Brodmann area 40 and 1 when combined with Fourier analysis and a source localization. Discussion: This study shows that electrophysiological trait markers of male sexual orientation can be identified using deep learning. These patterns are different from the differentiating signatures of males and females in a resting-state EEG.
Introduction
Sexual orientation is an important personal characteristic of every person. The individual orientation can be placed on a continuum between homo- and heterosexual alignment rather than applying a strict dichotomous division [1]. Throughout history, persons with sexual preferences for the same sex and thus a differing orientation from the majority have been facing persecution and violence, also in the Christian-coined western world [2‒4]. The American Psychiatric Association only removed homosexuality from the category of psychiatric diseases in 1973 [5]. However, people always tried to understand the reason for these differences in sexual orientation. One of the main questions thereby has been if either homosexuality is a hereditary trait or environmental factors influence the development of these preferences [1]. A main motivation for research in this area has been the search for notorious preventive measures for a long time [6]. Nowadays, in the western world, the freedom of choice including the sexual preferences has evolved as the widely accepted social agreement, although it is still a long way from providing safety and participation for every person, regardless of their sexual orientation.
Having reached this cultural developmental stage, it still is of high scientific interest, whether there exist biological patterns that differ between persons with different sexual orientations. The authors are aware of the ethical questions arising from this research, as outlined above. The approvement of biological traits in alignment with sexual orientation still could help further withdraw the basis of involuntary behavioral measures against persons with nonheterosexual orientation [7]. The fraternity hypothesis stated that the existence of older male brothers increased the probability of being gay in men, thereby attributing a significant impact to environmental factors on sexual orientation [8]. Others found genetic [9, 10], epigenetic [11], brain structure [12‒14], or immunological [15] differences in subjects with same sex sexual orientation, supplying evidence for a biological trait. Especially findings from large-scale studies using data from over 400.000 subjects from the UK Biobank and the 23andMe database revealed that there are genetic predispositions for same-sex sexual behavior [10]. However, the main outcome was that no genetic variant allowed for a prediction of the individual sexual orientation. The authors underline that their findings shed light on the complexity of sexual behavior because of still largely unknown interactions between genetics and environment.
In the light of structural differences between individuals with different variants of sexual orientation, it seems paramount to also search for possible functional differences. Electrophysiology, and electroencephalogram (EEG) in particular, provides a highly suitable framework for a noninvasive analysis of brain function for a large variety of conditions. In combination with advanced technologies such as deep learning, EEG data recently have been used to differentiate between males and females based on their brain function patterns during rest [16]. Although EEG research has been misused to identify pathological EEG patterns in males with homosexual orientation [17, 18], more recent research was more focused on mainly functional, non-pathological differences in groups of homosexual and heterosexual males (HeMs) [19]. When EEG data were used in these studies to differentiate between males and females [20] or groups of homosexual and HeMs [21], the data were always recorded during cognitive task performance (e.g., mental rotation). Results could therefore be related to sex-dimorphic cognitive task properties and less to functional differences by means of basic central nervous system activity during rest.
Within recent years, new techniques derived from the field of artificial intelligence research emerged that allow for powerful analysis of medical and electrophysiological data [22]. Especially so-called deep learning algorithms proved their superiority in the classification of time series data in comparison to conventional statistical analysis, given that enough labeled data are available [23]. These deep learning techniques stem from the functional principle of natural neuronal networks as can be found in the human brain [24]. By changing the connection strength between different nodes, i.e., neurons during the training of the network, the labeled input data will be processed to result in the desired output, i.e., the classification label. The network then can be tested with previously unseen testing data. Since it is appealing to classify brain-derived functional time series using a technique that follows the function of neuronal networks, in this study, we aimed to test whether there are differences in the electrophysiological patterns during rest between homosexual and heterosexual men using these deep learning. Assuming that cues of biological sex are detected and processed in similar ways within individuals who are interested in the same sex of potential partners [25], we assumed a resemblance in the resting state activity between homosexual men (HoM) and heterosexual women. Some research of the past showed similar functional activations in homosexual males and heterosexual females [12]. Consequently, we at first tested if (1) HoM might exhibit similar EEG patterns like heterosexual females in comparison to heterosexual men. That means HoM could be classified via a network that was trained on sex and not sexual orientation, receiving the label “female.” If this quite mechanistical hypothesis would not hold true, i.e., homosexual males would not exhibit similar EEG patterns as heterosexual females, it was further hypothesized that (2) HoM could be classified by applying a new network trained on HoM and heterosexual men, i.e., being classified via a network trained on sexual orientation and not sex. For testing (1), a previously trained male/female classification deep learning network (“SexNet”) [16] was applied to datasets of HoM, heterosexual men, and a third group of mixed males and females. For testing (2), a new network was trained to differentiate sexual orientation by using data from HoM or heterosexual men. To further elucidate the functional properties and anatomical distribution of potential EEG patterns for differentiation between homosexual and heterosexual men at the individual level, a gradient CAM (Grad-CAM) approach in combination with a source localization technique and Fourier transformation was used.
Materials and Methods
The study was approved by the Ethical Committee Zurich (EC-No 2014-0623). All participants gave written informed consent. The study was conducted following the rules established by the Declaration of Helsinki.
Subjects Sample 1 and Sample 2: EEG Recording
Sample 1 consisted of 39 HeMs and 38 HeMs recruited from 2017 to 2019 in Zurich and was reported in another study [26]. Ten additional subjects had been recruited but needed to be excluded, four due to regular intake of medication. Two more subjects showed a sexual orientation that showed no clear preference for on sex (bisexuality), based on a Klein Sexual Orientation Grid (KSOG) Score between 3and 4.9. Four subjects had to be excluded due to missing or bad EEG data.
Sample 2 consisted of 37 healthy controls with mixed sex (female = 16) and unknown sexual orientation, recorded with the same equipment under the same conditions as sample 1. Sample 2 was recorded in year 2010 as control subgroup for the ZINEP study [27].
Subjects Sample 3: EEG Recording and SexNet Training
In total, 1,300 EEG datasets were used for the SexNet training and validation. They were recorded and preprocessed using a standardized methodology and platform (Brain Resource Ltd., Australia) for which full details have been published elsewhere [28] and which is in large parts identical for sample 1 and 2. The used SexNet has previously been reported to differentiate men and women based on 80 s resting-state EEG recordings with an accuracy of 84% [16], which is a highly significant finding, given a random guess distribution of 50%. To avoid overestimation for transfer learning, we did not apply the top performance net from [16] but relied on an average model with a performance accuracy of 73%. Therefore, SexNet was trained on 24 channel EEG segments with a 128 Hz resolution from 1000 adults and tested on unseen 300 EEG datasets. The architecture of SexNet consisted of 9 layers with 256 × 24 as input matrix, convolutional layers with decreasing number of filters (300–50) and pooling function after each of the first four convolutional layers, and a dropout function of 25% after each convolutional layer. A rectifier linear unit served for activation. A final dense layer with softmax activation was used for classification probabilities. The rest of the training procedure was done as described in [16].
EEG Processing and SexOrientationNet
The EEGs were recorded at 2.5 kHz (BrainAmp, BrainProducts, Germany) during a 5-min resting state with eyes closed using a 32-electrode BrainCap (MR 32 standard, Easycap) referenced to FCz following the international 10 to 20 system. Impedances were kept below 20 kOhm. Eye movements were recorded using bipolar diagonal electrooculogram channels. Software analysis was done using Brain Vision Analyzer 2.0, Gilching, Germany: raw data were downsampled to 128 Hz and band pass filtered between 0.5 and 25 Hz plus notch filter. To compare the results with the outcome of the SexNet, 24 channels were kept for further analysis as follows: Fp1, Fp2, F7, F3, Fz, F4, F8, FC3, FCz, FC4, T3, C3, Cz, C4, T4, CP3, CPz, CP4, T5, P3, Pz, P4, T6, O1. EEG data were eye – artefact corrected using a regression-based approach Gratton, Coles, and Donchin [29]. Artefact rejection was done using an automated pipeline following different voltage and power criteria, detailed in [28]. Finally, the EEG was inspected visually by an experienced EEG-rater, and artefacts were marked and rejected for consecutive 2 s segments (example Fig. 1). Raw data of 4-min recordings were imported to Matlab software (R2019b), where a three-dimensional matrix of channels (24) × segment-length (2 sec with 2 × 128 = 256 timepoints) × epochs (120 with 2 sec duration) were stacked for each subject. Labeling of the subjects (homosexual man or heterosexual man in sample 1 and additionally heterosexual woman in sample 2 and sample 3) was done using one-hot coding. The matrix then was exported to the python environment.
The SexOrientationNet was deployed in a python (3.8) TensorFlow (version 1.2 GPU version) environment with a Keras backend (Version 2.2). The structure of the used network was identical to the structure of the SexNet, only the number of filters was between 100 and 300. However, training was not performed on sample 3 but on sample 1 (homosexual and heterosexual men) with a Leave One Out validation process (see below Sex Orientation network validation)
Assessment of Sexual Orientation
The KSOG [30] was used to check for self-reported sexual preference. This scale uses seven items on sexual orientation (sexual attraction, sexual behavior, sexual fantasies, emotional preference, social preference, self-identification, and heterosexual/gay lifestyle outcomes) that are addressed with responses on three dimensions (past, present, and future). Responses are given on a 7-point Likert scale ranging from 1 (other sex only/heterosexual only) to 7 (same sex only/gay only). Cronbach’s alpha of 0.96 and significant correlation with self-reported sexual orientation (r = 0.71) have been reported for the KSOG [31]. Average scores across the present time dimension were calculated. Values of 2.9 or less refer to heterosexual orientation; values of 5.0 or above refer to a homosexual orientation [32].
Deep Learning SexNet and SexOrientationNet Validation and Feature Extraction
Validation for the usage of the SexNet was done using separate training and test sets. Accuracy measures were used for assessing the quality of the classification in the test set.
Validation for the usage of the newly trained SexOrientationNet on homosexual and heterosexual men was done using the “Leave One Out Cross Validation.” This common practice splits data into k independent folds and subsequently trains in k−1 folds and tests in the complementary fold until the model has been trained and tested in every fold (i.e., in the complete dataset). This cross-validation is seen as the method of choice when it comes to deep learning with a limited amount of data and a separate training and test sets are not feasible [33].
Grad-CAM Feature Extraction
For feature extraction, the class activation maps allow to seek for distinctive features in the input data for deep learning based networks. The class activation maps show weighted heatmaps of the input data that allow for the identification of the most important data points for classification tasks, e.g., in medical imaging [34]. Their successor, Grad-CAM allows for the same approach without the necessity of global average pooling before the softmax layer by using gradient information that flows into the last convolutional layer of the network. Grad-CAM also outperforms other heatmap algorithms in medical imaging [35]. To apply Grad-CAM method, a rectified linear unit is used to focus on the decisive features for a certain class. This method has been used to detect EEG channels with highest information load in brain-computer-interface scenarios [36]. Following this procedure, data from the subject to classify were fed into the network, and from each layer of the network, the decisive weights were exported and averaged over all layers. This weight map then was projected on the input data, i.e., the EEG segments. Then, these 2-D maps were averaged over the time axis, yielding the 1-D activation array in space, i.e., electrode positions. This process was repeated for all subjects. The output Grad-CAM maps were then averaged for all homosexual and all heterosexual subjects. Feeding these arrays into a source localization algorithm such as the low-resolution electromagnetic tomography (eLORETA), it was possible to map the intracortical sources of electrophysiological activity that allowed the correct classification of a group.
EEG-Source Localization
Source localization analysis was done using eLORETA software package, version 20221229 (Roberto Pascual-Marqui/The KEY Institute for Brain-Mind Research, Zurich). The eLORETA algorithm is an inverse solution for EEG signals that has no localization error. In contrast to other source localization techniques, this is also applicable in the presence of measurement and structured biological noise [37]. The family of LORETA algorithms allows imaging of current density distributions inside the human cortex via constraining the solution to only gray matter-dense regions within 6,239 predefined 3D-voxels [38]. EEG assessment in conjunction with simultaneous recordings in other neuroimaging modalities cross-validated the family of LORETA algorithm and showed overlapping results of intracortical EEG-source estimates and the BOLD signal [39, 40] or glucose metabolic rate [41, 42]. To generate the input for the source localization, the projections of the network were trained on k−1 datasets and tested on the remaining dataset repeatedly, until all subjects of one group (hetero- or homosexual) were extracted from all layers for all EEG channels and averaged over all subjects of the group. The electrode activation matrix for the LORETA analysis was thus fed with the weights of the network, which allowed a correct classification. The LORETA transformation matrix was computed from the EEG channel positions used in the EEG recordings and the electrode activation matrix was projected into the intracortical solution space of the eLORETA software.
Frequency Analysis of the LORETA Regions
To determine which frequencies of the EEG input data were used for classification by the SexualOrientationNet, the output of all filters (ranging from 100–300 per layer) from all convolutional layers (n = 6) of the network were computed while using a loss function that maximized the activation of the filters. For each layer, the filter with the largest activation was projected into a 24 × 256 array, representing the EEG input with the highest potential of layer activation. A Fourier transformation was applied for the x-axis (i.e., EEG channel time series) to analyze the prevailing frequencies in the corresponding layers.
Estimating Statistical Threshold
For assessment of statistical significance of the SexualOrientationNet, we randomly assigned sexual orientation to each subject in a test set (n = 20), using the prior sexual orientation distribution (50% homosexual males). To set the p value for statistical significance at p ≤ 0.01, we performed 100 simulations in Matlab, following the principles of Monte Carlo simulation. The best classification accuracy reached was 70%, which was subsequently considered the significance threshold for p ≤ 0.01.
Comparison with fMRI Findings
To compare the sources of differentiating EEG activity with findings from neuroimaging studies using fMRI, the “Neurosynth” website (“neurosynth.org”) was used [43]. In total, 81 studies using fMRI and the context “sexual” were included into the sample. Works with the highest load in the analysis included studies on functional endophenotypes for sexual orientation [44], neural substrates of sexual desire [45], and correlates of gender differences [46]. The results of the uniformity analysis (z-scores from a one-way ANOVA testing) were corrected for multiple comparisons using the false discovery rate with a 0.01 criterion.
Results
Sociodemography
For sample 1 (n = 77), mean age of homosexual males (n = 38) was 24.7 years with SD 5.0, HeMs (n = 38) with 24.6 years and SD 4.9. Mean KSOG score for homosexual males was 1.9 with SD 0.40, for HeMs 5.7 SD 0.39. For sample 2 (n = 37 with 16 females), mean age was 28.6 years (28.4 years for females only) with SD 5.9. For sample 3 (n = 1,300), the mean age was 43.4 with 18.4 SD with 47% males [16]. Different personal variables and performances, including handedness, sexual desire, sexual excitation, and the attentional network task [47], were tested for differences between groups (homosexual males vs. HeMs) with no significant alterations between the groups (see [26] for details).
Classification of Homo- or Heterosexual Orientation Using a Pretrained SexNet
The SexNet trained on 1,000 and tested on 300 EEGs from women and men to differentiate sex (accuracy of 73%) was applied to three different groups (two groups from sample 1 with HoM and HeM controls) and data from sample 2 (mixed male female [MixMF]). In all datasets, HoM, HeM, and MixMF, males irrespective of their sexual orientation were classified (binary classification with a probability of being male >50% for the majority of all 120 segments per subject) as male with an accuracy of 89.2–97.5% with no significant differences between the three datasets (p > 0.05). Only when numeric probabilities of subject classification were tested for differences, the HoM dataset showed a significant higher probability for correctly classified sex in comparison to the correct classification of the MixMF group (p = 0.00, see Table 1).
. | HoM . | Heterosexual men . | Mixed sex . | Ho versus He . | Ho versus mix . | He versus mix . |
---|---|---|---|---|---|---|
Correct classification (binary) | 97.5% (39/40) | 92.3% (36/39) | 89.2% (33/37) | p = 0.30 | p = 0.13 | p = 0.64 |
Probability of correct classification | 88.8% (SD 14.2%) | 83.5% (SD 18.5%) | 75.8% (SD 21.6%) | p = 0.16 | p = 0.00* | p = 0.10 |
. | HoM . | Heterosexual men . | Mixed sex . | Ho versus He . | Ho versus mix . | He versus mix . |
---|---|---|---|---|---|---|
Correct classification (binary) | 97.5% (39/40) | 92.3% (36/39) | 89.2% (33/37) | p = 0.30 | p = 0.13 | p = 0.64 |
Probability of correct classification | 88.8% (SD 14.2%) | 83.5% (SD 18.5%) | 75.8% (SD 21.6%) | p = 0.16 | p = 0.00* | p = 0.10 |
Significant differences were only found when considering the general probability of a subject being classified correctly: homosexual males were classified correctly with higher probabilities than a mixed male/female cohort.
Classification of Sexual Orientation Using a New Trained Deep Net
To further test whether male subjects with different sexual orientation (homosexual or heterosexual) could be correctly classified using deep learning, a new network (SexOrientationNet) was trained on data from sample 1. The used Leave One Out training and testing procedure of the network for classification of homosexual or heterosexual men showed a performance of a 75.95% accuracy for the correct classification of HoM or heterosexual men for all segments of the dataset (Fig. 1). Following the majority vote on the 120 epochs of each subject when trained on the rest of the subjects, 26 out of 39 heterosexual men were classified correctly and 38 out of 38 HoM. A four-fold table of the results can be found in Figure 2 (bottom). The sensitivity to correctly classify a homosexual man as homosexual was 100%; the specificity was 67% with a positive predictive value of 76% and a negative predictive value of 100%, yielding a total accuracy of 83%. Taking into account that an accuracy of 70% is equivalent to a statistical threshold of p < 0.01 according to the Monte Carlo simulation, this result can be seen as highly significant. The number of epochs of learning for each trial (with one subject left out and an early stopping rule after four consecutive failed decreases of validation loss) ranged from 6 to 51. The averaged training and test accuracies and training and test losses for all segments (not only subjects) can be found in Figure 2.
Visualization and Localization of Differentiating Features at the Single EEG Segment Level
To analyze whether it is possible to extract the EEG patterns that showed discriminative features for sexual orientation classification in single subjects, a Grad-CAM approach was used. Therefore, the trained filters from all layers (ranging from 100–300 filters per layer) from a single leave-one-out run were averaged layer-wise, and the gradients of a single EEG epoch from the test subject were computed for each averaged layer filter (single layer averages can be found in Fig. 3, panel a). It is obvious that early layers focus on activity at 10 Hz at posterior sites (Fig. 3, panel a), while later layers have less differentiation due to the max-pooling steps in the algorithm. The gradient projections from all layer averages were summed up to give a visualization of the gradients on the raw EEG (Fig. 3, panel b). The averaged layer weights then were convoluted toward the EEG channel axis and the weights were used for the LORETA source reconstruction. The main focus of the differentiating features was found in the Brodmann area 7, parietal lobe at x = −3, y = −53, z = 57) for the shown activity (Fig. 3, panel c and d).
Visualization of Differentiating Features at the Group Level
As a next step, we trained 77 networks following the LOO approach and calculated an averaged Grad-CAM map for the remaining subject. Then, the maps were averaged for the two groups (homo- and heterosexual men, Fig. 3, panel a). Following the eLORETA source localization approach, the region with the activity that allowed for best labeling HeMs was located at the postcentral gyrus at the parietal lobe at Brodmann area 1 (X = −45, Y = −30, Z = 65, MNI cords, Fig. 4, panel b top). The region with the largest discriminative activity for HoM was found at the inferior parietal lobule at Brodmann area 40 (X = 40, Y = −55, Z = 60, MNI cords, Fig. 4, panel b, bottom).
Identification of Frequency Properties for Classification at Group Level
For assessment of the frequencies that are associated with the identification of male sexual orientation, the filters of all layers of the SexOrientationNet were fed backward with patterns that activated each filter the most, thus generating archetypical inputs for the decisions of the networks. These patterns (Fig. 4, panel c) can be seen as EEG traces that evoke the classifications “heterosexual” or “homosexual.” Since these patterns have the same dimension as the input EEG segments (24 channels × 256 points in time), these time series were forwarded to Fourier transformation to reveal the involved frequencies. Analysis was restricted to the closest electrodes of the intracortical clusters (Fig. 4, panel b). This process was done separately for heterosexual and homosexual classification patterns and for all layers. While there were clear peak frequencies around 17 Hz and 48 Hz in early layers (Fig. 4, panel d, picture 1 and 3 for layer 2), later layers resampled more complex patterns that did seem to comprise multiple frequencies (Fig. 4, panel d, picture 2 and 4).
Comparison with fMRI Research
Comparing the sources of the discriminating EEG features identified by the SexOrientationNet with meta-analytical findings from 81 fMRI studies that used a sexuality context in their paradigms revealed an overlap mainly in the parietal cortices, i.e., at the superior parietal lobe (Brodmann x = −26, y = −60, z = 48). As can be seen in Figure 5, both groups, HoM and HeM, show source clusters with high filter weights in this area.
Discussion
This study aimed at the differentiation of homosexual and heterosexual orientation in men using neurophysiological resting state data. Further goal was to test feature extraction from the used networks to gain new insights into electrophysiological patterns that differ between homosexual and heterosexual men. While the sexual orientation in men could not be classified by a network previously trained on sex (SexNet), a newly trained network on sexual orientation (SexOrientationNet) was able to distinguish between males with homo- or heterosexual orientation with a very high accuracy of 83%. Further, the Grad-CAM method was able to extract electrode-wise gradients that allowed for an identification of distinguishing patterns at the temporal dimension, i.e., in the EEG traces and in the spatial dimension, i.e., the anatomical source space. Additionally, these sources could be linked to frequency analysis to reveal associated oscillations at the group level. These findings provide evidence that deep learning in combination with EEG data can successfully classify sexual orientation at the group level and extract non-a priori-defined features on the single subject/segment and group level that might help to understand underlying electrophysiological and anatomical circuits. The further extraction of source-bound frequencies that contributed to the network to learn revealed the importance of specific EEG oscillations for differentiation.
Using the pretrained SexNet, all different groups from sample 1 and sample 2 showed very high classification accuracy for sex. However, contrary to the hypothesized classification of homosexual males as more akin to females, following potential similarities of sexual stimulus processing between homosexual males and females in contrast to HeMs, there was no difference for classification of homosexual males and HeMs. Strikingly, both variants of sexual orientations were classified as males when using a network trained on gender in another cohort. This counteracts assumptions that homosexual males are “femalized” woman. Interestingly, when not looking at the binary classification (yes/no) but into the probabilities for correct classification, the homosexual group even had a significant higher probability to be rated correctly in comparison to a mixed sex group. It is further interesting that the overall accuracy of this trial was higher than the results that this pretrained network achieved when being tested on 300 subjects from sample 3. The reason for a higher accuracy of 89–97.5% in comparison to 73% could be found in the fact that longer EEG epochs with a subsequently increased segment number per subject were used. The sample 1 and sample 2 datasets consisted of a three-fold number of segments per subject (240 s vs. 80 s) in comparison to the SexNet training (sample 3). A simulation on the association between numbers of segments for each subject (when classifying each subject into a specific group with >50% of classified segments in a binary labeling approach) and the overall probability of one segment being classified correctly shows a sharp increase of correct classification of subjects (Fig. 6) with increasing segment numbers. Hence, future studies on subject classification using deep learning and multiple EEG segments per subject should consider restricting the size of each segment to just contain the essential information for increasing the overall number of segments. This also strongly implies that deep learning approaches benefit from longer EEG recordings since this allows for more data on each subject to be classified. The increase of training samples has been found to go in line with a logarithmical performance increase in vison tasks [48].
When a new trained net was used for classification instead (SexOrientationNet), the network was able to accurately differentiate between homosexual males and HeMs. It is noteworthy that all homosexual subjects were classified correctly but only 2/3rd (26 out of 39) from the HeMs. Although this result must be taken with caution, since no replication has yet been done, it might imply that most homosexual males show distinct features (or miss some features), while HeMs sometimes share these features (or miss them, too) without being homosexual. However, only the heterosexual group reveals (or misses) a feature in some of them (the correctly classified) that is not there (or present) in the homosexual group. Thus, the SexOrientationNet revealed a very high specificity for classifying subjects as HeMs and a very high sensitivity for homosexual orientation.
The second aim of this work was the identification features that allowed discrimination between homosexual and HeMs without a definition of an a priori feature space. The extraction of features from the network by using the Grad-CAM approach and backward filter activation from discriminative tasks using deep learning allows the identification of nonlinear features from the raw data [49]. Following this path, the presented work was able to show that the shape of the parietal and occipital alpha waves was focused by the SexOrientationNet to label single subjects according to their sexual orientation. This is in line with previous EEG findings. One study found that heterosexual men rated the erotic video with higher general and sexual arousal than the homosexual participants. During observation of the neutral and erotic videos, both groups showed decreased amplitude of the alpha band in prefrontal and parietal cortices, indicating increased attention [50]. Another study revealed less suppression of the mu-rhythm, a specific central rhythm in the frequency range of alpha, in HoM as reaction to painful actions in comparison to heterosexual men [51]. Interestingly, Alexander and Sufka found increased alpha activity in homosexual males over right parietal cortices during cognitive tasks [19]. Ziogas et al. [52] related hemispheric alpha activity further to sexual arousal.
However, at the group level, the extracted frequencies showed peaks in the beta and gamma range, not within the alpha band. Again, EEG beta activity has been associated with processing of erotic content in homosexual males [50]. Further, at both, the single subject level and at the group level, the areas with the most important weights for differentiation by the network were in the parietal cortex. It is of special interest that the right Brodmann area 40 was source of EEG activity that yielded highest discriminative power for homosexual males and Brodmann area 1 had the highest load for HeMs. Parietal regions have been described in the context of sexual arousal in fMRI studies [53], while Brodmann area 1 is part of the somatosensory cortex and thus highly related to social perception [54] and sexual arousal [55]. The meta-analysis of 81 fMRI studies using paradigms with sexual content revealed a more likely activation of parietal regions in studies with sexual in comparison when no sexual content was present. Here, it must be mentioned that fMRI activation maps have to be regarded with caution since they cannot be interpreted as simple maps of areas with higher or lower neuronal activity, given the nature of the underlying blood oxygenation dependent signal [56, 57]. However, the involvement of the parietal and temporal cortex in activity generation in both homosexual and HeMs during sexual exposure [58] has been linked to arousal and attention networks [59]. The findings of this study underline the presence of specific functional patterns at defined cortical areas that are different for homosexual and HeMs.
Although this study clearly shows the potentials of deep learning-guided EEG analysis for discriminative tasks, it still bears some limitations. The first limitation can be found in the relatively small number of homosexual subjects and heterosexual controls in sample 1 for the training of the SexOrientationNet. Although the analysis was enriched with data from other samples with heterosexual subjects, it was not possible to separate a complete unseen test dataset; a Leave One Out approach had to be used instead. Although this can be seen as the method of choice in an environment with limited data, a larger dataset would have allowed for a more rigid testing procedure. Moreover, following this limitation, all conclusions drawn from this study have to be considered with caution until proved in an independent sample.
Another limitation was the low sampling frequency of the finally used EEG datasets. Although higher sampling rate was available for most datasets, at the point in time where the analysis was carried out, the authors did not have access to the computational power to calculate the networks for higher frequencies.
Conclusion
Homosexual males did not show female-like patterns in a pretrained network on sex but could be differentiated from HeMs in a newly trained network. It was possible to extract neurophysiological features and identify anatomical regions of interest for further research. Thus, deep learning in the EEG domain has the potential to open new avenues for research in the field by decoupling discriminative tasks from a priori-defined features.
Acknowledgments
Sincere thanks go to Christine Wyss, Tania Villar de Araujo, Xenia Binner, Laura Nanz, and Helena Pejic for their assistance.
Statement of Ethics
The study was approved by the Ethical Committee Zurich (EC-No 2014-0623). All participants gave written informed consent. The study was conducted following the rules established by the Declaration of Helsinki.
Conflict of Interest Statement
No conflicts of interest are reported by the authors.
Funding Sources
No external funding was provided for this study.
Author Contributions
Anastasios Ziogas designed the study, performed data acquisition, and helped in analyzing, drafting, revising, and approving the manuscript. Andreas Mokros, Wolfram Kawohl, Benedikt Habermeyer, and Elmar Habermeyer helped in the study design, result interpretation, and drafting and revising the manuscript. Mateo de Bardeci and Ilyas Olbrich helped in analyzing the data, interpreting the results, and drafting and revising the manuscript. Sebastian Olbrich helped in the design of the study, performed data analysis, and drafted and revised the manuscript.
Data Availability Statement
Data are not publicly available due to ethical reasons. Further inquiries can be directed to the corresponding author.