Introduction: It is unclear if sexual orientation is a biological trait that has neurofunctional footprints. With deep learning, the power to classify biological datasets without an a priori selection of features has increased by magnitudes. The aim of this study was to correctly classify resting-state electroencephalogram (EEG) data from males with different sexual orientation using deep learning and to explore techniques to identify the learned distinguishing features. Methods: Three cohorts (homosexual men, heterosexual men, and a mixed sex cohort), one pretrained network on sex classification, and one newly trained network for sexual orientation classification were used to classify sex. Further, Grad-CAM methodology and source localization were used to identify the spatiotemporal patterns that were used for differentiation by the networks. Results: Using a pretrained network for classification of males and females, no differences existed between classification of homosexual and heterosexual males. The newly trained network was able, however, to correctly classify the cohorts with a total accuracy of 83%. The retrograde activation using Grad-CAM technology yielded distinctive functional EEG patterns in the Brodmann area 40 and 1 when combined with Fourier analysis and a source localization. Discussion: This study shows that electrophysiological trait markers of male sexual orientation can be identified using deep learning. These patterns are different from the differentiating signatures of males and females in a resting-state EEG.

Sexual orientation is an important personal characteristic of every person. The individual orientation can be placed on a continuum between homo- and heterosexual alignment rather than applying a strict dichotomous division [1]. Throughout history, persons with sexual preferences for the same sex and thus a differing orientation from the majority have been facing persecution and violence, also in the Christian-coined western world [2‒4]. The American Psychiatric Association only removed homosexuality from the category of psychiatric diseases in 1973 [5]. However, people always tried to understand the reason for these differences in sexual orientation. One of the main questions thereby has been if either homosexuality is a hereditary trait or environmental factors influence the development of these preferences [1]. A main motivation for research in this area has been the search for notorious preventive measures for a long time [6]. Nowadays, in the western world, the freedom of choice including the sexual preferences has evolved as the widely accepted social agreement, although it is still a long way from providing safety and participation for every person, regardless of their sexual orientation.

Having reached this cultural developmental stage, it still is of high scientific interest, whether there exist biological patterns that differ between persons with different sexual orientations. The authors are aware of the ethical questions arising from this research, as outlined above. The approvement of biological traits in alignment with sexual orientation still could help further withdraw the basis of involuntary behavioral measures against persons with nonheterosexual orientation [7]. The fraternity hypothesis stated that the existence of older male brothers increased the probability of being gay in men, thereby attributing a significant impact to environmental factors on sexual orientation [8]. Others found genetic [9, 10], epigenetic [11], brain structure [12‒14], or immunological [15] differences in subjects with same sex sexual orientation, supplying evidence for a biological trait. Especially findings from large-scale studies using data from over 400.000 subjects from the UK Biobank and the 23andMe database revealed that there are genetic predispositions for same-sex sexual behavior [10]. However, the main outcome was that no genetic variant allowed for a prediction of the individual sexual orientation. The authors underline that their findings shed light on the complexity of sexual behavior because of still largely unknown interactions between genetics and environment.

In the light of structural differences between individuals with different variants of sexual orientation, it seems paramount to also search for possible functional differences. Electrophysiology, and electroencephalogram (EEG) in particular, provides a highly suitable framework for a noninvasive analysis of brain function for a large variety of conditions. In combination with advanced technologies such as deep learning, EEG data recently have been used to differentiate between males and females based on their brain function patterns during rest [16]. Although EEG research has been misused to identify pathological EEG patterns in males with homosexual orientation [17, 18], more recent research was more focused on mainly functional, non-pathological differences in groups of homosexual and heterosexual males (HeMs) [19]. When EEG data were used in these studies to differentiate between males and females [20] or groups of homosexual and HeMs [21], the data were always recorded during cognitive task performance (e.g., mental rotation). Results could therefore be related to sex-dimorphic cognitive task properties and less to functional differences by means of basic central nervous system activity during rest.

Within recent years, new techniques derived from the field of artificial intelligence research emerged that allow for powerful analysis of medical and electrophysiological data [22]. Especially so-called deep learning algorithms proved their superiority in the classification of time series data in comparison to conventional statistical analysis, given that enough labeled data are available [23]. These deep learning techniques stem from the functional principle of natural neuronal networks as can be found in the human brain [24]. By changing the connection strength between different nodes, i.e., neurons during the training of the network, the labeled input data will be processed to result in the desired output, i.e., the classification label. The network then can be tested with previously unseen testing data. Since it is appealing to classify brain-derived functional time series using a technique that follows the function of neuronal networks, in this study, we aimed to test whether there are differences in the electrophysiological patterns during rest between homosexual and heterosexual men using these deep learning. Assuming that cues of biological sex are detected and processed in similar ways within individuals who are interested in the same sex of potential partners [25], we assumed a resemblance in the resting state activity between homosexual men (HoM) and heterosexual women. Some research of the past showed similar functional activations in homosexual males and heterosexual females [12]. Consequently, we at first tested if (1) HoM might exhibit similar EEG patterns like heterosexual females in comparison to heterosexual men. That means HoM could be classified via a network that was trained on sex and not sexual orientation, receiving the label “female.” If this quite mechanistical hypothesis would not hold true, i.e., homosexual males would not exhibit similar EEG patterns as heterosexual females, it was further hypothesized that (2) HoM could be classified by applying a new network trained on HoM and heterosexual men, i.e., being classified via a network trained on sexual orientation and not sex. For testing (1), a previously trained male/female classification deep learning network (“SexNet”) [16] was applied to datasets of HoM, heterosexual men, and a third group of mixed males and females. For testing (2), a new network was trained to differentiate sexual orientation by using data from HoM or heterosexual men. To further elucidate the functional properties and anatomical distribution of potential EEG patterns for differentiation between homosexual and heterosexual men at the individual level, a gradient CAM (Grad-CAM) approach in combination with a source localization technique and Fourier transformation was used.

The study was approved by the Ethical Committee Zurich (EC-No 2014-0623). All participants gave written informed consent. The study was conducted following the rules established by the Declaration of Helsinki.

Subjects Sample 1 and Sample 2: EEG Recording

Sample 1 consisted of 39 HeMs and 38 HeMs recruited from 2017 to 2019 in Zurich and was reported in another study [26]. Ten additional subjects had been recruited but needed to be excluded, four due to regular intake of medication. Two more subjects showed a sexual orientation that showed no clear preference for on sex (bisexuality), based on a Klein Sexual Orientation Grid (KSOG) Score between 3and 4.9. Four subjects had to be excluded due to missing or bad EEG data.

Sample 2 consisted of 37 healthy controls with mixed sex (female = 16) and unknown sexual orientation, recorded with the same equipment under the same conditions as sample 1. Sample 2 was recorded in year 2010 as control subgroup for the ZINEP study [27].

Subjects Sample 3: EEG Recording and SexNet Training

In total, 1,300 EEG datasets were used for the SexNet training and validation. They were recorded and preprocessed using a standardized methodology and platform (Brain Resource Ltd., Australia) for which full details have been published elsewhere [28] and which is in large parts identical for sample 1 and 2. The used SexNet has previously been reported to differentiate men and women based on 80 s resting-state EEG recordings with an accuracy of 84% [16], which is a highly significant finding, given a random guess distribution of 50%. To avoid overestimation for transfer learning, we did not apply the top performance net from [16] but relied on an average model with a performance accuracy of 73%. Therefore, SexNet was trained on 24 channel EEG segments with a 128 Hz resolution from 1000 adults and tested on unseen 300 EEG datasets. The architecture of SexNet consisted of 9 layers with 256 × 24 as input matrix, convolutional layers with decreasing number of filters (300–50) and pooling function after each of the first four convolutional layers, and a dropout function of 25% after each convolutional layer. A rectifier linear unit served for activation. A final dense layer with softmax activation was used for classification probabilities. The rest of the training procedure was done as described in [16].

EEG Processing and SexOrientationNet

The EEGs were recorded at 2.5 kHz (BrainAmp, BrainProducts, Germany) during a 5-min resting state with eyes closed using a 32-electrode BrainCap (MR 32 standard, Easycap) referenced to FCz following the international 10 to 20 system. Impedances were kept below 20 kOhm. Eye movements were recorded using bipolar diagonal electrooculogram channels. Software analysis was done using Brain Vision Analyzer 2.0, Gilching, Germany: raw data were downsampled to 128 Hz and band pass filtered between 0.5 and 25 Hz plus notch filter. To compare the results with the outcome of the SexNet, 24 channels were kept for further analysis as follows: Fp1, Fp2, F7, F3, Fz, F4, F8, FC3, FCz, FC4, T3, C3, Cz, C4, T4, CP3, CPz, CP4, T5, P3, Pz, P4, T6, O1. EEG data were eye – artefact corrected using a regression-based approach Gratton, Coles, and Donchin [29]. Artefact rejection was done using an automated pipeline following different voltage and power criteria, detailed in [28]. Finally, the EEG was inspected visually by an experienced EEG-rater, and artefacts were marked and rejected for consecutive 2 s segments (example Fig. 1). Raw data of 4-min recordings were imported to Matlab software (R2019b), where a three-dimensional matrix of channels (24) × segment-length (2 sec with 2 × 128 = 256 timepoints) × epochs (120 with 2 sec duration) were stacked for each subject. Labeling of the subjects (homosexual man or heterosexual man in sample 1 and additionally heterosexual woman in sample 2 and sample 3) was done using one-hot coding. The matrix then was exported to the python environment.

Fig. 1.

Example of cleaned 2 s segment of EEG data used for classification of sexual orientation.

Fig. 1.

Example of cleaned 2 s segment of EEG data used for classification of sexual orientation.

Close modal

The SexOrientationNet was deployed in a python (3.8) TensorFlow (version 1.2 GPU version) environment with a Keras backend (Version 2.2). The structure of the used network was identical to the structure of the SexNet, only the number of filters was between 100 and 300. However, training was not performed on sample 3 but on sample 1 (homosexual and heterosexual men) with a Leave One Out validation process (see below Sex Orientation network validation)

Assessment of Sexual Orientation

The KSOG [30] was used to check for self-reported sexual preference. This scale uses seven items on sexual orientation (sexual attraction, sexual behavior, sexual fantasies, emotional preference, social preference, self-identification, and heterosexual/gay lifestyle outcomes) that are addressed with responses on three dimensions (past, present, and future). Responses are given on a 7-point Likert scale ranging from 1 (other sex only/heterosexual only) to 7 (same sex only/gay only). Cronbach’s alpha of 0.96 and significant correlation with self-reported sexual orientation (r = 0.71) have been reported for the KSOG [31]. Average scores across the present time dimension were calculated. Values of 2.9 or less refer to heterosexual orientation; values of 5.0 or above refer to a homosexual orientation [32].

Deep Learning SexNet and SexOrientationNet Validation and Feature Extraction

Validation for the usage of the SexNet was done using separate training and test sets. Accuracy measures were used for assessing the quality of the classification in the test set.

Validation for the usage of the newly trained SexOrientationNet on homosexual and heterosexual men was done using the “Leave One Out Cross Validation.” This common practice splits data into k independent folds and subsequently trains in k−1 folds and tests in the complementary fold until the model has been trained and tested in every fold (i.e., in the complete dataset). This cross-validation is seen as the method of choice when it comes to deep learning with a limited amount of data and a separate training and test sets are not feasible [33].

Grad-CAM Feature Extraction

For feature extraction, the class activation maps allow to seek for distinctive features in the input data for deep learning based networks. The class activation maps show weighted heatmaps of the input data that allow for the identification of the most important data points for classification tasks, e.g., in medical imaging [34]. Their successor, Grad-CAM allows for the same approach without the necessity of global average pooling before the softmax layer by using gradient information that flows into the last convolutional layer of the network. Grad-CAM also outperforms other heatmap algorithms in medical imaging [35]. To apply Grad-CAM method, a rectified linear unit is used to focus on the decisive features for a certain class. This method has been used to detect EEG channels with highest information load in brain-computer-interface scenarios [36]. Following this procedure, data from the subject to classify were fed into the network, and from each layer of the network, the decisive weights were exported and averaged over all layers. This weight map then was projected on the input data, i.e., the EEG segments. Then, these 2-D maps were averaged over the time axis, yielding the 1-D activation array in space, i.e., electrode positions. This process was repeated for all subjects. The output Grad-CAM maps were then averaged for all homosexual and all heterosexual subjects. Feeding these arrays into a source localization algorithm such as the low-resolution electromagnetic tomography (eLORETA), it was possible to map the intracortical sources of electrophysiological activity that allowed the correct classification of a group.

EEG-Source Localization

Source localization analysis was done using eLORETA software package, version 20221229 (Roberto Pascual-Marqui/The KEY Institute for Brain-Mind Research, Zurich). The eLORETA algorithm is an inverse solution for EEG signals that has no localization error. In contrast to other source localization techniques, this is also applicable in the presence of measurement and structured biological noise [37]. The family of LORETA algorithms allows imaging of current density distributions inside the human cortex via constraining the solution to only gray matter-dense regions within 6,239 predefined 3D-voxels [38]. EEG assessment in conjunction with simultaneous recordings in other neuroimaging modalities cross-validated the family of LORETA algorithm and showed overlapping results of intracortical EEG-source estimates and the BOLD signal [39, 40] or glucose metabolic rate [41, 42]. To generate the input for the source localization, the projections of the network were trained on k−1 datasets and tested on the remaining dataset repeatedly, until all subjects of one group (hetero- or homosexual) were extracted from all layers for all EEG channels and averaged over all subjects of the group. The electrode activation matrix for the LORETA analysis was thus fed with the weights of the network, which allowed a correct classification. The LORETA transformation matrix was computed from the EEG channel positions used in the EEG recordings and the electrode activation matrix was projected into the intracortical solution space of the eLORETA software.

Frequency Analysis of the LORETA Regions

To determine which frequencies of the EEG input data were used for classification by the SexualOrientationNet, the output of all filters (ranging from 100–300 per layer) from all convolutional layers (n = 6) of the network were computed while using a loss function that maximized the activation of the filters. For each layer, the filter with the largest activation was projected into a 24 × 256 array, representing the EEG input with the highest potential of layer activation. A Fourier transformation was applied for the x-axis (i.e., EEG channel time series) to analyze the prevailing frequencies in the corresponding layers.

Estimating Statistical Threshold

For assessment of statistical significance of the SexualOrientationNet, we randomly assigned sexual orientation to each subject in a test set (n = 20), using the prior sexual orientation distribution (50% homosexual males). To set the p value for statistical significance at p ≤ 0.01, we performed 100 simulations in Matlab, following the principles of Monte Carlo simulation. The best classification accuracy reached was 70%, which was subsequently considered the significance threshold for p ≤ 0.01.

Comparison with fMRI Findings

To compare the sources of differentiating EEG activity with findings from neuroimaging studies using fMRI, the “Neurosynth” website (“neurosynth.org”) was used [43]. In total, 81 studies using fMRI and the context “sexual” were included into the sample. Works with the highest load in the analysis included studies on functional endophenotypes for sexual orientation [44], neural substrates of sexual desire [45], and correlates of gender differences [46]. The results of the uniformity analysis (z-scores from a one-way ANOVA testing) were corrected for multiple comparisons using the false discovery rate with a 0.01 criterion.

Sociodemography

For sample 1 (n = 77), mean age of homosexual males (n = 38) was 24.7 years with SD 5.0, HeMs (n = 38) with 24.6 years and SD 4.9. Mean KSOG score for homosexual males was 1.9 with SD 0.40, for HeMs 5.7 SD 0.39. For sample 2 (n = 37 with 16 females), mean age was 28.6 years (28.4 years for females only) with SD 5.9. For sample 3 (n = 1,300), the mean age was 43.4 with 18.4 SD with 47% males [16]. Different personal variables and performances, including handedness, sexual desire, sexual excitation, and the attentional network task [47], were tested for differences between groups (homosexual males vs. HeMs) with no significant alterations between the groups (see [26] for details).

Classification of Homo- or Heterosexual Orientation Using a Pretrained SexNet

The SexNet trained on 1,000 and tested on 300 EEGs from women and men to differentiate sex (accuracy of 73%) was applied to three different groups (two groups from sample 1 with HoM and HeM controls) and data from sample 2 (mixed male female [MixMF]). In all datasets, HoM, HeM, and MixMF, males irrespective of their sexual orientation were classified (binary classification with a probability of being male >50% for the majority of all 120 segments per subject) as male with an accuracy of 89.2–97.5% with no significant differences between the three datasets (p > 0.05). Only when numeric probabilities of subject classification were tested for differences, the HoM dataset showed a significant higher probability for correctly classified sex in comparison to the correct classification of the MixMF group (p = 0.00, see Table 1).

Table 1.

Correct classification of sex of three different groups from sample 1 and 2. Men were classified as men irrespective of their sexual orientation

HoMHeterosexual menMixed sexHo versus HeHo versus mixHe versus mix
Correct classification (binary) 97.5% (39/40) 92.3% (36/39) 89.2% (33/37) p = 0.30 p = 0.13 p = 0.64 
Probability of correct classification 88.8% (SD 14.2%) 83.5% (SD 18.5%) 75.8% (SD 21.6%) p = 0.16 p = 0.00* p = 0.10 
HoMHeterosexual menMixed sexHo versus HeHo versus mixHe versus mix
Correct classification (binary) 97.5% (39/40) 92.3% (36/39) 89.2% (33/37) p = 0.30 p = 0.13 p = 0.64 
Probability of correct classification 88.8% (SD 14.2%) 83.5% (SD 18.5%) 75.8% (SD 21.6%) p = 0.16 p = 0.00* p = 0.10 

Significant differences were only found when considering the general probability of a subject being classified correctly: homosexual males were classified correctly with higher probabilities than a mixed male/female cohort.

Classification of Sexual Orientation Using a New Trained Deep Net

To further test whether male subjects with different sexual orientation (homosexual or heterosexual) could be correctly classified using deep learning, a new network (SexOrientationNet) was trained on data from sample 1. The used Leave One Out training and testing procedure of the network for classification of homosexual or heterosexual men showed a performance of a 75.95% accuracy for the correct classification of HoM or heterosexual men for all segments of the dataset (Fig. 1). Following the majority vote on the 120 epochs of each subject when trained on the rest of the subjects, 26 out of 39 heterosexual men were classified correctly and 38 out of 38 HoM. A four-fold table of the results can be found in Figure 2 (bottom). The sensitivity to correctly classify a homosexual man as homosexual was 100%; the specificity was 67% with a positive predictive value of 76% and a negative predictive value of 100%, yielding a total accuracy of 83%. Taking into account that an accuracy of 70% is equivalent to a statistical threshold of p < 0.01 according to the Monte Carlo simulation, this result can be seen as highly significant. The number of epochs of learning for each trial (with one subject left out and an early stopping rule after four consecutive failed decreases of validation loss) ranged from 6 to 51. The averaged training and test accuracies and training and test losses for all segments (not only subjects) can be found in Figure 2.

Fig. 2.

Top: illustration of the accuracies and losses of the training set and the test set by means of classification of total segment count of each set. Results are averaged over 77 runs of the network with one subject left out every time for training. Bottom: a four-fold table for the correct classification of homosexual and heterosexual men using the SexOrientationNetwork, trained on data from homosexual and heterosexual men with a total accuracy of 83%.

Fig. 2.

Top: illustration of the accuracies and losses of the training set and the test set by means of classification of total segment count of each set. Results are averaged over 77 runs of the network with one subject left out every time for training. Bottom: a four-fold table for the correct classification of homosexual and heterosexual men using the SexOrientationNetwork, trained on data from homosexual and heterosexual men with a total accuracy of 83%.

Close modal

Visualization and Localization of Differentiating Features at the Single EEG Segment Level

To analyze whether it is possible to extract the EEG patterns that showed discriminative features for sexual orientation classification in single subjects, a Grad-CAM approach was used. Therefore, the trained filters from all layers (ranging from 100–300 filters per layer) from a single leave-one-out run were averaged layer-wise, and the gradients of a single EEG epoch from the test subject were computed for each averaged layer filter (single layer averages can be found in Fig. 3, panel a). It is obvious that early layers focus on activity at 10 Hz at posterior sites (Fig. 3, panel a), while later layers have less differentiation due to the max-pooling steps in the algorithm. The gradient projections from all layer averages were summed up to give a visualization of the gradients on the raw EEG (Fig. 3, panel b). The averaged layer weights then were convoluted toward the EEG channel axis and the weights were used for the LORETA source reconstruction. The main focus of the differentiating features was found in the Brodmann area 7, parietal lobe at x = −3, y = −53, z = 57) for the shown activity (Fig. 3, panel c and d).

Fig. 3.

Panel (a) shows averaged filter weights from layer 1–6, superimposed on a single epoch input EEG time series. High gradients are red; low gradients are blue. Since the network convolutes and max-pooling is applied, the resolution of the layer gets coarse toward the deeper layers. Panel (b) illustrates the grand average of all filters, again superimposed on a single EEG segment of 2 s. Red color codes for the areas with highest “attention” of the network; blue color codes for the least “attention” for differentiation into labels “heterosexual” or “homosexual” male EEG segments. A clear focus on the parietal and occipital alpha waves can be seen. Panel (c) and (d) show the reconstruction of the associated sources of the weights in LORETA space.

Fig. 3.

Panel (a) shows averaged filter weights from layer 1–6, superimposed on a single epoch input EEG time series. High gradients are red; low gradients are blue. Since the network convolutes and max-pooling is applied, the resolution of the layer gets coarse toward the deeper layers. Panel (b) illustrates the grand average of all filters, again superimposed on a single EEG segment of 2 s. Red color codes for the areas with highest “attention” of the network; blue color codes for the least “attention” for differentiation into labels “heterosexual” or “homosexual” male EEG segments. A clear focus on the parietal and occipital alpha waves can be seen. Panel (c) and (d) show the reconstruction of the associated sources of the weights in LORETA space.

Close modal

Visualization of Differentiating Features at the Group Level

As a next step, we trained 77 networks following the LOO approach and calculated an averaged Grad-CAM map for the remaining subject. Then, the maps were averaged for the two groups (homo- and heterosexual men, Fig. 3, panel a). Following the eLORETA source localization approach, the region with the activity that allowed for best labeling HeMs was located at the postcentral gyrus at the parietal lobe at Brodmann area 1 (X = −45, Y = −30, Z = 65, MNI cords, Fig. 4, panel b top). The region with the largest discriminative activity for HoM was found at the inferior parietal lobule at Brodmann area 40 (X = 40, Y = −55, Z = 60, MNI cords, Fig. 4, panel b, bottom).

Fig. 4.

Panel (a) visualizes the averaged Grad-CAM results for the channels of highest activation for HeMs (top) and homosexual males (bottom). These matrices were averaged over time and used for the transformation matrices to identify the cortical sources of these activities (panel c). While HeMs showed highest weights in the inferior parietal gyrus (panel b, top), homosexual males showed highest weights in the superior temporal gyrus (panel b, bottom). From the backward activated filters of the SexOrientationNet (panel c, ×3x3 filters with largest weights of 1–6), the time series of nearby electrodes was Fourier transformed to obtain information about the associated frequencies. While early layers show peaks at 17 and 48 Hz (panel d, pictures 1 and 3), no clear peaks were found for later layers (panel d, picture 2 and 4).

Fig. 4.

Panel (a) visualizes the averaged Grad-CAM results for the channels of highest activation for HeMs (top) and homosexual males (bottom). These matrices were averaged over time and used for the transformation matrices to identify the cortical sources of these activities (panel c). While HeMs showed highest weights in the inferior parietal gyrus (panel b, top), homosexual males showed highest weights in the superior temporal gyrus (panel b, bottom). From the backward activated filters of the SexOrientationNet (panel c, ×3x3 filters with largest weights of 1–6), the time series of nearby electrodes was Fourier transformed to obtain information about the associated frequencies. While early layers show peaks at 17 and 48 Hz (panel d, pictures 1 and 3), no clear peaks were found for later layers (panel d, picture 2 and 4).

Close modal

Identification of Frequency Properties for Classification at Group Level

For assessment of the frequencies that are associated with the identification of male sexual orientation, the filters of all layers of the SexOrientationNet were fed backward with patterns that activated each filter the most, thus generating archetypical inputs for the decisions of the networks. These patterns (Fig. 4, panel c) can be seen as EEG traces that evoke the classifications “heterosexual” or “homosexual.” Since these patterns have the same dimension as the input EEG segments (24 channels × 256 points in time), these time series were forwarded to Fourier transformation to reveal the involved frequencies. Analysis was restricted to the closest electrodes of the intracortical clusters (Fig. 4, panel b). This process was done separately for heterosexual and homosexual classification patterns and for all layers. While there were clear peak frequencies around 17 Hz and 48 Hz in early layers (Fig. 4, panel d, picture 1 and 3 for layer 2), later layers resampled more complex patterns that did seem to comprise multiple frequencies (Fig. 4, panel d, picture 2 and 4).

Comparison with fMRI Research

Comparing the sources of the discriminating EEG features identified by the SexOrientationNet with meta-analytical findings from 81 fMRI studies that used a sexuality context in their paradigms revealed an overlap mainly in the parietal cortices, i.e., at the superior parietal lobe (Brodmann x = −26, y = −60, z = 48). As can be seen in Figure 5, both groups, HoM and HeM, show source clusters with high filter weights in this area.

Fig. 5.

Comparison of results from a Neurosynth meta-analysis of fMRI findings of studies related to sexual content and the weighted sources of the SexOrientationNet on EEG data differentiating HeMs and homosexual males. XYZ coordinates are all the same for top to bottom panels.

Fig. 5.

Comparison of results from a Neurosynth meta-analysis of fMRI findings of studies related to sexual content and the weighted sources of the SexOrientationNet on EEG data differentiating HeMs and homosexual males. XYZ coordinates are all the same for top to bottom panels.

Close modal

This study aimed at the differentiation of homosexual and heterosexual orientation in men using neurophysiological resting state data. Further goal was to test feature extraction from the used networks to gain new insights into electrophysiological patterns that differ between homosexual and heterosexual men. While the sexual orientation in men could not be classified by a network previously trained on sex (SexNet), a newly trained network on sexual orientation (SexOrientationNet) was able to distinguish between males with homo- or heterosexual orientation with a very high accuracy of 83%. Further, the Grad-CAM method was able to extract electrode-wise gradients that allowed for an identification of distinguishing patterns at the temporal dimension, i.e., in the EEG traces and in the spatial dimension, i.e., the anatomical source space. Additionally, these sources could be linked to frequency analysis to reveal associated oscillations at the group level. These findings provide evidence that deep learning in combination with EEG data can successfully classify sexual orientation at the group level and extract non-a priori-defined features on the single subject/segment and group level that might help to understand underlying electrophysiological and anatomical circuits. The further extraction of source-bound frequencies that contributed to the network to learn revealed the importance of specific EEG oscillations for differentiation.

Using the pretrained SexNet, all different groups from sample 1 and sample 2 showed very high classification accuracy for sex. However, contrary to the hypothesized classification of homosexual males as more akin to females, following potential similarities of sexual stimulus processing between homosexual males and females in contrast to HeMs, there was no difference for classification of homosexual males and HeMs. Strikingly, both variants of sexual orientations were classified as males when using a network trained on gender in another cohort. This counteracts assumptions that homosexual males are “femalized” woman. Interestingly, when not looking at the binary classification (yes/no) but into the probabilities for correct classification, the homosexual group even had a significant higher probability to be rated correctly in comparison to a mixed sex group. It is further interesting that the overall accuracy of this trial was higher than the results that this pretrained network achieved when being tested on 300 subjects from sample 3. The reason for a higher accuracy of 89–97.5% in comparison to 73% could be found in the fact that longer EEG epochs with a subsequently increased segment number per subject were used. The sample 1 and sample 2 datasets consisted of a three-fold number of segments per subject (240 s vs. 80 s) in comparison to the SexNet training (sample 3). A simulation on the association between numbers of segments for each subject (when classifying each subject into a specific group with >50% of classified segments in a binary labeling approach) and the overall probability of one segment being classified correctly shows a sharp increase of correct classification of subjects (Fig. 6) with increasing segment numbers. Hence, future studies on subject classification using deep learning and multiple EEG segments per subject should consider restricting the size of each segment to just contain the essential information for increasing the overall number of segments. This also strongly implies that deep learning approaches benefit from longer EEG recordings since this allows for more data on each subject to be classified. The increase of training samples has been found to go in line with a logarithmical performance increase in vison tasks [48].

Fig. 6.

Increase of probability for a correct classification of a single subject (for >50% of segments of one subject classified correctly) with different numbers of EEG segments (x-axis) from one recording and increasing overall probability for a correct classification of a single segment (colored graphs). Olbrich et al. [40].

Fig. 6.

Increase of probability for a correct classification of a single subject (for >50% of segments of one subject classified correctly) with different numbers of EEG segments (x-axis) from one recording and increasing overall probability for a correct classification of a single segment (colored graphs). Olbrich et al. [40].

Close modal

When a new trained net was used for classification instead (SexOrientationNet), the network was able to accurately differentiate between homosexual males and HeMs. It is noteworthy that all homosexual subjects were classified correctly but only 2/3rd (26 out of 39) from the HeMs. Although this result must be taken with caution, since no replication has yet been done, it might imply that most homosexual males show distinct features (or miss some features), while HeMs sometimes share these features (or miss them, too) without being homosexual. However, only the heterosexual group reveals (or misses) a feature in some of them (the correctly classified) that is not there (or present) in the homosexual group. Thus, the SexOrientationNet revealed a very high specificity for classifying subjects as HeMs and a very high sensitivity for homosexual orientation.

The second aim of this work was the identification features that allowed discrimination between homosexual and HeMs without a definition of an a priori feature space. The extraction of features from the network by using the Grad-CAM approach and backward filter activation from discriminative tasks using deep learning allows the identification of nonlinear features from the raw data [49]. Following this path, the presented work was able to show that the shape of the parietal and occipital alpha waves was focused by the SexOrientationNet to label single subjects according to their sexual orientation. This is in line with previous EEG findings. One study found that heterosexual men rated the erotic video with higher general and sexual arousal than the homosexual participants. During observation of the neutral and erotic videos, both groups showed decreased amplitude of the alpha band in prefrontal and parietal cortices, indicating increased attention [50]. Another study revealed less suppression of the mu-rhythm, a specific central rhythm in the frequency range of alpha, in HoM as reaction to painful actions in comparison to heterosexual men [51]. Interestingly, Alexander and Sufka found increased alpha activity in homosexual males over right parietal cortices during cognitive tasks [19]. Ziogas et al. [52] related hemispheric alpha activity further to sexual arousal.

However, at the group level, the extracted frequencies showed peaks in the beta and gamma range, not within the alpha band. Again, EEG beta activity has been associated with processing of erotic content in homosexual males [50]. Further, at both, the single subject level and at the group level, the areas with the most important weights for differentiation by the network were in the parietal cortex. It is of special interest that the right Brodmann area 40 was source of EEG activity that yielded highest discriminative power for homosexual males and Brodmann area 1 had the highest load for HeMs. Parietal regions have been described in the context of sexual arousal in fMRI studies [53], while Brodmann area 1 is part of the somatosensory cortex and thus highly related to social perception [54] and sexual arousal [55]. The meta-analysis of 81 fMRI studies using paradigms with sexual content revealed a more likely activation of parietal regions in studies with sexual in comparison when no sexual content was present. Here, it must be mentioned that fMRI activation maps have to be regarded with caution since they cannot be interpreted as simple maps of areas with higher or lower neuronal activity, given the nature of the underlying blood oxygenation dependent signal [56, 57]. However, the involvement of the parietal and temporal cortex in activity generation in both homosexual and HeMs during sexual exposure [58] has been linked to arousal and attention networks [59]. The findings of this study underline the presence of specific functional patterns at defined cortical areas that are different for homosexual and HeMs.

Although this study clearly shows the potentials of deep learning-guided EEG analysis for discriminative tasks, it still bears some limitations. The first limitation can be found in the relatively small number of homosexual subjects and heterosexual controls in sample 1 for the training of the SexOrientationNet. Although the analysis was enriched with data from other samples with heterosexual subjects, it was not possible to separate a complete unseen test dataset; a Leave One Out approach had to be used instead. Although this can be seen as the method of choice in an environment with limited data, a larger dataset would have allowed for a more rigid testing procedure. Moreover, following this limitation, all conclusions drawn from this study have to be considered with caution until proved in an independent sample.

Another limitation was the low sampling frequency of the finally used EEG datasets. Although higher sampling rate was available for most datasets, at the point in time where the analysis was carried out, the authors did not have access to the computational power to calculate the networks for higher frequencies.

Homosexual males did not show female-like patterns in a pretrained network on sex but could be differentiated from HeMs in a newly trained network. It was possible to extract neurophysiological features and identify anatomical regions of interest for further research. Thus, deep learning in the EEG domain has the potential to open new avenues for research in the field by decoupling discriminative tasks from a priori-defined features.

Sincere thanks go to Christine Wyss, Tania Villar de Araujo, Xenia Binner, Laura Nanz, and Helena Pejic for their assistance.

The study was approved by the Ethical Committee Zurich (EC-No 2014-0623). All participants gave written informed consent. The study was conducted following the rules established by the Declaration of Helsinki.

No conflicts of interest are reported by the authors.

No external funding was provided for this study.

Anastasios Ziogas designed the study, performed data acquisition, and helped in analyzing, drafting, revising, and approving the manuscript. Andreas Mokros, Wolfram Kawohl, Benedikt Habermeyer, and Elmar Habermeyer helped in the study design, result interpretation, and drafting and revising the manuscript. Mateo de Bardeci and Ilyas Olbrich helped in analyzing the data, interpreting the results, and drafting and revising the manuscript. Sebastian Olbrich helped in the design of the study, performed data analysis, and drafted and revised the manuscript.

Data are not publicly available due to ethical reasons. Further inquiries can be directed to the corresponding author.

1.
Bailey
JM
,
Vasey
PL
,
Diamond
LM
,
Breedlove
SM
,
Vilain
E
,
Epprecht
M
.
Sexual orientation, controversy, and science
.
Psychol Sci Public Interest
.
2016 Sep
17
2
45
101
.
2.
Gold
JH
.
Homosexuality: a review
.
Can Fam Physician
.
1983 Mar
29
521
4
.
3.
Groneberg
M
,
Funke
C
Combatting homophobia: experiences and analyses pertinent to education
Berlin
Auflage LIT
2011
.
4.
Röll
W
.
Homosexual inmates in the buchenwald concentration camp
.
J Homosex
.
1996
;
31
(
4
):
1
28
.
5.
De Block
A
,
Adriaens
PR
.
Pathologizing Sexual Deviance: A History
.
J Sex Res
.
2013
50
3–4
276
98
.
6.
Bieber
I
,
Bieber
TB
.
Male homosexuality
.
Can J Psychiatry
.
1979 Aug
24
5
409
21
.
7.
Independent Forensic Expert Group
.
Statement on conversion therapy
.
J Forensic Leg Med
.
2020 May
72
101930
.
8.
Blanchard
R
,
Bogaert
AF
.
Homosexuality in men and number of older brothers
.
Am J Psychiatry
.
1996 Jan
153
1
27
31
.
9.
Gavrilets
S
,
Rice
WR
.
Genetic models of homosexuality: generating testable predictions
.
Proc Biol Sci
.
2006 Dec 22
273
1605
3031
8
.
10.
Ganna
A
,
Verweij
KJH
,
Nivard
MG
,
Maier
R
,
Wedow
R
,
Busch
AS
.
Large-scale GWAS reveals insights into the genetic architecture of same-sex sexual behavior
.
Science
.
2019 Aug
365
6456
eaat7693
.
11.
Ngun
TC
,
Vilain
E
.
The biological basis of human sexual orientation: is there a role for epigenetics
.
Adv Genet
.
2014
;
86
:
167
84
.
12.
Savic
I
,
Lindström
P
.
PET and MRI show differences in cerebral asymmetry and functional connectivity between homo- and heterosexual subjects
.
Proc Natl Acad Sci U S A
.
2008 Jul
105
27
9403
8
.
13.
Burke
SM
,
Manzouri
AH
,
Savic
I
.
Structural connections in the brain in relation to gender identity and sexual orientation
.
Sci Rep
.
2017 20
7
1
17954
.
14.
Abé
C
,
Johansson
E
,
Allzén
E
,
Savic
I
.
Sexual orientation related differences in cortical thickness in male individuals
.
PLoS One
.
2014 Dec
9
12
e114721
.
15.
Bogaert
AF
,
Skorska
MN
,
Wang
C
,
Gabrie
J
,
MacNeil
AJ
,
Hoffarth
MR
.
Male homosexuality and maternal immune responsivity to the Y-linked protein NLGN4Y
.
Proc Natl Acad Sci U S A
.
2018 09
115
2
302
6
.
16.
van Putten
MJAM
,
Olbrich
S
,
Arns
M
.
Predicting sex from brain rhythms with deep learning
.
Sci Rep
.
2018 Feb
8
1
3069
.
17.
Silverman
D
,
Rosanoff
WR
.
Electro-encephalographic and neurological studies of homosexuals
.
J Nerv Ment Dis
.
1945
;
101
(
4
):
311
21
.
18.
Papatheophilou
R
,
James
S
,
Orwin
A
.
Electroencephalographic findings in treatment-seeking hamosexuals compared with heterosexuals: a controlled study
.
Br J Psychiatry
.
1975 Jul
127
1
63
6
.
19.
Alexander
JE
,
Sufka
KJ
.
Cerebral lateralization in homosexual males: a preliminary EEG investigation
.
Int J Psychophysiol
.
1993 Nov
15
3
269
74
.
20.
Razumnikova
OM
.
Gender differences in hemispheric organization during divergent thinking: an EEG investigation in human subjects
.
Neurosci Lett
.
2004 May
362
3
193
5
.
21.
Wegesin
DJ
.
Event-related potentials in homosexual and heterosexual men and women: sex-dimorphic patterns in verbal asymmetries and mental rotation
.
Brain Cogn
.
1998 Feb
36
1
73
92
.
22.
Rajpurkar
P
,
Chen
E
,
Banerjee
O
,
Topol
EJ
.
AI in health and medicine
.
Nat Med
.
2022 Jan
28
1
31
8
.
23.
Craik
A
,
He
Y
,
Contreras-Vidal
JL
.
Deep learning for electroencephalogram (EEG) classification tasks: a review
.
J Neural Eng
.
2019 Apr
16
3
031001
.
24.
LeCun
Y
,
Bengio
Y
,
Hinton
G
.
Deep learning
.
Nature
.
2015 May
521
7553
436
44
.
25.
Santtila
P
,
Mokros
A
,
Viljanen
K
,
Koivisto
M
,
Sandnabba
NK
,
Zappalà
A
.
Assessment of sexual interest using a choice reaction time task and priming: a feasibility study
.
Leg Criminol Psychol
.
2009
;
14
(
1
):
65
82
.
26.
Ziogas
A
,
Habermeyer
B
,
Kawohl
W
,
Habermeyer
E
,
Mokros
A
.
Automaticity of Early Sexual Attention: An Event-Related Potential Study
.
Sex Abuse
.
2022 Aug
34
5
507
6
.
27.
Ajdacic Gross
V
,
Müller
M
,
Rodgers
S
,
Warnke
I
,
Hengartner
MP
,
Landolt
K
.
The ZInEP Epidemiology Survey: background, design and methods
.
Int J Methods Psychiatr Res
.
2014 Jun
23
4
451
68
.
28.
Arns
M
,
Bruder
GE
,
Hegerl
U
,
Spooner
C
,
Palmer
DM
,
Etkin
A
.
EEG alpha asymmetry as a gender-specific predictor of outcome to acute treatment with different antidepressant medications in the randomized iSPOT-D study
.
Clin Neurophysiol
.
2016
;
127
(
1
):
509
19
.
29.
Gratton
G
,
Coles
MG
,
Donchin
E
.
A new method for off-line removal of ocular artifact
.
Electroencephalogr Clin Neurophysiol
.
1983 Apr
55
4
468
84
.
30.
Klein
F
,
Sepekoff
B
,
Wolf
TJ
.
Sexual orientation: a multi-variable dynamic process
.
J Homosex
.
1985
11
1–2
35
49
.
31.
Qualls
LR
,
Hartmann
K
,
Paulson
JF
.
Broad autism phenotypic traits and the relationship to sexual orientation and sexual behavior
.
J Autism Dev Disord
.
2018 Dec
48
12
3974
83
.
32.
Moore
DL
,
Norris
FH
.
Empirical investigation of the conflict and flexibility models of bisexuality
.
J Bisexuality
.
2005 Apr
5
1
5
25
.
33.
de Bardeci
M
,
Ip
CT
,
Olbrich
S
.
Deep learning applied to electroencephalogram data in mental disorders: a systematic review
.
Biol Psychol
.
2021 May
162
108117
.
34.
Shinde
S
,
Tupe-Waghmare
P
,
Chougule
T
,
Saini
J
,
Ingalhalikar
M
.
Predictive and discriminative localization of pathology using high resolution class activation maps with CNNs
.
Peerj Comput Sci
.
2021
;
7
:
e622
.
35.
Zhang
Y
,
Hong
D
,
McClement
D
,
Oladosu
O
,
Pridham
G
,
Slaney
G
.
Grad-CAM helps interpret the deep learning models trained to classify multiple sclerosis types using clinical brain magnetic resonance imaging
.
J Neurosci Methods
.
2021 Apr
353
109098
.
36.
Li
Y
,
Yang
H
,
Li
J
,
Chen
D
,
Du
M
.
EEG-based intention recognition with deep recurrent-convolution neural network: performance and channel selection by Grad-CAM
.
Neurocomputing
.
2020 Nov
415
225
33
.
37.
Pascual-Marqui
RD
Discrete, 3D distributed, linear imaging methods of electric neuronal activity. Part 1: exact, zero error localization
.
2007 Oct
.
38.
Pascual-Marqui
RD
,
Esslen
M
,
Kochi
K
,
Lehmann
D
.
Functional imaging with low-resolution brain electromagnetic tomography (LORETA): a review
.
Methods Find Exp Clin Pharmacol
.
2002
24
Suppl C
91
5
.
39.
Mulert
C
,
Jäger
L
,
Schmitt
R
,
Bussfeld
P
,
Pogarell
O
,
Möller
H-J
.
Integration of fMRI and simultaneous EEG: towards a comprehensive understanding of localization and time-course of brain activity in target detection
.
Neuroimage
.
2004 May
22
1
83
94
.
40.
Olbrich
S
,
Mulert
C
,
Karch
S
,
Trenner
M
,
Leicht
G
,
Pogarell
O
.
EEG-vigilance and BOLD effect during simultaneous EEG/fMRI measurement
.
Neuroimage
.
2009 Apr
45
2
319
32
.
41.
Pizzagalli
DA
,
Oakes
TR
,
Fox
AS
,
Chung
MK
,
Larson
CL
,
Abercrombie
HC
.
Functional but not structural subgenual prefrontal cortex abnormalities in melancholia
.
Mol Psychiatry
.
2004
;
9
(
4
):
325
405
.
42.
Guenther
T
,
Schönknecht
P
,
Becker
G
,
Olbrich
S
,
Sander
C
,
Hesse
S
.
Impact of EEG-vigilance on brain glucose uptake measured with [(18)F]FDG and PET in patients with depressive episode or mild cognitive impairment
.
Neuroimage
.
2011 May
56
1
93
101
.
43.
Yarkoni
T
,
Poldrack
RA
,
Nichols
TE
,
Van Essen
DC
,
Wager
TD
.
Large-scale automated synthesis of human functional neuroimaging data
.
Nat Methods
.
2011 Jun
8
8
665
70
.
44.
Ponseti
J
,
Bosinski
HA
,
Wolff
S
,
Peller
M
,
Jansen
O
,
Mehdorn
HM
.
A functional endophenotype for sexual orientation in humans
.
Neuroimage
.
2006 Nov
33
3
825
33
.
45.
Seok
JW
,
Sohn
JH
.
Neural substrates of sexual desire in individuals with problematic hypersexual behavior
.
Front Behav Neurosci
.
2015
;
9
:
321
.
46.
Strahler
J
,
Kruse
O
,
Wehrum-Osinsky
S
,
Klucken
T
,
Stark
R
.
Neural correlates of gender differences in distractibility by sexual stimuli
.
Neuroimage
.
2018 Aug
176
499
509
.
47.
Wang
YF
,
Jing
XJ
,
Liu
F
,
Li
ML
,
Long
ZL
,
Yan
JH
.
Reliable attention network scores and mutually inhibited inter-network relationships revealed by mixed design and non-orthogonal method
.
Sci Rep
.
2015 May
5
1
10251
.
48.
Sun
C
,
Shrivastava
A
,
Singh
S
,
Gupta
A
Revisiting unreasonable effectiveness of data in deep learning era
.
2017
. p.
843
52
. [cited 2022 Feb 2].
49.
Najafabadi
MM
,
Villanustre
F
,
Khoshgoftaar
TM
,
Seliya
N
,
Wald
R
,
Muharemagic
E
.
Deep learning applications and challenges in big data analytics
.
J Big Data
.
2015 Feb
2
1
1
.
50.
Amezcua-Gutiérrez
C
,
Marisela
HG
,
Fernández Guasti
A
,
Aguilar
MAC
,
Guevara
MA
.
Observing erotic videos with heterosexual content induces different cerebral responses in homosexual and heterosexual men
.
J Homosex
.
2020
;
67
(
5
):
639
57
.
51.
Lübke
KT
,
Sachse
C
,
Hoenen
M
,
Pause
BM
.
Mu-suppression as an indicator of empathic processes in lesbian, gay, and heterosexual adults
.
Arch Sex Behav
.
2020 Feb
49
2
635
44
.
52.
Ziogas
A
,
Habermeyer
E
,
Santtila
P
,
Poeppl
TB
,
Mokros
A
.
Neuroelectric correlates of human sexuality: a review and meta-analysis
.
Arch Sex Behav
.
2023
;
52
(
2
):
497
596
.
53.
Seok
JW
,
Sohn
JH
,
Cheong
C
.
Neural substrates of sexual arousal in heterosexual males: event-related fMRI investigation
.
J Physiol Anthropol
.
2016 Mar
35
8
.
54.
Keysers
C
,
Kaas
JH
,
Gazzola
V
.
Somatosensation in social perception
.
Nat Rev Neurosci
.
2010 Jun
11
6
417
28
.
55.
Mueller
SC
,
Wierckx
K
,
T’Sjoen
G
.
Neural and hormonal correlates of sexual arousal in transgender persons
.
J Sex Med
.
2020 Dec
17
12
2495
507
.
56.
Logothetis
NK
.
What we can do and what we cannot do with fMRI
.
Nature
.
2008 Jun
453
7197
869
78
.
57.
Logothetis
NK
,
Pauls
J
,
Augath
M
,
Trinath
T
,
Oeltermann
A
.
Neurophysiological investigation of the basis of the fMRI signal
.
Nature
.
2001 Jul
412
6843
150
7
.
58.
Hu
SH
,
Wei
N
,
Wang
QD
,
Yan
LQ
,
Wei
EQ
,
Zhang
MM
.
Patterns of brain activation during visually evoked sexual arousal differ between homosexual and heterosexual men
.
AJNR Am J Neuroradiol
.
2008 Nov
29
10
1890
6
.
59.
Stoléru
S
,
Grégoire
MC
,
Gérard
D
,
Decety
J
,
Lafarge
E
,
Cinotti
L
.
Neuroanatomical correlates of visually evoked sexual arousal in human males
.
Arch Sex Behav
.
1999 Feb
28
1
1
21
.