Malte Henningsen-Schomers will present MatCo research in a poster session at the ICON conference in Helsinki on 20.05.2022.

OBJECTIVES/RESEARCH QUESTION
Whether language influences perception and thought remains a subject of intense debate. We address this question in a brain-constrained neurocomputational model of fronto-occipital (extrasylvian) and frontotemporal (perisylvian) cortex including spiking neurons. The unsupervised neural network was simultaneously presented with word forms (phonological patterns, “labels”) in perisylvian areas and semantic grounding information (sensory-motor patterns, “percepts”) in extrasylvian areas representing either concrete or abstract concepts. Following the approach used in a previous simulation, each to-belearned concept was modeled as a triplet of partly overlapping percepts; the models were trained under two conditions: each instance of a perceptual triplet (patterns in extrasylvian areas) was repeatedly paired with patterns in perisylvian areas consisting of either (a) a corresponding word form (label condition), or (b) noise (no-label condition). We quantified the emergence of neuronal representations for the conceptually-related percepts using dissimilarity (Euclidean distance) of neuronal activation vectors during perceptual stimulation. Category learning was measured as the difference between within- and between-concept dissimilarity values (DissimDiff) of perceptual activation patterns.

RESULTS
A repeated-measures ANOVA with factors SemanticType (concrete/abstract) and Labelling showed main effects of both SemanticType and Labelling, and a significant interaction. We also quantified the “label effect” in percentage change from NoLabel to Label conditions, separately for between- and withincategory dissimilarities. This showed that the label effect was mainly driven by changes in between-category dissimilarity, was significantly larger for abstract than concrete concepts, and became even larger in the “deeper” layers of the model.

CONCLUSION
A brain-constrained neurocomputational simulation was performed to explore putative brain mechanisms of associating conceptual categories with linguistic labels. We found a clear Whorfian effect of category labels on the processing of conceptual instances: the model’s activity in response to perceptuo-motor grounding patterns was modulated depending on whether or not labels had been provided during the training phase. Labels were highly beneficial for semantic category learning performance, and this benefit was more strongly pronounced for abstract compared to concrete concepts and even more so in the deeper-lying semantic ‘hub’ areas of the model than in the primary areas, where stimulation was given.