What Does it Mean to 'See With the Mind's Eye?' Adventures in the human imagination CONOR FRIEDERSDORF DECEMBER 4, 2014 Imagine the table where you've eaten the most meals. Form a mental picture of its size, texture, and color. Easy, right? But when you summoned the table in your mind's eye, did you really see it? Or did you assume we've been speaking metaphorically? As it turns out, how people form mental images seems to vary significantly, a fact that's surprised those who've encountered it for more than a century. In 1880, Francis Galton published his classic paper "Statistics of Mental Imagery" after asking a series of subjects about images summoned by their minds. Some protested that they couldn't really see anything. "These questions presuppose assent to a proposition regarding the 'mind's eye' and the 'images' it sees," one subject wrote. "This points to some initial fallacy … It is only by a figure of speech that I can describe my recollection of a scene as a 'mental image' which I can 'see' with my 'mind's eye' ... I do not see it any more than a man sees the thousand lines of Sophocles which under due pressure he is ready to repeat. The memory possesses it." Many men and a yet larger number of women ... declared that they habitually saw mental imagery, and that it was perfectly distinct to them and full of color. The more I pressed and cross-questioned them, professing myself to be incredulous, the more obvious was the truth of their first assertions. They described their imagery in minute detail, and they spoke in a tone of surprise at my apparent hesitation in accepting what they said. I felt that I myself should have spoken exactly as they did if I had been describing a scene that lay before my eyes, in broad daylight, to a blind man who persisted in doubting the reality of vision. He then described how people with a talent for mental imagery see things in their mind's eye: 1. Brilliant, distinct, never blotchy. 2. Quite comparable to the real object. I feel as though I was dazzled, e.g., when recalling the sun to my mental vision. 3. In some instances quite as bright as an actual scene. 4. Brightness as in the actual scene. 5. Thinking of the breakfast table this morning, all the objects in my mental picture are as bright as the actual scene. 6. The image once seen is perfectly clear and bright. 7. Brightness at first quite comparable to actual scene. 8. The mental image appears to correspond in all respects with reality. I think it is as clear as the actual scene. 9. The brightness is perfectly comparable to that of the real scene. 10. I think the illumination of the imaginary image is nearly equal to that of the real one. 11. All clear and bright; all the objects seem to me well defined at the same time. 12. I can see my breakfast table or any equally familiar thing with my mind's eye, quite as well in all particulars as I can do if the reality is before me. Until reading this paper, I'd never imagined that other people have a much higher capacity than I do to form a mental image. Now I suspect that they do. Indeed, I don't even understand what it is to imagine something with "brightness." My breakfast table is bright when the sun shines in through the windows and dim in the dead of night. Shouldn't its brightness depend on when I conjure it? Yet a man without this capacity said, "Dim and not comparable in brightness to the real scene." To what extent, I wonder, was he speaking metaphorically? After pondering this same subject, blogger Scott Alexander asked his readers what characteristics they presumed to be universal only to discover that they're not. The range of responses was fascinating. Some readers talked about ASMR, "a perceptual phenomenon characterized as a distinct, pleasurable tingling sensation in the head, scalp, back, or peripheral regions of the body." Lots of people have it. Others talked of synesthesia, a condition that causes some to perceive numbers as if they have an inherent color, or to experience confusion as orange. One reader expressed a work preference that a lot of others said that they also share: As a programmer, I hate open spaces with passion. I hate the noise which makes it difficult to focus. And I hate the feeling of someone looking at my back; it makes me tense. Now I am in a room with only two other people, with a wall behind my back, and it feels great. It makes a huge difference, emotionally. I suspect that putting “needs to sit with a wall behind back” in my CV or trying to negotiate that into job contract would be very bad signaling. But if it changes how I feel 8 hours every day, of course it has an impact on my work, even on my total life satisfaction. I’m not even sure how many people feel like this. It took me a while to notice this explicitly. Another worker said: I have recently come to the conclusion that some people actually like their jobs. For almost all of my adult life I assumed that anyone who acts as if they didn’t hate their job must be either in denial or lying. But it’s been occurring to me that some of those people were really good at keeping the pretense. I’ve known people who could have retired but happily kept plowing. So I think I’ve been the victim of the typical mind fallacy and that the simplest explanation is the correct one and a lot of people actually don’t hate working. (And I envy them with every cell of my body.) There was a commenter who had trouble conceiving of "emotions as actual things of their own, rather than ways to describe complex biological states. I’m angry because my pulse is raised, my hackles are up, and I can’t think as completely as I’d like. I’m frustrated because I want to bite down on something. I’m sad because I’m crying, even if it is because of pain or even just having my head in the wrong position. I’m hungry because my stomach is empty, or my hands are shaking a little." Another was the anti-Elmo: It’s a bad idea to tickle me; I will reflexively and uncontrollably attempt to hurt anyone who does. Sometimes I can stop once I figure out what’s going on and if the tickler is someone I like, but I will always react first with violence. I have no idea why; it’s not like tickling is painful or anything. One reader was confused by crowds: I don’t get political rallies. You know the ones, where some major politician goes in front of a big crowd and pours on the charisma, and everybody is cheering and shouting all at once? I used to live in Iowa, so I’ve had the opportunity to be in a bunch of those crowds, and the whole thing always seemed… just completely baffling. Like, I’m standing next to a bunch of people who are cheering, and it’s 2008 and I really like Barack Obama and he’s standing less than ten meters away, and everything about this experience is carefully calculated to get people super intensely excited—and for some reason I’m not feeling a thing. It’s kind of disappointing! I get this way at any sort of rally, actually, and the parts of concerts where they’re not playing music, and parades—any sort of event where you’re supposed to get caught up in the enthusiasm of a big crowd. Is this common? I mean, I see a lot of people go to big crowds to yell about stuff, and they seem to enjoy it, but that’s hardly an unbiased sample. Another was confused by body image: I grew up female and always felt that something was not right somehow, that there was some kind of misalignment. Hormone therapy has given me a much more masculine appearance, but unfortunately tipped the scales in the other direction, and now I feel that my form is too masculine. I wish I knew of some method for making my body perfectly androgynous, because I think that would be the only way I could be completely content with it. As far as social interaction, I don’t seem to care very much whether I’m perceived as male or female; neither set of pronouns bothers me, though I’m more used to male pronouns because that’s what people default to. I have a lot of interests and hobbies that are considered feminine, and a lot that are considered masculine (I’ve not yet tried to figure out what the ratio is, though that could be a fun exercise). Only slightly relevant, I think, is that I’m also bothered by my weight (I’m about forty pounds heavier than what would constitute health for my height, which is actually slimmer than I was a few years ago—an excruciatingly slow process), but not just because I find my shape unappealing or because it’s unhealthy. Those are factors, but it’s more that when I see my reflection, it just seems wrong, like I’m wearing someone else’s body. I have the same reaction to my facial structure; the only thing that really looks right to me are my eyes. All of these issues together are a great source of stress for me. I have a bad habit of torturing myself with imagined scenarios wherein I’m given a brand new body, and of course escaping into these little fantasies makes it that much harder to actually do anything about it, because I get that little bit of reward feedback from the scene I’ve played in my head. I could read more examples with fascination for days. And with that in mind, I wonder if any readers are willing to share unexpected ways in which their experience of the world turns out to be different than what other people think and feel. Emails to conor@theatlantic.com are encouraged. ===== https://markcontiphoto.com/in-the-minds-eye/ ===== https://www.visionsciences.org/2019-122-symposia/ Vision Sciences Society Prefrontal cortex in visual perception and recognition Time/Room: Friday, May 17, 2019, 5:00 – 7:00 pm, Talk Room 2 Organizer(s): Biyu Jade He, NYU Langone Medical Center Presenters: Diego Mendoza-Halliday, Vincent B. McGinty, Theofanis I Panagiotaropoulos, Hakwan Lau, Moshe Bar Symposium Description To date, the role of prefrontal cortex (PFC) in visual perception and recognition remains mysterious. While it is well established that PFC neuronal activity reflects visual stimulus features in a wide range of dimensions (e.g., position, color, motion direction, faces, …), it is commonly thought that such feature encoding in PFC is only for the service of behaviorally relevant functions, such as working memory, attention, task rules, and report. However, recent emerging evidence is starting to challenge this notion, and instead suggests that contributions by the PFC may be integral for perceptual functions themselves. Currently, in the field of consciousness, an intense debate revolves around whether the PFC contributes to conscious visual perception. We believe that integrating insight from studies aiming to understand the neural basis of conscious visual perception with that from studies elucidating visual stimulus feature encoding will be valuable for both fields, and necessary for understanding the role of PFC in vision. This symposium brings together a group of leading scientists at different stages in their careers who all have made important contributions to this topic. The talks will address the role of the PFC in visual perception and recognition from a range of complementary angles, including neuronal tuning in nonhuman primates, neuroimaging and lesion studies in humans, recent development in artificial neural networks, and implications for psychiatric disorders. The first two talks by Mendoza-Halliday and McGinty will address neuronal coding of perceived visual stimulus features, such as motion direction and color, in the primate lateral PFC and orbitofrontal cortex, respectively. These two talks will also cover how neural codes for perceived visual stimulus features overlap or segregate from neural codes for stimulus features maintained in working memory and neural codes for object values, respectively. Next, the talk by Panagiotaropoulos will describe neuronal firing and oscillatory activity in the primate PFC that reflect the content of visual consciousness, including both complex objects such as faces and low-level stimulus properties such as motion direction. The talk by Lau will extend these findings and provide an updated synthesis of the literature on PFC’s role in conscious visual perception, including lesion studies and recent developments in artificial neural networks. Lastly, Bar will present a line of research that establishes the role that top-down input from PFC to the ventral visual stream plays in object recognition, touching upon topics of prediction and contextual facilitation. In sum, this symposium will present an updated view on what we know about the role of PFC in visual perception and recognition, synthesizing insight gained from studies on conscious visual perception and classic vision research, and across primate neurophysiology, human neuroimaging, patient studies and computational models. The symposium targets the general VSS audience, and will be accessible and of interest to both students and faculty. Presentations: Partially-segregated population activity patterns represent perceived and memorized visual features in the lateral prefrontal cortex Speaker: Diego Mendoza-Halliday, McGovern Institute for Brain Research at MIT, Cambridge MA Additional Authors: Julio Martinez-Trujillo, Robarts Research Institute, Western University, London, ON, Canada. Numerous studies have shown that the lateral prefrontal cortex (LPFC) plays a major role in both visual perception and working memory. While neurons in LPFC have been shown to encode perceived and memorized visual stimulus attributes, it remains unclear whether these two functions are carried out by the same or different neurons and population activity patterns. To systematically address this, we recorded the activity of LPFC neurons in macaque monkeys performing two similar motion direction match-to-sample tasks: a perceptual task, in which the sample moving stimulus remained perceptually available during the entire trial, and a memory task, in which the sample disappeared and was memorized during a delay. We found neurons with a wide variety of combinations of coding strength for perceived and memorized directions: some neurons preferentially or exclusively encoded perceived or memorized directions, whereas others encoded directions invariant to the representational nature. Using population decoding analysis, we show that this form of mixed selectivity allows the population codes representing perceived and memorized directions to be both sufficiently distinct to determine whether a given direction was perceived or memorized, and sufficiently overlapping to generalize across tasks. We further show that such population codes represent visual feature space in a parametric manner, show more temporal dynamics for memorized than perceived features, and are more closely linked to behavioral performance in the memory than the perceptual task. Our results indicate that a functionally diverse population of LPFC neurons provides a substrate for discriminating between perceptual and mnemonic representations of visual features. Mixed selectivity for visual features and economic value in the primate orbitofrontal cortex Speaker: Vincent B. McGinty, Rutgers University – Newark, Center for Molecular and Behavioral Neuroscience Rutgers University – Newark, Center for Molecular and Behavioral Neuroscience Primates use their acute sense of vision not only to identify objects, but also to assess their value, that is, their potential for benefit or harm. How the brain transforms visual information into value information is still poorly understood, but recent findings suggest a key role for the orbitofrontal cortex (OFC). The OFC includes several cytoarchitectonic areas within the ventral frontal lobe, and has a long-recognized role in representing object value and organizing value-driven behavior. One of the OFC’s most striking anatomical features is the massive, direct input it receives from the inferotemporal cortex, a ventral temporal region implicated in object identification. A natural hypothesis, therefore, is that in addition to well-documented value coding properties, OFC neurons may also represent visual features in a manner similar to neurons in the ventral visual stream. To test this hypothesis, we recorded OFC neurons in macaque monkeys performing behavioral tasks in which the value of visible objects was manipulated independently from their visual features. Preliminary findings include a subset of OFC cells that were modulated by object value, but only in response to objects that shared a particular visual feature (e.g. the color red). This form of ‘mixed’ selectivity suggests that the OFC may be an intermediate computational stage between visual identification and value retrieval. Moreover, recent work showing similar mixed value-feature selectivity in inferotemporal cortex neurons suggests that neural mechanisms of object valuation may be distributed over a continuum of cortical regions, rather than compartmentalized in a strict hierarchy. Mapping visual consciousness in the macaque prefrontal cortex Speaker: Theofanis I Panagiotaropoulos, Neurospin, Paris, France In multistable visual perception, the content of consciousness alternates spontaneously between mutually exclusive or mixed interpretations of competing representations. Identifying neural signals predictive of such intrinsically driven perceptual transitions is fundamental in resolving the mechanism and identifying the brain areas giving rise to visual consciousness. In a previous study, using a no-report paradigm of externally induced perceptual suppression, we have shown that functionally segregated neural populations in the macaque prefrontal cortex explicitly reflect the content of consciousness and encode task phase. Here I will present results from a no-report paradigm of binocular motion rivalry based on the optokinetic nystagmus (OKN) reflex read-out of spontaneous perceptual transitions coupled with multielectrode recordings of local field potentials and single neuron discharges in the macaque prefrontal cortex. An increase in the rate of oscillatory bursts in the delta-theta (1-9 Hz), and a decrease in the beta (20-40 Hz) bands, were predictive of spontaneous transitions in the content of visual consciousness that was also reliably reflected in single neuron discharges. Mapping these perceptually modulated neurons revealed stripes of competing populations, also observed in the absence of OKN. These results suggest that the balance of stochastic prefrontal fluctuations is critical in refreshing conscious perception, and prefrontal neural populations reflect the content of consciousness. Crucially, consciousness in the prefrontal cortex could be observed for faces and complex objects but also for low-level stimulus properties like direction of motion therefore suggesting a reconsideration of the view that prefrontal cortex is not critical for consciousness. Persistent confusion on the role of the prefrontal cortex in conscious visual perception Speaker: Hakwan Lau, UCLA, USA Is the prefrontal cortex (PFC) critical for conscious perception? Here we address three common misconceptions: (1) PFC lesions do not affect subjective perception; (2) PFC activity does not reflect specific perceptual content; and (3) PFC involvement in studies of perceptual awareness is solely driven by the need to make reports required by the experimental tasks rather than subjective experience per se. These claims are often made in high-profile statements in the literature, but they are in fact grossly incompatible with empirical findings. The available evidence highlights PFC’s essential role in enabling the subjective experience in perception, contra the objective capacity to perform visual tasks; conflating the two can also be a source of confusion. Finally we will also discuss the role of PFC in perception in the light of current machine learning models. If the PFC is treated as somewhat akin to a randomly connected recurrent neural network, rather than early layers of a convolution network, the lack of prominent lesion effects may be easily understood. What’s real? Prefrontal facilitations and distortions Speaker: Moshe Bar, Bar-Ilan University, Israel Additional Authors: Shira Baror, Bar-Ilan University, Israel By now, we know that visual perception involves much more than bottom-up processing. Specifically, we have shown that object recognition is facilitated, sometimes even afforded, by top-down projections from the lateral and inferior prefrontal cortex. Next we have found that the medial prefrontal cortex, in synchrony with the para-hippocampal cortex and the retrosplenial cortex form the ‘contextual associations network’, a network that is sensitive to associative information in the environment and which utilizes contextual information to generate predictions about objects. By using various behavioral and imaging methods, we found that contextual processing facilitates object recognition very early in perception. Here, we go further to discuss the overlap of the contextual associations network with the default mode network and its implications to enhancing conscious experience, within and beyond the visual realm. We corroborate this framework with findings that imply that top-down predictions are not limited to visual information but are extracted from social or affective contexts as well. We present recent studies that suggest that although associative processes take place by default, they are nonetheless context dependent and may be inhibited according to goals. We will further discuss clinical implications, with recent findings that demonstrate how activity in the contextual associations network is altered in visual tasks performed by patients experiencing major depressive disorder. To conclude, contextual processing, sustained by the co-activation of frontal and memory-relate brain regions, is suggested to constitute a critical mechanism in perception, memory and thought in the healthy brain. ===== Nothing is lost, only changed (the First Law of thermodynamics?)