Progress of electroencephalogram research is remarkable. Gradually, neural function is being elucidated. Our aim is to elucidate operation of the brain and its visual mechanism. We are also challenging on the application of BCI(Brain-Computer Interface).
Electroencephalogram (EEG) refers to the brain electrical activity recorded with electrodes on the scalp. EEGs are electrical signals of several tens of microvolts, which can be observed by amplification with an electroencephalograph.
Event-Related Potentials (ERP) is the brain response that originates in brain activity caused synchronizing with specific event. Studies on ERP can help us to understand human brain mechanism. In our laboratory, visual information processing system is investigated by measuring ERP during visual task.
The pupil is commonly called “Kurome (黒目): iris” in Japanese, and in the case of human eyes, the pupil size varies between 2 and 8 mm. Although it is known that the size of our pupil increase in a dark environment (dilation) and decrease in a bright environment (constriction), various other aspects such as cognitive load, emotional state, interest in things and surprises effects the size too. Which means the size of the pupil is also linked to our internal state. Our laboratory measures eye movements during various experiments as well as EEG measurement, and investigates all human cognitive mechanisms from pupil diameter information.
Study on “Face”
Face perception is important for our social interaction. Our study has been revealed that when we perceive faces, not only face features such as configuration and contour but also face color, affect our face perception. Aim we continue revealing the effect of the color information on face processing and consider its developmental aspects using tools such as EEG, fMRI, TMS.
Our memory is not always accurate, but accurate recollection is occasionally required, such as eyewitness memory. Therefore, it is required to get more accurate recollection. Electroencephalogram(EEG) reflects one’s cognitive state including memory state, thus, it might be that memory state could be estimate by EEG. Here, the purpose of this study is to investigate whether memory state is estimated by EEG, especially vague memory. To generate vague memory, chimeric face stimuli was used in this study. At first, participants were required to watch chimeric face stimuli, then, they were required to respond to whether they had seen the presented chimeric faces (old, conjunction, new) before and the confidence of thier choice while measuring thier EEG. As a result, when participates response that they did not to see it, the Late Positive Component(LPC) anplitude induced by the old stimulus increased more than the conjunctive stimulus and the new stimulus. The result suggests that even the witness mis-responds, the EEG can be used to reconigzed whether the they have already seen the face or not.
The face pareidolia phenomenon, incorrect perception of something that is not a face as a face, affects human cognition and behavior. Although some face pareidolia studies focus on event-related potentials (ERPs) as the physiological indicator, most of them investigated the physiological difference between face recognition and face pareidolia. Besides, physiological response induced by face pareidolia stimulus might be influenced by contexts other than face-likeness.
The current study estimates the ERP component that affects the face pareidolia phenomenon by measuring EEGs while evaluating face-like abstract figures’ face-likeness.
Study on “Emotion”
Human emotions play a key role for formation of a character. It is induced by accepting the sensory stimulus such as visual images and auditory sounds. Since the 1960s, emotion has been investigated in fields such as cognitive psychology and neuroscience, and its mechanism has been gradually revealed along with the development of measuring methods of physiological response such as fMRI and EEG. These findings are important researches for the application such as Brain-Machine Interface (BMI) which estimates human emotion, and many fields in engineering are investigating to incorporate in the method of emotion estimation in many technological applications. In our laboratory, we are investigating about the relationship between emotional states and physiological response by measuring EEG, fNRIS, and Eye-movement including pupillary response. In addition, we are trying to estimate human emotion by machine learning using physiological response to extract the temporal variation of human emotion in the future.
One of the associations between space and emotion is the vertical emotion metaphor, which associates the upper and lower side with positive and negative emotions. Previous studies showed that this metaphor allows emotional processing to induce attention to vertical space. However, previous studies examined overt attention with a gaze. To examine the covert attention without a gaze, we investigated the effect of emotional processing on covert attention to the upper and lower side. In this experiment, we set up a background with different luminance between the upper and lower side. And, as a physiological index, we used pupil size that changes depending on the brightness of the target of covert attention without a gaze. The change in pupil diameter during the presentation of emotion sounds was examined. The results suggest that covert attention is not affected by the vertical emotion metaphor. Also, positive and negative sounds enhanced subsequent attention to the upper side. These effects could be related to the high arousal level of emotional sounds.
Emotional states and traits are generally tested by subjective assessment using a questionnaire. However, questionnaires are not suitable for children and developmentally disabled people who cannot understand questions. In this study, we propose a new measurement index, pupil diameter, which reflects the response to emotional stimuli. The purpose of this study was to estimate feeling states and traits based on pupillary responses during an oddball task using emotional sound.
Study on “Time Perception”
We sometimes feel time shorter and longer even for the same duration. For example, playing game makes us feel the time flies by and boring meeting makes the time longer. This kind of experience is familiar to everybody. This example shows our feeling of time is not constant like a clock. Although there are various theories as to the mechanism of subjective perception of time, and it is still not clear. To reveal this mechanism, our laboratory focus on time perception of short scale within a few seconds by investigating the relation between behavior data and eye movement data.
Hue, which is one of the elements of color, is known to affect temporal perception. Speed perception is affected by the direction of moving objects, however the effect of hue has not been clarified. In addition, prior research on temporal and speed perception has noted that experimental environments deviate from reality. In this study, virtual reality (VR), which is considered to be close-to-reality environment, was used to investigate whether the hue and direction of moving objects in VR space affect speed perception. In the experiment, participants were presented with moving cars in different hues approaching from either right or left. Subsequently, participants reported whether the car was perceived faster or slower than the reference target.
Study on “Illusion”
Our experiences of perceptual misunderstanding are sometimes occurred by optical illusions to create the visual image that can be deceptive interpretations. Optical illusions trick our brains into believing things which may or may not be real. This is not on retina level, but on the cortical level processing in our brain. This kind of “an error of brain” is very valuable to know how our brain interprets things what we perceive in the world. For example, Figure1 left is called glare illusion which has a luminance gradation converging on the central region and evokes brightness enhancement and self-luminosity in the center white area. Interestingly, previous studies have reported that while observers are viewing the glare illusion and monitored their pupil with an eye-tracker, which resulted in a larger pupil constriction compared to its control object (i.e. Figure1 right). Since these two objects consist of the identical element of gradation balls, when human look at the object imitating an emitting light, the whole network among several brain areas may activate involving our memory or experience to prevent our eyes as an adaptive response in terms of the pupil constriction. We are investigating the brain mechanisms to optical illusions based on psychological responses such as EEG and pupillometry.
Pupil size changes not only while observing a reflective object, but also while looking at photographs and paintings which convey the sense of brightness. However, previous research has not been able to segregate the physical factors such as glare effects from the cognitive factors associated with the images. To further investigate the influence of these factors, a pupillometry experiment was conducted using several ambiguous images that could be perceived as both the sun and the moon. As a result, pupils contracted significantly when the participants perceived the image as the sun, independent of whether the image was the sun or the moon. These results indicates that the pupil diameter varies depending on how the ambiguous image is being perceived.