SAW 2025

Exploring the Role of Robot Interaction Style in Human-Robot Collaboration Conference Publication – In Press This preliminary work was presented at the 3rd Seeing and Acting Workshop (SAW 2025), an initiative sponsored by the EU’s Era Chair Action CogBooster and organized by the Faculty of Psychology and Educational Sciences of the University of Coimbra. Here, we presented the main study designed for the CoDRI project, envisioning continuous demeanor adaptation for collaborative robots, based on biofeedback from human workers. This work combines methodologies from two major fields, namely neurobiology and psychology, each with its own intricacies and dedicated approach. Given the complementarity of these fields, the goal is to construct meaningful feedback signalling during tasks where human and robot work together. Via reinforcement, AI methodology can then be used to control the robotic counterpart more autonomously. In terms of collaborative tasking, and considering the experimental nature of this work, simple cube stacking trials are planned to be shared between human worker and a Kinova robot arm. Objectives center around the construction of different configurations with the available material. In order to obtain neurophysiological data in a non-intrusive and minimally cumbersome way that can provide insight regarding the spatial activation of specific brain regions, the approach relies on fNIRS technology. Thus, an Artinis MK II headset with 10 transmitters and 8 receivers is provided for participants to wear during collaboration with the robot arm. Moreover, a 27-channel arrangement is employed to acquire signal over the prefrontal cortex (PFC), considering its variety of sub-areas associated with not only planning and executive decision, but also primal survival responses arising from the primitive limbic system. As for the psychological aspect of data collection, and in order to remain unobtrusive to ongoing collaboration, post-task surveying provides the necessary input to probe ergonomic impact. This surveying includes the filling out of the Frankenstein Syndrome Questionnaire, and the Human-Robot Interaction Trust and Perceived Safety scales, to gauge these factors and correlate their variation with observed activations of PFC sections. With both neurophysiological signalling demonstrating PFC activation, and user self-reported variation of human factors during collaborative tasks, it becomes possible to fit a mixed-effects model over the time-resolved neural data and the static ergonomic scoring. With this model, the natural next step is to bridge any observed correlations into feedback signal construction, which can adequately provide collaborative robots with information to adapt themselves and their behavior. To provide an objective example, during rapid non-anthropomorphic cube pick-up, left PFC activation may co-occur with a strong decrease in trust. Consequently, robot demeanor may incur a speed adaptation triggered by left PFC activation, with the goal of increasing or maintaining trust, at least. Replicating this experimental process, with variable conditions, tasks and observed cortical regions/ergonomic factors, is considered as a major goal of the CoDRI project. This is because it can lead to the production of a neuroergonomic effect mapping for human-robot collaboration, which by itself can mean considerable prospective applications involving feedback. Having presented the work at SAW, we then had the opportunity to attend the remaining sessions of the conference, with novel insight into topics tied to perception, cognitive mapping, navigational strategies, and several others.

fNIRS

Brief Overview of (f)NIRS Concept Knowledge In order to identify neural correlates triggered by specific HRI collaborative tasks, the project must resort to technology capable of probing neural activity. Here is where Brain-Computer Interfaces (BCI) come in. While most approaches typically rely on electroencephalography (EEG), this does not enable enough spatial granularity for activation to be associated with specific brain regions. Moreover, we still require BCI technology to be non-intrusive and non-cumbersome towards collaborative tasks. Thus, functional near-infrared spectroscopy (fNIRS) comes in handy. Unlike EEG, fNIRS technology uses optode transmitters and receivers, to measure light loss resultant from propagation in soft tissue. Different events, such as scattering and absorption, induce variation in the light travelling from emitter to receiver, and thus may be correlated with substance concentration in those tissues. More specifically, oxyhemoglobin Δ[𝑶2𝑯𝒃] and deoxyhemoglobin Δ[𝑯𝑯𝒃] concentrations are estimated over time, via their respective chromophores, in the surface area of the cerebral cortex where a cap with optodes is placed. By assuming tissue to remain unchanged, a modified Lambert-Beer law may be employed for such estimates. The idea behind concentration monitoring comes from the fact that, when neurons are active, they consume more oxygen. This factor leads to a local increase in deoxyhemoglobin and decrease in oxyhemoglobin. Consequentely, a signal can be considered, indicating the site and level of neural activity. To this end, optode transmitter-receiver arrangement and pair quantity dictate the number of channels on which concentrations are being estimated, thus influencing the granularity of  the activation mapping being generated. Pair distance is an impactful factor in fNIRS, as it determines the depth of brain tissue being measured and the proportion of the signal that comes from the scalp versus the brain. Here, distances are kept short (30mm), for cortex monitoring. Finally, optode arrangement should be placed over the cortical area meant to be monitored (e.g. prefrontal cortex or PFC). For each channel available, deoxyhemoglobin and oxyhemoglobin variation over time can be estimated, and thus a corresponding graph may be obtained. This is particularly useful when monitoring for localized activity, where the channel is placed. Evidently, when considering multi-channel arrangements, then a wider topographic mapping may be obtained for cortical activity, considering the localized variations of those concentrations. To learn more about fNIRS and the theory behind this technology, we suggest visiting Artinis’ post and suggested papers on the topic. In CoDRI, this level of cortical activation knowledge can be extremely useful as it pinpoints which area is undergoing more variation, while ergonomic factors of human-robot interaction fluctuate. Thus, correlations between spatial activation and ergonomic variation can be extracted, based on which appropriate feedback can then be generated for robots to adapt their demeanours. All this is possible as fNIRS provides enough temporal and spatial resolution, without hindering user mobility. Unlike techniques which typically explore brain activity in a spatial manner yet restrict the user, such as fMRI, fNIRS caps enable other activities to be performed, while signal is being acquired, thus making it ideal for collaborative robotics tasks. Here is a video of a live measurement, on a subject solving a 5×5 Rubik’s cube, for 2min30sec: https://codri.isr.uc.pt/wp-content/uploads/2025/10/fNIRS_mohammad_compressed.mp4

NEI

European Researcher’s Night Scientific Dissemination On September 26th, Bruno and Gustavo participated in the Coimbra chapter of the European Researchers’ Night, an initiative of the European Commission that offers a glimpse into the fascinating world of science. In this evening, attendees have the opportunity to immerse themselves in scientific projects by participating in games, discussions, experiments, and various other activities. It is also a great opportunity to engage with local researchers and get to know the work being done in the city. Public interest proved to be substantial, across all age buckets and backgrounds, given the constant and diverse stream of people stopping and, more importantly, staying by our table to listen to CoDRI’s frames of action and vision for safer, more sustainable and human-centered industry automation. Below you can find some of the feedback received during this event:

Exp.At 2025

EEG-triggered Awareness for a Collaborative Robot Conference Publication – In Press In this work, recently presented and published as part of the proceedings of IEEE’s 7th Experiment@ International Conference (exp.at’ 2025), in island of Faial, Portugal, we present a cobot capable of switching its operations between task completion and user tracking, based on data broadcast by a single-channel EEG headset (NeuroSky MindWave) worn by the user. The purpose of this adaptive application is to evaluate the impact on the user of having a cognition-sensitive robotic colleague, pondering ramifications towards danger incidence and productivity improvement. Given how awareness, or the ability of human workers to remain conscious of their surrounding operatives, is a strong contender to an ideal safety-productivity conciliation, the key aspect of this approach rests on being able to determine whether a user is aware of their actions and surroundings or not. In terms of electroencephalographic signalling, the partition of acquisitions into different frequencies determines the type of wave under scrutiny. Specifically, alpha wave activity has been extensively shown to decrease as attention enhances, in both human and animal studies. Thus, continuous monitoring can enable the modulation of robot activity in response to sustained human attention, or its lack thereof. While performing a collaborative task, and considering that the robot can determine worker attention via EEG signal interpretation, it should also demonstrate this knowledge in a manner perceptible to the user. Thus, imposing task completion contingent on sustained attention, the robot can also switch to a tracking demeanor while the human-directed part of a task is completed. By detecting the user body and translating its corresponding pose’s 2D coordinates into mirrored 3D joint twist velocities, the robot can demonstrate an enchanted snake-like behavior, focusing on worker actions. Naturally, this resorts to a body pose detector and corresponding coordinate homography, then normalized and used to scale joint velocities, which maintain robot movement. Ideally, human workers experiencing this adaptiveness can experience a reduction in uneasiness when working in proximity with industrial machinery and collaborative robots. There is value in this approach, as it reduces the sense of immediate danger, improves overall well-being, and possibly also reduces the rate of accidents. Naturally, positive effects are only expected to happen with habituation, as users become more aware of the collaborative robot’s interpersonal skills. Eventually, it becomes natural for human users to expect these robotic counterparts to ensure safety, much like human peers, thus fostering a healthier work environment. To exemplify the dynamic, here we see a young researcher experiencing a simplified EEG-based modulation of a simulated Kinova, at exp.at’s demo session.  To find out more, please read the publication online, soon available on IEEExplore.

COTEC – Demonstração

Demo at COTEC Summit Scientific Dissemination In the past May 14th, Bruno Ferreira presented a part of CoDRI’s work at the XVIII COTEC Europe Summit, which aimed to bring together business leaders, academia and policy decision makers to discuss how to proceed in the most concrete and agile way towards a new path for the future of European competitiveness. Here, he was able to showcase one of CoDRI’s most recent contributions – EEG-triggered Awareness for a Collaborative Robot. This work was recently accepted for publication in Exp.at 2025, an IEEE sponsored conference which is set to take place in the island of Faial, Portugal, this coming September. Envisioning automated safety procedures, itt proposes the usage of an attention score based on alpha wave activity, obtained via a single-channel EEG BCI, as indicative of human awareness towards robot movement. Here, the robot was set to track the person’s movement during a task, idealizing an industrial collaborative scenario, while they are considered aware. Otherwise, the robot should halt so as to ensure no harm may come to a worker oblivious of their surroundings. The demonstration went perfectly, allowing us to demonstrate how human signalling can indeed be used to modulate cobot behavior, with advantages towards human well-being. It is our hope that more events like this will help us spark interest in robot adaptation through integration of neuronal and psychological data, towards better, more-human like interactions with robotic agents. If you want to find out more, read this work’s detailed post.