Remote operation of highly automated vehicles (HAVs) may include occasional assistance from a human remote operator that is located outside the HAVs. Remote assistance typically delegates only highlevel guidance tasks to the remote operators such as authorizing a driving maneuver or specifying a new driving path. As remote assistance is fairly unexplored, there are still several research challenges. These challenges were discussed by experts from academia and industry in a multidisciplinary workshop at the 2023 IEEE Intelligent Vehicles Symposium. As a result of the workshop, this paper presents a list of most pressing research questions in the following areas: human-machine interaction and human factors, design of the remote station, design of the HAVs. It also outlines a roadmap for future research on remote assistance of HAV, thereby informing interdisciplinary studies and facilitating the benefits of HAVs before full autonomy can be reached.

Remote control technology for machines and robots has experienced significant advancement in many domains where visual information delivery is essential. Safety management at airports is one field that benefits from remote control systems, enabling operators to scan the airstrip for obstacles and debris. Depth perception and proper view position are critical in remote control systems, where operators need to accurately perceive 3D coordinates and maintain an appropriate perspective for performing tasks from a distance. This Demo paper presents a laboratory platform for an experimental study on human interaction with remote control systems using visual interfaces for inspection tasks to enhance airport safety management. The platform can evaluate the user experience of interfaces provided by First Person View, Third Person View, and Augmentation technologies. This platform enables exploration through controlled experiments and by user tests. These provide an avenue for assessing how these interfaces may affect human performance, depth perception, and user experience when conducting inspection tasks remotely. The findings will shed light on the strengths and limitations of each interface type, offering insights into their potential applications in various domains such as industrial inspection, surveillance, and remote exploration. 

Fully automated driving has posed more challenges than expected, and remote operation of heavy vehicles is increasingly getting attention. Therefore, human remote operators may have an essential role in compensating for the technological shortcomings in vehicle automation. This poses challenges in designing the work of human remote operators of automated heavy vehicles. This paper presents findings from a research project performed in collaboration between the RISE Research Institutes of Sweden and Scania. In the project, human-automation interaction requirements and challenges for remote operator work were explored through a simulator study. Before the study, three main operator tasks were defined: assessment, assistance, and remote driving. The simulation occurred in a transportation scenario where operators handled ten trucks driving on a public road and confined areas (transportation hub). Fifteen participants completed the study. The results provide examples and insights into classical automation-related challenges in a new context – the remote operation of heavy vehicles. Instances of challenges with situational awareness, out-of-the-loop, trust, and attention management were found and are discussed in relation to HMI design and requirements.

Autonomous shared ride vehicles may be prone to similar social issues and non-ideal passenger behaviors as today’s public transit. Such issues may include passengers littering, harassing others, and creating an environment that is generally unpleasant for riders. Transportation user experience designers should preemptively consider such scenarios early in their design work to help develop possible interfaces to manage social order and maintain good rider experience. Through a short video prototype, we present three possible non-ideal scenarios that may occur on shared autonomous shuttles and provide three potential solutions to begin a discussion around how to design for such non-ideal situations.

This study investigates interactive behaviors and communication cues of heavy goods vehicles (HGVs) and vulnerable road users (VRUs) such as pedestrians and cyclists as a means of informing the interactive capabilities of highly automated HGVs. Following a general framing of road traffic interaction, we conducted a systematic literature review of empirical HGV-VRU studies found through the databases Scopus, ScienceDirect and TRID. We extracted reports of interactive road user behaviors and communication cues from 19 eligible studies and categorized these into two groups: 1) the associated communication channel/mechanism (e.g., nonverbal behavior), and 2) the type of communication cue (implicit/explicit). We found the following interactive behaviors and communication cues: 1) vehicle-centric (e.g., HGV as a larger vehicle, adapting trajectory, position relative to the VRU, timing of acceleration to pass the VRU, displaying information via human-machine interface), 2) driver-centric (e.g., professional driver, present inside/outside the cabin, eye-gaze behavior), and 3) VRU-centric (e.g., racer cyclist, adapting trajectory, position relative to the HGV, proximity to other VRUs, eye-gaze behavior). These cues are predominantly based on road user trajectories and movements (i.e., kinesics/proxemics nonverbal behavior) forming implicit communication, which indicates that this is the primary mechanism for HGV-VRU interactions. However, there are also reports of more explicit cues such as cyclists waving to say thanks, the use of turning indicators, or new types of external human-machine interfaces (eHMI). Compared to corresponding scenarios with light vehicles, HGV-VRU interaction patterns are to a high extent formed by the HGV’s size, shape and weight. For example, this can cause VRUs to feel less safe, drivers to seek to avoid unnecessary decelerations and accelerations, or lead to strategic behaviors due to larger blind-spots. Based on these findings, it is likely that road user trajectories and kinematic behaviors will form the basis for communication also for highly automated HGV-VRU interaction. However, it might also be beneficial to use additional eHMI to compensate for the loss of more social driver-centric cues or to signal other types of information. While controlled experiments can be used to gather such initial insights, deeper understanding of highly automated HGV-VRU interactions will also require naturalistic studies. Copyright © 2022 Fabricius, Habibovic, Rizgary, Andersson and Wärnestål.

Small electric Autonomous Delivery Vehicles (ADV) can play an important role in future logistic chains under the last mile deliveries. In terminals where ADV are loaded with goods it is important that the interactions between the ADVS and the goods handling personnel is safe. Two workshops with developers of self-driving vehicles, researchers in the area of human-machine interaction and goods handling personnel form Postnord were conducted to identify challenges, needs and requirements regarding the design of ADV and the terminals för ADV. Due to COVID19, the workshops were carried out online and a video was shown to the participants demonstrating an ADV operating in a location representing a terminal. The two main objectives for this study were to gain understanding of the interactions between the ADV and human operators in the terminal and to identify high-level functional requirements for safe and efficient deployment of ADVs in terminals. The identified use cases related to (i) the ADV’s operations in the terminal, from entering to leaving the terminal and (ii) use cases where human operators interacted with the ADV, e.g. for loading/unloading goods. For each use case a high-level functional requirement was formulated. Human operators will most likely have important roles in delivery chains with ADV, such as loading and unloading of goods, as well as managing problems the ADV cannot solve. Consequently, how to design the human - ADV interactions will be critical from safety and efficiency points of view.

Autonomous vehicles are becoming a reality with great potential. However, persons with blindness, deafblindness, and deafness, who usually receive information and guidance from the driver, could miss information when travelling in an autonomous car. In this case study, 15 people with hearing and vision impairments explore and compare trips with and without vibrotactile guidance (using Ready-Ride or Ready-Move) in a simulated autonomous vehicle in real traffic (using a Wizard-of-Oz method). The study investigated if vibrotactile aid could enable persons with blindness, deafblindness, and deafness to use autonomous vehicles. Different phases of a trip (before, during, and after) were analysed. The study shows that people with functional impairments such as blindness, deafblindness, and deafness can perform trips independently if given information adapted to their needs through auditory, tactile, or visual information channels. It would be difficult for the target groups to travel without any additional communication aid, such as a vibrotactile guidance aid for all phases of the trip, especially for those with blindness. In all rides with the simulated autonomous car (Wizard-of-Oz set up) without vibrotactile guidance, the driver or assistant (in at least one phase of the trip) had to intervene for the research participants with blindness to complete the trip and continue the study. The study also highlights the usability of the vibrotactile guidance aid and identifies areas in need of improvement.

Reducing fuel consumption is one of the major benefits of platooning. While introducing platooning in mixed traffic, surrounding traffic will interfere with the platoon, risking a loss in fuel savings. In this work, a method for estimating potential fuel loss due to cut-ins in platoons is presented. Based on interviews with truck drivers with experience from platooning, and naturalistic data from previous research, we estimate the potential loss of fuel savings due to cut-ins and compare two scenarios with different amounts of traffic. The results show that platoons spend as much as 20% of time in cut-ins on typical European roads, reducing fuel savings in platooning from 13% down to 10%. Consequently, avoiding cut-ins has a positive environmental effect worth considering. 

Engaging in non-driving related tasks (NDRTs) while driving can be considered distracting and safety detrimental. However, with the introduction of highly automated driving systems that relieve drivers from driving, more NDRTs will be feasible. In fact, many car manufacturers emphasize that one of the main advantages with automated cars is that it “frees up time” for other activities while on the move. This paper investigates how well drivers are able to engage in an NDRT while in automated driving mode (i.e., SAE Level 4) in real traffic, via a Wizard of Oz platform. The NDRT was designed to be visually and cognitively demanding and require manual interaction. The results show that the drivers’ attention to a great extent shifted from the road ahead towards the NDRT. Participants could perform the NDRT equally well as when in an office (e.g. correct answers, time to completion), showing that the performance did not deteriorate when in the automated vehicle. Yet, many participants indicated that they noted and reacted to environmental changes and sudden changes in vehicle motion. Participants were also surprised by their own ability to, with ease, disconnect from driving. The presented study extends previous research by identifying that drivers to a high extent are able to engage in an NDTR while in automated mode in real traffic. This is promising for future of automated cars ability to “free up time” and enable drivers to engage in non-driving related activities.