3D Pedestrian Visualisation and Behaviour Modelling in Road-Crossing Scenarios

Please look at the website of the host laboratory SATIE by https://satie.ens-paris-saclay.fr/fr/presentation

and the website of UGE by https://fr.wikipedia.org/wiki/Universit%C3%A9_Gustave-Eiffel

As cities move toward increasingly smart and sustainable, particularly with integration of autonomous vehicles (AVs), ensuring the safety of vulnerable road users such as pedestrians has become a critical priority. While existing research on pedestrian simulation in virtual reality (VR) has largely focused on trajectory-based behaviour models (typically relying on position and velocity information), less attention has been given to pedestrians’ non-verbal behaviours. These include eye contact, intent-expressive gestures, and subtle postural cues when interacting with vehicles in road-crossing situations. The credibility of VR-based simulation is crucial, as the way pedestrians are modelled to express behaviour and intent directly shapes how drivers perceive and respond to different traffic situations. Enhancing the perceptual realism of virtual pedestrians—particularly in terms of their kinematic motion and intent expression—is therefore expected to lead to more natural and accurate human responses in simulated environments, closer to real-world interactions.
The objective of this internship is to develop a 3D skeleton-based pedestrian behaviour modelling pipeline that explicitly incorporates road-crossing intentions through kinematic representations of non-verbal cues. The project will focus on three main objectives: 1) Extraction of pedestrian behaviour data associated with diverse road-crossing intentions (e.g., crossing, waiting, looking, gesturing) from existing annotated datasets such as JAAD  and PIE ; 2) Reconstruction of temporally consistent 3D human skeletal motion from 2D visual data using state-of-the-art pose estimation and pose lifting techniques; and 3) Automatic reproduction and control of kinematic movements — including head rotation, hand gestures, and posture adjustments — by driving a 3D pedestrian avatar through scripted animation commands in a VR environment. 
The outcome of this internship will provide a foundational framework for more socially aware and expressive pedestrian agents in VR-based driving simulations. The developed pipeline and insights will serve as a key stepping stone toward a subsequent PhD project focusing on socially interactive agent modelling and immersive simulation of pedestrian–vehicle interactions, with direct applications in road safety research and human-centred evaluation of automated driving systems. 

Reference

Rasouli, A., Kotseruba, I. and Tsotsos, J.K., (2017). Are they going to cross? a benchmark dataset and baseline for pedestrian crosswalk behavior. In Proceedings of the IEEE international conference on computer vision workshops (pp. 206-213).

Rasouli, A., Kotseruba, I., Kunic, T. and Tsotsos, J.K., (2019). Pie: A large-scale dataset and models for pedestrian intention estimation and trajectory prediction. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 6262-6271).

Rock, T., Himmels, C., Peintner, J., Manger, C., & Cao, H. (2022). Realistic pedestrian models integrating motion-captured gestures of real humans. In 8th Symposium Driving Simulation (pp. 11-16). Automotive Solution Center for Simulation.

Jan, Q.H., Badella, Y.S. and Berns, K., (2024). Detailed Analysis of Pedestrian Activity Recognition in Pedestrian Zones Using 3D Skeleton Joints Using LSTM. SN Computer Science, 5(2), p.242.

Camara, F., Bellotto, N., Cosar, S., Weber, F., Nathanael, D., Althoff, M., Wu, J., Ruenz, J., Dietrich, A., Markkula, G., Schieben, A., Tango, F., Merat, N., & Fox, C. (2021). Pedestrian Models for Autonomous Driving Part II: High-Level Models of Human Behavior. IEEE Transactions on Intelligent Transportation Systems, 22(9), 5453–5472. https://doi.org/10.1109/TITS.2020.3006767

Myers C, Zane T, Van Houten R, Francisco VT. (2022). The effects of pedestrian gestures on driver yielding at crosswalks: A systematic replication. J Appl Behav Anal. 2022 Mar; 55(2):572-583. doi: 10.1002/jaba.905

We are looking for a motivated M2 (Master’s level) student with a strong interest in graphics, virtual reality, human behaviour modelling, and data-driven approaches. The ideal candidate should demonstrate:

(1) Programming experience in Python, with additional familiarity in Unity or Unreal Engine being a strong advantage; (2) Basic skills of data processing and machine learning; (3) Basic knowledge of 3D geometry and kinematics.