Social-Navigation-PyEnvs

A framework used to train robots within a social navigation context with a wide range of human motion models to simulate crowds of pedestrians.

Description

This repository contains a framework developed starting from CrowdNav [1] and Python-RVO2 [2] used to train and test learning-based algorithms for Social Navigation.

In order to simulate crowds of pedestrians the following models are implemented:

• Social Force Model (SFM) [3] and its variations [4], [5]
• Headed Social Force Model (HSFM) [6]
• Optimal Reciprocal Collision Avoidance (ORCA) [7]

The CrowdNav module [1] includes the following reinforcement learning algorithms for social robot navigation:

• Collision Avoidance with Deep RL (CADRL) [8]
• Long-short term memory RL (LSTM-RL) [9]
• Social Attentive RL (SARL) [10]

The simulator is built upon Pygame in order to provide a functional visualization tool and OpenAI Gym, which defines the standard API for RL environments. It also implements a laser sensor and a differential drive robot, which allow users to develop sensor-based algorithms.

Simulation videos: a comparative study of human motion models in reinforcement learning algorithms for social robot navigation

Cite this paper

If this repository or paper turns out to be useful for your research, please cite our paper:

@article{van2025comparative,
  title={A Comparative Study of Human Motion Models in Reinforcement Learning Algorithms for Social Robot Navigation},
  author={Van Der Meer, Tommaso and Garulli, Andrea and Giannitrapani, Antonio and Quartullo, Renato},
  journal={ACM Transactions on Human-Robot Interaction},
  year={2025},
  publisher={ACM New York, NY}
}

Human motion models

ORCA	SFM	HSFM

Baselines

BP	SSP	ORCA

Comparing policies

CADRL-HS-HSFM	LSTM-RL-HS-HSFM	SARL-HS-HSFM

Comparing scenarios

SARL-CC-HSFM	SARL-PT-HSFM	SARL-HS-HSFM

Comparing human motion models for training and testing

SARL-HS-ORCA	SARL-HS-SFM	SARL-HS-HSFM

Case study: evaluating scenarios with obstacles

SARL-CC-HSFM

References

• [1] CrowdNav.
• [2] Python-RVO2.
• [3] Helbing, D., Farkas, I., & Vicsek, T. (2000). Simulating dynamical features of escape panic. Nature, 407(6803), 487-490.
• [4] Moussaïd, M., Helbing, D., Garnier, S., Johansson, A., Combe, M., & Theraulaz, G. (2009). Experimental study of the behavioural mechanisms underlying self-organization in human crowds. Proceedings of the Royal Society B: Biological Sciences, 276(1668), 2755-2762.
• [5] Guo, R. Y. (2014). Simulation of spatial and temporal separation of pedestrian counter flow through a bottleneck. Physica A: Statistical Mechanics and its Applications, 415, 428-439.
• [6] Farina, F., Fontanelli, D., Garulli, A., Giannitrapani, A., & Prattichizzo, D. (2017). Walking ahead: The headed social force model. PloS one, 12(1), e0169734.
• [7] Van Den Berg, J., Snape, J., Guy, S. J., & Manocha, D. (2011, May). Reciprocal collision avoidance with acceleration-velocity obstacles. In 2011 IEEE International Conference on Robotics and Automation (pp. 3475-3482). IEEE.
• [8] Chen, Y. F., Liu, M., Everett, M., & How, J. P. (2017, May). Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 285-292). IEEE.
• [9] Everett, M., Chen, Y. F., & How, J. P. (2018, October). Motion planning among dynamic, decision-making agents with deep reinforcement learning. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3052-3059). IEEE.
• [10] Chen, C., Liu, Y., Kreiss, S., & Alahi, A. (2019, May). Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning. In 2019 international conference on robotics and automation (ICRA) (pp. 6015-6022). IEEE.