Add Parameterized N-Player Social Dilemma Games with Dynamic and Stochastic Payoffs

[sriramsowmithri9807]

Description

OpenSpiel currently includes several matrix and social dilemma games that are primarily limited to fixed 2-player settings with static payoff structures. While these are valuable for theoretical analysis, many real-world multi-agent reinforcement learning (MARL) problems involve:

• More than two agents
• Changing payoff dynamics over time
• Stochastic or noisy rewards

This issue proposes adding a parameterized social dilemma game that generalizes existing matrix games to better support modern MARL research and benchmarking.

Proposed Change

Introduce a new game, tentatively named param_social_dilemma, with the following capabilities:

1. Variable Number of Agents (N-Player)

• Support configurable agent counts (N ≥ 2)
• Maintain compatibility with OpenSpiel’s simultaneous-move game API

2. Dynamic Payoff Matrices

• Allow payoff matrices to:
• Change across timesteps or episodes
• Be generated from parameterized functions
• Enable experiments on non-stationary environments

3. Stochastic Rewards

• Add optional reward noise (e.g., Gaussian or discrete distributions)
• Useful for robustness and exploration studies

4. Python API Exposure

• Expose all parameters via the Python interface:
• Number of agents
• Payoff configuration
• Stochastic reward settings

Implementation Location

Core game logic:

open_spiel/games/

Python bindings and API:

open_spiel/python/games/

Tests and examples:

open_spiel/tests/
open_spiel/python/examples/

Potential Deliverables

• New game implementation: param_social_dilemma
• Python bindings with clear documentation

Unit tests covering:

• Different agent counts

• Deterministic vs stochastic rewards

• Example script demonstrating MARL usage

Willingness to Contribute

I’m happy to take this up and iterate based on maintainer feedback, starting with a minimal version (static N-player game) and progressively adding dynamic and stochastic extensions.

[alexunderch]

Also, pay attention to

• call for new games: Call for New Games #843
• how to add a game: https://github.com/google-deepmind/open_spiel/blob/master/docs/developer_guide.md#adding-a-game

[sriramsowmithri9807]

@alexunderch sure!!

[lanctot]

Hi @sriramsowmithri9807,

Nice idea. If you work on this, please place the base implementation in open_spiel/games/param_social_dilemma and make the base game inherit from SimultaneousMoveGame and states from SimultaneousMoveState (unless you do it all in Python, in which case put it under open_spiel/python/games. See also the IPD in that python games directory.

Would be cool to support Alexrod-style IPD and have a few of the well-known bots implemented (Tit-for-Tat, Grim Trigger, Always-Cooperate, Always-Defect, etc.)

[sriramsowmithri9807]

Thanks for the guidance @lanctot and the encouragement really appreciate it.

I’ll start with a base implementation under open_spiel/games/param_social_dilemma, inheriting from SimultaneousMoveGame and SimultaneousMoveState, and keep the design modular so it can be extended from Python as well. I’ll also review the existing IPD implementation in open_spiel/python/games as a reference.

Supporting Axelrod-style IPD sounds great. I’ll aim to include a small set of well-known baseline strategies (e.g., Tit-for-Tat, Grim Trigger, Always-Cooperate, Always-Defect) as part of the initial contribution or a follow-up, depending on scope.

I’ll share a draft PR once the base version is ready and iterate based on feedback.

[sriramsowmithri9807]

@lanctot @alexunderch checkout pull request of #1432 and let me know the changes if needed!!

[sriramsowmithri9807]

@lanctot hey check out the #1457 pr i think i made the changes!!

[sriramsowmithri9807]

Hey @lanctot @alexunderch checkout the PR of #1432

[lanctot]

Hi @sriramsowmithri9807

#1432 and #1457 are both closed.

[lanctot]

Both of them are still showing thousands of files changed.

[lanctot]

Oh wait did you mean #1482 ?

[sriramsowmithri9807]

Oh wait did you mean #1482 ?

Yes

[lanctot]

The tests failed. Can you look into them?

BTW to run all the tests locally, you can run scripts/build_and_run_tests.sh

[alexunderch]

@sriramsowmithri9807 hey! nice pr, but can you run even local tests (like, for the game you've implemented) because current failures don't go further than wrong constructor arguments:

[4](https://github.com/google-deepmind/open_spiel/actions/runs/22118660893/job/63938051940?pr=1482#step:7:3845)
  File "/Users/runner/work/open_spiel/open_spiel/open_spiel/python/tests/games_sim_test.py", line 320, in test_multiplayer_game
    game = pyspiel.load_game(game_info.short_name, dict_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pyspiel.SpielError: Unknown parameter 'players'. Available parameters are: dynamic_payoffs, max_game_length, num_actions, num_players, payoff_change_prob, payoff_matrix, reward_noise_std

Submitting blindly won't really help unless you don't know how to fix your errors and asking for help to identify their causes.
Also, this is a good practice (my bad too, sometimes) to run all tests before the push: bash scripts/build_and_run_tests.sh
Hope it helps!