Skip to content

[BUG]: vendor.nvidia.dcgm.enabled: false has no effect — DCGM prolog always wired into prologScriptRefs #210

Description

@basilnsage

Version

1.1.0

Which installation method(s) does this occur on?

Helm / Kubernetes manifests

Describe the bug.

Setting vendor.nvidia.dcgm.enabled: false (the default) does not prevent the DCGM prolog/epilog scripts from being added to prologScriptRefs/epilogScriptRefs in the Controller CR. As a result, the Slinky operator always injects Prolog=prolog-90-dcgm.sh and PrologFlags=Contain into slurm.conf, even on CPU-only clusters where DCGM is not intended to be used. This prevents workers from starting successfully on some CPU environments (minikube on macOS here).

Minimum reproducible example

# Render the chart with dcgm explicitly disabled (the default)
helm template my-cluster oci://ghcr.io/slinkyproject/charts/slurm \
  --version 1.1.0 \
  --set vendor.nvidia.dcgm.enabled=false \
  | grep -E "prologScriptRefs|epilogScriptRefs|dcgm"

# Observed output — DCGM refs are present despite enabled=false:
#   prologScriptRefs:
#     - name: my-cluster-slurm-prolog-dcgm
#   epilogScriptRefs:
#     - name: my-cluster-slurm-epilog-dcgm

# Also confirm with enabled= (empty string, the only working workaround):
helm template my-cluster oci://ghcr.io/slinkyproject/charts/slurm \
  --version 1.1.0 \
  --set "vendor.nvidia.dcgm.enabled=" \
  | grep -E "prologScriptRefs|epilogScriptRefs|dcgm"

# With empty string the refs are correctly absent.

Relevant log output

[2026-06-15T23:55:53] error: prolog failed: rc:127 output:[2026-06-15T23:55:53] error: _run_command_child_exec: execv(/usr/sbin/slurmd): No such file or directory

Additional environment details

Helm: 3.x
Kubernetes: minikube v1.35 (single-node, cgroup v2, IgnoreSystemd=yes)
slurm chart: 1.1.0
Slinky operator: 1.1.0
Slurm: 26.05.0

Other/Misc.

No response

Code of Conduct

  • I agree to follow Slurm Operator's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions