Skip to content

feat: kubernetes support for tls/x509 redis and more modernizations#349

Open
bradymiller wants to merge 21 commits intoopenemr:masterfrom
bradymiller:redis-tls-kub
Open

feat: kubernetes support for tls/x509 redis and more modernizations#349
bradymiller wants to merge 21 commits intoopenemr:masterfrom
bradymiller:redis-tls-kub

Conversation

@bradymiller
Copy link
Copy Markdown
Member

@bradymiller bradymiller commented Jan 30, 2023

fixes #347

Summary

Comprehensive security and architecture overhaul of the Kubernetes deployment, replacing the old Pipy proxy layer with direct mTLS connections and adding production-quality infrastructure.

mTLS by default (MariaDB + Redis)

  • All connections use mutual TLS (X509 client certificate verification) by default — Redis, Sentinel, MariaDB replication, OpenEMR-to-database, and phpMyAdmin-to-database
  • Certificates managed by cert-manager (upgraded from v1.11.0 to v1.20.2)
  • Clear downgrade instructions in README for TLS-only or plain TCP modes

Redis Sentinel with phpredis

  • Removed Pipy proxy layer — OpenEMR connects directly via SESSION_STORAGE_MODE=predis-sentinel using native phpredis with distributed session locking
  • Sentinel discovery for automatic failover (~1s recovery, tested)
  • Standardized sentinel port to 26379
  • Sentinel requirepass for auth even when mTLS is downgraded
  • Added replica-announce-ip so sentinel returns hostnames (required for TLS cert validation)
  • Three Redis ACL users with least-privilege: default (app sessions), replication (replica sync), admin (sentinel monitoring)

Secrets management

  • Moved all hardcoded passwords from ConfigMaps to Kubernetes Secrets (redis-credentials, mysql-replication-credentials)
  • ConfigMaps use placeholder tokens; init containers inject passwords at runtime via sed
  • OpenEMR uses secretKeyRef for all password env vars

Multi-node support

  • Replaced broken mauilion/hostpath-provisioner with in-cluster NFS provisioner (NFS Ganesha)
  • PVCs upgraded from ReadWriteOnce to ReadWriteMany with nfs StorageClass
  • Simplified Kind 4-node config (no more shared hostPath mounts)
  • Added Kind 1-node config with port mappings

Security hardening

  • NetworkPolicies: default-deny ingress with explicit allow rules per service (OpenEMR, MySQL, Redis, Sentinel, phpMyAdmin, NFS)
  • automountServiceAccountToken: false on all pods
  • phpMyAdmin changed from NodePort to ClusterIP (access via kubectl port-forward only)
  • OpenEMR service changed from LoadBalancer to NodePort with fixed ports (30080/30443)
  • Sentinel cert updated with proper dnsNames (SANs)

Infrastructure improvements

  • Liveness/readiness probes on all pods (Redis, Sentinel, MySQL, phpMyAdmin, OpenEMR)
  • busybox 1.28 → 1.37
  • OpenEMR image updated to 8.1.1 (requires 8.1.0+)
  • kub-up uses kubectl wait for cert-manager and node readiness instead of fixed sleeps
  • Kind configs map ports to localhost (http://localhost:8800, https://localhost:9800)

README

  • Updated architecture description and example output
  • MariaDB and Redis Connection Security sections with downgrade instructions (mTLS → TLS → TCP)
  • Redis Sentinel failover testing instructions
  • phpMyAdmin access via port-forward
  • Removed all references to deleted proxy files

Test plan

  • 1-node Kind cluster: all pods running, OpenEMR accessible
  • 4-node Kind cluster: pods distributed across nodes, NFS shared volumes working
  • Scaled OpenEMR to 10 replicas — sessions maintained across all pods
  • Redis failover: deleted master pod, sentinel promoted replica in ~1s, OpenEMR continued working
  • mTLS verified: Redis, Sentinel, and MariaDB connections all using X509 client certs

Comment thread kubernetes/redis/statefulset-redis.yaml Outdated
echo "Copying configuration file"
cp /tmp/redis/redis.conf /etc/redis/redis.conf
if [ "$(redis-cli -h sentinel -p 5000 ping)" != "PONG" ]; then
if [ "$(redis-cli --tls --cacert /certs/ca.crt -h sentinel -p 5000 ping)" != "PONG" ]; then
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue will probably crop up here if forcing x509, which can deal with when can test in future

Comment thread kubernetes/redis/statefulset-redis.yaml Outdated
else
echo "Sentinel found, finding master"
MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
MASTER="$(redis-cli --tls --cacert /certs/ca.crt -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue will probably crop up here if forcing x509, which can deal with when can test in future

for i in ${nodes//,/ }
do
MASTER=$(redis-cli --no-auth-warning --raw -h $i --user admin -a $REDIS_PASSWORD info replication | awk '{print $1}' | grep master_host: | cut -d ":" -f2)
MASTER=$(redis-cli --tls --cacert /certs/ca.crt --no-auth-warning --raw -h $i --user admin -a $REDIS_PASSWORD info replication | awk '{print $1}' | grep master_host: | cut -d ":" -f2)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue will probably crop up here if forcing x509, which can deal with when can test in future

@bradymiller bradymiller self-assigned this Jan 30, 2023
@bradymiller bradymiller changed the title Kubernete: Support for tls/x509 redis session connections (need to wait for next phpredis version release) Kubernete: Support for tls/x509 redis session connections Apr 9, 2023
@Jmevorach
Copy link
Copy Markdown
Contributor

Good news @bradymiller; the latest release of PHPRedis v6.0.2 (released on 12/16/23) has support for TLS connections and sentinel mode! This should satisfy 6 and 7 on the list above.

Additionally there's some good guidance on how to deploy a web application with Redis without the use of haproxy to Kubernetes that can be found here.

It's also possible to use nginx + cert-manager to set up encryption between services (i.e. OpenEMR and Redis) within a cluster as well which may eliminate the need entirely to implement x.509 connections for Redis by going down the current path.

Finally you should be able to configure a readiness probe within the cluster to detect when the OpenEMR containers are ready to serve traffic. You may be able to specify something like cat /root/instance-swarm-ready as the command for the readiness probe.

Hope this is helpful!

Generated-By: Claude
Generated-By: Claude
Generated-By: Claude
Generated-By: Claude
Generated-By: Claude
Generated-By: Claude
Generated-By: Claude
Generated-By: Claude
@bradymiller bradymiller marked this pull request as ready for review April 22, 2026 07:10
Copilot AI review requested due to automatic review settings April 22, 2026 07:10
@bradymiller bradymiller changed the title Kubernete: Support for tls/x509 redis session connections feat: kubernetes support for tls/x509 redis and more modernizations Apr 22, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Kubernetes deployment manifests to move Redis session storage to Sentinel-based failover with TLS/X509 (mTLS) and to support multi-node operation via ReadWriteMany volumes, along with additional hardening (network policies, probes, no SA token mounts).

Changes:

  • Add TLS/X509-enabled Redis + Sentinel configuration (and remove the redisproxy layer), wiring OpenEMR to Sentinel mode.
  • Introduce an in-cluster NFS provisioner + StorageClass and switch PVCs to ReadWriteMany.
  • Add default-deny ingress NetworkPolicies and update service exposure defaults (OpenEMR NodePort, phpMyAdmin ClusterIP/port-forward workflow).

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
kubernetes/volumes/website.yaml Switch PVC to NFS storage class + RWX.
kubernetes/volumes/ssl.yaml Switch PVC to NFS storage class + RWX.
kubernetes/volumes/letsencrypt.yaml Switch PVC to NFS storage class + RWX.
kubernetes/redis/statefulset-sentinel.yaml Generate TLS-enabled sentinel.conf and mount certs/secrets.
kubernetes/redis/statefulset-redis.yaml Inject Redis/ACL secrets, add TLS cert mounts and probes.
kubernetes/redis/service-sentinel.yaml Move sentinel service port to 26379.
kubernetes/redis/service-redisproxy.yaml Remove redisproxy Service.
kubernetes/redis/secret.yaml Add Secret holding Redis ACL passwords.
kubernetes/redis/deployment-redisproxy.yaml Remove redisproxy Deployment.
kubernetes/redis/configmap-pipy.yaml Remove pipy redisproxy ConfigMap.
kubernetes/redis/configmap-main.yaml Enable TLS-only Redis port and TLS client auth defaults.
kubernetes/redis/configmap-acl.yaml Template Redis ACL users for secret injection.
kubernetes/phpmyadmin/service.yaml Change phpMyAdmin to ClusterIP and rename ports.
kubernetes/phpmyadmin/deployment.yaml Add probes, disable SA token, and mount client cert/key.
kubernetes/phpmyadmin/configmap.yaml Force phpMyAdmin to use MySQL client cert/key.
kubernetes/openemr/service.yaml Switch OpenEMR to NodePort with fixed ports.
kubernetes/openemr/deployment.yaml Configure Sentinel session mode + Redis TLS/X509 envs; add probes/certs.
kubernetes/nfs/storageclass.yaml Add NFS StorageClass (openemr.org/nfs).
kubernetes/nfs/service.yaml Add Service exposing NFS provisioner ports.
kubernetes/nfs/rbac.yaml Add RBAC for the NFS provisioner.
kubernetes/nfs/deployment.yaml Add NFS provisioner Deployment + probes.
kubernetes/network/policies.yaml Add default-deny ingress + allow-list policies.
kubernetes/mysql/statefulset.yaml Inject replication password via secret; add probes; disable SA token.
kubernetes/mysql/replication-secret.yaml Add Secret for replication password.
kubernetes/mysql/configmap.yaml Template replication password and enforce X509 for replication.
kubernetes/kub-up.bat Add NFS + new certs/secrets + wait steps; remove redisproxy apply.
kubernetes/kub-up Add NFS + new certs/secrets + wait steps; remove redisproxy apply.
kubernetes/kub-down.bat Mirror resource deletions for new components and remove redisproxy deletions.
kubernetes/kub-down Mirror resource deletions for new components and remove redisproxy deletions.
kubernetes/kind-pvc-hostpath.yaml Remove hostpath provisioner manifests.
kubernetes/kind-config-4-nodes.yaml Replace hostpath mounts with NodePort mappings for kind.
kubernetes/kind-config-1-node.yaml Add 1-node kind config with NodePort mappings.
kubernetes/certs/sentinel.yaml Add cert-manager Certificate for Sentinel.
kubernetes/certs/redis.yaml Add cert-manager Certificate for Redis pods.
kubernetes/certs/redis-openemr-client.yaml Add cert-manager client Certificate for OpenEMR→Redis/Sentinel.
kubernetes/README.md Document mTLS defaults and downgrade paths; update kind/minikube usage.
docker/openemr/obsolete/7.0.2/openemr.sh Remove obsolete phpredis build support block.
docker/openemr/obsolete/7.0.1/openemr.sh Remove obsolete phpredis build support block.
README.md Update Kubernetes bullet description to reflect new capabilities.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +56 to 57
if [ "$(redis-cli $TLSPARAMETERS -h sentinel -p 26379 ping)" != "PONG" ]; then
echo "Sentinel not found to get the master info, defaulting to redis-0"
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

redis-cli ... ping is executed without authentication, but sentinel is configured with requirepass $REDIS_DEFAULT_PASSWORD. This will return NOAUTH and cause the script to incorrectly treat sentinel as unavailable. Add the appropriate AUTH (e.g., -a $REDIS_DEFAULT_PASSWORD, and user if needed) to this call.

Copilot uses AI. Check for mistakes.
Comment thread kubernetes/openemr/deployment.yaml Outdated
Comment on lines +22 to +26
image: busybox:1.37
command: ['sh', '-c', "until nslookup mysql.default.svc.cluster.local; do echo waiting for mysql; sleep 2; done"]
- name: init-redis-wait
image: busybox:1.28
command: ['sh', '-c', "until nslookup redis.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for redis; sleep 2; done"]
image: busybox:1.37
command: ['sh', '-c', "until nslookup redis.default.svc.cluster.local; do echo waiting for redis; sleep 2; done"]
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These initContainer nslookup checks hardcode the default namespace (mysql.default.svc.cluster.local / redis.default.svc.cluster.local). This will fail if the manifests are applied in any other namespace. Prefer using the in-namespace service names (mysql / redis) or inject POD_NAMESPACE via the Downward API and build the FQDN from that.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines 66 to 68
echo "Sentinel found, finding master"
MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
MASTER="$(redis-cli $TLSPARAMETERS -h sentinel -p 26379 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
echo "Master got: $MASTER, updating this in redis.conf"
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sentinel get-master-addr-by-name is queried without authenticating to sentinel. With requirepass enabled, this will fail and replication configuration will fall back to a potentially wrong master. Authenticate this redis-cli call the same way OpenEMR will (password, and username if applicable).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread kubernetes/network/policies.yaml Outdated
Comment on lines +118 to +121
ingress:
- ports:
- port: 80
- port: 443
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ingress rule specifies only ports and no from, which in NetworkPolicy semantics means "allow from all sources". If you intend to block other pods from reaching phpMyAdmin, add an appropriate from restriction (or omit ingress entirely to deny all).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread kubernetes/redis/statefulset-redis.yaml Outdated
Comment on lines +51 to +55
echo "Generating ACL file from template"
cp /tmp/acl/users.acl /conf/acl/users.acl
sed -i "s|__ADMIN_PASSWORD__|$REDIS_ADMIN_PASSWORD|g" /conf/acl/users.acl
sed -i "s|__REPLICATION_PASSWORD__|$REDIS_REPLICATION_PASSWORD|g" /conf/acl/users.acl
sed -i "s|__DEFAULT_PASSWORD__|$REDIS_DEFAULT_PASSWORD|g" /conf/acl/users.acl
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These sed substitutions write Secret values into the ACL template without escaping replacement metacharacters (notably & and backslashes). If a password includes these characters, users.acl will be malformed and Redis authentication will fail. Escape before substituting or generate the file with a tool that handles arbitrary values.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread kubernetes/network/policies.yaml Outdated
Comment on lines +105 to +107
# phpMyAdmin: allow ingress only from localhost/port-forward (no external)
# ClusterIP service already restricts this, but this policy ensures
# no other pod can reach phpMyAdmin either
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment here says this policy prevents other pods from reaching phpMyAdmin, but the actual rule below does not restrict sources (no from). As written, it allows any pod to connect on 80/443. Update the comment or the policy so they match.

Suggested change
# phpMyAdmin: allow ingress only from localhost/port-forward (no external)
# ClusterIP service already restricts this, but this policy ensures
# no other pod can reach phpMyAdmin either
# phpMyAdmin: allow ingress on http/https
# ClusterIP service limits external exposure, but this policy does
# not restrict which pods in the cluster may connect on 80/443

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread kubernetes/mysql/statefulset.yaml Outdated
cp /mnt/config-map/secondary.sql /docker-entrypoint-initdb.d
fi
# Inject replication password from secret
sed -i "s|__REPL_PASSWORD__|$MYSQL_REPL_PASSWORD|g" /docker-entrypoint-initdb.d/*.sql
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sed substitution will break for replication passwords containing characters with special meaning in sed replacement strings (notably &, backslashes, and potentially the chosen delimiter). Since this comes from a Secret, it should be treated as arbitrary bytes. Consider escaping the value before substitution or using a safer templating approach (e.g., envsubst/template file generation).

Suggested change
sed -i "s|__REPL_PASSWORD__|$MYSQL_REPL_PASSWORD|g" /docker-entrypoint-initdb.d/*.sql
escaped_repl_password=$(printf '%s' "$MYSQL_REPL_PASSWORD" | sed 's/[\\&|]/\\&/g')
sed -i "s|__REPL_PASSWORD__|$escaped_repl_password|g" /docker-entrypoint-initdb.d/*.sql

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines +46 to +50
echo "Copying configuration file and injecting passwords"
cp /tmp/redis/redis.conf /etc/redis/redis.conf
sed -i "s|__REPLICATION_PASSWORD__|$REDIS_REPLICATION_PASSWORD|g" /etc/redis/redis.conf
MY_FQDN="$(hostname).redis"
echo "replica-announce-ip $MY_FQDN" >> /etc/redis/redis.conf
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sed injection into redis.conf uses an unescaped Secret value. Passwords containing &, backslashes, or the chosen delimiter can corrupt the rendered config. Escape the value before substitution or use a safer templating mechanism.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines +35 to +39
# To downgrade to tls (no client certs): change to false
REDISX509=true
nodes=redis-0.redis,redis-1.redis,redis-2.redis
TLSPARAMETERS="--tls --cacert /certs/ca.crt"
if $REDISX509; then
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment implies TLS-only downgrade can be done by setting REDISX509=false, but the generated sentinel config later still hard-codes tls-auth-clients yes (which requires client certs). Update the comment to mention the additional required changes, or make the generated tls-auth-clients/cert/key lines conditional on REDISX509.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@bradymiller
Copy link
Copy Markdown
Member Author

bradymiller commented Apr 22, 2026

Just crazy, was able to resurrect this PR in one day with claude code. Testing well. Just some more housekeeping stuff to do (remove those changes in the old dockers and maybe some changes in the new dockers to clarify things; change mariadb tag to most recent long term release).

When 8.1.0 is released, can then bring this in (and will change the openemr docker tag from dev to 8.1.0 at that time)

@bradymiller
Copy link
Copy Markdown
Member Author

bradymiller commented Apr 22, 2026

TODO:

  • (remove those changes in the old dockers and maybe some changes in the new dockers to clarify things). - DONE
  • address copilot issues - DONE
  • run and address Aisle issues - DONE
  • Changed openemr tag to 8.1.0 - DONE
  • change mariadb tag to most recent long term release - STILL TODO
  • Final Testing - STILL TODO
  • When 8.1.0 is released, can then bring this in - STILL TODO

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 10 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +80 to 85
# To downgrade mysql connection security:
# tls (no client certs): change FORCE_DATABASE_X509_CONNECT to FORCE_DATABASE_SSL_CONNECT
# and remove mysql cert/key volume mounts below
# tcp (no encryption): also remove the env var above and remove mysql ca volume mount below
image: openemr/openemr:dev
name: openemr
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description/README indicate the deployment targets OpenEMR 8.1.x (and specifically mention updating to 8.1.1), but this manifest deploys openemr/openemr:dev. Using :dev can be incompatible with the documented env var/cert conventions and makes the deployment non-reproducible. Consider pinning to the intended released tag (e.g., 8.1.1) or updating the PR description/README to explicitly require :dev.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will change to 8.1.0 (night getting nightly build with changes to work with this)

Comment on lines +6 to +8
storageClassName: nfs
accessModes:
- ReadWriteOnce
- ReadWriteMany
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accessModes list item is not indented under accessModes:. This makes the PVC manifest invalid YAML. Indent - ReadWriteMany so it is nested under accessModes:.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

incorrect

Comment on lines 33 to +58
@@ -45,6 +52,10 @@ spec:
# On replicas use secondary configuration on initdb volume
cp /mnt/config-map/secondary.sql /docker-entrypoint-initdb.d
fi
# Inject replication password from secret (awk for safe handling of special chars)
for f in /docker-entrypoint-initdb.d/*.sql; do
awk -v val="$MYSQL_REPL_PASSWORD" '{gsub(/__REPL_PASSWORD__/, val); print}' "$f" > "$f.tmp" && mv "$f.tmp" "$f"
done
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The init container runs with set -x (set -ex), and the replication password is expanded into the awk -v val="$MYSQL_REPL_PASSWORD" ... command line. This will leak the Secret value into init-container logs. Drop -x (or disable xtrace around secret-handling) to avoid logging credentials.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines 84 to 88
SENTINEL_CONF="$SENTINEL_CONF
tls-auth-clients no"
fi
echo "$SENTINEL_CONF" > /etc/redis/sentinel.conf
cat /etc/redis/sentinel.conf
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The init container prints the generated /etc/redis/sentinel.conf via cat. That file contains requirepass and sentinel auth-pass values, so this will leak Redis credentials into pod logs. Avoid logging the full config (or redact secret-bearing lines before output).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread kubernetes/network/policies.yaml Outdated
Comment on lines +118 to +137
# NFS provisioner: allow ingress from all pods that need shared volumes
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-nfs-ingress
spec:
podSelector:
matchLabels:
app: nfs-provisioner
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
name: openemr
ports:
- port: 2049
- port: 111
- port: 20048
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allow-nfs-ingress restricts ingress to the NFS server pod to sources with podSelector: name=openemr, but NFS volume mounts are typically initiated from the node (kubelet) rather than from the application pod IP. With a NetworkPolicy-enforcing CNI, this can block NFS mounts even though the consuming pod is OpenEMR. Consider allowing ingress from node IPs (e.g., via ipBlock for the node CIDR) and include the needed TCP/UDP ports used by the NFS service.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines +6 to +8
storageClassName: nfs
accessModes:
- ReadWriteOnce
- ReadWriteMany
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accessModes list item is not indented under accessModes:. As written, this YAML will not parse (the - ReadWriteMany entry is at the same indentation level as the accessModes key). Indent the list item so it is a child of accessModes:.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

incorrect

Comment on lines +9 to +11
storageClassName: nfs
accessModes:
- ReadWriteOnce
- ReadWriteMany
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accessModes list item is not indented under accessModes:. This currently produces invalid YAML for the PVC. Indent - ReadWriteMany so it is a child element of accessModes:.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

incorrect

Comment thread kubernetes/mysql/statefulset.yaml Outdated
Comment on lines +55 to +57
# Inject replication password from secret (awk for safe handling of special chars)
for f in /docker-entrypoint-initdb.d/*.sql; do
awk -v val="$MYSQL_REPL_PASSWORD" '{gsub(/__REPL_PASSWORD__/, val); print}' "$f" > "$f.tmp" && mv "$f.tmp" "$f"
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awk's gsub() replacement string treats & (and backslashes) specially. If MYSQL_REPL_PASSWORD contains &, the rendered SQL will be corrupted (because & expands to the matched text). Either escape the replacement value before calling gsub, or use a templating approach that performs literal string replacement for arbitrary Secret values.

Suggested change
# Inject replication password from secret (awk for safe handling of special chars)
for f in /docker-entrypoint-initdb.d/*.sql; do
awk -v val="$MYSQL_REPL_PASSWORD" '{gsub(/__REPL_PASSWORD__/, val); print}' "$f" > "$f.tmp" && mv "$f.tmp" "$f"
# Inject replication password from secret.
# Escape backslashes and '&' so awk gsub() treats the value literally.
escaped_mysql_repl_password=${MYSQL_REPL_PASSWORD//\\/\\\\}
escaped_mysql_repl_password=${escaped_mysql_repl_password//&/\\&}
for f in /docker-entrypoint-initdb.d/*.sql; do
awk -v val="$escaped_mysql_repl_password" '{gsub(/__REPL_PASSWORD__/, val); print}' "$f" > "$f.tmp" && mv "$f.tmp" "$f"

Copilot uses AI. Check for mistakes.
Comment on lines +50 to +65
- name: SESSION_STORAGE_MODE
value: "predis-sentinel"
- name: REDIS_SENTINELS
value: "sentinel-0.sentinel|||sentinel-1.sentinel|||sentinel-2.sentinel"
- name: REDIS_MASTER
value: "mymaster"
- name: REDIS_SENTINELS_PASSWORD
valueFrom:
secretKeyRef:
name: redis-credentials
key: default-password
- name: REDIS_MASTER_PASSWORD
valueFrom:
secretKeyRef:
name: redis-credentials
key: default-password
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This manifest switches to SESSION_STORAGE_MODE=predis-sentinel and removes REDIS_SERVER, but the OpenEMR Docker entrypoint in this repo configures Redis sessions only when REDIS_SERVER is set (and it does not reference SESSION_STORAGE_MODE / REDIS_SENTINELS*). Unless the chosen OpenEMR image explicitly supports these new env vars, session storage will silently fall back to filesystem (breaking multi-replica session sharing). Either set the supported REDIS_SERVER/REDIS_PASSWORD/REDIS_* variables for the target image, or pin/update docs to an image version that implements the sentinel env var contract.

Suggested change
- name: SESSION_STORAGE_MODE
value: "predis-sentinel"
- name: REDIS_SENTINELS
value: "sentinel-0.sentinel|||sentinel-1.sentinel|||sentinel-2.sentinel"
- name: REDIS_MASTER
value: "mymaster"
- name: REDIS_SENTINELS_PASSWORD
valueFrom:
secretKeyRef:
name: redis-credentials
key: default-password
- name: REDIS_MASTER_PASSWORD
valueFrom:
secretKeyRef:
name: redis-credentials
key: default-password
- name: REDIS_SERVER
value: "redis"
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: redis-credentials
key: default-password

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wanted behavior

Comment on lines +133 to +143
path: redis-master-ca
- key: tls.crt
path: redis-master-cert
- key: tls.key
path: redis-master-key
- key: ca.crt
path: redis-sentinel-ca
- key: tls.crt
path: redis-sentinel-cert
- key: tls.key
path: redis-sentinel-key
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Redis certificate mount and filenames here don’t match what the OpenEMR entrypoint scripts in this repo expect for REDIS_TLS/REDIS_X509 (they look for /root/certs/redis/redis-{ca,cert,key} and copy them into sites/.../documents/certificates/). With the current mount at /redis-certs and paths like redis-master-ca, Redis TLS/X509 is likely not wired up for 8.1.x images. Align the mountPath/keys to the expected layout or update the OpenEMR image/docs to the actual expected cert locations.

Suggested change
path: redis-master-ca
- key: tls.crt
path: redis-master-cert
- key: tls.key
path: redis-master-key
- key: ca.crt
path: redis-sentinel-ca
- key: tls.crt
path: redis-sentinel-cert
- key: tls.key
path: redis-sentinel-key
path: redis-ca
- key: tls.crt
path: redis-cert
- key: tls.key
path: redis-key

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wanted behavior

@bradymiller
Copy link
Copy Markdown
Member Author

@aisle-analyzer

@aisle-research-bot
Copy link
Copy Markdown

aisle-research-bot Bot commented Apr 26, 2026

🔒 Aisle Security Analysis

We found 12 potential security issue(s) in this PR:

# Severity Title
1 🟠 High Hard-coded Redis credentials in Kubernetes Secret manifest
2 🟠 High Remote Kubernetes manifest applied from GitHub without integrity verification (cert-manager)
3 🟠 High NFS provisioner pod mounts hostPath and adds Linux capabilities (node filesystem access / privilege risk)
4 🟠 High Kind extraPortMappings expose OpenEMR NodePort on all host interfaces
5 🟠 High Hard-coded MySQL/MariaDB replication password committed in Kubernetes Secret
6 🟡 Medium Overly long-lived Redis/Sentinel mTLS certificates and overly-broad EKU usages
7 🟡 Medium SQL injection/config injection risk when templating replication password into MariaDB init SQL
8 🟡 Medium MySQL liveness/readiness probes expose root password via process arguments
9 🟡 Medium Overly permissive NetworkPolicy allows any pod to reach NFS/RPC ports on nfs-provisioner
10 🟡 Medium Redis ACL injection risk when templating passwords into users.acl
11 🔵 Low Redis Sentinel initContainer passes admin password via redis-cli -a (exposed in process arguments)
12 🔵 Low Redis initContainer passes passwords via redis-cli -a (exposed in process arguments)

Each finding is detailed in a separate comment below.


Analyzed PR: #349 at commit b8cc690

Last updated on: 2026-04-26T21:46:42Z

Comment on lines +5 to +8
stringData:
admin-password: adminpassword
replication-password: replicationpassword
default-password: defaultpassword
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. 🟠 Hard-coded Redis credentials in Kubernetes Secret manifest

Property Value
Severity High
CWE CWE-798

Description

kubernetes/redis/secret.yaml introduces static, weak credentials directly in the repo.

  • Anyone with read access to the repository (or to the applied Secret in the cluster) can obtain Redis admin, replication, and default passwords.
  • The admin user in the Redis ACL is configured with +@​all (full command access). If these leaked credentials are used, an attacker can gain full control over Redis data, including session storage.

Vulnerable code:

stringData:
  admin-password: a***
  replication-password: r***
  default-password: d***

Recommendation

Do not commit real/static credentials.

  • Generate strong passwords at deploy time and inject them via your secret manager (e.g., ExternalSecrets/Vault/SealedSecrets) or via CI/CD.
  • At minimum, replace these with placeholders and document that operators must override them.

Example using placeholders (force user-supplied values):

apiVersion: v1
kind: Secret
metadata:
  name: redis-credentials
stringData:
  admin-password: "__SET_ME__"
  replication-password: "__SET_ME__"
  default-password: "__SET_ME__"

Or generate and apply without committing to git:

kubectl create secret generic redis-credentials \
  --from-literal=admin-password="$(openssl rand -base64 32)" \
  --from-literal=replication-password="$(openssl rand -base64 32)" \
  --from-literal=default-password="$(openssl rand -base64 32)"

Last updated on: 2026-04-26T21:46:43Z

Comment thread kubernetes/kub-up
sleep 30

# bring in cert-manager and wait for its webhook to be ready
kubectl apply --validate=false -f https://github.com/cert-manager/cert-manager/releases/download/v1.20.2/cert-manager.yaml
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2. 🟠 Remote Kubernetes manifest applied from GitHub without integrity verification (cert-manager)

Property Value
Severity High
CWE CWE-494

Description

kubernetes/kub-up applies a remote cert-manager.yaml manifest directly from a GitHub release URL. This introduces a supply-chain/integrity risk:

  • The script fetches and applies YAML over the network at runtime (kubectl apply -f https://...).
  • No checksum/digest pinning, signature verification, or vendoring is performed.
  • If the fetched content is tampered with (compromised upstream, DNS/TLS interception in certain environments, or developer using a modified URL), arbitrary Kubernetes resources (including cluster-scoped RBAC / webhooks) could be installed in the cluster.

Vulnerable code:

kubectl apply --validate=false -f https://github.com/cert-manager/cert-manager/releases/download/v1.20.2/cert-manager.yaml

Recommendation

Avoid applying remote manifests without integrity controls. Options:

  1. Vendor the manifest in-repo (preferred for reproducibility) and apply the local file:
# commit the reviewed manifest under kubernetes/vendor/cert-manager/v1.20.2/cert-manager.yaml
kubectl apply --validate=false -f vendor/cert-manager/v1.20.2/cert-manager.yaml
  1. Verify a pinned checksum before applying (if you must download at runtime):
URL="https://github.com/cert-manager/cert-manager/releases/download/v1.20.2/cert-manager.yaml"
EXPECTED_SHA256="<pinned_sha256_here>"
curl -fsSL "$URL" -o /tmp/cert-manager.yaml
echo "$EXPECTED_SHA256  /tmp/cert-manager.yaml" | sha256sum -c -
kubectl apply --validate=false -f /tmp/cert-manager.yaml
  1. Use Helm with a pinned chart version and, where possible, verified provenance/signatures.

Also consider updating kub-down similarly since it deletes using the same remote URL.

Last updated on: 2026-04-26T21:46:44Z

Comment on lines +52 to +86
securityContext:
capabilities:
add:
- DAC_READ_SEARCH
- SYS_RESOURCE
args:
- "-provisioner=openemr.org/nfs"
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: SERVICE_NAME
value: nfs-provisioner
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: export-volume
mountPath: /export
livenessProbe:
tcpSocket:
port: nfs
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
tcpSocket:
port: nfs
initialDelaySeconds: 5
periodSeconds: 10
volumes:
- name: export-volume
hostPath:
path: /tmp/nfs-provisioner
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3. 🟠 NFS provisioner pod mounts hostPath and adds Linux capabilities (node filesystem access / privilege risk)

Property Value
Severity High
CWE CWE-269

Description

The NFS provisioner Deployment runs with elevated privileges compared to typical application pods:

  • A hostPath volume mounts a node directory (/tmp/nfs-provisioner) into the container at /export.
  • The container explicitly adds Linux capabilities DAC_READ_SEARCH and SYS_RESOURCE.
  • No hardening is set (e.g., runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation: false, seccompProfile).

If this pod is compromised, the attacker can directly interact with a writable area on the Kubernetes node filesystem via the hostPath mount, and the extra capabilities can expand what can be read/traversed (DAC_READ_SEARCH) and manipulated (SYS_RESOURCE). This increases the blast radius from a single pod compromise to potential node-level impact.

Vulnerable code:

securityContext:
  capabilities:
    add:
    - DAC_READ_SEARCH
    - SYS_RESOURCE
...
volumes:
- name: export-volume
  hostPath:
    path: /tmp/nfs-provisioner

Recommendation

Harden the provisioner pod and avoid hostPath where possible.

  • Prefer a managed storage backend or an NFS server running on dedicated nodes with tight controls.
  • If hostPath is unavoidable, restrict exposure and harden the pod:
    • Run as non-root (when possible) and set allowPrivilegeEscalation: false
    • Drop all capabilities by default and add only the minimum required
    • Use readOnlyRootFilesystem: true (if compatible)
    • Set a seccompProfile: RuntimeDefault
    • Consider hostPath type: DirectoryOrCreate and use a dedicated directory with strict node permissions

Example hardening snippet:

securityContext:
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  seccompProfile:
    type: RuntimeDefault
  capabilities:
    drop: ["ALL"]

Additionally, review whether DAC_READ_SEARCH and SYS_RESOURCE are truly required; remove them if not.

Last updated on: 2026-04-26T21:46:45Z

Comment on lines +5 to +10
- role: control-plane
extraPortMappings:
- containerPort: 30080
hostPort: 8800
- containerPort: 30443
hostPort: 9800
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4. 🟠 Kind extraPortMappings expose OpenEMR NodePort on all host interfaces

Property Value
Severity High
CWE CWE-284

Description

OpenEMR is exposed via a fixed NodePort and Kind extraPortMappings without a listenAddress, which causes the mapped ports to bind on 0.0.0.0 by default in Kind.

  • kubernetes/openemr/service.yaml sets OpenEMR to type: NodePort with fixed nodePorts 30080/30443.
  • kubernetes/kind-config-1-node.yaml (and similarly the 4-node config) maps those nodePorts to host ports 8800/9800.
  • Because listenAddress is not set, these host ports may be reachable from other machines on the network (not just localhost), contradicting the README guidance and potentially exposing a sensitive medical application unintentionally.
  • kubernetes/network/policies.yaml explicitly allows ingress to OpenEMR pods from anywhere on ports 80/443, so once traffic reaches the cluster, it is permitted.

Vulnerable configuration:

extraPortMappings:
- containerPort: 30080
  hostPort: 8800
- containerPort: 30443
  hostPort: 9800

Recommendation

Bind the Kind port mappings to localhost (or remove host port mappings entirely) to avoid unintended exposure.

Option A (recommended for local dev): set listenAddress: "127.0.0.1" for each mapping:

extraPortMappings:
- containerPort: 30080
  hostPort: 8800
  listenAddress: "127.0.0.1"
- containerPort: 30443
  hostPort: 9800
  listenAddress: "127.0.0.1"

Option B: avoid NodePort exposure:

  • Keep OpenEMR Service as ClusterIP and access via kubectl port-forward, or
  • Use an Ingress controller with proper TLS and IP allowlisting.

Also consider tightening allow-openemr-ingress to only allow from the ingress controller / specific CIDRs rather than from all sources.

Last updated on: 2026-04-26T21:46:46Z

metadata:
name: mysql-replication-credentials
stringData:
password: replsecret
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5. 🟠 Hard-coded MySQL/MariaDB replication password committed in Kubernetes Secret

Property Value
Severity High
CWE CWE-798

Description

kubernetes/mysql/replication-secret.yaml introduces a Kubernetes Secret with a static, predictable replication password.

  • The secret is committed to the repo with a usable value
  • It is consumed by the MariaDB StatefulSet init container to generate replication configuration
  • If deployed beyond local dev, this enables straightforward takeover of replication authentication (and may aid database compromise depending on network exposure/policies)

Vulnerable code:

stringData:
  password: <obfuscated>

Recommendation

Do not store real/default passwords in git.

Use one of:

  • External secret managers (ExternalSecrets/CSI driver)
  • SealedSecrets/SOPS-encrypted secrets
  • Require manual creation of the secret during deployment

Example (manual creation):

kubectl create secret generic mysql-replication-credentials \
  --from-literal=password="$(openssl rand -base64 32)"

Also ensure documentation highlights that replication credentials must be rotated/unique per environment and never reused across clusters.

Last updated on: 2026-04-26T21:46:47Z

Comment on lines +7 to +18
duration: 87660h # 10y
renewBefore: 360h # 15d
isCA: false
privateKey:
size: 2048
algorithm: RSA
encoding: PKCS1
usages:
- digital signature
- key encipherment
- server auth
- client auth
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6. 🟡 Overly long-lived Redis/Sentinel mTLS certificates and overly-broad EKU usages

Property Value
Severity Medium
CWE CWE-16

Description

The newly added cert-manager Certificate resources for Redis and Sentinel issue end-entity certificates with a 10-year validity and include both server and client authentication usages.

Implications:

  • duration: 87660h # 10y greatly increases the blast radius of key compromise and reduces the effectiveness of rotation/revocation in typical Kubernetes environments.
  • usages includes both server auth and client auth, which allows the same certificate/keypair to potentially be accepted as a client certificate where mTLS is enforced, undermining identity separation between servers and clients.

Vulnerable configuration:

spec:
  duration: 87660h # 10y
  usages:
    - server auth
    - client auth

Recommendation

Use shorter-lived leaf certificates and restrict EKUs to intended roles.

  • Reduce leaf certificate lifetime (commonly 30-90 days for workload certs; at most 1 year in many orgs).
  • For Redis and Sentinel server certificates, remove client auth.
  • For any client certificates, use a separate Certificate resource with only client auth.

Example (Redis server cert):

spec:
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  usages:
    - digital signature
    - key encipherment
    - server auth

Example (Redis client cert):

spec:
  duration: 2160h # 90d
  renewBefore: 360h
  usages:
    - digital signature
    - key encipherment
    - client auth

Also consider switching to encoding: PKCS8 unless a consuming component explicitly requires PKCS1.

Last updated on: 2026-04-26T21:46:48Z

Comment on lines +55 to +62
# Inject replication password from secret
# Uses bash parameter expansion for literal replacement (safe with &, \, etc.)
# Disable xtrace to prevent password leaking into logs
set +x
for f in /docker-entrypoint-initdb.d/*.sql; do
contents=$(<"$f")
printf '%s\n' "${contents//__REPL_PASSWORD__/$MYSQL_REPL_PASSWORD}" > "$f"
done
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7. 🟡 SQL injection/config injection risk when templating replication password into MariaDB init SQL

Property Value
Severity Medium
CWE CWE-89

Description

The MariaDB initContainer replaces __REPL_PASSWORD__ in SQL files with the raw secret value from MYSQL_REPL_PASSWORD and writes the result into /docker-entrypoint-initdb.d/*.sql, which MariaDB executes during initialization.

  • Input: MYSQL_REPL_PASSWORD from Kubernetes Secret mysql-replication-credentials
  • Transformation: bash string replacement with no SQL escaping/validation
  • Sink: SQL scripts in /docker-entrypoint-initdb.d executed by the mariadb image entrypoint

Because the placeholder is used inside single-quoted SQL strings (see primary.sql/secondary.sql), a password containing a single quote ('), newline, or ; can break out of the string and inject additional SQL (or at least break initialization in ways that could lead to insecure/undefined replication setup).

Vulnerable code:

printf '%s\n' "${contents//__REPL_PASSWORD__/$MYSQL_REPL_PASSWORD}" > "$f"

And the template context:

IDENTIFIED BY '__REPL_PASSWORD__';
...
MASTER_PASSWORD='__REPL_PASSWORD__',

Recommendation

Avoid templating raw secrets into SQL scripts without escaping.

Options:

  1. Validate and refuse unsafe characters (simple and robust for passwords used in SQL literals):
case "$MYSQL_REPL_PASSWORD" in
  *"'"*|*$'\n'*|*$'\r'*|*";"*)
    echo "Unsafe characters in replication password" >&2
    exit 1
    ;;
esac
  1. Properly escape for MySQL single-quoted literals (replace ' with ''):
escaped_pw=${MYSQL_REPL_PASSWORD//"'"/"''"}
for f in /docker-entrypoint-initdb.d/*.sql; do
  contents=$(<"$f")
  printf '%s\n' "${contents//__REPL_PASSWORD__/$escaped_pw}" > "$f"
done
  1. Prefer generating users via a SQL client command that safely handles parameters (where possible), rather than writing templated SQL files.

Also ensure the replication secret is generated from a restricted character set compatible with the chosen approach.

Last updated on: 2026-04-26T21:46:49Z

Comment on lines +84 to +99
livenessProbe:
exec:
command:
- bash
- -c
- mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}
initialDelaySeconds: 30
periodSeconds: 20
readinessProbe:
exec:
command:
- bash
- -c
- mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}
initialDelaySeconds: 15
periodSeconds: 10
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8. 🟡 MySQL liveness/readiness probes expose root password via process arguments

Property Value
Severity Medium
CWE CWE-532

Description

The MySQL StatefulSet configures liveness/readiness probes that execute mysqladmin with the root password provided on the command line.

  • MYSQL_ROOT_PASSWORD is sourced from a Kubernetes Secret.
  • The probe runs bash -c mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}.
  • Passing credentials via CLI arguments exposes the password to other processes in the same pod/node via process listings (e.g., /proc/<pid>/cmdline, ps), and it may also be captured by debugging/tracing tools.

Vulnerable code:

- mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}

Recommendation

Avoid putting passwords in process arguments. Use an option that does not expose the secret in argv.

Safer alternatives (pick one):

  1. Use MYSQL_PWD environment variable only for the probe process:
livenessProbe:
  exec:
    command:
    - bash
    - -ec
    - |
      MYSQL_PWD="$MYSQL_ROOT_PASSWORD" mysqladmin ping -u root
  1. Use a temporary defaults file (client option file) with restricted permissions:
- bash
- -ec
- |
  umask 077
  cat > /tmp/my.cnf <<'EOF'
  [client]
  user=root
  password='"$MYSQL_ROOT_PASSWORD"'
  EOF
  mysqladmin --defaults-extra-file=/tmp/my.cnf ping

Also consider using a dedicated low-privilege healthcheck user instead of root.

Last updated on: 2026-04-26T21:46:50Z

Comment on lines +118 to +135
# NFS provisioner: allow all ingress.
# NFS mounts originate from kubelet on nodes, not from pods, so
# pod-selector-based restrictions would block legitimate mounts.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-nfs-ingress
spec:
podSelector:
matchLabels:
app: nfs-provisioner
policyTypes:
- Ingress
ingress:
- ports:
- port: 2049
- port: 111
- port: 20048
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9. 🟡 Overly permissive NetworkPolicy allows any pod to reach NFS/RPC ports on nfs-provisioner

Property Value
Severity Medium
CWE CWE-284

Description

The allow-nfs-ingress NetworkPolicy permits ingress to the NFS provisioner without any from restrictions:

  • Because there is no from: clause, any pod in the namespace can connect to the exposed NFS/RPC services on the provisioner (subject to overall cluster networking).
  • NFS is a sensitive service; allowing arbitrary in-namespace access can enable unauthorized mounting/access attempts or facilitate lateral movement and data exfiltration if other pods are compromised.

Vulnerable code:

ingress:
- ports:
  - port: 2049
  - port: 111
  - port: 20048

This is inconsistent with the more restrictive policies used for MySQL/Redis/Sentinel in the same file.

Recommendation

Restrict which sources can reach the NFS provisioner.

Options (choose based on your CNI and kubelet/node traffic model):

  1. If your CNI supports it, restrict to node traffic using ipBlock for node CIDRs and/or namespace selectors for system components.

  2. Restrict to known client pods/namespaces (if mounts originate from pods in your architecture).

Example using ipBlock (replace with your node CIDR ranges):

ingress:
- from:
  - ipBlock:
      cidr: 10.0.0.0/16
  ports:
  - port: 2049
  - port: 111
  - port: 20048

Also consider limiting exposed ports to only those required and using NFSv4-only configurations when possible.

Last updated on: 2026-04-26T21:46:51Z

Comment on lines +53 to +59
echo "Generating ACL file from template"
cp /tmp/acl/users.acl /conf/acl/users.acl
contents=$(cat /conf/acl/users.acl)
contents="${contents//__ADMIN_PASSWORD__/$REDIS_ADMIN_PASSWORD}"
contents="${contents//__REPLICATION_PASSWORD__/$REDIS_REPLICATION_PASSWORD}"
contents="${contents//__DEFAULT_PASSWORD__/$REDIS_DEFAULT_PASSWORD}"
printf '%s\n' "$contents" > /conf/acl/users.acl
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10. 🟡 Redis ACL injection risk when templating passwords into users.acl

Property Value
Severity Medium
CWE CWE-94

Description

The Redis initContainer generates /conf/acl/users.acl by copying a template and performing raw string substitution with secret values (REDIS_*_PASSWORD).

  • Input: REDIS_ADMIN_PASSWORD, REDIS_REPLICATION_PASSWORD, REDIS_DEFAULT_PASSWORD from Kubernetes Secret redis-credentials
  • Transformation: shell parameter expansion replacement with no ACL escaping/validation
  • Sink: users.acl consumed by Redis ACL configuration

Redis ACL files are whitespace/newline delimited. If any password contains whitespace or newline characters, it can inject additional ACL tokens or extra lines, potentially enabling more permissions than intended (e.g., appending +@​all, ~*, or defining extra users). Even if Kubernetes secrets are usually admin-controlled, this is a real config-injection footgun, especially when secrets are sourced from external secret managers or user-provided values.

Vulnerable code:

contents="${contents//__ADMIN_PASSWORD__/$REDIS_ADMIN_PASSWORD}"
contents="${contents//__REPLICATION_PASSWORD__/$REDIS_REPLICATION_PASSWORD}"
contents="${contents//__DEFAULT_PASSWORD__/$REDIS_DEFAULT_PASSWORD}"
printf '%s\n' "$contents" > /conf/acl/users.acl

Recommendation

Do not inject raw secret values into Redis ACL files without ensuring they cannot alter ACL syntax.

Recommended fixes:

  1. Validate passwords to a safe character set for ACL > passwords (reject whitespace/newlines):
for v in "$REDIS_ADMIN_PASSWORD" "$REDIS_REPLICATION_PASSWORD" "$REDIS_DEFAULT_PASSWORD"; do
  case "$v" in
    *$'\n'*|*$'\r'*|*$'\t'*|*' '*)
      echo "Unsafe whitespace in Redis ACL password" >&2
      exit 1
      ;;
  esac
done
  1. Consider using hashed ACL passwords (Redis supports #<sha256> entries) so the file contains only hex:
  • Store SHA-256 (hex) in the secret, and template #${HASH} instead of >${PASSWORD}.
  1. Alternatively, avoid file templating entirely and configure ACLs via redis-cli ACL SETUSER ... at runtime with strict argument handling.

These approaches prevent newline/token injection into the ACL file.

Last updated on: 2026-04-26T21:46:52Z

for i in ${nodes//,/ }
do
MASTER=$(redis-cli --no-auth-warning --raw -h $i --user admin -a $REDIS_PASSWORD info replication | awk '{print $1}' | grep master_host: | cut -d ":" -f2)
MASTER=$(redis-cli $TLSPARAMETERS --no-auth-warning --raw -h $i --user admin -a $REDIS_PASSWORD info replication | awk '{print $1}' | grep master_host: | cut -d ":" -f2)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11. 🔵 Redis Sentinel initContainer passes admin password via redis-cli -a (exposed in process arguments)

Property Value
Severity Low
CWE CWE-532

Description

The Sentinel StatefulSet initContainer queries Redis nodes using redis-cli with -a $REDIS_PASSWORD.

  • REDIS_PASSWORD is sourced from a Kubernetes Secret.
  • redis-cli ... -a $REDIS_PASSWORD places the password in process arguments.
  • This can be recovered via /proc/<pid>/cmdline by other processes in the same pod while the initContainer runs.

Vulnerable code:

redis-cli ... --user admin -a $REDIS_PASSWORD info replication

Recommendation

Avoid -a <password> in command arguments.

Use REDISCLI_AUTH for the command instead:

MASTER=$(REDISCLI_AUTH="$REDIS_PASSWORD" \
  redis-cli $TLSPARAMETERS --no-auth-warning --raw -h "$i" --user admin info replication \
  | awk '{print $1}' | grep master_host: | cut -d ":" -f2)

As an additional safeguard, ensure set -x is not enabled in these scripts to prevent secret echoing.

Last updated on: 2026-04-26T21:46:53Z

Comment thread kubernetes/redis/statefulset-redis.yaml Outdated
Comment on lines 60 to 62
SENTINELAUTH="--no-auth-warning -a $REDIS_DEFAULT_PASSWORD"
if [ "$(redis-cli $TLSPARAMETERS $SENTINELAUTH -h sentinel -p 26379 ping)" != "PONG" ]; then
echo "Sentinel not found to get the master info, defaulting to redis-0"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

12. 🔵 Redis initContainer passes passwords via redis-cli -a (exposed in process arguments)

Property Value
Severity Low
CWE CWE-532

Description

The Redis StatefulSet initContainer uses redis-cli with -a $REDIS_DEFAULT_PASSWORD to query Sentinel.

  • REDIS_DEFAULT_PASSWORD is sourced from a Kubernetes Secret.
  • redis-cli is invoked with -a ... which places the password in the process command line.
  • Other processes with access to the same PID namespace (typically within the pod) can read /proc/<pid>/cmdline and recover the password while the initContainer runs.

Vulnerable code:

SENTINELAUTH="--no-auth-warning -a $REDIS_DEFAULT_PASSWORD"
redis-cli $TLSPARAMETERS $SENTINELAUTH -h sentinel -p 26379 ping

Recommendation

Avoid passing the password via -a on the command line.

Options:

  1. Prefer REDISCLI_AUTH environment variable for just the redis-cli invocation (still an env var, but not exposed in argv):
REDISCLI_AUTH="$REDIS_DEFAULT_PASSWORD" redis-cli $TLSPARAMETERS --no-auth-warning -h sentinel -p 26379 ping
  1. If using Redis 6+ ACL users, consider using --user and REDISCLI_AUTH:
REDISCLI_AUTH="$REDIS_DEFAULT_PASSWORD" redis-cli $TLSPARAMETERS --user default --no-auth-warning -h sentinel -p 26379 ping

Also ensure shell tracing (set -x) is disabled around any command that could reveal secrets.

Last updated on: 2026-04-26T21:46:53Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kubernetes: support redis over the wire encryption

4 participants