🔀 Merge pull request #2098 from Lissy93/feat/config-saving-robustness

Feat/config saving robustness

Author: Lissy93

docs/developing.md

@@ -79,6 +79,7 @@ You can set variables either in your environment, or using the [`.env`](https://
 - `IS_DOCKER` - Computed automatically on build. Indicates if running in container
 - `VITE_APP_VERSION` - Again, set automatically using package.json during build time
 - `BACKUP_DIR` - Directory for conf.yml backups
+- `DISABLE_CONFIG_BACKUPS` - Set to 'true' to skip the pre-save backup step (useful on read-only filesystems or where permissions don't allow it)
 
 ### Environment Modes
 

docs/management.md

@@ -235,7 +235,7 @@ docker run --rm -v some_volume:/volume -v /tmp:/backup alpine sh -c "rm -rf /vol
 
 All configuration and dashboard settings are stored in your `user-data/conf.yml` file. If you provide additional assets (like icons, fonts, themes, etc), these will also live in the `user-data` directory. So to backup all Dashy data, this is the only directory you need to backup.
 
-When you save config through the UI, Dashy automatically creates a timestamped backup in `user-data/config-backups/` (configurable via the `BACKUP_DIR` env var). If you break your config, check that directory for a recent copy.
+When you save config through the UI, Dashy automatically creates a timestamped backup in `user-data/config-backups/` (configurable via the `BACKUP_DIR` env var). If you break your config, check that directory for a recent copy. Backups can be disabled by setting `DISABLE_CONFIG_BACKUPS=true` (e.g. on read-only filesystems or where permissions don't allow it).
 
 Since Dashy is open source, there shouldn't be any need to backup the main container.
 

docs/troubleshooting.md

@@ -8,7 +8,18 @@
 ## Contents
 
 - [Config not saving](#config-not-saving)
-- [Unable to write conf.yml: EACCES: permission denied](#unable-to-write-confyml-eacces-permission-denied)
+  - [Permission denied or read-only filesystem](#permission-denied-or-read-only-filesystem-eacces-erofs)
+  - [Kubernetes ConfigMap mount is read-only](#kubernetes-configmap-mount-is-read-only)
+  - [SELinux or AppArmor blocks the write](#selinux-or-apparmor-blocks-the-write)
+  - [Backup step fails so save aborts](#backup-step-fails-so-save-aborts)
+  - [Save button is missing or returns 403](#save-button-is-missing-or-returns-403-forbidden)
+  - [Save unavailable on Vercel, Netlify or other static hosts](#save-unavailable-on-vercel-netlify-or-other-static-hosts)
+  - [/config-manager/save returns 404 or HTML](#config-managersave-returns-404-or-html)
+  - ["Invalid filename" when saving a sub-page](#invalid-filename-when-saving-a-sub-page)
+  - ["Cannot save to an external URL"](#cannot-save-to-an-external-url)
+  - [Saved successfully but the UI shows the old config](#saved-successfully-but-the-ui-shows-the-old-config)
+  - [Container crashes or restart loop after saving](#container-crashes-or-restart-loop-after-saving-310-and-311-only)
+  - [Intentionally read-only mode](#intentionally-read-only-mode)
 - [Refused to Connect in Web Content View](#refused-to-connect-in-modal-or-workspace-view)
 - [404 On Static Hosting](#404-on-static-hosting)
 - [404 from Mobile Home Screen](#404-after-launch-from-mobile-home-screen)
@@ -24,16 +35,11 @@
 - [High CPU or RAM Usage on Startup](#high-cpu-or-ram-usage-on-startup)
 - [Heap limit Allocation failed](#ineffective-mark-compacts-near-heap-limit-allocation-failed)
 - [Command failed with signal "SIGKILL"](#command-failed-with-signal-sigkill)
-- [Container Crashes or Restart Loop After Saving Config](#container-crashes-or-restart-loop-after-saving-config)
 - [Auth Validation Error: "should be object"](#auth-validation-error-should-be-object)
 - [App Not Starting After Update to 2.0.4](#app-not-starting-after-update-to-204)
 - [Keycloak Redirect Error](#keycloak-redirect-error)
 - [OIDC or Keycloak failure on numeric client IDs](#oidc-or-keycloak-failure-on-numeric-client-ids)
 - [Mount Type Mismatch](#mount-type-mismatch)
-- [Permission Denied Saving Config](#permission-denied-saving-config)
-- [Config not Saving on Vercel / Netlify / CDN](#config-not-saving-on-vercel--netlify--cdn)
-- [Config Not Updating](#config-not-updating)
-- [Config Still not Updating](#config-still-not-updating)
 - [Styles and Assets not Updating](#styles-and-assets-not-updating)
 - [DockerHub toomanyrequests](#dockerhub-toomanyrequests)
 - [Old image tags fail to pull](#old-image-tags-fail-to-pull)
@@ -65,43 +71,111 @@
 
 ## Config not saving
 
-### Possible Issue 1: Unable to call save endpoint from CDN/static server
-If you're running Dashy using a static hosting provider (like Vercel), then there is no Node server, and so the save config action will not work via the UI.
-You'll instead need to copy the YAML after making your changes, and paste that into your `conf.yml` directly. If you've connected Vercel to git, then these changes will take effect automatically, once you commit your changes. 
-Look here for more information: [https://dashy.to/docs/deployment#deploy-to-cloud-service](https://dashy.to/docs/deployment#deploy-to-cloud-service)
+There should be an error message, explaining the reason the config save failed. First check [browser console](#how-to-open-browser-console) (<kbd>F12</kbd> --> Console), and then your server-side logs in the terminal. Then, see the following sections for solutions to each possible error.
 
-If you're running on Netlify, there are some cloud functions which take care of all the server endpoints (like status checking), so these will work as expected.
+<a id="config-not-saving-on-vercel--netlify--cdn"></a>
+<a id="possible-issue-1-unable-to-call-save-endpoint-from-cdnstatic-server"></a>
+<a id="unable-to-write-confyml-eacces-permission-denied"></a>
+<a id="permission-denied-saving-config"></a>
+<a id="possible-issue-2-unable-to-save"></a>
+<a id="config-not-updating"></a>
+<a id="config-still-not-updating"></a>
+<a id="possible-issue-3-saved-but-not-updating"></a>
 
-See also [#1465](https://github.com/Lissy93/dashy/issues/1465)
+### Permission denied or read-only filesystem (EACCES, EROFS)
 
-### Possible Issue 2: Unable to save
-In Docker, double check that the file isn't read-only, and that the container actually has permissions to modify it. You shouldn't really be running it as a root user, and I'm not sure if it will work if you do-
+The container can't write to your `conf.yml` or its directory. Almost always an ownership mismatch: the host directory belongs to a different uid than the one Dashy runs as inside the container. Less commonly a read-only mount or an over-strict file mode.
 
-### Possible Issue 3: Saved but not updating
-After saving, the frontend will recompile, which may take a couple seconds (or a bit longer on a Pi or low-powered device). If it doesn't recompile, you can manually trigger a re-build.
+Dashy runs as UID=1000 (default non-root node user). You can see this by running `docker exec -it dashy id`. Then, check who owns the user-data directory, with: `docker exec -it dashy ls -la /app/user-data` - if it's not `1000` then that's the issue. And the solution is just to run `sudo chown -R 1000:1000 /path/to/your/user-data` to set the right owner.
 
----
+Fixes:
+1. **Hand the directory to uid 1000** (recommended). Keeps the container running as a non-root user, which is how Dashy is built to run `sudo chown -R 1000:1000 /path/to/your/user-data`
+2. **Run the container as your own user** if `chown` isn't practical (multi-user hosts, NAS appliances, host directories you don't want relabelled). Add `--user $(id -u):$(id -g)` to `docker run`, or set `user: "1000:1000"` (or your host uid:gid) on the service in `docker-compose.yml`.
+3. **Loosen a single-file mount** if its mode is `444`. Narrow case, only fixes that one symptom: `chmod 644 /path/to/conf.yml`
 
-## Unable to write conf.yml: `EACCES: permission denied`
+**Common mistakes**
+- **Using uid/gid 1001.** A common guess on Synology, Unraid and similar where 1001 is the host's first user. Dashy's container is 1000, not 1001.
+- **`chmod` alone for a UID mismatch.** Loosens permissions but doesn't change who owns the file. You need `chown`.
+- **`chmod -R 777` or `775`.** Works as a workaround, masks the real problem, weakens security. Use `chown` to the right uid instead.
 
-Most commonly the `conf.yml` file itself is mounted read-only, or its mode is `444`. Make it writable on the host:
+**Other gotchas:**
+- **Named Docker volumes** (created with `docker volume create`) inherit ownership from whatever first writes to them. If an older container set them up as root, the diagnose step will show that. Recreate the volume or `chown` the underlying directory under `/var/lib/docker/volumes/`.
+- **macOS hosts** rarely hit this. Docker Desktop transparently maps host uid to container uid through its VM. If saves are failing on macOS, look elsewhere first.
+- **Storage layers that ignore POSIX permissions** (some NAS app-data folders use FUSE, SMB or overlay mounts where `chmod` and `chgrp` are silent no-ops). Bind-mount user-data from a native filesystem path instead.
 
-```bash
-chmod 644 /path/to/your/conf.yml
-```
+### Kubernetes ConfigMap mount is read-only
+
+If you've mounted your `conf.yml` from a ConfigMap, writes will always fail with `EROFS` regardless of UID. ConfigMap volumes are read-only by design. Either treat the ConfigMap as the source of truth and edit it directly (saves through the UI won't work), or use a writable volume type like a `PersistentVolumeClaim` for `user-data/`.
+
+### SELinux or AppArmor blocks the write
+
+If you're on RHEL/Fedora, or systems with SELinux or AppArmour, and you've confirmed permissions are fine, and container's UID matches the host owner, but you still see `EACCES`.
+
+For SELinux, add the `:Z` flag to your volume mount so Docker relabels it for the container (e.g. `volumes: [ './user-data:/app/user-data:Z' ]`)
+
+For AppArmor, check `dmesg` for `apparmor="DENIED"` lines and adjust the profile. Disabling enforcement is a last resort.
+
+### Backup step fails so save aborts
+
+Before each save, Dashy backs up the current `conf.yml` to `user-data/config-backups/`. If that folder can't be written, the whole save aborts with `Unable to backup conf.yml`.
+
+Two ways out:
+1. Point `BACKUP_DIR` at a writable path
+2. Set `DISABLE_CONFIG_BACKUPS=true` to skip the backup step entirely
+
+### Save button is missing or returns 403 Forbidden
+
+Have you got auth setup? If so, make sure you are logged in as an admin, or set `type: admin` to your user in `conf.yml`.
+
+Beyond that, there's several other config options which prevent saving the config file, so if you didn't mean to add them, just remove from `conf.yml`
+- **`appConfig.preventWriteToDisk: true`** disables disk save and the button
+- **`appConfig.preventLocalSave: true`** disables the "Local" save option
+- **`appConfig.disableConfiguration: true`** hides the editor entirely. `disableConfigurationForNonAdmin: true` does the same just for non-admins.
+
+### Save unavailable on Vercel, Netlify or other static hosts
+Updating source config file on static hosts is not possible, since they have no Node server, nor have write access to modify any files.
+The "Local" save mode will still work (changes are just persisted in your browser), but the real solution is to copy/export the updated YAML and replace it in the source config file in your repo.
+
+Related: [#1465](https://github.com/Lissy93/dashy/issues/1465).
+
+### `/config-manager/save` returns 404 or HTML
+How are you running/serving Dashy?
+If you've got a reverse proxy which only forwards specific path prefixes then maybe you're missing the `/config-manager/*` API endpoints?
+Check the failed request in the browser's Network tab. If the response is HTML (a proxy error page) or a plain 404, your proxy isn't routing the path. Add `/config-manager/` to whatever you're forwarding, or simplify the rules so everything reaches Dashy.
+
+Or if you're serving up the compiled Vue app directly, instead of using the Node server, then the endpoint won't be available.
 
-As for backups, Dashy creates a timestamped backup of the existing config in `user-data/config-backups/` before writing the new one. If the host filesystem prevents that copy (read-only mount, wrong UID/GID, no execute bit on the parent directory), the save aborts before touching the live config.
+### "Invalid filename" when saving a sub-page
 
-Check that the user-data volume is writable by the container user:
+The save endpoint rejects sub-page filenames with path separators or non-yaml extensions. Check the `path:` value of the page in your `pages:` block. It needs to be a plain basename like `home.yml`, not `pages/home.yml` or `home.txt`.
+
+### "Cannot save to an external URL"
+
+The sub-page you are editing is loaded from a remote URL. Dashy can't write back to that URL.
+You will need to edit the file at it's origin yourself instead (click the Export to view the YAML).
+Or you could download the config to `user-data/something.yml`, and update `path:` to point to the local version.
+
+### Saved successfully but the UI shows the old config
+
+Two unrelated causes share this symptom:
+
+1. **Local storage overrides the file.** Dashy lets users save settings locally in browser storage, which take priority over `conf.yml`. Open Dashy in incognito to confirm. If the changes appear there, clear local settings via Config menu > "Clear Local Settings".
+2. **Docker isn't picking up file changes.** Some text editors save by replacing the inode, which breaks single-file bind mounts. Edit the file in place, or mount the parent directory rather than the single file. [More background](https://medium.com/@jonsbun/why-need-to-be-careful-when-mounting-single-files-into-a-docker-container-4f929340834).
+
+<a id="container-crashes-or-restart-loop-after-saving-config"></a>
+
+### Container crashes or restart loop after saving (3.1.0 and 3.1.1 only)
+
+If your container crashes or restart-loops right after clicking save, with logs like `ERR_HTTP_HEADERS_SENT` or `ERR_STREAM_WRITE_AFTER_END`, this was a known double-`res.end()` bug in 3.1.0 and 3.1.1. Fixed in v3.2.13 and later.
 
 ```bash
-docker exec -it dashy id
-docker exec -it dashy ls -la /app/user-data
+docker pull lissy93/dashy:latest
+docker compose up -d --force-recreate
 ```
 
-If you bind-mount from the host, make sure the host directory is owned by the container's UID (or world-writable). You can also override the backup location with the `BACKUP_DIR` env var.
+### Intentionally read-only mode
 
-And, if you intentionally want a read-only config and don't want users to even attempt saves, set `appConfig.preventWriteToDisk: true` in `conf.yml` - Dashy will then disable both the endpoint, and the save button itself, to make the experience more explicit.
+To stop anyone saving from the UI, set `appConfig.preventWriteToDisk: true` in `conf.yml`. This disables the endpoint and the save button. For Docker users, you can harden things further, by mounting the config and all user-data as read-only.
 
 ---
 
@@ -349,36 +423,6 @@ See also [#624](https://github.com/Lissy93/dashy/issues/624)
 
 ---
 
-## Container Crashes or Restart Loop After Saving Config
-
-If your Dashy container crashes or enters a restart loop the moment you click **Save to Disk** in the editor - particularly with logs that include something like:
-
-```text
-node:_http_outgoing:652
-    throw new ERR_HTTP_HEADERS_SENT('set');
-```
-
-or:
-
-```text
-Error [ERR_STREAM_WRITE_AFTER_END]: write after end
-```
-
-This was a known crash in Dashy 3.1.0 and 3.1.1 where a partial-failure code path in the save handler would call `res.end()` twice on the same response, causing Node to throw an unhandled rejection and exit the process. Docker would then restart the container, the user would save again, and the loop repeated. Several users also reported it was reliably triggered by entering edit mode while `statusCheck: true` was set, because the periodic background traffic increased the chance of overlapping responses.
-
-**This is fixed in v3.2.13 and later**
-
-```bash
-docker pull lissy93/dashy:latest
-docker compose up -d --force-recreate
-```
-
-If you can't upgrade right away, the temporary workaround that worked for affected users is to set `statusCheck: false` under `appConfig` in your `conf.yml`. This reduces the trigger frequency without changing the underlying bug.
-
-See also: [#1945](https://github.com/Lissy93/dashy/issues/1945), [#1935](https://github.com/Lissy93/dashy/issues/1935), [#1494](https://github.com/Lissy93/dashy/issues/1494)
-
----
-
 ## Auth Validation Error: "should be object"
 
 In V 1.6.5 an update was made that in the future will become a breaking change. You will need to update you config to reflect this before V 2.0.0 is released. In the meantime, your previous config will continue to function normally, but you will see a validation warning. The change means that the structure of the `appConfig.auth` object is now an object, which has a `users` property.
@@ -487,42 +531,6 @@ If you'd rather mount a single file (`-v ~/conf.yml:/app/user-data/conf.yml`), t
 
 ---
 
-## Permission Denied Saving Config
-
-If saving config from the UI fails with `EACCES` or `permission denied`, your mounted `user-data` directory is owned by a uid the container can't write to. Inside the container, Dashy runs as uid 1000 (the `node` user), and your host directory probably belongs to a different uid.
-
-Two fixes, pick whichever's less of a faff:
-
-1. Run the container as your own user. On `docker run`, add `--user $(id -u):$(id -g)`. In `docker-compose.yml`, uncomment the `user:` line and set it to your uid:gid (run `id -u` and `id -g` to find them).
-2. Hand the directory to uid 1000: `sudo chown -R 1000:1000 /path/to/your/user-data`.
-
-Most people are already uid 1000 on Linux and macOS, so this only tends to bite on NAS boxes, multi-user servers, or anywhere the first user isn't 1000.
-
----
-
-## Config not Saving on Vercel / Netlify / CDN
-
-If you're running Dashy using a static hosting provider (like Vercel), then there is no Node server, and so the save config action will not work via the UI.
-You'll instead need to copy the YAML after making your changes, and paste that into your `conf.yml` directly. If you've connected Vercel to git, then these changes will take effect automatically, once you commit your changes.
-
-If you're running on Netlify, there are some cloud functions which take care of all the server endpoints (like status checking), so these will work as expected.
-
-See also [#1465](https://github.com/Lissy93/dashy/issues/1465)
-
----
-
-## Config Not Updating
-
-Dashy has the option to save settings and config locally, in browser storage. Anything here will take precedence over whatever is in your config file, sometimes with unintended consequences. If you've updated the config file manually, and are not seeing changes reflected in the UI, then try visiting the site in Incognito mode. If that works, then the solution is just to clear local storage. This can be done from the config menu, under "Clear Local Settings".
-
----
-
-## Config Still not Updating
-
-Sometimes your text editor updates files [inode](https://linuxhandbook.com/inode-linux/), meaning changes will not be picked up by the Docker container. This [article](https://medium.com/@jonsbun/why-need-to-be-careful-when-mounting-single-files-into-a-docker-container-4f929340834) explains things further.
-
----
-
 ## Styles and Assets not Updating
 
 If you find that your styles and other visual assets work when visiting `ip:port` by not `dashy.domain.com`, then this is usually caused by caching. In your browser, do a hard-refresh (<kbd>Ctrl</kbd> + <kbd>F5</kbd>). If you use Cloudflare, then you can clear the cache through the management console, or set the cache level to Bypass for certain files, under the Rules tab.