run playwright via container

[TheFox0x7]

Enable running playwright tests on unsupported platforms as well

[silverwind]

I can test this on Arch Linux, which so far was unable to run playwright.

[TheFox0x7]

I am doing exactly that :)
few webkit and one from firefox timed out but it might be a local thing so I wanted to run it against CI.

unfortunately I think the --network=host is a must because otherwise I need to pass host (which works) but then request fixture can't get the address as it runs on host instead of in container.

[lunny]

Wouldn’t that require the developer to install Docker?

[TheFox0x7]

we have docker target for one. I'm working on skipping the deps install on non-debian/ubuntu installs though and toggling based on that.
Regardless, I think installing container runtime is simpler than changing distro to run tests.

[silverwind]

BTW another approach for limited e2e tests on Arch could be to detect chromium, firefox or webkit binaries in PATH and use that. The first two should work pretty well, webkit will be a gamble. Could make it use binaries on a best-effort basis 😆.

But overall I agree that if docker is available, it's the best fallback.

[silverwind]

#37315 will revert the webkit addition, to unstable. With webkit gone, it may be feasible to run chromium and firefox natively.

[TheFox0x7]

The first two should work pretty well.

Chromium works well. Firefox? I can't get it to work at all because some glib stuff or something. container should be more stable overall for systems not blessed by playwright.
Funnily enough I remember firefox working before so it's probably just a coin toss if it'll work or not.

I'll default to local unless OS is linux and it's not ubuntu/debian. I know I can gate it, just need to fight makefile a bit... though maybe instead I just should write a script like a sane person.

---

Does makefile and e2e even works under windows?

[silverwind]

Does makefile and e2e even works under windows?

In theory if you have go,node,docker,pnpm available, it may work but I think it's very likely you hit some issue. Real confirmation could only come if we could make CI run on windows runners which I think are not configured.

BTW webkit on macos runners should work reliably but again, I don't think our self-hosted runner infrastructure provides such runners.

[TheFox0x7]

It was more of a "should I be concerned about it" question. I don't think the test script for e2e would work as it needs bash but I don't exactly have a way of trying it. shell it is as conditionals in makefile are a pain

[silverwind]

No, for all we are concerned, only Linux and MacOS have to work.

[a1012112796]

maybe you can run test in docker by act_runner exec (should temporary update actions configuration file )

diff --git a/.github/workflows/pull-e2e-tests.yml b/.github/workflows/pull-e2e-tests.yml
index fde36383b9..2812ee7eb8 100644
--- a/.github/workflows/pull-e2e-tests.yml
+++ b/.github/workflows/pull-e2e-tests.yml
@@ -8,14 +8,14 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  files-changed:
-    uses: ./.github/workflows/files-changed.yml
-    permissions:
-      contents: read
+  # files-changed:
+  #   uses: ./.github/workflows/files-changed.yml
+  #   permissions:
+  #     contents: read
 
   test-e2e:
-    if: needs.files-changed.outputs.backend == 'true' || needs.files-changed.outputs.frontend == 'true' || needs.files-changed.outputs.e2e == 'true'
-    needs: files-changed
+    # if: needs.files-changed.outputs.backend == 'true' || needs.files-changed.outputs.frontend == 'true' || needs.files-changed.outputs.e2e == 'true'
+    # needs: files-changed
     runs-on: ubuntu-latest
     permissions:
       contents: read
@@ -35,7 +35,7 @@ jobs:
       - run: make frontend
       - run: make deps-backend
       - run: make gitea-e2e
-      - run: make playwright
+      - run: GITHUB_ACTIONS="" make playwright
       - run: make test-e2e
         timeout-minutes: 10
         env:

[TheFox0x7]

But... why? This is just "run in container" but with more steps and requiring act runner, docker daemon and installing browsers which are built-in in the playwright container. I don't follow.

[silverwind]

Goal should be that make test-e2e can run locally on platforms not officially supported by playwright. Either running natively or with a transparent docker wrapper are both acceptable to me, with a preference for native.