ci: the workloads checked on every other system a release is built for (#261)

* ci: the workloads checked on every other system a release is built for RISC-V, 32-bit ARM, POWER, IBM Z and LoongArch under an emulator, 32-bit x86 and Android natively, and FreeBSD, OpenBSD and NetBSD in virtual machines. Releases gain Linux on 386, ppc64le, s390x and loong64, and OpenBSD on arm64. * ci: qemu-user on Ubuntu 26.04, and the Android build loaded by Linux's loader
2026-10-14 19:23:58 -04:00
1 parent 8507cbea1c
commit 6217e418bd
@@ -129,6 +129,69 @@ jobs:
          go test ./packages/runtime
          -run '^(TestCountPrimes|TestPrimesSplitExecuteAggregate|TestIsqrtIsExactAcrossThePrimesRange|TestCountWordsOverTheWholeText|TestCountWordsIsUnaffectedByWhereRangesAreCut|TestWordCountProducesASortedTable|TestWordCountResultCIDIsTheSameHoweverTheJobIsSplit)$'

+  # The other systems and processors a release is built for, which GitHub
+  # has no runner of: the tests are compiled for each and run in an emulator
+  # of the processor, on the Android build's own processor (loaded by Linux's
+  # loader, since it asks for Android's, which it needs only to be placed in
+  # memory), or in a virtual machine of the BSD. s390x is big-endian, so it shows that an answer does
+  # not depend on byte order.
+  workload-elsewhere:
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - {runner: ubuntu-26.04, goos: linux, goarch: riscv64, via: qemu-riscv64}
+          - {runner: ubuntu-26.04, goos: linux, goarch: arm, via: qemu-arm}
+          - {runner: ubuntu-26.04, goos: linux, goarch: ppc64le, via: qemu-ppc64le}
+          - {runner: ubuntu-26.04, goos: linux, goarch: s390x, via: qemu-s390x}
+          - {runner: ubuntu-26.04, goos: linux, goarch: loong64, via: qemu-loongarch64}
+          - {runner: ubuntu-26.04, goos: linux, goarch: "386", via: ""}
+          - {runner: ubuntu-26.04-arm, goos: android, goarch: arm64, via: /lib/ld-linux-aarch64.so.1}
+          - {runner: ubuntu-26.04, goos: freebsd, goarch: amd64, via: vm}
+          - {runner: ubuntu-26.04, goos: openbsd, goarch: amd64, via: vm}
+          - {runner: ubuntu-26.04, goos: netbsd, goarch: amd64, via: vm}
+    name: workload-elsewhere (${{ matrix.goos }}, ${{ matrix.goarch }})
+    runs-on: ${{ matrix.runner }}
+    env:
+      KNOWN_ANSWERS: '^(TestCountPrimes|TestPrimesSplitExecuteAggregate|TestIsqrtIsExactAcrossThePrimesRange|TestCountWordsOverTheWholeText|TestCountWordsIsUnaffectedByWhereRangesAreCut|TestWordCountProducesASortedTable|TestWordCountResultCIDIsTheSameHoweverTheJobIsSplit)$'
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-go@v7
+        with:
+          go-version-file: go.mod
+      - name: Compile the tests for ${{ matrix.goos }}/${{ matrix.goarch }}
+        env:
+          CGO_ENABLED: "0"
+          GOOS: ${{ matrix.goos }}
+          GOARCH: ${{ matrix.goarch }}
+          GOARM: "7"
+        run: go test -c -o packages/runtime/runtime.test ./packages/runtime
+      - name: Install the emulator
+        if: startsWith(matrix.via, 'qemu')
+        run: sudo apt-get update && sudo apt-get install -y qemu-user
+      - name: Run the known-answer workloads
+        if: matrix.via != 'vm'
+        working-directory: packages/runtime
+        run: ${{ matrix.via }} ./runtime.test -test.v -test.run "$KNOWN_ANSWERS"
+      - name: Run the known-answer workloads on FreeBSD
+        if: matrix.goos == 'freebsd'
+        uses: vmactions/freebsd-vm@v1
+        with:
+          envs: KNOWN_ANSWERS
+          run: cd packages/runtime && ./runtime.test -test.v -test.run "$KNOWN_ANSWERS"
+      - name: Run the known-answer workloads on OpenBSD
+        if: matrix.goos == 'openbsd'
+        uses: vmactions/openbsd-vm@v1
+        with:
+          envs: KNOWN_ANSWERS
+          run: cd packages/runtime && ./runtime.test -test.v -test.run "$KNOWN_ANSWERS"
+      - name: Run the known-answer workloads on NetBSD
+        if: matrix.goos == 'netbsd'
+        uses: vmactions/netbsd-vm@v1
+        with:
+          envs: KNOWN_ANSWERS
+          run: cd packages/runtime && ./runtime.test -test.v -test.run "$KNOWN_ANSWERS"
+
  # A pool across machines, as near as one runner gets: each node in a
  # container with its own address and disk. It is what finds faults that
  # tests on one machine cannot, since those all run over loopback.
@@ -285,7 +285,7 @@ Sisyphus is an early prototype. The network loop works without AI: a coordinator
 - A first real workload, `transcode`: a video is cut into stretches, encoded by ffmpeg on different workers, and joined.
 - Container images as jobs: a worker whose owner allows it runs any image, one copy for each task, with its output stored and its log followed live.
 - Two stand-in workloads, `primes` and `wordcount`, that exercise the network rather than compute anything valuable.
- Tests that execute every statement of hand-written code, enforced in CI, and release builds of the node for Linux, macOS, Windows, the BSDs and Android, beside the desktop app's packages.
+- Tests that execute every statement of hand-written code, enforced in CI, and release builds of the node for Linux (eight processors, RISC-V and IBM Z among them), macOS, Windows, the BSDs and Android, and of the desktop app for Linux, Windows and macOS on x64 and arm64, with the workloads' answers checked on each system.

 **Not built yet**: more real workloads than the first (`transcode`). Unless a job asks for its tasks to be run by several workers and compared, a coordinator accepts whatever result a worker returns; and that comparison is the first step only ([#19](https://github.com/sisyphus-network/Sisyphus/issues/19)): it works for deterministic jobs that are not private, believes workers that agree, and costs a worker that is caught lying nothing but a probation, during which jobs that verify only some of their tasks verify all of its. So a pool is still only as trustworthy as the workers admitted to it. There is no blockchain, token or public network.

@@ -22,14 +22,21 @@ Pushing a tag such as `v0.1.0` makes GitHub build `sisyphusd` and publish the bi

 | System | Processors |
 | --- | --- |
-| Linux | x86-64, ARM64, 32-bit ARM (v7), RISC-V 64 |
+| Linux | x86-64, ARM64, 32-bit ARM (v7), RISC-V 64, 32-bit x86, POWER (ppc64le), IBM Z (s390x), LoongArch 64 |
 | macOS | Intel, Apple silicon |
 | Windows | x86-64, ARM64 |
-| FreeBSD | x86-64, ARM64 |
-| OpenBSD, NetBSD | x86-64 |
+| FreeBSD, OpenBSD | x86-64, ARM64 |
+| NetBSD | x86-64 |
 | Android | ARM64, for use in a terminal such as Termux |

-Only Linux on x86-64 is run by the test suite. The others are compiled and have not been run. Solaris, illumos, AIX and Plan 9 are not built because a dependency does not support them.
+The whole test suite runs on Linux x86-64. The known-answer workloads, which have to give the same result wherever they run, are also run on every other row of the table but OpenBSD and FreeBSD on ARM64:
+
+- on GitHub's own Linux, Windows and macOS machines, on x86-64 and ARM64;
+- under an emulator for Linux's other processors;
+- natively for Android;
+- in virtual machines for FreeBSD, OpenBSD and NetBSD.
+
+Solaris, illumos, AIX, Plan 9 and NetBSD on ARM64 are not built because a dependency does not support them.

 ## Try it on one machine

@@ -12,15 +12,19 @@ version=${1:?usage: build-release.sh <version> <output-dir>}
 out=${2:?usage: build-release.sh <version> <output-dir>}
 mkdir -p "$out"

-# Every target here builds without a C toolchain. Only linux/amd64 is run by
-# the test suite; the rest are compiled and otherwise untested. Not built:
-# Solaris, illumos, AIX and Plan 9 (a dependency does not support them), and
-# iOS and Android on x86, which need a platform SDK to link.
+# Every target here builds without a C toolchain. The whole test suite runs
+# on linux/amd64; the known-answer workloads also run on each of Linux,
+# Windows and macOS on x64 and arm64, on Linux's other processors under an
+# emulator, on Android's arm64, and on the BSDs in virtual machines (the
+# workload-architecture and workload-elsewhere jobs in ci.yml). Not built:
+# NetBSD on arm64, Solaris, illumos, AIX and Plan 9 (a dependency does not
+# support them), and iOS and Android on x86, which need a platform SDK.
 targets="
 	linux/amd64 linux/arm64 linux/arm linux/riscv64
+	linux/386 linux/ppc64le linux/s390x linux/loong64
 	darwin/amd64 darwin/arm64
 	windows/amd64 windows/arm64
-	freebsd/amd64 freebsd/arm64 openbsd/amd64 netbsd/amd64
+	freebsd/amd64 freebsd/arm64 openbsd/amd64 openbsd/arm64 netbsd/amd64
 	android/arm64
 "