rvnminers-A-and-N
42dcfec8cd
A new workload, container, runs an image once for each task of a job, with Docker. Each task is told which copy it is and how many there are. A stored file can be given to every task, and what a task prints and the files it leaves are stored and returned by CID. What it prints is also its log, as it goes. Memory and processor limits are enforced, a task has no network unless the job asks for it, and it runs as the node's own user. Cancelling the job or timing out a task kills its container. A worker runs containers only when started with --containers, because that lets whoever may submit jobs run what they like on the machine. Any coordinator takes such a job in; it waits for a worker that runs them. The planner is now told only of workloads that some connected worker runs, so that it asks for nothing the pool would leave waiting. Closes #9. Part of #10.
34 lines
1.1 KiB
Bash
Executable File
34 lines
1.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Runs a container image on the pool, one copy for each task, and prints
|
|
# what each task printed.
|
|
#
|
|
# examples/container.sh [tasks] [image] [command...]
|
|
#
|
|
# With no arguments it runs three copies of a small image, each of which
|
|
# says which copy it is. At least one worker of the pool must have been
|
|
# started with --containers.
|
|
set -euo pipefail
|
|
. "$(dirname "$0")/common.sh"
|
|
|
|
tasks=${1:-3}
|
|
image=${2:-alpine:3.20}
|
|
shift $(( $# > 2 ? 2 : $# ))
|
|
if [ $# -eq 0 ]; then
|
|
set -- sh -c 'echo "task $SISYPHUS_TASK_INDEX of $SISYPHUS_TASK_COUNT on $(hostname)"'
|
|
fi
|
|
|
|
# The command, as a JSON list.
|
|
command=$(printf '%s\n' "$@" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read().splitlines()))' 2>/dev/null) || {
|
|
echo "this example needs python3 to quote the command" >&2
|
|
exit 1
|
|
}
|
|
params="{\"image\":\"$image\",\"command\":$command}"
|
|
|
|
echo "running $image in $tasks tasks"
|
|
out=$(node job submit --workload container --tasks "$tasks" --params "$params")
|
|
echo "$out" | grep -v '^{'
|
|
# The result lists, for each task, the CID of what it printed.
|
|
for cid in $(echo "$out" | grep -o '"stdout":"[^"]*"' | cut -d'"' -f4); do
|
|
node blob get "$cid"
|
|
done
|