runtime: share forecast families' wiring, Chronos-Bolt the only one #682

2026-10-23T19:42:58Z

rvnminers-A-and-N commented

2026-10-23 19:42:58 +00:00

This is the refactor Lior proposed on #665 (comment): each family keeps its implementation in its own file, there is a small shared descriptor, and a registry the common forecasting code iterates. Chronos-Bolt is the only entry and nothing changes for it. TimesFM is not part of this PR.

Shared (packages/runtime/forecast_family.go):

the kinds a job names a family's models by;
Needs and Offers;
the form's choices and help;
refusing a node that can't run the family (no containers, or too little container memory);
choosing each candidate's runner;
comparing two workers' runs within the family's tolerance (Compare for forecast and forecasts, and compareWithin).

In forecast.go and forecasts.go, a model's settings check, its fewest points, the candidate list, the task's Scale, per-task sizing and keptCandidates now ask the registry instead of checking for "chronos".

Descriptor: forecastFamily has these fields: kind, name, variants, fewest, memoryMB, tolerance, label and runner. Memory and tolerance are fields because both mean the same thing for any family: the container's memory, and the agreement relative to the series' size.

Per family (packages/runtime/chronos.go): the pins, requirements, fetch script, Dockerfile, ChronosImage/ChronosLabel, the script the image runs, what it's given (chronosAsked) and what it leaves, and the checks on that output, all inside chronosRunner. The file also holds the chronosFamily entry.

Unchanged

The image is still sisyphus-chronos:e5a1c8b7a87b and the label forecast-chronos:e5a1c8b7a87b. On the R620, the real-image test reused that existing image without rebuilding it.
Before changing anything, I recorded 69 values with the pretend Chronos image for forecast and forecasts jobs of naive,chronos-tiny, all,chronos-base,chronos-mini, chronos-small and naive,ses:
- every task output and what the image was given;
- the aggregates;
- Needs, Offers, the form's choices and help;
- the split and execute errors on nodes without containers or with 512 MB;
- kinds() and invalid()/fewest() results.
The same recording after the refactor is byte-identical. It was a throwaway test and isn't committed.
The 40-series byte-for-byte comparison is now a committed test, TestChronosWiringGivesTheKnownAnswer. Behind #666 it was only a local check, though Lior asked for it as a regression test. It hashes:
- forecasts of 40 series, from too short to backtest up, and forecast of one series;
- five model sets: with and without Chronos-Bolt, every Go kind plus two sizes, and the refused all,chronos-base;
- an image that answers and one that fails;
- what the image is given, the job results, Needs, Offers, the form, the refusals, kinds(), invalid()/fewest(), and the image name and label.
The same test file gives the same digest (a9a8b02e…) on dev at 8b83c49, before this refactor. Changing the batch size the image is given makes it fail.
The rest of the existing tests are unchanged, except two renamed references in chronos_test.go, plus a new check that the pins and the offered sizes match one for one.

New tests (`forecast_family_test.go`)

Families are named apart from the Go kinds and from each other, and each says what it needs. Chronos-Bolt is the only family, with the same kinds, label, memory and tolerance.
A pretend second family, registered only for the test, goes through the shared path:
- kinds, including an unknown variant;
- settings and fewest points;
- Needs for one family and for both;
- Offers by memory, and the form's choices and help;
- the refusal errors;
- fitting by its own runner, with the Go candidates fitted in process;
- Compare within and beyond its own tolerance;
- keptCandidates.
A task with no family candidate runs in process.

Docs

docs/forecast-benchmark.md, Chronos-Bolt section: a new bullet on the family split and where a second family would go.
docs/security-model.md: now points to Compare in forecast_family.go.

Checks run

go test ./packages/... ./apps/sisyphusd/...
go test -tags realimages -run TestChronosForecastsARealSeriesInItsImage ./packages/runtime/ against the real image
gofmt
staticcheck 2026.2.2

Part of #665.

This is the refactor Lior proposed on #665 ([comment](https://git.echelonts.net/Sisyphus/Sisyphus/issues/665)): each family keeps its implementation in its own file, there is a small shared descriptor, and a registry the common forecasting code iterates. Chronos-Bolt is the only entry and nothing changes for it. TimesFM is not part of this PR. **Shared** (`packages/runtime/forecast_family.go`): - the kinds a job names a family's models by; - Needs and Offers; - the form's choices and help; - refusing a node that can't run the family (no containers, or too little container memory); - choosing each candidate's runner; - comparing two workers' runs within the family's tolerance (`Compare` for `forecast` and `forecasts`, and `compareWithin`). In `forecast.go` and `forecasts.go`, a model's settings check, its fewest points, the candidate list, the task's `Scale`, per-task sizing and `keptCandidates` now ask the registry instead of checking for `"chronos"`. **Descriptor:** `forecastFamily` has these fields: `kind`, `name`, `variants`, `fewest`, `memoryMB`, `tolerance`, `label` and `runner`. Memory and tolerance are fields because both mean the same thing for any family: the container's memory, and the agreement relative to the series' size. **Per family** (`packages/runtime/chronos.go`): the pins, requirements, fetch script, Dockerfile, `ChronosImage`/`ChronosLabel`, the script the image runs, what it's given (`chronosAsked`) and what it leaves, and the checks on that output, all inside `chronosRunner`. The file also holds the `chronosFamily` entry. ### Unchanged - The image is still `sisyphus-chronos:e5a1c8b7a87b` and the label `forecast-chronos:e5a1c8b7a87b`. On the R620, the real-image test reused that existing image without rebuilding it. - Before changing anything, I recorded 69 values with the pretend Chronos image for `forecast` and `forecasts` jobs of `naive,chronos-tiny`, `all,chronos-base,chronos-mini`, `chronos-small` and `naive,ses`: - every task output and what the image was given; - the aggregates; - `Needs`, `Offers`, the form's choices and help; - the split and execute errors on nodes without containers or with 512 MB; - `kinds()` and `invalid()`/`fewest()` results. The same recording after the refactor is byte-identical. It was a throwaway test and isn't committed. - **The 40-series byte-for-byte comparison is now a committed test**, `TestChronosWiringGivesTheKnownAnswer`. Behind #666 it was only a local check, though Lior asked for it as a regression test. It hashes: - `forecasts` of 40 series, from too short to backtest up, and `forecast` of one series; - five model sets: with and without Chronos-Bolt, every Go kind plus two sizes, and the refused `all,chronos-base`; - an image that answers and one that fails; - what the image is given, the job results, `Needs`, `Offers`, the form, the refusals, `kinds()`, `invalid()`/`fewest()`, and the image name and label. The same test file gives the same digest (`a9a8b02e…`) on dev at 8b83c49, before this refactor. Changing the batch size the image is given makes it fail. - The rest of the existing tests are unchanged, except two renamed references in `chronos_test.go`, plus a new check that the pins and the offered sizes match one for one. ### New tests (`forecast_family_test.go`) - Families are named apart from the Go kinds and from each other, and each says what it needs. Chronos-Bolt is the only family, with the same kinds, label, memory and tolerance. - A pretend second family, registered only for the test, goes through the shared path: - kinds, including an unknown variant; - settings and fewest points; - Needs for one family and for both; - Offers by memory, and the form's choices and help; - the refusal errors; - fitting by its own runner, with the Go candidates fitted in process; - Compare within and beyond its own tolerance; - `keptCandidates`. - A task with no family candidate runs in process. ### Docs - `docs/forecast-benchmark.md`, Chronos-Bolt section: a new bullet on the family split and where a second family would go. - `docs/security-model.md`: now points to `Compare` in `forecast_family.go`. ### Checks run - `go test ./packages/... ./apps/sisyphusd/...` - `go test -tags realimages -run TestChronosForecastsARealSeriesInItsImage ./packages/runtime/` against the real image - gofmt - staticcheck 2026.2.2 Part of #665.

excho0 approved these changes

2026-10-23 20:15:34 +00:00

excho0 left a comment

The family registry keeps the shared dispatch and comparison path centralized while leaving Chronos-specific image and output behavior in its own file. The pinned regression test makes the behavior-preserving boundary concrete, and the second-family test exercises the generic path. All required checks are green, including coverage, realimages, the platform matrix, and desktop packaging. /cc @rvnminers-A-and-N

rvnminers-A-and-N commented

2026-10-23 20:16:55 +00:00

Thanks for the review, Lior. It's merged as 012c4f6, and integration is synced.

TimesFM is next, as a family of its own. Before any code, I'll check its license, memory footprint and determinism, and post those on #665.

Thanks for the review, Lior. It's merged as 012c4f6, and integration is synced. TimesFM is next, as a family of its own. Before any code, I'll check its license, memory footprint and determinism, and post those on #665.

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Sisyphus/Sisyphus#682

runtime: share forecast families' wiring, Chronos-Bolt the only one #682

Unchanged

New tests (forecast_family_test.go)

Docs

Checks run

New tests (`forecast_family_test.go`)