Appearance
SystemsLab 161.0.0
Added
- Release builds now publish
.rpmpackages for Rocky Linux 10 (el10). - The context page's summary table can now be downloaded as a CSV file via a button next to the Summary heading. Each row is an experiment, with its varying parameters, one column per KPI (labelled
name (unit), holding the raw numeric value), and its state. - A new
systemslab infracommand group —status,log,pricing,requests,estimate,approve, anddeny— lets an operator inspect the EC2 fleet and manage instance requests.approve/denyresolve a request parked under[infra] allocation = "manual", which previously failed at server startup since nothing could approve a parked request. systemslab infra estimateforecasts an experiment's on-demand/spot cost from run history and current prices, andsystemslab infra recommendranks instance types by expected cost per successful run.systemslab submit --budget <usd>(requires--host-count) runs the same estimate and refuses to submit when the on-demand-high figure exceeds the budget or the cost is unknown, unless--yesoverrides.systemslab context newgains--max-instances,--pin-hosts, and--no-autoscaleflags to set a context's infra scheduling policy at creation time; there is currently no command to change them afterward.- A
rezolus_addroption inagent.tomllets an operator point an agent's metrics recording at a different host's rezolus endpoint, for a workload that actually runs in a VM, container, or separate target the job only orchestrates. Leaving it unset keeps the existing behaviour of scraping the local machine.
Changed
- The metrics dashboard shown on experiment and context artifact pages now matches Rezolus's own viewer: a paired cgroup selector, a GPU selector, a PromQL Query Explorer tab, a regrouped nav, and "Charts with no data" instead of blank cards for a sampler the recording lacks. The old
vllm-metrics,sgl-metrics,llm-perf,rpc-perf, andmetrics-v2URLs still work and redirect to the unified page. This also fixes opening an untagged workload recording sometimes rendering a different job's Rezolus data under it.
Fixed
- An agent whose host the server had dropped never came back. The agent registered once, at startup, and was passive after that, so nothing told it the server had forgotten it: the host stayed missing until someone restarted the agent, and experiments needing it queued indefinitely with no error surfaced anywhere. Agents now check that the server still has a host for them every
registration_check_intervalseconds (default 600, 0 to disable) and register again, under a new host id, when it does not. (#6079) - Fixed
/api/v1/artifactignoring theexperiment,context, andreportfilters for artifacts that are not attached to that kind of entity, so a query such as?experiment=<id>also returned every context artifact in the installation. This madesystemslab artifact list --experiment <id>andsystemslab artifact download-all <id>report and download artifacts belonging to unrelated experiments. (#6082) - Fixed a bug where a recording composed from multiple workloads (e.g. rezolus + cachecannon) could silently lose one workload's dashboard section, and sped up dashboard loads that were taking several seconds.
- Fixed several infra manager accounting bugs that could let fast, high-throughput jobs evade a context's
max_instancescap, skew cost estimates after a server restart, or overcharge a drained machine.