Spark

SparkLang

Three CPU train methods — not LoRA

A language beats another Python SDK because one reviewable .spark file covers train → status → expect, with dry-run fixtures you can run in CI without API keys.

For solo and small teams who want reviewable train-job pipelines in PRs — without maintaining SDK glue.

Offline-first by default: ./spark --dry-run. Live submit POSTs to a trainer you run — SPARK_TRAIN_URL. Reference methods (CPU only): spark_distill_cpu, spark_pref_pack, spark_playbook_fit — real weights / packs, not LoRA, not a voice GPU.

Download Spark Playground Getting started

Why a language?

Library glue scatters prompts, training scripts, and eval harnesses across repos. A .spark program is one reviewable surface — same file for dry-run CI and live ops.

Typical Python SDK soup

client = OpenAI(...)
resp = client.chat.completions.create(...)
# separate train script, eval notebook,
# CI secrets, ad-hoc shell glue…

One .spark program

model train dataset "data.jsonl" base "base" out "out/job-1" backend "http" -> job
model status "job-1" -> status
expect contains job fixture "…/want_accepted.txt"
expect contains status fixture "…/want_succeeded.txt"

# ./spark --dry-run train_eval.spark

The wedge

Dry-run by default

./spark --dry-run uses offline fixtures — no keys, no network. Pass --live only when you want real calls. 

./spark --dry-run examples/model_train.spark

Train backends wired

model train / model build submit via backend "http" to your trainer at SPARK_TRAIN_URL (POST …/jobs, GET …/jobs/{id}). Dry-run uses fixtures; live needs a service that implements the Trainer HTTP contract — in-repo reference: tools/spark-train-ref ships three CPU methods: spark_distill_cpu, spark_pref_pack, spark_playbook_fit (real weights.pt / pref_pack.json+ranker / playbooks.json+router — not LoRA). Captures: live-train-capture.txt, live-train-methods-capture.txt. 

model train dataset "…" base "…" out "…" backend "http" -> job
model status "job-id" -> status

Eval in the same file

After train, assert bound job and status fields with expect contains … fixture (exit 0/1) — gate of this job, not unrelated gateway aliases. 

expect contains job fixture "…/want_accepted.txt"
expect contains status fixture "…/want_succeeded.txt"

Train → status → expect

Dry-run writes fixture artifact paths (no GPU). Live POSTs to the trainer at SPARK_TRAIN_URL (backend "http"), or an allowlisted local-yield unit.

  1. 1 Train Submit dataset, base, and output path
  2. 2 Status Poll the job and bind artifact JSON
  3. 3 Expect assert bound job/status (exit 0/1) 
# examples/train_eval.spark — dry-run first
model train dataset "examples/fixtures/train/dataset.jsonl" base "fixture-base" out "out/train/job-dry-001" backend "http" -> job
model status "job-dry-001" -> status
expect contains job fixture "examples/fixtures/train/want_accepted.txt"
expect contains status fixture "examples/fixtures/train/want_succeeded.txt"

Fail path: examples/train_eval_fail.spark (exit 1). Live HTTP proof (three CPU methods, via ./spark --live / ./spark-train-http --method): live-train-methods-capture.txt. Model training docs · Build a Model · What's real today.

Also available

IDE ops, embed/retrieve, voice demos, HTTP get/post, and network/browser surfaces live in the language reference and what's real today — not the homepage thesis.

Learn Spark

Follow the trail: install → first dry-run → train→status→expect → gate a job.

Try it

# Dry-run train → status → expect — no network, no API key
./spark --dry-run examples/train_eval.spark

# Optional live submit — your trainer at SPARK_TRAIN_URL
export SPARK_TRAIN_URL=http://127.0.0.1:8090/v1
./spark --live examples/model_train.spark

See programming guide and language reference.