llms.txt · agent.md · grammar.ebnf · openapi.json · SKILL.md · examples

Spark

v0.8.0 · 67e37dc7ecca · verified 2026-09-04 · Changelog

real

SparkLang

SparkLang is a reviewable .spark language from train to patch to gate. Dry-run with fixtures; LoRA is a backend method when you have a GPU.

Example

CallsBack.ai / VoiceCore train_sam_lora.spark (redacted). Live 2026-09-03 numbers. Histogram floors of 20 fail on a thin freeze — that is the gate. 

# VoiceCore reference — Sam store persona LoRA (redacted for public)
# Dataset: #321 mini-freeze 140 rows
# sha256: 272b64eb1d40aaf8a186df6a2f6c8e7090a1d69d03ac4928ac75a246f8e439f0
# Live trainer: SPARK_TRAIN_URL (host redacted)
# Histogram floors of 20 fail on a thin freeze — that is the gate.
model code
model train dataset "examples/fixtures/train/dataset.jsonl" base "hf://Qwen/Qwen3-27B" out "out/train/job-lora-001" backend "http" method "lora_bf16" -> job
model status "job-lora-001" -> status
expect gte status $.metrics.golden_bucket 0.8
expect eq status $.metrics.grounding_facts true
expect eq status $.metrics.rachel_bleed 0
expect histogram_min status W2a 20
expect histogram_min status W2b 20
expect histogram_min status W-SNC 20
expect histogram_min status W-TF-RECOVER 20

Download train_sam_lora.spark

expectresultgot
gte golden_bucket ≥ 0.8PASS0.9929
eq grounding_facts truePASStrue
eq rachel_bleed 0PASS0
histogram_min W2a ≥ 20FAIL12
histogram_min W2b ≥ 20FAIL0
histogram_min W-SNC ≥ 20FAIL17
histogram_min W-TF-RECOVER ≥ 20PASS111

class_histogram (live)

{
  "W2a": 12,
  "W2b": 0,
  "W-SNC": 17,
  "W-TF-RECOVER": 111
}

adapter diff heatmap (live)

{
  "layer_count": 2,
  "delta_norms": {
    "base_model.model.layers.0.self_attn.q_proj.lora_A.weight": 12.1372,
    "base_model.model.layers.0.self_attn.q_proj.lora_B.weight": 0.0
  },
  "base": "hf://Qwen/Qwen3-27B",
  "rank": 16,
  "note": "live 2026-09-03 job-lora-sam-live-20260903; stub_weights"
}

What's real today

Same table as spark --status.

verbrealitysince
model trainreal0.6.0
model statusreal0.6.0
model inspectreal0.8.0
model diffreal0.8.0
model probereal0.8.0
model decompileexperimental0.8.0
model patchexperimental0.8.0
model evalreal0.8.0
model stagereal0.8.0
expect containsreal0.6.0
expect gtereal0.7.0
expect ltereal0.7.0
expect eqreal0.7.0
expect histogram_minreal0.7.0
expect scorereal0.7.0
http getreal0.6.0
http postreal0.6.0
spark_distill_cpureal0.6.0
spark_pref_packreal0.6.0
spark_playbook_fitreal0.6.0
spark_faq_indexreal0.6.0
lora_bf16real0.7.0
hf://real0.7.0
GitHub Action dry-runreal0.7.0

Four CPU methods + lora_bf16 + numeric expects + hf:// + GitHub Action + ModelScope inspect/diff/probe. decompile/patch experimental until probe AUROC ≥ 0.85.

Install

pip install sparklang
spark --dry-run examples/train_eval.spark
uses: Michael20251/sparklang/.github/actions/dry-run@main
docker compose -f tools/spark-train-ref/docker-compose.yml up
model code
model train dataset "examples/fixtures/train/dataset.jsonl" base "fixture-base" out "out/train/job-dry-001" backend "http" method "spark_distill_cpu" -> job
model status "job-dry-001" -> status
expect contains job fixture "examples/fixtures/train/want_accepted.txt"
expect contains status fixture "examples/fixtures/train/want_succeeded.txt"
curl -sS -X POST "$SPARK_TRAIN_URL/jobs" -H 'Content-Type: application/json' -d '{"dataset":"examples/fixtures/train/dataset.jsonl","base":"fixture-base","out":"out/train/job-dry-001","backend":"http","method":"spark_distill_cpu"}'