Part 1 of a series on Program-As-Weights, a framework to compile natural language program specs into fuzzy functions: what it is, how it's built, the local compiler I reproduced, and why it matters.
I’ve been looking into Program-As-Weights (PAW), a research project (arxiv.org/abs/2607.02512), with a core idea they call fuzzy-function programming: compiling a function from an English language spec using a large model once, to create a small locally-executable model that is called many times.
As background, many LLM features get built the same way: a request comes in, it goes to a hosted API, a frontier model handles it, and you wear the token cost 💸 and the network latency 🐌. For hard or novel problems that’s a reasonable trade. But if you look at what a lot of production LLM traffic actually is, it’s a similar narrow task repeated thousands of times a day. Paying frontier prices per call for that has always felt like overkill, and PAW is a potentially cheaper, faster alternative.
The way it works: you write a plain English spec for the task (e.g. routing a support message based on urgency). A large “compiler” model runs once to turn that spec into a small model adapter file. After that, every call runs on a small frozen model with the adapter loaded on your own hardware. There’s no API cost at run time, and it’s fast.
The problem it targets: fuzzy functions
The authors built PAW around a category of task they call fuzzy functions: everyday problems that resist clean rule based code but don’t need a chain of reasoning on every call. Things like:
routing a support message to the right team
pulling a price or a date out of messy free text
flagging the log lines that describe an outage
canonicalising company or country names that arrive in a dozen spellings
You’ll recognise these. You start with a regex, the regex grows arms, and at some point someone gives up and calls an LLM. The task is too fuzzy for if/else conditions, but doesn’t justify a frontier model per invocation. That awkward middle ground is where PAW aims.
Compile once, run many times
PAW borrows its shape from ordinary compilers where there are two phases, and the expensive one happens only once.
Compile (once per function). You hand a natural language spec to a 4B parameter compiler model, which the authors trained on an example dataset covering hundreds of categories of different fuzzy tasks. The compiler emits an adapter for your specific function that can be loaded and run by a smaller 0.6B model. Technically this adapter is a LoRA (Low Rank Adaptation) adapter, which is a small set of weight updates that specialises the behaviour of a frozen base model. The adapter file weighs in at only ~23MB.
Interpret (every call). A 0.6B parameter interpreter loads your function-specific adapter and handles every incoming call locally. The interpreter’s base weights remain frozen; switching functions simply means loading a different 23MB adapter.
From the calling code’s point of view, the result is just a Python function:
import programasweights as paw
# One time compile step:
program = paw.compile(
"Classify the incident severity using these rules: outages or data loss are critical; degraded performance is high; cosmetic issues are low. Output exactly one of: critical, high, low."
)
# Inference steps:
severity_fn = paw.function(program.id)
severity_fn("Typo on the About page") # returns "low"
severity_fn("Search results take 15s") # returns "high"
severity_fn("Production database is down") # returns "critical"The interface is simple: define a function in English, compile it once, call it like any other function.
Some concrete numbers
The compiler model (programasweights/paw-4b-qwen3-0.6b) is a custom trained version of a 4B base model (Qwen/Qwen3-4B-Instruct-2507) that you invoke once per function. The artefact the compiler produces is a ~23MB LoRA adapter (about 38.5M parameters at rank 64, quantised to Q4_0). The runtime is a frozen ~430MB interpreter model (programasweights/Qwen3-0.6B-GGUF-Q6_K) that runs fine on a laptop or a modest server, no GPU required.
In my local runs of the pipeline (more on that harness in post 3), compilation took around ~2 seconds per function on Apple Silicon via mps, and queries averaged 40ms to ~100ms depending on the task type. Those numbers are indicative since they’re from one machine and one test set. But even allowing for that, call latency is only a few tens of milliseconds on consumer grade hardware. Against a hosted API call priced per token with latency around 1 second or more, the difference adds up quickly.
Why I think this matters
If you’re an ML engineer, the useful mental model is that PAW moves the intelligence spend from run time to build time, a bit like ahead of time compilation moving work out of the hot path. It’s also worth being clear about what it is not: this is not LoRA fine-tuning. There’s no dataset to collect, no training loop, no per function gradient step. The compiler emits the LoRA adapter directly from the spec in a single forward pass; the learning was amortised into training the compiler model itself. That distinction is key, and post 5 digs into how it relates to hypernetworks and other prior work.
If you’re a technical PM, the relevant questions are cost, latency, and control. A compiled function has low marginal cost and runs entirely on infrastructure you control, which also means customer data doesn’t leave the building at inference time. (See caveats below for how I reproduced a local hosted compilation altrnative to keep your data local for that phase also.) The trade you’re making is capability: a 0.6B interpreter has a real ceiling, and PAW is upfront about targeting fuzzy functions rather than open ended generation or long chains of reasoning.
My takeaway: PAW is a new point on the cost and capability curve rather than a replacement for hosted LLMs. For bounded, high volume, stable tasks, it’s an option worthy of serious evaluation. You also take control of your inference: protection against frontier model version deprecation, and API rate limiting at scale.
The caveats
The interpreter is small, and it behaves like it. In my evals, tasks needing multi step reasoning, calendar arithmetic, or character level string manipulation degraded noticeably (post 4 has the details). PAW works best on short, well-specified input/output tasks like text classification, structured extraction, format repair, routing, and fuzzy matching.
Another important note: the public SDK gives you fully offline inference: once a function is compiled, every call happens on your own hardware. But the compile side is different, calling paw.compile() sends your spec to the hosted compiler at programasweights.com and hands back your binary fuzzy-function paw file. The compiler model itself is available, yet the code that drives it isn’t public at the time of writing.
I wanted an alternative which meant reproducing the correct inputs and outputs around the released compiler model, working from the paper and the various PAW resources online. The result is released in my programasweights-python SDK fork as paw-compiler: a fully-local CLI compiler plus a local FastAPI server that replicates the public compiler API. This enables fully offline compilation and inference. How that reproduction came together, and where the tricky parts were, is the subject of post 3.
What’s next
The obvious question after all this: how does a sentence of English actually become a set of neural network weights? That pipeline, spec to pseudo program to LoRA adapter to interpreter, is the subject of post 2.
Sources: the PAW paper (arxiv.org/abs/2607.02512), the released model card (huggingface.co/programasweights/paw-4b-qwen3-0.6b), and the public SDK (github.com/programasweights/programasweights-python). Latency figures are from my reproduction harness and are labelled as such.