My DecPort experiments found label-free, input-specific decision-behavior transfer across three frozen LLM backbones, while a preregistered final gate found no evidence that the learned shared DecisionCore itself was portable.
Yesterday I published Distinguishing Cross-Backbone Decision-Behavior Transfer from Frozen-Core Portability, the research article that closes the first DecPort investigation.
The result is deliberately split in two.
Input-specific decision behavior did transfer from a source decision system into three heterogeneous frozen language-model backbones without using target ground-truth labels during transfer. But the stronger idea I originally cared about did not survive: the particular learned shared DecisionCore did not provide a measurable advantage over a matched random core, and it did not beat a simpler target-specific distillation route.
That distinction is the reason the final verdict in the DecPort repository is NO-SHIP.
The original DecPort idea was straightforward. If a decision system has learned something reusable about how to turn model representations into decisions, perhaps that logic can be frozen once and connected to different language-model backbones through small adapters.
There are actually two claims hidden inside that idea.
The first is behavior transfer: can a target model learn to reproduce the source system's input-specific probability distributions without seeing the target labels?
The second is core portability: does the particular learned decision core contain reusable functionality that matters after the backbone changes?
Those are not the same claim. A target can imitate a source system successfully even if the frozen shared component contributes nothing special.
That became clear in the first accepted DecPort experiment using the released Open-Jev 2B system. Its decision head is a scalar Linear(2048 -> 1) readout. Across SmolLM2-360M, Gemma 3 270M, and TinyLlama-1.1B, adapters trained from Open-Jev's probability distributions transferred useful Choice and Score behavior. A mismatched-teacher control showed that the effect was input-specific.
But a matched random frozen core performed almost as well.
The reason was structural. With a rank-one linear readout and an expressive adapter in front of it, the adapter can absorb the readout direction. A positive transfer result therefore could not identify the pretrained head itself as the reusable part. The research progression records that experiment as evidence for behavior transfer, not evidence for core reuse.
The final experiment tried to remove that simple explanation rather than reinterpret the earlier result.
I replaced the linear core with a learned nonlinear network containing 1,118,977 parameters:
LayerNorm -> 2048 -> 512 -> GELU -> 128 -> GELU -> 1
The target-side adapter was deliberately weakened to a rank-128 linear map. The source core was trained once on a separate labeled source-only split, calibrated, frozen, and then reused for every target and seed. Target transfer still used only source-system probability outputs, not target labels.
The preregistered final-gate protocol fixed the architecture, data, loss, training budget, controls, and pass thresholds before the full run.
The experiment used five seeds, 4,500 unlabeled transfer decisions, 2,000 in-distribution evaluation decisions, and 1,500 out-of-distribution decisions. Every target was tested under five conditions:
| Condition | What it tests |
|---|---|
| Learned shared core | The DecPort hypothesis |
| Scale-matched random core | Whether the particular learned core matters |
| Mismatched teacher | Whether transfer is input-specific |
| Untrained adapter | Whether training adds useful signal |
| Target-specific module | Whether sharing the core is practically useful |
This separation ended up being more important than any single accuracy number.
Against the mismatched-teacher control, the learned-core condition improved in-distribution accuracy by about 10.5 to 16.9 percentage points across the three targets. The gain was positive in all five seeds for every target.
Out of distribution, the effect became smaller but remained positive: about 4.1 to 7.3 percentage points over the mismatched teacher.
That is the positive DecPort result I think is worth preserving. In this setting, a frozen target backbone can be trained to reproduce input-specific probabilistic behavior from another decision system using only its outputs as supervision. The transfer is not explained by a global class preference because the correctly paired teacher beats the type- and width-matched mismatched control.
The effect is not uniform across decision types. Choice and Score carry most of the useful transfer. Noul fails again: the students largely collapse toward a near-constant answer pattern and do not recover the source system's input-specific behavior.
OOD calibration is also substantially weaker than the in-distribution result. The transfer exists, but it should not be summarized as general decision portability.
The primary test compared the learned nonlinear core with a random core of the same architecture whose layer scales were matched to the learned core.
The difference was effectively zero:
| Target | ID gain over random core | OOD gain over random core |
|---|---|---|
| SmolLM2-360M | 0.0 pp | +0.6 pp |
| Gemma 3 270M | +1.5 pp | 0.0 pp |
| TinyLlama-1.1B | -1.5 pp | -1.1 pp |
None of the three targets came close to the preregistered threshold for demonstrating useful learned-core reuse.
The target-specific baseline made the practical result harder to rescue. It used the same unlabeled teacher outputs, objective, training budget, and a similarly small module attached directly to each target. It matched or beat the shared-core route almost everywhere. Only TinyLlama satisfied the parameter-efficiency match condition, because its DecPort adapter was less than half the size of its target-specific baseline.
The external JevBench check was also weak. On the public subset of 231 tasks, the learned-core systems scored 89, 73, and 74 correct for SmolLM2, Gemma, and TinyLlama respectively, against an expected 73.4 from uniform guessing. Only SmolLM2 cleared the preregistered margin.
The mechanical verdict therefore failed three of five criteria, including the primary one. The original reusable-core hypothesis was not supported in the tested setting.
For me, the most useful part of the project is not that one hypothesis failed. It is that the controls separated a real secondary effect from the mechanism I initially wanted to credit for it.
Without the random-core comparison, the positive transfer result could easily have been presented as evidence that a learned decision module had become portable. Without the mismatched-teacher control, it would have been harder to show that the target was learning input-specific behavior rather than a broad output prior. Without the target-specific baseline, even a genuine shared-core effect would say little about whether the architecture was worth using.
The final gate also made it difficult to move the goalposts after seeing the results. The thresholds and failure conditions were committed before the full experiment, and the verdict was computed from them rather than chosen afterward.
The run passed its audit. A second execution using the same code, configuration, data, hardware, and software environment reproduced all 93 compared result files with a maximum numeric difference of zero, and all 81 saved tensor artifacts were byte-identical. That is strong evidence that the archived measurement is internally reproducible.
It is not an independent replication, and I do not treat it as one.
The conclusion is intentionally narrow.
The experiment covers three target model families, one source representation boundary, the archived datasets and training budget, one nonlinear core design, and one set of preregistered thresholds. It does not prove that no portable decision core can exist.
The reproduction also used the same hardware and software environment as the original run. There has been no independent or cross-hardware replication.
JevBench was evaluated only on its 231-task public subset, not the full 534-task benchmark. TinyLlama also refused 37 tasks that exceeded its context limit, and those were scored as incorrect.
Finally, the negative result for the shared core should not erase the positive behavior-transfer result. The paper's main methodological point is exactly the opposite: decision-behavior transfer and decision-core portability are separate empirical questions. In DecPort, the first survived the controls and the second did not.
That is a less convenient conclusion than the original project hypothesis, but it is also a more useful one.
