Models
Each model here was converted with litetune and compared with its float reference on held-out data. A card says what the model is and which checkpoint the run used, and links to the section that measured it.
Every score below is the accuracy of the centroid decider verify fits, on held-out rows, against the float32 fine-tune as the reference. Every run, with the abstention it measured
270M parameters: the text tower of Google's EmbeddingGemma 2. An app uses it through flutter_edge_ai's EmbeddingModel.createDecider, which embeds the input and answers with the option whose examples have the nearest centroid. litetune fine-tunes the tower on one decision and writes the result into Google's own .litertlm file, so the app loads one file and nothing ships beside it.
What litetune does
Fine-tunes the whole text tower on a decision's rows, in float32, with the one recipe that was measured: there are no training flags. It runs in an environment of its own.
Maps every weight and norm of the fine-tune onto Google's graph, writes them as int4 (the default, 164,626,432 bytes, Google's size) or int8 (300,089,344 bytes), and reads the result back against Google's file byte for byte outside what it wrote. A weight it cannot place is refused rather than left as Google's.
Embeds every training and held-out row with the float32 fine-tune and with the file, on LiteRT-LM's CPU or, with --backend gpu, on a Mac's GPU. It fits the centroid decider on ten examples per option, five draws, and passes when every draw's upper 95 % bound on the accuracy difference is within 0.01.
Compiles the int8 file's text encoder ahead of time for a Qualcomm SoC: SM8550, SM8650, SM8750 or SM8850, the four Google publishes NPU files for. int8 only, on Linux x86-64 only. verify cannot run such a file; the SM8750 one has run on a phone, and no other SoC's file has run on a device.
What was measured
banking77, through litetune's own stages
77 options. prepare → tune → convert (int8 and int4) → verify --with-base, on prepare's split: 9,158 training and 2,617 held-out rows. Accuracy is the 10-shot centroid decider's mean over five draws; LiteRT-LM 0.18.0's EmbeddingEngine on an x86 host's CPU.
| embeddings | 10-shot accuracy | conversion cost | verify |
|---|---|---|---|
| Google's file (base model) | 0.9137 | — | — |
| fine-tuned, float32 (the reference) | 0.9312 | — | — |
| int8 file | 0.9314 | −0.0002 | passed |
| int4 file | 0.9276 | +0.0037 | passed |
The int4 cost resolved as above zero in three of the five draws; every draw's upper bound was under 0.01.
At ten examples per option the files gained +0.0177 (int8) and +0.0138 (int4) over Google's file. With a linear layer fitted on every training row, Google's file scored higher than the fine-tune: 0.9534 against 0.9389.
CLINC150, from the published package
150 options. The same stages from litetune 0.7.0 as installed from PyPI, on prepare's split: 13,650 training and 3,900 held-out rows. The same decider and runtime.
| embeddings | 10-shot accuracy | conversion cost | verify |
|---|---|---|---|
| Google's file (base model) | 0.9493 | — | — |
| fine-tuned, float32 (the reference) | 0.9791 | — | — |
| int8 file | 0.9792 | −0.0001 | passed |
| int4 file | 0.9765 | +0.0026 | passed |
Every draw's upper bound was under 0.01. The files gained +0.0299 (int8) and +0.0272 (int4) over Google's file at ten examples per option.
The CPU and the GPU on a Mac
verify on the banking77 int8 file, on one Mac, with LiteRT-LM 0.18.0 from PyPI, on the CPU and with --backend gpu.
| CPU | GPU | |
|---|---|---|
| verify | passed | passed |
| the file, 10-shot | 0.9317 | 0.9311 |
| the reference, 10-shot | 0.9312 | 0.9312 |
| cosine to the reference, mean | 0.99928 | 0.9969 |
| median per text | 43.43 ms | 11.71 ms |
The GPU computes in the precision LiteRT-LM picks, F16 unless the file declares otherwise, and its vectors sit further from the reference than the CPU's.
On a Qualcomm NPU
Measured once, on 2026-10-31. The banking77 int8 file with its text encoder compiled ahead of time for SM8750, run on a Galaxy S25 Ultra through LiteRT-LM 0.18.0's EmbeddingEngine C API with the Qualcomm dispatch and QNN runtime 2.50.0. The same 2,617 held-out rows; the cost is against the float32 reference, paired.
| side | 10-shot accuracy | cost | ms per text |
|---|---|---|---|
| float32 reference | 0.9312 | — | — |
| NPU, the int8 file compiled for SM8750 | 0.9316 | −0.0004 | 9.86 |
| the phone's CPU, the int8 file, 4 threads | 0.9314 | −0.0002 | 57.6 |
| NPU, Google's SM8750 file (base model) | 0.9169 | +0.0144 | 6.73 |
The NPU file's cost was unresolved in 5 of 5 draws. It is 311,705,600 bytes. convert --npu is int8 only: an int4 file compiled the same way did not load on the phone's NPU.
No text was longer than 105 tokens, so only the 128-token signature ran.
What was not
- Option descriptions, which the app adds to an option's centroid.
- Tasks that read distances directly, retrieval among them.
- The GPU on Linux, where the 0.18.0 package's GPU engine did not start in the one container tried, and Windows.
- The app's own runtime build, and a phone's CPU through the app.
- Training anywhere but the CPU.
- An NPU file for any SoC but SM8750 on a device.