zxc v0.14.0

Lossless compression C library built for ultra-fast decode. Faster than LZ4 (40–75% on ARM64), at a better ratio. Official Rust, Python, Node.js, Go and WASM bindings. Write-once/read-many, with optional seekable archives for O(1) random access.

Release notes

# Release Notes

v0.14.0 is a decode-speed release built on a format break.

**This release boosts decompression speed by up to 19% over v0.13.x, up to doubling our performance lead over LZ4 on ARM64 (+85% to +105%) and up to tripling it on x86_64 (+50% to +175%).**

Three independent changes get there: blocks stop describing what the decoder can derive (**format v8**), short back-references are copied 32 bytes at a time instead of 16, and at levels 1-5 the encoder keeps its matches far enough back that the widest SIMD copy path stays on. The PivCo Huffman kernels also picked up AVX-512 VBMI, 256-bit AVX2 and NEON work.

Nothing in the public API moved: no symbol added, none removed, **ABI unchanged** (`libzxc.so.4`). The break is in the container format.

> ⚠️ **Breaking change (format).** The container format is now **v8**. v0.14.0 writes v8 archives that v0.13.x and earlier decoders reject, and it cannot read v7 or older: the header version byte is checked for exact equality (`ZXC_ERROR_BAD_VERSION`). The **ABI is unchanged** (soname stays `libzxc.so.4`), so no re-link is required — but upgrading the library alone makes newly written archives v8.
>
> **Migrating v7 -> v8** is a one-time transcode — decompress with your old build, recompress with the new one:
>
> ```sh
> zxc-old -dc old.zxc | zxc-new -z -c > new.zxc
> ```
>
> Keep a v7 build around until your data at rest is converted; if the old archive used a dictionary, pass it to the decompress side (`zxc-old -dc -D dict.zxd`). Full guide (bulk migration, dictionaries, verification): **[docs/MIGRATION.md](https://github.com/hellobertrand/zxc/blob/v0.14.0/docs/MIGRATION.md)**.

## ZXC format v8

Compression is untouched — same parser, same match finder, same entropy coders. What changed is how a block describes itself, and every change pulls the same way: **stop writing down what the decoder can work out.** (#374)

- **Block sub-header: 16 -> 12 bytes.** Four reserved bytes nobody used.
- **Descriptors only when they carry information.** v7 wrote a fixed 32-byte table of section sizes; most of them follow from the header. v8 keeps the two that don't: the literal section's compressed size when it is RLE- or entropy-coded, and the token section's at level 7. GLO now carries 0, 4 or 8 descriptor bytes, GHI none. The extras section is the leftover payload.
- **`enc_litlen` -> `enc_tok`.** The field always described the token section, never literal lengths.
- **32 readable bytes must follow the literal section.** Literals are copied with a wild copy that overshoots, and those bytes keep the last copy inside the payload. Real blocks clear the bar without trying; the encoder pads into the extras when a tiny block doesn't. Always relied on, never written down — now normative, and decoders must reject blocks that break it.
- **`enc_lit`, `enc_tok` and `enc_off` are closed value sets.** Decoders reject unknown values instead of defaulting, which keeps the spare values free for a future version.

Per block this saves 25 to 34 bytes: invisible at the 512 KB default (levels 6 and 7 land 0.02 % smaller), worth about **2 %** at the 4 KB blocks where dictionaries live.

## Performance

Decompression on Silesia (202 MB), single-threaded, v0.13.3 -> v0.14.0 on the same reference hardware:

| Level | Apple M2 | Neoverse-V2 | EPYC Zen 5 | EPYC Zen 3 |
|:---|---:|---:|---:|---:|
| **-1** | +6.5 % | +2.8 % | +4.5 % | +3.4 % |
| **-2** | +7.7 % | +2.1 % | +6.2 % | +3.1 % |
| **-3** | **+19.0 %** | **+16.6 %** | **+10.9 %** | **+13.4 %** |
| **-4** | **+18.7 %** | **+16.6 %** | **+10.2 %** | **+13.2 %** |
| **-5** | **+18.0 %** | **+16.8 %** | +9.8 % | **+13.0 %** |
| **-6** | +10.3 % | +9.5 % | +3.9 % | +9.3 % |
| **-7** | +9.2 % | +6.2 % | +1.7 % | +6.2 % |

In absolute terms, level 3 decodes at **8.4 GB/s** on Apple M2 and **6.7 GB/s** on Zen 5; level 6 holds **6.7 GB/s** on M2 at a 36.28 % ratio, ahead of `lz4hc -9` on both axes on every machine …

Share this resource