oneDNN v3.13

oneAPI Deep Neural Network Library (oneDNN)

Release notes

# Performance Optimizations
## Intel 64/AMD64 Processors
* Improved performance on future Intel Core Ultra processors with Intel AVX10.2 instruction set support (codename Nova Lake).
* Improved performance of matmul on processors with Intel AMX instruction set support.
* Improved performance of `bf16`, `f16`, and `f32` matmul with unit M, N or K dimentions (GEMV-like) on processors with Intel AVX2 instruction set support.
* Improved performance of `u8`/`s8` matmul with `u4`/`s4` weights and grouped scales.
* Improved peformance of `bf16` and `f16` matmul with `f8` weights.
* Improved performance of `f8` [quantized Scaled Dot Product Attention (SDPA)] subgraph with Graph API.

[quantized Scaled Dot Product Attention (SDPA)]: https://uxlfoundation.github.io/oneDNN/v3.13/dev_guide_graph_sdpa_quantized.html

## Intel Graphics
* Improved performance for future integrated GPUs based on Xe3p-LPG architecture (codename Nova Lake P).
* Improved `u8`/`s8` convolution performance on Intel Arc B-series graphics.
* Improved `f16` and `u8`/`s8` matmul performance with `u8`/`s8` and `u4`/`s4` weights in non-transposed layout.

## AArch64 Processors
* Improved performance of `u8`/`s8` matmuls with `u8`/`s8`, `f16`, or `s32` outputs.
* Improved performance of `u8`/`s8` convolutions with `bf16` or `f32` outputs.
* Improved `u8`/`s8` layer normalization performance.
* Improved performance of convolution backpropagation and pooling on platforms with 128-bit SVE.
* Improved performance of `bf16` inner-product.
* Improved multi-threaded bnorm performance.
* Improved binary primitive and post-op performance.
* Improved performance of eltwise primitive with `gelu_erf` algorithm and post-op.

## RISC-V Processors
* Improved `f32` convolution, matmul, inner product, binary, eltwise, pooling, batch normalization, and group normalization primitive performance on processors with `V` extension support.
* Improved `f16` matmul, binary, eltwise, pooling, softmax, and layer normalization primitive performance on processors with `Zvfh` extension support.
* Improved `bf16` matmul primitive performance on processors with `Zvfbfwma` extension support.

# Functionality

## Functional API
* **[experimental]** Introduced support for eltwise and binary post-ops in matmul with grouped memory. Optimized implementation is available on Intel GPUs.
* **[experimental]** Extended grouped matmul with NVFP4 quantization scheme, including support for `f4_e2m1` tensors with `f8_e4m3` grouped scales and per-group binary post-op to implement global `fp32` scale. This is an experimental feature that requires opt-in with [`ONEDNN_EXPERIMENTAL_GROUPED_MEMORY=ON`] build option.

[`ONEDNN_EXPERIMENTAL_GROUPED_MEMORY=ON`]: https://uxlfoundation.github.io/oneDNN/v3.13/dev_guide_experimental.html#onednn-experimental-grouped-memory

## Graph API
* Introduced support for device-side seed, offset, and probability arguments for `Dropout` operation.

[`Dropout`]: https://uxlfoundation.github.io/oneDNN/v3.13/dev_guide_op_dropout.html

# Usability

## Common
* Introduced [user-managed scratchpad] support in Graph API.

[user-managed scratchpad]: https://uxlfoundation.github.io/oneDNN/v3.13/dev_guide_graph_scratchpad.html

## Intel Graphics
* Refactored verbose profiling on Intel GPUs to avoid spurious synchronizations with SYCL or OpenCL runtimes. The new implementation reports device time instead of host time and is compatible with SYCL Graph record/replay mode. 
* Reduced memory consumption of [Gated MLP subgraph] with Graph API.
* Enabled interoperability with SYCL Graph native recording mode for Intel GPUs.
* Introduced `ONEDNN_ZE_INCLUDE_DIR` and `ONEDNN_OCL_INCLUDE_DIR` build knobs to use Level Zero or OpenCL headers from a user-defined location instead of the vendored headers.
* **[experimental]** Introduced support for [persistent cache] with Level Zero runtime on GPU. Level Zero support is experimental.

## AArch64 Processors
*…

Share this resource