llms.txt

# cuDNN Frontend

> NVIDIA cuDNN Frontend (FE): a header-only C++ library and Python package (`nvidia-cudnn-frontend`, import `cudnn`) exposing the cuDNN Graph API, plus open-source CuTeDSL kernels (SDPA/Flash Attention, MoE grouped-GEMM fusions, fused normalizations) for Hopper and Blackwell GPUs.

Published documentation: https://docs.nvidia.com/deeplearning/cudnn/latest/developer/overview.html

## Start here

- [README](https://github.com/NVIDIA/cudnn-frontend/blob/main/README.md): overview, installation, feature highlights
- [AGENTS.md](https://github.com/NVIDIA/cudnn-frontend/blob/main/AGENTS.md): repo map, verified build/test/format commands, conventions
- [Contributing](https://github.com/NVIDIA/cudnn-frontend/blob/main/CONTRIBUTING.md): development environment and PR workflow
- [Python package guide](https://github.com/NVIDIA/cudnn-frontend/blob/main/python/cudnn/README.md): package structure, adding frontend-only APIs

## Graph operation reference

- [Attention (SDPA fwd/bwd, FP8)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Attention.md)
- [Matmul](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Matmul.md)
- [Convolutions](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Convolutions.md)
- [Normalizations (LayerNorm, RMSNorm, BatchNorm, InstanceNorm)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Normalizations.md)
- [MoE Grouped Matmul](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/MoeGroupedMatmul.md)
- [Pointwise](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Pointwise.md)
- [Block Scaling (MXFP8/NVFP4 quantization)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/BlockScaling.md)
- [RoPE](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/RoPE.md)
- [Causal Conv1d](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/CausalConv1d.md)
- [GNN simple aggregation](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/gnn/agg_simple.md)
- [FFT Causal Conv1d](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/FFTCausalConv1d.md)
- [Concatenate](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Concatenate.md), [Reshape](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Reshape.md), [Slice](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Slice.md), [Transpose](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Transpose.md), [Resampling](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/Resampling.md)

## Open-source (frontend-only) kernel APIs

- [FE OSS APIs overview — full catalog and usage pattern](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/overview.md)
- [Flex Attention API](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/attention/flex_attention.md) and [mask plan design](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/attention/flex_attention_design.md)
- [FROST SDPA support-matrix tracker — what the FROST SDPA engines serve, per architecture](https://github.com/NVIDIA/cudnn-frontend/blob/main/python/cudnn/sdpa/frost/SUPPORT_MATRIX_TRACKER.md)
- [Block-sparse attention (BSA)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/bsa.md), [DeepSeek sparse attention (DSA)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/dsa.md), [Native sparse attention (NSA)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/nsa.md)
- [GEMM fusions (amax, SwiGLU, sReLU, grouped/discrete MoE variants)](https://github.com/NVIDIA/cudnn-frontend/tree/main/docs/fe-oss-apis/gemm_fusions)
- [MoE with Expert Parallelism (MoeEP)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/operations/moe_ep.md)
- [Gated attention block (SM107): projection + QK-norm/RoPE + SDPA + sigmoid gate + out projection as one FROST block, bf16 / FP8 / MXFP8, optional MXFP4 weights and NVFP4 / MXFP4 output](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/gated_attention_block.md)
- [RMSNorm + RHT + Amax](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/rmsnorm_rht_amax.md), [RMSNorm + SiLU](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/fe-oss-apis/rmsnorm_silu.md)

## How-to guides

- [CUDA graphs](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/cuda-graphs.md)
- [Deviceless ahead-of-time compilation](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/deviceless-ahead-of-time-compilation.md)
- [Dynamic kernel cache](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/dynamic-kernel-cache.md)
- [Custom execution plans](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/custom-execution-plan.md)
- [Compile-time constants](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/CompileTimeConstants.md)
- [Adding PyTorch custom ops](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/adding_torch_custom_ops.md)
- [Python graph and execution backends](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/python_graph_and_execution_backends.md)
- [Composing multi-kernel blocks in Python (workspace, streams, fusion knobs, typed declines)](https://github.com/NVIDIA/cudnn-frontend/blob/main/docs/utilities/composing_kernel_blocks.md)

## Examples

- [C++ samples (Catch2)](https://github.com/NVIDIA/cudnn-frontend/tree/main/samples/cpp)
- [Python notebook tutorials](https://github.com/NVIDIA/cudnn-frontend/tree/main/samples/python)
- [SDPA training benchmark](https://github.com/NVIDIA/cudnn-frontend/tree/main/benchmark/attention_training)

## Optional

- [Release notes](https://github.com/NVIDIA/cudnn-frontend/releases)
- [PyPI package](https://pypi.org/project/nvidia-cudnn-frontend/)
- [cuDNN backend API reference](https://docs.nvidia.com/deeplearning/cudnn/latest/api/overview.html)
- [Acknowledgements](https://github.com/NVIDIA/cudnn-frontend/blob/main/ACKNOWLEDGEMENTS.md)