int8

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

An innovative library for efficient LLM inference via low-bit quantization

Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.

Reimplement RetinaFace use C++ and TensorRT

tensorrt int8 量化yolov5 onnx模型

TensorRT int8 量化部署 yolov5s 模型，实测3.3ms一帧！

a simple pipline of int8 quantization based on tensorrt.

Nodes to run Hunyuan Image 3 locally with BF16 and NF4 quantized options in Comfyui

RepVGG TensorRT int8 量化，实测推理不到1ms一帧！

👀 Apply YOLOv8 exported with ONNX or TensorRT(FP16, INT8) to the Real-time camera

Trustworthy onboard satellite AI in PyTorch→ONNX→INT8 with calibration, telemetry, and a PhiSat-2 EO tile-filter demo.

World's First NMS-Free YOLOv26n on ESP32-P4. Features end-to-end Int8 QAT and custom C++ optimizations achieving 30% faster inference than the official ESP-DL YOLOv11n (1.7s vs 2.4s).

nanodet int8 量化，实测推理2ms一帧！

Pure C implementation of FastEnhancer. 3.3x faster than the ONNX Runtime graph, same quality.

NCNN+Int8+YOLOv4 quantitative modeling and real-time inference

INT8 calibrator for ONNX model with dynamic batch_size at the input and NMS module at the output. C++ Implementation.

A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.

TensorRT Int8 Python version sample. TensorRT Int8 Python 实现例子。TensorRT Int8 Pythonの例です

离线可用的本地类型化决策：4 核 CPU 单题 15.6ms。Local & offline Jev / System One inference on CPU — ONNX + INT8, no torch at runtime. 支持 laya / kev / PlayJev

An stress and benchmark utility for NVIDIA GPUs. Measures performance across various precisions (FP64, FP32, TF32, FP16, INT8) and monitors real-time vitals like power, temperature, and clock speeds.