mirror of
https://github.com/opencv/opencv.git
synced 2026-11-21 08:45:29 -06:00
dnn : int8 optimisation - #30289 ### Correctness - **AVX-VNNI 1x1 conv, leftover pixels:** the last `H*W % 8` output pixels read the VNNI-repacked weights in the wrong order. Fixes the wrong top-1 of MobileNetV2 and DenseNet121 int8 with VNNI. - **Padded taps in scalar paths:** the depthwise (AVX2) tail and the NEON kxk scalar path skipped out-of-image taps instead of using the input zero point. The VNNI kxk kernel now never leaves a tile under 8 pixels. ### Speed - **First conv:** quantizing the float image is now parallel and vectorized. A float-input conv with up to 4 channels runs as a 1x1 conv over each pixel's window. First conv 5-6x faster (ResNet50: 0.88 -> 0.16 ms). `OPENCV_DNN_INT8_CONV_WINDOW_PACKING=0` disables the window packing. - **AVX-VNNI depthwise conv:** a new kernel transposes input rows into one row per channel and adds 4 taps for 8 pixels with one `pshufb` + `vpdpbusd`. Depthwise about 2x faster, output bit-identical. - **int8 MatMul at batch 1:** uses all threads instead of one. AlexNet int8: 2.75 -> 1.52 ms. - **Asymmetric padding:** convs with TF "SAME" padding at stride 2 are now fused to int8 instead of running in float between Dequantize and Quantize. Float convs in EfficientNet-Lite4 and MP hand pose: 5 -> 0-1. - **Broadcast int8 Add/Mul:** copies whole contiguous runs in parallel instead of computing an index per element. - **int8 Quantize/Dequantize:** AVX2 fast path for signed int8; per-tensor Q/DQ keep the block layout. ### Tests - Companion PR : opencv/opencv_extra#1436 (test data). ### Results (ms, i9-14900KS , default threads) | Model | 5.x before (ms) | this PR (ms) | ONNX Runtime 1.30 (ms) | Speedup vs 5.x | ORT / this PR | |---|---|---|---|---|---| | DenseNet121 int8 | 109.4 | 14.7 | 16.1 | 7.43x | 1.09x | | PP-OCRv3 det int8 | 65.7 | 12.7 | 33.1 | 5.15x | 2.60x | | SFace int8 | 8.1 | 3.0 | 19.0 | 2.75x | 6.43x | | EfficientNet-Lite4 int8 | 4.1 | 2.9 | 1.9 | 1.40x | 0.67x | | EfficientNet-Lite4 qdq | 4.6 | 3.1 | 2.0 | 1.49x | 0.65x | | MP hand pose int8 | 2.5 | 0.8 | 2.2 | 3.06x | 2.69x | | PPHumanSeg int8 | 2.6 | 1.9 | 5.6 | 1.41x | 3.02x | Co-authored with @Prasadayus ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake