Commit Graph

8 Commits

Author SHA1 Message Date
Savya Sanchi Sharma
0f51622bac Merge pull request #30439 from SavyaSanchi-Sharma:layers
dnn(cuda): transformer layer coverage and CUDA Graph replay on the new engine - #30439

  Follow-up to #30415 (ResNet layer coverage and cuDNN JIT conv/pooling on the new engine). Adds CUDA support on the new DNN engine
  for the layers transformer models need, and brings the CUDA path to parity with ONNX Runtime.
**Companion PR** opencv/opencv_extra#1448

  **Changes**
  - CUDA execs for Attention, Cast, ConstantOfShape, Expand, Gather, Not, Reduce, Shape, Slice, Softmax, Tile, Transpose, Concat,
  Gelu, LayerNormalization, MatMul, Split, Squeeze, Unsqueeze and the element-wise activations.
  - A graph with static shapes is recorded as one CUDA Graph on the second `forward()` and replayed from then on.
  - The cuDNN JIT convolution reads NCHW buffers in place; NHWC staging is kept only for FP16 grouped convolutions.
  - TF32 for cuBLAS GEMMs (`OPENCV_DNN_CUDA_ALLOW_TF32`), FP32 accumulation for FP16 GEMMs
  (`OPENCV_DNN_CUDA_FP16_HALF_ACCUMULATE`), and fused MatMul/Gemm post-ops.
  - Batch-broadcast MatMul and Shape are safe to capture in a CUDA Graph; the unused pointer-array `cublas::gemmBatched` is
  removed.
  - Tests: `forward()` runs three times on batch-broadcast MatMul and Shape models; `MatMul_init_bcast` runs on CUDA again;
  `AttentionSharedShapeReshape` uses the FP16 default tolerance on CUDA FP16.
  - Perf: new CUDA-only `*_CUDA_pinned` tests pin the input shapes.

  **Performance** (ms, median of 50, FP32 with TF32, RTX PRO 6000 Blackwell, idle GPU)

| Model | CPU | CUDA (cuDNN JIT) | CUDA (full cuDNN) | ORT 1.31 CUDA EP |
  |---|---|---|---|---|
  | ResNet-50 | 8.02 | 0.80 | 0.80 | 0.88 |
  | ResNet-18 | 3.27 | 0.44 | 0.44 | 0.48 |
  | MobileNetV2 | 1.99 | 0.28 | 0.27 | 0.40 |
  | DenseNet-121 | 17.42 | 1.48 | 1.58 | 1.58 |
  | EfficientNet-Lite4 | 5.03 | 0.53 | 0.54 | 0.73 |
  | ViT-B/32 | 11.06 | 1.02 | 1.02 | 0.95 |
  | ViT-Base/16 | 24.38 | 1.16 | 1.17 | 1.13 |
  | DeiT-Tiny | 4.58 | 0.65 | 0.65 | 0.68 |
  | MobileViT-XS | 6.93 | 0.72 | 0.72 | 0.77 |
  | YOLOv3 | 47.21 | 2.60 | 2.59 | 2.72 |
  | YOLOv4 | 109.38 | 3.58 | 3.61 | 4.18 |
  | YOLOv4-tiny | 7.31 | 0.64 | 0.64 | 0.66 |
  | BEiT-Base | 25.30 | 1.24 | 1.24 | 1.19 |
  | BERT | 10.00 | 0.97 | 0.97 | 0.99 |
  | SigLIP | 41.14 | 2.09 | 2.09 | 2.44 
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-11-03 10:29:32 +03:00
Savya Sanchi Sharma
4c20250a62 Merge pull request #29474 from SavyaSanchi-Sharma:extra_parsers
Remove Tensorflow Importer - #29474

[PR OpenCV Extra](https://github.com/opencv/opencv_extra/pull/1390)
[PR OpenCV Contrib](https://github.com/opencv/opencv_contrib/pull/4171)
Removed tensorflow importers

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-10-22 11:06:53 +03:00
Abhishek Gola
25beda402a Merge pull request #30157 from abhishek-gola:core-reduce-nd
core: reduceND, used by the DNN Reduce2 layer - #30157

Split out of #30082, which now carries only the resize half.

Adds `cv::reduceND`, which reduces an n-dimensional array over a set of axes with any of the `ReduceTypes`, extended with `REDUCE_PROD`, `REDUCE_L1`, `REDUCE_L2`, `REDUCE_LOG_SUM` and `REDUCE_LOG_SUM_EXP`. `cv::reduce` accepts n-dimensional input through it.

`sum`, `mean` and `minMaxIdx` now share one parallel driver (`stat_parallel.hpp`), so they parallelize over large arrays the same way `reduceND` does.

The new graph engine's `Reduce2` layer calls `cv::reduceND` instead of its own kernels, so `reduce2_kernels.simd.hpp` and the hand-written reduction loops in `reduce2_layer.cpp` go away.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-10-12 12:08:20 +03:00
Arghya Bose
8556b14874 dnn: fix the step bookkeeping of the reduce inner projection
Reducing over more than two axes walks the reduced sub-block with an odometer
over all of them but the innermost one. When an index carried into the next
axis, the step of that axis was assigned instead of being accumulated, so the
steps of every axis above the one that wrapped were lost and the corresponding
elements were read again and again:

    axes [0, 2, 3] on a 3x3x3x3 input, shape (3,3,3,3), steps (27, 9, 3, 1)
      projected steps: 0 3 6 27 30 33 27 30 33     <- should end in 54 57 60

Only the first output slice happened to be right, because the carry that
introduced the wrong step is the one that moves to the second one.

The same accounting exists for the unprojected axes just below, which
subtracts the step of the wrapped axis and adds the step of the axis it carries
into; the projected loop now does the same. This fixes test_mvn_expanded and
test_mvn_expanded_ver18 (a MeanVarianceNormalization decomposed into
primitives, denied as "Wrong answer" until now) plus any model that reduces
over three or more axes, since the inner projection also feeds the unprojected
axis steps. The OpenVINO backend still answers test_mvn_expanded_ver18
incorrectly, so it is filtered in
test_onnx_conformance_layer_filter__openvino.inl.hpp.

Found by comparing against onnxruntime: 9 of 200 random (rank, shape, axes set,
keepdims, op) combinations were wrong before and none are now; the two
conformance cases above and a new layer test fail without this change.
2026-10-05 21:28:46 +05:30
Abhishek Gola
0a8603b3ac Merge pull request #28889 from abhishek-gola:simd_kernel_speedup
SIMD Kernel speedup for DNN layers #28889

This PR add following speedups for Grounding Dino tiny model.

For Device:  Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
| Model | `ENGINE_NEW (Before)` | `ENGINE_NEW (After)` | `ENGINE_ORT` |
| :--- | :--- | :--- | :--- | 
| **Grounding Dino Tiny** | 3130 ms | 1872.06 ms| 1800.18 ms|

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-05-21 15:53:15 +03:00
Abhishek Gola
31cab2a055 Merge pull request #28574 from abhishek-gola:reduce_layer_empty_set
* added empty set support

* accept unnamed dynamic dims

* Fix for LSTM test failure

* removed converToND

* openvino failing test fix

* ARM CI issue fix

* ARM issue fix
2026-03-01 15:19:44 +03:00
Abhishek Gola
54b743dca8 Merge pull request #28575 from abhishek-gola:shape_inference_bug_fix
[BUG FIX] Incorrect shape assert bug fix in 5.x #28575

Closes: https://github.com/opencv/opencv/issues/28563

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-02-26 09:16:44 +03:00
Abhishek Gola
93385c6cdf Merge pull request #27816 from abhishek-gola:reduce_layer
Extended Reduce layer support in new DNN engine #27816

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-10-10 10:11:19 +03:00