Commit Graph

12 Commits

Author SHA1 Message Date
Abhishek Gola
2ec6a6bb65 Merge pull request #28588 from abhishek-gola:ORT_GPU_wrapper
Added OnnxRuntime GPU wrapper #28588

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-04-06 15:40:58 +03:00
Prasad Ayush Kumar
47ac80995f Merge pull request #28749 from Prasadayus:NMS-empty-detection-fix
Nms empty detection fix #28749

Requires opencv_extra: https://github.com/opencv/opencv_extra/pull/1330

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-04-02 18:37:23 +03:00
Abhishek Gola
e59506bbf5 Merge pull request #28595 from abhishek-gola:qdq_support
* added qdq fusion

* Added VNNI optimizations
2026-03-31 15:16:09 +03:00
Abhishek Gola
40ce5b4132 Merge pull request #28637 from abhishek-gola:old_dnn_tickets_cleanup
Added Output Tensor Names support in new DNN engine #28637

closes: https://github.com/opencv/opencv/issues/26201

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-03-26 15:07:56 +03:00
Vadim Pisarevsky
1b483ffea6 Merge pull request #28585 from vpisarev:dnn_block_layout_v5
Block layout-based convolution in DNN #28585

merge together with https://github.com/opencv/opencv_extra/pull/1321

Some core parts of the new engine in DNN module have been revised substantially:

1. all tests seem to pass, except for `Test_Graph_Simplifier.ResizeSubgraph`, which has been disabled because it does not take the newly added `TransformLayoutLayer` into account. The test should be reworked perhaps.
1. convolution and related operations (maxpool/avgpool) now use so-called block layout (`DATA_LAYOUT_BLOCK`), where `NxCxHxW` tensors are represented  as `NxC1xHxWxC0`, where `C1=(C + C0-1)/C0` and `C0` is a power-of-two (usually 4, 8, 16 or 32).
1. graph is now pre-processed and `TransformLayoutLayer` is inserted to convert data from NCHW or NHWC layout to the block layout or vice versa. The transformations are done in a lazy way only when they are really needed. For example, in the whole Resnet only 2 transformations are performed.
1. transformer-based models and other models that do not use convolutions will run as usual, without going to block layout.
1. there is yet another graph preprocessing stage added that embeds constant weights/scale and bias into convolution and batch norm layers.
1. 'batchnorm', 'activation' and 'adding a residual' are now fused with convolution, just like in the old engine. That brings some noticeable acceleration.
1. optimized convolution kernels have been added.
     * depthwise convolution, as well as maxpool and avgpool support C0=4, 8, 16 etc. _as long as_  C0 is divisible by the number of fp32 lanes in a SIMD register of the target platform (e.g. on ARM with NEON there must be `C0 % 4 == 0`, on x64 with AVX2 `C0 % 8 == 0`).
     * non-depthwise convolution only supports C0=8 for now. C0=8 seems to be a sweetspot for ARM with NEON, x64 with AVX2 or RISC-V with RVV (with 128- or 256-bit registers). For some platforms with dedicated matrix accelerators C0=16 or even C0=32 might be more efficient, but we could add the respective kernels later.
     * only fp32 kernels have been added. fp16/bf16 kernels might be added a little later.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
- [x] Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-03-13 17:09:27 +03:00
Abhishek Gola
95c66292b5 Merge pull request #28444 from abhishek-gola:added_ORT_wrapper
Added ONNX Runtime as an optional wrapper #28444

This PR adds ONNXRuntime (ORT) as an _optional_ wrapper, which can be enabled by adding **WITH_ONNXRUNTIME** flag in CMake command.

Using ORT wrapper the inference time for _resnet50.onnx model_ has come to _**~7ms**_ from _**~14ms**_.
Also, we are able to run models like `ssd_mobilenet_v1.onnx`.
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-02-27 15:47:12 +03:00
nklskyoy
b258926e54 Merge pull request #28058 from nklskyoy:onnx-importer2-dispatch-map
Onnx importer2 dispatch map #28058

I have noticed that PR[28032]( https://github.com/opencv/opencv/pull/28032) was incomplete - it fixed only the new ONNX importer. Also, building the dispatch map based on the parsed opset version hypotetically can cause trouble if the graph simplifier inserts a node with a different opset version which is not included in the dispatch map. So I removed this parameter in `buildDispatchMap_COM_MICROSOFT` and `buildDispatchMap_ONNX_AI` for now. It was marked as `CV_UNUSED` anyway. 

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2025-11-26 13:56:02 +03:00
Abhishek Gola
93385c6cdf Merge pull request #27816 from abhishek-gola:reduce_layer
Extended Reduce layer support in new DNN engine #27816

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-10-10 10:11:19 +03:00
Abhishek Gola
2470c07f1b Merge pull request #27698 from abhishek-gola:add_cast_layer
Added cast and castlike layers support in new DNN engine #27698

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-10-04 13:22:30 +03:00
Abhishek Gola
709eabda16 Merge pull request #27508 from abhishek-gola:if_layer_add
IfLayer add to new DNN engine #27508

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-07-16 09:01:54 +03:00
alexlyulkov
3672a14b42 Merge pull request #26394 from alexlyulkov:al/new-engine-tf-parser
Modified tensorflow parser for the new dnn engine #26394

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-11-27 09:15:20 +03:00
Vadim Pisarevsky
3cd57ea09e Merge pull request #26056 from vpisarev:new_dnn_engine
New dnn engine #26056

This is the 1st PR with the new engine; CI is green and PR is ready to be merged, I think.
Merge together with https://github.com/opencv/opencv_contrib/pull/3794

---

**Known limitations:**
* [solved] OpenVINO is temporarily disabled, but is probably easy to restore (it's not a deal breaker to merge this PR, I guess)
* The new engine does not support any backends nor any targets except for the default CPU implementation. But it's possible to choose the old engine when loading a model, then all the functionality is available.
* [Caffe patch is here: #26208] The new engine only supports ONNX. When a model is constructed manually or is loaded from a file of different format (.tf, .tflite, .caffe, .darknet), the old engine is used.
* Even in the case of ONNX some layers are not supported by the new engine, such as all quantized layers (including DequantizeLinear, QuantizeLinear, QLinearConv etc.), LSTM, GRU, .... It's planned, of course, to have full support for ONNX by OpenCV 5.0 gold release. When a loaded model contains unsupported layers, we switch to the old engine automatically  (at ONNX parsing time, not at `forward()` time).
* Some layers , e.g. Expat, are only partially supported by the new engine. In the case of unsupported flavours it switches to the old engine automatically (at ONNX parsing time, not at `forward()` time).
* 'Concat' graph optimization is disabled. The optimization eliminates Concat layer and instead makes the layers that generate tensors to be concatenated to write the outputs to the final destination. Of course, it's only possible when `axis=0` or `axis=N=1`. The optimization is not compatible with dynamic shapes since we need to know in advance where to store the tensors. Because some of the layer implementations have been modified to become more compatible with the new engine, the feature appears to be broken even when the old engine is used.
* Some `dnn::Net` API is not available with the new engine. Also, shape inference may return false if some of the output or intermediate tensors' shapes cannot be inferred without running the model. Probably this can be fixed by a dummy run of the model with zero inputs.
* Some overloads of `dnn::Net::getFLOPs()` and `dnn::Net::getMemoryConsumption()` are not exposed any longer in wrapper generators; but the most useful overloads are exposed (and checked by Java tests).
* [in progress] A few Einsum tests related to empty shapes have been disabled due to crashes in the tests and in Einsum implementations. The code and the tests need to be repaired.
* OpenCL implementation of Deconvolution is disabled. It's very bad and very slow anyway; need to be completely revised.
* Deconvolution3D test is now skipped, because it was only supported by CUDA and OpenVINO backends, both of which are not supported by the new engine.
* Some tests, such as FastNeuralStyle, checked that the in the case of CUDA backend there is no fallback to CPU. Currently all layers in the new engine are processed on CPU, so there are many fallbacks. The checks, therefore, have been temporarily disabled.

---

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2024-10-16 15:28:19 +03:00