Commit Graph

62 Commits

Author SHA1 Message Date
Abhishek Gola
2470c07f1b Merge pull request #27698 from abhishek-gola:add_cast_layer
Added cast and castlike layers support in new DNN engine #27698

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-10-04 13:22:30 +03:00
Alexander Smorkalov
350b211b57 Merge branch 4.x 2025-06-10 10:16:50 +03:00
Myron Rodrigues
344f8c6400 Merge pull request #27363 from MRo47:openvino-npu-support
Feature: Add OpenVINO NPU support #27363

## Why
- OpenVINO now supports inference on integrated NPU devices in intel's Core Ultra series processors.
- Sometimes as fast as GPU, but should use considerably less power.

## How
- The NPU plugin is now available as "NPU" in openvino `ov::Core::get_available_devices()`.
- Removed the guards and checks for NPU in available targets for Inference Engine backend.

## Test example

### Pre-requisites
- Intel [Core Ultra series processor](https://www.intel.com/content/www/us/en/products/details/processors/core-ultra/edge.html#tab-blade-1-0)
- [Intel NPU driver](https://github.com/intel/linux-npu-driver/releases)
- OpenVINO 2023.3.0+ (Tested on 2025.1.0)

### Example
```cpp
#include <opencv2/dnn.hpp>
#include <iostream>

int main(){
    cv::dnn::Net net = cv::dnn::readNet("../yolov8s-openvino/yolov8s.xml", "../yolov8s-openvino/yolov8s.bin");
    cv::Size net_input_shape = cv::Size(640, 480);
    std::cout << "Setting backend to DNN_BACKEND_INFERENCE_ENGINE and target to DNN_TARGET_NPU" << std::endl;
    net.setPreferableBackend(cv::dnn::DNN_BACKEND_INFERENCE_ENGINE);
    net.setPreferableTarget(cv::dnn::DNN_TARGET_NPU);

    cv::Mat image(net_input_shape, CV_8UC3);
    cv::randu(image, cv::Scalar(0, 0, 0), cv::Scalar(255, 255, 255));
    cv::Mat blob = cv::dnn::blobFromImage(
        image, 1, net_input_shape, cv::Scalar(0, 0, 0), true, false, CV_32F);
    net.setInput(blob);
    std::cout << "Running forward" << std::endl;
    cv::Mat result = net.forward();
    std::cout << "Output shape: " << result.size << std::endl; // Output shape: 1 x 84 x 6300
}
```

model files [here](https://limewire.com/d/bPgiA#BhUeSTBnMc)

docker image used to build opencv: [ghcr.io/mro47/opencv-builder](https://github.com/MRo47/opencv-builder/blob/main/Dockerfile)

Closes #26240

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2025-05-27 14:13:49 +03:00
Alexander Smorkalov
03983549fc Merge branch 4.x 2024-11-06 08:20:12 +03:00
Dmitry Kurtaev
0e80a97f87 Hotfix ie_ngraph.cpp in Debug 2024-10-29 10:20:51 +03:00
Dmitry Kurtaev
d193554a5f OpenVINO friendly output names from non-compiled Model 2024-10-23 09:29:05 +03:00
alexlyulkov
70df023317 Merge pull request #25605 from alexlyulkov:al/bool-dnn
Added bool support to dnn #25605

Added bool support to dnn pipeline (CPU, OpenVINO and CUDA pipelines).

Added bool support to these layers(CPU and OpenVINO):
- Equal, Greater, GreaterOrEqual, Less, LessOrEqual
- Not
- And, Or, Xor
- Where

Enabled all the conformance tests for these layers.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-06-06 12:52:06 +03:00
alexlyulkov
6af0394cd2 Merge pull request #25458 from alexlyulkov:al/dnn-openvino-int-support
Added int support for OpenVINO dnn backend #25458

Modified dnn OpenVINO integration to support type inference and int operations.

Added OpenVINO support to Cast, CumSum, Expand, Gather, GatherElements, Scatter, ScatterND, Tile layers.
I tried to add Reduce layer, but looks like OpenVINO uses float values inside Reduce operation so it can't pass our int tests.

OpenVINO uses int32 precision for int64 operations, so I've modified input values for int64 tests when backend is OpenVINO.

OpenVINO has a strange behavior with custom layers and int64 values. After model compilation OpenVINO may change types, so the model can have different output type. That's why these tests were disabled:
- Test_ArgMax_Int.random/0, where GetParam() = (4, NGRAPH/CPU)
- Test_ArgMax_Int.random/6, where GetParam() = (11, NGRAPH/CPU)
- Test_Reduce_Int.random/6, where GetParam() = (11, NGRAPH/CPU)
- Test_Reduce_Int.two_axes/6, where GetParam() = (11, NGRAPH/CPU)

Also these tests were temporary disabled, they didn't work on both 4.x and 5.x branches:
- Test_Caffe_layers.layer_prelu_fc/0, where GetParam() = NGRAPH/CPU
- Test_ONNX_layers.LSTM_Activations/0, where GetParam() = NGRAPH/CPU
- Test_ONNX_layers.Quantized_Convolution/0, where GetParam() = NGRAPH/CPU
- Test_ONNX_layers.Quantized_Eltwise_Scalar/0, where GetParam() = NGRAPH/CPU
- Test_TFLite.EfficientDet_int8/0, where GetParam() = NGRAPH/CPU


### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-05-15 11:51:59 +03:00
Oleg Pipikin
6da2ddcf0e Fix for OpenVINO 2024.0
Remove support OpenVINO lower than 2022.1 release
Remove legacy InferenceEngine wrappers
2024-03-18 15:05:50 +04:00
Alexander Alekhin
efc9837df1 Merge pull request #24892 from opencv-pushbot:gitee/alalek/dnn_avoid_16s_usage
DNN: avoid CV_16S usage for FP16 #24892

**Merge after**: #24918

TODO:
- [x] measure performance changes
- [x] optimize convertTo for OpenCL: #24918

12700K iGPU:

|Name of Test|0|1|1 vs 0 (x-factor)|
|---|:-:|:-:|:-:|
|AlexNet::DNNTestNetwork::OCV/OCL_FP16|7.441|7.480|0.99|
|CRNN::DNNTestNetwork::OCV/OCL_FP16|10.776|10.736|1.00|
|DenseNet_121::DNNTestNetwork::OCV/OCL_FP16|52.762|52.833|1.00|
|EAST_text_detection::DNNTestNetwork::OCV/OCL_FP16|60.694|60.721|1.00|
|EfficientNet::DNNTestNetwork::OCV/OCL_FP16|33.373|33.173|1.01|
|FastNeuralStyle_eccv16::DNNTestNetwork::OCV/OCL_FP16|81.840|81.724|1.00|
|GoogLeNet::DNNTestNetwork::OCV/OCL_FP16|20.965|20.927|1.00|
|Inception_5h::DNNTestNetwork::OCV/OCL_FP16|22.204|22.173|1.00|
|Inception_v2_SSD_TensorFlow::DNNTestNetwork::OCV/OCL_FP16|47.115|47.460|0.99|
|MPHand::DNNTestNetwork::OCV/OCL_FP16|6.760|6.670|1.01|
|MPPalm::DNNTestNetwork::OCV/OCL_FP16|10.188|10.171|1.00|
|MPPose::DNNTestNetwork::OCV/OCL_FP16|12.510|12.561|1.00|
|MobileNet_SSD_Caffe::DNNTestNetwork::OCV/OCL_FP16|17.290|17.072|1.01|
|MobileNet_SSD_v1_TensorFlow::DNNTestNetwork::OCV/OCL_FP16|19.473|19.306|1.01|
|MobileNet_SSD_v2_TensorFlow::DNNTestNetwork::OCV/OCL_FP16|22.874|23.404|0.98|
|OpenFace::DNNTestNetwork::OCV/OCL_FP16|9.568|9.517|1.01|
|OpenPose_pose_mpi_faster_4_stages::DNNTestNetwork::OCV/OCL_FP16|539.899|539.845|1.00|
|PPHumanSeg::DNNTestNetwork::OCV/OCL_FP16|18.015|18.769|0.96|
|PPOCRv3::DNNTestNetwork::OCV/OCL_FP16|63.122|63.540|0.99|
|ResNet_50::DNNTestNetwork::OCV/OCL_FP16|34.947|34.925|1.00|
|SFace::DNNTestNetwork::OCV/OCL_FP16|10.249|10.206|1.00|
|SSD::DNNTestNetwork::OCV/OCL_FP16|213.068|213.108|1.00|
|SqueezeNet_v1_1::DNNTestNetwork::OCV/OCL_FP16|4.867|4.878|1.00|
|VIT_B_32::DNNTestNetwork::OCV/OCL_FP16|200.563|190.788|1.05|
|VitTrack::DNNTestNetwork::OCV/OCL_FP16|7.528|7.173|1.05|
|YOLOX::DNNTestNetwork::OCV/OCL_FP16|132.858|132.701|1.00|
|YOLOv3::DNNTestNetwork::OCV/OCL_FP16|209.559|208.809|1.00|
|YOLOv4::DNNTestNetwork::OCV/OCL_FP16|221.357|220.924|1.00|
|YOLOv4_tiny::DNNTestNetwork::OCV/OCL_FP16|24.446|24.382|1.00|
|YOLOv5::DNNTestNetwork::OCV/OCL_FP16|43.922|44.080|1.00|
|YOLOv8::DNNTestNetwork::OCV/OCL_FP16|64.159|63.842|1.00|
|YuNet::DNNTestNetwork::OCV/OCL_FP16|10.177|10.231|0.99|
|opencv_face_detector::DNNTestNetwork::OCV/OCL_FP16|15.121|15.445|0.98|

Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2024-01-26 16:34:17 +03:00
Dmitry Kurtaev
c7ec0d599a Merge pull request #23987 from dkurt:openvino_int8_backend
OpenVINO backend for INT8 models #23987

### Pull Request Readiness Checklist

TODO:
- [x] DetectionOutput layer (https://github.com/opencv/opencv/pull/24069)
- [x] Less FP32 fallbacks (i.e. Sigmoid, eltwise sum)
- [x] Accuracy, performance tests (https://github.com/opencv/opencv/pull/24039)
- [x] Single layer tests (convolution)
- [x] ~~Fixes for OpenVINO 2022.1 (https://pullrequest.opencv.org/buildbot/builders/precommit_custom_linux/builds/100334)~~


Performace results for object detection model `coco_efficientdet_lite0_v1_1.0_quant_2021_09_06.tflite`:
| backend | performance (median time) |
|---|---|
| OpenCV | 77.42ms |
| OpenVINO 2023.0 | 10.90ms |

CPU: `11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz`

Serialized model per-layer stats (note that Convolution should use `*_I8` primitives if they are quantized correctly): https://gist.github.com/dkurt/7772bbf1907035441bb5454f19f0feef

---

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-09-28 16:24:43 +03:00
Dmitry Kurtaev
178fdbbda8 Merge pull request #24196 from dkurt:ov_backend_cleanups
Use ngraph::Output in OpenVINO backend wrapper #24196

### Pull Request Readiness Checklist

resolves https://github.com/opencv/opencv/issues/24102

* Use `ngraph::Output<ngraph::Node>>` insead of `std::shared_ptr<ngraph::Node>` as a backend wrapper. It lets access to multi-output nodes: 588ddf1b18/modules/dnn/src/net_openvino.cpp (L501-L504)
* All layers can be customizable with OpenVINO >= 2022.1. nGraph reference code used for default layer implementation does not required CPU plugin also (might be tested by commenting CPU plugin at `/opt/intel/openvino/runtime/lib/intel64/plugins.xml`).
* Correct inference if only intermediate blobs requested.


See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-09-05 18:08:28 +03:00
Dmitry Kurtaev
0245c0cd10 Merge pull request #24072 from dkurt:openvino_cpu_tests
Remove legacy nGraph logic #24072

### Pull Request Readiness Checklist

TODO:
- [x] Test with OpenVINO 2021.4 (tested locally)

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-08-02 14:39:11 +03:00
Alexander Alekhin
9a2a34f94e dnn(openvino): remove undefined status 2022-12-28 06:55:00 +00:00
Dmitry Kurtaev
8681686d8f Merge pull request #22957 from dkurt:new_openvino_api
Switch to new OpenVINO API after 2022.1 release

* Pass Layer_Test_Convolution_DLDT.Accuracy/0 test

* Pass test Test_Caffe_layers.Softmax

* Failed 136 tests

* Fix Concat. Failed 120 tests

* Custom nGraph ops. 19 failed tests

* Set and get properties from Core

* Read model from buffer

* Change MaxPooling layer output names. Restore reshape

* Cosmetic changes

* Cosmetic changes

* Override getOutputsInfo

* Fixes for OpenVINO < 2022.1

* Async inference for 2021.4 and less

* Compile model with config

* Fix serialize for 2022.1

* Asynchronous inference with 2022.1

* Handle 1d outputs

* Work with model with dynamic output shape

* Fixes with 1d output for old API

* Control outputs by nGraph function for all OpenVINO versions

* Refer inputs in PrePostProcessor by indices

* Fix cycled dependency between InfEngineNgraphNode and InfEngineNgraphNet.
Add InferRequest callback only for async inference. Do not capture InferRequest object.

* Fix tests thresholds

* Fix HETERO:GPU,CPU plugin issues with unsupported layer
2022-12-23 16:58:41 +00:00
Alexander Alekhin
b16f76eede Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2022-12-03 12:39:41 +00:00
Alexander Alekhin
74d0b4cc78 dnn(openvino): fix custom layers BlockingDesc 2022-12-03 01:34:10 +00:00
Alexander Alekhin
ca7f964104 dnn: use inheritance for OpenVINO net impl 2022-09-06 18:05:00 +00:00
Alexander Alekhin
5bf3c1df24 Merge pull request #21715 from ilyachur:change_type_info_creation 2022-03-14 09:18:58 +00:00
Ilya Churaev
419918076e Changed call of NodeTypeInfo constructor 2022-03-14 10:55:33 +03:00
Alexander Alekhin
57d3002ee1 Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2022-02-06 16:10:43 +00:00
Alexander Alekhin
b57ff73086 dnn(ngraph): fix outputs handling, drop 'unconnected' logic 2022-02-06 13:08:53 +00:00
Alexander Alekhin
67978b5746 dnn(ngraph): add debuging messages 2022-02-06 13:08:53 +00:00
Alexander Alekhin
aa5bc20c83 dnn(ngraph): fixup get_output_as_single_output_node() replacement patch 2022-02-06 10:35:59 +00:00
Alexander Alekhin
39b91c97f0 Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2021-07-16 10:35:42 +00:00
Alexander Alekhin
fbde0c6c96 dnn(ie): fix handling of 1D and non-32F outputs of InferenceEngine 2021-07-15 21:47:05 +00:00
Alexander Alekhin
8fad85edda Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2021-07-01 10:52:31 +00:00
Alexander Alekhin
f2057ce1ab dnn(ie): replace deprecated calls 2021-06-30 22:30:15 +00:00
Alexander Alekhin
170bf6d7af Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2021-05-01 09:44:24 +00:00
Alexander Alekhin
71bae7c23f dnn(ie): implicit usage of IE::GPU OpenCL kernels cache 2021-04-29 12:43:22 +03:00
Alexander Alekhin
d643a90efa Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2021-03-06 17:31:08 +00:00
Liubov Batanina
94533e12eb Determine layout 2021-03-04 13:05:01 +03:00
Omar Alzaibaq
a316b11aaa Merge pull request #18220 from Omar-AE:hddl-supported
* added HDDL VPU support

* changed to return True in one line if any device connected

* dnn: use releaseHDDLPlugin()

* dnn(hddl): fix conditions
2020-11-17 19:47:24 +00:00
Maksim Doronin
36f61f3879 [IE][VPU]: Refactor vpu configs
backported commit: 7fe87d9a5b
2020-10-05 20:27:52 +00:00
Maksim Doronin
7fe87d9a5b [IE][VPU]: Refactor vpu configs 2020-08-13 15:25:01 +03:00
Ilya Churaev
246de2b7f5 Replaced copy_with_new_args to clone_with_new_inputs 2020-08-03 23:08:29 +03:00
Ilya Churaev
e8c26963e9 Fixed removing is_parameter, is_constant, is_output 2020-08-03 23:08:22 +03:00
Maksim Shabunin
b698d0a6ee Merge pull request #17894 from ilyachur:feature/ichuraev/remove_copy_with_new_args 2020-07-21 08:19:42 +00:00
Ilya Churaev
aaf65bb3af Fixed removing is_parameter, is_constant, is_output 2020-07-21 08:45:16 +03:00
Ilya Churaev
0291dd3ef8 Replaced copy_with_new_args to clone_with_new_inputs 2020-07-20 16:05:01 +03:00
Ilya Churaev
a46ca98c57 Revert "Fixed removing is_parameter, is_constant, is_output" 2020-07-20 07:00:09 +03:00
Ilya Churaev
33db2ac928 Fixed removing is_parameter, is_constant, is_output 2020-07-17 14:45:11 +03:00
Dmitry Kurtaev
3bdd7bfbea Remove deprecated Inference Engine CPU extensions 2020-06-23 14:33:22 +03:00
Dmitry Kurtaev
8e3f5fb209 Remove deprecated Inference Engine CPU extensions 2020-06-23 10:10:29 +03:00
Ilya Lavrenov
676b818d6a Removed plugin dispatcher
backport of commit 74113737f3
2020-06-15 18:03:14 +00:00
Ilya Lavrenov
74113737f3 Removed plugin dispatcher 2020-06-15 16:02:52 +03:00
Alexander Alekhin
1644af841b Merge pull request #17536 from ilya-lavrenov:remove-error-listener 2020-06-12 22:18:04 +00:00
Ilya Lavrenov
9697e3ac24 Removed error lisneter usage 2020-06-12 20:29:11 +00:00
Alexander Alekhin
d2a9efd039 dnn: use OpenVINO 2020.3 defines 2020-06-12 20:24:08 +00:00
Alexander Alekhin
f0bef94a03 dnn: update network dump code, include ngraph serialization 2020-05-26 12:49:22 +00:00