Commit Graph

145 Commits

Author SHA1 Message Date
Savya Sanchi Sharma
0f51622bac Merge pull request #30439 from SavyaSanchi-Sharma:layers
dnn(cuda): transformer layer coverage and CUDA Graph replay on the new engine - #30439

  Follow-up to #30415 (ResNet layer coverage and cuDNN JIT conv/pooling on the new engine). Adds CUDA support on the new DNN engine
  for the layers transformer models need, and brings the CUDA path to parity with ONNX Runtime.
**Companion PR** opencv/opencv_extra#1448

  **Changes**
  - CUDA execs for Attention, Cast, ConstantOfShape, Expand, Gather, Not, Reduce, Shape, Slice, Softmax, Tile, Transpose, Concat,
  Gelu, LayerNormalization, MatMul, Split, Squeeze, Unsqueeze and the element-wise activations.
  - A graph with static shapes is recorded as one CUDA Graph on the second `forward()` and replayed from then on.
  - The cuDNN JIT convolution reads NCHW buffers in place; NHWC staging is kept only for FP16 grouped convolutions.
  - TF32 for cuBLAS GEMMs (`OPENCV_DNN_CUDA_ALLOW_TF32`), FP32 accumulation for FP16 GEMMs
  (`OPENCV_DNN_CUDA_FP16_HALF_ACCUMULATE`), and fused MatMul/Gemm post-ops.
  - Batch-broadcast MatMul and Shape are safe to capture in a CUDA Graph; the unused pointer-array `cublas::gemmBatched` is
  removed.
  - Tests: `forward()` runs three times on batch-broadcast MatMul and Shape models; `MatMul_init_bcast` runs on CUDA again;
  `AttentionSharedShapeReshape` uses the FP16 default tolerance on CUDA FP16.
  - Perf: new CUDA-only `*_CUDA_pinned` tests pin the input shapes.

  **Performance** (ms, median of 50, FP32 with TF32, RTX PRO 6000 Blackwell, idle GPU)

| Model | CPU | CUDA (cuDNN JIT) | CUDA (full cuDNN) | ORT 1.31 CUDA EP |
  |---|---|---|---|---|
  | ResNet-50 | 8.02 | 0.80 | 0.80 | 0.88 |
  | ResNet-18 | 3.27 | 0.44 | 0.44 | 0.48 |
  | MobileNetV2 | 1.99 | 0.28 | 0.27 | 0.40 |
  | DenseNet-121 | 17.42 | 1.48 | 1.58 | 1.58 |
  | EfficientNet-Lite4 | 5.03 | 0.53 | 0.54 | 0.73 |
  | ViT-B/32 | 11.06 | 1.02 | 1.02 | 0.95 |
  | ViT-Base/16 | 24.38 | 1.16 | 1.17 | 1.13 |
  | DeiT-Tiny | 4.58 | 0.65 | 0.65 | 0.68 |
  | MobileViT-XS | 6.93 | 0.72 | 0.72 | 0.77 |
  | YOLOv3 | 47.21 | 2.60 | 2.59 | 2.72 |
  | YOLOv4 | 109.38 | 3.58 | 3.61 | 4.18 |
  | YOLOv4-tiny | 7.31 | 0.64 | 0.64 | 0.66 |
  | BEiT-Base | 25.30 | 1.24 | 1.24 | 1.19 |
  | BERT | 10.00 | 0.97 | 0.97 | 0.99 |
  | SigLIP | 41.14 | 2.09 | 2.09 | 2.44 
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-11-03 10:29:32 +03:00
Abhishek Gola
6dc5ff5c1f Merge pull request #30181 from abhishek-gola:onnx-conformance-fixes-5x
dnn: fix ONNX conformance failures in layers and the graph engine - #30181

Fixes wrong results found by running the ONNX backend-scoreboard conformance suite (onnx 1.23.2) against the new DNN engine. No new ops are added; these are all ops OpenCV already claims to support but computed incorrectly.

Updated ONNX coverage is: `[scoreboard] passed=1740  failed=21  skipped=272  total=2033  (85.6% passed)`

**Graph engine / importer**
- Reshape+Reshape fusion now only fires when the outer shape is known and has no `0` entries, since a `0` copies a dim from the inner Reshape's output.
- Scan slices rank-1 inputs to scalars, so scalar loop-carried state keeps rank 0.
- Rank-0 tensors read from ONNX keep rank 0 instead of becoming 1x1.
- When a model is refused, `setInput()` now reports the actual import failure (including unknown ops that throw early) instead of a misleading "missing blob" error.

**Attention**
- Softcap is applied before the bias/mask, as the spec does, so masked keys no longer leak probability.
- `qk_matmul_output_mode` 1 and 2 now return the right tensors (checked against onnxruntime; the old denylist entries blaming the reference data were wrong).
- Attention-25 `left_window_size` / `right_window_size` are supported, including together with `attn_mask`.
- A fully masked row now gives all zeros instead of a uniform distribution.

**Layers**
- GRU: `direction="reverse"` is honoured.
- LSTM reverse: `Y_h` / `Y_c` are taken from the correct time step.
- SpaceToDepth: opset-28 `mode` (CRD / DCR) is supported.
- MaxPool: `storage_order` is respected for the indices output.
- OneHot: out-of-range indices produce an all-`off_value` row instead of wrapping.
- Mod with `fmod=0`: no longer truncates through int (wrong for float, lossy for int64).
- ReduceMax / ReduceMin on an empty bool set return the correct identity.
- Swish: `alpha` is respected (CPU and OpenCL).
- Quantize/DequantizeLinear: scale and zero_point may differ in shape when they're broadcast-compatible (e.g. rank-0 scale with `[1]` zero_point).
- BitShift: signed integer types are supported.

**Tests**
- 12 conformance tests removed from the denylists, a few OpenVINO filter entries added, and new 1D layer tests for the fixes.
- Full C++ conformance passes (5376/5376); `opencv_test_dnn` has no new failures compared with 5.x.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-10-14 12:55:00 +03:00
Abhishek Gola
65766b988a Merge pull request #30083 from abhishek-gola:dnn-eltwise-core-engine
Unify element-wise ops with the core engine - #30083

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-10-07 08:35:31 +03:00
Jaivardhan Bhola
193f8c4c5d Merge pull request #29618 from jaivardhan-bhola:vlm-ocr-support-engine-new
Added support for Granite Docling 258M and PaddleOCR-VL - #29618

Adds `com.microsoft::GroupQueryAttention` to the engine-new ONNX importer, required by GraniteDocling-258M's ,end-to-end samples for GraniteDocling-258M and PaddleOCR-VL-1.5. Verified on Linux x86_64 / GCC / CPU

**companion opencv_extra PR** [Vlm ocr support engine new opencv/opencv_extra#1412](https://github.com/opencv/opencv_extra/pull/1412) for test data


upstream/5.x already lowers `GroupQueryAttention` to `AttentionOnnxAi`, but that path rejects `do_rotary=1`:

```cpp
CV_CheckEQ(params.get<int>("do_rotary", 0), 0, "GroupQueryAttention: do_rotary=1 is not supported");
```

GraniteDocling-258M's export sets it. Confirmed two ways: building a Llama checkpoint from its published `text_config` (9 heads / 3 KV heads, `rope_theta=100000`) and running onnxruntime-genai's `model_builder`; and reading the community export at `onnx-community/granite-docling-258M-ONNX`, which carries 30 `GroupQueryAttention` nodes with `do_rotary=1`, `num_heads=9`, `kv_num_heads=3`, `head_dim=64`.


## Changes

**Importer** (`onnx_importer2.cpp`, +49)

Routing as above, plus the input validation the op needs: packed QKV rejected, past_key / past_value required as a pair, `do_rotary=1` requires cos/sin, unsupported trailing inputs rejected, and `seqlens_k` shorter than the buffer recognised as a shared-buffer export rather than silently mis-attended.

**Layer** (`group_query_attention_layer.cpp`, +380)

Attention is built on `fastGemmBatch` and `fused_softmax_softcap_mask`, the same kernels `attention_onnxai_layer.cpp` uses. Grouping falls out of the batched-GEMM offset table (query head → `h / groupSize`), so no broadcast copy is needed, and the second GEMM writes straight into the `[B, S, num_heads*D]` output. Supports rotary (full and partial), sliding window, softcap, growing and preallocated caches, and optional `present_*` outputs.

**Core** (`persistence_json.cpp`, +92)

`\uXXXX` escapes are UTF-16 code units, so anything above the BMP arrives as a surrogate pair. These were encoded as two separate 3-byte sequences CESU-8, which is not valid UTF-8 - for keys and values alike. Now recombined into one 4-byte sequence, with unpaired surrogates rejected.


**Tests / perf / samples**

`test_layer_group_query_attention.cpp` (+93), `test_tokenizer.cpp` (+28), `test_tokenizer.py` (+42), `perf_layer.cpp` (+125: `Layer_GroupQueryAttention` MHA_ShortCache and Grouped_WithCache, plus `Layer_Sign`), and two samples (`granite_docling_inference.py`, `paddleocr_vl_inference.py`).


### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-10-01 18:03:55 +03:00
Prasad Ayush Kumar
c093fe7d31 Merge pull request #30088 from Prasadayus:unify_graph_fusion
Unified DNN graph fusion behind fuseChains - #30088

Requires: https://github.com/opencv/opencv_extra/pull/1417

### What This PR Does

Follow-up to #29656, which added the backend-agnostic chain mechanism. That PR introduced it and made pointwise fusion its first consumer; this one moves the rest of the fusion onto it.

 - `Net::Impl::fuseBasic()` is gone. Its conv+BatchNorm, conv+activation and conv+residual branches now go through the chain mechanism.
  - `fuseBN` and the six other passes now run from inside `fuseChains()`, in their existing order.
  - `graph_fusion_basic.cpp` keeps `fuseBN()` and `fuseInstanceNormAffine()`.
  - New `FusionOps::foldInputScale` folds a scale sitting before a conv directly into its weights, across the dense, depthwise, grouped and MLAS-prepacked layouts.

  ### Results
  
 Measured on Intel Core i9-11900K
  | Model | 5.x | this PR | speedup |
  |-----------------------------|-----------|-----------|---------|
  | inception_v2 | 15.04 ms | 11.80 ms | 1.27 |
  | efficientnet-lite4 | 14.83 ms | 12.28 ms | 1.21 |
  | pose_estimation_mediapipe | 5.66 ms | 5.25 ms | 1.08 |
  | siglip_base_patch16_224 | 143.04 ms | 135.27 ms | 1.06 |
  | densenet121 | 25.99 ms | 24.76 ms | 1.05 |
  
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-09-30 09:04:57 +03:00
Savya Sanchi Sharma
61127d3812 Merge pull request #29656 from SavyaSanchi-Sharma:backendAgnostic
Added Backend Agnostic fusion in DNN - #29656

# What This Adds

A way for any layer, on any backend, to absorb trailing per-element math — with the pass knowing nothing about either layer and holding no list of op types.

- **Description**: `LayerMath` (one layer's math) → `AdjacencyGraph` (hash-consed chain)
- **Contract**: two `bool` virtuals on `Layer`
- **CPU implementations**: three, behind that contract

Pointwise fusion is the first consumer, not the point.

## The Problem

```cpp
ActivationLayer* activ = dynamic_cast<ActivationLayer*>(layer_ptr);
Conv2Layer* conv = getLayer<Conv2Layer>(newprog, conv_layer_idx);
if (conv) conv->fuseActivation(layer);
```

Guest type, host type, method name all hardcoded in the pass. `Clip` never fused (fails the cast), `Gemm` never fused (no virtual), no 2-op chain fused at all, and no backend could fuse anything without a CPU-specific method.


## Results

| Model                 | 5.x      | this PR  | speedup |
|-----------------------|----------|----------|---------|
| MPHand                | 2.34 ms  | 1.03 ms  | 2.27    |
| EfficientNet          | 9.82 ms  | 5.00 ms  | 1.96    |
| MPPose                | 5.14 ms  | 2.81 ms  | 1.83    |
| MobileNet_SSD_v1_ONNX | 12.58 ms | 8.38 ms  | 1.50    |
| BlazeFace             | 0.95 ms  | 0.77 ms  | 1.23    |
| DenseNet_121          | 20.56 ms | 19.08 ms | 1.08    |
| MobileNetv2_ONNX      | 2.07 ms  | 1.99 ms  | 1.04    |
| MobileViT_XS          | 6.33 ms  | 6.11 ms  | 1.04    |
| YuNet_320             | 1.33 ms  | 1.27 ms  | 1.04    |
| PPOCRv3               | 43.05 ms | 42.15 ms | 1.02    |
| BERT                  | 9.78 ms  | 9.68 ms  | 1.01    |
| BEiT_Base_Patch16_224 | 27.48 ms | 27.08 ms | 1.01    |
| DeiT_Tiny_Patch16_224 | 4.74 ms  | 4.72 ms  | 1.01    |
| MPPalm                | 1.88 ms  | 1.85 ms  | 1.01    |
| SSD                   | 50.19 ms | 49.82 ms | 1.01    |
| YOLOv4_tiny           | 7.10 ms  | 7.03 ms  | 1.01    |



### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-09-16 11:56:51 +03:00
velonica0
ba7e716840 Merge pull request #29836 from velonica0:dnn-blocked-pointwise-span
dnn: use the full vector width in blocked-layout pointwise kernels - #29836

Four block-layout kernels share one defect: they vectorize *across the channel block*, so they only use the vector unit while it happens to match `C0`. This PR fixes all four behind one shared helper, and along the way fixes an unrelated correctness bug found in the same function.

## 1. FP16 BatchNorm writes no output

`batch_norm2_layer.cpp`, the `CV_16F` arm of the accelerated block-layout path, wraps its whole body in a condition that cannot be true:

```cpp
} else if (type == CV_16F) {
    const hfloat* inptr = ...;
    if (type == CV_32F) {          // false by construction
```

There is no `else`, so every vector loop in that arm is dead and **nothing is written to the output**. A half-precision BatchNorm on a blocked tensor with `C0` equal to 1, 2 or 4 times the vector width returns whatever the destination buffer already held. The `CV_32F` and `CV_16BF` arms are correct; only the redundant wrapper is removed.

Confirmed with a sentinel-prefilled destination: before the fix the sentinel survives the call (`maxerr = 12346`) for all three `C0` values, after it the output is exact.

## 2. The kernels only use min(C0, VEC_SZ) lanes

`C0` is the block-layout channel block, fixed at 8 (`net_impl.hpp`, `DEFAULT_C0`), so these guards decide how much of the register gets used:

| kernel | guard | consequence with C0=8 |
|---|---|---|
| ChannelsPReLU | `C0 == VEC_SZ` | scalar at 4 lanes **and** at 16/32 |
| BatchNorm | `C0 == vlanes*{4,2,1}` | scalar at 16/32 |
| InstanceNorm | `c0 <= validC0 - VEC_SZ` | scalar at 16/32 |
| GroupNorm | `c0 <= c0_hi - VEC_SZ` | scalar at 16/32 |

PReLU is the worst case: an equality can only hold on an 8-lane build, so a default x86 SSE build (4 lanes) runs it scalar too. The other three use chunk loops that work at <= 8 lanes and fail above.

These files are not in the CPU-dispatch list, so `v_float32` is whatever `CPU_BASELINE` gives -- this is not RISC-V-specific.

### Approach

A block-layout plane is contiguous over `(H, W, C0)` and the coefficients repeat with period `C0`, so one register can span `vlanes/C0` pixels: replicate the coefficients across it and walk the plane flat.

All three spanning call sites now go through one helper, `cpu_kernels/blocked_pointwise.hpp::blockedSpanApply()`, parameterised by the per-element operation (`BlockedAffineOp`, `BlockedPReLUOp`). PReLU additionally gains a chunked path for `C0 > VEC_SZ`, which is what restores it on 4-lane targets. BatchNorm keeps its own unrolled loops and gains a whole-vector step plus a remainder loop. Pre-existing paths are untouched, so targets that already vectorized keep the same code.

**GroupNorm also gains a vector reduction.** Its mean/variance pass walked one channel at a time with a stride of `C0`, which no target vectorized at all, so that half speeds up everywhere rather than only on wide vectors. Both new GroupNorm paths are restricted to blocks owned entirely by one group; where a group boundary falls inside a block the old per-channel code still runs, because spanning would cross into channels another `parallel_for_` task is writing.

### Note for reviewers: the helper's `noinline` is load-bearing

`blockedSpanApply()` carries an explicit `noinline`. Inlined, GCC 15.2 on RISC-V speculates its stores into callers whose guard is false, which silently corrupted a neighbouring group's channels in `fastNormGroupBlockF32`. The symptom was exactly half the elements wrong at VLEN=1024 in the cases where a group splits a block; a runtime trace showed the guard evaluating false on every block, and inserting any call before the `if` made it disappear. Please do not remove the attribute.

## Benchmarks

SpacemiT K3, GCC 15.2, `CPU_BASELINE=RVV`, single thread, median of 11, pristine vs patched built back to back in one session. The board exposes two core types with different VLEN, so both columns are the same binary on the same machine.

Speedup, VLEN=256 / VLEN=1024:

| shape (NxC1xHxWxC0) | Ci | InstanceNorm | BatchNorm | GroupNorm |
|---|---|---|---|---|
| 1x32x56x56x8 | 256 | 1.01x / 14.38x | 0.91x / 11.62x | 1.85x / 14.56x |
| 1x16x28x28x8 | 128 | 1.01x / 16.27x | 0.98x / 3.95x | 1.91x / 16.01x |
| 1x8x112x112x8 | 64 | 1.02x / 12.76x | 1.03x / 1.37x | 1.98x / 12.86x |
| 1x16x56x56x4 | 64 | 3.01x / 12.47x | 4.49x / 6.18x | 3.01x / 11.78x |

ChannelsPReLU, measured separately the same way:

| shape | Ci | VLEN=256 | VLEN=1024 |
|---|---|---|---|
| 1x32x56x56x8 | 256 | 1.45x | 8.37x |
| 1x16x28x28x8 | 128 | 1.55x | 4.16x |
| 1x64x14x14x8 | 512 | 1.46x | 4.05x |
| 1x8x112x112x8 | 64 | 1.47x | 3.77x |
| 1x16x56x56x4 | 64 | 2.99x | 8.47x |
| 1x8x56x56x16 | 128 | 1.20x | 2.33x |

The VLEN=256 columns for InstanceNorm and BatchNorm are flat by construction -- `C0 == VEC_SZ` there, so those shapes already vectorized and the code is unchanged; the 0.91-1.03x spread is measurement noise. The rows that move at 256 are `C0=4` (block narrower than the vector), PReLU (broken at every width), and GroupNorm (reduction).

**Caveat on the BatchNorm numbers.** BatchNorm timings on this board are much less reproducible than the other three. The `1x8x112x112x8` case in particular measured anywhere from 1.4x to 6.9x across builds with byte-identical BatchNorm sources, and its pristine baseline moved by 26% between runs. The working set there is 3.06 MB, an exact multiple of 4096, and this hardware is sensitive to how source and destination alias in the cache; the numbers above are one back-to-back pair rather than a stable figure. InstanceNorm, GroupNorm and PReLU reproduced to within ~1% across every build.

## Testing

Verified against a scalar reference over 43 shape / `C0` / `Ci` / group combinations across the four layers, at VLEN 256 and 1024, at 1 and 8 threads, clean under `MALLOC_CHECK_=3`. Cases include partial trailing blocks, odd planes (7x7, 13x11) that exercise the remainder loops, `C0` from 2 to 64, and GroupNorm configurations where a group boundary falls inside a block -- those take the fallback and match bit-exactly, which is what confirms the ownership guard.

`opencv_test_dnn` was **not** run: the build used here is `BUILD_LIST=dnn`, which generates no dnn test target, and the board has no `opencv_extra` checkout. The blocked path's reachability was confirmed by inspection instead -- `ActivationLayer::getLayouts` passes the producer's layout through, and `useBlockLayout()` runs unconditionally in `finalizeGraph`, so these kernels are on the default path in real nets.

## Platform scope

Nothing here is behind a RISC-V `#ifdef`; this is universal-intrinsic code that compiles into every target.

| target (C0=8) | lanes | what changes |
|---|---|---|
| x86 SSE baseline (default) | 4 | PReLU newly vectorized; GroupNorm reduction newly vectorized |
| x86 AVX2 baseline | 8 | GroupNorm reduction; BatchNorm loop rewritten (same iterations) |
| x86 AVX-512 baseline | 16 | + all spanning paths go live |
| ARM64 NEON | 4 | as SSE baseline |
| ARMv7 NEON | 4 | `CV_SIMD_64F`=0, reduction path skipped |
| RVV 256 | 8 | PReLU, GroupNorm reduction |
| RVV >= 512 | 16/32 | everything |

Two changes reach a **default x86 build**: PReLU, which was scalar there because `C0 == VEC_SZ` cannot hold at 4 lanes, and the GroupNorm reduction, which was scalar everywhere. The latter changes GroupNorm's numerical output on those targets, since summation order differs -- measured at ~1e-7 relative here.

Measurements are RISC-V only; no x86 or ARM machine was available. The spanning branches were exercised on RVV at the same `vlanes/C0` ratios an AVX-512-baseline build would hit (2:1 and 4:1), but the x86-reachable changes above have had no x86 validation and are the part most worth checking in CI.
2026-09-06 11:02:41 +03:00
Prasad Ayush Kumar
30e2e0aaff Merge pull request #29786 from Prasadayus:layers_dtype_coverage_increase
Extend data type support in elementwise, gather and scatter layers - #29786

### PR Changes:

### Support added, per layer

  | Layer | Types added | Gate / kernel |
  |---|---|---|
  | Abs | 8U, 8S, 16U, 16S, 32U, 32S, 64U, 64S | Gate + new templated integer kernel |
  | Sign | 8U, 8S, 16U, 16S, 32U, 32S, 64U, 64S | Gate + same templated kernel |
  | Neg | 32S | Gate only. The kernel already had a `CV_32S` branch |
  | GatherElements | 64F, 16U, 16S, 32U, 64U | Gate + element-width dispatch |
  | Scatter | 64F, 16U, 16S, 32U, 64U | Gate + dispatch arms, all five reductions checked |
  | ScatterND | 64F, 16U, 16S, 32U, 64U | Gate + dispatch arms, same as Scatter |
  | GatherND | 64F, 16U, 16S, 32U, 64U | Gate + element-width dispatch |
  | Slice2 | (fix) | Kernel. Wrong-width copy, see below |


  `Abs`/`Sign` use one template over all widths with the signed/unsigned split resolved at compile time;
  unsigned `abs` short-circuits to `copyTo` and unsigned `sign` reduces to `x != 0`. `GatherElements` and
  `GatherND` only move elements, so their per-dtype arms collapsed to four widths. `GatherND` also unified a
  target-conditional gate that split `16F`/`32F` by target in a file with no OpenCL path.

  `Slice2` had two duplicated depth chains that both fell through to `run_parallel<float>`, a 4-byte copy, so
  `64F`/`64U` truncated and `16U`/`16S` read and wrote past the element. Reachable only when the innermost axis
  has `step != 1`, which is why the float32 slice tests passed. Now dispatches on `elemSize()`, matching
  `pad2_layer.cpp`.

  **Two further fixes:** signed-overflow UB in the int64 `Power`/`Neg` path (`sp[i] * scale` is undefined at
  `INT64_MIN`, now multiplied through `uint64_t`), and `CV_OCL_RUN` now skips integer depths, since the OCL
  activation kernels are float math and `CV_32S` would have gone through a 24-bit mantissa.

  **New accuracy tests in `test_int.cpp`**: `Test_Abs_Int`, `Test_Sign_Int`, `Test_Neg_Int`, `Test_Scatter_Int`,
  `Test_GatherND_Int`, with `Test_GatherElements_Int` and `Test_ScatterND_Int` widened to nine depths.

  Removed `test_slice_start_out_of_bounds` from the parser denylist.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-08-27 09:26:14 +03:00
Abhishek Gola
12fb9a6c24 using gpuMat instead of backend wrappers 2026-07-29 20:23:02 +05:30
Abhishek Gola
2c67582020 slope fix 2026-07-28 20:27:29 +05:30
Alexander Smorkalov
abb0115648 Merge branch 4.x 2026-07-09 12:17:24 +03:00
Tonu Samuel
4d839a16f0 dnn: SIMD for 13 transcendental activations in the 5.x engine
Port of #29377 to 5.x. Wires Log, Erf, Exp, Sin, Cos, Sinh, Cosh, Tan,
Softplus, BNLL, Asinh, Acosh, Atanh into the dispatched activation_kernels
registry, plus a Layer_Activation perf test. 1.4-12.3x across
M4/A76/Threadripper/Xeon, correct to <=5e-7 vs scalar.
2026-06-27 09:11:09 +03:00
Tõnu Samuel
b64a215e71 Merge pull request #29377 from tonuonu:perf-dnn-activation-simd
dnn: SIMD for transcendental activation layers (15 functors) 🧑‍💻🤖 #2937

# dnn: SIMD for transcendental activation layers (15 functors) 🧑‍💻🤖

### Summary

Many `BaseDefaultFunctor`-based activation functors fall back to the scalar `BaseDefaultFunctor::apply`
— one `libm` call per element. This PR adds vectorized `apply()` overrides (universal intrinsics,
matching the existing `MishFunctor`/`GeluFunctor`/`SigmoidFunctor` pattern, each with a scalar tail) to
the **15 transcendental activations** that have a clean SIMD path:

| functor | uses | | functor | uses |
|---|---|---|---|---|
| TanH | `v_exp` (1−2/(e²ˣ+1)) | | Asinh | `v_log`,`v_sqrt` (sign·log(\|x\|+√(x²+1))) |
| Log | `v_log` | | Acosh | `v_log`,`v_sqrt` |
| Erf | `v_erf` | | Atanh | `v_log` |
| Exp | `v_exp` | | Softplus | `v_exp`,`v_log` |
| Sin | `v_sin` | | BNLL | `v_exp`,`v_log` (max(x,0)+log1p(e^−\|x\|)) |
| Cos | `v_cos` | | GeluApproximation | `v_exp` (tanh) |
| Tan | `v_sin`/`v_cos` | | Sinh / Cosh | `v_exp` |

Functors compile at the baseline SIMD width (SSE on x86, NEON on ARM — same as the existing SIMD
functors). `Sqrt`/`Floor`/`Ceil`/`Round`/`Abs`/etc. are intentionally **not** included — those are
single cheap ops the compiler already auto-vectorizes (measured `Sqrt` at 0.95×). `Asin`/`Acos`/`Atan`
are omitted (no `v_atan`/`v_asin`/`v_acos` intrinsic).

### Performance

Real `opencv_perf_dnn` `Layer_Activation` (added here), 8×256×128×100 = 26.2 M-elem CV_32F blob, A/B
vs the scalar fallback, median speedup:

| functor | M4 clang/NEON | A76 gcc/NEON | Threadripper gcc | Xeon W-2235 gcc | functor | M4 | A76 | TR | Xeon |
|---|---|---|---|---|---|---|---|---|---|
| TanH | 9.1× | 2.3× | 7.3× | 7.3× | Sinh | 3.3× | 3.2× | 5.2× | 5.0× |
| Cos | 8.8× | 3.0× | 3.3× | 3.0× | Acosh | 3.2× | 2.9× | 2.7× | 2.6× |
| Log | 7.8× | 3.0× | 2.5× | 2.4× | Cosh | 2.8× | 3.2× | 2.6× | 2.5× |
| Sin | 7.8× | 3.0× | 3.4× | 2.9× | Exp | 2.4× | 2.5× | 1.4× | 1.5× |
| GeluApprox | 7.6× | 2.4× | 6.6× | 5.8× | BNLL | 4.2× | 2.0× | 1.6× | 1.4× |
| Atanh | 7.0× | 3.6× | 6.0× | 5.2× | Softplus | 2.0× | 2.0× | 3.3× | 2.7× |
| Tan | 6.8× | 3.6× | 6.6× | 6.0× | Erf | 4.2× | 2.4× | 3.9× | 3.5× |
| Asinh | 3.9× | 2.4× | 5.8× | 5.7× | | | | | |

Every functor is a win on every tested out-of-order core (worst case 1.4×). On the **in-order
Cortex-A55** (a55-tuned build) all 15 measure 0.985–0.99× — neutral, within noise: the SIMD
polynomials are dependency-chain-bound, which an in-order pipeline can neither accelerate nor (here)
slow down. So: meaningful wins on out-of-order ARM + x86, no regression on in-order ARM.

### Accuracy

Uses the `v_exp`/`v_log`/`v_erf`/`v_sin`/`v_cos` polynomials OpenCV already ships and relies on (Gelu
via `v_erf`, Mish/Sigmoid via `v_exp`). Verified vs scalar `libm` on 320 K random elements per functor
with domain-correct inputs (Acosh x≥1, Atanh |x|<1, …): **max abs error ≤ 3e-5** (most ≤ 2e-6), zero
elements exceeding rel>1e-3 & abs>1e-4. ONNX conformance and the layer accuracy tests exercise these ops.

### Pull Request Readiness Checklist

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch (4.x)
- [ ] There is a reference to the original bug report and related work — N/A (perf improvement)
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable — perf test added (`Layer_Activation`, `SANITY_CHECK_NOTHING`); accuracy covered by existing ONNX conformance/layer tests
- [ ] The feature is well documented and sample code can be built with the project CMake — N/A (internal optimization, no new API)
2026-06-26 11:06:53 +03:00
Abhishek Gola
ac70e5e8c9 block-layout ChannelsPReLU 2026-05-22 13:36:44 +05:30
Teddy-Yangjiale
4b1c861ab7 dnn: vectorize SigmoidFunctor using universal intrinsics 2026-05-12 02:22:19 +08:00
Abhishek Gola
3d77645a3a Merge pull request #28750 from abhishek-gola:activation_fusion
Extended fusion for activation functions in new DNN engine #28750

After this fusion, we see following improvements in YOLO models:

| Model | Before (`ENGINE_NEW`) | After (`ENGINE_NEW`) | `ENGINE_ORT` | % Improvement (Before v/s After) |
| :--- | :--- | :--- | :--- | :--- |
| **YOLOv8n** | 18.89  ms| 12.06 ms| 12.15 ms| 36.16% |
| **YOLOv5n** | 17.12  ms| 9.29 ms| 9.23 ms| 45.73% |
| **YOLOX-S** | 38.78  ms| 25.56 ms| 25.16 ms| 34.09% |

Device details: 
      - Model name: Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-04-14 09:08:14 +03:00
Vadim Pisarevsky
1b483ffea6 Merge pull request #28585 from vpisarev:dnn_block_layout_v5
Block layout-based convolution in DNN #28585

merge together with https://github.com/opencv/opencv_extra/pull/1321

Some core parts of the new engine in DNN module have been revised substantially:

1. all tests seem to pass, except for `Test_Graph_Simplifier.ResizeSubgraph`, which has been disabled because it does not take the newly added `TransformLayoutLayer` into account. The test should be reworked perhaps.
1. convolution and related operations (maxpool/avgpool) now use so-called block layout (`DATA_LAYOUT_BLOCK`), where `NxCxHxW` tensors are represented  as `NxC1xHxWxC0`, where `C1=(C + C0-1)/C0` and `C0` is a power-of-two (usually 4, 8, 16 or 32).
1. graph is now pre-processed and `TransformLayoutLayer` is inserted to convert data from NCHW or NHWC layout to the block layout or vice versa. The transformations are done in a lazy way only when they are really needed. For example, in the whole Resnet only 2 transformations are performed.
1. transformer-based models and other models that do not use convolutions will run as usual, without going to block layout.
1. there is yet another graph preprocessing stage added that embeds constant weights/scale and bias into convolution and batch norm layers.
1. 'batchnorm', 'activation' and 'adding a residual' are now fused with convolution, just like in the old engine. That brings some noticeable acceleration.
1. optimized convolution kernels have been added.
     * depthwise convolution, as well as maxpool and avgpool support C0=4, 8, 16 etc. _as long as_  C0 is divisible by the number of fp32 lanes in a SIMD register of the target platform (e.g. on ARM with NEON there must be `C0 % 4 == 0`, on x64 with AVX2 `C0 % 8 == 0`).
     * non-depthwise convolution only supports C0=8 for now. C0=8 seems to be a sweetspot for ARM with NEON, x64 with AVX2 or RISC-V with RVV (with 128- or 256-bit registers). For some platforms with dedicated matrix accelerators C0=16 or even C0=32 might be more efficient, but we could add the respective kernels later.
     * only fp32 kernels have been added. fp16/bf16 kernels might be added a little later.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
- [x] Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-03-13 17:09:27 +03:00
Abhishek Gola
31cab2a055 Merge pull request #28574 from abhishek-gola:reduce_layer_empty_set
* added empty set support

* accept unnamed dynamic dims

* Fix for LSTM test failure

* removed converToND

* openvino failing test fix

* ARM CI issue fix

* ARM issue fix
2026-03-01 15:19:44 +03:00
Alexander Smorkalov
6c995c768f Merge branch 4.x 2026-02-20 22:05:35 +03:00
Abhishek Gola
abc9100c9b added layer norm 2026-01-29 12:58:31 +05:30
Vincent Rabaud
5622958189 Replace pow with std::pow
The C pow casts to double while std::pow has overloads that can be
optimized by the compiler.
Also replace pow(*, 1./3) by cbrt.
2026-01-04 15:12:59 +01:00
Abhishek Gola
93385c6cdf Merge pull request #27816 from abhishek-gola:reduce_layer
Extended Reduce layer support in new DNN engine #27816

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-10-10 10:11:19 +03:00
Alexander Smorkalov
13a2919ec0 Fixed -Wunnecessary-virtual-specifier warning produced latest clang. 2025-08-19 12:28:15 +03:00
Alexander Smorkalov
bd67770dcb Merge branch 4.x 2025-07-22 09:47:19 +03:00
fengyuentau
77d2a5868a feat: support more cann operators 2025-06-25 04:57:01 +00:00
Alexander Smorkalov
350b211b57 Merge branch 4.x 2025-06-10 10:16:50 +03:00
Dmitry Kurtaev
1e3ab44cff Merge pull request #27307 from dkurt:tflite_face_blendshape_model
TFLite fixes for Face Blendshapes V2 #27307

### Pull Request Readiness Checklist

* Scalars support
* Better handling of 1D tensors
* New ops import: SUB, SQRT, DIV, NEG, SQUARED_DIFFERENCE, SUM
* Number of NHWC<->NCHW layouts compatibility improvements

resolves #27211

**Merge with extra**: https://github.com/opencv/opencv_extra/pull/1257

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-05-19 10:45:18 +03:00
Alexander Smorkalov
55a2ca58f0 Merge branch 4.x 2025-02-05 09:28:27 +03:00
Yuantao Feng
51ec7fedaf fix build 2024-12-17 10:17:15 +00:00
Alexander Smorkalov
100db1bc0b Merge branch 4.x 2024-08-28 15:06:19 +03:00
Alexander Smorkalov
67d0338c9c Merge pull request #26004 from ericmariasis:eric-mariasis-issue-26000
got rid of std prefix
2024-08-07 10:26:24 +03:00
ericmariasis
3f92884520 got rid of std prefix 2024-08-07 00:17:04 -04:00
Alexander Smorkalov
7e8f2a1bc4 Merge branch 4.x 2024-08-06 15:31:30 +03:00
Aven
796974cccc fix compilation errors caused by namespace
related: #25199
2024-08-04 05:04:03 +08:00
Alexander Smorkalov
672a662dff Merge branch 4.x 2024-07-26 09:10:36 +03:00
Yuantao Feng
23b244d3a3 Merge pull request #25881 from fengyuentau:dnn/cpu/optimize_activations_with_v_exp
dnn: optimize activations with v_exp #25881

Merge with https://github.com/opencv/opencv_extra/pull/1191.

This PR optimizes the following activations:

- [x] Swish
- [x] Mish
- [x] Elu
- [x] Celu
- [x] Selu
- [x] HardSwish

### Performance (Updated on 2024-07-18)

#### AmLogic A311D2 (ARM Cortex A73 + A53)

```
Geometric mean (ms)

            Name of Test              activations activations.patch activations.patch
                                                                              vs
                                                                         activations
                                                                          (x-factor)
Celu::Layer_Elementwise::OCV/CPU        115.859          27.930              4.15
Elu::Layer_Elementwise::OCV/CPU          27.846          27.003              1.03
Gelu::Layer_Elementwise::OCV/CPU         0.657           0.602               1.09
HardSwish::Layer_Elementwise::OCV/CPU    31.885          6.781               4.70
Mish::Layer_Elementwise::OCV/CPU         35.729          32.089              1.11
Selu::Layer_Elementwise::OCV/CPU         61.955          27.850              2.22
Swish::Layer_Elementwise::OCV/CPU        30.819          26.688              1.15
```

#### Apple M1

```
Geometric mean (ms)

               Name of Test                activations activations.patch activations.patch
                                                                                   vs
                                                                              activations
                                                                               (x-factor)
Celu::Layer_Elementwise::OCV/CPU              16.184          2.118               7.64
Celu::Layer_Elementwise::OCV/CPU_FP16         16.280          2.123               7.67
Elu::Layer_Elementwise::OCV/CPU               9.123           1.878               4.86
Elu::Layer_Elementwise::OCV/CPU_FP16          9.085           1.897               4.79
Gelu::Layer_Elementwise::OCV/CPU              0.089           0.081               1.11
Gelu::Layer_Elementwise::OCV/CPU_FP16         0.086           0.074               1.17
HardSwish::Layer_Elementwise::OCV/CPU         1.560           1.555               1.00
HardSwish::Layer_Elementwise::OCV/CPU_FP16    1.536           1.523               1.01
Mish::Layer_Elementwise::OCV/CPU              6.077           2.476               2.45
Mish::Layer_Elementwise::OCV/CPU_FP16         5.990           2.496               2.40
Selu::Layer_Elementwise::OCV/CPU              11.351          1.976               5.74
Selu::Layer_Elementwise::OCV/CPU_FP16         11.533          1.985               5.81
Swish::Layer_Elementwise::OCV/CPU             4.687           1.890               2.48
Swish::Layer_Elementwise::OCV/CPU_FP16        4.715           1.873               2.52
```

#### Intel i7-12700K

```
Geometric mean (ms)

            Name of Test              activations activations.patch activations.patch
                                                                    vs
                                                               activations
                                                                (x-factor)
Celu::Layer_Elementwise::OCV/CPU        17.106       3.560         4.81
Elu::Layer_Elementwise::OCV/CPU          5.064       3.478         1.46
Gelu::Layer_Elementwise::OCV/CPU         0.036       0.035         1.04
HardSwish::Layer_Elementwise::OCV/CPU    2.914       2.893         1.01
Mish::Layer_Elementwise::OCV/CPU         3.820       3.529         1.08
Selu::Layer_Elementwise::OCV/CPU        10.799       3.593         3.01
Swish::Layer_Elementwise::OCV/CPU        3.651       3.473         1.05
```

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-07-19 16:03:19 +03:00
Alexander Smorkalov
fc9208cff5 Merge branch 4.x 2024-07-17 10:08:16 +03:00
Yuantao Feng
e3858cc5a3 Merge pull request #25147 from fengyuentau:dnn/elementwise_layers/speedup
* added v_erf and implemented gelu acceleration via vectorization

* remove anonymous v_erf and use v_erf from intrin_math

* enable perf for ov and cuda backend
2024-07-08 14:24:36 +03:00
alexlyulkov
70df023317 Merge pull request #25605 from alexlyulkov:al/bool-dnn
Added bool support to dnn #25605

Added bool support to dnn pipeline (CPU, OpenVINO and CUDA pipelines).

Added bool support to these layers(CPU and OpenVINO):
- Equal, Greater, GreaterOrEqual, Less, LessOrEqual
- Not
- And, Or, Xor
- Where

Enabled all the conformance tests for these layers.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-06-06 12:52:06 +03:00
Alexander Smorkalov
cb6d295f15 Merge branch 4.x 2024-04-02 16:39:54 +03:00
Oleg Pipikin
6da2ddcf0e Fix for OpenVINO 2024.0
Remove support OpenVINO lower than 2022.1 release
Remove legacy InferenceEngine wrappers
2024-03-18 15:05:50 +04:00
Yuantao Feng
d4fd5157fa Merge pull request #24980 from fengyuentau:on-fly-quantization-removal
dnn cleanup: On-fly-quantization removal #2498

On-fly-quantization is first introduced via https://github.com/opencv/opencv/pull/20228.
We decided to remove it but keep int8 layers implementation because on-fly-quantization
is less practical given the fact that there has been so many dedicated tools for model
quantization.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-02-16 18:21:45 +03:00
Alexander Smorkalov
3a55f50133 Merge branch 4.x 2024-02-12 14:20:35 +03:00
Alexander Alekhin
efc9837df1 Merge pull request #24892 from opencv-pushbot:gitee/alalek/dnn_avoid_16s_usage
DNN: avoid CV_16S usage for FP16 #24892

**Merge after**: #24918

TODO:
- [x] measure performance changes
- [x] optimize convertTo for OpenCL: #24918

12700K iGPU:

|Name of Test|0|1|1 vs 0 (x-factor)|
|---|:-:|:-:|:-:|
|AlexNet::DNNTestNetwork::OCV/OCL_FP16|7.441|7.480|0.99|
|CRNN::DNNTestNetwork::OCV/OCL_FP16|10.776|10.736|1.00|
|DenseNet_121::DNNTestNetwork::OCV/OCL_FP16|52.762|52.833|1.00|
|EAST_text_detection::DNNTestNetwork::OCV/OCL_FP16|60.694|60.721|1.00|
|EfficientNet::DNNTestNetwork::OCV/OCL_FP16|33.373|33.173|1.01|
|FastNeuralStyle_eccv16::DNNTestNetwork::OCV/OCL_FP16|81.840|81.724|1.00|
|GoogLeNet::DNNTestNetwork::OCV/OCL_FP16|20.965|20.927|1.00|
|Inception_5h::DNNTestNetwork::OCV/OCL_FP16|22.204|22.173|1.00|
|Inception_v2_SSD_TensorFlow::DNNTestNetwork::OCV/OCL_FP16|47.115|47.460|0.99|
|MPHand::DNNTestNetwork::OCV/OCL_FP16|6.760|6.670|1.01|
|MPPalm::DNNTestNetwork::OCV/OCL_FP16|10.188|10.171|1.00|
|MPPose::DNNTestNetwork::OCV/OCL_FP16|12.510|12.561|1.00|
|MobileNet_SSD_Caffe::DNNTestNetwork::OCV/OCL_FP16|17.290|17.072|1.01|
|MobileNet_SSD_v1_TensorFlow::DNNTestNetwork::OCV/OCL_FP16|19.473|19.306|1.01|
|MobileNet_SSD_v2_TensorFlow::DNNTestNetwork::OCV/OCL_FP16|22.874|23.404|0.98|
|OpenFace::DNNTestNetwork::OCV/OCL_FP16|9.568|9.517|1.01|
|OpenPose_pose_mpi_faster_4_stages::DNNTestNetwork::OCV/OCL_FP16|539.899|539.845|1.00|
|PPHumanSeg::DNNTestNetwork::OCV/OCL_FP16|18.015|18.769|0.96|
|PPOCRv3::DNNTestNetwork::OCV/OCL_FP16|63.122|63.540|0.99|
|ResNet_50::DNNTestNetwork::OCV/OCL_FP16|34.947|34.925|1.00|
|SFace::DNNTestNetwork::OCV/OCL_FP16|10.249|10.206|1.00|
|SSD::DNNTestNetwork::OCV/OCL_FP16|213.068|213.108|1.00|
|SqueezeNet_v1_1::DNNTestNetwork::OCV/OCL_FP16|4.867|4.878|1.00|
|VIT_B_32::DNNTestNetwork::OCV/OCL_FP16|200.563|190.788|1.05|
|VitTrack::DNNTestNetwork::OCV/OCL_FP16|7.528|7.173|1.05|
|YOLOX::DNNTestNetwork::OCV/OCL_FP16|132.858|132.701|1.00|
|YOLOv3::DNNTestNetwork::OCV/OCL_FP16|209.559|208.809|1.00|
|YOLOv4::DNNTestNetwork::OCV/OCL_FP16|221.357|220.924|1.00|
|YOLOv4_tiny::DNNTestNetwork::OCV/OCL_FP16|24.446|24.382|1.00|
|YOLOv5::DNNTestNetwork::OCV/OCL_FP16|43.922|44.080|1.00|
|YOLOv8::DNNTestNetwork::OCV/OCL_FP16|64.159|63.842|1.00|
|YuNet::DNNTestNetwork::OCV/OCL_FP16|10.177|10.231|0.99|
|opencv_face_detector::DNNTestNetwork::OCV/OCL_FP16|15.121|15.445|0.98|

Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
2024-01-26 16:34:17 +03:00
Alexander Smorkalov
c739117a7c Merge branch 4.x 2024-01-19 17:32:22 +03:00
Yuantao Feng
024dfd54af dnn cann backend: add hardswish, layernorm and instasnce norm for cann and bug fix (#24462)
* add hardswish for cann

* gemm cann bug fix

* fix indentation

* cann: add layer norm

* cann: add instance norm

* add supportBackend

* cann: layer norm does not support axis=-1 due to 1d mat issue

* disable instance norm for now

* fix doc

* remove tensor desc initialization for 1D tensor
2023-11-15 17:57:52 +03:00
Alexander Smorkalov
34f34f6227 Merge branch 4.x 2023-11-08 14:39:48 +03:00
Alexander Smorkalov
97620c053f Merge branch 4.x 2023-10-23 11:53:04 +03:00
Yuantao Feng
d789cb459c Merge pull request #24231 from fengyuentau:halide_cleanup_5.x
dnn: cleanup of halide backend for 5.x #24231

Merge with https://github.com/opencv/opencv_extra/pull/1092.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2023-10-13 16:53:18 +03:00
Liutong HAN
a287605c3e Clean up the Universal Intrinsic API. 2023-10-13 19:23:30 +08:00