Commit Graph

229 Commits

Author SHA1 Message Date
Savya Sanchi Sharma
38a2f1556a Merge pull request #29807 from SavyaSanchi-Sharma:cudnnjit_p2
dnn(cuda): move the CUDA backend to UMat device buffers - #29807

This PR is in continuation with  #29658 and #30259 
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-10-19 13:06:41 +03:00
Abhishek Gola
eacc7a59ed Merge pull request #30277 from sanjayk9/dnn-attention-mlas-qkv
dnn: speed up Attention on CPU with MLAS; support 4D input (SAM3 decoder)
2026-10-19 13:07:06 +05:30
Savya Sanchi Sharma
d1f25babff Merge pull request #30309 from SavyaSanchi-Sharma:opti/lstm
dnn: speed up LSTM2 (gate math, input projection, recurrence) - #30309

Speeds up the ONNX LSTM layer (`LSTM2`) in `recurrent2_layers.cpp`:
- Default sigmoid/tanh gates are computed on raw rows with the dispatched activation kernels. Peephole, clip and custom activations keep the old path.
- The input projection runs before the per-direction loop, so the GEMM uses all threads. It goes through MLAS when available.
- Batch 1 (hidden size >= 128): hidden units are split across threads, with one `parallel_for_` per timestep.
- Batch > 1: one packed GEMM per step replaces one GEMV per row.

Ryzen 9 9950X, default threads, 5.x vs this PR:

| Model | 5.x, ms | PR, ms | x-factor |
|---|:-:|:-:|:-:|
| `DNNTestNetwork.CRNN` | 2.693 | 1.562 | 1.72 |
| CRNN EN 2021sep | 2.498 | 1.472 | 1.70 |
| crnn.onnx | 2.650 | 1.434 | 1.85 |
| LSTM2 bidirectional, batch 1, in 512, hidden 256, 24 steps (CRNN layer 1) | 0.710 | 0.212 | 3.36 
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-10-19 09:41:52 +03:00
sanjay
2c6130dc03 Merge 5.x: keep unconditional bias push, keep QKV grouping 2026-10-15 13:09:49 +05:30
sanjay
fa0d4a06a1 dnn: group a fused Attention's tokens the way its QKV reshape does
The fusion read only num_heads and head_dim from the QKV Reshape target,
so a graph reshaping Q/K/V to [B, T*S, H, D] fused the same way as one
reshaping to [B*T, S, H, D], and the fused layer computed the second for
both, giving a wrong result for the first.

The fusion now records the first two target entries, the batch and the
sequence length the attention runs over, each as a constant times a
product of input dims, or as inferred. It reads constants and chains of
Shape, Gather, Mul, Unsqueeze, Squeeze and Cast (as in the SAM3 decoder).
If an entry is computed some other way, or Q, K and V disagree, the block
is left unfused. The layer evaluates the record against the input shape
and runs over that grouping, which is a view of the same contiguous
tokens. Without a record, as for the ONNX Attention op, the input must be
3D.
2026-10-14 21:13:31 +05:30
Abhishek Gola
94a2431d7b Merge pull request #30103 from officialarghya29/fix/dnn-matmul-rank1-a-gemm
dnn: accept a rank-1 A in the MatMul -> Gemm rewrite
2026-10-14 14:16:27 +05:30
sanjay
d8baaa6e82 dnn: size the Attention output from the V width
The output has v_hidden values per token, but getMemoryShapes() sized it
from the input width, so a V wider than the input wrote past the output
buffer. Size it from v_hidden for both output ranks, and test V widths
below, equal to and above the input width.
2026-10-14 14:01:50 +05:30
sanjay
d5c8ad61fe dnn: faster transformer inference on CPU; accept rank > 3 Attention input
- Attention Q/K/V projection: pack each (Q/K/V, head) weight slice with
  MLAS and run all per-head GEMMs in one multithreaded
  mlasSgemmPackedBatch() call; the fastGemm loop stays as the fallback.
- MLAS: split each operation into 4 tiles per thread on hybrid CPUs.
- forward(): hand the private output Mats to the caller instead of
  copying them again.
- Attention: accept [..., S, C] input (SAM3 decoder).
2026-10-13 20:34:55 +05:30
Abhishek Gola
3b3a414c9c Merge pull request #30101 from officialarghya29/fix/dnn-gru-output-direction-seqlen
dnn: honour the GRU output selection, direction=reverse, sequence_lens and clip
2026-10-08 13:43:10 +05:30
Arghya Bose
27a035e866 dnn: address review feedback on the GRU fixes
- Move the activation clip clamp and holdFinishedRows into
  recurrent_activations.hpp and call them from both GRU and RNN; drop the
  redundant useClip flag, since clip > 0 is the switch.
- Select the GRU outputs positionally, matching getMemoryShapes(), instead
  of guessing Y / Y_h from their shape and keeping the old single-output
  may-be-Y_h policy.
- Reject direction=reverse together with two-direction weights instead of
  silently running both sweeps backwards, and drop the unreachable assert.
- Collapse the four inlined GRU references into one shared helper and drop
  the stale sentence from the SingleOutputIsSequenceY comment.
- Drop the hand-made hPrev allocation: hInternal.copyTo() already sizes and
  types it.
- Gate the GRU reference on the input timestep rather than the loop
  position, so reverse + sequence_lens skips the padding instead of the
  valid steps, and cover that combination with a test.
2026-10-07 20:38:16 +05:30
Abhishek Gola
99eb7ceb88 Merge pull request #30107 from officialarghya29/fix/dnn-global-pooling-nd
dnn: support global pooling for inputs with more than two spatial dimensions
2026-10-07 12:58:58 +05:30
Arghya Bose
6fa9357d32 dnn: support global pooling for inputs with more than two spatial dimensions
GlobalAveragePool and GlobalMaxPool accept N x C x D1 x ... x Dn inputs, but
the layer only worked for at most two spatial dimensions: the parameter
vectors derived from the layer params hold one pad/stride entry per recognized
dimension (kernel_d/h/w), so a 5-D input hit
"CV_CheckLE(addedDims, pads_begin.size())" in getMemoryShapes and the model
could not be loaded at all.

The quantized pooling layer shares the same parameters, so it silently read
past the pad/stride vectors and produced garbage output shapes (and read
garbage kernels) in the same situation.

Fix by giving the paddings (0) and the strides (1) as many entries as the
global kernel has spatial dimensions, both in the shape computation and in
finalize. The extra entries are inserted at the front so that the parameters
stay trailing-aligned with the spatial dimensions, and a check rejects inputs
with more spatial dimensions than the layer can describe instead of reading
the per-dimension flags out of bounds.
2026-10-06 18:30:09 +05:30
Abhishek Gola
33d76d5ade Merge pull request #30091 from abhishek-gola:core-shape-einsum
Add squeeze/unsqueeze/flatten/reshape for N-D arrays - #30091

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-10-06 12:44:51 +03:00
Abhishek Gola
59a96e43d2 Merge pull request #30108 from officialarghya29/fix/dnn-reduce2-inner-steps
dnn: fix the step bookkeeping of the reduce inner projection
2026-10-06 13:24:28 +05:30
Arghya Bose
8556b14874 dnn: fix the step bookkeeping of the reduce inner projection
Reducing over more than two axes walks the reduced sub-block with an odometer
over all of them but the innermost one. When an index carried into the next
axis, the step of that axis was assigned instead of being accumulated, so the
steps of every axis above the one that wrapped were lost and the corresponding
elements were read again and again:

    axes [0, 2, 3] on a 3x3x3x3 input, shape (3,3,3,3), steps (27, 9, 3, 1)
      projected steps: 0 3 6 27 30 33 27 30 33     <- should end in 54 57 60

Only the first output slice happened to be right, because the carry that
introduced the wrong step is the one that moves to the second one.

The same accounting exists for the unprojected axes just below, which
subtracts the step of the wrapped axis and adds the step of the axis it carries
into; the projected loop now does the same. This fixes test_mvn_expanded and
test_mvn_expanded_ver18 (a MeanVarianceNormalization decomposed into
primitives, denied as "Wrong answer" until now) plus any model that reduces
over three or more axes, since the inner projection also feeds the unprojected
axis steps. The OpenVINO backend still answers test_mvn_expanded_ver18
incorrectly, so it is filtered in
test_onnx_conformance_layer_filter__openvino.inl.hpp.

Found by comparing against onnxruntime: 9 of 200 random (rank, shape, axes set,
keepdims, op) combinations were wrong before and none are now; the two
conformance cases above and a new layer test fail without this change.
2026-10-05 21:28:46 +05:30
Abhishek Gola
0f18b2fb93 Merge pull request #30106 from officialarghya29/fix/dnn-softmax-pre13-opset
dnn: reduce the flattened tail in Softmax before opset 13
2026-10-05 17:07:09 +05:30
Arghya Bose
27c65912aa dnn: reduce the flattened tail in Softmax before opset 13
Softmax and LogSoftmax up to ONNX opset 12 coerce the input to the 2D tensor
[a_0*...*a_{axis-1}, a_axis*...*a_{n-1}] and reduce over its second dimension, so
axis=1 on a 2x3x4 input normalises the 12 values of the flattened tail per sample,
not the 3 values of that axis. The importer only picked the right default axis for
those opsets and the layer always reduced the single axis, which differed from
onnxruntime by up to 0.63 on such models. opset 12 is included as well: Softmax-11
is still the schema in effect there.

The layer now takes the pre-opset-13 semantics through a `coerced_2d` parameter
(the name the int8 Softmax already uses) and normalises the contiguous 2D view of
the blob. The OCL kernels reduce a single axis, so that case stays on the CPU path.

The Exp/ReduceSum/Div subgraph fold produces a Softmax node too, but it matches a
single-axis ReduceSum, which is not the pre-13 meaning, so it now marks the fused
node with coerced_2d=0 to keep the per-axis semantics of the subgraph it replaces.

Tests: a layer test comparing a 2x3x4 input against the definition, and two ONNX
models whose opset_import is 11 (data added to opencv_extra by the companion
change). Both fail on the previous code with normInf 0.63 and 4.38.

Reported-by: differential testing against ONNX Runtime 1.30 CPU.
2026-09-29 22:35:16 +05:30
Arghya Bose
8c20c0e141 dnn: honour the clip attribute in the GRU layer
`clip` bounds the input of the activations to [-clip, clip]. The layer never
read it, so a model asking for it computed the unclipped gates instead - off by
up to 0.56 on the cases checked here, with no warning.

Apply it to the two places the gates are activated: the z/r pair before
sigmoid() and the candidate before tanh(), i.e. after the recurrent bias has
been folded in. It stays off unless the attribute is present and positive, so
models without it keep the same code path.

Checked against ONNX Runtime 1.30 (CPU): GRU with clip agrees to ~1e-7 for
clip = 0.4 and 0.05, with and without linear_before_reset, forward, reverse and
bidirectionally, with an initial h and with sequence_lens. The new
Layer_GRU_Test_Accuracy_.Clip clamps the gate pre-activations in its reference
and fails on Y and Y_h without the patch.

The ONNX conformance GRU cases do not set clip, so nothing covered this.
2026-09-29 21:23:33 +05:30
Arghya Bose
b07e25370a dnn: honour sequence_lens in the GRU layer
The layer never read input #4, so a node with sequence_lens ran every batch
entry to the end of the padded sequence: Y was not zero-padded past each entry's
own length, and Y_h was the state after the whole sequence instead of the one
that entry finished with. RNN2 is the only recurrent layer that implemented it.

Take RNN2's semantics, which are the spec's: an entry stops updating once it
reaches its length, keeping the state it finished with, and contributes zeros to
Y from there on. That also holds for a lone reverse direction, where the scan
starts at the last timestep but the valid range still ends at each entry's own
length.

Checked against ONNX Runtime 1.30 (CPU): GRU with sequence_lens - forward,
reverse and bidirectionally, with and without linear_before_reset, with initial
h and with fewer entries than the batch - agrees to 6e-8 - 1.9e-7 after the
change (it was 0.2 - 0.9 off before). The new
Layer_GRU_Test_Accuracy_.SequenceLens covers a sample at full length, a
truncated one and an empty one, and fails on Y and Y_h without the patch.

The ONNX conformance suite does not cover this: its test_gru_seq_length and
test_rnn_seq_length cases are named for a longer test sequence but carry no
sequence_lens input at all.
2026-09-29 21:10:17 +05:30
Arghya Bose
04d3ba40d8 dnn: honour direction=reverse in the GRU layer
The layer only ever compared `direction` against "bidirectional", so a node
with direction=reverse ran forwards: Y came out in forward order and Y_h was
the state left after the last timestep instead of the first. Walking the
sequence backwards only happened for the second direction of a bidirectional
node, while LSTM2/RNN2 honour a lone reverse direction.

Parse the attribute into a `reverse` flag and use it wherever the scan order is
picked. Checked against ONNX Runtime 1.30: the same model with
direction=reverse now matches, and Layer_GRU_Test_Accuracy_.ReverseDirection
compares Y and Y_h against the recurrence evaluated from the last timestep
down. The ONNX conformance suite has no reverse-direction GRU case.
2026-09-29 20:34:09 +05:30
Arghya Bose
94f6b25708 dnn: accept a rank-1 A in the MatMul -> Gemm rewrite
fuseMatMulConstBToGemm() rewrites MatMul(A, const B) into a Gemm with
flatten_a=false so that A keeps its leading dims. ONNX MatMul accepts a
rank-1 A: it is promoted to (1, K) and the leading 1 is squeezed from the
result again, so Y is (N,). GemmLayerImpl::getMemoryShapes() rejected that
with "Tensor A must be n-dimensional (n >= 2)", so a model like
MatMul(A[3], const B[3,6]) only loaded while B was still a runtime input;
once the fusion fired it threw. finalize() and forward() indexed
shape_A[dims_A - 2] / shape_Y[dims_Y - 2] unconditionally and would have
read out of bounds for the same shapes.

flatten_a=false is only produced by that rewrite, so gate the relaxed rank
check on it: a plain ONNX Gemm (flatten_a defaults to true) still requires a
2-D A, which is what the op specifies. Where the extra dimension is needed,
treat the rank-1 A / (N,) Y as M == 1 and use A.view as (1, K), matching
what MatMul produced before the rewrite.

Add Test_Gemm.Rank1AFromMatMulRewrite covering fp32/fp64 with and without a
bias, plus a check that a plain Gemm still rejects a rank-1 A. It fails
before the fix with the reported assertion.
2026-09-29 19:52:47 +05:30
Arghya Bose
6c68b1c553 dnn: GRU with a single output must return Y, not Y_h
For the ONNX recurrent ops output #0 is Y, the hidden state of every
timestep, and output #1 is Y_h, the last one. A node that declares a single
output can only mean Y: asking for Y_h alone requires output #0 to be
declared as an empty name, which makes it a second output, so the declared
count never collapses to one.

The GRU nevertheless answered with Y_h in that case, while LSTM2 and RNN2
both answer Y for the same node. Give slot #0 the Y shape unconditionally
when a single output is requested, which makes resolveOutputPolicy() unused.

Add Layer_GRU_Test_Accuracy_.SingleOutputIsSequenceY, written from the ONNX
GRU equations for requiredOutputs == 1; it fails before the fix and passes
after. The existing ONNX conformance GRU cases declare ['', 'Y_h'] and are
unaffected.
2026-09-29 19:51:46 +05:30
Arghya Bose
6120bde7d1 dnn: fix packed-B offsets for broadcast batched MatMul
MatMulHelper::updatePackedBOffsets derived the per-slice stride of the
packed constant weight as packed_B_size / batch, where `batch` is the
number of matrices in the *output*, i.e. after broadcasting the batch
dims. fastGemmPackB stores one packed matrix per batch slice actually
present in B, so when B is broadcast over the output batch dims the two
counts differ and every slice beyond the first was read at the wrong
offset. The output had the correct shape but wrong values: only the
first index along a broadcast dim matched.

Remember the number of slices stored in B (B_batch) in compute() and use
it to derive the packed stride.

Add a regression test using A(2,1,3,4) @ B(1,5,4,6); it fails before the
fix and passes after.
2026-09-29 19:02:49 +05:30
Jaivardhan Bhola
84c2360c27 Merge pull request #29675 from jaivardhan-bhola:generalized-tokenizer
Generalize tokenizer loading to support method-based family dispatch in DNN - #29675

Companion PR: https://github.com/opencv/opencv_extra/pull/1402

### Changes
Added ALBERT and BERT support end-to-end, with `samples/dnn/albert_inference.py `and `samples/dnn/bert_inference.py `as validation samples, plus expanded coverage in modules/dnn/test/test_tokenizer.cpp. Required changes to `cv::dnn::dnn.hpp`, `graph_fusion_attention.cpp`, and `unicode.cpp/unicode.hpp` to support Unigram and WordPiece tokenizers.

To back ALBERT/BERT, generalized `cv::dnn::Tokenizer `from a single BPE implementation into a method-dispatched frontend, adding `core_wordpiece.cpp/hpp` (WordPiece) and `core_unigram.cpp/hpp` (Unigram) as new backends. `tokenizer.cpp` now routes by method across BPE, Gemma,, SentencePiece, Unigram, and WordPiece behind one shared interface.

Tested against the following samples and the output matches to old tokenizer:

```
gpt2_inference.py
qwen_inference.py
gemma3_inference.py
```
GPT2:

```
Preparing GPT-2 model...
Inferencing GPT-2 model...
Hello, I'm a language model, not a programming language. I'm a language model. I'm a language model. I'm a language model. I'm a language model. I'm a
```

Gemma3:
```
Preparing Gemma3 model...
Prompt:
<start_of_turn>user
What is OpenCV?<end_of_turn>
<start_of_turn>model

Inferencing Gemma3 model...
Response:
Okay, let's break down what OpenCV is.

**What is OpenCV?**

OpenCV (Open Source Computer Vision Library) is a powerful and
```

Qwen2.5:
```
Preparing Qwen2.5 model...
Prompt:
<|im_start|>user
What is OpenCV?<|im_end|>
<|im_start|>assistant

Inferencing Qwen2.5 model...
Response:
OpenCV is a set of computer vision libraries in C++ designed to be used for image and video processing. It provides a wide range of tools and functions for
```

### Tokenizer References 
Byte-level BPE: [tokenizers/src/pre_tokenizers/byte_level.rs](https://github.com/huggingface/tokenizers/blob/main/tokenizers/src/pre_tokenizers/byte_level.rs)
(This defines the byte-level mapping rules, which is used in conjunction with the [BPE model](https://www.google.com/search?q=https://github.com/huggingface/tokenizers/blob/main/tokenizers/src/models/bpe/mod.rs))

SentencePiece BPE (Metaspace): [tokenizers/src/pre_tokenizers/metaspace.rs](https://www.google.com/search?q=https://github.com/huggingface/tokenizers/blob/main/tokenizers/src/pre_tokenizers/metaspace.rs)
(This defines the rule for replacing whitespace with the U+2581 _ character and handling byte fallback)

Unigram: [tokenizers/src/models/unigram/mod.rs](https://www.google.com/search?q=https://github.com/huggingface/tokenizers/blob/main/tokenizers/src/models/unigram/mod.rs)
(This contains the core logic for the Unigram lattice scoring and probabilistic tokenization rules)

WordPiece: [tokenizers/src/models/wordpiece/mod.rs](https://www.google.com/search?q=https://github.com/huggingface/tokenizers/blob/main/tokenizers/src/models/wordpiece/mod.rs)
(This explicitly cites Schuster & Nakajima in the code comments and implements the greedy longest-match rule with the ## prefix)

### Pull Request Readiness Checklist

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on
      code under GPL or another license incompatible with OpenCV.
- [x] The PR is proposed to the proper branch (`5.x`).
- [x] There is a reference to the original bug report and related work.
- [x] There is accuracy test and test data in `opencv_extra`, same
      branch name (`generalized-tokenizer`) — `bert/`, `t5/` fixtures
      back the new C++ tests.
- [x] The feature is documented and sample code builds with project CMake.
2026-09-23 16:20:33 +03:00
Kumataro
d109e4e0c0 dnn: test: workaround for GCC 16 -Wnonnull false positive 2026-09-19 10:37:28 +09:00
Vincent Rabaud
551dbcac54 Merge pull request #29948 from vrabaud:comma_initializer
Remove deprecated CommaInitializer API - #29948

This goes hand in hand with https://github.com/opencv/opencv_contrib/pull/4217

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-09-16 13:59:50 +03:00
Abhishek Gola
c7dd924be3 Merge pull request #29594 from abhishek-gola:bitcast_matmul_dft_layers
Added Bitcast layer & extended MatMul and DFT layers support - #29594

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-09-01 10:23:57 +03:00
Abhishek Gola
89e18be549 Merge pull request #29642 from abhishek-gola:kv_cache_engine
Dynamic KV-cache support - #29642

The core idea is: reserveKVCache() API to pre-allocate memory for attention caches upfront, which eliminates allocation overhead during token decoding. For LLM inference, simply call reserveKVCache(prompt_len + max_new_tokens) before the prefill stage so the decode loop runs without page allocations, significantly reducing per-token latency for models like Gemma3 and Qwen.

Speedups after this PR on AMD Ryzen 9 9950X 16-Core Processor device:

Qwen2.5-0.5B-Instruct, fp32, CPU, tok/s:

```
Tokens	   Before   After	Speedup
64	       12.49	23.72	1.90×
128	       10.37	23.14	2.23×
256	       7.20	    22.40	3.11×
512	       4.25	    21.03	4.95×

```

Gemma 3 1B-it, fp32, CPU, 512 tokens :

```
Tokens	Before	After	Speedup
64	    6.99	11.84	1.69×
128	    5.84	11.72	2.01×
256	    4.17	11.50	2.76×
512	    2.47	11.15	4.51×
```

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-08-21 17:17:50 +03:00
Yang Guanyuhan
143b084be2 dnn: remove redundant ORT test skips 2026-07-31 23:42:48 +08:00
Yang Guanyuhan
7669897910 dnn: preserve Cast semantics after ONNX Mul 2026-07-31 00:21:42 +08:00
Yang Guanyuhan
0e36cafcf4 dnn: preserve Cast semantics after ONNX Gather 2026-07-30 22:22:31 +08:00
Abhishek Gola
57bcf14e78 moved to engine_opencv 2026-07-28 20:27:29 +05:30
Abhishek Gola
49b8f7c03f engine classic removed 2026-07-28 20:27:29 +05:30
Teddy-Yangjiale
dcae1f1dc1 dnn: add RISC-V RVV FP32 fastGemm micro-kernel and Pack-B support 2026-07-02 05:46:43 +08:00
omrope79
b67ad9a422 Merge pull request #28678 from omrope79:caffe-importer-cleanup
Caffe importer cleanup #28678

Merge with: https://github.com/opencv/opencv_extra/pull/1324

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
- [x] The feature is well documented and sample code can be built with the project CMake
2026-06-02 17:28:10 +03:00
Abhishek Gola
0908a2db6f Merge pull request #29104 from abhishek-gola:sdpa
Added SDPA layer (Scaled Dot Product Attention) #29104

Merge with: https://github.com/opencv/opencv_extra/pull/1374

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-05-27 21:13:03 +03:00
Prasad Ayush Kumar
be2c53c2b7 Merge pull request #29127 from Prasadayus:KV-cache
Add KV cache with paged attention and prefetch #29127

Closes: https://github.com/opencv/opencv/issues/27159

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-05-26 10:58:47 +03:00
nklskyoy
c2594b41bf Merge pull request #28840 from nklskyoy:key-value-cache
FP32 KV Cache #28840

OpenCV Extra: https://github.com/opencv/opencv_extra/pull/1348

## This PR introduces basic (Paged ) KV-Cache to use on CPU

### Summary:
1. To ensure proper gemm-prepacking,
1.1. The Page Size of Key Cache is currently hardcoded as `FAST_GEMM_F32_NR`(which is 8, 12 or 16 depending on CPU architecture)  
1.2. The Page Size of Values Cache is hardcoded as `FAST_GEMM_F32_PACKED_STRIDE_K`
2. there are two phases supported - prefill & generate. 
2.1. prefill grows cache by `N` tokens and is allowed **only** for empty cache
2.2. generate grows cache by 1 token. 
2.3. **Improtant**: it is currently not allowed to grow non-empty cache by more than one token at a time (thisbehaviour is sufficient for normal LLM querying, but should be extended if we want to implement speculative decoding)

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-05-25 13:39:02 +03:00
Abhishek Gola
f85a662563 Merge pull request #28971 from abhishek-gola:subgraph_argname_fix
Fixed subgraph name scoping in new DNN engine #28971

closes: https://github.com/opencv/opencv/issues/23663, https://github.com/opencv/opencv/issues/19977

OpenCV extra: https://github.com/opencv/opencv_extra/pull/1362

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-05-19 12:38:16 +03:00
Varun Jaiswal
562628eb71 Merge pull request #28836 from varun-jaiswal17:fix-dnn-nan-bugs
Fix dnn NaN bugs in GELU SIMD and softmax for large inputs #28836

### Bug Description

- **GELU SIMD bug:** `exp(2*inner)` overflows to `inf` for large inputs (e.g. `x=10.6` → `inner≈51`), causing `inf/inf = NaN`; fixed by clamping `inner` to [-9, 9] as `tanh` already saturates to ±1.0 beyond this range.
- **Softmax bug:** All `-inf` inputs (masked attention rows) produce `sum=0`, then `1/0 = inf` and `0*inf = NaN`; 
fixed by outputting zeros when `sum == 0`.
- Add regression tests for both fixes

Depends on : https://github.com/opencv/opencv/pull/28837

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-04-22 10:00:51 +03:00
Abhishek Gola
3d77645a3a Merge pull request #28750 from abhishek-gola:activation_fusion
Extended fusion for activation functions in new DNN engine #28750

After this fusion, we see following improvements in YOLO models:

| Model | Before (`ENGINE_NEW`) | After (`ENGINE_NEW`) | `ENGINE_ORT` | % Improvement (Before v/s After) |
| :--- | :--- | :--- | :--- | :--- |
| **YOLOv8n** | 18.89  ms| 12.06 ms| 12.15 ms| 36.16% |
| **YOLOv5n** | 17.12  ms| 9.29 ms| 9.23 ms| 45.73% |
| **YOLOX-S** | 38.78  ms| 25.56 ms| 25.16 ms| 34.09% |

Device details: 
      - Model name: Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-04-14 09:08:14 +03:00
Alexander Smorkalov
1b6fb61b89 Merge branch 4.x 2025-11-18 08:51:08 +03:00
Dmitry Kurtaev
5c02d32cca Merge pull request #28000 from dkurt:d.kurtaev:reset_winograd_impl
### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

resolves https://github.com/opencv/opencv/issues/27580

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-11-13 09:19:46 +03:00
Vadim Pisarevsky
bdab54f79e Merge pull request #27757 from vpisarev:matshape_inside_mat
Use MatShape instead of MatSize inside cv::Mat/cv::UMat #27757

**Merge together with https://github.com/opencv/opencv_contrib/pull/3996**
---

This PR continues cv::Mat/cv::UMat refactoring. See #26056, where `MatShape` was introduced. Now it's put inside cv::Mat/cv::UMat instead of a weird `MatSize`. MatSize is now an alias for MatShape:

**before:**

```
struct MatShape { ... };
struct MatSize { ... };

struct Mat {
    ...
    int dims;
    int rows;
    int cols;

    ...

    MatShape shape() const { ... /* constructs MatShape out of MatSize and returns it;
                                    layout is always 'unknown', because we don't store it */ }

    MatSize size; // size is not valid without the parent cv::Mat,
                  // because size.p may point to Mat::rows or to Mat::cols,
                  // depending on the dimensionality, and dims() returns Mat::dims.
    MatStep step; // may allocate memory, depending on the dimensionality.
    ...
};
```

**after:**

```
struct MatShape { ... };
typedef MatShape MatSize; // they are now synonyms

struct Mat {
    ...
    int dims;
    int rows;
    int cols;

    ...

    MatShape shape() const { return size; } // just return the embedded shape (including the proper layout information)

    MatSize size; // size is self-contained data structure that can be used without the parent cv::Mat.
                  // size.dims is now a copy of dims; size.p[*] contains copies of Mat::rows and Mat::cols when dims <= 2.
    MatStep step; // does not allocate extra memory buffers.
    ...
};
```

There are several reasons to do that:

1. the main reason is to be able to store data layout (MatShape::layout) inside each cv::Mat/cv::UMat. This is necessary for the proper shape inference in DNN module. In particular, it's necessary for the next step of DNN inference optimization where we introduce block-layout-optimized convolution and other operations. Later on, we can use layout information to support non-interleaved images (e.g. RRR...GGG...BBB...) or even batches of such images in core/imgproc modules.
2. the other reason is to represent 3D/4D/5D etc. tensors as cv::Mat/cv::UMat instances more conveniently, without extra dynamic memory allocation. Before this patch we allocated some memory buffers dynamically to store shape & steps for more than 2D arrays. Now the whole cv::Mat/cv::UMat header can be stored completely on stack/in a container. Creating another copy of Mat/UMat header is now done more efficiently.
3. the third reason is to introduce the new coding pattern: `dst.create(src.size, <dst_type>);`. The pattern is suitable for most of element-wise (including cloning) and filtering operations. This pattern does not only look crisp and self-documenting, it will also automatically copy shape (including layout) from the source tensor into the destination matrix/tensor.
4. in the future we might add `colorspace` member to MatShape that will allow to distinguish RGB from BGR or NV12. `dst.create(src.size, <dst_type>);` will then copy the colorspace information as well.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-09-15 15:03:34 +03:00
Abhishek Gola
38a72d57c1 Merge pull request #27656 from abhishek-gola:size_layer_add
Added Size layer to new DNN engine #27656

Merge with https://github.com/opencv/opencv_extra/pull/1274

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-08-12 14:38:21 +03:00
Abhishek Gola
709eabda16 Merge pull request #27508 from abhishek-gola:if_layer_add
IfLayer add to new DNN engine #27508

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-07-16 09:01:54 +03:00
nklskyoy
1df06488b1 Merge pull request #27238 from nklskyoy:mha-rope
Rotary positional embeddings #27238

Rotary positional embeddings let an attention block encode the relative position between tokens. Widely adopted in LLMs such as LLaMA—and equally applicable to Vision Transformers—the ONNX Runtime com.microsoft.Attention operator already exposes this via its `do_rotary` flag (see docs), and this PR adds support for that flag in OpenCV.

_Operator spec: https://github.com/microsoft/onnxruntime/blob/v1.16.1/docs/ContribOperators.md#com.microsoft.Attention_

Materials:
- RoFormer paper https://arxiv.org/abs/2104.09864
- RoPe for Vision Transformer https://github.com/naver-ai/rope-vit
- llama implementation https://github.com/meta-llama/llama/blob/main/llama/model.py#L80


Merge with https://github.com/opencv/opencv_extra/pull/1250

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2025-04-28 19:47:23 +03:00
Alexander Smorkalov
83a1621527 Fixed test LSTM warning. 2024-11-22 09:10:25 +03:00
Abduragim Shtanchaev
d0820dac38 Merge pull request #26391 from Abdurrahheem:ash/lstm-new-graph-engine-latest
LSTM layer for new graph engine. #26391

Merge with extra: https://github.com/opencv/opencv_extra/pull/1218

This PR updates/creates LSTM layer compatible with new graph engine. It is based on previous LSTM implementation with some modification on how initializers blobs are processed.

Note: Following tests are currently are disabled 


Two following two tests are disbled since ONNNRuntime does not support `layout=1` attiribute inference. See a detailed issue #26456 on this.
- `LSTM_layout_seq` 
- `LSTM_layout_batch`

Following test fails with the new engine as it is not able to deal with shapes of the form [?, C, H, W]
- `LSTM_Activations`

Works:
- [x] One directional case any batch type 
 - [x] Fix directional case when batch size large than 1
 - [x] Add peepholes attribute

TODO with the next PRs:
 - [ ] Activation support

Note: 
  

> Currently `LSTM_layout_seq`, `LSTM_layout_batch` are disabled as the tests are incorrect. They do not comply with the ONNX standard. Particularly test outputs are of incorrect dimensionality. They produce 3-dimentinal output instead of 4-dimentional. 

------

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-11-20 14:12:58 +03:00
alexlyulkov
a2fa1d49a4 Merge pull request #26208 from alexlyulkov:al/new-engine-caffe-parser
Modified Caffe parser to support the new dnn engine #26208

Now the Caffe parser supports both the old and the new engine. It can be selected using newEngine argument in PopulateNet.

All cpu Caffe tests work fine except:

- Test_Caffe_nets.Colorization
- Test_Caffe_layers.FasterRCNN_Proposal

Both these tests doesn't work because of the bug in the new net.forward function. The function takes the name of the desired target last layer, but uses this name as the name of the desired output tensor.
Also Colorization test contains a strange model with a Silence layer in the end, so it doesn't have outputs. The old parser just ignored it. I think, the proper solution is to run this model until the (number_of_layers - 2) layer using proper net.forward arguments in the test.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-10-28 11:32:07 +03:00