Re-enable & triage DISABLED tests for DNN module - #29702
Requires: https://github.com/opencv/opencv_extra/pull/1403
**co-authored by: @varun-jaiswal17**
### PR Changes:
## dnn test cleanup: re-enable stale-disabled tests, fix defect-blind tests, remove redundant coverage
### Removed (dead or unbuildable)
- `test_int8_layers.cpp` (1118 lines, removed entirely): cannot compile — `Net::quantize()`,
`getInputDetails`/`getOutputDetails` are all gone from `dnn.hpp`/`dnn/src`. Disabled in that same PR (#24980)
because on-the-fly quantization was removed — every test in this file called `net.quantize()` to calibrate and
run its own int8 conversion. Its own header comment said restore "when test models are quantized outside
OpenCV". Pre-quantized ONNX/TFLite test models already do that.
### Removed (redundant or assertion-free)
- `Tokenizer_BPE.Tokenizer_GPT2_Model`: line-for-line subset of `Tokenizer_GPT2` — same config, same input,
same roundtrip assertion.
- `Test_TensorFlow.read_inception`: printed `out.dims` and asserted nothing about the result;
`inception_accuracy` loads the same `.pb` and checks it against a reference.
- `Test_Caffe_nets` fixture + `INSTANTIATE`: registered **zero** `TEST_P` cases — dead scaffolding for Faster
R-CNN tests removed earlier.
- `Test_ONNX_nets.Squeezenet`: kernels {1×1, 3×3} and every op type already covered by dedicated layer tests.
- `Test_ONNX_nets.VGG16_bn`: single conv kernel (3×3), fully covered by dedicated layer tests; skipped by
default anyway under `mem_6gb`.
- `Test_ONNX_nets.CaffeNet`: identical op multiset, node count (24) and conv signatures to retained `Alexnet`.
- `Test_ONNX_nets.RCNN_ILSVRC13`: `Alexnet` minus `Softmax` (23 vs 24 nodes), identical conv signatures.
- `Test_ONNX_nets.Inception_v1`: same op set as retained `Googlenet` (+1 `Reshape`) — Inception v1 *is*
GoogLeNet.
### Given real assertions instead of stale expectations
- `Test_ONNX_layers.Elementwise_Sqrt`: moved `testONNXModels("sqrt")` below `#endif` — its only work line sat
inside `INF_ENGINE_VER_MAJOR_LT(2021040000)`, so without OpenVINO the body compiled to nothing and reported `[
OK ]` on all 3 backends.
- `Layer_Test_01D.Clip`: now calls `ClipLayer::create` with `"min"`/`"max"` — it set `lp.type = "Clip"` but
constructed `ReLU6Layer::create`, and `runLayer` never reads `layer->type`, so it just re-ran `ReLU6`.
- `Layer_Arg_Test`: removed the "disabled" comment, corrected the `convertTo` comment — the comment said the
test was disabled while it runs 8 cases, and the second said "convert to float" where the code converts to
`CV_64S`.
### Re-enabled as-is (stale disable reasons)
- `Test_ONNX_layers.LSTM`/`LSTM_bidirectional` (`test_onnx_importer.cpp:1551,1558`): disabled by #21522 (2022)
for poor 1-D-mat handling in the importer of that era; no longer reproduces.
- `Test_ONNX_layers.Split_sizes_0d` (`:1373`): disabled by #22652 for a Mul/0-d-tensor shape ambiguity (A×1 vs
1×A); dnn now supports real 1-D Mats, so the output matches the reference exactly.
- `DNNTestNetwork.YOLOv8n`
### Library fixes found while re-enabling
- `Test_ONNX_layers.LSTM_layout_seq`/`LSTM_layout_batch` (`test_onnx_importer.cpp:1721,1728`): `LSTM2` never
transposed `X` for ONNX `layout=1` (batch-first); fixed via `transposeND` gated on `layout==BATCH_SEQ_HID`
(`recurrent2_layers.cpp:172`). Fixture also had a leaked loop variable that made the reference a copy of the
input; rebuilt by hand since ORT itself refuses to run `layout=1`.
- `Test_Graph_Simplifier.ResizeSubgraph` (`test_graph_simplifier.cpp:61`): disabled by the block-layout PR
#28585; expectations updated for the `TransformLayout` pass that PR introduced. The test now covers 4 subgraphs rather than 6, because `GatherCastSubgraph` and `MulCastSubgraph` were removed by `0e36cafcf4` and `7669897910` (`Gather`/`Mul` -> `Cast` is no longer fused, since folding it away silently dropped the `Cast`'s dtype semantics). The dynamic-scale `Shape`/`Gather`/`Cast`/`Floor`/`Concat`/`Unsqueeze`/`Slice` chain these models use to compute Resize's scale factor therefore no longer collapses, and the `Mul` survives as `NaryEltwise`, which is why the expected layer lists grew
### Deliberately kept
- `ZFNet`: its **7×7** conv appears in no dedicated layer test, and its kernel set {7×7, 5×5, 3×3} differs from
`Alexnet`'s {11×11, 5×5, 3×3}.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Add Gemma3 tokenizer support for dnn #28837
- Adds Gemma3 tokenizer support
- Implements character-level BPE
- Adds 6 tests covering English, phrase, mixed case, numbers, special tokens, and encode/decode
- add gemma3_inference.py
Merge with:
- **Companion PR** : https://github.com/opencv/opencv_extra/pull/1346
- forward pass bug in gemma3_inference.py : https://github.com/opencv/opencv/pull/28836
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Add Qwen2.5 tokenizer support for dnn #28781
OpenCV extra: https://github.com/opencv/opencv_extra/pull/1334
Extended the dnn tokenizer support to qwen2.5 tokenization.
- Add QWEN2_5 pre-tokenizer regex pattern to utils.hpp
- Generalised buildTokenizerGPT to buildTokenizerFromJson to handle gpt2/gpt4/ and qwen2.5
- Add qwen2/qwen2.5 model type support with special token handling
- Add Qwen2.5 tests
- Add end-to-end qwen_inference script for Qwen2.5 ONNX model
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
GSoC 2025: Add Tokenizer Support to DNN Module #27534
merge with https://github.com/opencv/opencv_extra/pull/1276
### Summary
This pull request introduces initial support for a tokenizer module under `modules/dnn/src/tokenizer` as part of Google Summer of Code 2025 (Project: Tokenization for OpenCV DNN).
### Status
- [x] Project structure in place
- [x] Initial BPE tokenizer loading
- [x] Regex splitting (in progress)
- [x] Encoding logic for GPT-2 tokenizer (in progress)
- [ ] Documentation (to be improved)
### Goals
The goal is to support Hugging Face-compatible tokenization (e.g., GPT-2) natively in C++ to be integrated with DNN inference pipelines.
The core pipeline lives in `dnn/src/tokenizer/core_bpe.hpp` and `dnn/src/tokenizer/encoding.hpp`. For Unicode handling I’m using `dnn/src/tokenizer/unicode.hpp`, which is adapted from llama.cpp.
### Feedback
Please share early feedback on:
- General design structure
- Integration strategy with `dnn`
- Code organization or naming conventions
### Reference
Project: https://summerofcode.withgoogle.com/programs/2025/projects/79SW6eNK
