Merge pull request #29702 from Prasadayus:test_suite_cleanup

Re-enable & triage DISABLED tests for DNN module - #29702

Requires: https://github.com/opencv/opencv_extra/pull/1403

**co-authored by: @varun-jaiswal17**

### PR Changes:

 ## dnn test cleanup: re-enable stale-disabled tests, fix defect-blind tests, remove redundant coverage

  ### Removed (dead or unbuildable)
- `test_int8_layers.cpp` (1118 lines, removed entirely): cannot compile — `Net::quantize()`,
  `getInputDetails`/`getOutputDetails` are all gone from `dnn.hpp`/`dnn/src`. Disabled in that same PR (#24980)
  because on-the-fly quantization was removed — every test in this file called `net.quantize()` to calibrate and
  run its own int8 conversion. Its own header comment said restore "when test models are quantized outside
  OpenCV". Pre-quantized ONNX/TFLite test models already do that.

  ### Removed (redundant or assertion-free)
  - `Tokenizer_BPE.Tokenizer_GPT2_Model`: line-for-line subset of `Tokenizer_GPT2` — same config, same input,
  same roundtrip assertion.
  - `Test_TensorFlow.read_inception`: printed `out.dims` and asserted nothing about the result;
  `inception_accuracy` loads the same `.pb` and checks it against a reference.
  - `Test_Caffe_nets` fixture + `INSTANTIATE`: registered **zero** `TEST_P` cases — dead scaffolding for Faster
  R-CNN tests removed earlier.
  - `Test_ONNX_nets.Squeezenet`: kernels {1×1, 3×3} and every op type already covered by dedicated layer tests.
  - `Test_ONNX_nets.VGG16_bn`: single conv kernel (3×3), fully covered by dedicated layer tests; skipped by
  default anyway under `mem_6gb`.
  - `Test_ONNX_nets.CaffeNet`: identical op multiset, node count (24) and conv signatures to retained `Alexnet`.
  - `Test_ONNX_nets.RCNN_ILSVRC13`: `Alexnet` minus `Softmax` (23 vs 24 nodes), identical conv signatures.
  - `Test_ONNX_nets.Inception_v1`: same op set as retained `Googlenet` (+1 `Reshape`) — Inception v1 *is*
  GoogLeNet.

  ### Given real assertions instead of stale expectations
  - `Test_ONNX_layers.Elementwise_Sqrt`: moved `testONNXModels("sqrt")` below `#endif` — its only work line sat
  inside `INF_ENGINE_VER_MAJOR_LT(2021040000)`, so without OpenVINO the body compiled to nothing and reported `[
  OK ]` on all 3 backends.
  - `Layer_Test_01D.Clip`: now calls `ClipLayer::create` with `"min"`/`"max"` — it set `lp.type = "Clip"` but
  constructed `ReLU6Layer::create`, and `runLayer` never reads `layer->type`, so it just re-ran `ReLU6`.
  - `Layer_Arg_Test`: removed the "disabled" comment, corrected the `convertTo` comment — the comment said the
  test was disabled while it runs 8 cases, and the second said "convert to float" where the code converts to
  `CV_64S`.

  ### Re-enabled as-is (stale disable reasons)
  - `Test_ONNX_layers.LSTM`/`LSTM_bidirectional` (`test_onnx_importer.cpp:1551,1558`): disabled by #21522 (2022)
  for poor 1-D-mat handling in the importer of that era; no longer reproduces.
  - `Test_ONNX_layers.Split_sizes_0d` (`:1373`): disabled by #22652 for a Mul/0-d-tensor shape ambiguity (A×1 vs
  1×A); dnn now supports real 1-D Mats, so the output matches the reference exactly.
  - `DNNTestNetwork.YOLOv8n`

  ### Library fixes found while re-enabling
  - `Test_ONNX_layers.LSTM_layout_seq`/`LSTM_layout_batch` (`test_onnx_importer.cpp:1721,1728`): `LSTM2` never
  transposed `X` for ONNX `layout=1` (batch-first); fixed via `transposeND` gated on `layout==BATCH_SEQ_HID`
  (`recurrent2_layers.cpp:172`). Fixture also had a leaked loop variable that made the reference a copy of the
  input; rebuilt by hand since ORT itself refuses to run `layout=1`.
  - `Test_Graph_Simplifier.ResizeSubgraph` (`test_graph_simplifier.cpp:61`): disabled by the block-layout PR
  #28585; expectations updated for the `TransformLayout` pass that PR introduced. The test now covers 4 subgraphs rather than 6, because `GatherCastSubgraph` and `MulCastSubgraph` were removed by `0e36cafcf4` and `7669897910` (`Gather`/`Mul` -> `Cast` is no longer fused, since folding it away silently dropped the `Cast`'s dtype semantics). The dynamic-scale `Shape`/`Gather`/`Cast`/`Floor`/`Concat`/`Unsqueeze`/`Slice` chain these models use to compute Resize's scale factor therefore no longer collapses, and the `Mul` survives as `NaryEltwise`, which is why the expected layer lists grew

  ### Deliberately kept
  - `ZFNet`: its **7×7** conv appears in no dedicated layer test, and its kernel set {7×7, 5×5, 3×3} differs from
  `Alexnet`'s {11×11, 5×5, 3×3}.
  
 ### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake

modules/dnn/src/layers/recurrent2_layers.cpp

@@ -491,7 +491,18 @@ class LSTM2LayerImpl CV_FINAL : public LSTM2Layer
             // seq-major cell-state scratch: (seq, batch, dirs, hid), matching the recurrence.
             int cOutShape[] = {seqLenth, batchSize, numDirs, numHidden};
             Mat cOut = produceCellOutput ? Mat::zeros(4, cOutShape, output[0].type()) : Mat();
-            Mat xTs = input[0].reshape(1, batchSizeTotal);
+
+            // the recurrence below slices X by timestep, so it needs the seq-major order;
+            // under ONNX layout=1 the input arrives as (batch, seq, ...)
+            Mat xSeqFirst = input[0];
+            if (layout == BATCH_SEQ_HID)
+            {
+                std::vector<int> perm(input[0].dims);
+                std::iota(perm.begin(), perm.end(), 0);
+                std::swap(perm[0], perm[1]);
+                cv::transposeND(input[0], perm, xSeqFirst);
+            }
+            Mat xTs = xSeqFirst.reshape(1, batchSizeTotal);
 
             // seq-major Y assembly buffer; transposed into output[0] below.
             // Never reallocate output[0]'s header or it detaches from the graph.

modules/dnn/test/test_backends.cpp

@@ -115,7 +115,7 @@ public:
     Net net;
 };
 
-TEST_P(DNNTestNetwork, DISABLED_YOLOv8n) {
+TEST_P(DNNTestNetwork, YOLOv8n) {
     processNet("dnn/onnx/models/yolov8n.onnx", "", Size(640, 640), "output0");
     expectNoFallbacksFromIE(net);
     expectNoFallbacksFromCUDA(net);

modules/dnn/test/test_caffe_importer.cpp

@@ -52,37 +52,6 @@ static std::string _tf(TString filename)
     return findDataFile(std::string("dnn/") + filename);
 }
 
-class Test_Caffe_nets : public DNNTestLayer
-{
-public:
-    void testFaster(const std::string& proto, const std::string& model, const Mat& ref,
-                    double scoreDiff = 0.0, double iouDiff = 0.0)
-    {
-        checkBackend();
-        Net net = readNet(findDataFile("dnn/" + proto),
-                          findDataFile("dnn/" + model, false));
-        net.setPreferableBackend(backend);
-        net.setPreferableTarget(target);
-
-        if (target == DNN_TARGET_CPU_FP16)
-            net.enableWinograd(false);
-
-        Mat img = imread(findDataFile("dnn/dog416.png"));
-        resize(img, img, Size(800, 600));
-        Mat blob = blobFromImage(img, 1.0, Size(), Scalar(102.9801, 115.9465, 122.7717), false, false);
-        Mat imInfo = (Mat_<float>(1, 3) << img.rows, img.cols, 1.6f);
-
-        net.setInput(blob);
-        net.setInput(imInfo, "im_info");
-        // Output has shape 1x1xNx7 where N - number of detections.
-        // An every detection is a vector of values [id, classId, confidence, left, top, right, bottom]
-        Mat out = net.forward();
-        scoreDiff = scoreDiff ? scoreDiff : default_l1;
-        iouDiff = iouDiff ? iouDiff : default_lInf;
-        normAssertDetections(ref, out, ("model name: " + model).c_str(), 0.8, scoreDiff, iouDiff);
-    }
-};
-
 TEST(Reproducibility_SSD, Accuracy)
 {
     applyTestTag(
@@ -137,6 +106,4 @@ TEST(Test_Caffe, multiple_inputs)
     normAssert(out, first_image + second_image);
 }
 
-INSTANTIATE_TEST_CASE_P(/**/, Test_Caffe_nets, dnnBackendsAndTargets());
-
 }} // namespace

modules/dnn/test/test_graph_simplifier.cpp

@@ -26,8 +26,9 @@ class Test_Graph_Simplifier : public ::testing::Test {
         std::vector<std::string> layers;
         net.getLayerTypes(layers);
 
-        // remove Const, Identity (output layer), __NetInputLayer__ (input layer)
-        layers.erase(std::remove_if(layers.begin(), layers.end(), [] (const std::string l) { return l == "Const" || l == "Identity" || l == "__NetInputLayer__"; }), layers.end());
+        // remove Const, Identity (output layer), __NetInputLayer__ (input layer),
+        // TransformLayout (inserted by the block layout pass)
+        layers.erase(std::remove_if(layers.begin(), layers.end(), [] (const std::string l) { return l == "Const" || l == "Identity" || l == "__NetInputLayer__" || l == "TransformLayout"; }), layers.end());
         // Instead of 'Tile', 'Expand' etc. we may now have 'Tile2', 'Expand2' etc.
         // We should correctly match them with the respective patterns
         for (auto& l: layers) {
@@ -57,19 +58,18 @@ TEST_F(Test_Graph_Simplifier, LayerNormNoFusionSubGraph) {
     test("layer_norm_no_fusion", std::vector<std::string>{"NaryEltwise", "Reduce", "Sqrt"});
 }
 
-TEST_F(Test_Graph_Simplifier, DISABLED_ResizeSubgraph) {
-    /* Test for 6 subgraphs:
-        - GatherCastSubgraph
-        - MulCastSubgraph
+TEST_F(Test_Graph_Simplifier, ResizeSubgraph) {
+    /* Test for 4 subgraphs:
         - UpsampleSubgraph
         - ResizeSubgraph1
         - ResizeSubgraph2
         - ResizeSubgraph3
     */
-    test("upsample_unfused_torch1.2", std::vector<std::string>{"BatchNorm", "Resize"});
-    test("resize_nearest_unfused_opset11_torch1.3", std::vector<std::string>{"BatchNorm", "Convolution", "Resize"});
-    test("resize_nearest_unfused_opset11_torch1.4", std::vector<std::string>{"BatchNorm", "Convolution", "Resize"});
-    test("upsample_unfused_opset9_torch1.4", std::vector<std::string>{"BatchNorm", "Convolution", "Resize"});
+    test("upsample_unfused_torch1.2", std::vector<std::string>{"BatchNorm", "Cast", "Concat", "Floor", "Gather", "NaryEltwise", "Resize", "Shape", "Slice", "Unsqueeze"});
+    // In the models below the BatchNorm is folded into the preceding convolution by fuseBN().
+    test("resize_nearest_unfused_opset11_torch1.3", std::vector<std::string>{"Cast", "Concat", "Conv", "Floor", "Gather", "NaryEltwise", "Resize", "Shape", "Unsqueeze"});
+    test("resize_nearest_unfused_opset11_torch1.4", std::vector<std::string>{"Cast", "Concat", "Conv", "Floor", "Gather", "NaryEltwise", "Resize", "Shape", "Slice", "Unsqueeze"});
+    test("upsample_unfused_opset9_torch1.4", std::vector<std::string>{"Cast", "Concat", "Conv", "Floor", "Gather", "NaryEltwise", "Resize", "Shape", "Slice", "Unsqueeze"});
     test("two_resizes_with_shared_subgraphs", std::vector<std::string>{"NaryEltwise", "Resize"});
 }
 

modules/dnn/test/test_layers_1d.cpp

@@ -78,9 +78,9 @@ TEST_P(Layer_Test_01D, Clip)
 
     lp.type = "Clip";
     lp.name = "ClipLayer";
-    lp.set("min_value", 0.0);
-    lp.set("max_value", 1.0);
-    Ptr<ReLU6Layer> layer = ReLU6Layer::create(lp);
+    lp.set("min", 0.0);
+    lp.set("max", 1.0);
+    Ptr<ClipLayer> layer = ClipLayer::create(lp);
 
     Mat output_ref(output_shape.size(), output_shape.data(), CV_32F, 1.0);
     std::vector<Mat> inputs{input};
@@ -725,7 +725,6 @@ int arg_op(const std::vector<T>& vec, const std::string& operation) {
         CV_Error(Error::StsAssert, "Provided operation: " + operation + " is not supported. Please check the test instantiation.");
     }
 }
-// Test for ArgLayer is disabled because there problem in runLayer function related to type assignment
 typedef testing::TestWithParam<tuple<std::vector<int>, std::string>> Layer_Arg_Test;
 TEST_P(Layer_Arg_Test, Accuracy_01D) {
     std::vector<int> input_shape = get<0>(GetParam());
@@ -774,7 +773,7 @@ TEST_P(Layer_Arg_Test, Accuracy_01D) {
     runLayer(layer, inputs, outputs);
     ASSERT_EQ(1, outputs.size());
     ASSERT_EQ(shape(output_ref), shape(outputs[0]));
-    // convert output_ref to float to match the output type
+    // ArgLayer::getTypes() reports CV_64S; match it before comparing
     output_ref.convertTo(output_ref, CV_64SC1);
     normAssert(output_ref, outputs[0]);
 }

modules/dnn/test/test_onnx_importer.cpp

@@ -735,8 +735,8 @@ TEST_P(Test_ONNX_layers, Elementwise_Sqrt)
         applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_NN_BUILDER);
     if (backend == DNN_BACKEND_INFERENCE_ENGINE_NGRAPH)
         applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_NGRAPH);
-    testONNXModels("sqrt");
 #endif
+    testONNXModels("sqrt");
 }
 
 TEST_P(Test_ONNX_layers, Elementwise_not)
@@ -1374,8 +1374,7 @@ TEST_P(Test_ONNX_layers, Split)
     testONNXModels("split_neg_axis");
 }
 
-// Mul inside with 0-d tensor, output should be A x 1, but is 1 x A. PR #22652
-TEST_P(Test_ONNX_layers, DISABLED_Split_sizes_0d)
+TEST_P(Test_ONNX_layers, Split_sizes_0d)
 {
     if (backend == DNN_BACKEND_INFERENCE_ENGINE_NN_BUILDER_2019)
         applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_NN_BUILDER);
@@ -1551,14 +1550,12 @@ TEST_P(Test_ONNX_layers, LSTM_Activations)
     testONNXModels("lstm_cntk_tanh", pb, 0, 0, false, false);
 }
 
-// disabled due to poor handling of 1-d mats
-TEST_P(Test_ONNX_layers, DISABLED_LSTM)
+TEST_P(Test_ONNX_layers, LSTM)
 {
     testONNXModels("lstm", npy, 0, 0, false, false);
 }
 
-// disabled due to poor handling of 1-d mats
-TEST_P(Test_ONNX_layers, DISABLED_LSTM_bidirectional)
+TEST_P(Test_ONNX_layers, LSTM_bidirectional)
 {
     testONNXModels("lstm_bidirectional", npy, 0, 0, false, false);
 }
@@ -1721,20 +1718,14 @@ TEST_P(Test_ONNX_layers, LSTM_init_h0_c0)
     testONNXModels("lstm_init_h0_c0", npy, 0, 0, false, false, 3);
 }
 
-// epsilon is larger because onnx does not match with torch/opencv exactly
-// Test uses incorrect ONNX and test data with 3 dims instead of 4.
-// ONNNRuntime does not support layout=1 attiribute inference. See a detailed issue #26456
-TEST_P(Test_ONNX_layers, DISABLED_LSTM_layout_seq)
+TEST_P(Test_ONNX_layers, LSTM_layout_seq)
 {
     if(backend == DNN_BACKEND_CUDA)
         applyTestTag(CV_TEST_TAG_DNN_SKIP_CUDA);
     testONNXModels("lstm_layout_0", npy, 0.005, 0.005, false, false, 3);
 }
 
-// epsilon is larger because onnx does not match with torch/opencv exactly
-// Test uses incorrect ONNX and test data with 3 dims instead of 4.
-// ONNNRuntime does not support layout=1 attiribute inference. See a detailed issue #26456
-TEST_P(Test_ONNX_layers, DISABLED_LSTM_layout_batch)
+TEST_P(Test_ONNX_layers, LSTM_layout_batch)
 {
     if(backend == DNN_BACKEND_CUDA)
         applyTestTag(CV_TEST_TAG_DNN_SKIP_CUDA);
@@ -2516,11 +2507,6 @@ TEST_P(Test_ONNX_nets, RAFT)
     normAssert(ref0, outs[0], "", 1.5e-3, 3.2e-2);
 }
 
-TEST_P(Test_ONNX_nets, Squeezenet)
-{
-    testONNXModels("squeezenet", pb);
-}
-
 TEST_P(Test_ONNX_nets, Googlenet)
 {
 #if defined(INF_ENGINE_RELEASE) && INF_ENGINE_VER_MAJOR_EQ(2022010000)
@@ -2568,48 +2554,6 @@ TEST_P(Test_ONNX_nets, Googlenet)
     expectNoFallbacksFromIE(net);
 }
 
-TEST_P(Test_ONNX_nets, CaffeNet)
-{
-#if defined(OPENCV_32BIT_CONFIGURATION) && (defined(HAVE_OPENCL) || defined(_WIN32))
-    applyTestTag(CV_TEST_TAG_MEMORY_2GB);
-#else
-    applyTestTag(target == DNN_TARGET_CPU ? CV_TEST_TAG_MEMORY_512MB : CV_TEST_TAG_MEMORY_1GB);
-#endif
-
-#if defined(INF_ENGINE_RELEASE) && INF_ENGINE_VER_MAJOR_EQ(2019030000)
-    if (backend == DNN_BACKEND_INFERENCE_ENGINE_NN_BUILDER_2019 && target == DNN_TARGET_MYRIAD
-        && getInferenceEngineVPUType() == CV_DNN_INFERENCE_ENGINE_VPU_TYPE_MYRIAD_X)
-        applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_MYRIAD_X, CV_TEST_TAG_DNN_SKIP_IE_NN_BUILDER, CV_TEST_TAG_DNN_SKIP_IE_VERSION);
-#endif
-    testONNXModels("caffenet", pb);
-}
-
-TEST_P(Test_ONNX_nets, RCNN_ILSVRC13)
-{
-#if defined(OPENCV_32BIT_CONFIGURATION) && (defined(HAVE_OPENCL) || defined(_WIN32))
-    applyTestTag(CV_TEST_TAG_MEMORY_2GB);
-#else
-    applyTestTag(target == DNN_TARGET_CPU ? CV_TEST_TAG_MEMORY_512MB : CV_TEST_TAG_MEMORY_1GB);
-#endif
-
-#if defined(INF_ENGINE_RELEASE) && INF_ENGINE_VER_MAJOR_EQ(2019030000)
-    if (backend == DNN_BACKEND_INFERENCE_ENGINE_NN_BUILDER_2019 && target == DNN_TARGET_MYRIAD
-        && getInferenceEngineVPUType() == CV_DNN_INFERENCE_ENGINE_VPU_TYPE_MYRIAD_X)
-        applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_MYRIAD_X, CV_TEST_TAG_DNN_SKIP_IE_NN_BUILDER, CV_TEST_TAG_DNN_SKIP_IE_VERSION);
-#endif
-    // Reference output values are in range [-4.992, -1.161]
-    testONNXModels("rcnn_ilsvrc13", pb, 0.0046);
-}
-
-TEST_P(Test_ONNX_nets, VGG16_bn)
-{
-    applyTestTag(CV_TEST_TAG_MEMORY_6GB);  // > 2.3Gb
-
-    // output range: [-16; 27], after Softmax [0; 0.67]
-    const double lInf = (target == DNN_TARGET_MYRIAD) ? 0.038 : default_lInf;
-    testONNXModels("vgg16-bn", pb, default_l1, lInf, true);
-}
-
 TEST_P(Test_ONNX_nets, ZFNet)
 {
     applyTestTag(CV_TEST_TAG_MEMORY_2GB);
@@ -2836,16 +2780,6 @@ TEST_P(Test_ONNX_nets, DenseNet121)
     testONNXModels("densenet121", pb, default_l1, default_lInf, true, target != DNN_TARGET_MYRIAD);
 }
 
-TEST_P(Test_ONNX_nets, Inception_v1)
-{
-#if defined(INF_ENGINE_RELEASE) && INF_ENGINE_VER_MAJOR_LT(2021040000)
-    if ((backend == DNN_BACKEND_INFERENCE_ENGINE_NN_BUILDER_2019 ||
-         backend == DNN_BACKEND_INFERENCE_ENGINE_NGRAPH) && target == DNN_TARGET_MYRIAD)
-        applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_MYRIAD);
-#endif
-    testONNXModels("inception_v1", pb);
-}
-
 TEST_P(Test_ONNX_nets, Shufflenet)
 {
 #if defined(INF_ENGINE_RELEASE) && INF_ENGINE_VER_MAJOR_LT(2021040000)

modules/dnn/test/test_tf_importer.cpp

@@ -27,30 +27,6 @@ static std::string _tf(TString filename)
     return (getOpenCVExtraDir() + "/dnn/") + filename;
 }
 
-TEST(Test_TensorFlow, read_inception)
-{
-    Net net;
-    {
-        const string model = findDataFile("dnn/tensorflow_inception_graph.pb", false);
-        net = readNetFromTensorflow(model);
-        ASSERT_FALSE(net.empty());
-    }
-    net.setPreferableBackend(DNN_BACKEND_OPENCV);
-
-    Mat sample = imread(_tf("grace_hopper_227.png"));
-    ASSERT_TRUE(!sample.empty());
-    Mat input;
-    resize(sample, input, Size(224, 224));
-    input -= Scalar::all(117); // mean sub
-
-    Mat inputBlob = blobFromImage(input);
-
-    net.setInput(inputBlob, "input");
-    Mat out = net.forward();
-
-    std::cout << out.dims << std::endl;
-}
-
 TEST(Test_TensorFlow, inception_accuracy)
 {
     Net net;

modules/dnn/test/test_tokenizer.cpp

@@ -52,14 +52,6 @@ TEST(Tokenizer_BPE, Tokenizer_GPT2) {
     std::cout << word << std::endl;
 }
 
-TEST(Tokenizer_BPE, Tokenizer_GPT2_Model) {
-    std::string gpt2_model = _tf("gpt2/config.json");
-    Tokenizer tok = Tokenizer::load(gpt2_model);
-    auto ids = tok.encode("hello world");
-    auto text = tok.decode(ids);
-    EXPECT_EQ(text, "hello world");
-}
-
 TEST(Tokenizer_BPE, SimpleRepeated_GPT2) {
     Tokenizer gpt2_tok = Tokenizer::load(_tf("gpt2/config.json"));
     EXPECT_EQ(gpt2_tok.encode("0"), std::vector<int>({15}));