Files
opencv-MIRROR/modules
Vadim Pisarevsky c7bf418d06 core(dft): radix-8 stages and cache-set padding for the power-of-two lengths
The power-of-two part is now factored as r0 * 8^a * 4^b (r0 = 8 whenever the
rest can still be covered, radix-8 stages first, radix-4 last): 256 runs 2
middle stages instead of 3, 1024 3 instead of 4, 4096 3 instead of 5, 65536 5
instead of 7. The radix-8 butterfly follows FFTW's t1fv_8: the odd legs are
combined first (4-point DFT + the W8 rotations, kept as sums and folded into
the outputs with FMAs by +-1/sqrt(2)), then the even legs, whose pure
additions go straight into the outputs. GCC's AVX2 code for the kernel has 5
register spills per butterfly, so the 16-register budget is not the issue
that made an earlier radix-8 attempt slower. The last stage of C2C / even C2R
can be radix 8 too (radix8Last writes the interleaved scaled output).
Static tables gained the radix-8 stage twiddles for spans 4..128.

What did make the earlier attempt (and the radix-4 engine) slow at large n is
4K aliasing: with a span of a 4 KB multiple the r data legs, the r-1 twiddle
leg tables and the re/im arrays of one butterfly all map to one L1 set (12
ways on Intel, 8 on Zen), and 16-30 streams thrash it. One cache line of
padding between the twiddle legs (tw_stride) and between re and im (im_ofs)
fixes it: on its own it gives 1.3-1.4x at n >= 16384 for radix 4 already.

Apply time of one cv::dft() (Core Ultra 7 255H, AVX2, no IPP), speedup of
radix-8 + padding over the previous engine:
             32FC2  64FC2  32FC1  64FC1
   1024       1.17   1.07   1.09   1.09
   4096       1.37   1.32   1.04   1.04
  16384       1.63   1.51   1.22   1.32
  65536       1.76   1.38   1.39   1.33
 262144       1.70   1.46   1.35   1.30

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 16:50:34 +03:00
..
2026-07-09 12:17:24 +03:00