mirror of
https://github.com/opencv/opencv.git
synced 2026-11-17 06:47:47 -06:00
The power-of-two part is now factored as r0 * 8^a * 4^b (r0 = 8 whenever the
rest can still be covered, radix-8 stages first, radix-4 last): 256 runs 2
middle stages instead of 3, 1024 3 instead of 4, 4096 3 instead of 5, 65536 5
instead of 7. The radix-8 butterfly follows FFTW's t1fv_8: the odd legs are
combined first (4-point DFT + the W8 rotations, kept as sums and folded into
the outputs with FMAs by +-1/sqrt(2)), then the even legs, whose pure
additions go straight into the outputs. GCC's AVX2 code for the kernel has 5
register spills per butterfly, so the 16-register budget is not the issue
that made an earlier radix-8 attempt slower. The last stage of C2C / even C2R
can be radix 8 too (radix8Last writes the interleaved scaled output).
Static tables gained the radix-8 stage twiddles for spans 4..128.
What did make the earlier attempt (and the radix-4 engine) slow at large n is
4K aliasing: with a span of a 4 KB multiple the r data legs, the r-1 twiddle
leg tables and the re/im arrays of one butterfly all map to one L1 set (12
ways on Intel, 8 on Zen), and 16-30 streams thrash it. One cache line of
padding between the twiddle legs (tw_stride) and between re and im (im_ofs)
fixes it: on its own it gives 1.3-1.4x at n >= 16384 for radix 4 already.
Apply time of one cv::dft() (Core Ultra 7 255H, AVX2, no IPP), speedup of
radix-8 + padding over the previous engine:
32FC2 64FC2 32FC1 64FC1
1024 1.17 1.07 1.09 1.09
4096 1.37 1.32 1.04 1.04
16384 1.63 1.51 1.22 1.32
65536 1.76 1.38 1.39 1.33
262144 1.70 1.46 1.35 1.30
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>