Administrator/NANOKVM-MaixCDK-MIRROR
1
0
Fork 1
mirror of https://github.com/sipeed/MaixCDK.git synced 2026-11-09 17:04:38 -06:00
Code Issues Packages Projects Releases 1 Wiki Activity

feat(nn): add depth calibration params to Yolo26Depth API (#137)

Browse Source 
get_depth/get_depth_image now accept cal_a/cal_b so callers can pass
a per-camera scale calibration d_real = exp(a*log(d)+b); example app
exposes them as optional CLI args.

Co-authored-by: Taoxuan-168 <Taoxuan-168@users.noreply.github.com>
This commit is contained in:
Tao
2026-08-10 11:08:16 +08:00
committed by GitHub
parent fd4b8d2311
commit 489e19fb9b
2 changed files with 49 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
components/nn/include/maix_nn_yolo26_depth.hpp
View File

@@ -187,11 +187,16 @@ namespace maix::nn
* @param fit image resize fit mode if input image not equal to model' input size,
* will auto resize to model's input size then detect, and recover to image input size.
* Default Fit.FIT_CONTAIN, see image.Fit.
* @param cal_a calibration scale factor a, the depth is converted by d_real = exp(a * log(d) + b).
* Default 1.0 (no calibration). Set both cal_a/cal_b to fit your camera/scene.
* @param cal_b calibration offset b, see cal_a. Default 0.0 (no calibration).
* @throw If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error.
* @return result, a tensor.Tensor object. If in dual_buff mode, value can be None(in Python) or nullptr(in C++) when not ready. In C++, you need to delete it after use.
* @return result, a tensor.Tensor object with calibrated depth in meters. If in dual_buff mode, value can be None(in Python) or nullptr(in C++) when not ready. In C++, you need to delete it after use.
* @maixpy maix.nn.YOLO26Depth.get_depth
* @maixcdk maix.nn.YOLO26Depth.get_depth
*/
tensor::Tensor *get_depth(image::Image &img, image::Fit fit = image::FIT_CONTAIN)
tensor::Tensor *get_depth(image::Image &img, image::Fit fit = image::FIT_CONTAIN,
float cal_a = 1.0f, float cal_b = 0.0f)
{
if (_model == nullptr) return nullptr;
std::unique_ptr<tensor::Tensors> outputs(_model->forward_image(img, {}, {}, fit, false, false));
@@ -205,6 +210,7 @@ namespace maix::nn
_output_h = info.height;
_output_w = info.width;
tensor::Tensor *result = new tensor::Tensor(t->shape(), t->dtype(), t->data(), true);
_apply_calibration((float *)result->data(), info.width * info.height, cal_a, cal_b);
return result;
}
@@ -217,14 +223,19 @@ namespace maix::nn
* @param cmap Color map used convert grayscale distance estimation image to RGB image.
* Diiferent cmap will influence finally image.
* Default image.CMap.JET (near red/yellow, far blue).
* @param cal_a calibration scale factor a, the depth is converted by d_real = exp(a * log(d) + b).
* Default 1.0 (no calibration). Set both cal_a/cal_b to fit your camera/scene.
* @param cal_b calibration offset b, see cal_a. Default 0.0 (no calibration).
* @throw If error occurred, will raise err::Exception, you can find reason in log, mostly caused by args error or hardware error.
* @return result, a image::Image object. If in dual_buff mode, value can be None(in Python) or nullptr(in C++) when not ready. In C++, you need to delete it after use.
* @return result, a image::Image object with calibrated depth heatmap. If in dual_buff mode, value can be None(in Python) or nullptr(in C++) when not ready. In C++, you need to delete it after use.
* @maixpy maix.nn.YOLO26Depth.get_depth_image
* @maixcdk maix.nn.YOLO26Depth.get_depth_image
*/
image::Image *get_depth_image(image::Image &img, image::Fit fit = image::FIT_CONTAIN,
image::CMap cmap = image::CMap::JET)
image::CMap cmap = image::CMap::JET,
float cal_a = 1.0f, float cal_b = 0.0f)
{
std::unique_ptr<tensor::Tensor> depth(get_depth(img, fit));
std::unique_ptr<tensor::Tensor> depth(get_depth(img, fit, cal_a, cal_b));
if (!depth) return nullptr;
return depth_to_image(*depth, img.width(), img.height(), fit, cmap);
}
@@ -392,6 +403,22 @@ namespace maix::nn
return transform;
}
/**
* Apply log-affine depth calibration in place: d_real = exp(a * log(d) + b).
* With default a=1, b=0 the data is unchanged.
*/
void _apply_calibration(float *data, int count, float cal_a, float cal_b)
{
if (cal_a == 1.0f && cal_b == 0.0f)
return;
for (int i = 0; i < count; i++)
{
float d = data[i];
if (std::isfinite(d) && d > 0)
data[i] = expf(cal_a * logf(d) + cal_b);
}
}
/**
* Convert depth map to RGB888 heatmap: disparity(1/depth) + min-max normalize + cmap lookup.
* Return a new image::Image, caller should delete it.

25
examples/nn_yolo26_depth/main/src/main.cpp
View File

@@ -2,11 +2,11 @@
* YOLO26-depth example
*
* Usage:
* - 单张图片推理并保存热力图:
* ./nn_yolo26_depth mud_model_path image_path [output_path]
* 例: ./nn_yolo26_depth /tmp/yolo26n-depth.mud /tmp/bus.jpg /tmp/bus_heatmap.jpg
* - 实时摄像头推理并显示热力图:
* ./nn_yolo26_depth mud_model_path
* - 单张图片推理并保存热力图(可选传 cal_a cal_b 校准):
* ./nn_yolo26_depth mud_model_path image_path [output_path] [cal_a cal_b]
* 例: ./nn_yolo26_depth /tmp/yolo26n-depth.mud /tmp/bus.jpg /tmp/bus_heatmap.jpg 0.85 0.40
* - 实时摄像头推理并显示热力图(可选传 cal_a cal_b):
* ./nn_yolo26_depth mud_model_path [cal_a cal_b]
* 点击画面可添加/取消距离探针，最多保留 5 个，距离随画面实时更新。
*
* 依赖(设备上需存在):
@@ -166,6 +166,15 @@ int _main(int argc, char *argv[])
const char *model_path = argv[1];
bool dual_buff = false;
image::CMap cmap = image::CMap::JET;
// 可选深度校准参数(默认不校准): d_real = exp(cal_a * log(d_raw) + cal_b)
float cal_a = 1.0f;
float cal_b = 0.0f;
if (argc >= 6)
{
cal_a = std::stof(argv[4]);
cal_b = std::stof(argv[5]);
log::info("depth calibration enabled: cal_a=%f cal_b=%f", cal_a, cal_b);
}
log::info("model path: %s", model_path);
nn::YOLO26Depth model(model_path, dual_buff);
@@ -189,8 +198,8 @@ int _main(int argc, char *argv[])
img->width(), img->height(), model.input_width(), model.input_height());
}
// 推理得到深度热力图(自动 resize 回原图尺寸)
maix::image::Image *heatmap = model.get_depth_image(*img, image::FIT_CONTAIN, cmap);
// 推理得到深度热力图(自动 resize 回原图尺寸; 可传 cal_a/cal_b 校准)
maix::image::Image *heatmap = model.get_depth_image(*img, image::FIT_CONTAIN, cmap, cal_a, cal_b);
err::check_null_raise(heatmap, "get_depth_image failed");
// 保存热力图
@@ -246,7 +255,7 @@ int _main(int argc, char *argv[])
was_pressed = pressed;
}
std::unique_ptr<tensor::Tensor> depth(model.get_depth(*img, image::FIT_CONTAIN));
std::unique_ptr<tensor::Tensor> depth(model.get_depth(*img, image::FIT_CONTAIN, cal_a, cal_b));
if (!depth)
continue;