mirror of
https://github.com/sipeed/sipeed_wiki.git
synced 2026-09-11 00:22:49 -05:00
tof 修改
This commit is contained in:
lyx080
BIN
docs/hardware/zh/metasense/assets/tof-1.12.jpg
Normal file
BIN
docs/hardware/zh/metasense/assets/tof-1.12.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.0 MiB |
BIN
docs/hardware/zh/metasense/assets/tof-1.13.jpg
Normal file
BIN
docs/hardware/zh/metasense/assets/tof-1.13.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 70 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 79 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 47 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 300 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 60 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 53 KiB |
|
Before Width: | Height: | Size: 145 KiB After Width: | Height: | Size: 145 KiB |
BIN
docs/hardware/zh/metasense/metasense-a010/assets/ms-lamp.jpg
Normal file
BIN
docs/hardware/zh/metasense/metasense-a010/assets/ms-lamp.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
BIN
docs/hardware/zh/metasense/metasense-a010/assets/ms-mcu.jpg
Normal file
BIN
docs/hardware/zh/metasense/metasense-a010/assets/ms-mcu.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.1 MiB |
BIN
docs/hardware/zh/metasense/metasense-a010/assets/ms-people.jpg
Normal file
BIN
docs/hardware/zh/metasense/metasense-a010/assets/ms-people.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.3 MiB |
|
Before Width: | Height: | Size: 102 KiB After Width: | Height: | Size: 102 KiB |
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 57 KiB |
169
docs/hardware/zh/metasense/metasense-a010/at_command.md
Normal file
169
docs/hardware/zh/metasense/metasense-a010/at_command.md
Normal file
@@ -0,0 +1,169 @@
|
||||
# MetaSense-A010 二次开发手册
|
||||
|
||||
## AT 指令表
|
||||
|
||||
| AT | |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| +ISP<br>Image Signal Processor | :0: turn ISP off<br>=1: turn ISP on |
|
||||
| +BINN<br>full binning | =1: output 100x100 pixel frame<br>=2: output 50x50 pixel frame<br>=4: output 25x25 pixel frame<br> |
|
||||
| +DISP<br>display mux | =0: all off<br>=1: lcd display on<br>=2: usb display on<br>=3: lcd and usb display on<br>=4: uart display on<br>=5: lcd and uart display on<br>=6: usb and uart display on<br>=7: lcd, usb and uart display on |
|
||||
| +BAUD<br>uart baudrate | =0: 9600<br>=1: 57600<br>=2: 115200<br>=3: 230400<br>=4: 460800<br>=5: 921600<br>=6: 1000000<br>=7: 2000000<br>=8: 3000000 |
|
||||
| +UNIT<br>quantization unit | =0: auto<br>=1-10: quantizated by unit(mm) |
|
||||
| +FPS<br>frame per second | =1-19: set frame per second |
|
||||
| +Save<br>save config | : save current configuration |
|
||||
|
||||
|
||||
|输入|执行|注释|
|
||||
|---|---|---|
|
||||
|AT+ISP? |\r|返回当前ISP状态|
|
||||
|AT+ISP=? |\r|返回所有支持的ISP状态|
|
||||
|AT+ISP=< MODE >|\r|选择ISP状态|
|
||||
|
||||
参数：
|
||||
|< MODE > | 含义 |
|
||||
|----|----|
|
||||
|0 "STOP ISP" |立即关闭模组ISP，停止IR发射器|
|
||||
|1 "LAUNCH ISP" |计划启动模组ISP，实际出图需等待1～2秒|
|
||||
|
||||
BINN指令
|
||||
句法：
|
||||
| 输入 | 执行 | 注释 |
|
||||
|----|----|----|
|
||||
| AT+BINN? | \r | 返回当前BINN状态 |
|
||||
| AT+BINN=? | \r | 返回所有支持的BINN状态 |
|
||||
| AT+BINN= < MODE > | \r | 选择BINN状态 |
|
||||
|
||||
参数：
|
||||
| < MODE > | 含义 |
|
||||
|----------|------|
|
||||
| 1 "1x1 BINN" | 1x1相当于无binning，实际出图分辨率为100x100。 |
|
||||
| 2 "2x2 BINN" | 2×2binning，4个像素点合并成1个，实际出图分辨率为50×50计划启动模组ISP，实际出图需等待1～2秒。|
|
||||
| 4 "4x4 BINN" | 4×4binning，16个像素点合并成1个，实际出图分辨率为25×25。 |
|
||||
|
||||
|
||||
DISP指令
|
||||
请按需开启，避免资源过度占用
|
||||
句法：
|
||||
|
||||
| 输入 | 执行 | 注释 |
|
||||
|------|------|-----|
|
||||
| AT+DISP? | \r | 返回当前DISP状态 |
|
||||
| AT+DISP=? | \r | 返回所有支持的DISP状态 |
|
||||
| AT+DISP=< MODE > | \r | 选择DISP状态 |
|
||||
|
||||
参数：
|
||||
|
||||
|
||||
| < MODE > | 含义 |
|
||||
|----------|------|
|
||||
| 0 | all off |
|
||||
| 1 | lcd display on |
|
||||
| 2 | usb display on |
|
||||
| 3 | lcd and usb display on |
|
||||
| 4 | uart display on |
|
||||
| 5 | lcd and uart display on |
|
||||
| 6 | usb and uart display on |
|
||||
| 7 | lcd, usb and uart display on |
|
||||
|
||||
BAUD指令
|
||||
句法：
|
||||
|
||||
| 输入 | 执行 | 注释 |
|
||||
|------|-----|------|
|
||||
| AT+BAUD? | \r | 返回当前BAUD状态 |
|
||||
| AT+BAUD=? | \r | 返回所有支持的BAUD状态 |
|
||||
| AT+BAUD=< MODE > | \r | 选择BAUD状态 |
|
||||
|
||||
参数：
|
||||
|
||||
| < MODE > | 含义 |
|
||||
|----------|------|
|
||||
| 0 | 9600 |
|
||||
| 1 | 57600 |
|
||||
| 2 | 115200 |
|
||||
| 3 | 230400 |
|
||||
| 4 | 460800 |
|
||||
| 5 | 921600 |
|
||||
| 6 | 1000000 |
|
||||
| 7 | 2000000 |
|
||||
| 8 | 3000000 |
|
||||
|
||||
UNIT指令
|
||||
句法：
|
||||
|
||||
| 输入 | 执行 | 注释 |
|
||||
|------|------|------|
|
||||
| AT+UNIT? | \r | 返回当前UNIT值 |
|
||||
| AT+UNIT=? | \r | 返回所有支持的UNIT值 |
|
||||
| AT+UNIT=< UINT > | \r | 选择UNIT值 |
|
||||
|
||||
参数：
|
||||
| < UINT > | 含义 |
|
||||
|----------|------|
|
||||
| 0 "DEFAULT UNIT" | 采用默认量化策略，因tof特性导致成像近处精度优于远距离处，故放大近距离处差异，采用5.1*sqrt(x)将16bit的原数据量化为8bit
|
||||
1...9 "QUANTIZE UNIT"s
|
||||
|
||||
代表以x mm为单位进行量化，取值越小细节越多，同时可视距离越短，请合理设置
|
||||
FPS指令
|
||||
句法：
|
||||
输入
|
||||
执行
|
||||
注释
|
||||
AT+FPS?
|
||||
\r
|
||||
返回当前FPS值
|
||||
AT+FPS=?
|
||||
\r
|
||||
返回所有支持的FPS值
|
||||
AT+FPS=<FPS>
|
||||
\r
|
||||
选择FPS值
|
||||
参数：
|
||||
< FPS >
|
||||
含义
|
||||
1...19 "frame per second"
|
||||
|
||||
tof出图帧率，越大越流畅
|
||||
SAVE指令
|
||||
句法：
|
||||
输入
|
||||
执行
|
||||
注释
|
||||
AT+SAVE
|
||||
\r
|
||||
固化TOF摄像头当前配置，事后需要复位
|
||||
|
||||
多机 和 AE 指令建议加入
|
||||
ANTIMMI指令
|
||||
句法：
|
||||
输入
|
||||
执行
|
||||
注释
|
||||
AT+ANTIMMI?
|
||||
\r
|
||||
返回当前ANTIMMI状态
|
||||
AT+ANTIMMI=?
|
||||
\r
|
||||
返回所有支持的ANTIMMI状态
|
||||
AT+ANTIMMI=<MODE>
|
||||
\r
|
||||
选择ANTIMMI状态
|
||||
参数：
|
||||
<MODE>
|
||||
含义
|
||||
-1
|
||||
disable anti-mmi
|
||||
0
|
||||
auto anti-mmi
|
||||
1-41
|
||||
manual anti-mmi usb display on
|
||||
|
||||
图像数据包说明
|
||||
上电默认启动ISP并在显示屏显示图像，同时输出图像数据到uart和usb
|
||||
图像数据封装成包（未稳定）：
|
||||
1. 包头2字节：0X00、0XFF
|
||||
2. 包长度2字节：当前包剩余数据的字节数
|
||||
3. 其他内容16字节：包括包序号、包长度、分辨率等等
|
||||
4. 图像帧
|
||||
5. 校验1字节：之前所有字节的“和”低八位
|
||||
6. 包尾1字节：0XDD
|
||||
921
docs/hardware/zh/metasense/metasense-a010/code.md
Normal file
921
docs/hardware/zh/metasense/metasense-a010/code.md
Normal file
@@ -0,0 +1,921 @@
|
||||
# 源码总览
|
||||
|
||||
- [源码总览](#源码总览)
|
||||
- [tof_mainpy](#tof_mainpy)
|
||||
- [streampy](#streampy)
|
||||
- [calvolumespy](#calvolumespy)
|
||||
- [calibratepy](#calibratepy)
|
||||
|
||||
## tof_mainpy
|
||||
|
||||
```python
|
||||
from fpioa_manager import fm
|
||||
from machine import UART
|
||||
import lcd, image
|
||||
|
||||
# lcd.init(invert=True)
|
||||
lcd.init()
|
||||
img = image.Image()
|
||||
|
||||
fm.register(24, fm.fpioa.UART1_TX, force=True)
|
||||
fm.register(25, fm.fpioa.UART1_RX, force=True)
|
||||
|
||||
uart_A = UART(UART.UART1, 115200, 8, 0, 0, timeout=1000, read_buf_len=4096)
|
||||
|
||||
def uart_readBytes():
|
||||
return uart_A.read()
|
||||
|
||||
|
||||
def uart_hasData():
|
||||
return uart_A.any()
|
||||
|
||||
|
||||
def uart_sendCmd(cmd):
|
||||
uart_A.write(cmd)
|
||||
|
||||
uart_sendCmd(b"AT+BAUD=5\r")
|
||||
|
||||
uart_A.deinit()
|
||||
|
||||
uart_A = UART(UART.UART1, 921600, 8, 0, 0, timeout=1000, read_buf_len=4096)
|
||||
|
||||
jetcolors = [
|
||||
(128, 0, 0), (132, 0, 0), (136, 0, 0), (140, 0, 0), (144, 0, 0), (148, 0, 0), (152, 0, 0), (156, 0, 0), (160, 0, 0), (164, 0, 0), (168, 0, 0), (172, 0, 0), (176, 0, 0), (180, 0, 0), (184, 0, 0), (188, 0, 0), (192, 0, 0), (196, 0, 0), (200, 0, 0), (204, 0, 0), (208, 0, 0), (212, 0, 0), (216, 0, 0), (220, 0, 0), (224, 0, 0), (228, 0, 0), (232, 0, 0), (236, 0, 0), (240, 0, 0), (244, 0, 0), (248, 0, 0), (252, 0, 0), (255, 0, 0), (255, 4, 0), (255, 8, 0), (255, 12, 0), (255, 16, 0), (255, 20, 0), (255, 24, 0), (255, 28, 0), (255, 32, 0), (255, 36, 0), (255, 40, 0), (255, 44, 0), (255, 48, 0), (255, 52, 0), (255, 56, 0), (255, 60, 0), (255, 64, 0), (255, 68, 0), (255, 72, 0), (255, 76, 0), (255, 80, 0), (255, 84, 0), (255, 88, 0), (255, 92, 0), (255, 96, 0), (255, 100, 0), (255, 104, 0), (255, 108, 0), (255, 112, 0), (255, 116, 0), (255, 120, 0), (255, 124, 0), (255, 128, 0), (255, 132, 0), (255, 136, 0), (255, 140, 0), (255, 144, 0), (255, 148, 0), (255, 152, 0), (255, 156, 0), (255, 160, 0), (255, 164, 0), (255, 168, 0), (255, 172, 0), (255, 176, 0), (255, 180, 0), (255, 184, 0), (255, 188, 0), (255, 192, 0), (255, 196, 0), (255, 200, 0), (255, 204, 0), (255, 208, 0), (255, 212, 0), (255, 216, 0), (255, 220, 0), (255, 224, 0), (255, 228, 0), (255, 232, 0), (255, 236, 0), (255, 240, 0), (255, 244, 0), (255, 248, 0), (255, 252, 0), (254, 255, 1), (250, 255, 6), (246, 255, 10), (242, 255, 14), (238, 255, 18), (234, 255, 22), (230, 255, 26), (226, 255, 30), (222, 255, 34), (218, 255, 38), (214, 255, 42), (210, 255, 46), (206, 255, 50), (202, 255, 54), (198, 255, 58), (194, 255, 62), (190, 255, 66), (186, 255, 70), (182, 255, 74), (178, 255, 78), (174, 255, 82), (170, 255, 86), (166, 255, 90), (162, 255, 94), (158, 255, 98), (154, 255, 102), (150, 255, 106), (146, 255, 110), (142, 255, 114), (138, 255, 118), (134, 255, 122), (130, 255, 126),
|
||||
(126, 255, 130), (122, 255, 134), (118, 255, 138), (114, 255, 142), (110, 255, 146), (106, 255, 150), (102, 255, 154), (98, 255, 158), (94, 255, 162), (90, 255, 166), (86, 255, 170), (82, 255, 174), (78, 255, 178), (74, 255, 182), (70, 255, 186), (66, 255, 190), (62, 255, 194), (58, 255, 198), (54, 255, 202), (50, 255, 206), (46, 255, 210), (42, 255, 214), (38, 255, 218), (34, 255, 222), (30, 255, 226), (26, 255, 230), (22, 255, 234), (18, 255, 238), (14, 255, 242), (10, 255, 246), (6, 255, 250), (2, 255, 254), (0, 252, 255), (0, 248, 255), (0, 244, 255), (0, 240, 255), (0, 236, 255), (0, 232, 255), (0, 228, 255), (0, 224, 255), (0, 220, 255), (0, 216, 255), (0, 212, 255), (0, 208, 255), (0, 204, 255), (0, 200, 255), (0, 196, 255), (0, 192, 255), (0, 188, 255), (0, 184, 255), (0, 180, 255), (0, 176, 255), (0, 172, 255), (0, 168, 255), (0, 164, 255), (0, 160, 255), (0, 156, 255), (0, 152, 255), (0, 148, 255), (0, 144, 255), (0, 140, 255), (0, 136, 255), (0, 132, 255), (0, 128, 255), (0, 124, 255), (0, 120, 255), (0, 116, 255), (0, 112, 255), (0, 108, 255), (0, 104, 255), (0, 100, 255), (0, 96, 255), (0, 92, 255), (0, 88, 255), (0, 84, 255), (0, 80, 255), (0, 76, 255), (0, 72, 255), (0, 68, 255), (0, 64, 255), (0, 60, 255), (0, 56, 255), (0, 52, 255), (0, 48, 255), (0, 44, 255), (0, 40, 255), (0, 36, 255), (0, 32, 255), (0, 28, 255), (0, 24, 255), (0, 20, 255), (0, 16, 255), (0, 12, 255), (0, 8, 255), (0, 4, 255), (0, 0, 255), (0, 0, 252), (0, 0, 248), (0, 0, 244), (0, 0, 240), (0, 0, 236), (0, 0, 232), (0, 0, 228), (0, 0, 224), (0, 0, 220), (0, 0, 216), (0, 0, 212), (0, 0, 208), (0, 0, 204), (0, 0, 200), (0, 0, 196), (0, 0, 192), (0, 0, 188), (0, 0, 184), (0, 0, 180), (0, 0, 176), (0, 0, 172), (0, 0, 168), (0, 0, 164), (0, 0, 160), (0, 0, 156), (0, 0, 152), (0, 0, 148), (0, 0, 144), (0, 0, 140), (0, 0, 136), (0, 0, 132), (0, 0, 128)
|
||||
]
|
||||
|
||||
def show(frameData, res):
|
||||
resR = res[0]
|
||||
resC = res[1]
|
||||
for y in range(resR):
|
||||
for x in range(resC):
|
||||
pixel_cmap_rgb = jetcolors[frameData[y*resR + x]]
|
||||
img.set_pixel(110 + x, 70 + y, pixel_cmap_rgb)
|
||||
lcd.display(img)
|
||||
img.clear()
|
||||
|
||||
FRAME_HEAD = b"\x00\xFF"
|
||||
FRAME_TAIL = b"\xCC"
|
||||
|
||||
from struct import unpack
|
||||
# send_cmd("AT+BINN=2\r")
|
||||
uart_sendCmd(b"AT+DISP=5\r")
|
||||
uart_sendCmd(b"AT+FPS=10\r")
|
||||
|
||||
# while True:
|
||||
# if uart_hasData():
|
||||
# print(uart_readBytes())
|
||||
|
||||
rawData = b''
|
||||
while True:
|
||||
if not uart_hasData():
|
||||
continue
|
||||
rawData += uart_readBytes()
|
||||
idx = rawData.find(FRAME_HEAD)
|
||||
if idx < 0:
|
||||
continue
|
||||
rawData = rawData[idx:]
|
||||
# print(rawData)
|
||||
# check data length 2Byte
|
||||
dataLen = unpack("H", rawData[2: 4])[0]
|
||||
# print("len: "+str(dataLen))
|
||||
frameLen = len(FRAME_HEAD) + 2 + dataLen + 2
|
||||
frameDataLen = dataLen - 16
|
||||
|
||||
if len(rawData) < frameLen:
|
||||
continue
|
||||
# get data
|
||||
frame = rawData[:frameLen]
|
||||
# print(frame.hex())
|
||||
rawData = rawData[frameLen:]
|
||||
|
||||
frameTail = frame[-1]
|
||||
# print("tail: "+str(hex(frameTail)))
|
||||
_sum = frame[-2]
|
||||
# print("checksum: "+str(hex(_sum)))
|
||||
# check sum
|
||||
# spi has no checksum but i add one
|
||||
if frameTail != 0xdd and _sum != sum(frame[:frameLen - 2]) % 256:
|
||||
continue
|
||||
|
||||
frameID = unpack("H", frame[16:18])[0]
|
||||
# print("frame ID: "+str(frameID))
|
||||
|
||||
resR = unpack("B", frame[14:15])[0]
|
||||
resC = unpack("B", frame[15:16])[0]
|
||||
res = (resR, resC)
|
||||
# print(res)
|
||||
# frameData=[ unpack("H", frame[20+i:22+i])[0] for i in range(0, frameDataLen, 2) ]
|
||||
frameData = [unpack("B", frame[20+i:21+i])[0]
|
||||
for i in range(0, frameDataLen, 1)]
|
||||
|
||||
show(frameData, res)
|
||||
|
||||
del frameData
|
||||
```
|
||||
|
||||
## streampy
|
||||
|
||||
```python
|
||||
from PIL import Image
|
||||
import requests
|
||||
import matplotlib.pyplot as plt
|
||||
import struct
|
||||
import numpy as np
|
||||
import cv2
|
||||
|
||||
|
||||
def frame_config_decode(frame_config):
|
||||
'''
|
||||
@frame_config bytes
|
||||
|
||||
@return fields, tuple (trigger_mode, deep_mode, deep_shift, ir_mode, status_mode, status_mask, rgb_mode, rgb_res, expose_time)
|
||||
'''
|
||||
return struct.unpack("<BBBBBBBBi", frame_config)
|
||||
|
||||
|
||||
def frame_config_encode(trigger_mode=1, deep_mode=1, deep_shift=255, ir_mode=1, status_mode=2, status_mask=7, rgb_mode=1, rgb_res=0, expose_time=0):
|
||||
return struct.pack("<BBBBBBBBi",
|
||||
trigger_mode, deep_mode, deep_shift, ir_mode, status_mode, status_mask, rgb_mode, rgb_res, expose_time)
|
||||
|
||||
|
||||
def frame_payload_decode(frame_data: bytes, with_config: tuple):
|
||||
deep_data_size, rgb_data_size = struct.unpack("<ii", frame_data[:8])
|
||||
frame_payload = frame_data[8:]
|
||||
# 0:16bit 1:8bit, resolution: 320*240
|
||||
deepth_size = (320*240*2) >> with_config[1]
|
||||
deepth_img = struct.unpack("<%us" % deepth_size, frame_payload[:deepth_size])[
|
||||
0] if 0 != deepth_size else None
|
||||
frame_payload = frame_payload[deepth_size:]
|
||||
|
||||
# 0:16bit 1:8bit, resolution: 320*240
|
||||
ir_size = (320*240*2) >> with_config[3]
|
||||
ir_img = struct.unpack("<%us" % ir_size, frame_payload[:ir_size])[
|
||||
0] if 0 != ir_size else None
|
||||
frame_payload = frame_payload[ir_size:]
|
||||
|
||||
status_size = (320*240//8) * (16 if 0 == with_config[4] else
|
||||
2 if 1 == with_config[4] else 8 if 2 == with_config[4] else 1)
|
||||
status_img = struct.unpack("<%us" % status_size, frame_payload[:status_size])[
|
||||
0] if 0 != status_size else None
|
||||
frame_payload = frame_payload[status_size:]
|
||||
|
||||
assert(deep_data_size == deepth_size+ir_size+status_size)
|
||||
|
||||
rgb_size = len(frame_payload)
|
||||
assert(rgb_data_size == rgb_size)
|
||||
rgb_img = struct.unpack("<%us" % rgb_size, frame_payload[:rgb_size])[
|
||||
0] if 0 != rgb_size else None
|
||||
|
||||
if (not rgb_img is None) and (1 == with_config[6]):
|
||||
jpeg = cv2.imdecode(np.frombuffer(
|
||||
rgb_img, 'uint8', rgb_size), cv2.IMREAD_COLOR)
|
||||
if not jpeg is None:
|
||||
rgb = cv2.cvtColor(jpeg, cv2.COLOR_BGR2RGB)
|
||||
rgb_img = rgb.tobytes()
|
||||
else:
|
||||
rgb_img = None
|
||||
|
||||
return (deepth_img, ir_img, status_img, rgb_img)
|
||||
|
||||
|
||||
HOST = '192.168.233.1'
|
||||
PORT = 80
|
||||
|
||||
|
||||
def post_encode_config(config=frame_config_encode(), host=HOST, port=PORT):
|
||||
r = requests.post('http://{}:{}/set_cfg'.format(host, port), config)
|
||||
if(r.status_code == requests.codes.ok):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def get_frame_from_http(host=HOST, port=PORT):
|
||||
r = requests.get('http://{}:{}/getdeep'.format(host, port))
|
||||
if(r.status_code == requests.codes.ok):
|
||||
# print('Get deep image')
|
||||
deepimg = r.content
|
||||
# print('Length={}'.format(len(deepimg)))
|
||||
(frameid, stamp_msec) = struct.unpack('<QQ', deepimg[0:8+8])
|
||||
# print((frameid, stamp_msec/1000))
|
||||
return deepimg
|
||||
|
||||
|
||||
def show_frame(fig, frame_data: bytes):
|
||||
config = frame_config_decode(frame_data[16:16+12])
|
||||
frame_bytes = frame_payload_decode(frame_data[16+12:], config)
|
||||
|
||||
depth = np.frombuffer(frame_bytes[0], 'uint16' if 0 == config[1] else 'uint8').reshape(
|
||||
240, 320) if frame_bytes[0] else None
|
||||
|
||||
ir = np.frombuffer(frame_bytes[1], 'uint16' if 0 == config[3] else 'uint8').reshape(
|
||||
240, 320) if frame_bytes[1] else None
|
||||
|
||||
status = np.frombuffer(frame_bytes[2], 'uint16' if 0 == config[4] else 'uint8').reshape(
|
||||
240, 320) if frame_bytes[2] else None
|
||||
|
||||
rgb = np.frombuffer(frame_bytes[3], 'uint8').reshape(
|
||||
(480, 640, 3)) if frame_bytes[3] else None
|
||||
|
||||
ax1 = fig.add_subplot(221)
|
||||
if not depth is None:
|
||||
# center_dis = depth[240//2, 320//2]
|
||||
# if 0 == config[1]:
|
||||
# print("%f mm" % (center_dis/4))
|
||||
# else:
|
||||
# print("%f mm" % ((center_dis/5.1) ** 2))
|
||||
# depth = depth.copy()
|
||||
|
||||
# l,r= 200,5000
|
||||
# depth_f = ((depth.astype('float64') - l) * (65535 / (r - l)))
|
||||
# depth_f[np.where(depth_f < 0)] = 0
|
||||
# depth_f[np.where(depth_f > 65535)] = 65535
|
||||
|
||||
# depth = depth_f.astype(depth.dtype)
|
||||
|
||||
# depth[240//2, 320//2 - 5:320//2+5] = 0x00
|
||||
# depth[240//2-5:240//2+5, 320//2] = 0x00
|
||||
ax1.imshow(depth, cmap='jet_r')
|
||||
ax2 = fig.add_subplot(222)
|
||||
if not ir is None:
|
||||
ax2.imshow(ir, cmap='gray')
|
||||
ax3 = fig.add_subplot(223)
|
||||
if not status is None:
|
||||
ax3.imshow(status)
|
||||
ax4 = fig.add_subplot(224)
|
||||
if not rgb is None:
|
||||
ax4.imshow(rgb)
|
||||
|
||||
|
||||
if post_encode_config(frame_config_encode(1, 1, 255, 0, 2, 7, 1, 0, 0)):
|
||||
# 打开交互模式
|
||||
plt.ion()
|
||||
figsize = (12, 12)
|
||||
fig = plt.figure('2D frame', figsize=figsize)
|
||||
while True:
|
||||
p = get_frame_from_http()
|
||||
show_frame(fig, p)
|
||||
# 停顿时间
|
||||
plt.pause(0.001)
|
||||
# 清除当前画布
|
||||
fig.clf()
|
||||
|
||||
plt.ioff()
|
||||
```
|
||||
|
||||
## calvolumespy
|
||||
|
||||
```python
|
||||
from PIL import Image, ImageDraw
|
||||
import requests
|
||||
import matplotlib.pyplot as plt
|
||||
import struct
|
||||
import numpy as np
|
||||
import cv2
|
||||
|
||||
HOST = '192.168.233.1'
|
||||
PORT = 80
|
||||
|
||||
def depth2xyz(xp, yp, z, fx, fy, cx, cy, depth_scale=1000):
|
||||
# h,w=np.mgrid[0:depth_map.shape[0],0:depth_map.shape[1]]
|
||||
z = z/depth_scale
|
||||
x = (xp-cx)*z/fx
|
||||
y = (yp-cy)*z/fy
|
||||
# xyz=np.dstack((x,y,z))
|
||||
# xyz=cv2.rgbd.depthTo3d(depth_map,depth_cam_matrix)
|
||||
return [x, y, z]
|
||||
|
||||
|
||||
def polygon_area(polygon):
|
||||
area = 0
|
||||
q = polygon[-1]
|
||||
for p in polygon:
|
||||
area += p[0] * q[1] - p[1] * q[0]
|
||||
q = p
|
||||
return abs(area) / 2.0
|
||||
|
||||
def get_lenscoeff(host=HOST, port=PORT):
|
||||
r = requests.get('http://{}:{}/getinfo'.format(host, port))
|
||||
if(r.status_code == requests.codes.ok):
|
||||
lenscoeff_bin = r.content
|
||||
(_fx,_fy,_cx,_cy) = struct.unpack('<ffff', lenscoeff_bin[41:41+4*4])
|
||||
# print((frameid, stamp_msec/1000))
|
||||
return (_fx,_fy,_cx,_cy)
|
||||
|
||||
|
||||
diff_low = 30
|
||||
diff_high = 500
|
||||
fx = 2.265142e+02
|
||||
fy = 2.278584e+02
|
||||
cx = 1.637246e+02 # cx
|
||||
cy = 1.233738e+02 # cy
|
||||
|
||||
(fx,fy,cx,cy) = get_lenscoeff()
|
||||
|
||||
def cal_volume(d_bk, d_bg):
|
||||
img_h, img_w = d_bk.shape[0], d_bk.shape[1]
|
||||
d_bk = d_bk.astype(np.float32) # cvt to mm
|
||||
d_bg = d_bg.astype(np.float32)
|
||||
|
||||
diff = (d_bg-d_bk).astype(np.int16)
|
||||
diff1 = diff.copy()
|
||||
diff1 = np.where(diff1 < diff_low, 0, diff1)
|
||||
diff1 = np.where(diff1 > diff_high, 0, diff1)
|
||||
diff1 = (np.where(diff1 > 0, 1, 0)*255).astype(np.uint8)
|
||||
# plt.imshow(diff1)
|
||||
|
||||
# print(d_bk.shape) (240, 320)
|
||||
output = np.zeros((img_h, img_w, 3), np.uint8)
|
||||
|
||||
num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(
|
||||
diff1, connectivity=8)
|
||||
# print('num_labels = ',num_labels)
|
||||
# 连通域的信息：对应各个轮廓的x、y、width、height和面积
|
||||
# print('stats = ',stats)
|
||||
res = list()
|
||||
max_stats = list()
|
||||
for i in range(5):
|
||||
max_label = 1+np.argmax(stats[1:, 4])
|
||||
# print('stats[max_label] = ', stats[max_label])
|
||||
if i > 0 and stats[max_label][4] < 700:
|
||||
break
|
||||
max_stat = stats[max_label]
|
||||
max_stats.append(max_stat)
|
||||
stats[max_label][4] = 0
|
||||
|
||||
mask = (labels == max_label)
|
||||
# (np.random.rand(3)*255).astype(np.uint8)
|
||||
output[:, :, :][mask] = [200, 0, 0]
|
||||
# plt.imshow(output)
|
||||
|
||||
# kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
|
||||
# eroded = cv2.erode(output, kernel)
|
||||
# dilated = cv2.dilate(output, kernel)
|
||||
|
||||
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
|
||||
# output = dilated
|
||||
output=cv2.morphologyEx(output, cv2.MORPH_OPEN, kernel)
|
||||
output=cv2.morphologyEx(output, cv2.MORPH_CLOSE, kernel)
|
||||
|
||||
|
||||
volumes = []
|
||||
# points = []
|
||||
# areas = []
|
||||
for yp in range(img_h):
|
||||
for xp in range(img_w):
|
||||
if mask[yp, xp]:
|
||||
x1, y1, z1 = depth2xyz(
|
||||
xp, yp, d_bk[yp, xp], fx, fy, cx, cy, depth_scale=1)
|
||||
x0, y0, z0 = depth2xyz(
|
||||
xp, yp, d_bg[yp, xp], fx, fy, cx, cy, depth_scale=1)
|
||||
|
||||
x = xp-1
|
||||
if x < 0:
|
||||
x = 0
|
||||
y = yp
|
||||
xl, yl, zl = depth2xyz(
|
||||
x, y, d_bk[y, x], fx, fy, cx, cy, depth_scale=1)
|
||||
x = xp+1
|
||||
if x >= img_w:
|
||||
x = img_w-1
|
||||
y = yp
|
||||
xr, yr, zr = depth2xyz(
|
||||
x, y, d_bk[y, x], fx, fy, cx, cy, depth_scale=1)
|
||||
x = xp
|
||||
y = yp-1
|
||||
if y < 0:
|
||||
y = 0
|
||||
xt, yt, zt = depth2xyz(
|
||||
x, y, d_bk[y, x], fx, fy, cx, cy, depth_scale=1)
|
||||
x = xp
|
||||
y = yp+1
|
||||
if y >= img_h:
|
||||
y = img_h-1
|
||||
xb, yb, zb = depth2xyz(
|
||||
x, y, d_bk[y, x], fx, fy, cx, cy, depth_scale=1)
|
||||
|
||||
area_a = polygon_area(
|
||||
[[xt, yt], [xl, yl], [xb, yb], [xr, yr]])/2
|
||||
|
||||
dz = z0-z1
|
||||
dx = z1/fx
|
||||
dy = z1/fy
|
||||
area_b = dx*dy*2/2
|
||||
|
||||
area = (area_a+area_b)/2 # avg get better acc
|
||||
volume = area*dz
|
||||
# areas.append(area)
|
||||
volumes.append(volume)
|
||||
# points.append((x1, y1, dz))
|
||||
# areas = np.array(areas)
|
||||
volumes = np.array(volumes)
|
||||
# points = np.array(points)
|
||||
|
||||
res.append("{}:{} cm3".format(i, int(np.sum(volumes)/1000)))
|
||||
# print(res)
|
||||
|
||||
img_pil = Image.fromarray(output)
|
||||
draw = ImageDraw.Draw(img_pil)
|
||||
for i in range(len(max_stats)):
|
||||
max_stat = max_stats[i]
|
||||
draw.rectangle([(max_stat[0], max_stat[1]),
|
||||
(max_stat[0] + max_stat[2], max_stat[1] + max_stat[3])], outline="red")
|
||||
draw.text((max_stat[0], max_stat[1]), res[i], fill=(255, 255, 255))
|
||||
output = np.array(img_pil)
|
||||
|
||||
return output
|
||||
|
||||
|
||||
def frame_config_decode(frame_config):
|
||||
'''
|
||||
@frame_config bytes
|
||||
|
||||
@return fields, tuple (trigger_mode, deep_mode, deep_shift, ir_mode, status_mode, status_mask, rgb_mode, rgb_res, expose_time)
|
||||
'''
|
||||
return struct.unpack("<BBBBBBBBi", frame_config)
|
||||
|
||||
|
||||
def frame_config_encode(trigger_mode=1, deep_mode=1, deep_shift=255, ir_mode=1, status_mode=2, status_mask=7, rgb_mode=1, rgb_res=0, expose_time=0):
|
||||
return struct.pack("<BBBBBBBBi",
|
||||
trigger_mode, deep_mode, deep_shift, ir_mode, status_mode, status_mask, rgb_mode, rgb_res, expose_time)
|
||||
|
||||
|
||||
def frame_payload_decode(frame_data: bytes, with_config: tuple):
|
||||
deep_data_size, rgb_data_size = struct.unpack("<ii", frame_data[:8])
|
||||
frame_payload = frame_data[8:]
|
||||
# 0:16bit 1:8bit, resolution: 320*240
|
||||
deepth_size = (320*240*2) >> with_config[1]
|
||||
deepth_img = struct.unpack("<%us" % deepth_size, frame_payload[:deepth_size])[
|
||||
0] if 0 != deepth_size else None
|
||||
frame_payload = frame_payload[deepth_size:]
|
||||
|
||||
# 0:16bit 1:8bit, resolution: 320*240
|
||||
ir_size = (320*240*2) >> with_config[3]
|
||||
ir_img = struct.unpack("<%us" % ir_size, frame_payload[:ir_size])[
|
||||
0] if 0 != ir_size else None
|
||||
frame_payload = frame_payload[ir_size:]
|
||||
|
||||
status_size = (320*240//8) * (16 if 0 == with_config[4] else
|
||||
2 if 1 == with_config[4] else 8 if 2 == with_config[4] else 1)
|
||||
status_img = struct.unpack("<%us" % status_size, frame_payload[:status_size])[
|
||||
0] if 0 != status_size else None
|
||||
frame_payload = frame_payload[status_size:]
|
||||
|
||||
assert(deep_data_size == deepth_size+ir_size+status_size)
|
||||
|
||||
rgb_size = len(frame_payload)
|
||||
assert(rgb_data_size == rgb_size)
|
||||
rgb_img = struct.unpack("<%us" % rgb_size, frame_payload[:rgb_size])[
|
||||
0] if 0 != rgb_size else None
|
||||
|
||||
if (not rgb_img is None) and (1 == with_config[6]):
|
||||
jpeg = cv2.imdecode(np.frombuffer(
|
||||
rgb_img, 'uint8', rgb_size), cv2.IMREAD_COLOR)
|
||||
if not jpeg is None:
|
||||
rgb = cv2.cvtColor(jpeg, cv2.COLOR_BGR2RGB)
|
||||
rgb_img = rgb.tobytes()
|
||||
else:
|
||||
rgb_img = None
|
||||
|
||||
return (deepth_img, ir_img, status_img, rgb_img)
|
||||
|
||||
|
||||
def post_encode_config(config=frame_config_encode(), host=HOST, port=PORT):
|
||||
r = requests.post('http://{}:{}/set_cfg'.format(host, port), config)
|
||||
if(r.status_code == requests.codes.ok):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def get_frame_from_http(host=HOST, port=PORT):
|
||||
r = requests.get('http://{}:{}/getdeep'.format(host, port))
|
||||
if(r.status_code == requests.codes.ok):
|
||||
# print('Get deep image')
|
||||
deepimg = r.content
|
||||
# print('Length={}'.format(len(deepimg)))
|
||||
(frameid, stamp_msec) = struct.unpack('<QQ', deepimg[0:8+8])
|
||||
# print((frameid, stamp_msec/1000))
|
||||
return deepimg
|
||||
|
||||
|
||||
def show_frame(fig, frame_data: bytes):
|
||||
config = frame_config_decode(frame_data[16:16+12])
|
||||
frame_bytes = frame_payload_decode(frame_data[16+12:], config)
|
||||
|
||||
depth = np.frombuffer(frame_bytes[0], 'uint16' if 0 == config[1] else 'uint8').reshape(
|
||||
240, 320) if frame_bytes[0] else None
|
||||
|
||||
# ir = np.frombuffer(frame_bytes[1], 'uint16' if 0 == config[3] else 'uint8').reshape(
|
||||
# 240, 320) if frame_bytes[1] else None
|
||||
|
||||
# status = np.frombuffer(frame_bytes[2], 'uint16' if 0 == config[4] else 'uint8').reshape(
|
||||
# 240, 320) if frame_bytes[2] else None
|
||||
|
||||
rgb = np.frombuffer(frame_bytes[3], 'uint8').reshape(
|
||||
(480, 640, 3)) if frame_bytes[3] else None
|
||||
|
||||
ax1 = fig.add_subplot(122)
|
||||
if not depth is None:
|
||||
# center_dis = depth[240//2, 320//2]
|
||||
# if 0 == config[1]:
|
||||
# print("%f mm" % (center_dis/4))
|
||||
# else:
|
||||
# print("%f mm" % ((center_dis/5.1) ** 2))
|
||||
# depth = depth.copy()
|
||||
|
||||
# l,r= 200,5000
|
||||
# depth_f = ((depth.astype('float64') - l) * (65535 / (r - l)))
|
||||
# depth_f[np.where(depth_f < 0)] = 0
|
||||
# depth_f[np.where(depth_f > 65535)] = 65535
|
||||
|
||||
# depth = depth_f.astype(depth.dtype)
|
||||
|
||||
# depth[240//2, 320//2 - 5:320//2+5] = 0x00
|
||||
# depth[240//2-5:240//2+5, 320//2] = 0x00
|
||||
|
||||
if not UPDATE_BG[1] is None:
|
||||
ax1.imshow(cal_volume(depth, UPDATE_BG[1]))
|
||||
else:
|
||||
ax1.imshow(depth)
|
||||
|
||||
if UPDATE_BG[0]:
|
||||
UPDATE_BG[1] = depth
|
||||
|
||||
# ax2 = fig.add_subplot(222)
|
||||
# if not ir is None:
|
||||
# ax2.imshow(ir, cmap='gray')
|
||||
# ax3 = fig.add_subplot(223)
|
||||
# if not status is None:
|
||||
# ax3.imshow(status)
|
||||
ax4 = fig.add_subplot(121)
|
||||
if not rgb is None:
|
||||
ax4.imshow(rgb)
|
||||
|
||||
|
||||
UPDATE_BG = [False, None]
|
||||
|
||||
if post_encode_config(frame_config_encode(1, 0, 255, 0, 2, 7, 1, 0, 0)):
|
||||
# 打开交互模式
|
||||
def on_key_press(event):
|
||||
if event.key == ' ':
|
||||
UPDATE_BG[0] = True
|
||||
elif event.key == 'c':
|
||||
UPDATE_BG[1] = None
|
||||
|
||||
plt.ion()
|
||||
figsize = (12, 12)
|
||||
fig = plt.figure('2D frame', figsize=figsize)
|
||||
fig.canvas.mpl_connect('key_press_event', on_key_press)
|
||||
|
||||
print("按下空格键更新背景图，按下c键清空背景图")
|
||||
while True:
|
||||
p = get_frame_from_http()
|
||||
show_frame(fig, p)
|
||||
if UPDATE_BG[0]:
|
||||
UPDATE_BG[0] = False
|
||||
print("update bg success!")
|
||||
# 停顿时间
|
||||
plt.pause(0.001)
|
||||
# 清除当前画布
|
||||
fig.clf()
|
||||
|
||||
plt.ioff()
|
||||
```
|
||||
|
||||
## calibratepy
|
||||
|
||||
```python
|
||||
# -*- coding:utf-8 -*-
|
||||
|
||||
import requests
|
||||
import struct
|
||||
import shutil
|
||||
import logging
|
||||
|
||||
|
||||
from PIL import Image
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import cv2
|
||||
import os
|
||||
import json
|
||||
|
||||
# 默认的warning级别，只输出warning以上的
|
||||
# 使用basicConfig()来指定日志级别和相关信息
|
||||
|
||||
logging.basicConfig(level=logging.INFO # 设置日志输出格式
|
||||
, filename="cali.log" # log日志输出的文件位置和文件名
|
||||
, filemode="w" # 文件的写入格式，w为重新写入文件，默认是追加
|
||||
# 日志输出的格式
|
||||
# -8表示占位符，让输出左对齐，输出长度都为8位
|
||||
# 时间输出的格式
|
||||
, format="%(asctime)s - %(name)s - %(levelname)-9s - %(filename)-8s : %(lineno)s line - %(message)s", datefmt="%Y-%m-%d %H:%M:%S"
|
||||
)
|
||||
|
||||
logging.info("Starting Cali...")
|
||||
|
||||
HOST = '192.168.233.233'
|
||||
PORT = 80
|
||||
|
||||
rgb_dir = 'RGB/'
|
||||
tof_dir = 'TOF/'
|
||||
|
||||
|
||||
logging.info("Clear {} and {} dir...".format(rgb_dir, tof_dir))
|
||||
|
||||
if os.path.exists(rgb_dir):
|
||||
shutil.rmtree(rgb_dir)
|
||||
|
||||
if os.path.exists(tof_dir):
|
||||
shutil.rmtree(tof_dir)
|
||||
|
||||
os.makedirs(rgb_dir)
|
||||
os.makedirs(tof_dir)
|
||||
|
||||
logging.info("Clear directories success!")
|
||||
|
||||
IMAGECOUNT = 0
|
||||
|
||||
|
||||
IMG_WIDTH = 800 # Do not change this
|
||||
IMG_HEIGHT = 600 # and this
|
||||
patternSize = (8, 6) # modifly this for your Chessboard Corners
|
||||
squareSize = 28.3 # modifly this for chessboard box size (in mm)
|
||||
|
||||
|
||||
def frame_config_decode(frame_config):
|
||||
'''
|
||||
@frame_config bytes
|
||||
|
||||
@return fields, tuple (trigger_mode, deep_mode, deep_shift, ir_mode, status_mode, status_mask, rgb_mode, rgb_res, expose_time)
|
||||
'''
|
||||
return struct.unpack("<BBBBBBBBi", frame_config)
|
||||
|
||||
|
||||
def frame_config_encode(trigger_mode=1, deep_mode=1, deep_shift=255, ir_mode=1, status_mode=2, status_mask=7, rgb_mode=1, rgb_res=0, expose_time=0):
|
||||
return struct.pack("<BBBBBBBBi",
|
||||
trigger_mode, deep_mode, deep_shift, ir_mode, status_mode, status_mask, rgb_mode, rgb_res, expose_time)
|
||||
|
||||
|
||||
def frame_payload_decode(frame_data: bytes, with_config: tuple):
|
||||
deep_data_size, rgb_data_size = struct.unpack("<ii", frame_data[:8])
|
||||
frame_payload = frame_data[8:]
|
||||
# 0:16bit 1:8bit, resolution: 320*240
|
||||
deepth_size = (320*240*2) >> with_config[1]
|
||||
deepth_img = struct.unpack("<%us" % deepth_size, frame_payload[:deepth_size])[
|
||||
0] if 0 != deepth_size else None
|
||||
frame_payload = frame_payload[deepth_size:]
|
||||
|
||||
# 0:16bit 1:8bit, resolution: 320*240
|
||||
ir_size = (320*240*2) >> with_config[3]
|
||||
ir_img = struct.unpack("<%us" % ir_size, frame_payload[:ir_size])[
|
||||
0] if 0 != ir_size else None
|
||||
frame_payload = frame_payload[ir_size:]
|
||||
|
||||
status_size = (320*240//8) * (16 if 0 == with_config[4] else
|
||||
2 if 1 == with_config[4] else 8 if 2 == with_config[4] else 1)
|
||||
status_img = struct.unpack("<%us" % status_size, frame_payload[:status_size])[
|
||||
0] if 0 != status_size else None
|
||||
frame_payload = frame_payload[status_size:]
|
||||
|
||||
assert(deep_data_size == deepth_size+ir_size+status_size)
|
||||
|
||||
rgb_size = len(frame_payload)
|
||||
assert(rgb_data_size == rgb_size)
|
||||
rgb_img = struct.unpack("<%us" % rgb_size, frame_payload[:rgb_size])[
|
||||
0] if 0 != rgb_size else None
|
||||
|
||||
if (not rgb_img is None) and (1 == with_config[6]):
|
||||
jpeg = cv2.imdecode(np.frombuffer(
|
||||
rgb_img, 'uint8', rgb_size), cv2.IMREAD_COLOR)
|
||||
if not jpeg is None:
|
||||
rgb = cv2.cvtColor(jpeg, cv2.COLOR_BGR2RGB)
|
||||
rgb_img = rgb.tobytes()
|
||||
else:
|
||||
rgb_img = None
|
||||
|
||||
return (deepth_img, ir_img, status_img, rgb_img)
|
||||
|
||||
|
||||
def post_encode_config(config=frame_config_encode(), host=HOST, port=PORT):
|
||||
r = requests.post('http://{}:{}/set_cfg'.format(host, port), config)
|
||||
if(r.status_code == requests.codes.ok):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def get_frame_from_http(host=HOST, port=PORT):
|
||||
r = requests.get('http://{}:{}/getdeep'.format(host, port))
|
||||
if(r.status_code == requests.codes.ok):
|
||||
# print('Get deep image')
|
||||
deepimg = r.content
|
||||
# print('Length={}'.format(len(deepimg)))
|
||||
(frameid, stamp_msec) = struct.unpack('<QQ', deepimg[0:8+8])
|
||||
# print((frameid, stamp_msec/1000))
|
||||
return deepimg
|
||||
|
||||
|
||||
def show_frame(fig, frame_data: bytes, save_flag=False):
|
||||
config = frame_config_decode(frame_data[16:16+12])
|
||||
frame_bytes = frame_payload_decode(frame_data[16+12:], config)
|
||||
|
||||
depth = np.frombuffer(frame_bytes[0], 'uint16' if 0 == config[1] else 'uint8').reshape(
|
||||
240, 320) if frame_bytes[0] else None
|
||||
|
||||
ir = np.frombuffer(frame_bytes[1], 'uint16' if 0 == config[3] else 'uint8').reshape(
|
||||
240, 320) if frame_bytes[1] else None
|
||||
|
||||
status = np.frombuffer(frame_bytes[2], 'uint16' if 0 == config[4] else 'uint8').reshape(
|
||||
240, 320) if frame_bytes[2] else None
|
||||
|
||||
rgb = np.frombuffer(frame_bytes[3], 'uint8').reshape(
|
||||
(480, 640, 3)) if frame_bytes[3] else None
|
||||
|
||||
ax1 = fig.add_subplot(221)
|
||||
if not depth is None:
|
||||
# center_dis = depth[240//2, 320//2]
|
||||
# if 0 == config[1]:
|
||||
# print("%f mm" % (center_dis/4))
|
||||
# else:
|
||||
# print("%f mm" % ((center_dis/5.1) ** 2))
|
||||
# depth = depth.copy()
|
||||
|
||||
# l,r= 200,5000
|
||||
# depth_f = ((depth.astype('float64') - l) * (65535 / (r - l)))
|
||||
# depth_f[np.where(depth_f < 0)] = 0
|
||||
# depth_f[np.where(depth_f > 65535)] = 65535
|
||||
|
||||
# depth = depth_f.astype(depth.dtype)
|
||||
|
||||
# depth[240//2, 320//2 - 5:320//2+5] = 0x00
|
||||
# depth[240//2-5:240//2+5, 320//2] = 0x00
|
||||
ax1.imshow(depth, cmap='jet_r')
|
||||
ax2 = fig.add_subplot(222)
|
||||
if not ir is None:
|
||||
ax2.imshow(ir, cmap='gray')
|
||||
if save_flag:
|
||||
Image.fromarray(ir).resize((800, 600)).save(
|
||||
tof_dir+'IR-{}.png'.format(IMAGECOUNT))
|
||||
ax3 = fig.add_subplot(223)
|
||||
if not status is None:
|
||||
ax3.imshow(status)
|
||||
ax4 = fig.add_subplot(224)
|
||||
if not rgb is None:
|
||||
ax4.imshow(rgb)
|
||||
if save_flag:
|
||||
Image.fromarray(rgb).resize((800, 600)).save(
|
||||
rgb_dir+'RGB-{}.png'.format(IMAGECOUNT))
|
||||
|
||||
|
||||
def main():
|
||||
global IMAGECOUNT
|
||||
if post_encode_config(frame_config_encode(1, 1, 255, 0, 2, 7, 1, 0, 0)):
|
||||
KEEP_STREAM = [True]
|
||||
save_once = [False]
|
||||
|
||||
def on_key_press(event):
|
||||
if event.key == ' ':
|
||||
save_once[0] = True
|
||||
elif event.key == 'y':
|
||||
KEEP_STREAM[0] = False
|
||||
|
||||
# 打开交互模式
|
||||
plt.ion()
|
||||
figsize = (12, 12)
|
||||
fig = plt.figure('2D frame', figsize=figsize)
|
||||
fig.canvas.mpl_connect('key_press_event', on_key_press)
|
||||
|
||||
print("按下空格键抓取图片，一次一张，保持稳定后才按，请不要长按连续拍摄，否则程序会识别出错退出，从而需要重新开始！")
|
||||
print("抓取不同角度图片50张左右，之后按y键程序将进入下一阶段，自动完成等待正常退出即可，图片过少程序也会识别出错退出，从而需要重新开始！")
|
||||
|
||||
while KEEP_STREAM[0]:
|
||||
p = get_frame_from_http()
|
||||
|
||||
show_frame(fig, p, save_once)
|
||||
if save_once[0]:
|
||||
save_once[0] = False
|
||||
IMAGECOUNT += 1
|
||||
tmp = "saved {} pictures".format(IMAGECOUNT)
|
||||
logging.info(tmp)
|
||||
print(tmp)
|
||||
# 停顿时间
|
||||
plt.pause(0.001)
|
||||
# 清除当前画布
|
||||
fig.clf()
|
||||
|
||||
plt.ioff()
|
||||
|
||||
rgblist = [a for a in os.listdir(
|
||||
rgb_dir) if a[-3:] == 'png' or a[-3:] == 'jpg']
|
||||
toflist = [a for a in os.listdir(
|
||||
tof_dir) if a[-3:] == 'png' or a[-3:] == 'jpg']
|
||||
if(len(rgblist) != len(toflist)):
|
||||
print("error pic count not match")
|
||||
else:
|
||||
print("there are ", len(rgblist), ' pairs')
|
||||
# for r,t in zip(rgblist,toflist):
|
||||
# print(r,t)
|
||||
rgbimg = []
|
||||
tofimg = []
|
||||
for r, t in zip(rgblist, toflist):
|
||||
rimg = cv2.imread(rgb_dir+r)
|
||||
rimg = cv2.cvtColor(rimg, cv2.COLOR_BGR2RGB)
|
||||
timg = cv2.imread(tof_dir+t) # , cv2.IMREAD_GRAYSCALE)
|
||||
rimg = cv2.resize(rimg, (IMG_WIDTH, IMG_HEIGHT))
|
||||
timg = cv2.resize(timg, (IMG_WIDTH, IMG_HEIGHT))
|
||||
rgbimg.append(rimg)
|
||||
tofimg.append(timg)
|
||||
|
||||
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.001)
|
||||
# pt_found_t,ptst = cv2.findChessboardCorners(tofimg[0], patternSize);
|
||||
# print(pt_found_t,ptst)
|
||||
# PLT.figure()
|
||||
# PLT.imshow(tofimg[0])
|
||||
# PLT.show()
|
||||
reject_nums = 0
|
||||
imgp_rgb = []
|
||||
imgp_tof = []
|
||||
for r, t in zip(rgbimg, tofimg):
|
||||
pt_found_r, ptsr = cv2.findChessboardCornersSB(
|
||||
r, patternSize, flags=cv2.CALIB_CB_NORMALIZE_IMAGE)
|
||||
pt_found_t, ptst = cv2.findChessboardCornersSB(
|
||||
t, patternSize, flags=cv2.CALIB_CB_NORMALIZE_IMAGE)
|
||||
if(not (pt_found_r and pt_found_t)):
|
||||
reject_nums += 1
|
||||
# print(pt_found_r,pt_found_t)
|
||||
continue
|
||||
ptsr = cv2.cornerSubPix(cv2.cvtColor(
|
||||
r, cv2.COLOR_RGB2GRAY), ptsr, (3, 3), (-1, -1), criteria)
|
||||
ptst = cv2.cornerSubPix(cv2.cvtColor(
|
||||
t, cv2.COLOR_RGB2GRAY), ptst, (3, 3), (-1, -1), criteria)
|
||||
imgp_rgb.append(ptsr)
|
||||
imgp_tof.append(ptst)
|
||||
perview1 = cv2.drawChessboardCorners(r, patternSize, ptsr, pt_found_r)
|
||||
perview2 = cv2.drawChessboardCorners(t, patternSize, ptst, pt_found_t)
|
||||
# PLT.figure()
|
||||
# PLT.imshow(perview1)
|
||||
# PLT.show()
|
||||
# PLT.figure()
|
||||
# PLT.imshow(perview2)
|
||||
# PLT.show()
|
||||
tmp = "reject: {}, total: {}".format(reject_nums, len(rgbimg))
|
||||
logging.warning(tmp)
|
||||
print(tmp)
|
||||
|
||||
if reject_nums == len(rgbimg):
|
||||
print("失败，请重新运行该程序，请保持标定板所有方格一直在画面中!")
|
||||
return None
|
||||
|
||||
xi = np.linspace(0, patternSize[0]-1, patternSize[0])
|
||||
yi = np.linspace(0, patternSize[1]-1, patternSize[1])
|
||||
zpos = np.zeros((patternSize[1], patternSize[0]))
|
||||
boardpos = np.array(np.meshgrid(xi, yi))*squareSize
|
||||
boardpos = np.concatenate(([boardpos[0]], [boardpos[1]], [zpos]))
|
||||
boardpos = np.transpose(boardpos, (1, 2, 0))
|
||||
# print(boardpos)
|
||||
cameraMatrix1 = np.zeros((3, 4))
|
||||
cameraMatrix2 = np.zeros((3, 4))
|
||||
distCoeffs1 = np.zeros((1, 5))
|
||||
distCoeffs2 = np.zeros((1, 5))
|
||||
R = np.zeros((3, 3)),
|
||||
T = np.zeros((3, 1)),
|
||||
# cbs=np.array([boardpos]*len(imgp_rgb))
|
||||
# cbs=cbs.reshape((cbs.shape[0],cbs.shape[1]*cbs.shape[2],cbs.shape[3]))
|
||||
cbs = np.array([boardpos.astype(np.float32)]*len(imgp_rgb))
|
||||
cbs = cbs.reshape((cbs.shape[0], cbs.shape[1]*cbs.shape[2], cbs.shape[3]))
|
||||
# print(cbs)
|
||||
ret, cameraMatrix1, distCoeffs1, rvecs, tvecs = cv2.calibrateCamera(
|
||||
cbs, imgp_rgb, (IMG_WIDTH, IMG_HEIGHT), cameraMatrix1, distCoeffs1)
|
||||
ret, cameraMatrix2, distCoeffs2, rvecs, tvecs = cv2.calibrateCamera(
|
||||
cbs, imgp_tof, (IMG_WIDTH, IMG_HEIGHT), cameraMatrix2, distCoeffs2)
|
||||
flags = cv2.CALIB_FIX_INTRINSIC+cv2.CALIB_USE_INTRINSIC_GUESS + \
|
||||
cv2.CALIB_FIX_PRINCIPAL_POINT+cv2.CALIB_FIX_FOCAL_LENGTH+cv2.CALIB_FIX_ASPECT_RATIO
|
||||
ret, K1, D1, K2, D2, R, T, E, F = cv2.stereoCalibrate(
|
||||
cbs, imgp_rgb, imgp_tof, cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, (IMG_WIDTH, IMG_HEIGHT), flags=flags)
|
||||
print("ret", ret)
|
||||
print("K1", K1)
|
||||
print("D1", D1)
|
||||
print("K2", K2)
|
||||
print("D2", D2)
|
||||
print("R", R)
|
||||
print("T", T)
|
||||
print("E", E)
|
||||
print("F", F)
|
||||
param = {
|
||||
"R_Matrix_data": R.reshape(-1).tolist(),
|
||||
"T_Vec_data": T.reshape(-1).tolist(),
|
||||
"Camera_Matrix_data": cameraMatrix1.reshape(-1).tolist(),
|
||||
"Distortion_Parm_data": distCoeffs1.reshape(-1).tolist(),
|
||||
}
|
||||
|
||||
j = json.dumps(param, indent=4)
|
||||
logging.info(j)
|
||||
print(j)
|
||||
with open('CameraParms.json', 'w') as fp:
|
||||
fp.write(j)
|
||||
fp.close()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
```
|
||||
@@ -52,11 +52,11 @@ MetaSense-A010 是由 Sipeed 所推出的一款由 BL702 + 炬佑 100x100 TOF
|
||||
|
||||
**COMTOOL 软件包**
|
||||
Windows 系统连接：下载压缩包后解压安装即可。
|
||||
Linux 系统：不提供软件压缩包，需用户自行编译。
|
||||
Linux 系统：不提供软件压缩包，需用户自行编译 [获取跳转](https://github.com/sipeed/COMTool)。
|
||||
注意：Win 7 及以下系统需装驱动，可自行前往 FTDI 官网下载。
|
||||
|
||||
### 上电互动预览
|
||||
将设备通电后，可在设备上自带 LCD 屏实时预览 color map 后的深度伪彩图
|
||||
将设备通电后，可在设备上自带 LCD 屏实时预览 color map 后的深度伪彩图。
|
||||
|
||||
|
||||
### PC 预览和微调
|
||||
@@ -84,45 +84,33 @@ COMTOOL 上位机的配置控件说明
|
||||
- FPS 设置出图帧率（不宜过高，根据对接设备的性能合理设置即可，减小帧率可以减少传输数据量）
|
||||
- Ev 曝光间隙控制（最左代表 AE，其他是固定曝光时间）
|
||||
|
||||
## 案例：远近中物体实拍
|
||||
物体摆放距离形成差异，模组通过捕捉到的深度值的差异显示冷暖色调，更直观了解 `TOF` 技术。
|
||||
|
||||
|
||||
|
||||
## 案例：检测人流
|
||||
高精度，大分辨率的实时监测人流走动的情况并统计。
|
||||
|
||||
高精度，大分辨率的实时监测人流走动的情况统计。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 案例：键盘灯跟随
|
||||
|
||||
实现超酷炫的键盘灯跟随，实时跟踪手部的位置，再根据手部的位置映射键盘灯。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 案例：接入 MCU
|
||||
|
||||
MS-A010 拥有强大的兼容性，可基于串口协议外接 K210 bit 这样的单片机开发板或树莓派之类的 linux 开发板来进行二次开发。
|
||||
[MS-A010 外接 K210 bit 源码获取](./metasense-a010/../code.html#tof_mainpy)
|
||||
|
||||
|
||||
|
||||
获取链接：
|
||||
|
||||
|
||||
## 二次开发：串口协议
|
||||
|
||||
可参考上方的案例：**K210 bit 外接 MCU**
|
||||
|
||||
| AT | |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| +ISP<br>Image Signal Processor | :0: turn ISP off<br>=1: turn ISP on |
|
||||
| +BINN<br>full binning | =1: output 100x100 pixel frame<br>=2: output 50x50 pixel frame<br>=4: output 25x25 pixel frame<br> |
|
||||
| +DISP<br>display mux | =0: all off<br>=1: lcd display on<br>=2: usb display on<br>=3: lcd and usb display on<br>=4: uart display on<br>=5: lcd and uart display on<br>=6: usb and uart display on<br>=7: lcd, usb and uart display on |
|
||||
| +BAUD<br>uart baudrate | =0: 9600<br>=1: 57600<br>=2: 115200<br>=3: 230400<br>=4: 460800<br>=5: 921600<br>=6: 1000000<br>=7: 2000000<br>=8: 3000000 |
|
||||
| +UNIT<br>quantization unit | =0: auto<br>=1-10: quantizated by unit(mm) |
|
||||
| +FPS<br>frame per second | =1-19: set frame per second |
|
||||
| +Save<br>save config | : save current configuration |
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -130,7 +118,7 @@ MS-A010 拥有强大的兼容性，可基于串口协议外接 K210 bit 这样
|
||||
### 接入 ROS1
|
||||
|
||||
**1. 准备工作**
|
||||
首先，准备适用的环境：Linux 系统
|
||||
首先，准备适用的环境：`Linux` 系统
|
||||
可使用虚拟机 `virtual box` 或者 `vmware` 也可安装双系统，安装方法请自行查询。
|
||||
|
||||
**2. 安装运行**
|
||||
@@ -151,13 +139,13 @@ rosrun sipeed_tof_ms_a010 a010_publisher _device:="/dev/ttyUSB0"
|
||||
|
||||
**4. RVIZ2 预览**
|
||||
打开 `rviz2` 后，在界面左下角的 `Add`->`By topic`->`PointCloud2或/depth` ->`Image 添加` ->`Display/Global Options/Fixed Frame` 需要修改成 `tof`，才能正常显示点云，根据添加的内容，左侧会显示 `Image` 而中间则显示点云。
|
||||
|
||||
|
||||
|
||||
|
||||
### 接入 ROS2
|
||||
|
||||
**1. 准备工作**
|
||||
首先，准备适用的环境：Linux 系统
|
||||
首先，准备适用的环境：`Linux` 系统
|
||||
可使用虚拟机 `virtual box` 或者 `vmware` 也可安装双系统，安装方法请自行查询。
|
||||
|
||||
**2. 安装运行**
|
||||
@@ -175,9 +163,9 @@ source install/setup.sh
|
||||
|
||||
**3. RQT 查看帧率**
|
||||
|
||||
|
||||
|
||||
|
||||
**4. RVIZ2 预览**
|
||||
打开 `rviz2` 后，在界面左下角的 `Add`->`By topic`->`PointCloud2或/depth` ->`Image 添加` ->`Display/Global Options/Fixed Frame` 需要修改成 `tof`，才能正常显示点云，根据添加的内容，左侧会显示 `Image` 而中间则显示点云。
|
||||
|
||||
|
||||
|
||||
|
||||
BIN
docs/hardware/zh/metasense/metasense-a075v/assets/tof-15.jpg
Normal file
BIN
docs/hardware/zh/metasense/metasense-a075v/assets/tof-15.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 494 KiB |
|
Before Width: | Height: | Size: 156 KiB After Width: | Height: | Size: 156 KiB |
BIN
docs/hardware/zh/metasense/metasense-a075v/assets/tog-14.jpg
Normal file
BIN
docs/hardware/zh/metasense/metasense-a075v/assets/tog-14.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 81 KiB |
@@ -18,7 +18,9 @@ MS-A075V 是由 Sipeed 所推出的一款具有 RGB 功能的 3D TOF 摄像机
|
||||
## 产品开箱指南
|
||||
### 准备工作
|
||||
|
||||
**接线说明**：
|
||||
**接线说明**
|
||||
|
||||
|
||||
**安装驱动**：上电前一定要确保网络环境中没有使用 192.168.233.0/24 的地址段，MS-A075V 会使用 RNDIS 并设置自己的 ip 地址为 http://192.168.233.1
|
||||
Windows 系统需安装驱动才可正常运行。
|
||||
|
||||
@@ -28,7 +30,7 @@ Windows 系统需安装驱动才可正常运行。
|
||||
### 网页上位机预览
|
||||
|
||||
1. 把设备使用 type-c 线与电脑链接，MS-A075V 的风扇会开始工作，产品正面镜头处就会闪烁红灯。
|
||||
2. 此时可打开浏览器输入 http://192.168.233.233 预览 3D 点云图，上电后有延迟需等待一段时间后，系统和程序才会启动完成。
|
||||
2. 此时可打开浏览器输入 http://192.168.233.1 预览 3D 点云图，上电后有延迟需等待一段时间后，系统和程序才会启动完成。
|
||||
3. 使用网页上位机快速预览 演示图（正面和侧面）：
|
||||
|
||||
<html>
|
||||
@@ -67,6 +69,10 @@ Windows 系统需安装驱动才可正常运行。
|
||||
- **SavePointCloud**：可保存一帧 3D 点云图，保存格式为 pcd ，同样可以通过上述提供的脚本预览。
|
||||
注意：raw 数据可通过开放的接口获取，开发者进行解析即可基于此二次开发，但点云（pointcloud）是基于 raw 数据和相机内参进行计算得到的，无相应接口提供。
|
||||
|
||||
## 案例：远中近点云实拍
|
||||
高精度的映射物品摆放距离的差异，点云图可直观清楚感受到更真实的可视化。
|
||||
|
||||
|
||||
## 案例：避障小车
|
||||
|
||||
模组可搭载小车或无人机来回移动获取障碍物的远近深度值，并通过差异判断画面中是否有障碍物，做出快速反应并精准规避障碍物（例程暂未开源，待整理公开)。
|
||||
@@ -93,10 +99,11 @@ Windows 系统需安装驱动才可正常运行。
|
||||
### 解包推流
|
||||
理解了上述 `python SDK` 数据获取和解码的逻辑后，我们可以尝试进阶版，连续获取解码并调用第三方 `python` 图像库，例如：matplotlib 进行实时显示。而 `toturial.py` 给出了获取一帧数据的逻辑实现，通过 plt 显示并外套循环即可做到实时显示。
|
||||
|
||||
**解包推流**：
|
||||
**解包推流**：`python stream.py` [点我查看stream.py内容](./../metasense-a010/code.html#streampy)
|
||||
**使用方式**：装好所有的依赖包后即可 `python stream.py` 运行。
|
||||
|
||||
|
||||
[233](./../metasense-a010/code.html#calibratepy)
|
||||
|
||||
### 检测体积
|
||||
基于第三方 `python` 包，理解了上述数据获取和解码的逻辑后，再次进阶，不但持续显示多帧并且再通过 SDK 获取相机内参后计算出初略的点云，做累加得到总体积。限制：要求俯视图可以看到除底面外的所有细节
|
||||
@@ -111,7 +118,7 @@ Windows 系统需安装驱动才可正常运行。
|
||||
### 接入 ROS1
|
||||
|
||||
**1. 准备工作**
|
||||
首先，准备适用的环境：Linux 系统
|
||||
首先，准备适用的环境：`Linux` 系统
|
||||
可使用虚拟机 `virtual box` 或者 `vmware` 也可安装双系统，安装方法请自行查询。
|
||||
|
||||
**2. 安装运行**
|
||||
@@ -136,7 +143,7 @@ rosrun sipeed_tof_cpp publisher
|
||||
|
||||
### 接入 ROS2
|
||||
**1. 准备工作**
|
||||
首先，准备适用的环境：Linux 系统
|
||||
首先，准备适用的环境：`Linux` 系统
|
||||
可使用虚拟机 `virtual box` 或者 `vmware` 也可安装双系统，安装方法请自行查询。
|
||||
|
||||
**2. 安装运行**
|
||||
|
||||
@@ -4,9 +4,10 @@ title: MetaSense 系列
|
||||
|
||||
## MetaSense 是什么?
|
||||
|
||||
MetaSense 系列摄像头模组分为 MetaSense-A075V 和 MetaSense-A010 两款产品。
|
||||
- MS-A010 是由 Sipeed 所推出的一款由 BL702 + 炬佑 100x100 TOF 模组所组成的极致性价比的 TOF 3D 传感器模组，最大支持 100x100 的分辨率和 8 位精度，并且带有 240×135 的 LCD 显示屏可实时预览 color map 后的深度图。
|
||||
- 而 MS-A075V 是一款具有 RGB 功能的 3D TOF 摄像机模组，该模组可以实现免驱的即插即用，实现实时彩色 3D 显示。
|
||||
MetaSense 系列产品搭载 TOF 深度摄像头，目前主要有 MetaSense-A010 和 MetaSense-A075VS 两款产品。
|
||||
- MS-A010 是一款由 BL702 + 炬佑 100x100 TOF 模组所组成的极致性价比的 TOF 3D 传感器模组，最大支持 100x100 的分辨率和 8 位精度，并且带有 240×135 的 LCD 显示屏可实时预览 color map 后的深度图。
|
||||
- 而 MS-A075V 是一款具有 RGB 功能的 3D TOF 摄像机模组，该模组可以实现 Linux 免驱的即插即用，实现实时彩色 3D 显示。
|
||||
|
||||
|
||||
<img src="./assets/tof-1.6.jpg" alt="summary" width=100%>
|
||||
|
||||
@@ -15,7 +16,7 @@ MetaSense 系列摄像头模组分为 MetaSense-A075V 和 MetaSense-A010 两款
|
||||
|
||||
|
||||
|
||||
| |<p style="white-space:nowrap">MateSense-010</p > | <p style="white-space:nowrap">MateSense-A075V</p > |
|
||||
| |<p style="white-space:nowrap">MateSense-A010</p > | <p style="white-space:nowrap">MateSense-A075V</p > |
|
||||
| :----------- |:----------------------------------------------- | :------------------------------------------------- |
|
||||
| | |  |
|
||||
| 接口 | 1.25mm 串口连接器 \*1<br>Type-C USB2.0 \*1 | 1.25mm 串口连接器 \*1 <br>Type-C USB2.0 \*1 |
|
||||
@@ -26,7 +27,6 @@ MetaSense 系列摄像头模组分为 MetaSense-A075V 和 MetaSense-A010 两款
|
||||
| 测量范围 |0.2-2.5m | 0.15-1.5m |
|
||||
| 测量精度 |<=1%/cm | <=1%/cm |
|
||||
|
||||
相关交流社区及社群：
|
||||
|
||||
|
||||
|
||||
@@ -34,56 +34,49 @@ MetaSense 系列摄像头模组分为 MetaSense-A075V 和 MetaSense-A010 两款
|
||||
|
||||
|
||||
### 案例：远中近物体实拍
|
||||
|
||||
高精度的映射物品摆放距离的差异，点云图可直观感受到更真实的可视化，
|
||||
|
||||
|
||||
|
||||
### 案例：人流统计
|
||||
可高精度、大分辨率的实时监测人流走动的情况并统计。
|
||||
|
||||
高精度，大分辨率的实时监测人流走动的情况统计。
|
||||
|
||||
|
||||
|
||||
|
||||
### 案例：小车避障
|
||||
|
||||
可搭载于小车移动并判断画面是否有障碍物，模组自带 LCD 屏幕精准显示距离并做出反应规避障碍物。
|
||||
|
||||
|
||||
|
||||
### 案例：键盘灯跟随
|
||||
|
||||
实现超酷炫的键盘灯跟随，实时跟踪手部的位置，再根据手部的位置映射键盘灯。
|
||||
|
||||
|
||||
|
||||
|
||||
### 案例：体积测量
|
||||
|
||||
通过 SDK 获取到的相机内参后计算出初略的点云，做累加得到全部的总体积，达到体积测量的效果。
|
||||
|
||||
|
||||
|
||||
### 案例：外接 MCU
|
||||
### [案例：外接 MCU](./metasense-a010/code.html#tof_mainpy)
|
||||
|
||||
MS-A010 拥有强大的兼容性，基于串口协议，可外接 K210 bit 这样的单片机开发板或树莓派之类的 linux 开发板来进行二次开发
|
||||
|
||||
|
||||
MS-A010 拥有强大的兼容性，基于串口协议的数据传输，可外接 K210 bit 这样的单片机开发板或树莓派之类的 linux 开发板来进行二次开发。
|
||||
|
||||
|
||||
|
||||
### 案例：接入 ROS1 + ROS2
|
||||
|
||||
双支持 ROS 系统，开放 ROS1+ROS2 接入功能包，可快速获得深度数据及深度图。
|
||||
<html>
|
||||
<img src="./assets/../metasense-a075v/assets/tof-11.jpg" width=48%>
|
||||
<img src="./assets/../metasense-a075v/assets/tof-12.jpg" width=48%>
|
||||
<img src="./assets/tof-1.13.jpg" width=49%>
|
||||
<img src="./assets/tof-1.12.jpg" width=49%>
|
||||
</html>
|
||||
|
||||
|
||||
## 快速了解 TOF 技术
|
||||
|
||||
1. TOF: Time of flight（飞行时间）它是一种测距的方法，通过测量超声波/微波/光等信号在发射器和反射器之间的“飞行时间”来计算出两者之间的距离。能够实现 TOF 测距的传感器就是 TOF 传感器。种类较多，使用较多的是通过红外或者激光进行测距的 TOF 传感器。
|
||||
2. RGBD：RGB 摄像头加 TOF 深度摄像头组成的一整个模组。图源自[百度](https://baike.baidu.com/item/TOF/19952376?fr=aladdin)。
|
||||
|
||||
2. 物体摆放距离形成差异，模组通过捕捉到的深度值的差异显示冷暖色调，冷暖色随着对距离的映射发生变化，距离越近色调呈暖调（橘红）而越远色调呈冷调（蓝色）
|
||||
|
||||
|
||||
## 更多
|
||||
关于 MS-010 更详细的资料获取：
|
||||
关于 MS-075 更详细的资料获取：
|
||||
关于 MS-010 更详细的资料获取：[点击跳转](https://wiki.sipeed.com/hardware/zh/metasense/metasense-a010/metasense-a010.html)
|
||||
关于 MS-075 更详细的资料获取：[点击跳转](https://wiki.sipeed.com/hardware/zh/metasense/metasense-a075v/metasense-a075v.html)
|
||||
4
pages/index/en/metasense.md
Normal file
4
pages/index/en/metasense.md
Normal file
@@ -0,0 +1,4 @@
|
||||
---
|
||||
layout: redirect
|
||||
redirect_url: https://wiki.sipeed.com/hardware/en/metasense/index.html
|
||||
---
|
||||
4
pages/index/en/metasense10.md
Normal file
4
pages/index/en/metasense10.md
Normal file
@@ -0,0 +1,4 @@
|
||||
---
|
||||
layout: redirect
|
||||
redirect_url: https://wiki.sipeed.com/hardware/en/metasense/metasense-a010/metasense-a010.html
|
||||
---
|
||||
4
pages/index/en/metasense75.md
Normal file
4
pages/index/en/metasense75.md
Normal file
@@ -0,0 +1,4 @@
|
||||
---
|
||||
layout: redirect
|
||||
redirect_url: https://wiki.sipeed.com/hardware/en/metasense/metasense-a075v/metasense-a075v.html
|
||||
---
|
||||
Reference in New Issue
Block a user