Merge pull request #993 from liangzhuohua/docs/maixcam2-xai-guide

Docs/maixcam2 xai guide
This commit is contained in:
taonyx
2026-07-28 16:59:11 +08:00
committed by GitHub
17 changed files with 715 additions and 0 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 221 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 132 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 104 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 330 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 397 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 436 KiB

View File

@@ -0,0 +1,355 @@
---
title: "MaixCAM2 x AI: Build a Two-Axis Gimbal Tracker with an AI Agent"
---
This page uses a real MaixCAM2 development recording as an example to show how to use an AI Agent to build a UART4 two-axis gimbal red-object tracker with the MaixCAM2 built-in camera. The recording used Codex to call maixpy-skill. To make the setup easier to reproduce on Windows, macOS, and Linux, this page only demonstrates OpenCode for installation and model configuration. The Codex UI shown in some images is only from the recorded session and is not the OpenCode interface.
## Project Goal and Final Result
The goal is to detect a red object in the camera image and keep it near the center of the frame by driving a two-axis gimbal.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/11_final_tracking.gif" alt="Figure 1: Final dynamic tracking. Source: this recording, cropped to show the gimbal, target object, and device screen." width="640" style="max-width: 100%; height: auto;">
</p>
When the object moves, the gimbal adjusts accordingly. This is the target behavior for the following development, debugging, and acceptance steps.
## Video Tutorial
<iframe src="//player.bilibili.com/player.html?isOutside=true&bvid=BV13W3w6REa2&p=1" scrolling="no" allowfullscreen style="width:90%; max-width:960px; aspect-ratio:16/9; height:auto; border:0; display:block; margin:0 auto;"></iframe>
If the embedded video does not play, open it on Bilibili: [MaixCAM2 x MCP auto debugging video tutorial](https://www.bilibili.com/video/BV13W3w6REa2/).
## Choose an AI Agent for Development
You can use Codex, Claude Code, or OpenCode. The recorded Agent was Codex, while this page only covers OpenCode installation and configuration to avoid duplicating multiple Agent interfaces. The later task description, device connection, and acceptance workflow do not depend on the Agent UI shown in the recording.
The Agent should at least be able to:
- read and modify the local project;
- run local tools;
- connect to the target device;
- read runtime logs;
- view debug images, or ask the user to confirm the real image when it cannot view them directly.
### Install OpenCode
Purpose: prepare an Agent that can work on the local project from the development computer.
Steps:
1. Open the [OpenCode download page](https://opencode.ai/download).
2. Select the Desktop installer for your operating system. The official page provides download entries for macOS, Windows, and Linux.
3. After installation, start OpenCode and open or create the local working directory for this project.
4. Before continuing, confirm that the Agent can read files in the directory and run local tools.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/01_opencode_download.png" alt="Figure 2: OpenCode official download page. Source: OpenCode download page." width="640" style="max-width: 100%; height: auto;">
</p>
Note: the download page and client UI may change over time. Follow the system packages and versions shown on the current official download page.
### Install cc-switch and Configure OpenCode Models
Purpose: use cc-switch to manage OpenCode providers and model settings.
Steps:
1. Download the version for your system from the [cc-switch website](https://ccswitch.io) or the [official Releases page](https://github.com/farion1231/cc-switch/releases).
2. Open cc-switch and select the OpenCode configuration entry.
3. Add or import a provider configuration, then select the model you want to use.
4. Enable the configuration, restart or reopen OpenCode, and confirm that the current model and provider match your expectation.
The cc-switch repository states that it supports Windows, macOS, and Linux, and includes OpenCode configuration management. Whether DeepSeek, Doubao, GPT, Claude, or other models are available depends on the cc-switch version, provider, and account permissions shown in your actual UI.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/02_ccswitch_home.png" alt="Figure 3: cc-switch homepage and configuration UI example. Source: ccswitch.io." width="640" style="max-width: 100%; height: auto;">
</p>
### Install uv
Purpose: prepare a Python environment management tool for installing and running maixpy-skill later.
maixpy-skill needs a Python runtime when installing and executing helper scripts for device development. Installing only a system-level Python is not recommended: built-in Python versions, `pip` permissions, and dependency isolation differ across operating systems, which can lead to packages being installed into the wrong environment, system Python pollution, or version mismatches. `uv` can manage Python versions, virtual environments, and dependency installation in a unified way. If a suitable Python version already exists, it can use it directly; if the required version is missing, it can install and manage it as needed.
Steps:
1. Install uv.
macOS and Linux:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
Windows PowerShell:
```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
2. Reopen the terminal and confirm that `uv` has been added to `PATH`:
```bash
uv --version
```
After uv is installed, continue by asking the Agent to install maixpy-skill. If maixpy-skill later needs a Python version or extra Python packages, the Agent will manage them through uv instead of modifying the system Python environment directly.
## Install maixpy-skill
Purpose: give the Agent the workflows and device operation capabilities needed for MaixCAM series development.
Prerequisite: download [maixpy-skill.zip](https://dl.sipeed.com/fileList/MaixCAM/MaixCAM2/Software/maixpy-skill.zip) and extract it to a local path that the Agent can access.
You can send this directly to OpenCode:
> Open this link: https://dl.sipeed.com/fileList/MaixCAM/MaixCAM2/Software/maixpy-skill.zip, then download and install maixpy-skill. After installation, check whether it supports MaixCAM2 device connection, development mode, program execution, log reading, and debug-image artifact retrieval. Do not print the device password in the conversation or logs.
After installation, ask the Agent to report:
- whether the skill has been discovered and enabled;
- whether device connection, execution, log, and debug-image capabilities are available;
- where runtime records are stored in the current project;
- missing dependencies or information that still needs user confirmation.
OpenCode uses `SKILL.md` to define reusable Agent Skills. The actual loading path and discovery rules should follow the current OpenCode documentation.
## Prepare the MaixCAM2 Development Environment and Peripherals
Scope: MaixCAM2, the dedicated UART4 two-axis servo gimbal, and the red-object tracking project.
Checklist:
- MaixCAM2, using its built-in camera without an external camera;
- a development computer that can access the same LAN;
- dedicated UART4 two-axis servo gimbal, model `RLU-C45`;
- red object;
- enough free space for safe gimbal movement.
Steps:
1. Power on the MaixCAM2.
2. Connect the device to Wi-Fi in system settings, or use the USB network method described in the official documentation, so the computer and device can reach each other.
3. Check the device address in "Settings -> Device info" on the device.
4. Connect the gimbal to UART4 and confirm that power, ground, and signal wires are reliable.
5. Keep cables, fingers, and obstacles away from the gimbal travel range.
6. Place the red object inside the MaixCAM2 built-in camera view.
The MaixCAM2 quick-start documentation explains that the device needs network access for first use. After connecting to Wi-Fi, you can check the IP address in device information. It also explains that the computer can connect to the device through Wi-Fi or USB networking.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/03_hardware_and_agent.jpg" alt="Figure 4: MaixCAM2, two-axis gimbal, and red object during recording. Source: this recording, cropped to show the physical setup." width="480" style="max-width: 100%; height: auto;">
</p>
Note: UART4 pin mapping, servo IDs, power requirements, and protocol details depend on the gimbal model. Do not treat values from a reference project as universal settings.
## Submit the Initial Development Request
Purpose: provide the goal, reference project, validation order, safety constraints, and final behavior in one message, reducing guesses on critical conditions.
Replace `[device address]` with the current device address and send:
> Load maixpy skill and connect to MaixCAM2 at `root@[device address]`.
>
> I have connected a Sipeed-tested two-axis gimbal. The open-source reference code is at https://github.com/sipeed/MaixPy/tree/main/projects/demo_block_tracking. The gimbal is connected through UART4. Please develop a red-object tracker with fast response, real-time tracking, high precision, and stable behavior. You can use the MaixPy API documentation at https://wiki.sipeed.com/maixpy/doc/en/index.html, and my computer already has uv installed.
>
> I have placed the red object in the camera view. Before closed-loop tracking, first make small gimbal movements to confirm coordinates and actual movement direction. The real gimbal movement direction may not match the code direction, so calibrate it first. For example, command the gimbal to move left and then ask me which direction it actually moved. Do the same for other directions.
>
> The original safety range may be inaccurate. You can remove the old gimbal limits temporarily, let me manually move it to the mechanical limits, then read those positions for calibration. Use conservative values to set a suitable safe range.
>
> Please greatly increase the tracking speed and use three-stage gain: large steps for fast chasing when far away, fast convergence at medium distance, and low-gain braking near the center. If either direction overshoots, it must be able to move back in the opposite direction and relocate the object; otherwise it is only moving one way. If the object leaves the trackable angle, stop tracking and return to center.
>
> Also note: during development, when exiting a test, restore a simple MaixCAM2 built-in camera pass-through display example to avoid leaving the device with a black screen for a long time during AI coding.
`projects/demo_block_tracking` is a project directory in the official MaixPy repository. It contains the application configuration, main program, and servo-related implementation.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/04_task1_input.jpg" alt="Figure 5: Task 1 execution screen. Source: this recording, cropped to show the physical scene and Agent task area." width="640" style="max-width: 100%; height: auto;">
</p>
Note: do not publish private LAN addresses, passwords, or access tokens in public documentation. The reference project is used to understand the protocol and project structure. Direction, center position, limits, color thresholds, and control parameters must be validated again on the current hardware.
## Step-by-Step Debugging and Manual Confirmation
Task 1 should be completed in stages, with the Agent reporting each result. When the Agent cannot determine the actual hardware state through APIs, the user should provide observations such as "the positive horizontal command actually turns right" or "the gimbal hits the bracket at this position".
### Gimbal Micro-Movement and Direction Calibration
Purpose: confirm that both axes can communicate, move slightly, return to center, and establish local safety constraints.
Steps:
1. Probe whether both servos are online.
2. Read the current angle or position.
3. Move only one axis at a time with a small command.
4. Observe the actual direction and record the positive/negative direction mapping for that axis.
5. Return that axis to center, then test the other axis.
6. Try to read firmware limits. If they cannot be read, use small probing movements and user observation to establish conservative limits.
Pass criteria: both axes are online, directions are recorded, the gimbal can return to center, and movements do not approach collision positions.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/05_micro_motion.jpg" alt="Figure 6: Small gimbal movement and device feedback. Source: this recording, cropped to show gimbal posture and Agent output." width="640" style="max-width: 100%; height: auto;">
</p>
The image shows the gimbal and device feedback. The recording alone is not enough to prove that servo firmware limits were finally read successfully; this should be confirmed separately in the runtime logs.
### Red-Object Detection Verification
Purpose: confirm that the vision input is correct before adding gimbal control.
Steps:
1. Run only the MaixCAM2 built-in camera and red-object detection, without sending gimbal tracking commands.
2. Detect red candidate regions and filter out noise with very small areas or pixel counts.
3. Select the largest candidate region and output its center coordinates, area, and frame rate.
4. Move the object and confirm that the coordinates change accordingly.
5. Move the object out of view and confirm that the program reports no target instead of continuing to use old coordinates.
Pass criteria: the target can be repeatedly detected and coordinates can be obtained; when the object disappears, old target coordinates are not reused.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/06_red_detection.jpg" alt="Figure 7: Red-object detection result, with detection box shown on the device screen. Source: this recording, cropped to show the physical device and target box on screen." width="480" style="max-width: 100%; height: auto;">
</p>
The device screen shows the red target detection box. The exact threshold, bounding box, and frame-rate values should be confirmed from the current runtime logs and debug images.
### Add the Three-Stage Closed-Loop Control Strategy
Purpose: let the two-axis gimbal track the target according to the error between the object and the frame center, while avoiding overshoot from high-speed movement and oscillation near the center.
Suggested control rules:
| Error range | Behavior | Validation focus |
| --- | --- | --- |
| Far | large steps and high-speed chasing | does not exceed safe limits |
| Medium | fast convergence with acceleration limiting | does not noticeably cross the center |
| Near center | low-gain braking; hold position after entering the dead zone | no continuous oscillation |
Each axis should maintain its own direction mapping, limits, speed, acceleration, and control state. After the target is lost continuously for a threshold period, the program should stop chasing old coordinates and return to center at a safe speed.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/07_closed_loop_debug.jpg" alt="Figure 8: Closed-loop control debugging stage. Source: this recording, cropped to show physical state and Agent debug output." width="640" style="max-width: 100%; height: auto;">
</p>
The Agent is investigating possible causes such as "movement too aggressive". The recording alone cannot prove all three-stage gain parameters; the actual parameters should be confirmed in the source code or logs.
### Correct the Closed-Loop Direction Mapping
After directly using the reference implementation, the real installation direction did not match the expected direction. The gimbal moved in the wrong direction and the target left the trackable range.
Debugging steps:
1. Keep the object fixed.
2. Test only one axis at a time.
3. Compare the target offset in the image with the actual gimbal movement direction.
4. Modify only the direction mapping for the corresponding axis.
5. Test left, right, up, and down again.
6. After the direction is correct, adjust gain and speed.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/08_wrong_direction.gif" alt="Figure 9: Wrong direction in the first closed-loop test. Source: this recording, cropped to show target position and actual gimbal movement." width="640" style="max-width: 100%; height: auto;">
</p>
Note: before direction is confirmed, do not tune PID, speed, or thresholds first. Otherwise, it is hard to tell whether the issue is wrong control direction or bad parameters.
### Suppress Oscillation and Overshoot
After fixing the pitch direction, the gimbal still oscillated left and right when the target was static. After further tuning, tracking became stable.
Recommended order:
1. Confirm again that the direction mapping is correct.
2. Increase the center dead zone.
3. Lower maximum speed and acceleration near the center.
4. Limit the single-step position change.
5. Clear integral and history state after entering the dead zone.
6. Allow reverse correction after the error crosses the center.
7. Stop chasing and return to center safely when the target is continuously lost.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/09_wobble.gif" alt="Figure 10: Oscillation before the fix. Source: this recording, cropped as a looped animation." width="640" style="max-width: 100%; height: auto;">
</p>
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/09_stable_tracking.jpg" alt="Figure 11: Stable tracking after the fix. Source: this recording, cropped to show the gimbal and target." width="640" style="max-width: 100%; height: auto;">
</p>
## Automated Position Regression Test
Purpose: use repeatable fixed-position tests instead of relying on "it looks fine once".
Send this to the Agent:
> Generate an automated position test for the current gimbal. First move the gimbal away from the color block, then run the program and check whether it can automatically track back to the color block. Choose random offset positions, but do not move the red block out of the detectable range.
Check:
- whether the directions are correct;
- whether returning to center is stable;
- whether repeated runs create accumulated drift;
- whether the gimbal approaches or exceeds safe limits;
- whether each step is recorded in logs or a test summary.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/10_auto_test_request.jpg" alt="Figure 12: Automated test request. Source: this recording, cropped to show the test requirement and execution area." width="640" style="max-width: 100%; height: auto;">
</p>
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/10_auto_test.gif" alt="Figure 13: Automated position test. Source: this recording, cropped as a looped animation." width="640" style="max-width: 100%; height: auto;">
</p>
The recording also includes multiple repeated position checks. If you publish a more complete acceptance report, consider adding screenshots or a result table from those clips.
## Finalize the Implementation and Run Dynamic Acceptance Testing
After the automated test passes, freeze the current implementation as the final dynamic tracking version, then perform manual dynamic testing.
Send this to the Agent:
> The automated position test passed. Please finalize the current dynamic tracking implementation, disable continuous debug-image saving, keep graceful exit and restore MaixCAM2 built-in camera pass-through display after exit. I will now move the red object manually and verify the real tracking behavior. Please refer to the MaixPy documentation for the exit method; the implementation must exit gracefully.
Manual acceptance steps:
1. Keep the object still at the center.
2. Move it slowly left, right, up, and down.
3. Change direction quickly at medium distance.
4. Move it from the center to the edge.
5. Move it out of the frame and then back in.
Pass criteria: both axes move in the correct direction; far targets can be chased; there is no continuous oscillation near the center; after overshoot, the gimbal can move back in the opposite direction; after target loss, it does not continue chasing old coordinates.
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/11_final_tracking.gif" alt="Figure 14: Final dynamic tracking acceptance. Source: this recording, cropped as a looped animation." width="640" style="max-width: 100%; height: auto;">
</p>
## Acceptance Results and Deliverable Archive
After acceptance, collect these deliverables from the Agent:
1. current project source code;
2. UART4, servo IDs, direction mapping, center position, and safe limits validated on the current hardware;
3. red-detection threshold and validation conditions;
4. automated test entry point, runtime records, and conclusion;
5. a small set of debug images and runtime logs;
6. known limitations and safety boundaries.
When checking the deliverables, confirm that continuous debug-image saving is disabled in the release version, that the application has a graceful exit path, and that it restores the MaixCAM2 built-in camera pass-through display after exit as required.
## General Task Template
Replace the bracketed fields with your own project requirements:
> Load maixpy skill and connect to my MaixCAM2. I want to develop [target function]. Use the MaixCAM2 built-in camera by default; if another sensor is needed, the input is [input]. The device should perform [action], and the success criteria are [acceptance criteria]. Please verify device connection, peripheral communication, and input data separately before integrating the full control logic. Do not directly reuse direction, center, limit, or parameter values from other devices; confirm each item on the current hardware. After debugging, provide the project source code, test results, logs/debug artifacts, and verification video. When the program exits, restore a simple MaixCAM2 built-in camera pass-through display.
### References
1. [OpenCode download page](https://opencode.ai/download) for desktop downloads and supported systems.
2. [OpenCode Agent Skills documentation](https://opencode.ai/docs/skills/) for reusable Agent behavior defined with `SKILL.md`.
3. [OpenCode model configuration documentation](https://opencode.ai/docs/models/) for provider/model configuration and model selection rules.
4. [cc-switch repository](https://github.com/farion1231/cc-switch) for supported platforms, Agent tools, and configuration management capabilities.
5. [uv installation documentation](https://docs.astral.sh/uv/getting-started/installation/) for uv installation methods and commands.
6. [MaixCAM2 MaixPy quick start](https://wiki.sipeed.com/maixpy/doc/en/README_MaixCAM2.html) for network connection, device address, computer connection, and development environment notes.
7. [MaixPy `demo_block_tracking` project directory](https://github.com/sipeed/MaixPy/tree/main/projects/demo_block_tracking) for the reference project used here.
8. [MaixPy UART documentation](https://wiki.sipeed.com/maixpy/doc/en/peripheral/uart.html) for UART peripheral usage.

View File

@@ -171,6 +171,8 @@ items:
file: maixcam/maixcam2_os.md
- label: Cammeras & Lens
file: maixcam/maixcam2_camera_lens.md
- label: Develop with AI Agent
file: maixcam/maixcam2_xAI.md
- label: Develop with MaixPy
url: https://wiki.sipeed.com/maixpy/
- label: Develop with MaixCDK

View File

@@ -0,0 +1,356 @@
---
title: MaixCAM2 x AI用 AI Agent 开发二轴云台物块追踪
---
本文以一次实际录制的 MaixCAM2 开发过程为例,说明如何使用 AI Agent 开发基于 MaixCAM2 内置摄像头的 UART4 二轴云台红色物块追踪器。本次录制使用 Codex 调用 maixpy-skill为方便 Windows、macOS 和 Linux 用户复刻,安装与模型配置部分只演示 OpenCode。图中的 Codex 界面用于呈现实录过程,不是 OpenCode 的操作界面。
## 项目目标与最终效果
目标是识别画面中的红色物块,并让两轴云台持续将物块保持在画面中心附近。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/11_final_tracking.gif" alt="图 1最终动态跟随。来源本次实录裁剪保留云台、目标物和设备屏幕。" width="640" style="max-width: 100%; height: auto;">
</p>
物块移动时,云台会随之调整;这是后续开发、调试和验收的目标状态。
## 视频教程
<iframe src="//player.bilibili.com/player.html?isOutside=true&bvid=BV13W3w6REa2&p=1" scrolling="no" allowfullscreen style="width:90%; max-width:960px; aspect-ratio:16/9; height:auto; border:0; display:block; margin:0 auto;"></iframe>
如果页面内视频无法播放,可以打开 B 站视频:[【电赛特辑】MaixCAM2 x MCP 自动调试之视频详解](https://www.bilibili.com/video/BV13W3w6REa2/)。
## 选择开发用 AI Agent
可使用 Codex、Claude CodeCC或 OpenCode。实录中的 Agent 是 Codex本文只展开 OpenCode 的安装和配置,避免重复介绍多个 Agent 的界面。后续任务描述、设备连接和验收步骤不依赖图中的 Agent 界面。
Agent 至少需要具备以下能力:
- 读取和修改本地工程;
- 执行本地工具;
- 与目标设备建立连接;
- 读取运行日志;
- 查看调试图片,或在无法查看时要求用户确认实际画面。
### OpenCode 安装
目的:在开发电脑上准备可执行本地工程任务的 Agent。
步骤:
1. 打开 [OpenCode 官方下载页](https://opencode.ai/download)。
2. 选择与当前系统对应的 Desktop 安装包;官方页面列出了 macOS、Windows 和 Linux 的下载入口。
3. 完成安装后启动 OpenCode并打开或创建本次项目的本地工作目录。
4. 在继续之前确认 Agent 能读取该目录中的文件并执行本地工具。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/01_opencode_download.png" alt="图 2OpenCode 官方下载页。来源OpenCode 下载页。" width="640" style="max-width: 100%; height: auto;">
</p>
注意:下载页和客户端界面会随版本变化;以官方下载页当前显示的系统包和版本为准。
### cc-switch 安装与 OpenCode 模型配置
目的:使用 cc-switch 管理 OpenCode 的服务商和模型配置。
步骤:
1. 从 [cc-switch 官方网站](https://ccswitch.io) 或 [官方 Releases](https://github.com/farion1231/cc-switch/releases) 下载当前系统版本。
2. 打开 cc-switch选择 OpenCode 对应的配置入口。
3. 添加或导入服务商配置,再选择需要使用的模型。
4. 启用该配置后,重新启动或重新打开 OpenCode并确认当前模型与服务商显示符合预期。
cc-switch 官方仓库说明其支持 Windows、macOS、Linux并包含 OpenCode 的配置管理能力。DeepSeek、豆包、GPT、Claude 等模型是否可选,取决于当前 cc-switch 版本、服务商和账户权限,需以实际界面为准。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/02_ccswitch_home.png" alt="图 3cc-switch 官方主页及配置管理界面示意。来源ccswitch.io。" width="640" style="max-width: 100%; height: auto;">
</p>
### uv 安装
目的:为后续安装和运行 maixpy-skill 准备 Python 环境管理工具。
maixpy-skill 在安装和执行设备辅助脚本时需要 Python 运行环境。这里不建议只手动安装一个系统级 Python不同系统自带的 Python 版本、`pip` 权限和依赖隔离方式差异较大,容易出现包安装到错误环境、污染系统 Python、版本不匹配等问题。`uv` 可以统一管理 Python 版本、虚拟环境和依赖安装;如果本机已有合适的 Python它可以直接使用如果缺少所需版本也可以按需安装和管理。
步骤:
1. 安装 uv。
macOS 和 Linux
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
Windows PowerShell
```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
2. 重新打开终端,确认 `uv` 已经加入 `PATH`
```bash
uv --version
```
安装完成后,继续让 Agent 安装 maixpy-skill。后续如果 maixpy-skill 需要 Python 版本或额外 Python 包Agent 会通过 uv 管理依赖,而不是直接修改系统 Python 环境。
## 安装 maixpy-skill
目的:让 Agent 获得 MaixCAM 系列设备开发所需的工作流和设备操作能力。
前置条件:下载 [maixpy-skill.zip](https://dl.sipeed.com/fileList/MaixCAM/MaixCAM2/Software/maixpy-skill.zip),解压到本地可访问的位置。
可直接向 OpenCode 发送:
> 访问这个链接https://dl.sipeed.com/fileList/MaixCAM/MaixCAM2/Software/maixpy-skill.zip ,下载并安装 maixpy-skill。安装完成后检查它是否支持 MaixCAM2 的设备连接、开发模式、程序运行、日志读取和调试图片产物获取;不要在对话或日志中输出设备密码。
安装完成后,要求 Agent 汇报:
- skill 是否已经被发现并启用;
- 设备连接、运行、日志和调试图片能力是否可用;
- 当前项目中保存运行记录的位置;
- 缺失的依赖或待用户确认的信息。
OpenCode 使用 `SKILL.md` 定义可复用的 Agent Skills具体加载位置和发现规则应以当前 OpenCode 文档为准。
## 准备 MaixCAM2 开发环境和外设
适用范围MaixCAM2、专用 UART4 二轴舵机云台和红色物块追踪项目。
准备清单:
- MaixCAM2使用其内置摄像头不需要另配摄像头
- 可访问同一局域网的开发电脑;
- 专用 UART4 二轴舵机云台(型号:`RLU-C45`
- 红色物块;
- 可供云台安全运动的空间。
步骤:
1. 给 MaixCAM2 供电并开机。
2. 在设备设置中连接 Wi-Fi或使用官方文档说明的 USB 网络方式,使电脑与设备可互通。
3. 在设备的“设置 → 设备信息”中确认设备地址。
4. 将云台接入 UART4确认供电、地线和信号线连接可靠。
5. 清理云台机械行程附近的线缆、手指和障碍物。
6. 将红色物块放入 MaixCAM2 内置摄像头视野。
MaixCAM2 快速开始文档说明,首次使用需连接网络;连接 Wi-Fi 后可在设备信息中查看 IP。该文档也说明可通过 Wi-Fi 或 USB 网络方式连接电脑和设备。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/03_hardware_and_agent.jpg" alt="图 4录制中的 MaixCAM2、二轴云台和红色物块。来源本次实录裁剪为实物设备画面。" width="480" style="max-width: 100%; height: auto;">
</p>
注意UART4 的具体引脚映射、舵机 ID、供电规格和协议取决于云台型号。本文不把参考工程中的数值视为通用配置。
## 提交初始开发需求
目的:一次性提供目标、参考工程、验证顺序、安全约束和最终行为,减少 Agent 在关键条件上猜测。
将下面的 `[设备地址]` 换成当前设备地址后发送:
> 加载 maixpy skill连接 maixcam2 `root@[设备地址]`。
>
> 我连接了矽速测试过的二轴云台,他们有开源源码在 https://github.com/sipeed/MaixPy/tree/main/projects/demo_block_tracking 。我通过 uart4 连接了该云台,请给我开发一个红色物块追踪器,要求高速响应、实时追踪,并且保证精度,稳定追踪。你可以使用 https://wiki.sipeed.com/maixpy/doc/zh/index.html 这个里面的API函数进行开发我的电脑中有uv。
>
> 我已经把红色物体放在视野当中了,请先让二轴微动确认坐标和实际的移动方向,因为实际云台移动的方向和代码的方向不一致,所以请你先执行一下校准云台的操作,你可以控制云台向左移动,然后询问我云台的移动方向,其他方向亦是如此。
>
> 原来的安全范围限定不一定准确,你可以解除云台的限制,让我手动掰到限位,然后你读取那个位置来进行校准,然后适当保守些,设置合适的安全范围。
>
> 请大幅提速追踪,采用三段式增益:远距离大步高速追赶、中距离快速收敛、近中心低增益制动。并且当某方向甩动过头时要求能反向回来重新定位物体,否则就是单向运动。物体离开了可追踪角度则放弃追踪并居中回来。
>
> 另外注意:开发过程中请在退出测试的时候,恢复一个简单的 MaixCAM2 内置摄像头画面直传显示屏的例程,避免 AI coding 的时候,设备长时间黑屏空置。
`projects/demo_block_tracking` 是 MaixPy 官方仓库中的项目目录,当前目录包含应用配置、主程序和舵机相关实现。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/04_task1_input.jpg" alt="图 5任务一执行画面。来源本次实录裁剪保留实物画面和 Agent 任务区。" width="640" style="max-width: 100%; height: auto;">
</p>
注意:不要将内网地址、密码或访问令牌发布到公开文档。参考项目用于理解协议和工程结构;当前硬件的方向、中心、限位、颜色阈值和控制参数必须重新验证。
## 分阶段调试与人工确认
任务一应由 Agent 分阶段完成并汇报每一步结果。用户在 Agent 无法从接口确定实际硬件状态时补充观察结果,例如“水平轴正向命令实际向右转”或“云台到某位置会碰到支架”。
### 云台微动与方向校准
目的:确认两个轴可通信、可小幅运动、可回中,并建立本机的安全约束。
步骤:
1. 分别探测两个舵机是否在线。
2. 读取当前角度或位置。
3. 一次只对一个轴发送小幅动作。
4. 观察实际转向,记录该轴的正反方向。
5. 让该轴回到中心,再测试另一个轴。
6. 尝试读取固件限位;若不可读取,采用小幅探测和人工观察建立保守限位。
通过标准:两轴在线、方向已记录、可回中、动作未接近机械碰撞位置。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/05_micro_motion.jpg" alt="图 6云台微动与设备反馈。来源本次实录裁剪保留云台姿态和 Agent 输出。" width="640" style="max-width: 100%; height: auto;">
</p>
画面可见云台与设备反馈。录像不足以证明最终成功读取了舵机固件限位;此项应在运行日志中单独确认。
### 红色物块检测验证
目的:在接入云台控制前确认视觉输入正确。
步骤:
1. 仅运行 MaixCAM2 内置摄像头和红色物块检测,不发送云台追踪命令。
2. 识别红色候选区域,并过滤面积或像素数过小的噪声。
3. 选取最大候选区域,输出中心坐标、面积和帧率。
4. 移动物块并确认坐标同步变化。
5. 移出物块,确认程序报告未检测到目标,而不是继续使用旧坐标。
通过标准:可重复识别目标并获得坐标;物块消失后不输出旧目标坐标。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/06_red_detection.jpg" alt="图 7红色物块识别结果设备屏幕显示检测框。来源本次实录裁剪保留实物设备和屏幕中的目标检测框。" width="480" style="max-width: 100%; height: auto;">
</p>
设备屏幕显示红色目标的检测框。检测阈值、识别框和帧率的具体数值需以当前运行日志和调试图为准。
### 接入三段式闭环控制策略
目的:让两轴云台依据物块相对画面中心的误差追踪目标,同时避免高速运动带来的超调和近中心摆动。
建议将控制规则拆为三段:
| 误差区间 | 行为 | 验证重点 |
| --- | --- | --- |
| 远距离 | 大步长、高速度追赶 | 不越过安全限位 |
| 中距离 | 快速收敛,并限制加速度 | 不明显冲过中心 |
| 近中心 | 低增益制动;进入死区后保持当前位置 | 不持续摆动 |
每个轴应独立维护方向、限位、速度、加速度和控制状态。目标连续丢失达到阈值后,应停止追赶旧坐标并按安全速度回中。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/07_closed_loop_debug.jpg" alt="图 8闭环控制调试阶段。来源本次实录裁剪保留实物状态与 Agent 调试输出。" width="640" style="max-width: 100%; height: auto;">
</p>
Agent 正在针对追踪行为排查“动作过猛”等可能原因。录像无法单独证明三段式增益的全部参数,实际参数需在工程源码或日志中确认。
### 闭环方向映射修正
直接使用参考实现后,实际安装方向不一致,云台向错误方向运动,目标离开可追踪范围。
排查步骤:
1. 固定物块位置。
2. 一次只测试一个轴。
3. 对照目标在画面中的偏移和云台实际转向。
4. 只修改对应轴的方向映射。
5. 重新测试左、右、上、下四个方向。
6. 方向正确后再调整增益和速度。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/08_wrong_direction.gif" alt="图 9首次闭环方向错误。来源本次实录裁剪保留目标位置与云台实际转向。" width="640" style="max-width: 100%; height: auto;">
</p>
注意:方向未确认前,不应先调 PID、速度或阈值否则无法区分是控制方向错误还是参数问题。
### 抑制摆动与超调
俯仰方向修复后,目标静止时云台仍在两侧来回摆;继续调整后,追踪恢复稳定。
处理顺序:
1. 再次确认方向正确。
2. 增加中心死区。
3. 降低中心附近的最大速度和加速度。
4. 限制单次位置变化。
5. 进入死区时清空积分和历史状态。
6. 误差跨过中心后允许反向重新定位。
7. 目标持续丢失时停止追赶并安全回中。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/09_wobble.gif" alt="图 10修复前的摆动。来源本次实录裁剪为循环动图。" width="640" style="max-width: 100%; height: auto;">
</p>
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/09_stable_tracking.jpg" alt="图 11修复后的稳定追踪。来源本次实录裁剪保留云台和目标。" width="640" style="max-width: 100%; height: auto;">
</p>
## 自动化位置回归测试
目的:用可重复的固定位置测试替代“一次看起来正常”的主观判断。
向 Agent 提交:
> 请为当前云台生成自动化位置测试。你先控制云台偏离色块,然后运行程序,然后看看程序会不会自动跟踪回色块。偏离的位置随机,但是不要让红色的色块超出识别的范围。
检查项:
- 方向是否正确;
- 回中是否稳定;
- 重复执行后是否产生累积偏差;
- 是否接近或越过安全限位;
- 每一步的结果是否记录在日志或测试摘要中。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/10_auto_test_request.jpg" alt="图 12提交自动化测试请求。来源本次实录裁剪保留测试需求和执行区。" width="640" style="max-width: 100%; height: auto;">
</p>
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/10_auto_test.gif" alt="图 13自动化位置测试。来源本次实录裁剪为循环动图。" width="640" style="max-width: 100%; height: auto;">
</p>
录制中还保留了多轮重复位置验证。若发布更完整的验收报告,建议补充这些片段中的截图或结果表。
## 固化最终实现并进行动态验收
自动化测试通过后,将当前实现固化为最终动态跟随版本,再进行人工动态测试。
向 Agent 提交:
> 自动化位置测试通过。请固化当前最终动态跟随实现,关闭持续保存调试图片,保留正常退出和退出后 MaixCAM2 内置摄像头直传恢复。现在开始人工移动红色物块我将根据真实跟随效果进行验收。程序退出的方法请参考MaixPy文档实现一定要优雅。
人工验收步骤:
1. 物块静止在中心。
2. 缓慢向左、右、上、下移动。
3. 在中距离快速改变方向。
4. 从中心移到边缘。
5. 离开画面后重新进入。
通过标准:两轴方向正确;远距离能追赶;近中心不持续摆动;超调后能反向回正;目标丢失后不会继续追赶旧坐标。
<p align="center">
<img src="../../assets/maixcam/maixcam2_xai/11_final_tracking.gif" alt="图 14最终动态跟随验收。来源本次实录裁剪为循环动图。" width="640" style="max-width: 100%; height: auto;">
</p>
## 验收结果与交付物归档
验收通过后,应从 Agent 获取:
1. 当前工程源码;
2. 当前硬件验证过的 UART4、舵机 ID、方向、中心位置和安全限位
3. 红色检测阈值和验证条件;
4. 自动化测试入口、运行记录和结论;
5. 少量调试图像与运行日志;
6. 已知限制和安全边界;
检查交付时,应确认正式版本已关闭持续保存调试图片,应用具备正常退出路径,并按需求恢复 MaixCAM2 内置摄像头直传显示。
## 通用任务模板
将方括号替换为自己的项目需求:
> 加载 maixpy skill连接我的 MaixCAM2。我要开发一个[目标功能]:默认使用 MaixCAM2 内置摄像头;如需其他传感器,输入是[输入]。设备需要执行[动作],成功标准是[验收标准]。请先分别验证设备连接、外设通信和输入数据,再接入完整控制逻辑。不要直接复用其他设备的方向、中心、限位或参数;请在当前硬件上逐项确认。调试结束后给出工程源码、测试结果、日志/调试产物和确认视频;程序退出时恢复一个简单的 MaixCAM2 内置摄像头直传显示。
### 参考资料
1. [OpenCode 官方下载页](https://opencode.ai/download)(桌面端下载入口与支持系统)。
2. [OpenCode Agent Skills 文档](https://opencode.ai/docs/skills/)`SKILL.md` 形式的可复用 Agent 行为)。
3. [OpenCode 模型配置文档](https://opencode.ai/docs/models/)(服务商/模型配置与模型选择规则)。
4. [cc-switch 官方仓库](https://github.com/farion1231/cc-switch)(支持平台、支持的 Agent 工具与配置管理能力)。
5. [uv 官方安装文档](https://docs.astral.sh/uv/getting-started/installation/)uv 安装方式和安装命令)。
6. [MaixCAM2 MaixPy 快速开始](https://wiki.sipeed.com/maixpy/doc/zh/README_MaixCAM2.html)(网络连接、设备地址、电脑连接与开发环境说明)。
7. [MaixPy `demo_block_tracking` 项目目录](https://github.com/sipeed/MaixPy/tree/main/projects/demo_block_tracking)(本次参考工程)。
8. [MaixPy UART 文档](https://wiki.sipeed.com/maixpy/doc/zh/peripheral/uart.html)UART 外设使用)。

View File

@@ -173,6 +173,8 @@ items:
file: maixcam/maixcam2_os.md
- label: 摄像头和镜头选择
file: maixcam/maixcam2_camera_lens.md
- label: 使用 AI Agent 开发
file: maixcam/maixcam2_xAI.md
- label: 使用 MaixPy 开发
url: https://wiki.sipeed.com/maixpy/
- label: 使用 MaixCDK 开发