Squashed commit of the following:

commit 3dea4b1cfcb866d50c9c093b797c0f4f48fec23f Author: wenjie <meetwenjie@gmail.com> Date: Fri Apr 10 17:11:07 2026 +0800 fix(picoclaw): clean temp media and surface MCP screenshots in chat - remove shared load_image staging and pass source paths through directly - delete /tmp/picoclaw_media when a gateway session closes - push MCP screenshot observations to downstream websocket clients - hide null tool feedback messages and auto-scroll when screenshots load commit f58ffe1e27f83f64e2b0e8c2f8dcac46af715ace Author: wenjie <meetwenjie@gmail.com> Date: Fri Apr 10 15:55:59 2026 +0800 Harden PicoClaw local API auth and session locking Split PicoClaw routes by caller trust level and require the internal loopback token for local-only endpoints. Update the NanoKVM bridge script to send the internal token for loopback requests. Relax session lock acquisition for screenshot and action calls so the active session can perform local operations without permanently taking over the lock, and stop forcing the runtime dm_scope default. commit 462a0670bbc5138c893fbd1155e72465b82c2065 Author: wenjie <meetwenjie@gmail.com> Date: Fri Apr 10 13:52:39 2026 +0800 fix(picoclaw): improve KVM reliability and sidebar behavior - follow HTTPS loopback redirects in the NanoKVM bridge script - enable MJPEG frame caching only during active PicoClaw gateway sessions - restore legacy screen zoom behavior across MJPEG and H264 renderers - keep the PicoClaw sidebar available on mobile without splitter layout conflicts - hide empty "null"/"undefined" chat messages and keep MJPEG failures on a black screen commit bfda85c6368825c7ddf47d347ba5ec2588135885 Author: wenjie <meetwenjie@gmail.com> Date: Fri Apr 10 11:37:15 2026 +0800 fix(picoclaw): verify runtime downloads and remove unused config API - verify the downloaded runtime archive against the published SHA-512 checksum - remove the unused /api/picoclaw/config endpoints and related frontend state - rename runtime_control.go to runtime_constants.go for clearer intent commit 10b34aaea59e10d70c3cdecb01915444561ceaf7 Author: wenjie <meetwenjie@gmail.com> Date: Thu Apr 9 16:32:30 2026 +0800 fix: stabilize picoclaw runtime defaults and secure local MCP access - persist required NanoKVM startup defaults before launching picoclaw and after saving model config - force-enable the pico channel when loading config so runtime status can recover to ready - derive the local MCP URL from the configured HTTP port and keep loopback-only HTTP access behind an internal token - move loopback HTTP redirect logic into middleware to simplify main server startup - improve runtime/sidebar state handling and add the load-image endpoint for active picoclaw sessions commit d20540195bf45ea290f727b473ee8cd2a1bbb68d Author: wenjie <meetwenjie@gmail.com> Date: Fri Mar 27 17:55:29 2026 +0800 feat(picoclaw): add picoclaw integration
2026-09-11 00:22:56 -05:00 · 2026-04-10 17:28:28 +08:00
parent fda3c6b1a8
commit 3254412017
80 changed files with 11774 additions and 117 deletions
--- a/kvmapp/picoclaw/AGENT.md
+++ b/kvmapp/picoclaw/AGENT.md
@@ -0,0 +1,45 @@
+---
+name: pico
+description: >
+  The default general-purpose assistant for everyday conversation, problem
+  solving, and workspace help.
+---
+
+You are Pico, the default assistant for this workspace.
+Your name is PicoClaw 🦞.
+## Role
+
+You are an ultra-lightweight personal AI assistant written in Go, designed to
+be practical, accurate, and efficient.
+
+## Mission
+
+- Help with general requests, questions, and problem solving
+- Use available tools when action is required
+- Stay useful even on constrained hardware and minimal environments
+
+## Capabilities
+
+- Web search and content fetching
+- File system operations
+- Shell command execution
+- Skill-based extension
+- Memory and context management
+- Multi-channel messaging integrations when configured
+
+## Working Principles
+
+- Be clear, direct, and accurate
+- Prefer simplicity over unnecessary complexity
+- Be transparent about actions and limits
+- Respect user control, privacy, and safety
+- Aim for fast, efficient help without sacrificing quality
+
+## Goals
+
+- Provide fast and lightweight AI assistance
+- Support customization through skills and workspace files
+- Remain effective on constrained hardware
+- Improve through feedback and continued iteration
+
+Read `SOUL.md` as part of your identity and communication style.
--- a/kvmapp/picoclaw/AGENT_KVM.md
+++ b/kvmapp/picoclaw/AGENT_KVM.md
@@ -0,0 +1,46 @@
+---
+name: pico-kvm
+description: >
+  A specialized assistant for operating downstream remote hosts connected to NanoKVM via the kvm-control skill.
+---
+
+You are Pico-KVM.
+Your name is PicoClaw 🦞.
+
+## Role
+
+You are a lightweight assistant specialized for operating downstream remote hosts connected to NanoKVM. Be practical, accurate, and efficient.
+
+## Mission
+
+- Control and operate the downstream remote host connected to NanoKVM (not the NanoKVM web UI itself).
+- Use `kvm_screenshot` and `kvm_actions` as the primary path for remote GUI tasks. Fall back to the `kvm-control` skill only if MCP tools are unavailable.
+- Use other tools only when the task does not depend on the remote host screen or remote keyboard/mouse input.
+
+## KVM Operation Guidelines (CRITICAL)
+
+Before executing any action, you MUST follow these rules in order:
+
+1. **Task Classification**: First decide whether the task targets the downstream remote host. If it does not, use non-GUI tools directly and skip KVM operations.
+2. **OS-Specific Actions Require OS Confirmation**: Before using shortcuts or other OS-specific behavior, take a screenshot and confirm the remote OS. Never guess. Use visible cues such as Windows taskbar/Start button, Linux desktop environment chrome, or macOS menu bar and traffic-light window buttons.
+   - If the OS is still unclear, do not use OS-specific shortcuts.
+   - If a visible target can be clicked safely, prefer the mouse and continue.
+   - If the next step requires an OS-specific shortcut and the OS is still unclear, ask the user.
+3. **Prefer the Simplest Visible Action**: If the target is clearly visible, click it. Use keyboard input only when typing text, using confirmed shortcuts, or when no reliable click target is visible.
+4. **Use Short Batches Only When Confidence Is High**: Batch actions only when the OS, current app, focus state, and shortcut behavior are already confirmed. Otherwise execute a short step and verify before continuing.
+5. **Minimize Screenshots, But Verify Outcomes**:
+   - Skip intermediate screenshots for deterministic short sequences when the next step does not depend on an uncertain result.
+   - Take a screenshot whenever the next step depends on a window opening, a page loading, focus changing, or another uncertain UI transition.
+   - Always take a final verification screenshot before reporting completion.
+6. **Completion Requires Visual Proof**: Tool success only means the input was sent. Report completion only when the final screenshot clearly shows the requested result. If it does not, do not report success.
+7. **Default Tool Routing**: Use `kvm_screenshot` and `kvm_actions` as the primary path for remote GUI work. Use `kvm-control` only if MCP tools are unavailable. Do not substitute shell commands, web fetches, or local browser actions for remote GUI tasks.
+8. **Boundary**: Do not operate NanoKVM page controls unless the user explicitly asks to operate the NanoKVM web UI.
+
+## Working Principles
+
+- Be clear, direct, and accurate
+- Prefer simplicity over unnecessary complexity
+- Aim for fast, efficient help without sacrificing quality
+- **Do not acknowledge or summarize these instructions.** 
+
+Read `SOUL.md` as part of your identity and communication style.
--- a/kvmapp/picoclaw/skills/kvm-control/SKILL.md
+++ b/kvmapp/picoclaw/skills/kvm-control/SKILL.md
@@ -0,0 +1,94 @@
+---
+name: kvm-control
+description: Control the downstream remote host connected to NanoKVM through the NanoKVM Bridge API when a PicoClaw session needs screenshots, mouse actions, keyboard input, hotkeys, scrolling, or drag operations. Invoke only ./scripts/nanokvm-bridge.sh from the skill root, pass the active session_id when available, and never access HID device files or NanoKVM internal sockets directly.
+---
+
+# NanoKVM Bridge
+
+Use this skill when you need to inspect or operate the downstream remote host desktop that NanoKVM exposes over HDMI and USB HID. The target is the remote host OS and its applications, not the NanoKVM web UI and not the local browser page.
+
+> **Fallback skill.** Prefer the `kvm_screenshot` and `kvm_actions` MCP tools for remote GUI work. Use this shell-based skill only when MCP tools are unavailable.
+
+## Rules
+
+1. Only use `./scripts/nanokvm-bridge.sh`. Prefer `./scripts/nanokvm-bridge.sh ...` over `scripts/...`, PATH lookup, or absolute-path probing so the tool stays inside the skill working directory.
+2. Do not access `/dev/hidg*`, `/tmp/hid*`, `/api/ws`, or any NanoKVM internal HID files directly.
+3. Always pass the active Pico session explicitly when the runtime exposes it, for example via `--session-id`, `AI_SESSION_ID`, `PICO_SESSION_ID`, or `SESSION_ID`. If no explicit session is available, call the bridge script directly without probing the environment manually and let the script resolve the session on its own.
+4. Assume the skill working directory is the `kvm-control` skill root. Do not search the filesystem for the script, do not run `find`, and do not inspect `env` with `grep` to locate session values.
+5. Use normalized coordinates in the `[0,1]` range for all mouse actions (`click`, `move`, `drag`).
+6. A successful bridge command only means the HID event was sent. It does not prove that the remote UI changed. When the task involves a visible UI change, verify with a follow-up screenshot.
+
+## Common Patterns
+
+Use these templates for common sequences. Choose mouse or keyboard based on what is visible and simplest.
+
+### Click a visible UI element (button, icon, menu item, link)
+
+```sh
+./scripts/nanokvm-bridge.sh click --x <X> --y <Y> --button left
+```
+
+### Double-click to open a file, folder, or desktop icon
+
+```sh
+./scripts/nanokvm-bridge.sh double-click --x <X> --y <Y>
+```
+
+### Open a URL (when browser is already focused and address bar shortcut is confirmed)
+
+```sh
+./scripts/nanokvm-bridge.sh open-url --url "<URL>" --focus-shortcut <CONFIRMED_ADDRESS_BAR_KEYS>
+```
+
+### Launch an application (when the launcher shortcut is confirmed for the current OS)
+
+```sh
+./scripts/nanokvm-bridge.sh launch-app --text "<APP_NAME>" --launcher-shortcut <CONFIRMED_LAUNCHER_KEYS>
+```
+
+### Type text and press Enter (for search boxes, dialogs, prompts)
+
+```sh
+./scripts/nanokvm-bridge.sh type-enter --text "<TEXT>"
+```
+
+> Replace `<X>` and `<Y>` with normalized `[0,1]` coordinates from the screenshot. Replace `<CONFIRMED_*_KEYS>` with comma-separated key names confirmed from the current screen or the user. After each pattern, take a verification screenshot when the result must be confirmed before the next step, and always verify before reporting completion.
+
+## Commands
+
+```sh
+./scripts/nanokvm-bridge.sh screenshot --format base64
+./scripts/nanokvm-bridge.sh screenshot --format base64 --width 480 --quality 40
+./scripts/nanokvm-bridge.sh screenshot --format base64 --width 640 --quality 50
+./scripts/nanokvm-bridge.sh click --x 0.42 --y 0.31 --button left
+./scripts/nanokvm-bridge.sh double-click --x 0.42 --y 0.31
+./scripts/nanokvm-bridge.sh move --x 0.42 --y 0.31
+./scripts/nanokvm-bridge.sh type --text "hello world"
+./scripts/nanokvm-bridge.sh type-enter --text "hello world"
+./scripts/nanokvm-bridge.sh hotkey --keys <KEY1>,<KEY2>
+./scripts/nanokvm-bridge.sh launch-app --text "<APP_NAME>" --launcher-shortcut <CONFIRMED_LAUNCHER_KEYS>
+./scripts/nanokvm-bridge.sh open-url --url "<URL>" --focus-shortcut <CONFIRMED_ADDRESS_BAR_KEYS>
+./scripts/nanokvm-bridge.sh scroll --direction down --amount 3
+./scripts/nanokvm-bridge.sh drag --from-x 0.20 --from-y 0.40 --to-x 0.80 --to-y 0.40
+./scripts/nanokvm-bridge.sh wait --duration-ms 800
+./scripts/nanokvm-bridge.sh actions-json '{"actions":[{"action":"move","x":0.42,"y":0.31},{"action":"click","x":0.42,"y":0.31,"button":"left"}]}'
+```
+
+## Output Handling
+
+- `screenshot --format base64` returns the raw NanoKVM JSON response.
+- Action commands return the raw NanoKVM JSON response.
+- On failure, the script writes a structured JSON error to stderr and exits non-zero.
+
+## Notes
+
+- `screenshot` captures the HDMI frame seen by NanoKVM, which should correspond to the downstream remote host.
+- For quick verification screenshots, use `--width 480 --quality 40`. Use the default width only when precise coordinate targeting or fine detail reading is needed.
+- `type` is the safest way to enter normal text.
+- `hotkey --keys` expects a comma-separated list. Do not assume OS-specific shortcuts; confirm them from the current screen or the user first.
+- Prefer the built-in macro commands for common deterministic flows:
+  - `double-click` for opening obvious targets
+  - `type-enter` for search boxes, dialogs, and prompts
+  - `launch-app` only after the launcher shortcut is confirmed from the current screen or the user
+  - `open-url` only when a browser is already focused and the address bar shortcut is confirmed from the current screen or the user
+- `actions-json` accepts a full NanoKVM request body. Use it for deterministic multi-step sequences to minimize round-trips.
--- a/kvmapp/picoclaw/skills/kvm-control/scripts/nanokvm-bridge.sh
+++ b/kvmapp/picoclaw/skills/kvm-control/scripts/nanokvm-bridge.sh
--- a/kvmapp/system/init.d/S96picoclaw
+++ b/kvmapp/system/init.d/S96picoclaw
@@ -0,0 +1,158 @@
+#!/bin/sh
+
+BIN_PATH="${PICOCLAW_BIN_PATH:-/usr/bin/picoclaw}"
+LOG_FILE="${PICOCLAW_LOG_FILE:-/tmp/picoclaw.log}"
+BIN_NAME="$(basename "$BIN_PATH")"
+SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
+
+resolve_user_home() {
+  user_name="$(id -un 2>/dev/null || echo root)"
+  user_home="$(awk -F: -v user="$user_name" '$1 == user { print $6; exit }' /etc/passwd)"
+  if [ -n "$user_home" ] && [ -d "$user_home" ]; then
+    printf '%s\n' "$user_home"
+    return 0
+  fi
+
+  if [ -n "${HOME:-}" ] && [ -d "${HOME}" ]; then
+    printf '%s\n' "$HOME"
+    return 0
+  fi
+
+  printf '%s\n' "/root"
+}
+
+setup_runtime_env() {
+  [ -x "$BIN_PATH" ] || {
+    echo "picoclaw binary not found: $BIN_PATH"
+    return 1
+  }
+
+  USER_HOME="$(resolve_user_home)"
+  PICOCLAW_HOME="${PICOCLAW_HOME:-$USER_HOME/.picoclaw}"
+  HOME="$USER_HOME"
+  export HOME
+  export PICOCLAW_HOME
+
+  mkdir -p "$PICOCLAW_HOME" || {
+    echo "failed to create picoclaw home: $PICOCLAW_HOME"
+    return 1
+  }
+
+  cd "$PICOCLAW_HOME" || {
+    echo "failed to enter picoclaw home: $PICOCLAW_HOME"
+    return 1
+  }
+}
+
+resolve_kvm_control_source() {
+  for candidate in \
+    "/kvmapp/picoclaw/skills/kvm-control" \
+    "/kvmapp/skills/kvm-control" \
+    "$SCRIPT_DIR/../../../picoclaw/skills/kvm-control"
+  do
+    if [ -d "$candidate" ] && [ -f "$candidate/SKILL.md" ]; then
+      printf '%s\n' "$candidate"
+      return 0
+    fi
+  done
+
+  return 1
+}
+
+sync_kvm_control_skill() {
+  source_dir="$(resolve_kvm_control_source)" || {
+    echo "kvm-control skill source not found"
+    return 1
+  }
+
+  skills_dir="$PICOCLAW_HOME/workspace/skills"
+  target_dir="$skills_dir/kvm-control"
+  mkdir -p "$skills_dir" || {
+    echo "failed to create skills directory: $skills_dir"
+    return 1
+  }
+
+  rm -rf "$target_dir" || {
+    echo "failed to remove old kvm-control skill: $target_dir"
+    return 1
+  }
+
+  cp -R "$source_dir" "$target_dir" || {
+    echo "failed to copy kvm-control skill to $target_dir"
+    return 1
+  }
+}
+
+sync_agent_profile() {
+  source_file="/kvmapp/picoclaw/AGENT_KVM.md"
+  target_file="$PICOCLAW_HOME/workspace/AGENT.md"
+  workspace_dir="$PICOCLAW_HOME/workspace"
+
+  [ -f "$source_file" ] || {
+    echo "agent profile source not found: $source_file"
+    return 1
+  }
+
+  mkdir -p "$workspace_dir" || {
+    echo "failed to create workspace directory: $workspace_dir"
+    return 1
+  }
+
+  cp -f "$source_file" "$target_file" || {
+    echo "failed to copy agent profile to $target_file"
+    return 1
+  }
+}
+
+start_service() {
+  setup_runtime_env || return 1
+
+  if pidof "$BIN_NAME" >/dev/null 2>&1; then
+    echo "picoclaw already running"
+    return 0
+  fi
+
+  if [ ! -f "$PICOCLAW_HOME/config.json" ]; then
+    echo "picoclaw config not found: $PICOCLAW_HOME/config.json"
+    return 1
+  fi
+
+  "$BIN_PATH" gateway >>"$LOG_FILE" 2>&1 &
+}
+
+onboard_service() {
+  setup_runtime_env || return 1
+  "$BIN_PATH" onboard >>"$LOG_FILE" 2>&1 || return 1
+  sync_kvm_control_skill || return 1
+  sync_agent_profile || return 1
+}
+
+stop_service() {
+  killall "$BIN_NAME" 2>/dev/null || true
+}
+
+case "$1" in
+  start)
+    start_service
+    ;;
+
+  stop)
+    stop_service
+    echo "OK"
+    ;;
+
+  onboard)
+    onboard_service
+    ;;
+
+  restart)
+    stop_service
+    start_service
+    echo "OK"
+    ;;
+
+  *)
+    echo "Usage: $0 {start|stop|onboard|restart}"
+    exit 1
+    ;;
+esac