Humanoid Robot Perception & Control
Humanoid Robot Perception & Control Integration Internship
Project Overview
The project built a visual-following system on the Fourier N1 humanoid robot platform. The core pipeline runs object detection on a remote Orin compute unit and sends results back to the robot for arm control — a design driven by the N1's limited onboard compute. The system enables the robot to detect a target object, estimate its 3D position, and track it with the right arm.
The algorithm design of the visual following pipeline was led by the team. My involvement focused on system integration, UI development, and on-site engineering work described below.
My Contributions
Algorithm understanding & system integration
- Studied and documented the visual-following algorithm architecture: RGB-D capture → YOLO detection → 3D centroid estimation → arm joint command
- Integrated components across the N1-to-Orin communication link, understanding how each module interacts
Web UI & frontend–backend connection
- Developed the web-based control and monitoring UI
- Connected the frontend status display to the backend control system
- Added RTT (round-trip time) monitoring so latency between Orin inference and robot control was visible during debugging
SLAM & environment mapping
- Used SLAM (RTAB-Map) for offline environment mapping during bring-up sessions
- Managed rosbag recording for data collection
On-site debugging
- Resolved camera device ownership conflicts between multiple processes
- Fixed network routing and ROS topic remapping issues across the N1–Orin link
- Diagnosed and cleared process scheduling conflicts during live bring-up
System architecture diagram, UI screenshots, and demo footage will be added here.
人形机器人感知与控制联调 实习
项目内容
本项目在 Fourier N1 人形机器人平台上构建视觉跟随系统。核心链路将目标检测运行在远端 Orin 算力节点上，推理结果传回机器人用于手臂控制——这一设计由 N1 本机算力有限决定。系统实现了机器人检测目标物体、估计其 3D 位置并以右臂进行跟随。
视觉跟随算法的设计由团队主导；我的参与主要集中在系统集成、UI 开发与现场工程工作，详见下方。
我的工作
算法理解与系统集成
- 学习并梳理视觉跟随算法架构：RGB-D 采集 → YOLO 检测 → 3D 中心估计 → 手臂关节指令
- 在 N1 到 Orin 的通信链路中集成各模块，理解各组件间的交互方式
Web UI 与前后端连接
- 开发基于 Web 的控制与监控 UI 界面
- 打通前端状态展示与后端控制系统的连接
- 增加 RTT（往返时间）监控，使调试时 Orin 推理与机器人控制之间的延迟可视化
SLAM 与环境建图
- 使用 SLAM（RTAB-Map）进行现场 bring-up 阶段的离线环境建图
- 管理 rosbag 数据录制
现场联调
- 解决多进程间的相机设备归属冲突
- 修复 N1–Orin 链路上的网络路由与 ROS topic 重映射问题
- 诊断并处理现场 bring-up 过程中的进程调度冲突
系统架构图、UI 截图与 Demo 视频将在后续补充。