Spatial-Temporal Whole-Body Motion Control for Mobile Manipulator

Project Background

During my internship at AgileX Robotics, I was responsible for developing the simulation environment, motion control system, state estimator, and ROS driver for an omnidirectional mobile manipulator. The goal of this project was to extend the workspace for data-collection for manipulation.

Methods

Our technical approach includes the following key components:

System Modeling and Simulation
- Physics-based kinematic and dynamic modeling
- High-fidelity simulation using Isaac Sim and Isaac Lab
Whole-Body Motion Control
- Optimization algorithms based on model predictive control (using ocs2 platform)
- Learning algorithms using asymmetric PPO
State Estimation
- Odometry estimation based on Lidar-Inertial Odometry
- Velocity estimation based on Extended Kalman Filter
- Real-time motion state monitoring
System Integration
- Development of a complete ROS driver package, including:
  - ROS2 driver for robot base
  - ROS driver for robot pillar
  - ROS2 driver for robot arm
  - ros1 noetic to ros2 humble bridge
- Dockerize the deployment procedure

Key Achievements

The main achievements of this project include:

Validation of the control effectiveness of reinforcement learning and MPC
Implementation of end-effector trajectory tracking in both simulation and physical environments
Integration of a complete ROS driver package and control system into the company's product line

Application Scenarios

Data collection
Indoor service scenarios within human workspaces

Technical Challenges and Solutions

During the project, we faced and resolved several technical challenges:

Calibration: Ensured coordinated movement of the chassis and manipulator through a unified optimization framework
State Estimation Accuracy: Improved state estimation accuracy using multi-sensor fusion strategies
Dynamic Modeling and Simulation: It is quiet a challenge to model omnidirectional wheel base dynamics without CAD information. Thus, I gave up and focused on the kinematics.
Rotation Accuracy in Learning Methods: Used a more precise but less generalizable MPC instead of Reinforcement Learning

Summary and Reflection

This internship experience allowed me to deeply understand the complexity of mobile manipulator systems and independently complete the entire process from theoretical modeling to actual deployment. By solving real engineering problems, I enhanced my system thinking and ability to tackle complex issues.