Residual RL--MPC for Robust Microrobotic Cell Pushing Under Time-Varying Flow

TL;DR

This paper introduces a hybrid control approach combining MPC with a learned residual policy to enhance the robustness and accuracy of microrobotic cell pushing in dynamic microfluidic flows.

Contribution

It presents a novel residual RL--MPC controller that improves microrobotic manipulation under flow disturbances by integrating a learned policy with traditional MPC.

Findings

Enhanced robustness over pure MPC and PID methods.

Successful generalization to unseen trajectories.

Optimal residual correction bound identified for best performance.

Abstract

Contact-rich micromanipulation in microfluidic flow is challenging because small disturbances can break pushing contact and induce large lateral drift. We study planar cell pushing with a magnetic rolling microrobot that tracks a waypoint-sampled reference curve under time-varying Poiseuille flow. We propose a hybrid controller that augments a nominal MPC with a learned residual policy trained by SAC. The policy outputs a bounded 2D velocity correction that is contact-gated, so residual actions are applied only during robot--cell contact, preserving reliable approach behavior and stabilizing learning. All methods share the same actuation interface and speed envelope for fair comparisons. Experiments show improved robustness and tracking accuracy over pure MPC and PID under nonstationary flow, with generalization from a clover training curve to unseen circle and square trajectories. A residual-bound sweep identifies an intermediate correction limit as the best trade-off, which we use in all benchmarks.