Intelligent control of fish-like robots by data-driven dynamic modeling and reinforcement learning

Fish-like robots have advantages over traditional underwater vehicles, in performances like maneuverability, agility, and energy efficiency. However, the motion planning and control of fish-like robots is challenging due to the complexity of dynamic modeling, the intricate three-dimensional fluid-structure interaction, system nonlinearity, the added-mass effect, and the underactuated dynamics system nature. Traditional methods rely on surrogate modeling, kinematics-based dynamic modeling, and CFD-simulations-based modeling, which is either of low fidelity or requires massive computation power. While fish don't know the theory of fluids, body dynamics equations, or fluids/structure interactions simulations, they are masters of swimmers. Moreover, they learned all the skills by interacting with the environment. Inspired by this, we propose to use an artificial neural network (ANN) to directly fit the two-dimensional dynamics model of the fish-like robot and its interaction with surrounding environments. The high fidelity of the ANN is validated by experiments and used as a simulation environment. Deep Reinforcement learning, specifically, PPO (Proximal Policy Optimization) is used to train the fish to reach a static target with pose requirements in this ANN-based simulation environment. Experimental results have validated the effectiveness of the data-driven dynamic model. The fish-like robot can reach the preset goals with a positional accuracy of 0.05 meters and an orientation tolerance of 0.25 radians. The new method can also be potentially used to train other robotic systems to achieve control tasks.