Archived
This content is available here for research, reference, and/or recordkeeping.
Abstract
This research develops and implements a novel reinforcement learning (RL) architecture to address the trajectory-tracking problem in bipedal robotic systems under articulated-joint constraints. The proposed RL framework extends previously designed adaptive controllers characterized by state-dependent gain structures. The learning mechanism comprises two hierarchical adaptation layers: the first employs an adaptive dynamic programming (ADP) formulation to approximate the Bellman value function using a class of continuous-time dynamic neural networks. In contrast, the second uses an iterative optimization scheme based on the deep deterministic policy gradient (DDPG) algorithm. The resulting control strategy minimizes a robust performance index defined over the tracking trajectories of a system with uncertain and nonlinear dynamics representative of bipedal locomotion. The dynamic programming formulation ensures robustness to bounded parametric uncertainties and external perturbations. By approximating the Hamilton–Jacobi–Bellman (HJB) value function using neural network structures, a closed-loop controller design is systematically established. Numerical simulations demonstrate the convergence of the tracking error to a region centered at the origin with a size that depends on the approximation quality of the selected neural network. To assess the effectiveness of the proposed approach, a conventional state-feedback control design is adopted as a benchmark, revealing that the suggested method produces a lower cumulative tracking error norm (0.023 vs. 0.037 rad·s) in the trajectory-tracking control problem for all robotic joints while simultaneously reducing the control effort required to complete motion tasks.
Document Type
Article
Publication Date
2026
Digital Object Identifier (DOI)
https://doi.org/10.3390/app16010001
Funding Information
This research was financially supported by the Tecnologico de Monterrey Challenge-Based Research Program project ID IJXT070-23EG60002.
Repository Citation
Rincon-Martinez, Karla; Yu, Wen; and Chairez, Isaac, "Adaptive Control for a Robotic Bipedal Device Using a Hybrid Discrete-Continuous Reinforcement Learning Strategy" (2026). Mathematics Faculty Publications. 50.
https://uknowledge.uky.edu/math_facpub/50
Notes/Citation Information
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.