Date Available
12-12-2024
Year of Publication
2024
Document Type
Doctoral Dissertation
Degree Name
Doctor of Philosophy (PhD)
College
Engineering
Department/School/Program
Computer Science
Advisor
Dr. Brent Harrison
Abstract
Reinforcement Learning (RL) is an approach to allowing computer agents to try and learn how to solve problems by learning what actions are best to take in a given situation. RL is effective for learning what to do in an environment, but as the problem grows larger, the amount of information needed grows exponentially, making RL less effective on complex problems. A big challenge, often called the curse of dimensionality, is that the number of states and possible number of actions in an environment can grow too large to sufficiently test every possible combination of state and action. One method that has been used to improve the performance of RL is to introduce hierarchies into the learning algorithms. The hierarchies impart guidance to the agent, attempting to limit the RL agents from performing tasks that do not benefit it in achieving good results. By limiting the tasks available this makes the problem smaller and more tractable. A big part of the challenge for using hierarchies in RL has been the creation of hierarchies. Typically hierarchies are hand-crafted by humans and custom created for each problem. I discuss methods that use prior successful episodes, called traces, to directly convert them into a hierarchy that an agent can use to learn. I also look at methods for using natural language as a descriptive language for the hierarchy and then convert the natural language representation to a hierarchy. Once a hierarchy is created I developed a way to apply the hierarchy into advanced neural networks, allowing them to be used in more complex environments. Based on the results from my research I show that there are multiple ways to generate hierarchies and they can be applied to a variety of environments, including complex text-based games.
Digital Object Identifier (DOI)
https://doi.org/10.13023/etd.2024.490
Recommended Citation
Mobley, Roy, "Finding Hierarchies to Improve Learning in Hierarchical Reinforcement Learning" (2024). Theses and Dissertations--Computer Science. 147.
https://uknowledge.uky.edu/cs_etds/147
