Archived
This content is available here strictly for research, reference, and/or recordkeeping and as such it may not be fully accessible. If you work or study at University of Kentucky and would like to request an accessible version, please use the SensusAccess Document Converter.
Date Available
5-7-2021
Year of Publication
2021
Document Type
Master's Thesis
Degree Name
Master of Science (MS)
College
Engineering
Department/School/Program
Computer Science
Faculty
Dr. Brent Harrison
Faculty
Dr. Zongming Fei
Abstract
We present Markov Decision Processes with Embedded Agents (MDPEAs), an extension of multi-agent POMDPs that allow for the modeling of environments that can change the actuators, sensors, and learning function of the agent, e.g., a household robot which could gain and lose hardware from its frame, or a sovereign software agent which could encounter viruses on computers that modify its code. We show several toy problems for which standard reinforcement-learning methods fail to converge, and give an algorithm, `just-copy-it`, which learns some of them. Unlike MDPs, MDPEAs are closed systems and hence their evolution over time can be treated as a Markov chain. In future work, we hope MDPEAs can be extended to model even fully embedded agents acting in real digital or physical environments.
Digital Object Identifier (DOI)
https://doi.org/10.13023/etd.2021.135
Recommended Citation
Miles, Luke Harold, "Markov Decision Processes with Embedded Agents" (2021). Theses and Dissertations--Computer Science. 106.
https://uknowledge.uky.edu/cs_etds/106
