This dissertation presents algorithms that allow robots to generate optimal behavior from first principles. Instead of hard-coding every desired behavior, we encode the task as a cost function, and use numerical optimization to find action sequences that can accomplish the task. Using the theoretical framework of optimal control, we develop methods for generating autonomous motor behavior in high-dimensional domains of legged locomotion. We identify three foundational problems that limit the

more »

... lication of existing optimal control algorithms, and present guiding principles that address these issues. First, some traditional algorithms use global optimization, where every possible state is considered. This approach cannot be applied in continuous domains, where every additional mechanical degree of freedom exponentially increases the volume of state space. In order to sidestep this curse of dimensionality, we focus on trajectory optimization, which finds locally-optimal solutions while scaling only polynoimally with state dimensionality. Second, many algorithms of optimal control and reinforcement ii More people than I can count contributed to this project, and it's hard to express my gratefulness to all of them. First and foremost, I owe my family infinite gratitude for their support and continuing inspiration: my grandfather, who was a biology teacher, my parents, who are avid readers and thinkers, and my wife, Anna, who shares my passion and devotion to the academic world. I feel fortunate to have spent the past few years in St. Louis, and I am grateful for many things: the communal lifestyle, the extended social circle of the MFA students in creative writing, and the overall amiable atmosphere. I felt at home in the Media and Machines lab, where I had many insightful conversations with friends and colleagues, including (but not limited to) Michael Dixon, Nathan Jacobs, and Austin Abrams. The superbly-managed CSE department always took good care of me, and I thank