{"id":40738,"date":"2024-03-18T14:08:37","date_gmt":"2024-03-18T18:08:37","guid":{"rendered":"https:\/\/www.bu.edu\/cise\/?p=40738"},"modified":"2024-03-20T13:33:50","modified_gmt":"2024-03-20T17:33:50","slug":"reinforcement-learning-a-more-efficient-way-for-robots-to-learn","status":"publish","type":"post","link":"https:\/\/www.bu.edu\/cise\/reinforcement-learning-a-more-efficient-way-for-robots-to-learn\/","title":{"rendered":"Reinforcement Learning: A More Efficient Way for Robots to Learn"},"content":{"rendered":"
\"\"
Vittorio Giammarino, PhD Candidate (SE)<\/figcaption><\/figure>\n

Robot nurses \u2014 myth or reality? Although this may sound far-fetched, there are already hospitals in which robots assist nurses by bringing them tools, allowing the nurses to focus on providing care to their patients more efficiently. Vittorio Giammarino, a fifth-year PhD candidate (SE) at Boston University, hopes that his work can be useful for applications like these.\u00a0<\/span><\/p>\n

At the Center for Information and System Engineering (CISE), Giammarino is working under Ioannis Paschalidis, the Director of the Rafik B. Hariri Institute for Computing and Computational Science & Engineering and a Distinguished Professor of Engineering, on the Multidisciplinary University Research Initiative (MURI) grant from the Department of Defense entitled <\/span>Neuro-Autonomy: Neuroscience-Inspired Perception, Navigation, and Spatial Awareness for Autonomous Robots<\/span><\/i><\/a>.<\/span>\u00a0<\/span><\/p>\n

Giammarino\u2019s work centers around machine learning, more specifically reinforcement learning,\u00a0 a method for robots to learn how to select good actions. \u201cBefore, control was mainly mathematical, and now we realize that modeling everything is hard,\u201d explains Giammarino. \u201cSo what we try to do is let the agent interact with the environment and try to learn by itself, by trial and error, or experience.\u201d<\/span><\/p>\n

Designing robots that can learn through reinforcement learning \u2014 as humans and animals do \u2014 would make robots more computationally efficient. \u201cWe want to build algorithms that do not require all this data, cleaning and annotation and all the expenses behind collecting data when there is something that can be built and can be cheaply put into the field and can learn by itself,\u201d Giammarino says.\u00a0<\/span><\/p>\n

However, Giammarino and his collaborators quickly realized that this would not be an easy feat; in humans and animals, the learning process takes a lifetime and is largely inefficient. To combat this inefficiency, Giammarino and his collaborators decided to use behavioral data from humans and animals to enhance and improve this learning process for robots, starting with simple experiments such as getting from point A to point B in a room.<\/span><\/p>\n

During the research process, Giammarino\u2019s role is to begin by thinking about the problem and how to address it. To do this, he and his collaborators come up with a research question and then look for solutions to these questions from data. Their data comes from experiments involving humans and animals, many of which are conducted in the Center for Systems Neuroscience labs led by Michael Hasselmo and Chantal Stern. The first step is to leverage imitation learning techniques, getting the robot to imitate what animals in humans do in similar settings. Then, they try to decrease the complexity of their algorithm \u2014 making it easier and quicker for the robot to follow \u2014 before letting it improve upon what it has learned through reinforcement learning. Once these are developed, neuroscientists become involved, followed by a discussion and defense phase, during which the team looks for problems in their work and tries to improve upon them.\u00a0<\/span><\/p>\n

\n\/cise\/files\/2024\/03\/rodent_cropped0001-2500.mp4<\/a><\/video><\/div>\n

 <\/p>\n

One recently published paper that Giammarino worked on, <\/span>\u201cOpportunities and Challenges from Using Animal Videos in Reinforcement Learning for Navigation,\u201d<\/span><\/a> was presented in <\/span>Yokohama, Japan last summer. In the paper, he and his collaborators focus on the problem of imitation learning from visual observations, where the learning agent has access to videos of experts as its sole learning source. Further, they address the challenges that arise from this framework and describe how they plan to tackle these problems. Currently,<\/span> Giammarino is working on the paper, <\/span>\u201cAdversarial Imitation Learning from Visual Observations using Latent Information,\u201d<\/span><\/a> in which he investigates the use of observations in animal videos to improve reinforcement learning efficiency and performance in navigation tasks.<\/span><\/p>\n

\/cise\/files\/2024\/03\/humanoid_comparison.mp4<\/a><\/video><\/div>\n

 <\/p>\n

\u201cVittorio is working on a very challenging and important set of problems,\u201d said Paschalidis, Giammarino\u2019s Ph.D. advisor. \u201cHe has made important progress, particularly tackling the very realistic setup where one may not be able to observe the true internal states and corresponding actions of an expert we wish to imitate,\u201d added Paschalidis.\u00a0 <\/span><\/p>\n

Giammarino completed his undergraduate studies at the University of Bologna in Italy <\/span>and Tongji University in China, where he majored in Automation Engineering. He also received a Master of Science from the Delft University of Technology in Systems and Control. In the future, he hopes to pursue a career in an industrial-related field such as robotics, recommendation systems, or energy.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Robot nurses \u2014 myth or reality? Although this may sound far-fetched, there are already hospitals in which robots assist nurses by bringing them tools, allowing the nurses to focus on providing care to their patients more efficiently. Vittorio Giammarino, a fifth-year PhD candidate (SE) at Boston University, hopes that his work can be useful for […]<\/p>\n","protected":false},"author":23345,"featured_media":40770,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[205],"tags":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts\/40738"}],"collection":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/users\/23345"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/comments?post=40738"}],"version-history":[{"count":5,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts\/40738\/revisions"}],"predecessor-version":[{"id":40788,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts\/40738\/revisions\/40788"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/media\/40770"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/media?parent=40738"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/categories?post=40738"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/tags?post=40738"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}