BU\u2019s Center for Philosophy & History of Science<\/a>. \u201cAlso, because I work on long-term evolution, when I consider all the people worried about the long-term impact of AI, I wonder what evolutionary theory has to offer about some of those questions.\u201d<\/p>\nArts x Sciences<\/em> spoke with Powell about philosophical and ethical considerations surrounding the development and use of artificial intelligence.<\/p>\nArts x Sciences<\/em>: What about AI interests you, given your background in philosophy?<\/strong>
\nRachell Powell:<\/strong> There\u2019s obviously a vast range of different kinds of AI. They\u2019re going to have radically different implications, and there are different approaches to dealing with their philosophical issues. To simplify things, one thing we can say is that there\u2019s this big bifurcation between the people who are concerned about the shorter-term risks of imaginable AI\u2014or AI that\u2019s being implemented right now or might be implemented in the coming months and years\u2014and the people who are working on the really theoretical, long-term risk sides of AI.<\/p>\nThe question for me is, how can my work on the evolution of cognition, evolutionary theory, and enhancement ethics help illuminate some of these issues?<\/p>\n
Do you think there\u2019s a right and wrong side to this issue?
\n<\/strong>The people who are concerned about the short-term risks focus on these big issues that we need to deal with right now. And they don\u2019t think we should spend too much time talking about what some people call artificial general intelligence or sentient AI or super intelligences, or about these big risks to humanity. I think there\u2019s something very reasonable about that view.<\/p>\nThen there are a lot of scholars working in this area of philosophy called longtermism. It involves hot-button and controversial philosophical and moral theories, including that we really need to think about ethics for long-term humanity existence. Because when you do the ethical computations, and you add up all the good stuff that comes out of [the long-term thinking], it\u2019s going to dwarf all the good stuff that we get right now from making these small, shorter-term tweaks to our societies. They believe we really need to think about the extinction of the species due to these kinds of super intelligences. Because if we\u2019re gone, our entire future is gone.<\/p>\n
Our conceptual resources are finite. Our monetary resources are finite. So, these two sides are competing for resources and attention. I understand the concern on both sides, actually.<\/p>\n
My interest is in the existential risks that AI might pose to continued species\u2019 existence or to life on the planet.<\/p>\n
Rachell Powell<\/p>\n<\/blockquote>\n
What are the moral and ethical considerations?
\n<\/strong>My interest is in the existential risks that AI might pose to continued species\u2019 existence or to life on the planet, but also these deeper theoretical questions about things like the nature of intelligence, the nature of mind, the moral status of increasingly sophisticated machines. What is going to be their moral political place in our society?<\/p>\nThere are three interrelated aspects of the more theoretical side of AI that interest me in particular as a philosopher of biology and as a bioethicist. One is what people call the control problem, how you\u2019re going to control AI. Another is the nature and evolution of intelligence. And the third one is the moral status of these machines, because I work on the moral status of nonhuman animals, embryos, people with brain damage, and so forth. These are all related things.<\/p>\n
Let\u2019s start with the control problem and go from there.
\n<\/strong>Part of the control problem is that, okay, we\u2019re going to make these increasingly sophisticated machines. Well, how are we going to be able to control what they do or want? How are we going to be able to control what they become? And those are big, big, big, big problems.<\/p>\nOne might wonder why these are philosophical problems or how they relate to the philosophy of biology. Here\u2019s the way I see it: One of the big differences so far between living things and machines is that machines don\u2019t really want anything. They\u2019re not motivated to do things. They follow executing programs, but they don\u2019t really have drive. Living things have drives; they want stuff. Things that happen to them have meaning to them.<\/p>\n
So far, we haven\u2019t been able to design machines with these kinds of agencies and these kinds of motivations. They can do very sophisticated things, like make art and dominate chess. They also do a lot of really dumb things. But I don\u2019t like slamming the dumb stuff, because eventually they are not going to do those dumb things anymore. And then who\u2019s laughing?<\/p>\n
What about the evolution of AI?
\n<\/strong>One of the key questions is whether we\u2019re going to be able to design AIs with these kinds of motivational systems. And if we do, then are those motivational systems going to be persistent under self-modification and self-improvement? In other words, when these machines want to improve their functionalities, if they have access to their own source codes and they can make modifications that they deem to increase efficiency, for example, are we going to be able to control the stability of what they want?<\/p>\nAnd if these AI systems develop agency, then what?
\n<\/strong>The difference here is that these AI systems are also going to be smart as hell. In some respects, they\u2019re going to be smarter than us. If they have access to their own design, to their own source codes, they might modify their motivations. And if we try to put impediments to that, they may see that as a threat, and they could try to maximize their utility functions, however those have been set, and we don\u2019t even know that that\u2019s going to remain stable.<\/p>\nI sometimes think about the classic movie 2001: A Space Odyssey<\/em>. There\u2019s this AI on the spaceship, and in a famous scene, the astronaut is trying to shut down the AI, and the AI says, \u201cJust what do you think you are doing, Dave?\u201d I think that\u2019s legit. Machines may begin to resist these kinds of modifications. This becomes a really serious set of questions that we need to deal with.<\/p>\nI think when machines are successfully embodied\u2014which they\u2019re not right now\u2014then all bets will be off. At that point, you\u2019re going to have this robust living system with really key differences between what we\u2019ve normally been dealing with, and I don\u2019t know that we will be able to control the outcomes. At some point, AI will cross that threshold, with machines becoming more embodied and starting to actually care about things and being able to get around in the world.<\/p>\n
![\"Rachell](\"\/cas\/files\/2023\/11\/IMG_8257-cropped.jpg\")