![\"Computers](\"\/cise\/files\/2016\/12\/V.-Saligrama-450x300.jpg\")
It had to happen. In an era when the nation\u2019s president-elect has been routinely criticized for his sexist remarks about women<\/a>, BU researchers, working with Microsoft colleagues, have discovered your computer itself may be sexist. By Rich Below It may say \u201cboss\u201d is a man\u2019s job, BU and Microsoft researchers discover It had to happen. In an era when the nation\u2019s president-elect has been routinely criticized for his sexist remarks about women, BU researchers, working with Microsoft colleagues, have discovered your computer itself may be sexist. Or rather, they\u2019ve discovered […]<\/p>\n","protected":false},"author":1500,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[26],"tags":[132,161],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts\/19953"}],"collection":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/users\/1500"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/comments?post=19953"}],"version-history":[{"count":2,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts\/19953\/revisions"}],"predecessor-version":[{"id":34045,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/posts\/19953\/revisions\/34045"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/media?parent=19953"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/categories?post=19953"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bu.edu\/cise\/wp-json\/wp\/v2\/tags?post=19953"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nOr rather, they\u2019ve discovered that the biased data we fallible humans feed into computers can lead the machines to regurgitate our bias. And there are potential real-world consequences from that.
\nThose findings are in a paper<\/a> produced by the team, whose two BU members are Venkatesh Saligrama<\/a>, a College of Engineering professor of electrical and computer engineering with a College of Arts & Sciences computer science appointment, and Tolga Bolukbasi<\/a> (ENG\u201918).
\nThe team studied\u00a0word embeddings\u2014algorithms\u00a0that one member of the team described to National Public Radio<\/a> as dictionaries for computers.\u00a0Word embeddings allow computers to make word associations. To take a hypothetical example NPR used,\u00a0a tech company looking to hire a computer programmer can use an embedding that knows \u201ccomputer programmer\u201d is related to terms such as \u201cJavaScript\u201d or \u201cartificial intelligence.\u201d A computer program with that word embedding could cull r\u00e9sum\u00e9s that contain such related words. So far, so harmless.
\nBut word embeddings can recognize word relationships only by studying batches of writing. The researchers particularly focused on\u00a0word2vec,\u00a0a publicly accessible embedding nourished on texts from Google News<\/a>, an aggregator of journalism articles. Turns out that those articles contain gender stereotypes, as the researchers found when they asked the embedding to find analogies similar to \u201che\/she.\u201d
\nThe embedding spit back worrisome analogies involving jobs. For \u201che\u201d occupations, it came up with words like \u201carchitect,\u201d \u201cfinancier,\u201d and \u201cboss,\u201d while \u201cshe\u201d\u00a0jobs included\u00a0\u201chomemaker,\u201d \u201cnurse,\u201d and \u201creceptionist.\u201d
\nTheoretically, these distinctions could promote real-world inequity. Companies increasingly rely<\/a> on computer software to analyze job applications. Say that hypothetical tech company seeking a computer programmer used embeddings to weed through r\u00e9sum\u00e9s.
\n\u201cWord embeddings also rank terms related to computer science closer to male names than female names,\u201d the BU-Microsoft team says in its paper, to be presented at this week\u2019s Neural Information Processing System<\/a> (NIPS) conference in Barcelona, the top annual meeting on machine learning . \u201cIn this hypothetical example, the usage of word embedding makes it even harder for women to be recognized as computer scientists and would contribute to widening the existing gender gap in computer science.\u201d
\n\u201cThese are machine learning algorithms that are looking at documents, and whatever bias exists in our everyday world is being carried into these word embeddings,\u201d Salingrama says. \u201cThe algorithm itself is pretty agnostic. It doesn\u2019t care whether there exists an underlying bias or no biases in the document itself.\u2026It is just picking up on what words co-occur with what other words.\u201d The bias is in the data set being examined, like Google News.
\n\u201cWhat our paper uncovers is that just because a machine does stuff agnostically doesn\u2019t mean it is going to be unbiased.\u2026What machine learning is about is: you look at the world, and then you learn from the world. The machine is also going to learn biases that exist in the world it observes.\u201d
\nThe researchers didn\u2019t just decide on their own which pairings were sexist and which weren\u2019t; they ran each analogy<\/a> by 10 people using Amazon Mechanical Turk<\/a>, the crowdsourcing online marketplace. If a majority deemed an analogy sexist, the researchers accepted their judgment.
\nThe researchers say that they wrote their own algorithms that maintain appropriate gender-based associations while screening out sexist stereotypes. \u201cIt sounds like an ugly problem, because there are many, many, many, many words, and it seems very hard to go individually and remove these biases,\u201d says Saligrama. But the computer\u2019s ability to make word associations enables it, when fed some biased words, to predict other words that could be similarly sexist, he says. \u201cSo it is able to then remove biases\u2026without a human being performing, word by word, the whole dictionary.\u201d
\nThey will make their algorithms publicly available shortly on the computer code-sharing platform GitHub<\/a>, he and Bolukbasi say.
\nThe two plan to pursue additional research. They\u2019ve begun to look at racial bias in Google News articles, and they hope to expand their study beyond English. \u201cWe\u2019ve thought about how to quantify bias among different languages, when you look at gender or when you look at bias,\u201d says Saligrama. \u201cDo certain languages have more bias versus others? We don\u2019t know the answer to that.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"