(This post was originally written in August of 2018.)<\/p>\n
–<\/p>\n
Get a bunch of humans together to make some decisions, and some of them are bound to get it wrong. \u00a0Humans are wildly different from each other, with so many different life experiences that lead up to a decision that there is almost never 100% consensus on anything. \u00a0But it doesn\u2019t stop there. Humans are so different from each other that not only will many of them fail, many of them will fail in <\/span>wildly different ways<\/span><\/i>.<\/span><\/p>\n I swear I\u2019m going somewhere with this, so bear with me.<\/span><\/p>\n This diversity of failure is an important property of a system that I think a lot of people overlook. \u00a0It has two important consequences: First, it means that if we were actually wrong about the best answer in the first place, then there\u2019s a good chance that <\/span>someone<\/span><\/i> will chance on the better solution. \u00a0This gives us a chance to discover this improvement and move to it, or just let the evolution of our answer slowly select for it over many iterations. \u00a0Second, though, it means that we did not create a systemic problem by failing in only one way. It means that even if we undershot our answer some of the time, we did not undershoot our answer <\/span>all<\/span><\/i> of the time – sometimes we overshot as well. \u00a0Hopefully, overshooting some of the time balances out some of the cost of undershooting.<\/span><\/p>\n Let me give an example. \u00a0Let\u2019s say ten people are going to a potluck, and must decide how much food, and what kind, to bring. \u00a0If they don\u2019t bring enough, then people will be hungry. If they bring too much, the leftovers will spoil and go to waste. \u00a0Everyone has some kind of estimate of how much food the party will eat in total. Inevitably, some people are going to overestimate the amount, and some will underestimate it. \u00a0But on the whole, the errors probably balance each other out and the final amount of food is probably within the bounds of acceptable. In fact, if this party continues to have potlucks, and everyone brings a slightly different amount of food every time, it\u2019ll still continue to work out on average, though the variance says that sometimes there will be too much food left over, and sometimes there won\u2019t be enough.<\/span><\/p>\n Now let\u2019s say we design an algorithm to do the same thing. \u00a0It should calculate the amount of food the party wants and buy it for the potluck. \u00a0Since it\u2019s the same people every time, its estimate doesn\u2019t change. (Later we\u2019ll deal with an algorithm that\u2019s slightly more sophisticated, but for now, let\u2019s say it\u2019s deterministic.)<\/span><\/p>\n If the algorithm were “perfect,” it would get this right every time. \u00a0Unfortunately, the chance of this algorithm being perfect is pretty much nil.<\/span><\/p>\n Instead, the algorithm is probably going to be wrong in some direction or another. \u00a0The point is that it will always be wrong <\/span>in the same way<\/span><\/i>. \u00a0There will either always be too much food, or always too little food. \u00a0This creates a <\/span>systemic<\/span><\/i> problem that wasn\u2019t there when we failed in a heterogeneous way.<\/span><\/p>\n \u201cBut wait!\u201d I hear you saying. \u00a0\u201cWouldn\u2019t the party realize that they were always buying too little food, and just tell the algorithm to buy more?\u201d<\/span><\/p>\n That approach works great if the party is in charge of the algorithm. \u00a0But in my example, this algorithm is actually a service used by hundreds of millions of people. \u00a0It\u2019s not going to change its approach just for one party of 10 – that\u2019s more expensive and complicated for it, and anyway, everyone has to keep using the potluck service that their friends use, so no one is going to switch to a more flexible, competing service. \u00a0Sometimes it might not be a service at all, but is instead mandated by the government, but that\u2019s beside the point.<\/span><\/p>\n This doesn\u2019t even get to the problem where the food ordered by the algorithm is almost certainly objectionable to some people due to allergies, religion, or preferences. \u00a0Ordering through that algorithm rather than allowing for a diversity of failures has now locked certain people out of eating at the potluck <\/span>at all<\/span><\/i> – not just in one party, but in <\/span>all of them<\/span><\/i>.<\/span><\/p>\n You could argue that a good food-choosing algorithm wouldn\u2019t do this, but personal experience has confirmed to me that even in scenarios where food is <\/span>trying<\/span><\/i> to cater to everyone involved, it can <\/span>still<\/span><\/i> be true that there are no consumable food options for a lot of the population. \u00a0It\u2019s not a given that a food-choosing algorithm would be sensitive to this kind of thing, especially if the responsible parties are not incentivized to care.<\/span><\/p>\n Okay, so now that we\u2019ve talked about the potluck example, let\u2019s talk about automated content moderation.<\/span><\/p>\n –<\/span><\/p>\n Daniele Citron<\/span><\/a>, a privacy expert and professor at the University of Maryland Francis King Carey School of Law, was invited to give a <\/span>talk<\/span><\/a> at the Cyber Alliance this week on content moderation and how recent developments have given some cause for concern.<\/span><\/p>\n Until recently, Facebook and other social media platforms have taken a First Amendment approach to content moderation, at least in the context of stalking. \u00a0In general, they would err on the side of allowing the speech. In addition, their moderation was done by humans in a fairly ad-hoc manner – a user would report a post, then human moderators would make a decision on whether to allow the post or not. \u00a0The main regulation governing this in the U.S. was (and still is) Section 230 of the Communications Decency Act. This act includes protection for \u201cgood samaritan\u201d under- or over-filtering of content; that is, as long as you can demonstrate that you were trying to do a good job filtering your content, you won\u2019t be punished for being a little overzealous in either direction.<\/span><\/p>\n As of 2015, the main method for making difficult filtering decisions was human moderation. \u00a0Three things changed over the next couple years that irrevocably altered the playing field: The first change accompanied the 2015 terrorist attacks on Paris and Brussels. \u00a0The second was the Russian disinformation campaign during the 2016 U.S. presidential election, and the third was the shockwave as the public learned of companies like Cambridge Analytica and their involvement in elections.<\/span><\/p>\n