I know how models are built, because I build them myself, so I know that I'm embedding my values into every single algorithm I create and I am projecting my agenda onto those algorithms. Cathy O'Neil More Quotes by Cathy O'Neil More Quotes From Cathy O'Neil Occupy provided me a lens through which to see systemic discrimination. Cathy O'Neil lenses discrimination The most important goal I had in mind was to convince people to stop blindly trusting algorithms and assuming that they are inherently fair and objective. Cathy O'Neil goal mind people The training one receives when one becomes a technician, like a data scientist - we get trained in mathematics or computer science or statistics - is entirely separated from a discussion of ethics. Cathy O'Neil training data statistics An insurance company might say, "Tell us more about yourself so your premiums can go down." When they say that, they're addressing the winners, not the losers. Cathy O'Neil winner loser might That's what we do when we work in Silicon Valley tech startups: We think about who's going to benefit from this. That's almost the only thing we think about. Cathy O'Neil valleys benefits thinking People are starting to be very skeptical of the Facebook algorithm and all kinds of data surveillance. Cathy O'Neil algorithms data people People felt like they were friends with Google, and they believed in the "Do No Evil" thing that Google said. They trusted Google more than they trusted the government, and I never understood that. Cathy O'Neil government evil people The NSA buys data from private companies, so the private companies are the source of all this stuff. Cathy O'Neil nsa data stuff The public trusts big data way too much. Cathy O'Neil data too-much way I think there's inherently an issue that models will literally never be able to handle, which is that when somebody comes along with a new way of doing something that's really excellent, the models will not recognize it. They only know how to recognize excellence when they can measure it somehow. Cathy O'Neil issues excellence thinking You'll never be able to really measure anything, right? Including teachers. Cathy O'Neil including able teacher I think big data companies only like good news. So I think they're just hoping that they don't get sued, essentially. Cathy O'Neil news data thinking Google is so big you have no idea what a given person does. Cathy O'Neil google doe ideas For whatever reason, I have never separated the technical from the ethical. Cathy O'Neil ethical reason My fantasy is that there is a new regulatory body that is in charge of algorithmic auditing. Cathy O'Neil fantasy body I set up a company, an algorithmic auditing company myself. I have no clients. Cathy O'Neil company clients I wanted to prevent people from giving them too much power. I see that as a pattern. I wanted that to come to an end as soon as possible. Cathy O'Neil patterns giving people With recidivism algorithms, for example, I worry about racist outcomes. With personality tests [for hiring], I worry about filtering out people with mental health problems from jobs. And with a teacher value-added model algorithm [used in New York City to score teachers], I worry literally that it's not meaningful. That it's almost a random number generator. Cathy O'Neil new-york teacher meaningful Obviously the more transparency we have as auditors, the more we can get, but the main goal is to understand important characteristics about a black box algorithm without necessarily having to understand every single granular detail of the algorithm. Cathy O'Neil black important goal Especially from my experience as a quant in a hedge fund - I naively went in there thinking that I would be making the market more efficient and then was like, oh my God, I'm part of this terrible system that is blowing up the world's economy, and I don't want to be a part of that. Cathy O'Neil would-be world thinking
