With bots and fake news influencing elections, algorithms replacing bail hearings, and biased risk assessments shaping sentencing, we are beginning to grapple with the reality that automated decision-making systems are responsible for some of our most profound political decisions. In response, the data science community is having a slow-burn ethical crisis.
Tom Simonite at Wired recently reported on an effort to create a code of ethics for data science at the Data for Good Exchange on February 6. DJ Patil, Chief Data Scientist of the United States Office of Science and Technology Policy under President Obama, referred to the event as the “Constitutional Convention” for the field.
The draft they released of Community Principles for Ethical Data Sharing, produced by representatives primarily from industry and academia, offers some ambitious, if vague, elementary principles, including:
- We will help foster a data science community culture that is actively open, welcoming, and inclusive for people from diverse backgrounds;
- [We will] disclose how data is processed and stored, and remove data related to race, gender, disability, and religion, where possible [to mitigate bias]; and
- I will consider my responsibility to solve problems of consequence to people and society.
You can witness the field’s baby steps towards a list of ethical principles here, or contribute to the ongoing, crowd-sourced effort here.
Though I believe a professional code of conduct should be an internal effort, led by those actively doing data science day to day, I think the targets of the most invasive and punitive automated decision-making systems also have valuable insight to offer.
In my reporting for Automating Inequality, families told me story after story about how the digital tools they interact with worsen bias, subvert democracy, and violate their basic human rights.
Their experiences suggest that we need to think more broadly about who the “community of data science” includes. These systems impact us all – but they don’t impact us equally. Without including the voices and experiences of those who face automation’s most dire affects, data scientists may miss the opportunity to push the field towards social justice.
Oath of Digital Non-Harm
I swear to fulfill, to the best of my ability, the following covenant:
I will respect all people for their integrity and wisdom, understanding that they are experts in their own lives, and will gladly share with them all the benefits of my knowledge.
I will use my skills and resources to create bridges for human potential, not barriers. I will create tools that remove obstacles between resources and the people who need them.
I will not use my technical knowledge to compound the disadvantage created by historic patterns of racism, classism, able-ism, sexism, homophobia, xenophobia, transphobia, religious intolerance, and other forms of oppression.
I will design with history in mind. To ignore a four-century- long pattern of punishing the poor is to be complicit in the “unintended,” but terribly predictable consequences that arise when equity and good intentions are assumed as initial conditions.
I will integrate systems for the needs of people, not data. I will choose system integration as a mechanism to attain human needs, not to facilitate ubiquitous surveillance.
I will not collect data for data’s sake, nor keep it just because I can.
When informed consent and design convenience come into conflict, informed consent will always prevail.
I will design no data-based system that overturns an established legal right of the poor.
I will remember that the technologies I design are not aimed at data points, probabilities, or patterns, but at human beings.
2 thoughts on “A Hippocratic Oath for Data Science”