ALG Blog Post 1: Exceptions to a Data Driven Rule

My response to an case study that discusses the ethical problems of data driven algorithms.

Case Study:
The Right to Be an Exception to a Data-Driven Rule

Summary

The case study talks about how biases can creep into data driven algorithms, and allow them to make decisions that are rooted in stereotypes. The case study argues that data driven algorithms can be useful. However this requires that data driven algorithms are certain only if the levels of individualization and certainty are high enough to justify the level of harm that would result from that recommendation.

Discussion Questions

1. What is a data-driven rule, and what does it mean to be a data-driven exception? Is an exception the same as an error?

Data driven rules are guidelines built into complex sorting algorithims. Data driven exceptions are if an element that is sorted, is put into an incorrect group. Exceptions are wrong but not errors. Humans decide what is and is not an wrong based on caveats. These algorithims are not human in the sense that they cannot apply these exceptions to the sorting algorithims.

1. In addition to those listed above, what other factors differentiate data-driven decisions from human ones?

Humans use context and slowly move through data. Algorithims are fast and don’t look at each context that corresponds with the data, they match and sort. Because algorithims look at large data sets and do not look at each case deeply, the unwanted sorting can be amplified.

1. Beyond what is discussed above, what are some of the benefits and downsides of individualization?

Individualization can narrow down sorting algorithims consideration with more context, thus making more fair well rounded decisions. This however can risk your privacy, by requiring more data.

1. Why is uncertainty so critical to the right to be an exception? When the stakes are high (e.g., in criminal sentencing), is there any evaluation metric (e.g., accuracy) that can justify the use of a data-driven rule without the consideration of uncertainty?

Uncertainty is crucial, because if you have someone that is being sentenced to death, depending on the uncertainty, they could be killed unjustifiably. There is no one metric to justify the use of a data driven rules, because of the massive number of incalculable variables.

New Question

Is it ethical to use data driven rules when the uncertainty is high but the stakes are low, like music recommendations?

I chose this question because I find music recommendations very helpful and I don’t see a problem with it. In fact the uncertainty can sometimes show me music That I would have never saught out. However there may be concerns for privacy and data collection.

Reflection

Reading through this case study made me think about certainty and how it is unethical to be uncertain when taking action that could result in harm. In a perfect world you can be a law abiddening citizen and not have to worry about data driven algorithms making incorrect sorts, or judgements when you are trying to get a loan or a job. From what I have gathered it seems to me that these can be used in cases where the harm does not outweigh the potential for good. This would not include loans, and job hiring. It may be college applications systems.