Defining and Demystifying Automated Decision Systems

64 Pages Posted: 26 Mar 2021 Last revised: 18 Jun 2021

See all articles by Rashida Richardson

Rashida Richardson

Northeastern University School of Law

Date Written: March 24, 2021

Abstract

Government agencies are increasingly using automated decision systems to aid or supplant human decision-making and policy enforcement in various sensitive social domains. They determine who will have their food subsidies terminated, how much healthcare benefits a person is entitled to, and who is likely to be a victim of a crime. Yet, existing legislative and regulatory definitions fail to adequately describe or clarify how these technologies are used in practice and their impact on society. This failure to adequately describe and define “automated decision systems” leads to such systems evading scrutiny that policymakers are increasingly recognizing is warranted and potentially impedes avenues for legal redress. Such oversights can have concrete consequences for individuals and communities, such as increased law enforcement harassment, deportation, denial of housing or employment opportunities, and death.

This article is the first in law review literature to provide two clear and measured definitions of “automated decision systems” for legislative and regulatory purposes and to suggest how these definitions should be applied. The definitions and analytical framework offered in this article clarify automated decision systems as prominent modes of governance and social control that warrant greater public scrutiny and immediate regulation. The definitions foreground the social implications of these technologies in addition to capturing the multifarious functions these technologies perform as they relate to rights, liberties, public safety, access, and opportunities. To demonstrate the significance and practicality of these definitions I analyze and apply them to two modern use cases: teacher evaluation systems and gang databases. I then explore how policymakers should determine exemptions and evaluate two technologies routinely used in government: email filters and accounting software. This law review provides a much-needed intervention in global public policy discourse and interdisciplinary scholarship regarding the regulation of emergent, data- driven technologies.

Keywords: legislation, regulation, technology policy, automation, automated decision systems, artificial intelligence, definitions, education, criminal justice

Suggested Citation

Richardson, Rashida, Defining and Demystifying Automated Decision Systems (March 24, 2021). Maryland Law Review, Forthcoming, Available at SSRN: https://ssrn.com/abstract=3811708

Rashida Richardson (Contact Author)

Northeastern University School of Law ( email )

416 Huntington Avenue
Boston, MA 02115
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1,065
Abstract Views
5,938
Rank
38,431
PlumX Metrics