The Case for Designing Tech for Social Cohesion: The Limits of Content Moderation and Tech Regulation

26 Pages Posted: 21 Feb 2023 Last revised: 7 Apr 2023

Date Written: March 3, 2023

Abstract

Based on nearly 60 interviews with staff at tech companies, critics of big tech, civil society groups impacted by tech-amplified social media, and new tech startups, research revealed three distinct but complementary narratives or approaches to thinking about polarization and social cohesion in digital spaces. The “User-Centered” Narrative describes harmful content online as generated by users, with social media products and search engines acting as a mirror of society. Several interviewees described the defeating feeling of playing “whack a mole” against the growing tide of individual and state-sponsored industrialized harmful digital content. This narrative points to the need for content moderation on user-generated content and digital media literacy to help the public navigate information and communication on the internet.

The “Tech Design Regulation” Narrative describes harmful content as amplified by tech product designs including the affordances and algorithms that are optimized for user engagement, advertising, and shareholder profit. Many social media companies optimize their product designs for user engagement to maximize their ad-based profits. Machine learning algorithms promote emotionally alarming and divisive content which tends to garner more attention, just as cars slow down driving past a car accident and as news outlets use the “if it bleeds, it leads” principle to prioritize alarming news. From this point of view, some tech products incentivize harmful content that drives toxic polarization. This narrative presses for government regulation to extend beyond privacy to regulating tech profit models, algorithms, affordances, and designs that amplify toxic content.

The “Social Cohesion by Design” Narrative describes tech products that amplify and scale social cohesion by designing affordances and algorithms optimized for these purposes. These digital products can support human agency to participate in civic action, bridge divided communities, and build trust between the public and institutions.

The first half of this article provides explores the complex relationship between toxic polarization and digital spaces and analyzes these three frames or paradigms for understanding the role of digital spaces in toxic polarization. The second half of the paper focuses on examples and case studies of “social cohesion by design” also known as "peacetech."

Keywords: content moderation; tech regulation; social cohesion; toxic polarization; peacetech; peacebuilding; bridging

JEL Classification: k23; z11

Suggested Citation

Schirch, Lisa, The Case for Designing Tech for Social Cohesion: The Limits of Content Moderation and Tech Regulation (March 3, 2023). Yale Journal of Law and the Humanities, Forthcoming, Available at SSRN: https://ssrn.com/abstract=4360807 or http://dx.doi.org/10.2139/ssrn.4360807

Lisa Schirch (Contact Author)

University of Notre Dame ( email )

100 Hesburgh Center
Notre Dame, IN 46556-5646
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
456
Abstract Views
1,446
Rank
116,076
PlumX Metrics