Recommendation algorithms promise social media users a personalized experience, but some experts worry they promote doomscrolling, brain rot, and myriad mental health problems. Australia recently proposed legislation to allow users to opt out of algorithms with the goal of protecting children online and giving people more control over what they see on their feeds.
We asked faculty members from different fields of study to share their views and concerns about recommendation algorithms. Their responses have been edited for clarity and length.
‘Dark nudging’ from the Garden of Eden to AI age
— Cass Sunstein, Robert Walmsley University Professor, Harvard Law School, and co-author of the book “Algorithmic Harm: Protecting People in the Age of Artificial Intelligence”It is standard to define manipulation as a form of trickery or covert influence. Some definitions suggest that manipulation treats people as “tools or fools.” My preferred definition sees manipulation as an influence that fails to respect people’s capacity for reflective and deliberative choice. The basic idea is that manipulators exploit a lack of information on the part of their victims or take advantage of our behavioral biases (such as a focus on the short term, unrealistic optimism, and limited attention). You can think of manipulation as a form of “dark nudging.” The problem is the same, from the Garden of Eden to the era of algorithms and AI.
I like recommendation algorithms! They can help us put our time and our money in the right places. If an algorithm tells me about a new book on behavioral economics or constitutional law, I will be grateful. Personalization is often highly beneficial. I don’t want to see anything about hockey, advanced physics, Jell-O, Beethoven, or Bach, even though a lot of people do. The concern here points to manipulation, as when recommendation algorithms exploit people’s lack of information or their behavioral biases. That can be a real problem.
We need laws and regulations to protect against the worst forms of manipulation, just as we need them to protect against deception. We need a new right — the right not to be manipulated. As individuals, we can and should pause a few beats and ask: “On reflection, do I really want to buy what they’re selling? With my time or my money? They might be trying to trick me!”
Feeds like ‘fun house mirrors’
— Emily Weinstein, Lecturer on Education and co-director of the Center for Digital Thriving, Harvard Graduate School of EducationI get to do a lot of deep listening to young people, and usually our aim is not just to understand what they see when they’re behind their screens, but how they see it. I’ve been doing this work for more than 15 years. Recently, the “algorithm” seems to have shifted from a supporting player to a leading role in teens’ telling of their experiences with tech. They say things like: “My algorithm is just not great on that app” or “Lately, my algorithm is trash.”
Young people shape their feeds in ways they’re well aware of, as well as in ways that they say they don’t understand. Their feeds shape them, too. One 16-year-old recently told me that her feed is filled with posts where, “It almost feels like everyone’s pushing the rush of your life — like get married at 18, have kids by 20. That’s just not everybody’s reality, but it makes it seem like it’s supposed to be.” Other teens have feeds that look completely different.
When you sit alongside teens as they scroll, it’s abundantly clear how they are being profiled and pulled. Their interests (and their identities) are mirrored back with distortions like fun house mirrors. Another high schooler who is an avid gamer showed me his feed was full of “edits” (rapid-cut montages of movie scenes and TV shows), games, and ads for sports betting and online gambling.
Algorithms can contribute to teens (and all of us) feeling inundated by narrowly themed content. For one 17-year-old I spoke with in the spring, the algorithm was saturating her feed in prom content: “I get it, I get it, it’s prom season, but it’s like every single time I refresh, it’s prom. But I don’t think it really changes how I feel about prom itself. It’s just like when I’m looking at it on my phone, I’m like, bro, can I see something else?”
Our personalized algorithms transport us into very different worlds, and even when we understand how they work, it doesn’t inoculate us from their impacts. For researchers, algorithms can also make it hard to capture evidence of the effects of these technologies, because everyone’s experiences are so varied and different by design.
Responsibility for digital agency can’t rest just on the shoulders of our young people, who are being asked to withstand designs that prey on psychological vulnerabilities and developmental sensitivities. We need more control over algorithms, and features that allow us to make changes to them. Young people also deserve to understand how and why agency can be hard to exercise — in particular the incentive structures that are behind design choices that impact them.
‘Guard your attention very carefully’
— Rebecca Lemov, Professor of the History of Science, and author of the book “The Instability of Truth: Brainwashing, Mind Control and Hyper-Persuasion”Hyper-persuasion is a word I use to describe the compelling and even coercive parts of persuasion seen in the operations of modern digital media. An early example that serves as a harbinger is the 2014 Facebook Experiment by the Proceedings of the National Academy of Sciences journal, in which 689,003 users of the site found themselves unwittingly “nudged” into slightly more depressive (or slightly less despondent) states, as measured by their behavior after having their personal feeds manipulated. This was the power of social media — to induce “massive-scale emotional contagion,” almost like turning a dial on the user’s psyche. Little by little, as we engage with these technologies, we are ungrounded. The algorithmically-driven version of this is seen in how almost all of us — be honest! — can be prone to getting caught in a loop, a flow state, a doomscroll, or another daily type of dissociative mind state.
Social media interactions and AI chatbots, with their targeted algorithms, can become microenvironments. One of the key principles, really the foundation, of classical brainwashing is what Robert Jay Lifton called milieu control. Everything else — the emotional engineering of a captive, their ideological realignment (mostly temporary), or a radical personality change — follows from this control of the local environment, especially via control of what comes in and what goes out. Milieu, after all, means place or surroundings. You can imagine a highly controlling abusive cult or a prisoner of war camp that has no fences or barbed wire because it is so remote (this was the milieu in which U.S. GIs captured in the Korean War found themselves, where they underwent communist re-education). Although these circumstances seem extreme compared with the microenvironments created by your seemingly tiny interactions with your phone, for example, the same dynamics are at play. The device, with its algorithmic targeting, selects or controls what comes in and out of view for you, to which you react, which then continues to feed the selection process. This is why the word siloing is often used. It amounts to a type of invisible control.
My recommendation is to guard your attention very carefully. If it goes astray, or you find yourself in a mindless attention-sapping loop, bring your attention back to awareness. Strive to check in with your own feedback, as experienced in the body. There’s a subtle feeling that can arise when any of us encounters draining or (even) undermining messages. You can ask intermittently, how does this make me feel? It’s not a matter of avoiding what makes you uncomfortable but rather of making a deliberate choice that giving my attention to this is meaningful.
‘Is my phone listening in?’
— Sitan Chen, Assistant Professor of Computer ScienceAn algorithm is any procedure that takes an input, performs some clearly specified computation, and produces an output. The first algorithms that we learn, in grade school, are how to add, multiply, and divide numbers. In machine learning, the process of training a model on data, either to predict (e.g., “Is this individual eligible for a loan?” or “Is this chest X-ray that of a healthy patient?”) or to generate (e.g., images, videos, text), is an algorithm called “gradient descent,” whose roots go back to 19th-century mathematics. Its input is data, and its output is a bunch of numbers, in some cases trillions of numbers, specifying a model. Somewhat confusingly, these models themselves are also algorithms. They are designed to take in some input, for instance a picture of someone’s face, or some prompt in plain English, and output a response, for instance whether the person has checked into their flight, or an answer to a math question.
Algorithms and scientific progress have gone hand in hand since antiquity. More recently, algorithms for numerical simulation have been instrumental in the study of complex physical and biological systems. Algorithms for search and sequence alignment enabled the sequencing of the human genome. Algorithms for reconstructing an image from noisy measurements are run every time a doctor performs an MRI or CT scan.
Recommendation algorithms are pervasive in social media. Their basic functionality is to take as input a user’s browsing patterns and output suggestions for content meant to maximize user engagement and drive sales. The implications for addiction, among other mental health concerns, are well documented. In terms of privacy, not only is it unclear to the user which aspects of their interactions with social media are being used to drive recommendations (“Is my phone listening in on my conversations?”), but these interactions ultimately also go into training the machine learning model that is making recommendations to other users, possibly revealing to them traces of a user’s activity they never intended to share. There is now over two decades of scholarship on this latter kind of vulnerability, and a massive body of work, e.g., on differential privacy, proposing interventions that algorithm designers could employ to mitigate these risks.
Legally, ‘We could, and should, do much better’
— Leah Plunkett, Meyer Research Lecturer on Law and faculty associate with the Berkman Klein Center for Internet & SocietyMy overarching concerns about the impact of recommendation algorithms on young people’s lives are about two types of decisions: those that are made about young people without their knowledge or full consent, and those that are made by young people without their full awareness or meaningful consent, about the role of a recommendation algorithm.
Recommendation algorithms appear in tools, services, and platforms that might inform access to major life opportunities such as employment, insurance, etc., but also in social and emotional learning programs used by schools. And in social media, the feed being pushed is usually through the secret sauce of a recommendation algorithm, and for kids and teens, they might feel that a lot of the suggested content is organic and natural. The recommendation algorithm really can lure you into a false sense that this content is really meant for you. It can also get very insidious, even risky and dangerous, if the content that is being fed to you to keep your attention is depicting activities, emotions, experiences, relationship dynamics, and worldviews that are harmful or risky to minors.
I think the law in the United States has largely failed to provide ethical, practical protections for kids on social media and many other tech platforms. Ideally, the federal legislature would have done comprehensive individual consumer privacy for the digital age a couple of decades ago, and at the very least should have done it for minors, and they haven’t. We have seen many leading tech companies move fast and break things, fail early and often, act all the way up against, sometimes even past, the boundaries of what the law permits. So, to the extent that we have seen tech companies avail themselves of the freedoms entitled to them in this country to build lawful businesses, I would say shame on companies that have chosen to conduct their affairs past what the law permits, or even close to the boundary, and it is long past time for the places where that has happened to change.
I am not anti-tech. I am not anti-social media. I am not anti-innovation. I think the formal and informal governance mechanisms here are complicated, and I think that most people, whether they are federal lawmakers, state regulators, tech executives, or vice principals, are trying their best to regulate the way these companies operate and protect our kids and teens. I also think we could, and should, do much better.
Fact-Checked by Irfan Ahmad.
Read next:
• Study Finds People May Embrace AI Advice That Confirms Their Views and Dismiss It When Challenged
• Google’s content recommendation algorithm is a black box. Here’s how it decides what to show you
by External Contributor via Digital Information World
