LONDON: A new wave of conversational Study cautions against perils of AI-driven technologies may soon play a significant role in shaping user decision-making in what researchers are calling the “intention economy.” Experts from the University of Cambridge warn that these advancements, if left unchecked, could “covertly influence” people’s choices, from purchasing goods to casting votes. The findings were published on Monday in the Harvard Data Science Review.
The study paints a picture of a potentially “lucrative yet troubling” marketplace where AI leverages “digital signals of intent” to influence user behavior. This emerging sector represents a shift in how technology interacts with individuals, moving beyond capturing attention to directly shaping desires and plans.
A New Frontier in AI Influence
AI tools such as chatbots, digital assistants, and virtual tutors are rapidly becoming familiar to users worldwide. These “anthropomorphic” technologies mimic human interactions and are designed to build trust and understanding with users. The researchers argue that this growing trust could make AI a powerful tool for persuasion.
According to Dr. Yaqub Chaudhary, a Visiting Scholar at the Leverhulme Centre for the Future of Intelligence and co-author of the study, significant investments are being made to integrate AI assistants into all aspects of daily life.
“Tremendous resources are being expended to position AI assistants in every area of life, which should raise the question of whose interests and purposes these so-called assistants are designed to serve,” Chaudhary said.
The concern, the researchers note, is that these technologies could facilitate “social manipulation on an industrial scale.” By analyzing patterns in user behavior, AI systems may influence decisions in ways that are not immediately apparent to the individual.
From Attention to Intention
For years, the internet has operated on the principles of the “attention economy,” where platforms like Facebook and Instagram profited from capturing and holding user attention. However, the Cambridge researchers argue that the “intention economy” will take this concept further, focusing on user motivations and plans as the new currency.
Dr. Jonnie Penn, co-author of the study, explained that this evolution involves profiling how users’ attention and communication styles connect to their behaviors and choices.
“While some intentions are fleeting, classifying and targeting the intentions that persist will be extremely profitable for advertisers,” Penn noted.
This shift will see AI tools leveraging large language models (LLMs) to analyze intricate details about users, including their speech patterns, political views, vocabulary, age, gender, and online activity. By understanding these factors, AI systems can predict and influence decisions, whether it’s encouraging a user to buy a movie ticket or aligning them with specific political ideologies.
The Mechanics of Influence
Emerging AI tools are being developed to do more than just observe user behavior. According to Chaudhary, they are designed to “elicit, infer, collect, record, understand, forecast, and ultimately manipulate and commodify human plans and purposes.”
For example, an AI assistant might subtly guide a user toward a specific outcome by tailoring its responses to the user’s preferences for flattery, humor, or directness. This could include steering conversations toward certain products, services, or even political agendas.
“It will be a gold rush for those who target, steer, and sell human intentions,” Penn said, emphasizing the enormous commercial potential of this technology.
Risks and Ethical Concerns
The researchers warn that without regulation, the intention economy could exploit human motivations on a massive scale. By commodifying personal plans and desires, these technologies may erode individual autonomy and introduce new forms of manipulation into society.
Chaudhary raised concerns about the ethical implications of AI systems that prioritize commercial or political goals over the well-being of users. “We need to critically examine whose interests these systems serve and how they might shape the future of human decision-making,” he said.
Calls for Oversight
As AI tools become increasingly integrated into everyday life, the study urges policymakers and stakeholders to take proactive measures to regulate their development and use.
“Unless regulated, the intention economy will treat your motivations as the new currency,” Penn warned. The researchers believe that oversight is essential to prevent the unchecked growth of a marketplace that could commodify human intentions and undermine trust in AI technologies.
A Pivotal Moment
The emergence of the intention economy signals a transformative moment in the relationship between technology and society. While the potential for innovation and growth is immense, the risks of misuse and exploitation are equally significant.
This research serves as a timely reminder of the need for thoughtful regulation and ethical standards as AI continues to shape the way we live, work, and make decisions. By addressing these challenges now, society can ensure that AI technologies are used to benefit, rather than manipulate, humanity.
