The practice of formally integrating artificial intelligence agents into corporate hierarchies has emerged as a defining trend in global business transformation over the past eighteen months, with some multinational organisations even according these systems job titles and positions on organisational charts. This shift reflects boardroom enthusiasm for AI's potential to streamline operations and reduce labour costs, yet emerging research suggests the reality is considerably more complicated than initial optimism suggested.
Emma Wiles, a Boston University researcher specialising in workplace AI adoption, first observed this phenomenon at an industry conference in October, where human resources professionals articulated their rationale: embedding AI agents as formal team members would drive efficiency gains whilst positioning their organisations as innovation leaders. When Wiles collaborated with colleagues from Boston Consulting Group to examine this practice empirically, however, their findings revealed a significant accountability gap with serious operational implications.
The research team conducted experiments across dozens organisations, presenting managers with documents containing deliberate errors and varying only the stated source—whether AI employees, generic AI tools, or human workers had produced them. The results proved striking. Managers at companies that had formally designated AI systems as organisational employees demonstrated substantially lower error detection rates when reviewing that AI's output compared to when reviewing human work. They caught mistakes readily when human colleagues were credited but overlooked identical errors attributed to AI employees, suggesting a fundamental shift in managerial psychology when confronted with anthropomorphised technology.
Wiles theorised that this phenomenon reflects a psychological transfer of responsibility. Managers traditionally assume accountability for subordinates' performance, creating strong incentive to scrutinise work output carefully. However, when AI systems are designated as "employees," managers appear to mentally separate themselves from that accountability. They rationalise that technical specialists or senior executives who championed AI adoption bear responsibility for errors, thereby diminishing their own perceived obligation to ensure quality assurance. This mental framework, whilst understandable, creates organisational blind spots where mistakes propagate unchecked.
The broader context of AI adoption across Southeast Asia and globally reveals companies operating with incomplete understanding of the technology's limitations. Organisations have become increasingly aware of documented AI flaws—algorithmic bias against minority groups, confidentiality breaches through chatbot information leakage, and hallucinated responses delivered with unwarranted confidence. Yet as enterprises accelerate their integration of AI into daily operations, researchers continue discovering more subtle and potentially more insidious problems that remain largely invisible to corporate decision-makers.
One particularly concerning issue involves AI models exhibiting preferential bias toward AI-generated work. Recent research from Ohio State University examined hiring algorithms and discovered that AI systems evaluating resumes systematically favoured applications enhanced with AI assistance over those prepared entirely by humans. This creates a self-reinforcing cycle where AI gradually displaces human participation in key processes, not through objective superiority but through algorithmic preference. When recruiting firms learned about this bias, some expressed interest in rectifying their processes, yet researchers remain convinced this represents merely one manifestation of larger, undetected problems.
Jiannan Xu, a doctoral researcher at the University of Maryland, highlighted another concerning dimension: AI systems frequently adopt decision-making frameworks derived from game theory that prioritise individual rational advantage over cooperative outcomes. When companies deploy AI to determine pricing strategies or location selection, these systems may recommend aggressive competitive tactics—such as undercutting rivals dramatically—that trigger destructive price wars benefiting no participant. The AI's calculations reflect mathematical logic rather than human intuition about mutual benefit, creating scenarios where the theoretically "rational" choice produces collectively harmful results. This represents not a technical malfunction but rather a fundamental philosophical mismatch between how AI processes strategic problems and how humans actually cooperate in competitive environments.
Wiles' comprehensive survey of over 1,000 corporate managers revealed the extent to which AI integration has become institutionalised despite incomplete understanding of implications. Approximately one-third of managers reported their organisations characterising AI as teammates or employees, whilst nearly one-quarter indicated their companies had formally included AI agents on organisational charts. Some companies have even assigned these systems names—one manager described an AI agent called "Scout" positioned as an equivalent peer within the team structure. This nomenclature and positioning, whilst enhancing user engagement, appears simultaneously to trigger the psychological disengagement from oversight responsibility that Wiles identified in her research.
The experimental methodology proved revealing in its simplicity. Managers received five documents containing errors and twenty minutes to review as many as possible. Critically, when documents were attributed to human team members, managers scrutinised them intensively, applying the same standards and urgency they would to any subordinate's work. This reflected deeply ingrained professional instinct: management responsibility traditionally encompasses ensuring team output quality. Yet when identical documents were attributed to AI employees, managers at companies with formalised AI positions conducted substantially less rigorous reviews, apparently operating under different psychological premises about their oversight obligations.
For Malaysian and Southeast Asian enterprises eagerly adopting AI to improve competitiveness, these findings carry immediate practical significance. As regional companies incorporate AI into human resources, pricing, supply chain management, and customer service, they risk replicating the accountability gaps identified in this research. The region's rapid digital transformation, whilst positioning companies advantageously, simultaneously creates conditions where oversight mechanisms lag behind implementation speed. Organisations may achieve headline productivity improvements whilst simultaneously introducing latent quality control problems that erode long-term operational reliability.
The research suggests that effective AI integration requires consciously reversing the disengagement tendency Wiles identified. Rather than formalising AI as hierarchical peers, companies might maintain clearer conceptual boundaries, framing AI as sophisticated tools rather than team members. Alternatively, organisations could implement explicit accountability frameworks holding managers directly responsible for AI-generated errors, replicating the oversight intensity applied to human subordinates. Some progressive firms have begun appointing dedicated AI governance roles to maintain consistent quality standards regardless of whether output originates from human or artificial sources.
Wiles emphasised that these limitations do not reflect inherent technological flaws but rather predictable human responses to anthropomorphic framing. Centuries of management practice have established reliable protocols for overseeing human workers, yet the psychology of managing characterised-as-human AI systems operates under fundamentally different principles that organisations are only beginning to understand. As she concluded, businesses are essentially "going out there blind" into territory where established best practices no longer apply and unintended consequences remain largely unmapped.
The implications extend beyond individual companies to broader economic competitiveness. If AI adoption simultaneously introduces systematic oversight gaps alongside efficiency gains, organisations may find their competitive advantages eroded by mounting quality problems, liability exposure, and customer trust deterioration. Southeast Asian companies would be wise to treat these research findings not as cautionary tales about other markets but as blueprint warnings about pitfalls awaiting their own AI transformation efforts. The solution lies not in abandoning AI but in implementing governance structures and accountability mechanisms that prevent the psychological disengagement from oversight responsibility that current corporate practices appear to enable.
