The hidden cost of enterprise AI: 6.4 hours a week babysitting bots

While AI is proliferating across the workplace, it is introducing a new productivity paradox: While the technology makes work feel faster, it actually pushes more burden onto employees to provide context, perform quality checks, then rinse and repeat across numerous disparate tools.

This, according to a new survey of 6,000 full-time digital workers by Glean’s Work AI Institute, results in two emerging behaviors: “botsitting,” all the unrecognized work that goes into making AI actually usable; and “botshitting,” shipping AI-generated work that is unverified, not that well understood, or perhaps not even trustworthy. The survey report was co-authored by experts from Work AI Institute, Emory University, Stanford University, UC Berkeley, UC Santa Barbara, UNC Charlotte, University College London, and University of Notre Dame.

“It’s definitely in many ways a vicious cycle that feeds itself,” said Rebecca Hinds, head of Glean’s research center the Work AI Institute, a research collaborative of AI experts. Enterprises need to begin understanding and addressing the “massive, massive human labor that’s at the core of this.”

Workers are using AI more, getting more frustrated

There’s no doubt that AI is quickly becoming a central teammate in the workplace. Glean’s Work AI Institute found that 87% of digital workers are using AI: It is already automating more than a quarter of their work and saving about 11 hours a week.

Still, only 13% say the use of AI has significantly improved their company’s performance, and their time savings are being eaten up by the same technology that is producing them. Employees lose about one-third of their work week (6.4 hours) botsitting: feeding AI context, supervising outputs, debugging errors, cleaning up AI-generated work, and switching between AI tools.

“We’re seeing high, high rates of multiple tool usage, and often those tools aren’t connected,” said Hinds.

In terms of context-feeding, large language models (LLMs) are trained on the vast corpus of the internet, but not always on enterprise-specific data. Thus, employees often have to provide additional information around their company’s products, customers, services, or other details.

“They’re often feeling frustrated when the tools don’t understand enough about day to day work to be useful,” said Hinds. Also, because employees are using multiple tools, they often have to repeat the same prompt over and over.

“It’s exhausting for workers to not only do this, but to have the work be unrecognized, often unrewarded and unacknowledged within the organization,” she said.

Similarly, workers are having to catch outputs that might look polished and finished on the surface, but could be wrong, incomplete, or missing important context. Debugging is the biggest driver of exhaustion, because it is often conducted by people who didn’t necessarily contribute to the initial output, Hinds noted, so they first have to dig up background information.

However, “not all botsitting is bad,” Hinds emphasized. “Certainly, we want workers to have some level of ownership and oversight.”

But when it is unnecessary, it can lead to botshitting, where users ship AI-generated work they haven’t verified because they’re overwhelmed or time-constrained. Sixty-nine percent of users admit to doing so, and 41% say they sometimes deliver work they could not explain if asked. Another 28% blame AI for mistakes they themselves caused.

“Botshitting is offloading your critical human thinking, judgment, and understanding,” Hinds explained. “You’re offloading that work that absolutely needs to remain with the human.”

Workers using multiple AI agents are significantly more likely to do this, she added, because agents are so scalable, and can spiral out of control if they don’t have the right controls or permissions built around them, causing overwhelmed users to give up on their verification efforts.

“You don’t often see the negative impacts until 3, 4, 5, steps down the line,” said Hinds. “Then it requires all of this cleanup work, detective work, to understand where did the agent go wrong.”

Using AI … but not too much

Interestingly, more than half of the workers surveyed said they get more day-to-day help from AI than they get from their managers, and consider it easier to collaborate with than humans.

Still, they seem to be facing a Goldilocks problem when it comes to sharing their use of AI. Among self-identified high AI achievers, 54% are using unapproved tools or using approved tools in noncompliant ways, and 36% are hiding how much AI helps them.

As Hinds explained, depending on the context and the level of psychological safety an organization has provided, it can be “differentially beneficial or harmful” to show you’re using AI, and, on the flip side, to conceal that you’re using it too much, because that might make you less valuable, or perceived as less valuable, she said.

It’s a complicated balance, because, she noted, “there’s massive pressure in so many organizations to demonstrate AI fluency, to demonstrate you’re a power user.”

What successful organizations are doing differently

In fact, the report said, “The companies pulling ahead are doing something different. They aren’t spending a greater share of their AI time using AI. They’re spending a greater share on the work around it: setting context, defining what ‘good’ looks like, building judgment, and deciding what should never have been handed to a model in the first place.”

The most transformative organizations are addressing AI challenges proactively: Providing training and support, treating AI as an opportunity to redesign work, and formally rewarding AI skills. In addition, it noted, the hardest skill to build is knowing when not to use AI.

It is “not just clicks of the tool, not just tokens used, but real skills, real learning,” said Hinds. In addition to investing in workers, these organizations are clearly stating AI strategy and clarifying the “why” behind it. Governance should also be “living and breathing,” with companies continuously re-evaluating policies.

And it needs to happen at every level, top execs included, said Hinds: “It’s being able to see the executives use the technology, sharing both the success stories and the failures.”

Successful companies are also actively using metrics anchored in existing key performance indicators (KPIs). They are measuring quality, efficiency, and employee engagement in different ways, and putting data in the hands of employees so they can assess their own adoption and success.

“It’s less about surveillance and more about feedback in terms of how we work collectively,” said Hinds.

What’s “fascinating but perhaps not surprising,” she said, is that workers are increasingly using AI itself as a teacher, and prefer it over other learning channels. This speaks to the importance of low-code, no-code tools, with low learning curves and organizational context, that are embedded directly into workflows.

“It is starkly different from what we’ve seen with previous technologies,” she said.

This article originally appeared on CIO.com.

Source:: Computer World