Many companies have spent the past few years experimenting with AI. Now, as the technology moves into business workflows in critical sectors like healthcare and finance, they are confronting a harder problem: how to stay in control once software starts taking action on its own
Recent recognition of IBM as a Leader in the inaugural Gartner® Magic Quadrant™ for AI Governance Platforms comes as organizations seek ways to manage increasingly capable AI systems. Companies are moving beyond AI pilots and using the technology inside everyday business processes that require visibility into risk, controls and compliance
“The conversation around AI has shifted dramatically over the last two years,” Dinesh Nirmal, SVP of IBM Software, told IBM Think in an interview. “Companies are no longer asking whether they should use AI; they’re asking how to deploy it responsibly across the enterprise.”
When AI joins the workflow
The first wave of enterprise AI largely centered on experimentation. Companies tested chatbots, explored large language models (LLMs) and searched for ways to improve productivity
Many of those projects remained separate from the systems that run businesses day-to-day. IBM said in a blog post that it is beginning to change as organizations weave AI into customer service, administrative and operational tasks
That shift brings a new set of questions. IBM said companies increasingly want to know where AI is being used, what information it can access and who is responsible for its actions. Those concerns become more pressing as organizations deploy multiple models across departments and begin experimenting with AI agents, software systems that can perform tasks rather than simply generate content
“The organizations that succeed with AI will be the ones that can scale innovation while maintaining visibility, accountability and control,” Nirmal said
Responsible AI in healthcare
Healthcare offers an early glimpse of what those challenges look like when AI moves from experimentation into environments where mistakes can carry real consequences
The challenge becomes easier to see in healthcare. Hospitals want AI to reduce paperwork, streamline operations and free up clinicians’ time. They also need to know who remains responsible when technology becomes part of decisions and workflows that affect patient care
Those competing priorities shaped how ViClinic developed its agentic healthcare operating system, or AHOS, a platform designed to deploy AI agents within clinical and administrative workflows. The company built the platform using IBM technologies, including watsonx
“We quickly realized that AI adoption in healthcare is not primarily a technology challenge. It is a governance challenge,” Dr. Boris Jinjolava, Founder and CEO of ViClinic, a healthcare technology company and IBM partner, told IBM Think in an interview
Healthcare organizations wanted more than sophisticated AI tools, Jinjolava said. They wanted to know how those systems would fit into existing clinical processes and who would remain responsible for the outcome
That concern shaped ViClinic’s approach from the beginning. “Health systems need confidence that AI actions are transparent, auditable, supervised and aligned with organizational policies,” he said
Jinjolava said healthcare organizations need to know what an AI system did, why it did it and who reviewed the result before it became part of a patient’s record
The autonomy dilemma
Those concerns become even more important as organizations begin deploying AI agents. Unlike traditional AI tools that answer questions or generate content, AI agents can perform tasks, move information between systems and help coordinate activities across a workflow
Organizations see agents as a way to automate tasks that once required employees to move information, complete paperwork or coordinate activities across teams. Jinjolava said the challenge is ensuring those systems operate within clearly defined boundaries
“AI should support decisions and workflows, not act as an uncontrolled autonomous system,” he said
The company designed the system so clinicians remained responsible for approving key outputs. “Rather than maximizing automation, we prioritized transparency, accountability and human oversight,” he said
Building trust before scale
Building trust often comes down to deciding who reviews what and when. Jinjolava said governance frameworks help healthcare organizations establish approval thresholds, escalation paths and review processes before introducing AI into critical workflows
Those safeguards allow organizations to begin with limited use cases, evaluate outcomes and expand adoption over time. Jinjolava said that the approach gives healthcare organizations an opportunity to test AI-assisted workflows before introducing them into broader operations
The same questions confronting hospitals are beginning to emerge across industries as organizations experiment with more autonomous AI systems. IBM said organizations increasingly want tools that provide a clearer view of AI models, vendors and systems as AI becomes part of more business activities
For years, companies worried about whether individual AI models would produce inaccurate answers. Jinjolava said the bigger challenge may be what happens when those systems become part of larger business processes
“The risk shifts from a single model to an entire workflow,” he said
The latest AI trends, brought to you by experts
Get curated insights on the most important—and intriguing—AI news. Subscribe to our weekly Think newsletter. See the IBM Privacy Statement
Thank you! You are subscribed
Sascha Brodsky
IBM
Gartner Magic Quadrant for AI Governance Platforms 2026
Explore the latest Gartner® Magic Quadrant™ for AI Governance Platforms and see why IBM stands out as a leader
