25% of SMB execs can't explain their AI outputs: why governance and audit trails matter
Published by AINave Editorial • Reviewed by Ramit
A new survey cited by TechRadar Pro reveals a troubling gap in AI explainability for small businesses: one in four AI-enabled startup executives cannot explain how their AI systems arrived at a given output, even as 85% of small businesses rely on AI for sensitive financial work. This blind trust creates real regulatory and investor risk, especially under UK GDPR Article 15, and signals a clear pain point for builders who design AI tools sold to SMBs. The findings come from a Startup.co.uk survey and were reported by TechRadar Pro.
The survey data: leaders in the dark
Of the executives surveyed, 25% said they could not explain AI-generated outputs, 12% could do so only with significant difficulty, and just 22% reported being able to explain them easily. Meanwhile, the same executives are trusting AI with core financial operations: 37% use it for accounts payable automation, 32% for audits and compliance, 31% for spend management, 27% for performance insights, and 26% for fraud detection. The gap between reliance and understanding is stark.
Why this matters for AI builders
If you are building AI tools for SMBs, these numbers are a warning. Your customers are using your product for tasks that require traceability. A founder who cannot explain how a fraud detection flag was raised or why an accounts payable amount was calculated faces immediate problems when an investor, customer, or regulator asks for justification. The survey editor Zohra Huda framed it bluntly: "Founders are letting AI manage their fraud detection and accounting, but if a stakeholder asks how the numbers were calculated, a quarter of them can't answer." For builders, this means explainability features, audit logs, and human-readable reasoning paths are not nice-to-haves. They are core requirements for the product to be safe for deployment in finance.
The regulatory threat is real
The risks extend beyond investor confidence. Under UK GDPR Article 15, individuals have the right to know how their data is processed. If an SMB cannot explain what its AI does with customer financial data, it may be in breach. The UK Information Commissioner's Office can fine up to £17.5 million or 4% of global turnover for verified breaches. That kind of exposure should push any SMB founder to demand better AI governance, and any builder to deliver it. AI risk management for SMBs must include audit trails that survive regulatory scrutiny.
What builders should do differently
Practical steps include exposing model reasoning through structured outputs (e.g., chain-of-thought summaries), maintaining immutable logs of AI decisions, and providing clear documentation for how financial calculations are derived. If your AI handles accounts payable or fraud detection, you should treat explainability as a compliance feature, not an optional overlay. The survey suggests many leaders cannot even describe what their AI does, so products that surface decisions in plain language will have a genuine compliance advantage.
Caveats on the data
This article is based on a secondary report of the Startup.co.uk survey, as published by TechRadar Pro. The exact sample size, survey methodology, and date of the survey were not included in the available material. Readers should treat the percentages as directional indicators rather than precisely measured statistics. The regulatory warnings about GDPR and ICO penalties are well-established, but the direct link between AI explainability gaps and enforcement actions remains hypothetical without a specific case.
