Why Finance Needs AI That Knows When To Stop Thinking

The financial industry is racing to deploy artificial intelligence across trading, risk management, and customer service, but a fundamental flaw threatens to undermine all that progress: AI has no inherent concept of accountability or consequences. A leading voice on the Forbes Technology Council argues that finance specifically needs AI that knows when to stop thinking—systems that can recognise when further computation is wasteful, harmful, or simply wrong.

The challenge, as outlined in the Forbes article, is that most AI models are optimised to maximise some objective function without regard for the cost of continued reasoning. In finance, where milliseconds can mean millions, this can lead to over-optimisation, runaway trading algorithms, or models that fabricate ‘insights’ from noise. Unlike human traders who understand risk, liability, and ethical boundaries, AI operates in a vacuum of consequences.

This problem has become urgent as financial institutions embed AI deeper into core operations. According to a 2025 survey by the Bank for International Settlements, over 70% of central banks and large commercial banks now use machine learning for market surveillance, credit scoring, or portfolio optimisation. Yet fewer than one-third have formal policies governing when an AI system should stop, pause, or escalate a decision to a human. The gap between capability and governance is widening.

The article highlights several concrete risks. In algorithmic trading, AI agents can enter feedback loops, buying and selling the same assets repeatedly, amplifying volatility. In loan underwriting, models may produce statistically sound but ethically indefensible decisions, such as denying mortgages to entire zip codes. Without a built-in ‘stop thinking’ mechanism, these systems can generate output indefinitely, even when the marginal value of additional computation is negative.

Named examples include the 2010 Flash Crash, where high-frequency trading algorithms caused a trillion-dollar market swing, and the 2012 Knight Capital disaster, where a faulty algorithm lost $440 million in 45 minutes. These were not failures of prediction but failures of stopping—the systems lacked a circuit breaker for their own reasoning. The article argues that modern AI, particularly large language models used for financial analysis, repeats this pattern by generating verbose, plausible-sounding answers that may be entirely fabricated.

Broader implications touch on regulation and trust. The U.S. Securities and Exchange Commission has proposed rules requiring brokers to explain AI-driven decisions, but enforcement lags. The European Union’s AI Act classifies financial applications as ‘high risk,’ mandating human oversight, but companies are still defining what ‘oversight’ means. Informed observers, including AI safety researchers at the Future of Life Institute, warn that without built-in stopping criteria, AI systems could cause systemic financial crises that regulators cannot anticipate.

What happens next depends on whether the finance industry treats ‘stopping’ as a design feature rather than an afterthought. Some firms are experimenting with reinforcement learning constraints that penalise excessive computation. Others are building ‘consequence-aware’ models that factor in the cost of being wrong. The real milestone will be the first major regulatory mandate that requires AI systems to articulate when—and why—they stop thinking. Until then, the markets are running on algorithms that never learn to hesitate.