Bank of America and Capital One are using smaller AI models to cut costs while expanding their artificial intelligence budgets, according to recent remarks from bank executives. The largest U.S. lenders are triaging complex queries to advanced systems while routing simpler tasks to compact, fine-tuned models. This approach helps the banks manage expenses without sacrificing performance.
Smart Routing and Budgets
Bank of America plans to double its AI spending next year, focusing on improving the financial assistant Erica. Chief Technology and Information Officer Hari Gopalkrishnan described a system called Orchestra that acts as a routing layer. The bank uses a small classifier model to decide which AI model should handle a specific user request. This method lets the bank direct queries to the most cost-effective option rather than using expensive models for every interaction.
Hari Gopalkrishnan said they didn’t want to be beholden to any given model. He added that the team expected to find simple classifier models capable of handling basic decision tasks without the need for large language models. The executives at Bank of America see this strategy as an effective way to control token spend and maintain flexibility across their AI operations.
Capital One has also adopted smaller models for its chat concierge service. The bank took open weight models, fine-tuned them with proprietary data, and compressed them into smaller versions. Chief Enterprise AI Officer Prem Natarajan reported that these compact models reduced latency by five times and increased dealership engagement by 50%.
Data and Guardrails
Both banks emphasize that a strong data foundation is essential for their AI efforts. Capital One has built a modern cloud tech stack and invested in machine learning to ensure high-quality data. The bank relies on well-managed, well-curated datasets to power its applications. Executives stress that the decisions made in the foundational layers of the architecture either enable or constrain the capabilities available at the top.
