Microsoft Backs Startup Focused on Making AI More Efficient - BNN ...

4 Jun 2024

BNN

(Bloomberg) -- Touchcast Inc., a startup that caches frequently asked generative artificial intelligence queries to make AI systems more efficient, is raising $100 million from venture backers, including OpenAI investor Microsoft Corp.

New York-based Touchcast stores and serves up responses for commonly used AI prompts, reducing the computational resources and energy required for using models like OpenAI’s GPT-4. The aim is to help developers and businesses deploy generative AI more broadly with lower costs and faster response times.

The Accenture Plc-backed Touchcast will have a valuation of at least $350 million after the new funding, Chief Executive Officer Edo Segal said, declining to provide the details of the capital Microsoft has infused. Touchcast is in discussions with large Silicon Valley investors, he said in an interview.

Businesses are deploying AI systems to streamline daily tasks, but skyrocketing demand has laid bare a critical bottleneck in electronics, particularly GPUs, the scarce chips used to train and operate AI models. Touchcast’s approach could be an innovative way to make better use of computing and energy resources, Segal said.

The startup recently launched what it calls cognitive cache content delivery technology, which is akin to having several small library desks in different parts of a giant library instead of a single main desk, making it easier to access material. The technology helps improve the performance of the AI system.

“When users ask the same question, they go to the large language model over and over,” Segal said. In the era of the AI-powered web, “a new class of infrastructure is critical to deliver these magical experiences at internet scale, sustainably.”

Touchcast is working with Microsoft to create systems to speed up and broaden the distribution of generative AI queries. It’s deploying the cognitive caching technology across Microsoft’s Azure data centers, helping to run Azure OpenAI tools to service customers and bring down costs significantly.

“The explosive growth of generative AI is on track to unlock trillions in economic value,” Segal said, “but its fate hinges on solving an escalating compute and energy crisis.”