Argmax CEO Atila Orhon on Developing On-Device AI Alongside Salesforce Ventures

“We highly value investors who can bet on a founding team obsessed with supercharging a fledgling market within the next couple of years.”

Caroline Fiegel and Jason Spinell

November 26, 2024

After making Argmax the first investment out of Salesforce Ventures new $500M Generative AI fund, investors Caroline Fiegel and Jason Spinnell sat down with CEO Atila Orhon to discuss Argmax’s origins, how the company has changed over time, why Argmax partnered with Salesforce Ventures, and more. The following is a transcript of their conversation, lightly edited for clarity and style.

Caroline Fiegel: What inspired you to build Argmax?

Atila Orhon: I dropped out of a PhD to join Apple in 2018 so I could build machine learning (ML) technology that billions of people use everyday. Over the years, I wore many hats: training ML models that ship as iOS features, developing tools for efficient on-device inference for other engineers, and eventually technical leadership. In 2022, we open-sourced some of our work in efficient on-device inference for Transformers and the reception changed my view on the state of the industry entirely. Things we had figured out several years ago were still bleeding edge problems for many developers and enterprises who are extremely competent in every other topic. Realizing that we were living in the future at Apple due to our early bet on on-device ML, I wanted to pull the future forward for the broader industry.

I was working on Apple Intelligence in late 2023 and it was super fun. I loved my leadership (I still do, we go hiking on weekends) and my peers but I realized that 2024 was the perfect time to start building to be ready to go-to-market when the market window opens in 2025. So in November 2023 I left Apple and founded Argmax.

I felt on-device inference was not predestined to be a huge market unless a strong player emerged to break the perceived limitations, similar to what Tesla achieved for the EV market. Why would a young startup be that strong player and not the incumbents? Because the incumbents are not fully incentive-aligned with developers who are building cross-platform products that will run on devices that were sold 5 years ago, in some cases competing with the incumbent’s first-party applications as a third-party.

Jason Spinell: How has your vision for Argmax evolved over time?

Atila Orhon: In the first month, we were dead focused on bringing frontier LLMs to mobile phones by investing heavily in model compression R&D. We were probably too aggressive in pulling the future forward and did not stop to think who needed that problem solved.

While interviewing prospective customers, we quickly realized that projects that are close to production had much more practical problems that were addressable by rigorous engineering using (mostly) existing technology. With that insight, we pivoted to building frameworks to bring popular and useful Foundation Models On Device (FMOD), starting with WhisperKit.

We became an open-source first company because we believe this is the only way to make an industry-wide impact and make on-device inference the dominant solution for Foundation Models in production. The primary benefit of open-source is that most companies that productionize Foundation Models do not reinvent the wheel — they adopt open-source systems and architectures as a starting point and tweak or fine-tune on top. Our tools and frameworks stay compatible with proprietary models of prospective customers because of their shared open-source origins.

Caroline Fiegel: Tell us about the team behind Argmax.

Atila Orhon: I met Zach Nagengast (Head of Software at Argmax) on GitHub when he dropped a huge Pull Request (PR) on Apple’s Stable Diffusion repo that I had launched. Pedro Cuenca, an ML engineer from Hugging Face, and I had been working on the exact same feature (for diffusion nerds: SDXL in Core ML) for several weeks and we had even drafted a blog post on it. When we saw Zach’s PR drop a few hours before we were about to publish our work, Pedro was like “OMG.” I reached out to Zach over Twitter and he was super cool about sunsetting his PR to let us publish ours. Over the next few months, Zach kept contributing more and more. These open-source PRs ended up becoming an unofficial 3-month long coding interview before I was convinced that he should be the chief software architect at Argmax. He joined the founding team on Day 1.

I met Brian Keene (Head of Performance at Argmax) when we joined Apple around the same time in 2018. We were on sister teams. He was always working closer to the Metal (for Apple nerds: pun intended). He was an architect on the OG team that built the first on-device inference engine that was preinstalled in iOS, a version of which backs Core ML to this day. When I heard that he serendipitously became an Apple alumni around the same time as me, we teamed up right away. There are less than a handful folks in the world who have a holistic understanding of on-device inference performance and Brian was a natural choice for the chief performance role.

Gary L. (VP of Product) was a godsend. He was independently convinced of our vision before I even uttered a single word during his interviews (which lasted a total of 7 hours across 3 sessions because he created a GTM proposal and correctly guesstimated our approach and progress up to that point). His experience match was uncanny: He witnessed consumer hardware inflections and challenges of on-device processing at MediaTek, where he spent 7 years. He also spent 7 years at Google where he was instrumental in one of Google’s first major on-device ML projects at scale: Google Lens. He joined just a few months after the founding when many folks said that “perhaps a VP of Product is too early for your company phase.” He proved me right by pioneering a new business model for our unique and opinionated technology platform in his first few months. I’m glad we invested early in onboarding him because it would have been too late to figure that all out now.

Jason Spinell: Why was Argmax seeking a new round of funding, and what kind of partners were you looking for?

Earlier this year, our open-source traction led to commercial inbound that was too strong to ignore. We decided to double down on our most in-demand project (WhisperKit) and accelerate our roadmap in Speech AI by growing the team earlier than previously expected. This turned out to be a great opportunity to seek out a new partner whose view of the future was aligned with ours.

We highly value investors who can bet on a founding team obsessed with supercharging a fledgling market within the next couple of years. Both Salesforce Ventures and General Catalyst stood out based on their market view and confidence in our approach.

Caroline Fiegel: Why did you choose Salesforce Ventures as a partner?

Atila Orhon: I interviewed several founders who partnered with Salesforce Ventures in the past and their references made a very positive impression on me.

After the initial partner meeting, your team did a quick Salesforce internal survey which showed that Argmax technology was already being used at Salesforce and that was the canary in the coal mine that led to Salesforce Ventures’ decision to lead Argmax’s seed round. It’s a very high bar that an investor would already be using your product before they invest in your company but I am glad Salesforce and Salesforce Ventures exceeded that bar and we are thrilled to partner with them.

Jason Spinell: What are you most excited about as you look ahead to the next phase of your company’s growth?

Atila Orhon: Our business model aligns our incentives with that of our customers: If their products powered by Argmax go viral, we go viral. So the next commercial phase of our company is all about supporting our customers as they go into production with their next-gen Argmax-powered products. I am most excited about demonstrating the correlation between our customers’ success metrics and the adoption of Argmax tools and frameworks.

If you’re a founder building across AI, enterprise SaaS, or impact, we’d love to talk. To get in with Caroline Fiegel, email her at cfiegel@salesforce.com. To get in touch with Jason Spinell, email him at jspinell@salesforce.com.

The Long Game: How Salesforce Ventures is Reimagining VC

Read Article

15 Years of Salesforce Ventures

Read Article