AI Infrastructure Explained

Innovative applications of AI have captured the public’s imagination over the past year and a half. What’s less appreciated or understood is the infrastructure powering these AI-enabled technologies. But as foundational models get more powerful, we’ll need a strong technology stack that balances performance, cost, and security to enable widespread AI adoption and innovation.

As such, Salesforce Ventures views AI infrastructure as a crucial part of the market to build and invest in. Within AI infrastructure, a few key elements are most critical: Graphic Processing Units (known as “GPUs”), software that enables usage of GPUs, and the cloud providers that link the hardware and software together.

Understanding these three elements—how they work, how they’re delivered, and what the market looks like—will help founders and innovators execute more effectively and identify new opportunities. In this write-up, we present our analysis of the AI infrastructure stack, starting with the basics of GPUs and software components and moving to the ways products are delivered and how the market is segmented. 

GPU Hardware

GPUs are the hardware that powers AI. 

While many traditional servers utilize computer processing units (CPUs), these processors aren’t designed to support parallel computing, which is needed for specialized tasks like deep learning, video gaming / animations, autonomous vehicles, and cryptography. The key difference between GPUs and CPUs is that CPUs have fewer processing cores, and these cores are generalized for running many types of code. GPU cores are simpler and specialized for data-parallel numerical computations, meaning they can perform the same operation on many data points in parallel. 

GPUs are organized into nodes (single server/computing unit), racks (enclosures designed to house multiple sets of computing units and components so they can be stacked), and clusters (a group of connected nodes) within data centers. Users have access to the GPUs at these data centers through virtualization into cloud instances. Data centers that house GPUs must be configured differently than traditional CPU data centers. This is because GPUs require much higher bandwidth for communication between nodes during distributed training, making proprietary interconnects such as InfiniBands necessary. 

The density of GPU servers and their high power draw calls for planning around power provisioning, backup, and liquid cooling to ensure uptime. Further, the topology (physical interconnection layout) of GPUs is also different from CPUs—GPU interconnect topologies are specially designed to deliver maximum bi-sectional bandwidth to satisfy the communication demands of massively parallel GPU workloads. By contrast, CPU interconnects focus more on low-latency data sharing. 

In terms of GPU manufacturers, NVIDIA currently dominates with an 80%+ market share. Other noteworthy players include AMD and Intel. Demand for GPUs currently far outweighs supply, prompting intense competition amongst manufacturers and encouraging customers to experiment with building their own AI chips. 

Aside from big tech companies, there are also a few startups attempting chip design. These new entrants are oftentimes focused on optimizing the design for specific use cases or AI workloads. For example, Groq’s LPU (Language Processing Unit) is solving for cost-effective latency, and Etched’s Sohu chip is designed to run transformer-based models efficiently. We’re excited to see the innovation in the space and market supply and demand dynamics at work. 

GPU Software

Supporting these GPUs are the various software solutions that interact directly with the GPU clusters and are installed on different levels (i.e., nodes, racks, or clusters). The following is a non-comprehensive list of the types of software associated with GPU workloads:

• Operating systems: The operating system handles the scheduling of processes and threads across CPUs and GPUs. It allocates memory and I/O appropriately. Examples of GPU operating systems include CentOS, RHEL, and Ubuntu. 
• GPU drivers: GPU drivers are vendor-provided parallel computing platforms that control the GPU hardware. Examples of GPU drivers include NVIDIA, CUDA, and the open-source AMD ROCm.
• Cluster management / job scheduling: Cluster management software allocates GPUs to submitted jobs based on constraints and availability, distributes batch jobs and processes across the cluster, manages queues and priorities for diverse workloads, and integrates with provisioning tools. Examples include Kubernetes and Slurm.
• Provisioning tools: Provisioning tools provide containers / isolated environments for applications or jobs to run on the cluster and allow for portability to different environments. Examples include Docker and Singularity. 
• Monitoring software: Monitoring software tracks specialized metrics and data specific to AI operations. Examples include Prometheus, Grafana, and Elastic Stack.
• Deep learning frameworks: Frameworks that are specifically designed to take advantage of GPU hardware. These are essentially libraries for programming with tensors (multi-dimensional arrays that represent data and parameters in deep neural networks) and are used to develop deep learning models. Examples of deep learning frameworks include TensorFlow and PyTorch.
• Compilers: Compilers are development environments to build optimized code. Examples include the NVCC compiler for NVIDIA’s CUDA code and HCC/HIP compilers for AMD ROCm GPU code.

These softwares help infrastructure teams provision, maintain, and monitor GPU cloud resources.

The hardware / software combination matters a lot for the type of AI workloads being performed. For example, distributed training (training an AI model as fast and effectively as possible) typically requires multiple servers with best-in-class GPUs and high node-to-node bandwidth. Meanwhile, production inference (running the job of an AI application) needs GPU clusters that are configured in a way to handle thousands of requests simultaneously, usually relying on optimized inference engines like TensorRT, vLLM, or proprietary stacks, such as Together AI’s Inference Engine.

In the next section, we provide an overview of what we consider the current landscape of cloud infrastructure that can accommodate various AI workloads. 

The 3 Types of GPU Cloud Providers

Salesforce Ventures’ view of the market is that there are currently three types of GPU cloud providers. Each provider has its own benefits depending on the desired use case. 

Hyperscalers

The first type of GPU cloud provider are Hyperscalers: cloud computing providers that operate massive, globally distributed data centers and cloud infrastructure and offer a wide array of computing services. While Hyperscalers haven’t historically focused on GPUs, they’ve recently expanded their offerings to GPUs to capture the immense market demand for AI technologies. Notable Hyperscalers include household names like AWS, Google Cloud, Azure, Oracle, and IBM Cloud.

In terms of infrastructure, Hyperscalers own their GPUs but co-locate these GPUs with colocation data center operators or maintain their own data centers where their chips are housed. Software provided by Hyperscalers is largely product dependent—some might have lower-level software just for managing GPU clusters, while others provide higher-level abstractions that are MLOps focused.

For example, AWS has EC2 instances that offer different types of GPUs (e.g., T4, A10, A100, H100, etc.) as well as deep learning software solutions (e.g., AWS Deep Learning AMIs) that include the training frameworks, dependencies, and tools developers can utilize to build and deploy models. Meanwhile, AWS Bedrock offers API endpoints of popular open-source and closed-source models, abstracting away the interim steps of deployment.

Specialized Cloud Providers

The second type of GPU cloud provider is what we call “Specialized Cloud Providers.” Unlike Hyperscalers, these organizations are focused on providing GPU-specific infrastructure for AI and high-performance computing workloads. Examples of specialized cloud providers include CoreWeave, Lambda Labs, Massed Compute, Crusoe, and RunPod.

Like Hyperscalers, Specialized Cloud Providers own their GPUs, but either co-locate them with colocation data center operators or operate their own data centers. These providers either offer bare-metal GPU clusters (hardware units networked together) or GPU clusters with a basic software layer that enables users to operate the clusters and virtualization layers to spin up cloud instances—similar to the EC2 instances at AWS.

Both Hyperscalers and Specialized Cloud Providers require massive upfront capital outlays to buy and install the GPUs in their data centers, and sometimes build and operate the data centers themselves (if they’re not “colocated”).

Inference-as-a-Service / Serverless Endpoints

The third type of GPU cloud provider encompasses a broader array of companies that we bucket under “Inference-as-a-Service” or “Serverless Endpoints.” These are newer entrants to the market that offer software abstraction on top of GPU clouds so users only interact with the API endpoints where models are fine-tuned and deployed for inference. Examples of companies in this space include Together AI, which we recently led a round of financing in, Fireworks.ai, Baseten, Anyscale, Modal, OctoML, Lepton AI, and Fal, among others.

Most Inference-as-a-Service providers get their GPUs from Hyperscalers or Specialized Cloud Providers. Some rent GPUs and make a margin on top of the unit cost, some form revenue-sharing partnerships, and some are pure passthrough (i.e., revenue goes directly to the GPU supplier). These companies typically have a software layer with the highest level of abstraction so that users likely don’t have visibility into what GPU cloud provider or specific SKU of GPU/networking configuration is utilized. Users also rely on these companies to perform MLOps-type value-adds such as autoscaling, resolving cold starts, and maintaining the best performance possible. Oftentimes, Serverless Endpoint companies build their own proprietary stack of optimizations to improve cost and performance (often a balance between latency and throughput). 

Inference-as-a-Service has gained traction with the new wave of generative AI because these providers take away many steps around provisioning and maintaining infrastructure. If a startup opted to use a product from a cloud vendor, it’d likely need to:

• Select a GPU instance (based on model performance requirements), 
• Launch that instance, 
• Install and set up the necessary software (e.g., GPU drivers, deep learning libraries), 
• Containerize the model, 
• Transfer the container to the EC2 instance, 
• Install serving software depending on model format and deployment requirements, and
• Configure the serving software to load the model and expose it as an API endpoint. 

However, if a startup was to deploy an LLM with a provider like Together AI, all it’d need to do is select the relevant model from the Together Playground, launch the model using the serverless endpoint provided by Together AI, and build the inference endpoint into a generative AI application using the API key and Python/Javascript SDK. Together AI also performs maintenance for its users.

Also note that some players in this broad category offer products targeting distributed training workloads, including Together AI and Foundry. Given the need for larger GPU clusters to train models (vs. serving/running inference), these products have a different form factor from Serverless Endpoints.

Organizing AI Infrastructure

We hope our organization of the current AI infrastructure stack can inform founders on how to orchestrate their own infrastructure and spark ideas for how to innovate and improve upon the current technologies. In our next post, we’ll dive deeper into the viability of the various types of GPU cloud providers, and detail where we see opportunities for innovation.

—

If you’re a founder building in AI, we’d love to talk. Salesforce Ventures is currently investing in best-in-class AI tooling and horizontal or vertical applications. To learn more, email me at emily@salesforceventures.com

Selling to Utilities: A Guide for Early-Stage Climate & SaaS Startups

Utilities sit at the center of the clean energy transition. These enterprises own the energy grid infrastructure that powers our homes and businesses. Importantly, they must meet sustainability targets while ensuring reliable and affordable electricity supply to customers.

At Salesforce Ventures Impact Fund, we’ve been investing in climate tech for the past seven years. We understand how critical utility innovation is to a cleaner future. At the same time, we recognize the difficulties startups face selling these pioneering technologies to utility companies. 

For many early-stage startups, earning a contract with a utility provider is a major breakthrough that can set a business on a path to success. However, the process of selling to utilities can be beset by long sales-cycles, numerous decision-makers, varying regulations, and a culture that can be risk averse to emerging technologies. 

For these reasons, Salesforce Ventures recently hosted a workshop to support portfolio companies selling to regulated utilities. The session featured Larry Goldstein, Senior Director of Product Strategy for Salesforce’s Energy & Utilities Cloud, and Leo Trudel, Director of Innovation and Technology at Indigo Advisory Group, a digital strategy consulting firm for electric utilities.

Larry and Leo shared practical advice for startups hoping to break into the utilities market. In this guide, we present an overview of their framework, supplemented with our own insights and best practices for a utility go-to-market (GTM) motion. 

Understand your ideal customer profile (ICP)

The ideal customer profile defines the qualities of the stakeholders being targeted by a sales pitch. Utilities are large enterprises with multiple stakeholders involved in purchasing decisions. Selling into utilities requires a strategic selling approach that includes mapping these stakeholders, their key drivers, and their influence on the decision making process. While every utility company is different, there are generally five groups of stakeholders that startups should focus on during the sales process, each with their own set of priorities:

1. C-suite: Utility company executives are primarily concerned with achieving their key performance indicators (KPIs). These KPIs may pertain to customer satisfaction, net zero goals, safety, reliability, or operational efficiency. Startups should aim to present the C-suite with proof points of other utility companies that hit their KPIs using the startup’s solution.
2. Business users: Business users will look to integrate a new product or service if they believe it will help them achieve success on a project or program. The business user’s need can help push an IT organization that otherwise may not be interested in integrating an emerging technology from an early-stage company. 
3. IT: IT will evaluate new software based on a number of criteria. IT will likely first consider whether an existing software can address the problem or if they can build the solution internally. IT will then consider ease of integration with the utility’s existing workflows and systems. Next, IT will conduct a security review process, with specific criteria for cloud-hosted software. It’s important to remember that large utilities typically have hundreds of applications, so they will be comparing new solutions to existing platforms. Also note that IT stakeholders are heavily influenced by their third-party systems integrators (SIs). SIs have their own set of priorities, which sometimes differ from those of the utility and third party vendors. When selling to IT, the business user will be the startup’s best ally throughout the sale, as IT ultimately serves the interests of the business user. Pushback from IT typically occurs when the team feels they can build the solution internally, the software poses security or skill set concerns (i.e., IT doesn’t understand the software enough to support it or has an SI that understands it), or there are budget or timing constraints.
4. Procurement: The procurement department oversees all software acquisitions. Given the lengthy cycles associated with utility sales, these stakeholders are most concerned with vendor viability (i.e., how well-capitalized the vendor is and who its customers are).
5. Regulatory affairs: Regulatory affairs is focused on meeting the utility’s regulatory requirements and regulatory stakeholder management. The regulatory team is also the lead in utility filings and proceedings related to utility funding—including the utility general rate case. The rate case is the primary regulatory funding proceeding for any investor-owned utility, as well as program-specific filings such those for energy efficiency and electric vehicle programs (more on rate cases further down). Engaging the regulatory team can help startups understand if and how software and implementation costs can be covered by various funding sources. For example, startups can explore whether the cost of the software license can be capitalized (CapEx) or treated as an operational expense (OpEx). Regulatory has an agenda and goals with each of their regulatory filings. If a startup’s software solution helps in supporting that regulatory agenda, it can earn a strong advocate within the utility.

Given every utility company operates somewhat differently, a best practice is to rewrite the ICPs for each individual utility provider being sold into. 

“Be rigorous in mapping your stakeholders—the people who make decisions, influence decisions, or who can shut you down,” Goldstein said. “Understand who they are, what their title is, what their concerns are, and what their motivations are.”

While utility sales cycles can be lengthy, generating interest and excitement from internal customer groups can be accomplished using ordinary sales motions. If startups present a solution that solves a business problem the utility can’t accomplish on its own, requires little training, has good data transparency, and a positive market reputation, they’ll be able to generate interest. 

Learn the culture of utilities

Utilities companies must deliver safe and reliable service to customers. As such, these organizations move cautiously and are often slow to change as a means of managing risk. Commonly, utilities are also siloed, with communication between departments restricted as a result of longstanding cultural norms and mandatory security protocols. Further, utility companies are not particularly tech-forward relative to their industry counterparts, meaning they’re not overly familiar with SaaS solutions or how they can support the business.

Given this culture, startups selling to utilities must wear three different hats:

1. Evangelist: Startups must promote their product and find or develop internal champions to promote it to colleagues (e.g., business users, regulatory affairs). Because utility companies are siloed, startups must take the initiative to move horizontally through the organization to ensure all potential stakeholders are aware of their solution and see value in it. Note that utilities prioritize reliability in virtually all purchases they make. They often associate reliability with brands that have solid, long-standing reputations. For this reason, utility companies may avoid doing business with startups. As such, it can be advantageous for startups to emphasize any high-profile partnerships (including well-known investors, affiliates, systems integrators) to foster credibility with stakeholders. Given regulated utilities do not directly compete, they tend to communicate and share information with each other and often look to other utilities as references on a proposed solution.
2. Educator: Startups must explain how to think about the solution and how stakeholders should evaluate it. Operators at utility companies have typically been in their role for many years and may not know how best to gauge the value of a SaaS tool. So if, for example, the product in question uses computer vision to assess asset health, explain to Procurement that the best way to evaluate their options is to administer tests that are graded on processing time and accuracy. 
3. Sales Engineer / Systems Architect: A startup should be able to propose architecture for how its solutions can fit into the utility’s existing environment both from a systems replacement and integration perspective. Because utilities are risk-averse, startups should expect roadblocks and come to the table with ideas on how to navigate them. 

It’s important for startups to learn the “idiosyncrasies” of the utility in order to engage in the most effective fashion.

“Each utility is different, and you’re not going to figure out how these organizations buy software unless you do your research and talk to a bunch of internal stakeholders,” Trudel said. “And then once you understand those dynamics, it’s up to you to execute a strategy that gets everybody on board and the deal moving forward.”

Understand where the money comes from

Utilities receive the majority of their funding through regulatory filings and proceedings. The largest of these, as previously mentioned, is the general rate case.

Rate casing is the process by which a utility company sets the rate it charges customers for services to match the costs incurred to provide those services. The rate case process involves preparing and filing a rate case with the regulatory body that governs the utility, evidence gathering, and several rounds of public hearings (this video provides a good overview of the rate case process).

Having a rate case approved essentially enables the utility to pass on increases in costs (say, from a new SaaS tool) to customers. Supporting a utility provider with their rate case can help a startup secure the sale. Because the rate case process is different for every utility (utilities report into different federal and state regulators), startups should look into whether their target customers have rate-cased SaaS products in the past. If they have, seek to understand how the utility successfully rate cased the software, as this can provide insight into how future SaaS expenses can be passed along to customers. Note that rate case eligibility typically applies to CapEx and not ongoing expenses, such as recurring SaaS fees. 

“One thing that can be really helpful is looking at old rate cases that have included SaaS software, and then asking the regulatory affairs folks how they handled that process,” Trudel said. “You can then play the role of sales engineer and figure out what you can do to get your software included in the rate case based on the rules the utility company must abide by.”

There are 50 different state regulators as well as numerous federal regulators governing more than 5,000 utilities providers in the U.S. Each of these regulators has a different level of regulatory oversight and compliance related requirements. What’s more, within each utility there are numerous individuals who have varying knowledge of regulatory processes and compliance procedures. 

This mosaic of compliance rules and stakeholder interpretations means there’s no single solution for winning rate case approval that applies to all customers. SaaS companies should engage with stakeholders to gain knowledge of each client’s interpretation of their unique regulatory frameworks, share best practices and anecdotes of other utilities that won rate case approval for SaaS products, and see if there’s a path forward for securing funding via rate increases, rather than from utilities’ profit margins. 

Because rate-casing is a complex and time-intensive process, startups should also consider other sources of capital that may be used to compensate vendors. Again, this requires understanding the idiosyncrasies of the utility provider. There are regulatory programs for energy efficiency, electric vehicles, and income qualified programs that provide funding specific to meeting those regulatory program goals.

Understanding what funding source(s) are relevant and can be applied to software license and implementation costs is critical to getting a SaaS purchase project funded—as is understanding how those software and implementation costs are allowed to be treated (i.e., CapEx vs. OpEx).

Utility sales in summary

Utility sales is a complex enterprise selling cycle that can feel daunting to a startup. But startups that take the time and effort to understand the organization, stakeholder dynamics, and funding mechanisms described above can find success. The value of these contracts—both financial and reputational—often makes utility sales a worthwhile investment. 

Climate tech startups have incredible solutions from which utilities could benefit. We hope the sales strategies we’ve detailed in this guide can facilitate increased adoption of these solutions in the coming years.

—

Salesforce Ventures Impact Fund hosts frequent workshops with Salesforce experts designed to address our portfolio companies’ recurring challenges and chart a path to success. The Impact Fund is currently investing in enterprise software startups that drive measurable social and environmental impact. To learn more about the Salesforce Ventures Impact Fund, visit our website.

Disclaimer

The information provided in this article does not, and is not intended to, constitute legal or financial advice; instead, all information, content, and materials available are for general informational purposes only. Readers should contact their attorney or financial representative to obtain advice with respect to any particular legal or financial matter. Opinions of the referenced presenters and/or author are their own and do not necessarily reflect the official position of Salesforce.

Salesforce Ventures Founder John Somorjai on the ‘Ask More of AI’ Podcast

John Somorjai, Chief Corporate Development and Investments Officer at Salesforce Ventures, recently sat down with Clara Shih, CEO of Salesforce AI, on the “Ask More of AI” podcast for a wide-ranging conversation on the origins of Salesforce Ventures, Salesforce Ventures’ AI investment portfolio, how Salesforce Ventures applies its values in all investment decisions, and much more.

Here were a few of our favorite takeaways*.

*Quotes have been edited for clarity and concision.

On what inspired John to launch Salesforce Ventures… 

“We started talking about creating a venture firm towards the end of 2008. It was the middle of the financial crisis, and we had many AppExchange partners who were struggling to raise funding. We had created this incredible ecosystem of partners that would tightly integrate with our products and it was really important for our business and our customers that they be able to access these solutions.”

“So we worked with Marc (Benioff) to create a venture arm that would fund within the broader Salesforce ecosystem and allow Salesforce to become the anchor investor of a fundraising round and pull in other investors. Companies we invested in during that time include DocuSign and Box.”

“From our origins, we could have never imagined the success Salesforce Ventures has had. The world changed and enterprises realized they could run their companies more efficiently by moving their systems to the cloud. So Salesforce, and all the companies that grew up around us, really benefited from changes in the industry.”

On what differentiates Salesforce Ventures from other VC firms…

“Through that early experience, we realized we have a few advantages as investors. One is we understand buying signals and have a good sense for the types of companies customers would want to buy from. Additionally, we have in-house experts that understand enterprise software and how to create the best products for the cloud that are secure, reliable, and scalable. So we can leverage the expertise of our entire organization to make better investment decisions.”

On launching Salesforce Ventures’ Generative AI Fund…

“Generative AI is one of the most transformative technologies we’ve seen in a long time in our industry. When we started to see interesting business models take off with this technology, we decided we wanted to be at the forefront of deploying capital into the industry to make sure we’re getting into the best companies while building an ecosystem of partners around our internal AI efforts.”

On Salesforce Ventures’ investment in application-layer AI companies…

“Notable application layer investments include Runway, Typeface, and AutogenAI. What all of these companies have in common is how much productivity they add back to the company. McKinsey says AI is going to save $4 trillion every year for companies. That’s an enormous productivity uplift. I think these companies are really on the cusp of breaking through and becoming very, very successful large businesses because when you can drive that much efficiency for a business, who’s not going to want to buy?”

On how Salesforce Ventures leads with its values… 

“One of the greatest things about Salesforce is how much we value giving back. Our 1-1-1 model stipulates 1% of employee time is dedicated to volunteerism, 1% of our product is given away for free, and we put 1% of our equity into a 501(c)(3) foundation, which has now been able to give away over $700M in grants. We’ve brought our 1-1-1 model to our portfolio, and now have 190+ companies using it. They’re building on our success and helping their communities.”

“Another example is our focus on diversity and inclusion. We invest in many minority founders, and fund the Black Venture Institute, which trains black operators to become check writers. So I think everything we do with our values internally, we try to bring that to the companies we work with.”

_

Watch John’s full conversation with Clara here:

In Conversation With Anthropic Co-Founder Tom Brown

Salesforce Ventures recently hosted a dinner and networking event for a group of portfolio companies and Fortune 500 executives in San Francisco. The evening was highlighted by a fireside chat between Tom Brown, co-founder of Salesforce Ventures’ portfolio company Anthropic, and Salesforce President and Chief Product Officer David Schmaier. 

The duo discussed the origins of Anthropic, the imperative of AI safety, techniques for generating better LLM outputs, generative AI use cases, improving AI accuracy, the prospect of artificial general intelligence (AGI), and how to ensure AI will be used as a force for good in the world. 

Their conversation featured a ton of great insights for founders, builders, and AI enthusiasts alike. Here were a few of our top takeaways*…

*Quotes have been edited for clarity and concision

On Anthropic’s approach to creating ‘harmless’ AI…

“Many models are trained using reinforcement learning from human feedback (RLHF). The idea behind RLHF is you reward and punish the model for doing well or not doing well on the paths you care about. We had people upvoting or downvoting how well the model does a task. That’s how you can make the model become a harmless assistant.” 

“I think we noticed that as the models were getting smarter, they started to do most tasks well. We developed constitutional AI to turn a model into the entity that upvotes or downvotes another model. A person can write up a constitution of what it means to be helpful, harmless, and honest, and then a model will read the interactions between the human and the assistant and consider if the assistant is acting in accordance with the constitution. This is a way to take a simple document and turn it into a model personality.”

On the impact of ‘stacking’ LLMs to generate higher-quality outputs…

“We have Claude 2.1, which is a large model, and then we have Claude Instant, which is a smaller model. Depending on the task, sometimes you’ll want a smaller model because it’s faster and cheaper. For example, Midjourney is one of our customers. Whenever you put any prompt into Midjourney to generate text, it’ll pass it through Claude Instant and Claude Instant checks if it’s violating Midjourney’s terms of service. And if it thinks it might be, it’ll give a little message to the user saying ‘this might be against our terms. Do you want to appeal it?’ And if you hit yes, it goes to Claude’s Instant’s boss, which is Claude 2.1, who thinks about it a little bit longer, and maybe says, ‘Sorry, Claude Instant was totally wrong. You’re fine actually.’”

On building domain-specific models…

“There’s two different ways I think about building a domain-specific model. One is that you take a large model and fine tune it to make it better at a specific task. The other is you build a narrow model that’s good at one specific thing. Claude Instant is faster and cheaper than Claude 2.1, but less performant. You can fine tune either one, but Claude 2.1 will still do better. So I think that’s my normal mental model for what’s going on with fine tuning. You could imagine someone building a very narrow model that’s very good at one specific thing, but it feels like fine tuning a larger model is the thing people have successfully done rather than making a super small model good at some narrow field.”

On new AI use cases….

“I think code is a place where the models do great. Retrieval augmented generation (RAG) for quality assurance is a broad area that the models can do and be really useful at. I feel like customer service is an area that fits that very well.”

On cutting down on AI hallucinations…

“We have a bunch of internal metrics for hallucination rates that we measure and a team that’s fully focused on bringing that number down. Claude 2.1 came out last month, about four months after Claude 2. On our internal dashboards it reduced the error rate by 2x. It still has some hallucinations, but there are half as many as there used to be. So if, for example, you’re seeing 98% accuracy with your model now and you need to get to 99%, maybe it’ll only take four months. If you need to get 99.9%, it might take a year or something like that.”

“Also, if it’s a harder task, the model is more likely to hallucinate or make a mistake. It’s the same as if a person is trying to juggle a bunch of tasks at once. You’ll make more mistakes than if you’re just focused on one thing.”

On the prospect of AGI…

“I think we already have weak AGI. You can talk with ChatGPT or Claude and it’s somewhat intelligent, and with each year it’s improving.”

“Right now you may prompt the model to build a successful startup. It’ll try to do it, but it’ll get stuck. But as we add more compute, the model gets smarter. The model’s IQ goes up. So I think it would be surprising if the models suddenly stopped getting better. And then it’s a question of how much better can they get? One thing we don’t know is how far the model has to go for it to be an AI entrepreneur that could do a great job. But every year it seems like we’re getting more IQ points so at some point I think we’ll get models that are better at engineering than I am.”

On whether AI will be used for good or evil…

“I feel like I’m cautiously optimistic in general. People aren’t angels and people aren’t devils. People are people. So I think people will use AI for all sorts of different stuff and it’s up to us as a society to make sure that the benefits outweigh the costs. I’ve been really heartened by the recent regulatory updates. I’ve always worried that the government would get involved and do things that don’t quite make sense. But the recent AI executive order seems quite sensible. So I think I’m much more optimistic now than I was a year and a half ago.”

—

To learn more about Salesforce Ventures’ investment in Anthropic, click here. To read more about our Generative AI fund, click here.

FY23 Year-in-Review

Forget Leadership Tips: Stay True to You

At Salesforce Ventures, we believe that values-led organizations are more successful and contribute more to the communities around them. We know that the most sought-after organizations foster communities where people share a sense of values, purpose, identity, trust, and belonging. Along those lines, on a brisk day in early November, we hosted “Uplift,” a full-day mindfulness and leadership event to support female-identifying and non-binary leaders, including founders and investors.

On a stage bathed in stained glass-tinted light, Salesforce President and Chief Financial Officer, Amy Weaver, and Salesforce Ventures Chief Operating Officer, Khushboo Patel, got together to discuss leadership and dove into topics such as authenticity, identity, and advocacy. 

The importance of diversifying senior management

Their shared perspective was that as organizations begin to diversify their ranks of senior management, society needs to become more open-minded to, and welcoming of, new styles of leadership. This includes what a leader looks and sounds like, as well as how they behave and the behaviors they reinforce on their teams. “I think we’ve made strides in getting more people of every background, gender, and race into the workplace,” said Weaver. “What we haven’t done as well is show differences in styles of leadership. And that’s what we really have to concentrate on.”

“In the next ten-plus years, there will be a different group of people leading many companies. How do we meet them in the middle? How do we tailor our approach so that it’s a great working relationship?”

— Khushboo Patel

One of the challenges in attracting, retaining, and elevating underrepresented talent is that, in large part, many companies are still expecting the next generation of leaders to be more like the legacy model. Leaders from underrepresented groups can be perceived to be quiet, and lacking in gravitas, or, conversely, they can be judged as too aggressive —  a largely unwelcome personality trait for women and people of color. Either way, they can struggle to be heard in meetings, build trust, maximize their performance, and convey confidence.

“It’s great to get people in the door but if you’re saying the only way to move up is to act like every other leader who came behind you or in front of you, that’s not diversity. That is not authenticity.”

– Amy Weaver

Striking the right tone can be a balancing act. Trying too hard to act like a more traditional leader can feel uncomfortable and inauthentic. Yet being yourself can also end up backfiring — especially if you’re in an organization where you are among the first underrepresented leaders in a senior position. 

With so many expectations, how are underrepresented people supposed to find their way?

Weaver, who has had a number of senior roles in her career, including Executive Vice President and General Counsel of Univar Solutions Inc., and Senior Vice President and Deputy General Counsel at Expedia Inc., says finding your leadership style is a constant evolution. It takes practice and patience. Weaver is a soft-spoken person. Years ago, in preparation for an important meeting with a senior male executive, her mentor advised her to use four-letter words and demand what she wanted. She refused. It didn’t feel authentic and she didn’t think it would work. She was right. She went in as herself, highlighted her record, and got what she came for. 

“It’s not enough to say niceness isn’t a weakness,” Weaver says. “We have to treat it as a strength. We’ve got to reward the people who are acting this way. We need to elevate them, make sure that is the behavior we are modeling, and not for a second think that it is a negative.”

Sometimes, you need to give yourself and the people around you some grace as you find the right approach. 

“We’ve got to show that there are different ways to lead. A leader does not have to be the loudest. A leader does not have to be pounding the table. They do not have to be the person who’s speaking the most. We’ve got to make room for that and encourage that. We’ve got to make room for the fact that there are going to be different ways of succeeding.”

– Amy Weaver

Patel added that this goes both ways: When you “have the floor,” you should go out of your way to ask others what they think — especially those who haven’t spoken up in a while. Then listen to what they say. That will go a long way toward building a culture where people aren’t afraid to speak up when they disagree with the rest of the room. “When you’re feeling empowered to speak your mind, you have psychological safety to voice your opinion and concern,” says Patel. 

Lately, multiple companies and investment firms have been in the news for decisions they made that were counter to the organizations’ historical norms and stated values. Those decisions ended up putting them in precarious situations where billions of dollars were put at risk. Much has been written about the idea that there must have been people in those rooms who disagreed with the choices being made, but didn’t feel they could state their opinions without experiencing backlash. 

“Existing leaders need to look around and leverage all the people around them and ask, ‘Who is missing? Who is integral to this decision? Who can add value to this company, this firm, or this initiative? Are the right folks at the table?”

– Khushboo Patel

The biggest takeaway from Weaver and Patel is that if you are an emerging leader and you look and sound different from the historical standard, don’t be dismayed. There are allies and advocates all around you. Stay true to your values. And when you have the chance, extend the same opportunity to the next generation.

For more takeaways from Uplift, visit: salesforceventures.com/uplift