Could AI Change the Way We Do Meetings?

Over the many years I’ve been leading organizations, I’ve consistently had people complain about spending too much time in meetings. While I’ve tried many different ways to reduce meeting overload, most of my efforts have failed to have a lasting impact, and, inevitably,  meetings have crept back onto calendars. Often, I found that when I reduced the size of a meeting, someone would feel left out and ask to be invited back in.  

There is a better way to improve meeting culture, but it would require more fundamental changes to how organizations think about meeting cadence and attendance. By using transcription tools to publish, organize, and share meeting content more broadly across an organization, leaders could reduce meeting load while still maintaining a culture of transparency.

With advances in AI, we’re finally getting to a point where this solution is feasible. In the coming years, it will likely become commonplace for meetings to be recorded, transcribed, and summarized.  

The idea that meeting content could be leveraged more effectively is something that my guest for this week’s newsletter, Krish Ramineni, is highly passionate about. Krish is the co-founder and CEO of Fireflies.ai, an AI meeting assistant. Fireflies works by joining meetings across various video conferencing platforms, transcribing the meeting, taking notes with summaries and action items, and providing a searchable database of previous conversations.

In August, IVP partner Michael Miao and I sat down with Krish to discuss his approach to solving customer pain points, how he’s using AI technology, and his broader vision (along with some of the cool, future projects he’s working on).

Start with the customer pain point: recall and sharing from meetings

“One of the biggest issues that led me to start Fireflies is when I realized how much valuable information is lost in meetings,” Krish told me. “I can go back to an email I sent two years ago, but I can’t remember a conversation I had two hours ago. That’s a big problem, especially given the fact that meetings are not only a fundamental part of knowledge work, but that they often cost thousands of dollars when you consider the expense of getting four people in a room. The thesis for Fireflies was to unearth the insights buried inside these valuable conversations. Ideally, Fireflies will provide perfect recall of everything you've ever talked about.”

How is Fireflies leveraging AI and LLMs? 

Fireflies has been using ML models for years, long before OpenAI entered into public consciousness, but the introduction of GPT3 made the bulk of Fireflies’ work much, much easier. With GPT3 it is now possible to build out a chat interface, summarize transcripts in notes format, and customize writing styles. With LLMs, Fireflies could now mimic each user’s style including specific preferences for bullet lists or more detailed notes all without needing multiple ML models.

But is the quality of LLMs good enough to rely on for summarization?

When it comes to accurate summarization, Krish thinks the models are improving all the time. “They get people 80% of the way there,” he said. “Our current models often summarize items I completely forgot were discussed.” But the technology isn’t perfect. Fireflies’ output still occasionally requires human intervention to tweak information that’s missing or unnecessary. 

“You're competing with the friction of having to spend 30 to 40 minutes creating a manual summary,” Krish said. It may not get the perfect version or perfect draft the first time, but it saves significant time.

Choosing an LLM: Pick the best solution and run with it

For founders weighing their options when it comes to which LLM to use, Krish says don’t overthink it. The reality is that whichever LLM you choose to work with will likely fall short of your customer’s needs at least some percentage of the time.

“I came to the realization early on that certain models work well for certain things while other models work better for other things,” Krish said. “For example, maybe I need to train an Open Source LLM for an enterprise customer because of how they want the data handled.”

Currently, Fireflies is working with multiple LLMs including models from Anthropic, OpenAI, and Cohere. “We want to use the best-in-class for different things, and we want to recommend different options to our users,” he said. “I think optionality is the most important thing that any team can have.”

Other ways that Fireflies is using LLMs 

Early on, Fireflies set out to incorporate LLMs into every aspect of its product. To do so, Krish looked at the manual processes that people do every day at work. For example, Krish regularly passes along feedback to his team after a customer meeting. Now, he uses the Fireflies meeting summarization to highlight a 30-second clip from customer meetings to share relevant feedback and bug reports more broadly with the Fireflies’ team. 

In the future, you’ll be able to make specific complex requests of Fireflies such as, “Can you please go through all these customer conversations, create clips every time the customer talks about a feature request, and make it into a highlight reel?” Additionally, it will enable users to upload meeting content into a variety of formats including emails, podcasts, or blog posts.  

Building Apps on a Conversational Platform

Krish is a big proponent of software that lets users tailor the product to their own needs. “One of my favorite products back in the day was Trello, but they were horizontal to a fault. If you needed any level of customization, it was very, very difficult,” he said. “Today, you have these great companies like Monday, AirTable, and Notion that give a lot of credit to the end-users who can toy with the product. People essentially run their lives on this type of software.”

Krish believes that LLMs will allow users to optimize software for their own personal needs to an even greater extent than ever before. “All of these LLMs are just agents or teammates,” he said. “I want people to think of Fireflies as a teammate that can do really personalized, individualized work for them.”

This means that different users will be able to create highly detailed prompts that generate output in the users’ desired voice, style, and format. For instance, a recruiter could customize prompts to summarize interviews while a CEO could generate board meeting content. For every role and use case, users are offered a wide range of different prompts and temperature controls for the model.

“We’re taking the conversational platform and allowing people to build apps on top. Note-taking is one app. There are hundreds of other apps people can build and customize each one of those apps to their liking.”

To enable this, Fireflies has recently rolled out a new feature called Apps which allows users to create editable building blocks that generate content into customized formats. These building blocks are prompts that people can customize and integrate with their workflow. For example, summarizing a sales call, updating Salesforce based on the information, and automatically sending a Slack message to people noting their action items.

“Our goal is to give control back to the end user,” said Krish. “The case for LLMs and AI is to go into a system and tell it exactly what you want the product experience to be like for you.”

Fine-tuning models to create custom workflows: Introducing the “Krish Bot”

Fireflies is working on an internal project to showcase how fine-tuning (the process of training a model on a narrow dataset to perform highly specific tasks) can build more intelligent agents.

As an example, Krish had Fireflies review every sales demo he’s ever done, fine-tune a model with this same sales demo data, and then build a bot that trains new salespeople. Now, Krish offloads part of his work training salespeople to the “Krish bot” which answers questions on Slack.

This is where smaller LLMs may perform better than a general-purpose LLM. In addition, these models can be put in private storage and containerized, so they also meet the security and privacy requirements of enterprises. 

The “Krish Bot” paints a picture of Fireflies' long-term vision for the future of AI-assisted work: training AI on routine aspects of employees’ daily duties to create specialized bots that automate work. “We treat these training efforts in the same way we consider training actual employees,” says Krish. “Instead of having ten consultants, you can have ten AI bots that can work for you 24/7.”

The Future of Meeting Summarization

In the future, it’s inevitable that summarization tools built as a simple wrapper on top of LLMs will become commoditized. A great product will need to perform end-to-end workflows that integrate with other tools, solve problems that depend on more than text summarization, and ideally create a community where people share their solutions.

I’m excited to see how Fireflies and other companies like it will progress towards the goal of becoming a knowledge base for enterprises–and significantly reducing the amount of time we all spend in meetings in the process!