Glean’s AI assistant: Answers at your fingertips, complexity behind the scenes.

It’s been a few months since my last issue of Practical Intelligence and today, in addition to an exciting guest for the newsletter, I have some news to share: I’m heading back into an operating role. 

At IVP, I’ve spent the past months digging into the fast-paced world of AI and working alongside an amazing team. Now, I’ll be building an AI-powered work assistant at Glean as President, Product and Technology. Among the many AI companies I’ve gotten to know during my time at IVP, Glean is a special company. If you’re interested in learning more about why I’m joining Glean, you can read about it here. 

It’s fitting, then, that my guest for this week's newsletter is none other than Glean’s CEO and founder, Arvind Jain. Arvind is an outstanding technologist whom I’ve known for years, ever since we met at Google in 2011. I’ve spent the last few weeks learning a lot about Glean, so it was a pleasure to have another conversation with Arvind that I could share with my readers.

While this is my last issue of this newsletter from IVP, my journey with Practical Intelligence will continue in my new role at Glean. I will dig deeper into the role that AI plays in helping people work, speak to industry leaders, and engage in conversations about this emerging and exciting field. 

Thank you to all my readers for your time and feedback so far. Please enjoy my conversation with Arvind:

Tamar: For people who aren’t familiar with Glean, could you explain what the product does?

Arvind: Think of Glean as ChatGPT or Google for your company. ChatGPT answers general questions, while Glean answers questions specific to your work. Glean connects with all of the data and knowledge that sits across many different systems inside your company. Employees can ask questions and Glean will use your company knowledge in a safe and secure way to help answer their questions.

Tamar: As you started building Glean, what were some of the problems you ran into?

Arvind: We underestimated how difficult it is to bring in data at each organization and integrate individual applications. Initially, we thought this would be easy given the fact that businesses have standardized the way they store information through SaaS APIs. But we soon ran into a number of issues. For one, it’s difficult to understand the governance behind data. Every SaaS application has built its own version of mission control permissions, which we replicate inside Glean’s search systems. This turned out to be a lot of work.

Another key challenge for our team is that we had to unlearn a lot of what we knew about search from working at Google. In enterprise systems, there are fewer queries, so there’s less data to learn about how people are actually using search. We had to think very carefully about how people were engaging with information inside their companies and use those signals to build ranking algorithms for our search.

Also, in enterprise systems you’re working with all different kinds of data including tickets, documents, video recordings and emails. You have to figure out how to rank across many disparate sources of content.

Tamar: Glean’s core technology is a retrieval engine you built. When you built it, Retrieval-Augmented Generation or what’s known as “RAG,” wasn’t something that people were talking about. Today, everyone is talking about it. Can you tell me about the retrieval engine you built and how it’s different from what other companies are using?

Arvind: When people use the term RAG or search, they’re all talking about the same thing:  a system where you can ask questions and get relevant information. 

Right now, the attention in the market is on something called vector search or semantic search. These are retrieval systems that use language model technology and embeddings to match original questions with documents in the embedding space. It’s not a keyword-based system. 

The issue with these systems is that while embedding search is powerful, it’s also unpredictable. 

This is why we use a hybrid search system at Glean: We use traditional Information Retrieval (IR) techniques where we look at words in a query and match based on those words and their synonyms. But we also use AI to embed the queries and embed the documents to match semantically. Both of these factors come into consideration for Glean’s rankings, and the combination of the two yields better results.

For instance, if an employee is searching for a product roadmap within their company there are probably 10,000 documents that might come up. But you want the most relevant results for this employee. Not only should it be an authoritative source that’s used by the entire company, but it should be relevant to the employee’s team and include only the information they’re allowed to access within the company. This is where Glean differs from a traditional RAG, because we are a permissions-aware RAG engine using traditional keyword-based techniques, as well as modern embedding techniques within an enterprise.

Tamar: Security for enterprise search is much harder than people think because there are so many corner cases. For instance, when there is a document that can only be accessed by people who have the link to it, how do you approach security to understand who has authority to read what within an organization?

Arvind: It’s fundamental to the product that we get this right—it’s table stakes for security. Glean was developed following modern security principles, such as the zero-trust security model, strong authentication practices, and the principle of least privilege.

You mentioned an example where there’s a document that’s only accessible to people who have a link to it. In this case, we would only show you the result if you had viewed the link before. We will actually surface the exact link that was shared, whether it was in an email or in a Slack. But there are all sorts of tricky issues you run into, depending on the software you’re using and its various permissions. It took us multiple years to get this right across all of our app integrations.

Tamar: I’d love to learn more about your customers. Where are Glean’s customers getting the most value from the product and how are they using it in their own companies?

Arvind: What we hear most often is that our customers are using Glean as a company-wide AI assistant. For employees, the sky's the limit in terms of use cases. They use it to ask general questions about company policies and complex issues. Engineers use it to troubleshoot issues or write code for a new task. Support teams use it to resolve customer cases. Salespeople use it to prepare for customer meetings and find all of the latest action items related to a particular customer. So it really depends on the company and the team.

A lot of our customers have gone beyond thinking about Glean as a search product; they’re  thinking of it as a way to sort information more efficiently. We are seeing many applications of integrating Glean into daily workflows when an employee needs information from multiple sources to do their job.

Tamar: ChatGPT has obviously had a big impact on all AI companies, including Glean. Now every CIO is saying that they need an AI solution for enterprise search. What’s the biggest mistake you see CIOs making when attempting to figure out how AI is going to help their companies? 

Arvind: In the nearly thirty years of my career, I’ve never seen a new technology excite people in the way that AI has. Every department and business is eager to embrace AI because the use cases are almost unbounded. The biggest mistake I see is not building a central policy and strategy around how you’re going to bring new AI tools inside your company. As a result, data can become littered across hundreds of different systems and you’ll lose control of governance. CIOs need to proactively take charge and have a holistic strategy to keep information safe.

The big thing with AI is that every AI system wants to have access to your enterprise data. So our recommendation for CIOs is to have a centralized function as well as to assign someone with an official task of overseeing all of the AI efforts within an organization.

Tamar: The technology around AI is changing fast. What’s your vision for Glean in the future?

Arvind: AI has the potential to change how we work. In the future, every person who works will have an amazing personal assistant, powered by AI, so that they can make better decisions faster than ever before. Our vision is to democratize access to these personal assistants so that everyone has the luxury of an AI that can help them do their work. 

(Conversation was edited and condensed for clarity)

Thank you for reading! As I continue working on Practical Intelligence, I’m interested in speaking to leaders in the enterprise space about how they’re using AI, what’s working and what can be improved. Who do you think I should interview next?