To use Chroma with Semantic Kernel, visit the sample notebook on the Semantic Kernel GitHub repo. And if you like the repo, please give it a star!

In this interview with Jeff Huber, CEO and co-founder of Chroma, a leading AI-native vector database, Jeff discusses how Chroma bridges the gap between AI models and production by leveraging embeddings and offering powerful document retrieval capabilities. Jeff highlights Chroma’s role in preventing hallucinations in AI models, enhancing accuracy, and ensuring developer productivity. He also shares insights into the challenges faced by LLM AIs and how Chroma aims to address them, emphasizing the importance of programmable memory for future AI development.

“Hallucinations happen when the AI model does not have enough factual information to ground it’s response.” —Jeff Huber, CEO of Chroma

SK: Can you tell us about your background and how you got involved in the field of AI and database development?

Jeff Huber: My cofounder Anton and I started Chroma to help developers bridge the gap between demo and production while building AI-based applications. We had built many such systems before, and had personally felt how hard this is. Our thesis with Chroma is that embeddings specifically have a ton of value that is underexplored – this has led to us being obsessed about how they are generated, stored, searched and analyzed.

SK: How does Chroma’s integration with AI SDKs like Semantic Kernel enhance the capabilities of AI models?

JH: Chroma’s integration with AI SDKs like Semantic Kernel allows developers to leverage the real power of embeddings in their AI models. By using Semantic Kernel and AI models from OpenAI, Azure OpenAI Service, and Hugging Face, developers can easily generate embeddings for their data and utilize the similarity and dissimilarity features of embeddings to enhance the capabilities of their AI models. This integration provides a bridge between AI models and the specific data they need to process.

“Chroma is designed first and foremost for developer productivity and happiness.”

SK: Can you provide an example of how Chroma’s embedding-based document retrieval has been beneficial for developers?

JH: Certainly, hundreds of thousands of developers have used Chroma to build everything from a small experiment with LangChain at a hackathon to complex research projects like Voyager. We’ve been really impressed by the diversity and technical achievements of developers building with Chroma.

SK: How does Chroma prevent hallucinations in AI models and ensure the accuracy of their outputs?

JH: Hallucinations come when the model does not have enough factual information to ground it’s response. Retrieval-augmented-generation, which is the technical term for pulling in relevant documents from Chroma, gives the language model factual information to use when answering your questions, massively decreasing the chance that it makes something up.

SK: What are the key takeaways for developers who choose to use Chroma in their AI projects?

JH: Chroma is designed first and foremost for developer productivity and happiness. We have a thriving community of developers who have put Chroma through its paces and we’ve worked really hard to keep the API incredibly simple and easy to learn and use.

Embeddings are magical but when “magic” doesn’t work—it can be incredibly difficult to know why!

SK: How does Chroma’s in-memory mode contribute to faster query processing?

JH: Chroma offers 2 versions, an in-memory and single-node version. Developers love the in-memory version as it is lightning fast to get up and running in a Python script or even Jupyter notebook. Chroma will soon release a distributed version which will scale forever in the cloud.

SK: What challenges do you foresee in the future of LLM AIs, and how does Chroma address those challenges?

JH: Embeddings are magical but when “magic” doesn’t work—it can be incredibly difficult to know why! Chroma is working on a suite of tools to help developers build reliable, useful and powerful production AI-applications using embeddings.

SK: What sets Chroma apart from other databases in the market, and how do you envision its role in the future of AI development?

JH: Fundamentally we need the ability for language models to learn and remember things outside the weights of the model itself. This ability we call programmable memory and is essential to making it easy to develop reliable and powerful systems that humans can understand and interpret. Chroma is the first and only AI-native vector database and we will continue to lead the charge here.

About Jeff Huber

Jeff Huber is CEO and co-founder of Chroma. The Chroma open-source project is coordinated by a small team of full-time employees who work at a company also called Chroma. They proudly work in the sunny neighborhood of Potrero Hill in San Francisco.