Announcing New Vector Stores: Faiss, SQL Server, and Pinecone
We are thrilled to announce the availability of three new Vector Stores and Vector Store Record Collections: Faiss, SQL Server, and Pinecone. These new connectors will enable you to store and retrieve vector data efficiently, making it easier to work with your own data and data models.
Faiss
Faiss is a library for efficient similarity search and clustering of dense vectors. It builds on the built-in InMemoryCollection, by creating Faiss indexes on the side, which are then used for the actual vector search.
Setup
Install Semantic Kernel with the Faiss extra:
pip install semantic-kernel[faiss]And then create your collection:
from semantic_kernel.connectors.memory.faiss import FaissCollection
collection = FaissCollection(
collection_name="my_collection",
data_model_type=MyDataModel,
)With this connector you can also use your own indexes for one or all vectors fields, this allows you to create highly optimized indexes for your specific use case, potentially even leveraging GPU’s see the documentation for more info on how to set that up.
SQL Server
SQL Server is a relational database management system developed by Microsoft. Azure SQL Server now supports vector data types and a set of vector distance/scoring functions, these can now be used with Semantic Kernel. Currently this only works with Flat indexes.
The Python SQL Server Collection uses pyodbc to connect, that can be installed by the command below, but you will need to install the ODBC driver for SQL Server separately, this differs by platform, see the Azure SQL Documentation for details.
Setup
pip install semantic-kernel[sql]
Then you can create the collection like this:
from semantic_kernel.connectors.memory.sql_server import SqlServerCollection
collection = SqlServerCollection(
collection_name="my_collection",
data_model_type=MyDataModel,
)Environment Variables
- SQL_SERVER_CONNECTION_STRING: The connection string for your SQL Server database.
For more info, see our documentation.
Pinecone
Pinecone is a vector database that provides fast and scalable vector search capabilities. It is designed to handle large-scale vector data and offers seamless integration with machine learning models.
Setup
First install the pinecone client package, by using the extra:
pip install semantic-kernel[pinecone]
and then you can create the collection:
from semantic_kernel.connectors.memory.pinecone import PineconeCollection
collection = PineconeCollection(
collection_name="my_collection",
data_model_type=MyDataModel,
)Environment Variables
- PINECONE_API_KEY: Your Pinecone API key.
For more info, see the documentation.
Get Started Today
We are excited to see what you will build with these new connectors. Whether you are working on a simple chatbot, or building multi-agent group chats running inside of a larger process, these connectors will provide you with the tools you need to succeed. Get started today and take your projects to the next level!
Happy coding!
0 comments
Be the first to start the discussion.