{"id":5305,"date":"2022-12-08T10:00:29","date_gmt":"2022-12-08T18:00:29","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=5305"},"modified":"2022-12-08T09:51:38","modified_gmt":"2022-12-08T17:51:38","slug":"altgraph-graph-workloads-with-azure-cosmos-db-for-nosql","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/altgraph-graph-workloads-with-azure-cosmos-db-for-nosql\/","title":{"rendered":"AltGraph \u2013 Graph workloads with Azure Cosmos DB for NoSQL"},"content":{"rendered":"

AltGraph<\/a> is the idea of implementing \u201cgraph workloads\u201d not with a graph database, but with the more general-purpose Azure Cosmos DB for NoSQL<\/a> (formerly known as the \u201cSQL API\u201d).\u00a0 Azure Cosmos DB for NoSQL is highly performant and offers the best integration with the rest of Azure.\u00a0 It also offers the lowest learning curve since most of the industry already understands SQL (Structured Query Language), the query language used by this database.<\/p>\n

AltGraph was the topic of episode #59 of Azure Cosmos DB Live TV<\/a>, and has since been expanded to include the v2 IMDb graph.<\/p>\n

 <\/p>\n

Perception<\/strong><\/h2>\n

Many customers I speak with initially visualize their use-cases as a graph, such as the following diagram.\u00a0 They then reason that since the application is visualized<\/em> as a graph means that the underlying datastore needs to be implemented<\/em> as a graph database.\u00a0 It is my assertion that this is not true \u2013 the underlying data can be stored and traversed in alternative and possibly much more efficient datastores, such as Azure Cosmos DB for NoSQL.\u00a0 JavaScript libraries can be used to create the graph visualizations.<\/p>\n

\"sample<\/a><\/p>\n

 <\/p>\n

Application Architecture<\/strong><\/h2>\n

A graph application is typically built with much more than just a database<\/em> at the center of the solution.\u00a0 A typical solution integrates several services, and this is where Azure and Azure Cosmos DB shines \u2013 the ease of integrations.\u00a0 In the diagram below, the native integrations that Azure Cosmos DB supports are shown in red<\/strong><\/span>.\u00a0 These are implemented in a reliable and scalable way by the Azure PaaS services themselves, so you, the customer, don\u2019t have to write \u201cglue code\u201d or explicit ETL to integrate them.<\/p>\n

Four of these integrations I\u2019d like to call out here are Azure Functions<\/a>, Azure Cognitive Search<\/a>, Azure Synapse Link for Azure Cosmos DB<\/a>, and Azure Synapse Analytics<\/a>.\u00a0 With Azure Functions you can observe the Azure Cosmos DB change-stream, and thus have event-driven logic in your application for real-time processing.\u00a0 Azure Cognitive Search supports automatic indexing of Azure Cosmos DB containers.\u00a0 For some applications their queries\/searches can be much more fluent and cost-effective by using this search engine.\u00a0 Azure Synapse Link implements the HTAP pattern (Hybrid Transactional Analytic Processing) to automatically copy your documents to a low-cost datastore for processing in a \u201cBig Data\u201d batch environment such as Apache Spark in Azure Synapse Analytics.<\/p>\n

Thus, the total Azure and Azure Cosmos DB-based graph application can leverage the \u201cbest tool for the job\u201d, as well as reduce costs, with these efficiencies.<\/p>\n

Conversely, it is an antipattern to use one database for both real-time and batch\/analytic purposes, as the DBMS often won\u2019t scale under the workload, and typically offers sub-optional support for one or more of these access patterns.<\/p>\n

\"altgraph<\/a><\/p>\n

 <\/p>\n

So, what again is AltGraph?<\/strong><\/h2>\n

AltGraph isn\u2019t a product, or another Azure PaaS service.\u00a0 Instead, it\u2019s a set of reference applications that demonstrate how to build \u201cgraph applications\u201d with Azure Cosmos DB for NoSQL<\/em> and leverage its\u2019 features to achieve performant, integrated, and low-cost solutions.\u00a0 It currently demonstrates two different graph use-cases, with two different implementations.<\/p>\n

The first is a Bill-of-Materials use-case that uses data from the NPM (Node Package Manager) ecosystem.\u00a0 The implementation uses the concept of RDF graph database \u201ctriples\u201d, but implemented with Azure Cosmos DB.\u00a0 Roughly 20-million triples can be stored efficiently in a single Azure Cosmos DB logical partition, which makes their queries both fast and low-cost.\u00a0 Optionally, these can be cached using Azure Cache for Redis<\/a> or the Azure Cosmos DB integrated cache.\u00a0 Additionally, in-memory graph traversal logic can be utilized that is 1000x faster than DBMS disk IO.<\/p>\n

A \u201cTriple\u201d is a data structure that links a subject, an object, and a predicate.\u00a0 For example, in the relationship \u201cChris works at Microsoft\u201d the subject is Chris, the object is Microsoft, and the predicate is \u201cworks at\u201d.\u00a0 Triples can be visualized like this:<\/p>\n

\"rdf<\/a><\/p>\n

 <\/p>\n

In the AltGraph reference implementation, the Triple documents look like the following.\u00a0 They have additional attributes to enable fast vertex document point-reads in Azure Cosmos DB.\u00a0 The optional subjectTags and objectTags allow the application to efficiently \u201cpeek\u201d into the adjacent vertex documents, for their primary attributes, without actually reading those documents.<\/p>\n

\"sample<\/a><\/p>\n

 <\/p>\n

The second reference application is a rich Social Network graph that uses IMDb (Internet Movie Database) data.\u00a0 This large dataset is very relatable and can be used to explore the \u201cSix Degrees of Kevin Bacon\u201d graph challenge.\u00a0 The implementation of this reference app uses the JGraphT<\/a> Java library which offers very fast in-memory processing as well as a comprehensive set of graph traversal algorithms such as Dijkstra Shortest Path, Centrality, and Page Rank.\u00a0 Over 1-million movie and actor vertices are stored in the memory of the JVM, as well as 3.9 million edges <\/em>in just 1.5GB of memory.\u00a0 Larger in-memory graphs are possible.\u00a0 Using a combination of in-memory graph traversals, and point-reads to Azure Cosmos DB, can result in a very performant and cost-effective solution.<\/p>\n

This reference application screenshot shows \u201cTwo degrees of Kevin Bacon\u201d and uses D3.js for interactive JavaScript visualizations.\u00a0 The graph traversal took only 3 milliseconds to execute and consumed zero Azure Cosmos DB request units<\/em> (RUs).<\/p>\n

\"network<\/a><\/p>\n

 <\/p>\n

Are other implementations of \u201cAltGraph\u201d possible?\u00a0 Yes!\u00a0 I\u2019m sure that each customer will create their own implementations, perhaps inspired by the examples in this GitHub repository.\u00a0 The Microsoft team is here to support you in those efforts and ensure that you are successful.<\/p>\n

 <\/p>\n

Get started:<\/strong><\/h2>\n