{"id":21431,"date":"2023-10-10T09:00:35","date_gmt":"2023-10-10T16:00:35","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/azuregov\/?p=21431"},"modified":"2023-10-10T09:00:35","modified_gmt":"2023-10-10T16:00:35","slug":"decoding-ai-part-3","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/azuregov\/decoding-ai-part-3\/","title":{"rendered":"Decoding AI: Part 3, Making data speak human"},"content":{"rendered":"

Welcome back to Part 3 of our Decoding AI: A Government Perspective series<\/strong>. In our <\/em>last installment<\/em><\/a>, we explored the nuanced landscape of anomaly detection and how it intersects with generative AI in shaping the future of public governance. As promised, today’s dialogue will pivot toward semantic search and its role in converting complex data into easily interpretable, actionable insights. \u2013 Siddhartha Chaturvedi, Miri Rodriguez <\/em><\/p>\n

Semantic search: A cognitive leap beyond keywords<\/strong><\/p>\n

For a long time, traditional search engines have been revered for their impressive ability to sift through and parse millions of documents in the blink of an eye. This remarkable feat of processing power and speed has revolutionized the way we access information. However, despite their advanced capabilities, these engines often stumbled when it came to understanding the nuances of human language. Specifically, they struggled with comprehending the context or sentiment embedded within a user\u2019s query. This limitation highlighted the complexity and subtlety of human communication, which goes beyond mere words and involves a deeper understanding of context, tone, and emotion.<\/p>\n

This is where semantic search swings in, adding a layer of comprehension to the raw computing power of generative AI. Utilizing Natural Language Processing (NLP), it can decipher not just the query’s literal text but the intention behind it.<\/p>\n

\"Image<\/a><\/p>\n

This image shows how a semantic search engine interprets and responds to a natural language query. The top part of the image illustrates the query analysis, where the search engine breaks down the query into semantic components and relationships. The bottom part of the image illustrates the data retrieval, where the search engine accesses and displays relevant data sources that match the query. Semantic search engines aim to provide more natural and interactive search experiences for users. \u2013 Generated by Bing Chat (GPT4)<\/em><\/span><\/p>\n

Imagine the transformative potential this capability can have in the public sector. For instance, immigration officers could sift through vast legal documents, not merely for generic terms like \u201cvisa statuses,\u201d but pinpointing nuanced conditions such as \u201ctemporary work visas granted under economic hardship.\u201d This level of precision could significantly enhance efficiency and decision-making in immigration processes. It’s a shift from data retrieval to knowledge discovery, streamlining governmental operations like never before.<\/p>\n

Potential use cases<\/strong><\/p>\n

Semantic search lends itself to multiple applications within government:<\/p>\n

Climate resilience and adaptation<\/strong><\/p>\n

Agencies focused on environmental protection and climate adaptation can employ semantic search sift through meteorological data, scientific research, and crowd-sourced data, then translated to the right format leveraging LLMs. With engines like Microsoft Planetary Computer<\/a>, we can even bring in Geospatial data, but more on that in the next blog. This can yield an intricate understanding of climate-related risks and potential adaptive measures, simplified for understanding and access, aiding in policy formulation and community engagement, supercharging investments to build a climate ready nation<\/a>.<\/p>\n

Citizen services<\/strong><\/p>\n

Imagine a citizen wanting to understand tax codes or healthcare benefits. A semantically intelligent portal could dissect the context of the user’s query, tapping into underlying databases and generating human-readable summaries or action plans, thereby accelerating service delivery.<\/p>\n

Integrating generative AI: The ‘translate’ function for data<\/strong><\/p>\n

Semantic search is not an island; it works best when integrated with generative AI. Consider a scenario where an anomaly is detected in public healthcare records. The semantic search first narrows down the possible causes from medical literature and guidelines. Then generative AI kicks in, translating the jargon-filled text into layman’s terms complete with suggestions for policymakers.<\/p>\n

\"Image<\/a><\/p>\n

This image shows how a generative engine can produce new and realistic content based on a user\u2019s input. The user can enter any natural language query or prompt, and the generative engine will create content that matches the query. The image illustrates the data flow between the user and the generative engine, Generative engines are a type of artificial intelligence that can learn from existing data and generate novel data that reflects the characteristics of the underpinning training data. \u2013 Generated by Bing Chat (GPT4)<\/em><\/span><\/p>\n

The confluence of technologies: Framework for synergy<\/strong><\/p>\n

So, what’s the ideal architecture for this? While the architecture of integrative AI is still evolving, we suggest a way of thinking that has three parts. The first part is about people. We want to make AI that helps people and understand them. The second part is about nature. We want to make AI that fits well with the things that people already do and use. The third part is about process. We want to make AI that can change and grow with the needs of people. Semantic search is a type of AI that helps with the first part. It can understand what people mean when they search for something. Generative AI is a type of AI that helps with the second and third parts. It can create new things that match what people want. It can also work well with different situations and scales.<\/p>\n

Let’s get started:<\/strong><\/p>\n

    \n
  1. Prepare your semantic search<\/strong>\n