{"id":68672,"date":"2023-09-24T06:59:14","date_gmt":"2023-09-24T14:59:14","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/devops\/?p=68672"},"modified":"2024-02-22T23:23:19","modified_gmt":"2024-02-23T07:23:19","slug":"revolutionizing-requirement-gathering-azure-devops-meets-azure-openai-using-semantic-kernel","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/devops\/revolutionizing-requirement-gathering-azure-devops-meets-azure-openai-using-semantic-kernel\/","title":{"rendered":"Revolutionizing Requirement Gathering: Azure DevOps Meets Azure OpenAI using Semantic kernel"},"content":{"rendered":"<p>This blog is a deep dive into the future of requirement gathering. This blog explores how Azure DevOps and Azure OpenAI are joining forces to transform the way we manage project requirements. From automated requirement generation to intelligent analysis, learn how these powerful tools are reshaping the landscape of project management. Stay tuned for an enlightening journey into the world of AI-powered requirement gathering!<\/p>\n<h2>Setting up environment<\/h2>\n<p><strong>Pre-requisite<\/strong><\/p>\n<ul>\n<li>Visual studio code<\/li>\n<\/ul>\n<p>\u00a0 \u00a0 Please install below extension<\/p>\n<p>\u00a0 \u00a0 &#8211; Jupyter (Publisher- Microsoft)<\/p>\n<p>\u00a0 \u00a0 &#8211; Python (Publisher- Microsoft)<\/p>\n<p>\u00a0 \u00a0 &#8211; Pylance (Publisher- Microsoft)<\/p>\n<p>\u00a0 \u00a0 &#8211; Semantic Kernel Tools (Publisher- Microsoft)<\/p>\n<ul>\n<li>Python<\/li>\n<\/ul>\n<p>\u00a0 Please install below python packages<\/p>\n<p>\u00a0 \u00a0 &#8211; PIP<\/p>\n<p>\u00a0 \u00a0 &#8211; Semantic-kernel<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/vivekgarudi\/Semantic-Kernal-Azure\/tree\/main\/PlugIn-for-creating-Azure-DevOps-features-from-Requirment-text\">Download<\/a>\u00a0the content from GitHub\u00a0repo<\/li>\n<\/ul>\n<h2>Define the Semantic Function to generate\u00a0feature description-<\/h2>\n<p>Now that you have below mentioned folder\u00a0structure.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2024\/02\/image1.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2024\/02\/image1.png\" alt=\"Image image1\" width=\"385\" height=\"239\" class=\"alignnone size-full wp-image-68715\" srcset=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2024\/02\/image1.png 385w, https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2024\/02\/image1-300x186.png 300w\" sizes=\"(max-width: 385px) 100vw, 385px\" \/><\/a><\/p>\n<h3>Create semantic function for generating Feature description.<\/h3>\n<p>The first step is to define a semantic function that can interpret the input string and map it to a specific action. In our case, the action is to generate feature description from title. The function could look something like this:<\/p>\n<ol>\n<li>\n<p>Create folder structure \u00a0 \u00a0<\/p>\n<ul>\n<li>Create \/plugins folder \u00a0 \u00a0 <\/li>\n<li>\n<p>Create folder for semantic plugins inside Plugins folder, in this case its &#8220;AzureDevops&#8221;. (For more details on <a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/ai-orchestration\/plugins\/?tabs=Csharp#what-is-a-plugin\">plugins<\/a>) \u00a0 \u00a0<\/p>\n<\/li>\n<li>\n<p>Create Folder for semantic function inside the skills folder ie &#8216;\/plugin\/AzureDevops&#8217;, in this case &#8220;FeatureDescription&#8221; (For more details on <a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/ai-orchestration\/plugins\/?tabs=Csharp#adding-functions-to-plugins\">functions<\/a>)<\/p>\n<\/li>\n<\/ul>\n<\/li>\n<li>\n<p>Define semantic function \u00a0 \u00a0<\/p>\n<ul>\n<li>Once we have folder structure in place lets define the function by having<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>\u00a0 \u00a0 \u00a0 \u00a0 &#8216;config.json&#8217; with below JSON content for more details on content refer here.<\/p>\n<pre><code>{\n  \"schema\": 1,\n  \"description\": \"get standard feature title and description\",\n  \"type\": \"completion\",\n  \"completion\": {\n    \"max_tokens\": 500,\n    \"temperature\": 0.0,\n    \"top_p\": 0.0,\n    \"presence_penalty\": 0.0,\n    \"frequency_penalty\": 0.0\n   },\n     \"input\": {\n          \"parameters\": [\n               {\n               \"name\": \"input\",\n               \"description\": \"The feature name.\",\n               \"defaultValue\": \"\"\n               }\n          ]\n     }\n}\n<\/code><\/pre>\n<p>In above file, we are defining semantic function which accept &#8216;input&#8217; parameter to perform &#8220;get standard feature title and description&#8221; as mentioned in Description section.<\/p>\n<p>\u00a0 \u00a0 &#8211; Now, let&#8217;s\u00a0put the single shot prompt for our semantic function in &#8216;skprompt.txt&#8217;. where &#8216;{{input}}&#8217; where our input ask would be replaced.<\/p>\n<pre><code>    Create feature title and description for {{$input}}  in below format\n    Feature Title:\"[Prodive a short title for the feature]\"\n    Description: \"[Provide a more detailed description of the feature's purpose, the problem it addresses, and its significance to the product or project.] \n\n    User Needs- \n    [Outline the specific user needs or pain points that this feature aims to address.] \n\n    Functional Requirements:-\n    - [Requirement 1] \n    - [Requirement 2] \n    - [Requirement 3] \n    - ... \n\n    Non-Functional Requirements:-\n    - [Requirement 1] \n    - [Requirement 2] \n    - [Requirement 3] \n    - ... \n\nFeature Scope: [Indicates the minimum capabilities that feature should address. Agreed upon between Engineering Leads and Product Mangers] \"\n<\/code><\/pre>\n<h2>Execute above semantic function in action.<\/h2>\n<ul>\n<li>\n<p>Rename &#8220;.env.example&#8217; as &#8216;.env&#8217; and update the parameters with actual values<\/p>\n<\/li>\n<li>\n<p>Open notebook &#8220;Create-Azure-Devops-feature-from-requirement-text&#8221; in visual\u00a0studio code and follow the steps mentioned to test<\/p>\n<\/li>\n<\/ul>\n<p><strong>Step 1<\/strong> Install all python libraries<\/p>\n<pre><code>!python -m pip install semantic-kernel==0.3.10.dev0 !python -m pip install azure-devops\n<\/code><\/pre>\n<p><strong>Step 2<\/strong> Import Packages required to prepare a semantic kernel instance first.<\/p>\n<pre><code>import os\nfrom dotenv import dotenv_values\nimport semantic_kernel as sk\nfrom semantic_kernel import ContextVariables, Kernel # Context to store variables and Kernel to interact with the kernel\nfrom semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, OpenAIChatCompletion # AI services\nfrom semantic_kernel.planning.sequential_planner import SequentialPlanner # Planner\n\nkernel = sk.Kernel() # Create a kernel instance\nkernel1 = sk.Kernel() #create another kernel instance for not having semanitc function in the same kernel \n\nuseAzureOpenAI = True\n\n# Configure AI service used by the kernel\nif useAzureOpenAI:\n    deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()\n    kernel.add_chat_service(\"chat_completion\", AzureChatCompletion(deployment, endpoint, api_key))\n    kernel1.add_chat_service(\"chat_completion\", AzureChatCompletion(deployment, endpoint, api_key))\nelse:\n    api_key, org_id = sk.openai_settings_from_dot_env()\n    kernel.add_chat_service(\"chat-gpt\", OpenAIChatCompletion(\"gpt-3.5-turbo\", api_key, org_id))\n<\/code><\/pre>\n<p><strong>Step 3<\/strong> Importing skills and function from folder<\/p>\n<pre><code># note: using skills from the samples folder\nplugins_directory = \".\/plugins\"\n\n# Import the semantic functions\nDevFunctions=kernel1.import_semantic_skill_from_directory(plugins_directory, \"AzureDevOps\")\nFDesFunction = DevFunctions[\"FeatureDescription\"]  \n<\/code><\/pre>\n<p><strong>Step 4<\/strong> calling the semantic function with feature title to generate feature description based on predefined template<\/p>\n<pre><code>resultFD = FDesFunction(\"Azure Resource Group Configuration Export and Infrastructure as Code (IAC) Generation\")\nprint(resultFD)\n<\/code><\/pre>\n<h2>Create native function to create features in Azure DevOps<\/h2>\n<p>\u00a0&#8211; Create file &#8220;native_function.py&#8221; under &#8220;AzureDevOps&#8221; or <a href=\"https:\/\/github.com\/vivekgarudi\/Semantic-Kernal-Azure\/blob\/main\/PlugIn-for-creating-Azure-DevOps-features-from-Requirment-text\/plugins\/AzureDevops\/native_function.py\">download<\/a>\u00a0the file from repo.<\/p>\n<p>\u00a0&#8211; Copy the code base and update Azure Devops parameter. you can access this as context parameter but for simplicity\u00a0of this exercise, we kept it as hardcoded. Please find below code flow<\/p>\n<p>\u00a0 \u00a0 \u00a0 \u00a0 &#8211; Importing python packages<\/p>\n<p>\u00a0 \u00a0 \u00a0 \u00a0 &#8211; Defining class &#8216;feature&#8217; and native function as &#8220;create&#8221; under &#8220;@sk_function&#8221;.<\/p>\n<p>\u00a0 \u00a0 \u00a0 \u00a0 &#8211; Call semantic function to generate feature description.<\/p>\n<p>\u00a0 \u00a0 \u00a0 \u00a0 &#8211; Use this description to create Azure DevOps feature.<\/p>\n<pre><code>from semantic_kernel.skill_definition import (\n    sk_function,\n    sk_function_context_parameter,\n)\n\nfrom semantic_kernel.orchestration.sk_context import SKContext\nfrom azure.devops.v7_1.py_pi_api import JsonPatchOperation\n\nfrom azure.devops.connection import Connection\nfrom msrest.authentication import BasicAuthentication\nimport base64\nfrom semantic_kernel import ContextVariables, Kernel\nimport re\nclass feature:\n    def __init__(self, kernel: Kernel):\n        self._kernel = kernel\n    _function(\n        description=\"create a Azure DevOps feature with description\",\n        name=\"create\",\n    )\n    _function_context_parameter(\n        name=\"title\",\n        description=\"the tile of the feature\",\n    )\n    _function_context_parameter(\n        name=\"description\",\n        description=\"Description of the feature\",\n    )\n    async def create_feature(self, context: SKContext) -&gt; str:\n        feature_title = context[\"title\"]\n        get_feature = self._kernel.skills.get_function(\"AzureDevOps\", \"FeatureDescription\")\n        fdetails = get_feature(feature_title)\n        # Define a regular expression pattern to match the feature title\n        pattern = r\"Feature Title:\\s+(.+)\"\n        # Search for the pattern in the input string\n        match = re.search(pattern, str(fdetails))\n        # Check if a match was found\n        if match:\n            feature_title = match.group(1)\n        # Define a regular expression pattern to match the feature description\n        # Split the string into lines\n        lines = str(fdetails).split('\\n')\n        lines = [line for index, line in enumerate(lines) if index not in [0]]\n        description = '\\n'.join(lines)\n        jsonPatchList = [] \n        #description=context[\"description\"]\n        targetOrganizationName= \"XXX\"\n        targetProjectName= \"test\"\n        targetOrganizationPAT = \"XXXXXX\"\n        workItemCsvFile= \"abc\"\n        teamName = \"test Team\"\n        areaName = teamName\n        iterationName =\"Sprint 1\"\n        targetOrganizationUri='https:\/\/dev.azure.com\/'+targetOrganizationName\n        credentials = BasicAuthentication('', targetOrganizationPAT)\n        connection = Connection(base_url=targetOrganizationUri, creds=credentials)\n        userToken = \"\" + \":\" + targetOrganizationPAT\n        base64UserToken = base64.b64encode(userToken.encode()).decode()\n        headers = {'Authorization': 'Basic' + base64UserToken}\n        core_client = connection.clients.get_core_client()\n        targetProjectId = core_client.get_project(targetProjectName).id\n        workItemObjects = [\n                {\n                    'op': 'add',\n                    'path': '\/fields\/System.WorkItemType',\n                    'value': \"Feature\"\n                },\n                {\n                    'op': 'add',\n                    'path': '\/fields\/System.Title',\n                    'value': feature_title\n                },\n                {\n                    'op': 'add',\n                    'path': '\/fields\/System.State',\n                    'value': \"New\"\n                },\n                {\n                    'op': 'add',\n                    'path': '\/fields\/System.Description',\n                    'value': description\n                },\n                {\n                    'op': 'add',\n                    'path': '\/fields\/Microsoft.VSTS.Common.AcceptanceCriteria',\n                    'value': \"acceptance criteria\"\n                },      \n                {\n                    'op': 'add',\n                    'path': '\/fields\/System.IterationPath',\n                    'value': targetProjectName+\"\\\\\"+iterationName\n                }\n            ]\n        jsonPatchList = JsonPatchOperation(workItemObjects)\n        work_client = connection.clients.get_work_item_tracking_client()\n        try:\n            WorkItemCreation = work_client.create_work_item(jsonPatchList.from_, targetProjectName, \"Feature\")\n        except Exception as e:\n            return feature_title+\"Feature created unsuccessfully\"\n        return feature_title+\" Feature created successfully\"\n<\/code><\/pre>\n<p>\u00a0<\/p>\n<h2>Let&#8217;s execute native function<\/h2>\n<p>Let&#8217;s\u00a0go back to\u00a0notebook.<\/p>\n<p><strong>Step 5<\/strong> Importing native function<\/p>\n<pre><code>from plugins.AzureDevops.native_function import feature\nmath_plugin = kernel.import_skill(feature(kernel1), skill_name=\"AzureDevOps\")\nvariables = ContextVariables()\n<\/code><\/pre>\n<p>\u00a0 <strong>Step 6<\/strong> Executing native function by putting\u00a0natural language queries in title field<\/p>\n<pre><code>variables[\"title\"] = \"creating a nice pipelines\"\nvariables[\"description\"] = \"test\"\nresult = await kernel.run_async(\n                math_plugin[\"create\"], input_vars=variables\n            )\nprint(result)\n<\/code><\/pre>\n<h2>Use of Sequential planner to dynamical create N number of features.<\/h2>\n<p><strong>Step 7<\/strong> Initiate sequential planner with semantic kernel<\/p>\n<pre><code>from plugins.AzureDevops.native_function import feature\nplanner = SequentialPlanner(kernel)\n# Import the native functions\nAzDevplugin = kernel.import_skill(feature(kernel1), skill_name=\"AzureDevOps\")\nask = \"create two Azure DevOps features for one with title creating user and one with creating work items with standard feature title and description\"\nplan = await planner.create_plan_async(goal=ask)\nfor step in plan._steps:\n        print(step.description, \":\", step._state.__dict__)\n<\/code><\/pre>\n<p>This would generate a plan to meet the goal which is above case is &#8220;create two Azure DevOps features for one with title creating user and one with creating work items with standard feature title and description&#8221; using available function in kernel.<\/p>\n<p><strong>Step 8<\/strong> once the plan is created, we can use this plan and execute it to create multiple features.<\/p>\n<pre><code>&lt;br \/&gt;print(\"Plan results:\")\nresult = await plan.invoke_async(ask)\nfor step in plan._steps:\n        print(step.description, \":\", step._state.__dict__)\n<\/code><\/pre>\n<p>This will create two features one for user and one for work item. Using this block, you can create a semantic function-based solution that can interpret natural language requirement document or transcript of reequipment call and use it to create features in azure DevOps. You can increase the accuracy of this solution by brining multi-shot prompt and historical data using collections.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This blog is a deep dive into the future of requirement gathering. This blog explores how Azure DevOps and Azure OpenAI are joining forces to transform the way we capture project requirements. From automated requirement generation to intelligent analysis, learn how these powerful tools are reshaping the landscape of project management. Stay tuned for an enlightening journey into the world of AI-powered requirement gathering!<\/p>\n","protected":false},"author":148446,"featured_media":68715,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-68672","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"acf":[],"blog_post_summary":"<p>This blog is a deep dive into the future of requirement gathering. This blog explores how Azure DevOps and Azure OpenAI are joining forces to transform the way we capture project requirements. From automated requirement generation to intelligent analysis, learn how these powerful tools are reshaping the landscape of project management. Stay tuned for an enlightening journey into the world of AI-powered requirement gathering!<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/68672","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/users\/148446"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/comments?post=68672"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/68672\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media\/68715"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media?parent=68672"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/categories?post=68672"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/tags?post=68672"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}