{"id":3645,"date":"2023-12-14T17:02:08","date_gmt":"2023-12-15T01:02:08","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/surface-duo\/?p=3645"},"modified":"2024-01-03T16:04:24","modified_gmt":"2024-01-04T00:04:24","slug":"android-openai-chatgpt-29","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/surface-duo\/android-openai-chatgpt-29\/","title":{"rendered":"OpenAI Assistant functions on Android"},"content":{"rendered":"

\n Hello prompt engineers,\n<\/p>\n

\n This week, we are taking one last look at the new Assistants API<\/a>. Previous blog posts have covered the Retrieval<\/a> tool with uploaded files and the Code interpreter<\/a> tool. In today\u2019s post, we\u2019ll add the askWikipedia<\/code> function that we\u2019d previously built<\/a> to the fictitious Contoso employee handbook document chat. \n<\/p>\n

Configure functions in the playground<\/h2>\n

\n We\u2019ll start by configuring the assistant in the OpenAI playground. This isn\u2019t required \u2013 assistants can be created and figured completely in code \u2013 however it\u2019s convenient to be able to test interactively before doing the work to incorporate the function into the JetchatAI Kotlin sample app<\/a>.\n<\/p>\n

\n Adding a function declaration is relatively simple \u2013 click the Add<\/strong> button and then provide a JSON description of the function\u2019s capabilities. Figure 1 shows the Tools pane where functions are configured:\n<\/p>\n

\n \"ScreenshotFigure 1: Add a function definition in the playground (shows askWikipedia<\/code> already defined)<\/em>\n<\/p>\n

\n The function configuration is just an empty text box, into which you enter the JSON that describes your function \u2013 its name, what it does, and the parameters it needs. This information is then used by the model to determine when the function could be used to resolve a user query.\n<\/p>\n

\n The JSON that describes the askWikipedia<\/code> function from our earlier post<\/a> is shown in Figure 2. It has a single parameter – query<\/code> – which the model will extract from the user\u2019s input. \n<\/p>\n

  {\r\n    \"name\": \"askWikipedia\",\r\n    \"description\": \"Answer user questions by querying the Wikipedia website. Don't call this function if you can answer from your training data.\",\r\n    \"parameters\": {\r\n      \"type\": \"object\",\r\n      \"properties\": {\r\n        \"query\": {\r\n          \"type\": \"string\",\r\n          \"description\": \"The search term to query on the wikipedia website. Extract the subject from the sentence or phrase to use as the search term.\"\r\n        }\r\n      },\r\n      \"required\": [\r\n        \"query\"\r\n      ]\r\n    }\r\n  }<\/pre>\n
Figure 2: JSON description of the askWikipedia<\/code> function<\/em>\n<\/p>\n
\n  In theory, this should be enough for us to test whether the model will call the function; however, my test question \u201cwhat is measles?\u201d would always be answered by the model without calling askWikipedia<\/code>. Notice that my function description<\/code> included the instruction \u201cDon’t call this function if you can answer from your training data.\u201d \u2013 this was because in the earlier example, the function was being called too often!\n<\/p>\n
\n  Since this assistant\u2019s purpose is to discuss health plans, I updated the system prompt (called Instructions<\/code> in the Assistant playground) to the following (new text in bold):\n<\/p>\n
\n    answer questions about employee benefits and health plans from uploaded files. ALWAYS include the file reference with any annotations. use the askWikipedia function to answer questions about medical conditions<\/strong>\n  <\/p><\/blockquote>\n<\/ul>\n
\n  After this change, the test question now triggers a function call!\n<\/p>\n
\n  Figure 3 shows what a function call looks like in the playground \u2013 because function calls are delegated to the implementing application, the playground needs you to interactively supply a simulated function return value. In this case, I pasted in the first paragraph of the Wikipedia page for \u201cmeasles\u201d, and the model summarized that into its chat response.\n<\/p>\n
\n  \"ZoomedFigure 3: Testing a function call in the playground<\/em>\n<\/p>\n
\n  With the function configured and confirmation that it will get called for the test question, we can now add the function call handler to our Android assistant in Kotlin.\n<\/p>\n
Assistant function calling in Kotlin<\/h2>\n
\n  In the original Assistant API implementation<\/a>, there is a loop after the message is added to a run (shown in Figure 4) where the chat app waits for the model to respond with its answer:\n<\/p>\n
  do {\r\n     delay(1_000)\r\n     val runTest = openAI.getRun(thread!!.id, run.id)\r\n  } while (runTest.status != Status.Completed)<\/pre>\n
Figure 4: Checking message status until the run is completed and the model response is available<\/em>\n<\/p>\n
\n  Now that we\u2019ve added a function definition to the assistant, the runTest.status<\/code> can change to RequiresAction<\/code> (instead of Completed<\/code>), which indicates that the model is waiting for one (or more) function return values to be supplied. \n<\/p>\n
\n  The AskWikipediaFunction<\/code> class already contains code to search Wikipedia, so we just need to add some code to detect when a function call has occurred and provide the return value. The code shown in Figure 5 does that in these steps:\n<\/p>\n
    \n
  1. \n  Detect when the status becomes RequiresAction<\/code>\n<\/li>\n
  2. \n  Get the run\u2019s steps<\/code>\n<\/li>\n
  3. \n  Examines the latest stepDetails<\/code> for information about what action is required\n<\/li>\n
  4. \n  If a tool call is indicated, check if it\u2019s for the FunctionTool<\/code>\n<\/li>\n
  5. \n  Get the function name and parameters (which are in json key:value format)\n<\/li>\n
  6. \n  Execute the function locally with the parameters provided\n<\/li>\n
  7. \n  Create a toolOutput<\/code> with the function return value\n<\/li>\n
  8. \n  Send the output back to the assistant \n<\/li>\n<\/ol>\n
    \n  Once the assistant receives the function return value, it can construct its answer back to the user. This will set the status<\/code> to Completed<\/code> (exiting the do<\/code> loop) and the code will display the model\u2019s response to the user.\n<\/p>\n
      do\r\n  {\r\n      delay(1_000)\r\n      val runTest = openAI.getRun(thread!!.id, run.id)\r\n      if (runTest.status == Status.RequiresAction) { \/\/ run is waiting for action from the caller\r\n          val steps = openAI.runSteps(thread!!.id, run.id) \/\/ get the run steps\r\n          val stepDetails = steps[0].stepDetails \/\/ latest step\r\n          if (stepDetails is ToolCallStepDetails) { \/\/ contains a tool call\r\n              val toolCallStep = stepDetails.toolCalls!![0] \/\/ get the latest tool call\r\n              if (toolCallStep is ToolCallStep.FunctionTool) { \/\/ which is a function call\r\n                  var function = toolCallStep.function\r\n                  var functionResponse = \"Error: function was not found or did not return any information.\"\r\n                  when (function.name) {\r\n                      \"askWikipedia\" -> {\r\n                          var functionArgs = argumentsAsJson(function.arguments) ?: error(\"arguments field is missing\")\r\n                          val query = functionArgs.getValue(\"query\").jsonPrimitive.content\r\n                          \/\/ CALL THE FUNCTION!!\r\n                          functionResponse = AskWikipediaFunction.function(query)\r\n                  }\r\n                  \/\/ Package the function return value\r\n                  val to = toolOutput {\r\n                      toolCallId = ToolId(toolCallStep.id.id)\r\n                      output = functionResponse\r\n                  }\r\n                  \/\/ Send back to assistant\r\n                  openAI.submitToolOutput(thread!!.id, run.id, listOf(to))\r\n                  delay(1_000) \/\/ wait before polling again, to see if status is complete\r\n              }\r\n          }\r\n      }\r\n  } while (runTest.status != Status.Completed)<\/pre>\n
    Figure 5: Code to detect the RequiresAction<\/code> state and call the function<\/em>\n<\/p>\n
    \n  Note that there are a number of shortcuts in the code shown \u2013 when multiple functions are declared, the assistant might orchestrate multiple function calls in a run, and there are multiple places where better error\/exception handling is required. You can read more about how multiple function calls might behave in the OpenAI Assistant documentation<\/a>.\n<\/p>\n
    \n  Here is a screenshot of JetchatAI for Android showing an assistant conversation using the askWikipedia<\/code> function:\n<\/p>\n
    \n  \"JetchatAI\nFigure 6: JetchatAI #assistant-chat showing a response generated by a function call<\/em>\n<\/p>\n
    Feedback and resources<\/h2>\n
    \n  You can view the complete code for the assistant function call in this pull request<\/a> for JetchatAI. \n<\/p>\n
    \n  Refer to the OpenAI blog<\/a> for more details on the Dev Day announcements, and the openai-kotlin repo<\/a> for updates on support for the new features like the Assistant API.\u00a0\n<\/p>\n
    \n  We\u2019d love your feedback on this post, including any tips or tricks you\u2019ve learned from playing around with ChatGPT prompts.\u00a0\n<\/p>\n
    \n  If you have any thoughts or questions, use the feedback forum<\/a> or message us on Twitter @surfaceduodev<\/a>.\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"
    Hello prompt engineers, This week, we are taking one last look at the new Assistants API. Previous blog posts have covered the Retrieval tool with uploaded files and the Code interpreter tool. In today\u2019s post, we\u2019ll add the askWikipedia function that we\u2019d previously built to the fictitious Contoso employee handbook document chat. Configure functions in […]<\/p>\n","protected":false},"author":570,"featured_media":3653,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[741],"tags":[734,733],"class_list":["post-3645","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-chatgpt","tag-openai"],"acf":[],"blog_post_summary":"
    Hello prompt engineers, This week, we are taking one last look at the new Assistants API. Previous blog posts have covered the Retrieval tool with uploaded files and the Code interpreter tool. In today\u2019s post, we\u2019ll add the askWikipedia function that we\u2019d previously built to the fictitious Contoso employee handbook document chat. Configure functions in […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/3645","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/users\/570"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/comments?post=3645"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/3645\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media\/3653"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media?parent=3645"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/categories?post=3645"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/tags?post=3645"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}