{"id":3623,"date":"2023-11-30T19:09:06","date_gmt":"2023-12-01T03:09:06","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/surface-duo\/?p=3623"},"modified":"2024-01-03T16:04:43","modified_gmt":"2024-01-04T00:04:43","slug":"android-openai-chatgpt-27","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/surface-duo\/android-openai-chatgpt-27\/","title":{"rendered":"OpenAI Assistant on Android"},"content":{"rendered":"<p>\n  Hello prompt engineers,\n<\/p>\n<p>\n  This week we\u2019re continuing to discuss the new Assistant API announced at OpenAI Dev Day. There is <a href=\"https:\/\/platform.openai.com\/docs\/assistants\/how-it-works\/managing-threads-and-messages\">documentation available<\/a> that explains how the API works and shows python\/javascript\/curl examples, but in this post we\u2019ll implement in Kotlin for Android and Jetpack Compose. You can review the code in this <a href=\"https:\/\/github.com\/conceptdev\/droidcon-sf-23\/pull\/24\">JetchatAI pull request<\/a>.\n<\/p>\n<h2>OpenAI Assistants<\/h2>\n<p>\n  A few weeks ago, we demonstrated <a href=\"https:\/\/devblogs.microsoft.com\/surface-duo\/openai-assistants\/\">building a simple Assistant<\/a> in the OpenAI Playground \u2013 uploading files, setting a system prompt, and performing RAG-assisted queries \u2013 mimicking this <a href=\"https:\/\/github.com\/azure-samples\/azure-search-openai-demo\">Azure demo<\/a>. To refresh your memory, Figure 1 shows the Assistant configuration:\n<\/p>\n<p>\n  <img decoding=\"async\" width=\"1099\" height=\"821\" src=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/word-image-3623-1.png\" class=\"wp-image-3624\" alt=\"Screenshot of the OpenAI Assistant Playground showing the test configuration\" srcset=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/word-image-3623-1.png 1099w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/word-image-3623-1-300x224.png 300w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/word-image-3623-1-1024x765.png 1024w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/word-image-3623-1-768x574.png 768w\" sizes=\"(max-width: 1099px) 100vw, 1099px\" \/><br\/><em>Figure 1: OpenAI Assistant Playground (key explained in <a href=\"https:\/\/devblogs.microsoft.com\/surface-duo\/openai-assistants\/\">this post<\/a>)<\/em>\n<\/p>\n<p>\n  We are going to reference this specific assistant, which has been configured in the playground, from an Android app. After the assistant is created, a unique identifier is displayed under the assistant name (see Figure 2), which can be referenced via the API.\n<\/p>\n<p>\n  <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-close-up-of-a-computer-screen-description-autom.png\" class=\"wp-image-3625\" alt=\"\" width=\"300\" srcset=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-close-up-of-a-computer-screen-description-autom.png 571w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-close-up-of-a-computer-screen-description-autom-300x78.png 300w\" sizes=\"(max-width: 571px) 100vw, 571px\" \/><br\/><em>Figure 2: Assistant unique identifier <\/em>\n<\/p>\n<h2>Building in Kotlin <\/h2>\n<p>\n  The <a href=\"https:\/\/github.com\/aallam\/openai-kotlin\/\">openai-kotlin<\/a> GitHub repo contains some <a href=\"https:\/\/github.com\/aallam\/openai-kotlin\/blob\/main\/guides\/Assistants.md\">basic instructions<\/a> for accessing the Assistant API in Kotlin. This guidance has been adapted to work in <em>JetchatAI<\/em>, as shown in Figure 3. Comparing this file (<strong>AssistantWrapper.kt<\/strong>) to earlier examples (eg. <strong>OpenAIWrapper.kt<\/strong>) you\u2019ll notice it is significantly simpler! Using the Assistant API means:\n<\/p>\n<ul>\n<li>\n    No need for tracking token usage and message history for the sliding window \u2013 the Assistant API will automatically manage the model input size limit.\n  <\/li>\n<li>\n    No need to keep sending past messages with each request, since they are stored on the server. Each user query is added to a thread which is then run against a configured assistant, entirely on the server.\n  <\/li>\n<li>\n    No need to manually load or chunk document contents \u2013 we can upload documents and they will be chunked and stored on the server. Embeddings will automatically be generated.\n  <\/li>\n<li>\n    No need to generate an embedding for the user\u2019s query or do the vector similarity comparisons in Kotlin. The RAG will be done by the Assistant API on the server.\n  <\/li>\n<\/ul>\n<p>\n  Many lines of code over multiple files in older examples can be reduced to the <code>chat<\/code> function shown in Figure 3:\n<\/p>\n<pre>suspend fun chat(message: String): String {\r\n    if (assistant == null) { \/\/ open assistant and create a new thread every app-start\r\n       assistant = openAI.assistant(id = AssistantId(Constants.OPENAI_ASSISTANT_ID)) \/\/ from the Playground\r\n       thread = openAI.thread()\r\n    }\r\n    val userMessage = openAI.message (\r\n       threadId = thread!!.id,\r\n       request = MessageRequest(\r\n          role = Role.User,\r\n          content = message\r\n       )\r\n    )\r\n    val run = openAI.createRun(\r\n       threadId = thread!!.id,\r\n       request = RunRequest(assistantId = assistant!!.id)\r\n    )\r\n    do\r\n    {\r\n       delay(1_000)\r\n       val runTest = openAI.getRun(thread!!.id, run.id)\r\n    } while (runTest.status != Status.Completed)\r\n    val messages = openAI.messages(thread!!.id)\r\n    val message = messages.first() \/\/ bit of a hack, get the last one generated\r\n    val messageContent = message.content[0]\r\n    if (messageContent is MessageContent.Text) {\r\n       return messageContent.text.value\r\n    }\r\n    return \"&lt;Assistant Error&gt;\" \/\/ TODO: error handling\r\n}<\/pre>\n<p><em>Figure 3: new <code>chat<\/code> function using a configured Assistant (by Id) using Kotlin API<\/em>\n<\/p>\n<p>\n  Assistants can also support function calling, but we haven\u2019t included any function demonstration in this sample.\n<\/p>\n<h2>Try it out<\/h2>\n<p>\n  To access the new Assistant channel in <em>JetchatAI<\/em>, first choose <strong>#assistant-chat <\/strong>from the chat list:\n<\/p>\n<p>\n  <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-4.png\" class=\"wp-image-3626\" alt=\"Screenshot of the JetchatAI app showing the channel menu and how to choose the assistant chat\" width=\"300\" srcset=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-4.png 1132w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-4-287x300.png 287w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-4-981x1024.png 981w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-4-768x802.png 768w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-4-24x24.png 24w\" sizes=\"(max-width: 1132px) 100vw, 1132px\" \/><br\/><em>Figure 4: Access the #assistant-chat channel in the Chats menu of JetchatAI<\/em>\n<\/p>\n<p>\n  You can now ask questions relating to the content of the five source PDF documents that were uploaded in the playground. The documents relate to employee policies and health plans of the fictitious Contoso\/Northwind organizations. There are two example queries and responses in the screenshot in Figure 5:\n<\/p>\n<p>\n  <img decoding=\"async\" width=\"1895\" height=\"2335\" src=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-5.png\" class=\"wp-image-3627\" alt=\"Screenshot of the JetchatAI app showing two user queries answered by the assistant\" srcset=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-5.png 1895w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-5-243x300.png 243w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-5-831x1024.png 831w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-5-768x946.png 768w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-5-1247x1536.png 1247w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/11\/a-screenshot-of-a-chat-description-automatically-5-1662x2048.png 1662w\" sizes=\"(max-width: 1895px) 100vw, 1895px\" \/><br\/><em>Figure 5: user queries against the Assistant loaded with fictitious \u2018employee policy\u2019 PDFs as the data source<\/em>\n<\/p>\n<h2>Next steps<\/h2>\n<p>\n  The Assistant API is still in preview, so it could change before the official release. The current <em>JetchatAI<\/em> implementation needs some more work in error handling, resolving citations, checking for multiple assistant responses, implementing functions, and more. The takeaway from this week is how much simpler the Assistant API is to use versus implementing the Chat API directly.\n<\/p>\n<h2>Resources and feedback<\/h2>\n<p>\n  Refer to the <a href=\"https:\/\/openai.com\/blog\/new-models-and-developer-products-announced-at-devday\" target=\"_blank\" rel=\"noopener\">OpenAI blog<\/a> for more details on the Dev Day announcements, and the <a href=\"https:\/\/github.com\/aallam\/openai-kotlin\/\">openai-kotlin repo<\/a> for updates on support for the new features.\u00a0\n<\/p>\n<p>\n  We\u2019d love your feedback on this post, including any tips or tricks you\u2019ve learned from playing around with ChatGPT prompts.\u00a0\n<\/p>\n<p>\n  If you have any thoughts or questions, use the <a href=\"http:\/\/aka.ms\/SurfaceDuoSDK-Feedback\" target=\"_blank\" rel=\"noopener\">feedback forum<\/a> or message us on <a href=\"https:\/\/twitter.com\/surfaceduodev\" target=\"_blank\" rel=\"noopener\">Twitter @surfaceduodev<\/a>.\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello prompt engineers, This week we\u2019re continuing to discuss the new Assistant API announced at OpenAI Dev Day. There is documentation available that explains how the API works and shows python\/javascript\/curl examples, but in this post we\u2019ll implement in Kotlin for Android and Jetpack Compose. You can review the code in this JetchatAI pull request. [&hellip;]<\/p>\n","protected":false},"author":570,"featured_media":3627,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[741],"tags":[734,733],"class_list":["post-3623","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-chatgpt","tag-openai"],"acf":[],"blog_post_summary":"<p>Hello prompt engineers, This week we\u2019re continuing to discuss the new Assistant API announced at OpenAI Dev Day. There is documentation available that explains how the API works and shows python\/javascript\/curl examples, but in this post we\u2019ll implement in Kotlin for Android and Jetpack Compose. You can review the code in this JetchatAI pull request. [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/3623","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/users\/570"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/comments?post=3623"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/3623\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media\/3627"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media?parent=3623"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/categories?post=3623"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/tags?post=3623"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}