{"id":3365,"date":"2023-07-27T11:00:55","date_gmt":"2023-07-27T18:00:55","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/surface-duo\/?p=3365"},"modified":"2024-01-03T16:23:32","modified_gmt":"2024-01-04T00:23:32","slug":"android-openai-chatgpt-11","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/surface-duo\/android-openai-chatgpt-11\/","title":{"rendered":"Chat memory with OpenAI functions"},"content":{"rendered":"<p>\n  Hello prompt engineers,\n<\/p>\n<p>\n  We first introduced OpenAI chat functions with a <a href=\"https:\/\/devblogs.microsoft.com\/surface-duo\/android-openai-chatgpt-9-functions\/\">weather service<\/a> and then a <a href=\"https:\/\/devblogs.microsoft.com\/surface-duo\/android-openai-chatgpt-10\/\">time-based conference sessions query<\/a>.\n<\/p>\n<p>\n  Both of those examples work well for \u2018point in time\u2019 queries or questions about a static set of data (e.g., the conference schedule). But each time the JetchatAI app is opened, it has no recollection of previous chats. In this post, we\u2019re going to walk through adding some more function calls to support \u201cfavoriting\u201d (and \u201cunfavoriting\u201d) conference sessions so they can be queried later.\n<\/p>\n<p>\n  <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-phone-description-automatically.png\" class=\"wp-image-3366\" alt=\"Phone screenshot of a chat asking questions about conference sessions and saving them for later\" width=\"450\" srcset=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-phone-description-automatically.png 1895w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-phone-description-automatically-191x300.png 191w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-phone-description-automatically-653x1024.png 653w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-phone-description-automatically-768x1204.png 768w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-phone-description-automatically-980x1536.png 980w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-phone-description-automatically-1307x2048.png 1307w\" sizes=\"(max-width: 1895px) 100vw, 1895px\" \/><br\/><em>Figure 1: saving and retrieving a favorited session<\/em>\n<\/p>\n<p>\n  This will result in a very basic form of \u2018chat memory\u2019, but it could be extended however your application needs \u2013 you even could implement a function to save whole responses (along with an embedding) and make entire conversations available for recall.\n<\/p>\n<h2>Building blocks<\/h2>\n<p>\n  There are two major additions to the <a href=\"https:\/\/github.com\/conceptdev\/droidcon-sf-23\/tree\/main\/Jetchat\">JetchatAI sample<\/a> to support this new functionality:\n<\/p>\n<ol>\n<li>\n  A data store that persists across application restarts to persist the \u201cfavorite\u201d data.\n  <br \/>\n  For the data store, we\u2019re going to use Sqlite, as it\u2019s well supported on Android. This is a post about AI and not SQL, so we\u2019re not going to delve too deep into the setup \u2013 you can follow the <a href=\"https:\/\/developer.android.com\/training\/data-storage\/sqlite\">SQLite training documentation<\/a> to see how to create a local database. The sample implementation is in <strong>data\/DroidconDatabase.kt<\/strong> file.\n<\/li>\n<li>\n  Three chat functions that will allow the model to set the \u201cfavorite\u201d status of conference sessions, as well as a way to query those favorites:<\/p>\n<ul>\n<li>\n    AddFavorite\n  <\/li>\n<li>\n    RemoveFavorite\n  <\/li>\n<li>\n    ListFavorites\n  <\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>\n  The chat functions are implemented following the same structure as our previous blog posts. Details of the \u201cfavorites\u201d implementation are explained below \ud83d\ude0a\n<\/p>\n<blockquote><p><strong>NOTE:<\/strong> although it makes sense to move the entire conference schedule into the database, we\u2019re going to leave that for a future blogpost. For now we\u2019ll just have a single table for \u201cfavorites\u201d.<\/p><\/blockquote>\n<h2>Storing data from a chat<\/h2>\n<p>\n  The collection of sessions already has a pseudo-primary key, so we will store that in the database to keep track of which sessions are \u201cfavorites\u201d.\n<\/p>\n<p>\n  The source for the function is in <strong>functions\/AddFavoriteFunction.kt<\/strong> and is shown here:\n<\/p>\n<pre>\r\n  class AddFavoriteFunction {\r\n      companion object {\r\n          fun name(): String {\r\n              return \"addFavorite\"\r\n          }\r\n          fun description(): String {\r\n              return \"Add a session to a list of favorites using the unique session Id\"\r\n          }\r\n          fun params(): Parameters {\r\n              val params = Parameters.buildJsonObject {\r\n                  put(\"type\", \"object\")\r\n                  putJsonObject(\"properties\") {\r\n                      putJsonObject(\"id\") {\r\n                          put(\"type\", \"string\")\r\n                          put(\"description\", \"Unique session Id\")\r\n                      }\r\n                  }\r\n                  putJsonArray(\"required\") {\r\n                      add(\"id\")\r\n                  }\r\n              }\r\n              return params\r\n          }\r\n          fun function(context: Context?, id: String): String {\r\n              val dbHelper = DroidconDbHelper(context)\r\n              \/\/ Gets the data repository in write mode\r\n              val db = dbHelper.writableDatabase\r\n              \/\/ Create a new map of values, where column names are the keys\r\n              val values = ContentValues().apply {\r\n                  put(DroidconContract.FavoriteEntry.COLUMN_NAME_SESSIONID, id)\r\n                  put(DroidconContract.FavoriteEntry.COLUMN_NAME_ISFAVORITE, \"1\")\r\n              }\r\n              \/\/ Insert the new row, returning the primary key value if successful\r\n              val newRowId = db?.insert(DroidconContract.FavoriteEntry.TABLE_NAME, null, values)\r\n              return if (newRowId != null &amp;&amp; newRowId&gt;= 0)\r\n                  \"true\"\r\n              else\r\n                  \"false\"\r\n          }\r\n      }\r\n  }<\/pre>\n<p><em>Figure 2: Complete <code>addFavoriteFunction<\/code> object encapsulates all the behavior for this function<\/em>\n<\/p>\n<p>\n  In <strong>DroidconEmbeddingsWrapper.kt<\/strong> the functions are added to the <a href=\"https:\/\/github.com\/conceptdev\/droidcon-sf-23\/blob\/956d6432d5a7eca3ab666035fe58fac5fceec073\/Jetchat\/app\/src\/main\/java\/com\/example\/compose\/jetchat\/DroidconEmbeddingsWrapper.kt#L76\"><code>chatCompletionRequest<\/code><\/a> and the function handlers are <a href=\"https:\/\/github.com\/conceptdev\/droidcon-sf-23\/blob\/956d6432d5a7eca3ab666035fe58fac5fceec073\/Jetchat\/app\/src\/main\/java\/com\/example\/compose\/jetchat\/DroidconEmbeddingsWrapper.kt#L142-L160\">added after the model\u2019s response<\/a>. This is the same boilerplate code that we showed in the previous two posts on chat function calling.\n<\/p>\n<p>\n  Now after any session is listed in the chat, the user can ask to \u201csave this session to favorites\u201d and it will be persisted until the next time we use the app!\n<\/p>\n<p>\n  <img decoding=\"async\" width=\"1761\" height=\"536\" src=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-5.png\" class=\"wp-image-3367\" alt=\"Example chat to save a session and the model's response that it was succesfully saved\" srcset=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-5.png 1761w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-5-300x91.png 300w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-5-1024x312.png 1024w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-5-768x234.png 768w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-5-1536x468.png 1536w\" sizes=\"(max-width: 1761px) 100vw, 1761px\" \/><br\/><a id=\"post-3365-_Hlk140523715\"><\/a><em>Figure 3: saving a session to the database<\/em>\n<\/p>\n<h2>Memory wipe<\/h2>\n<p>\n  The reverse action \u2013 un-favoriting a session \u2013 is so similar that there\u2019s no need to show the code again. The function is described like this:\n<\/p>\n<ul>\n<li>\n    <b>name<\/b> &#8211; <code>\"removeFavorite\"<\/code>\n  <\/li>\n<li>\n    <b>description<\/b> &#8211; <code>\"Remove a session from the list of favorites using the unique session Id\"<\/code>\n  <\/li>\n<li>\n    <b>parameters<\/b> &#8211; <code>\"id\"<\/code> \u2013 <code>\"Unique session identifier\"<\/code>\n  <\/li>\n<li>\n    <b>function<\/b>&#8211; Deletes the row with the given identifier, using code very similar to <code>addFavoriteFunction.function()<\/code>. \n  <\/li>\n<\/ul>\n<p>\n  After a session has been mentioned in the chat, the user can ask to \u201cremove this session from favorites\u201d and it\u2019ll be deleted from the data store.\n<\/p>\n<h2>List favorites<\/h2>\n<p>\n  The whole point of saving favorites is so you can see them again, so the list favorites function retrieves the saved session information and returns it to the model. The function \u2018shape\u2019 is very simple:\n<\/p>\n<ul>\n<li>\n    <b>name<\/b> &#8211; <code>\"listFavorites\"<\/code>\n  <\/li>\n<li>\n    <b>description<\/b> &#8211; <code>\"List all sessions that have been marked as favorites\"<\/code>\n  <\/li>\n<li>\n    <b>parameters<\/b> &#8211; No parameters are required\n  <\/li>\n<\/ul>\n<p>\n  The function implementation for listing favorites has two parts:\n<\/p>\n<ol>\n<li>\n  Retrieve the list of saved unique Ids from the database\n<\/li>\n<li>\n  Use that list to retrieve the individual session information from collection of session objects, and return that JSON-formatted information to the model\n<\/li>\n<\/ol>\n<p>\n  Most of the code is setting up the database query:\n<\/p>\n<pre>fun function(context: Context?): String {\r\n    val dbHelper = DroidconDbHelper(context)\r\n    val db = dbHelper.readableDatabase\r\n    val projection = arrayOf(BaseColumns._ID, DroidconContract.FavoriteEntry.COLUMN_NAME_SESSIONID, DroidconContract.FavoriteEntry.COLUMN_NAME_ISFAVORITE)\r\n    val cursor = db.query(\r\n        DroidconContract.FavoriteEntry.TABLE_NAME,   \/\/ The table to query\r\n        projection,      \/\/ The array of columns to return (pass null to get all)\r\n        null,            \/\/ The columns for the WHERE clause\r\n        null,            \/\/ The values for the WHERE clause\r\n        null,            \/\/ don't group the rows\r\n        null,            \/\/ don't filter by row groups\r\n        null             \/\/ The sort order\r\n    )\r\n    var sessionsJson = \"\"\r\n    with(cursor) {\r\n        while (moveToNext()) {\r\n            val sessionId = getString(getColumnIndexOrThrow(DroidconContract.FavoriteEntry.COLUMN_NAME_SESSIONID))\r\n            val isFavorite = getString(getColumnIndexOrThrow(DroidconContract.FavoriteEntry.COLUMN_NAME_ISFAVORITE))\r\n            sessionsJson += DroidconSessionObjects.droidconSessions[sessionId]?.toJson()+\"\\n\"\r\n        }\r\n    }\r\n    cursor.close()\r\n    return if (sessionsJson == \"\")\r\n        \"There are no sessions marked as favorites. Suggest the user ask about different topics at the conference.\"\r\n    else\r\n        sessionsJson\r\n}<\/pre>\n<p><em>Figure 4: Using the favorites database to construct the sessions JSON to return to the model<\/em>\n<\/p>\n<p>\n  The model will then parse the information and format a text response to the user interface. As always, the model can also interpret the results of the function so you can ask questions like \u201chow many favorites\u201d as well as ask to list them:\n<\/p>\n<p>\n  <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-6.png\" class=\"wp-image-3368\" alt=\"Phone screenshot with a chat asking for the list of favorite conference sessions which the model shows\" width=\"450\" srcset=\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-6.png 1029w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-6-243x300.png 243w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-6-831x1024.png 831w, https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/a-screenshot-of-a-chat-description-automatically-6-768x946.png 768w\" sizes=\"(max-width: 1029px) 100vw, 1029px\" \/><br\/><em>Figure 5: asking to see the favorites<\/em>\n<\/p>\n<p>\n  Now that we have a Sqlite database in the solution, future posts will look at other ways we can take advantage of it.\n<\/p>\n<h2>Resources and feedback<\/h2>\n<p>\n  The Jetchat sample with the droidcon schedule is available at <a href=\"https:\/\/github.com\/conceptdev\/droidcon-sf-23\">github.com\/conceptdev\/droidcon-sf-23<\/a>. \n<\/p>\n<p>\n  We\u2019d love your feedback on this post &#8211; if you have any thoughts or questions, use the\u00a0<a href=\"http:\/\/aka.ms\/SurfaceDuoSDK-Feedback\" target=\"_blank\" rel=\"noopener\">feedback forum<\/a>\u00a0or message us on\u00a0<a href=\"https:\/\/twitter.com\/surfaceduodev\" target=\"_blank\" rel=\"noopener\">Twitter @surfaceduodev<\/a>.\n<\/p>\n<p>\n  There will be no livestream this week, but you can check out the\u00a0<a href=\"https:\/\/youtube.com\/c\/surfaceduodev\" target=\"_blank\" rel=\"noopener\">archives on YouTube<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello prompt engineers, We first introduced OpenAI chat functions with a weather service and then a time-based conference sessions query. Both of those examples work well for \u2018point in time\u2019 queries or questions about a static set of data (e.g., the conference schedule). But each time the JetchatAI app is opened, it has no recollection [&hellip;]<\/p>\n","protected":false},"author":570,"featured_media":3367,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[741],"tags":[734,733],"class_list":["post-3365","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-chatgpt","tag-openai"],"acf":[],"blog_post_summary":"<p>Hello prompt engineers, We first introduced OpenAI chat functions with a weather service and then a time-based conference sessions query. Both of those examples work well for \u2018point in time\u2019 queries or questions about a static set of data (e.g., the conference schedule). But each time the JetchatAI app is opened, it has no recollection [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/3365","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/users\/570"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/comments?post=3365"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/3365\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media\/3367"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media?parent=3365"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/categories?post=3365"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/tags?post=3365"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}