Chat memory with OpenAI functions

Craig Dunn

Hello prompt engineers,

We first introduced OpenAI chat functions with a weather service and then a time-based conference sessions query.

Both of those examples work well for ‘point in time’ queries or questions about a static set of data (e.g., the conference schedule). But each time the JetchatAI app is opened, it has no recollection of previous chats. In this post, we’re going to walk through adding some more function calls to support “favoriting” (and “unfavoriting”) conference sessions so they can be queried later.

Figure 1: saving and retrieving a favorited session

This will result in a very basic form of ‘chat memory’, but it could be extended however your application needs – you even could implement a function to save whole responses (along with an embedding) and make entire conversations available for recall.

Building blocks

There are two major additions to the JetchatAI sample to support this new functionality:

1. A data store that persists across application restarts to persist the “favorite” data.
   For the data store, we’re going to use Sqlite, as it’s well supported on Android. This is a post about AI and not SQL, so we’re not going to delve too deep into the setup – you can follow the SQLite training documentation to see how to create a local database. The sample implementation is in data/DroidconDatabase.kt file.
2. Three chat functions that will allow the model to set the “favorite” status of conference sessions, as well as a way to query those favorites:
   • AddFavorite
   • RemoveFavorite
   • ListFavorites

The chat functions are implemented following the same structure as our previous blog posts. Details of the “favorites” implementation are explained below 😊

NOTE: although it makes sense to move the entire conference schedule into the database, we’re going to leave that for a future blogpost. For now we’ll just have a single table for “favorites”.

Storing data from a chat

The collection of sessions already has a pseudo-primary key, so we will store that in the database to keep track of which sessions are “favorites”.

The source for the function is in functions/AddFavoriteFunction.kt and is shown here:

  class AddFavoriteFunction {
      companion object {
          fun name(): String {
              return "addFavorite"
          }
          fun description(): String {
              return "Add a session to a list of favorites using the unique session Id"
          }
          fun params(): Parameters {
              val params = Parameters.buildJsonObject {
                  put("type", "object")
                  putJsonObject("properties") {
                      putJsonObject("id") {
                          put("type", "string")
                          put("description", "Unique session Id")
                      }
                  }
                  putJsonArray("required") {
                      add("id")
                  }
              }
              return params
          }
          fun function(context: Context?, id: String): String {
              val dbHelper = DroidconDbHelper(context)
              // Gets the data repository in write mode
              val db = dbHelper.writableDatabase
              // Create a new map of values, where column names are the keys
              val values = ContentValues().apply {
                  put(DroidconContract.FavoriteEntry.COLUMN_NAME_SESSIONID, id)
                  put(DroidconContract.FavoriteEntry.COLUMN_NAME_ISFAVORITE, "1")
              }
              // Insert the new row, returning the primary key value if successful
              val newRowId = db?.insert(DroidconContract.FavoriteEntry.TABLE_NAME, null, values)
              return if (newRowId != null && newRowId>= 0)
                  "true"
              else
                  "false"
          }
      }
  }

Figure 2: Complete addFavoriteFunction object encapsulates all the behavior for this function

In DroidconEmbeddingsWrapper.kt the functions are added to the chatCompletionRequest and the function handlers are added after the model’s response. This is the same boilerplate code that we showed in the previous two posts on chat function calling.

Now after any session is listed in the chat, the user can ask to “save this session to favorites” and it will be persisted until the next time we use the app!

Figure 3: saving a session to the database

Memory wipe

The reverse action – un-favoriting a session – is so similar that there’s no need to show the code again. The function is described like this:

• name – "removeFavorite"
• description – "Remove a session from the list of favorites using the unique session Id"
• parameters – "id" – "Unique session identifier"
• function– Deletes the row with the given identifier, using code very similar to addFavoriteFunction.function().

After a session has been mentioned in the chat, the user can ask to “remove this session from favorites” and it’ll be deleted from the data store.

List favorites

The whole point of saving favorites is so you can see them again, so the list favorites function retrieves the saved session information and returns it to the model. The function ‘shape’ is very simple:

• name – "listFavorites"
• description – "List all sessions that have been marked as favorites"
• parameters – No parameters are required

The function implementation for listing favorites has two parts:

1. Retrieve the list of saved unique Ids from the database
2. Use that list to retrieve the individual session information from collection of session objects, and return that JSON-formatted information to the model

Most of the code is setting up the database query:

fun function(context: Context?): String {
    val dbHelper = DroidconDbHelper(context)
    val db = dbHelper.readableDatabase
    val projection = arrayOf(BaseColumns._ID, DroidconContract.FavoriteEntry.COLUMN_NAME_SESSIONID, DroidconContract.FavoriteEntry.COLUMN_NAME_ISFAVORITE)
    val cursor = db.query(
        DroidconContract.FavoriteEntry.TABLE_NAME,   // The table to query
        projection,      // The array of columns to return (pass null to get all)
        null,            // The columns for the WHERE clause
        null,            // The values for the WHERE clause
        null,            // don't group the rows
        null,            // don't filter by row groups
        null             // The sort order
    )
    var sessionsJson = ""
    with(cursor) {
        while (moveToNext()) {
            val sessionId = getString(getColumnIndexOrThrow(DroidconContract.FavoriteEntry.COLUMN_NAME_SESSIONID))
            val isFavorite = getString(getColumnIndexOrThrow(DroidconContract.FavoriteEntry.COLUMN_NAME_ISFAVORITE))
            sessionsJson += DroidconSessionObjects.droidconSessions[sessionId]?.toJson()+"\n"
        }
    }
    cursor.close()
    return if (sessionsJson == "")
        "There are no sessions marked as favorites. Suggest the user ask about different topics at the conference."
    else
        sessionsJson
}

Figure 4: Using the favorites database to construct the sessions JSON to return to the model

The model will then parse the information and format a text response to the user interface. As always, the model can also interpret the results of the function so you can ask questions like “how many favorites” as well as ask to list them:

Figure 5: asking to see the favorites

Now that we have a Sqlite database in the solution, future posts will look at other ways we can take advantage of it.

Resources and feedback

The Jetchat sample with the droidcon schedule is available at github.com/conceptdev/droidcon-sf-23.

We’d love your feedback on this post – if you have any thoughts or questions, use the feedback forum or message us on Twitter @surfaceduodev.

There will be no livestream this week, but you can check out the archives on YouTube.

Craig Dunn Principal SW Engineer, Surface Duo Developer Experience

Follow