\n Following the function calling documentation<\/a> and the example provided by the OpenAI kotlin client-library<\/a>, a real-time \u201cweather\u201d data source will be added to the chat. Figures 1 and 2 below show how the chat response before and after implementing the function:\n<\/p>\n \n \n \n A \u201cchat function\u201d isn\u2019t actually code running in the model or on the server. Rather, as part of the chat completion request, you tell that model that you can execute code on its behalf, and describe (in both plain text and with a JSON structure) what parameters your code can accept.\n<\/p>\n \n The model will then decide, based on the user\u2019s input and the capabilities you described, whether your function might be able to help with the completion. If so, the model response indicates \u201cplease execute your code with these parameters, and send back the result\u201d. Rather than show this response to the user, your code extracts the parameters, runs the code, and sends the result to the model.\n<\/p>\n \n The model now reconsiders how to respond to the user input, using all the context from earlier in the chat plus<\/em> the result of your function. It may use all, some, or none of the function result, depending on how well it matches the user\u2019s initial request. The model may also re-phrase, paraphrase, or otherwise interpret your function\u2019s results to best fit the completion.\n<\/p>\n \n If you have provided multiple functions, the model will choose the best one (according to its capabilities), and if none seem to be applicable then it will respond with a completion without calling any functions.\n<\/p>\n \n The code added to the JetchatAI sample<\/a> is a variation of the example provided in the OpenAI Kotlin client library docs<\/a>. Before adding code make sure to update your build.gradle<\/strong> file to use the latest version of the client library (v3.3.1) which includes function calling support:\n<\/p>\n \n Also make sure you\u2019re referencing a chat model that supports the function calling feature, either \n The basic code for making chat completion requests includes just the model name and the serialized conversation (including the most recent entry by the user). You can find this in the OpenAIWrapper.kt<\/strong> file in the JetchatAI sample:\n<\/p>\n Figure 3: executing a chat completion without functions<\/em>\n<\/p>\n \n To provide the model with a function (or functions), the completion request needs these additional parameters:\n<\/p>\n \n The updated chat completion request is shown below:\n<\/p>\n Figure 4: a function for getting the current weather added to the chat completion request<\/em>\n<\/p>\n \n To keep the code readable the function parameters (and implementation) are in the OpenAIFunctions.kt<\/strong> file. The parameters are set in the \n There is also an element to specify which parameters are Figure 5: describe the shape of the function parameters using JSON<\/em>\n<\/p>\n \n With just the above changes the model will detect that you have declared a function and attempt to use it if appropriate (according to the user\u2019s input). To actually respond to the model\u2019s request, we need to add more code handling the chat completion.\n<\/p>\n \n The basic code for handling a chat response is very simple \u2013 it extracts the most recent message content (from the model), adds it to the conversation history and renders it on screen:\n<\/p>\n Figure 6: handling a chat response without functions<\/em>\n<\/p>\n \n Because we have added a function to the response, it\u2019s possible that the model decided to \u201ccall\u201d it, so we need to handle that case after the chat completion by checking the \n This additional round-trip to the model is hidden from the user \u2013 they do not see the response with the function call, nor do they have any direct input into the request that is sent back with the function\u2019s results. The subsequent<\/em> response \u2013 where the model has considered the function\u2019s results and incorporated them into a final response \u2013 is what is rendered in the UI.\n<\/p>\n Figure 7: Code to handle a function call request from the model<\/em>\n<\/p>\n \n The above code will successfully define a function for OpenAI and handle the case where the model wishes to call the function, but the actual implementation of the \n To make this demo work without too many dependencies, it uses the free weather data API from weather.gov<\/a>. Note that this only works for locations within the United States, and the provider requests a unique user-agent be sent with each request (which can be set in Constants.kt<\/strong>) \u2013 failure to add a user-agent string may result in failed requests.\n<\/p>\n \n The full implementation of \n This particular API cannot work directly with latitude and longitude, but instead requires the location be converted to a grid reference.\n<\/p>\n Figure 8: code to convert a lat\/long location into a grid reference<\/em>\n<\/p>\n \n Using the grid reference another web service call will return the weather forecast. The response includes multiple days of information, but the code retrieves just the most current data.\n<\/p>\n Figure 9: code to get a weather forecast for a grid location<\/em>\n<\/p>\n \n Once the current weather forecast data has been extracted, use the Figure 10: a simple data class is used to format the output as JSON<\/em>\n<\/p>\n \n As mentioned above, the model doesn\u2019t require a specific\/declared response format \u2013 it will use all the context you provide (e.g., the key names as well as the data values) to help it understand the data and incorporate it in the response to the user.\n<\/p>\n \n The model cannot know your location by default, so with the implementation already shown if the user asks \u201cwhat\u2019s the weather\u201d the model will respond with a question about their location. To help make the chat seem smarter we could use the location services APIs to determine the device\u2019s location and ground each message with that information \u2013 but for now the demo uses a hardcoded default location that you can set in Constants.kt<\/strong>.\n<\/p>\n Figure 11: simple grounding data added to the prompt<\/em>\n<\/p>\n \n With this added to the prompt (but not shown in the UI), it\u2019s easy to ask for local weather or to give another location:\n<\/p>\n \n \n You can find the code for this post (plus more) in this pull request<\/a> on the JetchatAI repo<\/a>.\n<\/p>\n \n If you have any questions, use the\u00a0feedback forum<\/a>\u00a0or message us on\u00a0Twitter @surfaceduodev<\/a>.\n<\/p>\n![\"Chat](\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/word-image-3339-1.png\")
![\"Chat](\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/word-image-3339-2.png\")
How chat functions work<\/h2>\n
How to implement a function<\/h2>\n
com.aallam.openai:openai-client:3.3.1<\/pre>\n
gpt-3.5-turbo-0613<\/code> or gpt-4<\/code> (see Constants.kt<\/strong> in the sample project). Don\u2019t forget that you\u2019ll need to add your OpenAI developer key to the constants class as well.\n<\/p>\nAdd the function call to the chat<\/h3>\n
val chatCompletionRequest = chatCompletionRequest {\r\n model = ModelId(Constants.OPENAI_CHAT_MODEL)\r\n messages = conversation\r\n}<\/pre>\n\n
functions<\/code> \u2013 a collection of function<\/code>, each of which has:<\/p>\n\n
name<\/code> \u2013 key used to track which function is called\n <\/li>\ndescription<\/code> \u2013 explanation of what the function does \u2013 the model will use this to determine if\/when to call the function\n <\/li>\nparameters<\/code> \u2013 a JSON representation of the function\u2019s parameters following the OpenAPI specification (shown in Figure 5)\n <\/li>\n<\/ul>\n<\/li>\nfunctionCall<\/code> \u2013 the ability to control how functions are called. Auto<\/code> means the model will decide, but you can also turn the functions off (FunctionMode.None<\/code>) OR force a specific function to be called with FunctionMode.Named<\/code> followed by the function\u2019s name.\n <\/li>\n<\/ul>\nval chatCompletionRequest = chatCompletionRequest {\r\n model = ModelId(Constants.OPENAI_CHAT_MODEL)\r\n messages = conversation\r\n functions {\r\n function {\r\n name = \"currentWeather\"\r\n description = \"Get the current weather in a given location\"\r\n parameters = OpenAIFunctions.currentWeatherParams()\r\n }\r\n }\r\n functionCall = FunctionMode.Auto\r\n}<\/pre>\ncurrentWeatherParams<\/code> function, which returns the attributes of all the function parameters, such as:\n<\/p>\n\n
name<\/code> – each parameter has a name, such as \"latitude\"<\/code> and \"longitude\"<\/code>\n <\/li>\ntype<\/code> – data type, such as \"string\"<\/code>. This can also be an enum<\/code> as shown in Figure 5.\n <\/li>\ndescription<\/code> \u2013 an explanation of the parameter\u2019s purpose \u2013 the model will use this to understand what data should be provided.\n <\/li>\n<\/ul>\nrequired<\/code>, which will also help the model collect information from the user. The parameters for the weather function are described like this:\n<\/p>\nfun currentWeatherParams(): Parameters {\r\n val params = Parameters.buildJsonObject {\r\n put(\"type\", \"object\")\r\n putJsonObject(\"properties\") {\r\n putJsonObject(\"latitude\") {\r\n put(\"type\", \"string\")\r\n put(\"description\", \"The latitude of the requested location, e.g. 37.773972 for San Francisco, CA\")\r\n }\r\n putJsonObject(\"longitude\") {\r\n put(\"type\", \"string\")\r\n put(\"description\", \"The longitude of the requested location, e.g. -122.431297 for San Francisco, CA\")\r\n }\r\n putJsonObject(\"unit\") {\r\n put(\"type\", \"string\")\r\n putJsonArray(\"enum\") {\r\n add(\"celsius\")\r\n add(\"fahrenheit\")\r\n }\r\n }\r\n }\r\n putJsonArray(\"required\") {\r\n add(\"latitude\")\r\n add(\"longitude\")\r\n }\r\n }\r\n return params\r\n}<\/pre>\nImplement the function call behaviour<\/h3>\n
\/\/ this value is added to the UI\r\nval chatResponse = completion.choices[0].message?.content ?: \"\"\r\n\/\/ add the response to the conversation history for subsequent requests\r\nconversation.add(\r\n ChatMessage(\r\n role = ChatRole.Assistant,\r\n content = chatResponse\r\n )\r\n)<\/pre>\n
functionCall<\/code> property of the response. If a function name is provided, this indicates the model wishes to \u201cexecute\u201d that function, so we need to extract the parameters, perform the function, and send the function\u2019s output back to the model for consideration.\n<\/p>\nif (completionMessage.functionCall != null) {\r\n \/\/ handle function\r\n val function = completionMessage.functionCall\r\n if (function!!.name == \"currentWeather\")\r\n {\r\n val functionArgs = function.argumentsAsJson() ?: error(\"arguments field is missing\")\r\n \/\/ CALL THE FUNCTION with the parameters sent back by the model\r\n val functionResponse = OpenAIFunctions.currentWeather( \/\/ implementation to come\u2026\r\n functionArgs.getValue(\"latitude\").jsonPrimitive.content,\r\n functionArgs.getValue(\"longitude\").jsonPrimitive.content,\r\n functionArgs[\"unit\"]?.jsonPrimitive?.content ?: \"fahrenheit\"\r\n )\r\n \/\/ add the model\u2019s \"call a function\" response to the history \u2013 this doesn\u2019t show in the UI\r\n conversation.add(\r\n ChatMessage(\r\n role = completionMessage.role,\r\n content = completionMessage.content ?: \"\", \/\/ required to not be empty\r\n functionCall = completionMessage.functionCall\r\n )\r\n )\r\n \/\/ add the response to the \"function call\" to the history \u2013 this doesn\u2019t show in the UI\r\n conversation.add(\r\n ChatMessage(\r\n role = ChatRole.Function, \/\/ this is a new role type\r\n name = function.name,\r\n content = functionResponse\r\n )\r\n )\r\n \/\/ send the function request\/response back to the model\r\n val functionCompletionRequest = chatCompletionRequest {\r\n model = ModelId(Constants.OPENAI_CHAT_MODEL)\r\n messages = conversation }\r\n val functionCompletion: ChatCompletion = openAI.chatCompletion(functionCompletionRequest)\r\n \/\/ show the interpreted function response as chat completion in the UI\r\n chatResponse = functionCompletion.choices.first().message?.content!!\r\n conversation.add(\r\n ChatMessage(\r\n role = ChatRole.Assistant,\r\n content = chatResponse\r\n )\r\n )\r\n }\r\n}<\/pre>\ncurrentWeather()<\/code> code has not yet been discussed. The next section explains how a basic web service client for weather.gov<\/a> can be implemented, although the OpanAI model does not care about the local implementation of the function call.\n<\/p>\nWire up a web service<\/h2>\n
currentWeather<\/code> is in OpenAIFunctions.kt<\/strong> in the JetchatAI sample. There are three steps to using this API:\n<\/p>\n\n
\n This is a web service call with latitude and longitude parameters. It will return 404<\/b> for locations outside the USA. Parse the JSON result to determine the grid location (office<\/code>, gridX<\/code>, gridY<\/code>).\n<\/li>\n
\n Create another web service call to get the weather forecast for the given \u2018grid location\u2019. Parse the JSON result to get the forecast information (name<\/code>, temperature<\/code>, temperatureUnit<\/code>, detailedForecast<\/code>).\n<\/li>\n
\n Although there is no strict requirement for formatting, in this case the code sends JSON back to the model using the data class weatherInfo<\/code>. The model will use the keys and values to interpret the data and use it in the completion shown to the user.\n<\/li>\n<\/ol>\n1. Get grid location<\/h3>\n
val gridUrl = \"https:\/\/api.weather.gov\/points\/$latitude,$longitude\"\r\nval gridResponse = httpClient.get(gridUrl) {\r\n contentType(ContentType.Application.Json)\r\n}\r\n\/\/\u2026 then extract grid info\r\nval responseText = gridResponse.bodyAsText()\r\nval responseJson =Json.parseToJsonElement(responseText).jsonObject[\"properties\"]!!\r\nvar office = responseJson.jsonObject[\"gridId\"]?.jsonPrimitive?.content<\/pre>\n2. Get weather forecast<\/h3>\n
val forecastUrl =\r\n \"https:\/\/api.weather.gov\/gridpoints\/$office\/$gridX,$gridY\/forecast\"\r\nval forecastResponse = httpClient.get(forecastUrl) {\r\n contentType(ContentType.Application.Json)\r\n}\r\n\/\/\u2026 then extract forecast info\r\nJson.parseToJsonElement(responseText).jsonObject[\"properties\"]!!\r\nval periods = responseJson.jsonObject[\"periods\"]!!\r\nval period1 = periods.jsonArray[0]!!\r\nvar name = period1.jsonObject[\"name\"]?.jsonPrimitive?.content!!\r\nvar temperature = period1.jsonObject[\"temperature\"]?.jsonPrimitive?.content!!<\/pre>\n3. Format result<\/h3>\n
WeatherInfo<\/code> class to format as JSON to include in the chat response:\n<\/p>\nval weatherInfo = WeatherInfo(\r\n latitude,\r\n longitude,\r\n temperature,\r\n unit,\r\n listOf(name, detailedForecast)\r\n)\r\nreturn weatherInfo.toJson()<\/pre>\n
One last thing \u2013 grounding<\/h2>\n
val groundedMessage = \"Current location is ${Constants.TEST_LOCATION}.\\n\\n$message\"<\/pre>\n![\"Android](\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/07\/word-image-3339-3.png\")
Feedback and resources<\/h2>\n