\n Last week, we talked about the different API endpoint options and showed you some examples of how to use the \n The Images API<\/a> uses DALL-E models<\/a> to interact with or generate images based on user prompts. You can use the API in three ways:\n<\/p>\n \n If you\u2019re curious about the output from the \n In this blog post, we\u2019ll be focusing specifically on the image edits endpoint<\/a>. As mentioned above, this endpoint accepts two inputs: a text prompt and an image. The text prompt describes what the final edited image should look like, while the image provided should contain a masked transparent area to show which parts of the image can be edited. As with all the image APIs, the image input must be square and less than 4MB. To learn more, check out the API reference<\/a> and use the editor portion of preview app<\/a>.\n<\/p>\n \n Now that we understand what the image editing endpoint requires for inputs, let\u2019s try to build an Android app that targets this endpoint. Our end goal will be to create a camera filter that lets users capture and edit their photos with OpenAI.\n<\/p>\n \n We\u2019ll need to write code to:\n<\/p>\n \n There are multiple ways you can do this in Android, but today we\u2019ll be using CameraX. CameraX is a Jetpack library designed to make camera app development easier, with lots of documentation<\/a>, samples<\/a>, and codelabs<\/a> available for use.\n<\/p>\n \n To capture images with CameraX, you must:\n<\/p>\n \n For more detailed implementation information, check out the documentation on image capture with CameraX<\/a>.\n<\/p>\n \n The next step is to let users apply a transparent mask to part of the captured image, which will allow OpenAI to fill in the masked area based on the text prompt.\n<\/p>\n \n We can do this by creating a simple custom view that overrides the \n Once the user is satisfied with the mask they\u2019ve drawn, then all that\u2019s left to do is combine the original image with the mask drawing. We can accomplish this with the help of the Finally, once we\u2019ve processed the masked image, and collected additional user input for the text prompt, we just need to build an OpenAI request to the image editing endpoint. If you need help getting started with OpenAI API keys and other setup, please refer to our previous blog post<\/a>.\n<\/p>\n \n As described in the API reference<\/a>, the image editing endpoint accepts 2-7 fields in the request body:\n<\/p>\n \n All of the default values work well for our purposes, so we\u2019ll only need to supply the \n This code snippet shows how you can use the OkHttp library<\/a> to build a multipart request with a masked image bitmap and prompt string:\n<\/p>\n \n The image edits endpoint response will send back \n And just like that, you have your edited image! You can display this in your app with coil\u2019s AsyncImage<\/a> if you\u2019re using Jetpack Compose, or open an \n Here are some examples of the image editing camera filter in action:\n<\/p>\nEdits<\/code> API. Today, we\u2019ll be focusing more on the Images<\/code> API and how you can set up some interesting interactions in your Android apps, such as a camera filter with editing capabilities.\n<\/p>\nImages API overview<\/h2>\n
\n
Images<\/code> API, you can check out the DALL-E preview app<\/a> and provide some of your own prompts. You can describe image style and content with lots of details, or provide more abstract concepts to see what the model will come up with! The preview app shows examples of prompts and generated images to give you an idea of where to start \u2013 here are some image prompts we tested out ourselves:\n<\/p>\n![\"four](\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/04\/image1.jpg\")
<\/a><\/p>\nIntegrating image editing into an Android app<\/h2>\n
\n
Capture images<\/h3>\n
\n
android.hardware.camera.any<\/code>, android.permission.CAMERA<\/code>, android.permission.RECORD_AUDIO<\/code> and android.permission.WRITE_EXTERNAL_STORAGE<\/code>\n <\/li>\nPreviewView<\/code> so users can see a preview of their photo \u2013 this requires setting up a ProcessCameraProvider<\/code>, a CameraSelector<\/code>, and a Preview<\/code> object with a SurfaceProvider<\/code>\n <\/li>\ntakePhoto<\/code> with the appropriate image file metadata\n <\/li>\n<\/ul>\nCreate image mask<\/h3>\n
onDraw<\/code> and onTouchEvent<\/code> methods to let the user color in the desired mask area. To learn more about setting up custom views for drawing input, check out the documentation on custom drawing with Canvas<\/a> and tracking touch input<\/a>.\n<\/p>\nPorterDuff.Mode.SRC_OUT<\/code><\/a>, which will delete pixels from the original image wherever the mask overlaps it.\n<\/p>\n\/\/ customView \u2013 reference to custom view that tracks user input\r\n\/\/ imageBitmap \u2013 reference to (cropped) original image bitmap\r\n\/\/ extract masked area from custom view\r\nval maskBitmap = Bitmap.createBitmap(imageBitmap.width, imageBitmap.height, Bitmap.Config.ARGB_8888)\r\nval canvas = Canvas(maskBitmap)\r\ncustomView?.draw(canvas)\r\n\r\n\/\/ combine mask with original image\r\nval paint = Paint()\r\npaint.xfermode = PorterDuffXfermode(PorterDuff.Mode.SRC_OUT)\r\n\r\ncanvas.drawBitmap(imageBitmap, 0f, 0f, paint)<\/pre>\n
\n Build OpenAI request\n<\/h2>\n
\n
256x256<\/code>, 512x512<\/code>, or 1024x1024<\/code> (default)\n <\/li>\nurl<\/code> (default) or b64_json<\/code>\n <\/li>\nimage<\/code> and prompt<\/code> fields when building our request. Since an image file is included in the request, we will need to build a multipart request.\n<\/p>\nval maskedImage = ByteArrayOutputStream().use {\r\n maskBitmap.compress(Bitmap.CompressFormat.PNG, 100, it)\r\n it.toByteArray().toRequestBody(\"image\/png\".toMediaType())\r\n}\r\n\r\nval requestBody = MultipartBody.Builder()\r\n .setType(MultipartBody.FORM)\r\n .addFormDataPart(\"prompt\", prompt)\r\n .addFormDataPart(\"image\", \"image.png\", maskedImage)\r\n .build()\r\n\r\nval request = Request.Builder()\r\n .url(\"https:\/\/api.openai.com\/v1\/images\/edits\")\r\n .addHeader(\"authorization\", \"Bearer ${OPENAI_KEY}\")\r\n .post(requestBody)\r\n .build()\r\n\r\nval client = OkHttpClient.Builder().build()\r\nval response = client.newCall(request).execute()<\/pre>\nn<\/code> images in the specified response_format<\/code>. It\u2019s not uncommon for the requests to take 5-10 seconds, so don\u2019t worry if it takes a bit. Since we used the default values for the request, we\u2019ll only be getting back one image in url format. To parse this from the response body, we can add just a few simple lines of code:\n<\/p>\nval jsonContent = response.body?.string() ?: \"\"\r\nval data = Json.parseToJsonElement(jsonContent).jsonObject[\"data\"]?.jsonArray\r\nval imageUrl = data?.map { it.jsonObject[\"url\"] }?.get(0).toString()<\/pre>\nInputStream<\/code> to download the image bitmap from the url \u2013 depending on which method you choose, you may also need to add the android.permission.INTERNET<\/code> permission to your Android manifest. \n<\/p>\n![\"Four](\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/04\/image2.jpg\")
<\/a><\/p>\n![\"Two](\"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-content\/uploads\/sites\/53\/2023\/04\/image3.jpg\")
<\/a><\/p>\nAdditional thoughts on crafting prompts<\/h2>\n