This article walks through the development of a technique for running Spark jobs in parallel on Azure Databricks. The technique enabled us to reduce the processing times for JetBlue’s reporting threefold while keeping the business logic implementation straight forward. The technique can be re-used for any notebooks-based Spark workload on Azure Databricks.
Post by this author
In this article, we introduce a technique to rapidly pre-label training data for image segmentation models such that annotators no longer have to painstakingly hand-annotate every pixel of interest in an image. The approach is implemented in Python and OpenCV and extensible to any image segmentation task that aims to identify a subset of visually distinct pixels in an image.
This article describes how to use GitHub, Travis CI and Docker Compose to build a simple continuous delivery pipeline to deploy Linux Docker containers to a Service Fabric cluster of Linux hosts.
This article covers how to take a standard Python web service consisting of an application tier, a WSGI server, and a Nginx reverse proxy and deploy it via Linux containers to a Linux cluster managed by Azure Service Fabric using only simple tooling like Docker Compose.
This story covers how to get started with transfer-learning and build image classification models in Python with the Custom Vision Service. We compare the results with the popular Tensorflow-based models Inception and MobileNet.
The ability to correctly identify entities, such as places, people, and organizations, adds a powerful level of natural language understanding to applications. This post introduces a MIT-licensed one-click deployment to Azure for web services that lets developers get started with a wide range of natural language tasks in 5 minutes or less, by consuming simple HTTP services for language identification, tokenization, part-of-speech-tagging and named entity recognition.
This post describes in detail the Azure Cognitive Services speech-to-text WebSocket protocol and shows how to implement the protocol in Java. This enables us to transcribe audio to text in near real-time. We then show how to feed the transcribed radio into a pipeline based on Spark Streaming for further analysis, augmentation, and aggregation. The Java client is reusable across a wide range of text-to-speech scenarios that require time-efficient speech-to-text transcription in more than 10 languages including English, French, Spanish, German and Chinese.