ISE Developer Blog
Solving global tech challenges, sharing insights, and empowering developers
Latest posts
Running Parallel Apache Spark Notebook Workloads On Azure Databricks
This article walks through the development of a technique for running Spark jobs in parallel on Azure Databricks. The technique enabled us to reduce the processing times for JetBlue's reporting threefold while keeping the business logic implementation straight forward. The technique can be re-used for any notebooks-based Spark workload on Azure Databricks.
Developing and Deploying a Churn Prediction Model with Azure Machine Learning Services
Deep learning algorithms capable of learning and predicting customer behavior are allowing businesses to intervene with the right retention offers at the right time. CSE recently partnered with Majid Al Futtaim Ventures (MAF) to design and deploy a machine learning solution to predict attrition.
Real-Time Time Series Analysis at Scale for Trending Topics Detection
This code story describes a collaboration with ZenCity around detecting trending topics at scale. We discuss the datasets, data preparation, models used and the deployment story for this scenario.
Improving Safety and Efficiency in BMW Manufacturing Plants with an Open Source Platform for Managing Inventory Delivery
Over the course of twelve months Microsoft and BMW partnered three different times to help BMW with its vision for technical transformation. An open-source package called ROS-Industrial was used to help provide the building blocks for the robotics work.
Social Stream Pipeline on Databricks with auto-scaling and CI/CD using Travis
This code story describes CSE's work with ZenCity to create a data pipeline on Azure Databricks supported by a CI/CD pipeline on TravisCI. The aim of the collaboration was to create a pipeline capable of processing a stream of social posts, analyzing them, and identifying trends.
Active Learning for Object Detection in Partnership with Conservation Metrics
CSE teamed up with Conservation Metrics to use Active Learning to allow for more efficient data labeling for object detection projects.
How to Build A K8S Http API For Helm, and Serve Micro-services Using A Single IP
The Commercial Software Engineering team (CSE) partnered with Axonize to automate the process of deploying apps to Kubernetes, and expose these apps to the internet via a single IP. This post is about enabling applications in your Kubernetes cluster to programmatically install helm charts and expose them through a single public facing IP.
Attaching and Detaching an Edge Node From a HDInsight Spark Cluster when running Dataiku Data Science Studio (DSS)
Earlier this year, Dataiku and Microsoft joined forces to add extra flexibility to DSS on HDInsight, and also to allow Dataiku customers to attach a persistent edge node on an HDInsight cluster – something which was previously not a feature supported by the most recent edition of Azure HDInsight.
Infrastructure as Code – On demand GPU clusters with Terraform & Jenkins
Developing robust algorithms for self-driving cars requires sourcing event data from over 10 billion hours of recorded driving time. CSE worked with Cognata, a startup developing simulation platforms for autonomous vehicles, to build a Jenkins pipeline and Terraform solution that enabled our partner to dynamically scale GPU resources for their simulations.