Microservices and Docker containers: Architecture, Patterns and Development guidance (Updated for .NET Core 2.0)

Updated for .NET Core 2.0 “wave” of technologies (Nov. 15th 2017)

Earlier this year, we published this eBook and sample application offering guidance for architecting microservices and Docker containers based applications.

We have recently published updates to the eBook (2nd edition) and sample application to bring them in line with the latest releases of .NET Core 2.0 many other updates coming along as part of the same “wave” of technologies.

This is the update list summary for both, the eBook and the sample application:

Use of .NET Core 2.0 and .NET Standard Library 2.0
Use of Entity Framework Core 2.0 with new features like owned entities for Value Object pattern implementation
Use of ASP.NET Core 2.0 Docker multi-arch images in dockerfiles (targeting Linux and Windows Containers)
- Support in docker-compose deployment files for Docker Linux and Windows containers
Updated code for logging and configuration in ASP.NET Core 2.0
Use of the new IHostedService for the OrderGracePeriod background task
Aligned to the new MVC and SPA-Angular templates in Visual Studio 2017 targeting ASP.NET Core 2.0
Migrated to ASP.NET Core Identity 2.0
Idempotent commands implementation
Use of MediatR 3 with Behaviors instead of Decorators in addition to Domain Events
Use of Azure Application Insights for monitoring the microservices/containers.
Use of NoSql database for a new added microservice supporting MongoDB container and Azure Cosmos DB deployment
- In addition, eShopOnContainers uses SQL Server and Redis with local container deployment or in Azure PaaS services.
Support for Kubernetes deployment in Azure (Linux containers deployment)
Support for Service Fabric deployment in Azure (Linux and Windows containers deployment)
Event Bus and Integration Events supporting local RabbitMQ container or Azure Service Bus implementations
NServiceBus and Sagas support with forked version of eShopOnContainers for NServiceBus.
- Check the generated integration event visualizations from Particular ServiceInsight based on runtime information.
- Kudos to folks from Particular Software for implementing this additional forked version!

These are just the new additions for the .NET Core 2.0 “wave”. For a full list of implemented features and technologies used at eShopOnContainers see this Roadmap and features Wiki post.

Introduction

As part of the series of posts announced at this initial blog post (.NET Application Architecture Guidance) that explores each of the architecture areas currently covered by our team, this current blog post focuses on “Microservices and Docker containers: Architecture, Patterns and Development guidance”.

Just as a reminder, the four introductory blog posts of this series will be the following:

The microservices architecture is emerging as an important approach for distributed mission-critical applications. In a microservice-based architecture, the application is built on a collection of services that can be developed, tested, deployed, and versioned independently. In addition, enterprises are increasingly realizing cost savings, solving deployment problems, and improving DevOps and production operations by using containers (Docker engine based as de facto standard).

Microsoft has been releasing container innovations for Windows and Linux by creating products like Azure Container Service (AKS) and Azure Service Fabric, and by partnering with industry leaders like Docker, Mesosphere, and Kubernetes. These products deliver container solutions that help companies build and deploy applications at cloud speed and scale, whatever their choice of platform or tools.

As mentioned, Docker Containers are becoming the de facto standard in the container industry, supported by the most significant vendors in the Windows and Linux ecosystems. (Microsoft is one of the main cloud vendors supporting Docker.) In the future, Docker Containers will probably be ubiquitous in any datacenter in the cloud or on-premises.

Keeping this in mind, we have introduced two guides/ebooks in this space:

• ‘.NET Microservices – Architecture for Containerized .NET Applications’ guide/eBook

• ‘Containerized Docker Application Lifecycle with Microsoft Platform and Tools’ guide/eBook

Alright, lets now dive deeper about what each of these guides/ebooks offer for you today!

Introducing the ‘.NET Microservices – Architecture for Containerized .NET Applications’ Guide/eBook

You can browse online pages or download the full eBook (.PDF) of the first edition of this eBook that offers guidance on those mentioned subjects and from our perspective from the .NET team: .NET Microservices: Architecture for Containerized .NET Applications. Available as a free .PDF eBook download:

http://aka.ms/MicroservicesEbook

Also available as online content at docs.microsoft.com:

https://aka.ms/microservices-guide-online-msft-docs

As highlighted and in addition to the .PDF eBook, this guide is also available as online version (online web pages) at docs.microsoft.com, so you can directly go to specific pages of the content, as a reference. The best thing about this approach is that you can comment on every page and since it is based on GitHub, you can even submit Pull Requests with content updates that we can approve and merge.

This guide is an introduction to developing microservices-based applications and managing them using containers. It discusses architectural design and implementation approaches using .NET Core and Docker containers. To make it easier to get started with containers and microservices, the guide focuses on a reference containerized and microservice-based application that you can explore. The sample application is available at the eShopOnContainers GitHub repo. It provides foundational development and architectural guidance primarily at a development environment level with a focus on two main technologies: Docker and .NET Core. Our intention is that you read this guide when thinking about your application design without focusing on the infrastructure (cloud or on-premises) of your production environment. You will make decisions about your infrastructure later when you create your production-ready applications. Therefore, this guide is intended to be cloud infrastructure agnostic and more development-environment-centric.

Who should use this guide

We wrote this guide for developers and solution architects who are new to Docker-based application development and to microservices-based architecture and also for someone who knows what microservices are, but wants to know how to do it (or do it better) on .NET. This guide is for you if you want to learn how to architect, design, and implement proof-of-concept applications with Microsoft development technologies (with special focus on .NET Core) and with Docker containers. You will also find this guide useful if you are a technical decision maker, such as an enterprise architect, who wants an architecture and technology overview before you decide on what approach to select for new and modern distributed applications.

How to use this guide

The first part of this guide introduces Docker containers, discusses how to choose between .NET Core and the .NET Framework as a development framework, and provides an overview of microservices. This content is for architects and technical decision makers who want an overview but who do not need to focus on code implementation details. The second part of the guide starts with the “Development process for Docker based applications” section. It focuses on development and microservice patterns for implementing applications using .NET Core and Docker. This section will be of most interest to developers and architects who want to focus on code and on patterns and implementation details.

Related microservice and container-based reference application: eShopOnContainers

The eShopOnContainers application (still in BETA state) is a sample reference app for .NET Core and microservices that is designed to be deployed using Docker containers. The application consists of multiple subsystems, including several e-store UI front ends (a Web app and a native mobile app). It also includes the back-end microservices and containers for all required server-side operations.

This microservice and container-based application source code is open source and available at the eShopOnContainers GitHub repo.

The following diagram shows a high-level overview of the architecture of the sample application:

The sample application includes the following backend services:

• An identity microservice, which uses ASP.NET Core Identity and IdentityServer4.

• A catalog microservice, which is a data-driven, create, read, update, delete (CRUD) service that consumes an SQL Server database using EntityFramework Core.

• An ordering microservice, which is a domain-driven service that implements domain driven design patterns.

• A basket microservice, which is a data-driven CRUD service that uses Redis Cache

• A Marketing and Location microservices as example of No-SQL databases (MongoDB and CosmosDB).

• An MVC watchdogs web app to query and show info from the HealthChecks (not shown in the above architecture diagram).

These backend services are implemented as microservices using ASP.NET Core Web API, and are deployed as unique containers within a single Docker host. Client apps then communicate with the backend services through a Representational State Transfer (REST) web interface.

The microservices are also communicated between them with asynchronous messages based on an Event Bus currently implemented on top of RabbitMQ, but the event bus could also be based on any other broker or service bus like Azure Service Bus, NServiceBus, MassTransit, etc.

In addition, the sample application ships with three client apps:

• An MVC application developed with ASP.NET Core.

• A Single Page Application (SPA) developed with Angular 2 and Typescript.

• A cross-platform mobile app developed with Xamarin.Forms.

Key takeaways about Microservices and Containers

As a summary, the following are the most important conclusions from this guide.

Benefits of using containers. Container-based solutions provide the important benefit of cost savings because containers are a solution to deployment problems caused by the lack of dependencies in production environments. Containers significantly improve DevOps and production operations.

Containers will be ubiquitous. Docker-based containers are becoming the de facto standard in the container industry, supported by the most significant vendors in the Windows and Linux ecosystems. This includes Microsoft, Amazon AWS, Google, and IBM. In the near future, Docker will probably be ubiquitous in both cloud and on-premises datacenters.

Containers as unit of deployment. A Docker container is becoming the standard unit of deployment for any server-based application or service.

Microservices. The microservices architecture is becoming the preferred approach for distributed and large or complex mission-critical applications based on multiple independent subsystems in the form of autonomous services. In a microservice-based architecture, the application is built as a collection of services that can be developed, tested, versioned, deployed, and scaled independently; this can include any related autonomous database.

Domain-driven design and SOA. The microservices architecture patterns derive from service-oriented architecture (SOA) and domain-driven design (DDD). When you design and develop microservices for environments with evolving business rules shaping a particular domain, it is important to take into account DDD approaches and patterns, like the Bounded Context pattern.

Microservices challenges. Microservices offer many powerful capabilities, like independent deployment, strong subsystem boundaries, and technology diversity. However, they also raise many new challenges related to distributed application development, such as fragmented and independent data models, resilient communication between microservices, eventual consistency, and operational complexity that results from aggregating logging and monitoring information from multiple microservices. These aspects introduce a higher level of complexity than a traditional monolithic application. As a result, only specific scenarios are suitable for microservice-based applications. These include large and complex applications with multiple evolving subsystems; in these cases, it is worth investing in a more complex software architecture, because it will provide better long-term agility and application maintenance.

Containers for any application. Containers are convenient for microservices, but are not exclusive for them. Containers can also be used with monolithic applications, including legacy applications based on the traditional .NET Framework and modernized through Windows Containers. The benefits of using Docker, such as solving many deployment-to-production issues and providing state of the art Dev and Test environments, apply to many different types of applications.

Orchestrators. Container-based orchestrators like the ones provided in Azure Container Service (Kubernetes, Mesos DC/OS, and Docker Swarm) and Azure Service Fabric are indispensable for any production-ready microservice-based and for any multi-container application with significant complexity, scalability needs, and constant evolution. This guide has introduced orchestrators and their role in microservice-based and container-based solutions. If your application needs are moving you toward complex containerized apps, you will find it useful to seek out additional resources for learning more about orchestrators.

Introducing the ‘Containerized Docker Application Lifecycle with Microsoft Platform and Tools’ Guide/eBook

This guide targets a different profile, not just developers, but also IT professionals, IT managers and enterprise developers who would like to know about what technologies and Tools Microsoft offers when working with containers and what products you can use to implement an end-to-end DevOps application lifecycle when using Docker, you can also download a shorter and higher-level eBook, available here:

https://aka.ms/DockerLifeCycleeBook

Visit the .NET Application Architecture Center and grab it all!

Remember that this ‘Microservices Architecture and Containers guidance‘ explained at this blog post is part of a broader .NET Architecture guidance which covers additional subjects and you can check out at the .NET Application Architecture Center page where you can download the multiple eBooks/Guides and visit the reference applications from there, and of course, feel free to provide feedback by dropping a note below or on the feedback form at the architecture page.

Happy coding from the .NET team!

Cesar de la Torre

.NET Product Group

Twitter: @cesardelatorre

Microservices and Docker containers: Architecture, Patterns and Development guidance (Updated for .NET Core 2.0)