Converting multiple sources to FHIR using Liquid

Problem Statement

In our recent engagement, we were working to solve a data integration scenario for our customer. The patient data is collated in different upstream systems with their own format and protocols. We had to work on unifying the data across these different sources to build a single view of a patient for health care services. We decided to represent this single patient view in FHIR format which is a global standard of data modelling in the healthcare industry. Our next challenge was to find a simple way to convert different formats into FHIR. Whilst exploring different options, we decided to use Liquid templates to enable us to convert varied formats into standard FHIR data model.

We referenced the Azure Health Data Services architecture guide and health architectures which provided us with the pattern to ingest data in multiple formats into one standard. These patterns leverage Liquid templates as a mechanism for mapping those formats into the aforementioned standard.

Liquid templates are a powerful tool for dynamic content rendering in various contexts. In this blog post, we’ll dive into the fundamentals of Liquid templates, and how we used them to convert multiple data formats into a standard unified model.

Terminologies

Before we move ahead in this post, we’ll define some terms used in this blog.

HL7v2 (Health Level Seven version 2): is a messaging standard used for transmitting various types of healthcare data, such as patient demographics, lab results, and clinical orders. It employs a structured message format based on segments and fields, allowing systems to communicate in a standardized manner.

FHIR (Fast Healthcare Interoperability Resources): is a newer standard that uses modern web technologies like JSON and XML to represent healthcare data in a more flexible and interoperable way. FHIR resources encapsulate discrete units of healthcare information, such as patients, observations, and medications, making it easier to exchange and consume data across different systems.

What is a Liquid Template?

Let’s familiarize ourselves with Liquid templates and their working before proceeding. Liquid is a lightweight templating language designed to inject dynamic content into static templates. It originated in the Shopify ecosystem but has gained popularity beyond e-commerce platforms. Here are the key components of Liquid:

Objects

Objects contain the data that Liquid will display on a page or inside a component. Objects and variables are enclosed in {{ ... }} e.g. if the person Bruce Wayne is represented as an object and has “Bruce” as firstName and “Wayne” as lastName, this is how the below example would work.

Input:

{{ person.firstName }}

Output:

Bruce

Filters

Filters modify or transform the output data before rendering. They are used in a similar fashion to Objects within curly braces {{ }} and can be separated by pipe character |. For example to append a string, we could do:

Input

{{ "Bruce"  | append "Wayne" }}

Output:

Bruce Wayne

Multiple filters can be daisy-chained as well. For example:

Input

{{ "bruce wayne" | capitalize | prepend: "Hello " }}

Output:

Hello BRUCE WAYNE

A detailed list of filters can be found on the Shopify liquid documentation.

How we used Liquid

To address the problem statement at the start of this post, we transformed patient records and data from various formats into the UK FHIR standard. We utilized Liquid templates to map diverse data types into a consistent FHIR format.

We utilized two in-built functionalities of FHIR on Azure Health Data Services to convert and validate input data.

convert-data: By using the $convert-data operation in the FHIR service, you can leverage Liquid templates to convert health data from various formats to FHIR.
validate: $validate is an operation in FHIR, that allows you to ensure that a FHIR resource conforms to a specified profile.

Now, we’ll show how we mapped CSV and Hl7v2 to FHIR format.

Converting Patient data in CSV to FHIR format

As part of the integration of the solution, we received CSV files with patients data from services like personal demographic and organisations data.

Creating Liquid template

The first step was to create a Liquid template to convert incoming patient data into a FHIR compliant Patient resource. The Liquid template below will output JSON to represent FHIR resource Bundle. It will iterate over the list of patients passed to the template and create an array of entry objects. We will use for loop of Liquid for the iteration.

{
  "resourceType": "Bundle",
  "type": "transaction",
  "entry": [
  {% for item in msg.patients %}
    {
      "resource": {
        "resourceType": "Patient",
        "id": "{{ item.Id }}",
        "meta": {
          "source": "Organization/{{ item.OrganizationCode }}",
          "profile": [
            "https://fhir.hl7.org.uk/StructureDefinition/UKCore-Patient"
          ]
        },
        "identifier": [
          {
            "system": "https://fhir.xxx.uk/Id/national-health-number",
            "value": "{{ item.NationalHealthNumber }}"
          }
        ],
        "name": [
          {
            "family": "{{ item.FamilyName }}",  
            "given": ["{{ item.GivenName }}", "{{ item.OtherGivenName }}"]
          }
        ],
        "gender": {% case item.Gender %}  
          {% when '0' %}  
          "unknown"  
          {% when '1' %}  
          "male"  
          {% when '2' %}  
          "female"  
          {% else %}  
          "unknown"  
          {% endcase %},
        "telecom": [
        {% if item.TelephoneNumber != "" %}  
          {   
            "system": "phone",  
            "value": "{{ item.TelephoneNumber }}",  
            "use": "home"  
         },  
        {% endif %}
        ]
      }
    }
    {% unless forloop.last %},{% endunless %}  
    {% endfor %}
  ]
}

The below JSON content was generated after the Liquid template was executed.

{
  "resourceType": "Bundle",
  "type": "transaction",
  "entry": [
    {
      "resource": {
        "resourceType": "Patient",
        "id": "REDACTED",
        "meta": {
          "source": "Organization/REDACTED",
          "profile": [
            "https://fhir.hl7.org.uk/StructureDefinition/UKCore-Patient"
          ]
        },
        "identifier": [
          {
            "system": "https://fhir.xxx.uk/Id/national-health-number",
            "value": "REDACTED"
          }
        ],
        "name": [
        {
          "family": "REDACTED",  
          "given": "REDACTED"
        }
        ],
        "gender": "REDACTED",
        "telecom": [
          {
            "system": "phone",
            "value": "REDACTED",
            "use": "home"
          }
        ]
      }
    }
  ]
}

Please note that the term “REDACTED” is used here to indicate where PII has been removed. In practice, you may choose to replace these with anonymized identifiers or remove the fields entirely, depending on the requirements of your data handling policy.

Calling the Liquid template from C# code

To create the FHIR JSON from Liquid template, we used the below C# code. We have presented a skeleton of the actual code here to ease the focus on the conversion. We will use the convert-data endpoint and ValidateData endpoint of FHIR Service on Azure Health Data Services.


// This function consumes the CSV content, converts into json and uses FHIR functions to use Liquid template for final Patient representation.

public async Task ConsumeCsvContentAndCreateEntriesInFHIR(string csvSourceData) 
{
    var patientsData = ConvertFromCsvToJson(csvSourceData); // the skeletal implementation of Convert is given below
    var fhirCallResponse = await CallFHIRConvertAndUpdateResource(patientsData); // the skeletal implementation of Convert is given below
}

private string ConvertFromCsvToJson(string source)  
{   
    var csvContent = GetCsvContent(source);
    var records = ConvertCsvToRecords(csvContent);
    return JsonSerializer.Serialize(new { patients = records });  
}

private async Task<Result> CallFHIRConvertAndUpdateResource(string patients)  
{  
    var conversionRequest = new ConvertDataRequest(patients, <LiquidTemplateForPatients>);  
    var fhirTransactionModel = fhirClient.ConvertData(conversionRequest);
    var validationResult = await fhirClient.ValidateData(fhirTransactionModel.Value);
    if (validationResult.isFailure) return validationResult; // we only proceed after validating that the converted values are as per the FHIR profile
    var transactionResult = fhirClient.TransactionAsync<Patient>(fhirTransactionModel.Value);
    return transactionResult;
}

Converting Patient data in Hl7v2 to FHIR format

We also integrated an upstream data source using Hl7v2 protocol. For this purpose of this post, we use the same Liquid template as defined above and used the below C# code to convert the Hl7v2 into FHIR.

Hl7v2 Input

A sample HL7v2 message to represent a patient could be represented as below:

MSH|^~\&|SendingApp|SendingFac|ReceivingApp|ReceivingFac|20240101000000||ADT^A01|HL7MSG00001|P|2.4
PID|1|123456789|10101010|Doe^John||19700101|M

Calling the Liquid template from C# code

We will use the convert-data endpoint and ValidateData endpoint of FHIR Service on Azure Health Data Services below to convert HL7v2 to FHIR.


public async Task<Result> Ingest(string hl7v2Message)  
{  

    var conversionRequest = new ConvertDataRequest(hl7v2Message, <LiquidTemplateForPatients>);
    var conversionResult = await fhirClient.ConvertData(conversionRequest);  
    var validationResult = await fhirClient.ValidateData(conversionResult.Value);
    if (validationResult.isFailure) return validationResult; // we only proceed after validating that the converted values are as per the FHIR profile
    var transactionResult = await fhirClient.TransactionAsync<Bundle>(conversionResult.Value);    
    return transactionResult;  
}

Output

The below JSON content was generated after the Liquid template executed

{
  "resourceType": "Bundle",
  "type": "transaction",
  "entry": [
    {
      "resource": {
        "resourceType": "Patient",
        "id": "123456789",
        "meta": {
          "source": "Organization/REDACTED",
          "profile": [
            "https://fhir.hl7.org.uk/StructureDefinition/UKCore-Patient"
          ]
        },
        "identifier": [
          {
            "system": "https://fhir.xxx.uk/Id/national-health-number",
            "value": "REDACTED"
          }
        ],
        "name": [
          {
            "family": "John",  
            "given": "Doe"
          }
        ],
        "gender": "M",
      }
    }
  ]
}

The above concludes the code sample and examples of how we used Liquid template to convert patients data in CSV and HL7v2 format into a standard FHIR format.

Why Are Liquid Templates a Good Idea?

Out of the Box Solution

Azure Health Data Services provide out of the box Liquid templates for conversion to FHIR resources from multiple sources.
This out of the box functionality saves a huge amount of development time.
More details and examples can be found on FHIR Service page on Azure Health Data Services documentation.

Flexibility and Reusability

Dynamic Content: Liquid templates enable us to provide data from various sources (e.g., databases, APIs) into the templates.
Reusable Components: By separating content from logic, we can reuse templates across different components or even projects. This modular approach simplifies maintenance and promotes consistency.

Portability

Platform-Agnostic: Liquid isn’t tied to a specific platform. We can use it in web applications, email marketing tools, or document generation pipelines.
Cross-System Compatibility: Since Liquid is widely supported, we can migrate templates between systems without major modifications.

Performance

Efficient Rendering: Liquid templates are precompiled, resulting in faster rendering times. The system evaluates placeholders only when needed, reducing unnecessary computations.
Lightweight
Since they have simple syntax and streamlined functionality, Liquid templates maintain a lean footprint while still empowering users to create dynamic content with ease.
This efficiency makes them ideal for applications where performance and simplicity are paramount.

Using VSCode Extensions for Liquid Templates

If you’re working with Liquid templates, we recommend using the Visual Studio Code (VSCode) extension to enhance your development experience. The extension provides extensive editing support like syntax highlighting, autocompletion etc.

Liquid VS Code Extension.

A detailed user guide for using VSCode Extension can be found on shopify VSCode extension page.

Acknowledgments

I’d like to thank all the folks on Microsoft teams and the customer teams, who contributed to developing the solution.

Converting multiple sources to FHIR using Liquid

Problem Statement

Terminologies

What is a Liquid Template?

Objects

Input:

Output:

Tags

Input:

Output:

Filters

Input

Output:

Input

Output:

How we used Liquid

Converting Patient data in CSV to FHIR format

Creating Liquid template

Calling the Liquid template from C# code

Converting Patient data in Hl7v2 to FHIR format

Hl7v2 Input

Calling the Liquid template from C# code

Output

Why Are Liquid Templates a Good Idea?

Out of the Box Solution

Flexibility and Reusability

Portability

Performance

Lightweight

Using VSCode Extensions for Liquid Templates

Acknowledgments

Author

Read next

Measuring Differentials of Product Images in AI-generated Ads

Semantic Router using Azure AI Search

Problem Statement

Terminologies

What is a Liquid Template?

Objects

Input:

Output:

Tags

Input:

Output:

Filters

Input

Output:

Input

Output:

How we used Liquid

Converting Patient data in CSV to FHIR format

Creating Liquid template

Calling the Liquid template from C# code

Converting Patient data in Hl7v2 to FHIR format

Hl7v2 Input

Calling the Liquid template from C# code

Output

Why Are Liquid Templates a Good Idea?

Out of the Box Solution

Flexibility and Reusability

Portability

Performance

Lightweight

Using VSCode Extensions for Liquid Templates

Acknowledgments

Author

Read next

Measuring Differentials of Product Images in AI-generated Ads

Semantic Router using Azure AI Search

Stay informed