Using a Private Ethereum Consortium Network to Store and Validate Documents

Ami Turgman

This blog post co-authored with Rabeea Abu-Saleh

Background

Food retailers in Germany are required to prove that their food is below thresholds for regulated substances, such as pesticides. Currently, producers must send their probes for these substances to laboratories and only receive their results days later. In the meantime, they must wait to find out whether they should harvest their produce. Additionally, distributors and retailers have no proof that the paper documenting the laboratory results is actually based on a sample from the producer’s own farm or that the laboratory is the one who produced it.

Recent incidents where pesticide residue was inadequately identified in vegetables by major supermarket chains have heightened consumer awareness of food production. Changes in legislation have also impacted the demand for food safety assurance along the entire supply chain. As a result, there is a need for more cost-effective procedures for detecting environmental contaminants in the food industry.

The Problem

In this supply chain flow, there is a need to be able to track the origin of a specific set of produce and retrieve its entire history, like the levels of pesticides measured before harvest. In addition, since part of the data is considered to be private, there is a need to protect and manage access to this sensitive data.

As a solution to these problems, iBERA, a startup that operates in the biosensors domain, developed a device that can be used by the farmers to test produce and receive instant results. We recently worked with iBERA to create a blockchain-based solution for storing, tracking, providing and validating these pesticide level measurements.

The Solution

The iBERA blockchain ecosystem approach we developed enables seamless sharing of data along the supply chain. In the first stage, we concentrated on providing secure access to quality related data (for example, laboratory analysis documents) and immutable storage of this data for the whole food supply chain. In further development stages, other types of information will be added, along with process automation functionality with smart contracts (e.g., automated payment procedures, triggering of transactions after passing specific quality gates).  This distributed ledger system has complete interoperability with all kinds of existing legacy systems in the supply chain by operating with specific APIs and connectors. Our approach creates a streamlined, visible and verified data flow between all actors.

We implemented two separate scenarios:

Supplementing the existing process

In this scenario, we support the existing flow in which the farmer sends a sample to a remote lab. The lab then sends the results back to the producer on an email as an attachment. We implemented a Microsoft Outlook add-in that can be deployed on any of the participants’ Outlook clients. Using this add-in, the producer can choose to store the document.

Behind the scenes, we store the document off-chain, on blob storage, while keeping the document’s hash on the blockchain; we refer to this data as the document’s proof. When the retailer asks for the document, it will receive both the original document as an attachment and a reference to the document’s proof that was stored on the blockchain. The retailer can validate that the document wasn’t tampered by calculating the hash and comparing it to the hash stored on the blockchain.

Note: It would be better if the lab could store the documents on the blockchain directly in the first place, but since this required changes in the existing flow like the need to work with another partner, we have provided the ability for the producer to store it themselves. The intention is that when more participants, like the lab, are onboarded to the consortium network, they will interact with the blockchain directly.

Integrating iBERA’s mobile app with the blockchain

In this scenario, a mobile app running on an Android device gets the measurements from the device and sends the data to the blockchain using iBERA’s REST API. Since the data that is uploaded is relatively small, we keep it on the blockchain as part of the proof. Since the data is already stored on-chain, there’s no need to validate the data as we did in the previous scenario because it can’t be tampered with anyway.

The following is a high-level diagram of the supply chain scenario, as more participants join the network:

In our solution, iBERA represents all of the producers who work directly with its API and iBERA is a central entity that all producers trust. We leverage the distributed, immutable ledger of blockchain technology when interacting with different entities in a larger supply chain scenario. iBERA is the first participant in the network that we expect other food retailers, distributors, regulators, and more to join.

Architecture

We leveraged the existing Ethereum Consortium ARM template and used it as a baseline for our deployment scripts, adding the rest of the components as described in the following architecture diagram.

Outlook Add-in

As previously described, the Outlook add-in allows users to add manage attachments by adding them to the blockchain and providing the attachment and the proof to anyone who asks for it. It validates that the attachment is authentic by calculating the hash and comparing it to the hash that was stored on the blockchain.

The add-in includes 3 operations:

Store Attachments

When applying this operation to an email with attachments, this operation stores the attached files to Azure storage (as blobs), and creates a proof for every attachment that is stored on the blockchain, using the Service API.

The proof is JSON in the following structure:

{
  "trackingId": “<tracking-id>”,
  "owner": "<owner-id>",
  "encryptedProof": {
    "blobName": "<random-inique-id>/<file-name>"
  },
  "publicProof": {
    "documentHash": "<hash_value>",
    "encryptedProofHash": "<hash_value>"
  },
  "previousTrackingId": "<prev-tracking-id>"
}

The result of this operation is a new email that includes an array of the tracking IDs of the created proofs.

Provide Proof

This operation creates a new reply email that includes:

• Proof from the blockchain, according to the tracking IDs from the original email’s body
• Relevant files (documents) of the proofs, as attachments

Currently, if there is more than one tracking ID in the email body, this operation will only get the proof and file of the first tracking ID.

Validate Proof

When applying this operation to an email that includes an attachment and a proof, the solution calculates the hash for the attachment and compares it with the document hash value from the email body. In addition, it verifies that the hash is equal to the one that was stored on the blockchain. If all are equal, it means that the document is valid.

Outlook Integration API

The backend logic of the Outlook add-in is hosted in the Outlook Integration API service. It uses a dedicated Azure Storage Account for storing the attachments and works against Microsoft Exchange APIs to download the attachments. In addition, it calculates the hash of the documents before storing them on the blockchain.

To perform all the blockchain-related operations (store, validate and get proofs), the Outlook Integration API works directly with the Service API.

Service API

The Service web app is the main entry point to the private Ethereum Consortium Network. It is responsible for managing all the operations with the blockchain network like storing and retrieving proofs from the blockchain. Its API can be accessed via the Outlook add-in (through the Outlook Integration web app), or directly from any other web client (Client Authentication was not implemented as part of this solution).

The Services API works against a smart contract that’s deployed on the private Ethereum network. The smart contract provides two main APIs for storing and retrieving proofs on/from the ledger.

The main operations in this web app are:

[PUT] /api/proof

• Stores a new proof on the blockchain network.
• The request must include a proof in the body. The result of this operation will be the generated tracking ID (if it wasn’t provided as part of the request) and the Ethereum transaction’s hash.

[GET] /api/proof/<tracking-id>

• Retrieves a proof from the blockchain network according to a provided tracking ID.

As part of storing the proofs on the blockchain, the Services API creates a dedicated encryption key per proof to encrypt its private content. It uses a dedicated Azure Storage Account to manage those keys. When a user requests a proof, the Service APIs get the proof from the blockchain and decrypts the private content only if the user that requested the proof was the one who added it to the blockchain. Others will be able to get the proof, but will not be able to access the private data that was encrypted.

Private Ethereum Consortium Network

The Ethereum Consortium is a fully operational, multi-member, private Ethereum network. It consists of a set of load-balanced transaction nodes with which an application or a user can interact to submit transactions. In addition, it consists of a set of mining nodes that do the actual processing of the transaction.

The number and the VM sizes of the transactions and the mining nodes can be configured before running the deployment script. The deployment script deploys the smart contract right after the deployment of the Ethereum Consortium is completed.

The code

The code is referenced under the main repository, and is separated into 3 different repositories on Github:

• Outlook Add-In
• Services API
• Smart Contract

Please refer to the repositories for more details and explanations of how to deploy the solution.

Options for Reuse

Our solution can be reused by anyone who is looking to implement scenarios similar to iBERA’s supply chain validation scenario we have discussed.

Our solution can also be used as a basis for a similar architecture where there are services that interact with a Private Ethereum Consortium Network and Smart Contracts deployment. You can start by deploying this solution which will save you months of work (if you were to develop it yourself from scratch), then modify it according to your own scenario/needs.

We also encourage you to contribute to this code by adding more functionality and submitting pull requests. You can also reach out to us in the comments below.

Contributors

Header Image: Blockchain, used under CC By 2.0 Creative Commons

Ami Turgman

Follow