Using a Layer 7 Proxy for Ethereum Blockchain Client’s JSON-RPC Endpoint
Webjet is a leading online international travel provider based in Melbourne, Australia. Over the past year, Webjet embarked on exploring and proving how to make their wholesale hotel booking processes more efficient using Blockchain with Ethereum and Smart Contracts. You can read more about the Webjet story here: Webjet and Microsoft build first-of-a-kind travel industry blockchain solution – Microsoft News Centre Australia.
This walk-through shows how, we validated API Management as a Layer 7 proxy in front of an Ethereum client using the Ethereum JSON-RPC JavaScript library web3.js for this project. In this scenario, utilizing Microsoft Azure API Management is necessary to ensure that only internal traffic from an Azure tenant via Virtual Networks (VNET) can access the Ethereum client’s JSON-RPC listening endpoint. In this example, the Ethereum client is Go Ethereum. Utilizing API Management provides additional risk mitigation features; however, this extra security also creates a blocker for the Geth JSON-RPC client libraries making calls that require custom HTTP Headers, such as the Ethereum JavaScript API. For addressing this blocker, a JavaScript library and NPM package was created. This library overrides the behavior of the web3.js HttpProvider, and permits injection of HTTP Headers that API Management can use for authorization and authentication of callers.
Typically, an Ethereum client’s JSON-RPC endpoint is unprotected and uses only HTTP with no option to validate callers of the endpoint or to use HTTPS (TLS). This approach represents a possible attack vector for abuse such as a denial-of-service (DoS), snooping, or worse. Some guidance exists from other Blockchain platforms, such as Bitcoin, to never use their version of JSON-RPC on the internet. The guidance from Ethereum is to ensure that the port is blocked from any internet usage. If there is a desire to present this endpoint either internally or externally, Azure API Management can assist in managing the service for security purposes, while adding other features such as analytics and tracking. Whether internal or external, this open port is an attack vector that needs to be part of a secure infrastructure design.
Azure API Management provides HTTPS for a secure channel; request authorization can leverage API Keys or OAuth tokens issued by Azure Active Directory. Azure API Management provides request throttling, traffic analysis, tracking, and other features that provide better insight into how the Ethereum JSON-RPC endpoint is being used and by whom. For runtime and developer diagnostics, Azure API Management has a trace option that helps identify transport issues between Azure API Management and the JSON-RPC endpoint. The goal of this solution is to ensure that only known callers are permitted to pass.
As part of this solution for diagnosing requests when calling the JSON-RPC endpoint, the NPM library discussed below adds additional logging capabilities to view the JSON-formatted Requests and Responses that are part of the Ethereum JSON-RPC API specification.
Blockchain in the Enterprise and Consortiums
Many organizations are testing and piloting Distributed Apps (DAPP) backed by Smart Contracts and Blockchain. Ethereum is one of the leading Smart Contract and Blockchain implementations available for Public, Consortium, and Private usage. While the Ethereum Organization provides several open source implementations, there are third-party options as well, such as Parity and BlockApps. Ethereum is utilized in Public usage as a permissionless blockchain, as well as within Consortiums as both a permissionless and permission-based one.
Blockchain is Growing and Disruptive
Blockchain has had a growth rate of 60+% and Bitcoin is the most well-known blockchain public platform. As the technology backing Bitcoin, blockchain is also the heart of the technology that is projected to disrupt areas including government records, banking, capital markets, renewable energy, and even digital identity and reputation.
Solution
The solution presented here is broken into two different parts. First is the Azure API Management configuration steps, and key configuration choices. Second is how to extend the web3.js JavaScript library in a manner that easily permits consumers to inject HTTP Headers that Azure API Management can use to identify callers.
Typical Deployment
Blockchain is a distributed ledger backed by technology that ensures integrity and non-repudiation through cryptographically ensured immutability of history. For blockchain to be viable, the nodes (in Ethereum they are called Ethereum Clients), are distributed among numerous entities throughout the world. The more independent entities running nodes, the greater the assurance that no single entity or group can control the blockchain and potentially rewrite history. Each node achieves consensus by interacting with peers in this mesh network. The process of validation involves ensuring that new transactions within blocks are correct both from a cryptographic perspective and from a state perspective, which can be affected by the execution of the logic contained within a Smart Contract that is part of a transaction.
Project Bletchley
Microsoft’s Project Bletchley provides templates that can deploy a multi-node, multiparty Ethereum cluster using the Ethereum Geth client into Azure. A view of what that might look like is below:
For a brief explanation of the annotations on the drawing, the “TN Subnet” (Transaction Subnet) are Geth client nodes that are NOT mining and have the JSON-RPC endpoint enabled to allow applications to make API calls. The diagram also shows SA which is an Azure Storage Account that backs the Virtual Machine. The other Subnet shown represents other “consortium” members whose Geth client nodes are mining. Mining is what processes transactions submitted to the network. Miners compete to package transactions into blocks that are then appended to the blockchain, and verified by other members of the network so that they earn Ether (the currency in Ethereum). On Private and Consortium networks, Ether may or may not be used as the unit of value, but Ether is still required to have transactions processed by the network with the current Ethereum Proof of Work model.
Ethereum Templates
A team within Microsoft is working with partners to build upon and extend the Project Bletchley templates as needed and provide feedback to the product team. Two examples are a multi-region consortium template and another that provides a choice and a mix of Ethereum Clients (for example, Geth and Parity).
Part 1 – API Management
For Azure API Management, one goal is providing a smart Layer 7 proxy that validates callers through HTTP headers, in addition to providing protection from anonymous callers, throttling, limits, analysis, and other features. Roughly, it looks like the following diagram:
API Management Authorization Features
Azure API Management can employ a simple token or API Key approach in addition to OAuth bearer tokens that can be granted by Azure Active Directory. These tokens are added to HTTP requests as the: Authorization : Bearer <token> header.
Azure API Management can also throttle based on various policies, as well as authorize only certain calls to specific clients, all through a configuration-oriented approach.
API Management Configuration
For this partner, Azure API Management is set to permit access to internal resources only. This control is done through the Virtual Network settings on the configuration blade. In the diagram below, if the Virtual Network is set to “Internal,” only clients from the same identified VNET and subnet, as shown in the drop-down, are permitted. From that VNET, any other traffic via gateways and VNET Peering is supported as well, given the proper configuration is in place.
From a logical perspective, the diagram below illustrates how API Management is configured and placed in the appservices/default virtual network and subnet. Once configured, the endpoint for Azure API management receives an IP address that is part of the subnet range; in this example, it is assigned 10.9.0.5. Note also that this IP address has no DNS name automatically created, so name resolution must be provided either through custom DNS or hosts files.
API Management Internal vs. External Accessibility
API Management can be created so it becomes a peer to the local private Virtual Network, as shown in the following screen shot:
API Management Endpoint Publication
The Ethereum JSON-RPC protocol uses only an HTTP POST verb, and the body contains JSON for requests and responses. The Geth client must be running internally with the --rpc --rpcaddr 0.0.0.0 command line switches, which enables the JSON-RPC endpoint. To create the configuration in Azure API Management, establish the API Settings as shown in the following section.
API Settings for Geth RPC
POST Operation for Geth
The following is the ONLY operation required for a proxy of the Geth client JSON-RPC endpoint. It must be a POST and the template is empty.
API Management Analytics
Part 2 – Alternative web3.js HttpProvider
In order to complete this solution, it is necessary to inject HTTP headers into requests from applications. For this requirement, when using the web3.js JavaScript library from Ethereum, an npm module was created to extend that library. This module provides a type that is used in place of HttpProvider from web3.js. Once in place, the constructor offers a headers parameter that is a key/value object that should have any headers required for Azure API Management to authorize requests.
Normally, the JSON-RPC endpoint on an Ethereum client provides no authentication or authorization boundary. All validation happens after the client receives the request and initially processes it. The only validation is if the transaction request is signed for sending requests. For local state calls, no verification is done whatsoever. These read-only requests are never blocked. With no protection at Layer 7, this method represents a potential DoS risk vector.
With this approach, by placing a Layer 7 proxy in front of an Ethereum client’s JSON-RPC endpoint, the ability to authorize and validate applications is possible. In addition, the proxy helps secure the channel with HTTPS.
Using the module
First, you need to install the npm package and then reference it from your project.
npm install --save httpheaderprovider
Adding to your web3.js project
In your JavaScript project, instead of creating an instance of the web3.providers.HttpProvider, create an instance of HttpHeaderProvider with the same parameters. However, add another parameter that contains key/value property object with the HTTP header name and value.
var Web3 = require(‘web3’);
var web3 = new Web3();
var HttpHeaderProvider = require(‘httpheaderprovider’);
var headers = {
“Ocp-Apim-Subscription-Key”: “mykeyfromtheapiportal”
}
var provider = new HttpHeaderProvider(‘https://scicoria.azure-api.net’, headers);
web3.setProvider(provider);
var coinbase = web3.eth.coinbase;
console.log(coinbase);
var balance = web3.eth.getBalance(coinbase);
console.log(balance.toString(10));
Debug Mode
The module supports emitting debug information by adding an environment variable before running:
# linux/macOS
export DEBUG=ethereumex:httpheaderprovider
# windows cmd
SET DEBUG=ethereumex:httpheaderprovider
# posh
$env:DEBUG="ethereumex:httpheaderprovider"
Debug messages
Once debug is enabled, the console shows messages similar to the following that include the request/response and the JSON-RPC message text.
talking to remote Geth on https://.azure-api.net/geth/
ethereumex:httpheaderprovider constructor:begin +0ms
ethereumex:httpheaderprovider constructor:end +2ms
ethereumex:httpheaderprovider Request: {"jsonrpc":"2.0","id":1,"method":"web3_clientVersion","params":[]} +1ms
ethereumex:httpheaderprovider prepareRequest:begin +1ms
ethereumex:httpheaderprovider setting headers +13ms
ethereumex:httpheaderprovider prepareRequest:end +0ms
ethereumex:httpheaderprovider Result: {"jsonrpc":"2.0","id":1,"result":"Geth/v1.5.4-stable-b70acf3c/linux/go1.7.3"} +371ms
ethereum client info: Geth/v1.5.4-stable-b70acf3c/linux/go1.7.3
ethereumex:httpheaderprovider Request: {"jsonrpc":"2.0","id":2,"method":"net_peerCount","params":[]} +1ms
ethereumex:httpheaderprovider prepareRequest:begin +0ms
ethereumex:httpheaderprovider setting headers +1ms
ethereumex:httpheaderprovider prepareRequest:end +0ms
ethereumex:httpheaderprovider Result: {"jsonrpc":"2.0","id":2,"result":"0x19"} +249ms
Peer Count: 25
ethereumex:httpheaderprovider Request: {"jsonrpc":"2.0","id":3,"method":"eth_coinbase","params":[]} +3ms
ethereumex:httpheaderprovider prepareRequest:begin +0ms
ethereumex:httpheaderprovider setting headers +0ms
ethereumex:httpheaderprovider prepareRequest:end +0ms
ethereumex:httpheaderprovider Result: {"jsonrpc":"2.0","id":3,"result":"0x13015840b5b4641f3ad441e36ec428d7a1c9934c"} +431ms
web3.eth.coinbase: 0x13015840b5b4641f3ad441e36ec428d7a1c9934c
ethereumex:httpheaderprovider Request: {"jsonrpc":"2.0","id":4,"method":"eth_getBalance","params":["0x13015840b5b4641f3ad441e36ec428d7a1c9934c","latest"]} +2ms
ethereumex:httpheaderprovider prepareRequest:begin +0ms
ethereumex:httpheaderprovider setting headers +0ms
ethereumex:httpheaderprovider prepareRequest:end +1ms
ethereumex:httpheaderprovider Result: {"jsonrpc":"2.0","id":4,"result":"0x8cf23f909c0fa000"} +277ms
web3.eth.getBalance(coinbase) 10156250000000000000
Approach
The module presented utilizes prototypical inheritance and creates a virtual function for HttpProvider.prepareRequest. The approach in the library uses pre-es2015 JavaScript. However, to make it clear what is being done, here is how it might look using a JavaScript es2015 compatible module:
class HttpHeaderProvider extends Web3.providers.HttpProvider {
constructor(host, headers) {
debug('in prv constructor');
super(host);
}
...
prepareRequest(async) {
debug('in prepare');
var request = super.prepareRequest(async);
if (this.headers){
debug('setting headers')
for (var header in this.headers){
request.setRequestHeader( header, this.headers[header]);
}
}
return request;
}
}
Light
Dark
0 comments