Integrating Web3 technologies with Azure Devops

Although the various technologies that make up what’s been dubbed “Web3” are unlikely to replace the massive infrastructure and software investments we’ve made during the past three decades, there’s still something interesting there. The first question we need to ask is, what problems can they solve?

Web3 proponents suggest that at heart, it’s a massive set of consumer technologies that can replace the web’s transactional foundations. I think of it as a more limited tool, one that’s able to build on blockchain technologies to support a subset of enterprise applications with a focus on electronic data interchange (EDI). That’s because once you strip back the blockchain to its essence, it’s an immutable data structure that can be shared between untrusted partners in a trusted manner. That makes it useful in supply chains where electronic documents have a contractual and legal basis that’s enshrined in international treaties and where one end of the supply chain has only an indirect relationship with the other.

Microsoft’s work on proof-of-membership consensus blockchains, run by consortia of untrusted organizations is an interesting option here, offering a fast and low-impact alternative to proof-of-work and proof-of-stake systems. At the same time, recent releases of SQL Server now provide an immutable ledger for applications that don’t need to be distributed between different entities.

You can think of these blockchain-based services as something like the electronic equivalent of the bills of lading used to describe a ship’s cargo, something that travels through several different business systems without alteration and where you may not know all the different entities that interact with documents and contracts. Those entities could be any of the original manufacturers, shippers, warehouses, cargo ships, customs agents, customs offices, and many more. All need access to the documents, and many need to add their own signatures as part of a complex multiparty approval process.

We could build these into an enterprise blockchain, but we need to start thinking about how we use them within a modern development environment. We’re already building distributed systems at scale for cloud-native applications using devops and CI/CD platforms, so can we use these techniques for Web3?

Using enterprise tools with the blockchain

Microsoft’s Donovan Brown was tasked with looking at how developers should work with these distributed application platforms. Now part of Mark Russinovich’s CTO incubation team on Azure, Brown is best known for his devops work, so it wasn’t surprising that he quickly started bringing popular Web3 platforms into a devops framework. He’s had some good results. I recently had a conversation with him about how he was able to use these technologies with enterprise tools.

If we’re to make these tools ready for use in the enterprise, they need to become part of the way we build code, integrating with both our development platforms and our build and test tools. It’s important that the tools we use avoid the many public disasters associated with Web3, especially with handling commerce and other key information and value flows. We don’t want a smart contract for a bill of lading that can be hijacked to change the cargo being delivered to our warehouses or even diverted to another destination.

Part of the problem Brown identified was an explosion of tools that offered slightly different sets of features. It’s a landscape that makes it hard to get on board, as there’s no obvious toolchain and no real set of best practices to help you build that toolchain. That means there’s a need to identify the mature tools that support enterprise best practices, with the intent to wrap it in a GitHub Codespace or make it available in one of Microsoft’s Dev Box virtual development environments. Otherwise getting started is hard, with no easy route to onboard new developers on your team.

Choosing tools is only part of the problem and possibly the easiest to overcome. The biggest issue is that if you’re using development best practices, it’s very difficult to shoehorn these new tools into existing pipelines. As Brown says, “As I dug deeper into it, I realized, wow, these tools aren’t even designed to be put into a pipeline.” There’s too much reliance on simple publishing techniques, writing code on your own and simply publishing it without formal testing. That approach is all very well for self-hosted experiments and prototypes, but it doesn’t scale to delivering enterprise-grade code.

Building a devops pipeline for smart contracts in Azure

How can you bring them into a devops pipeline? First, we need to stop thinking of Web3 technologies as isolated from the rest of the enterprise application stack. Once we do, we can find points of integration, for example, putting smart contracts into a test harness and using test-first development techniques.

Brown has now been able to build an Ethereum-based distributed application environment that uses Azure Pipelines with Dev, QA, and Production outputs, with Azure Static Web Apps hosting the application front end. Dev deployments run in a private Ethereum instance on Azure Containers. The biggest problem for a developer taking this approach is deploying a smart contract to different environments.

It turns out that smart contracts hard-code an address that’s automatically added to the contract JSON when it’s compiled. This requires rebuilding the entire contract on each deployment, requiring multiple rebuilds for each environment. As Brown notes, this is a devops antipattern. You should only need to compile once, adding environment-specific values at runtime. This required some work to rewrite the application front-end code to support an external source for the network address. This approach makes it easier to use the service when a contract address can’t be found, using an Azure Function to deliver the address when queried.

This allows Brown’s code to only build the front end once, supporting it being used at each stage of the deployment pipeline. He could then use standard JavaScript testing frameworks with his application. All the steps necessary to build and deploy each environment from a GitHub repository could be built into a single Azure Pipeline, deleting environments as each step is validated. Tools like Azure Container Apps help here, allowing quick deployment of build artifacts.

From this start, it’s possible to add support for additional frameworks in each environment, as well as infrastructure as code tools like Bicep, and system management scripts in the Azure CLI and PowerShell to ensure that you have a repeatable environment and that you’re able to deliver a ready-to-run application and all the servers and services necessary to support it. Working in Azure using both infrastructure-as-a-service and platform-as-a-service tools allows you to remove unused environments after they’re no longer needed, saving money and ensuring that your application and its environment are an idempotent distribution, each change to your code requiring a complete redeployment of the entire application and supporting infrastructure.

Towards a maturity model for blockchain technologies

Brown’s work goes a long way to showing how Web3 technologies can be built into a familiar enterprise environment as part of a modern application framework. There’s no need to step outside familiar tools—GitHub, Azure Devops, Azure Container Apps, VS Code. It’s clear, however, that changes are necessary in how Web3 frameworks work with environment variables and dynamic resources. They’re not designed to work in a multistage pipeline, and changes are necessary if they’re to offer the appropriate level of maturity for use at scale in enterprise applications. There’s also a need for better telemetry so that developers can get a clearer look at how their applications and contracts perform.

The result is an interesting hybrid of the familiar and the new. That’s a good thing, as it makes it easier for a developer to adopt new technologies and bring them into existing development processes. It’s important for companies like Microsoft to take a deep look at new technologies, as they can accelerate the maturation of emerging innovations. Companies can offer an incubation pathway from the experimental to the enterprise, one that’s informed by many years of enterprise application development experience, inside and outside their own platforms.