Building out Web3 and using blockchain technology to decentralize control over networks, platforms, and data is a powerful idea that can help revive the open internet. But blockchain advocates believe that smart contracts won’t eat the world until decentralized applications are able to directly interface with Web2, just as conventional consumer web services do today. Only then will blockchains onboard billions of Web2 users to Web3 services.
Blockchains typically cannot directly interact with systems and data that are hosted outside of their respective networks. Legacy web tech largely communicates via cloud infrastructure and servers using the standard web protocol HTTP, forcing dapps to rely on external “oracles” for communicating with HTTP-based sources. Oracles basically function as bridges that connect on-chain software with off-chain data and services.
Because blockchains like Ethereum (ETH) and Solana (SOL) can’t offer native oracle solutions, dapp developers deal with the added programming complexity, costs, latency, and security risks of using trusted middleware to query external data. They commonly integrate centralized oracles and oracle networks such as Chainlink (LINK) to allow their smart contracts to access off-chain infrastructure, but they need to regularly ensure that these oracle services are reliably returning valid responses that are not compromised.
Smart contracts benefit from an ability to ingest off-chain data from the real world. Decentralized exchanges, for example, often obtain exchange rates from centralized exchanges like Coinbase. This is why financial smart contracts, automated market makers, and trading platforms generally depend on oracles while navigating associated vulnerabilities and exploits, such as price oracle manipulation. What if there were a direct API for dapps to access external data? What if developers no longer had to incorporate and pay for third-party intermediaries to send on-chain requests and relay off-chain responses?
In a sign of how blockchain technology continues to push forward, the Internet Computer (ICP) recently implemented an API that enables smart contracts and dapps to directly send HTTP outcalls, including secured HTTPS traffic. “If you develop a canister smart contract on the Internet Computer blockchain, it will be able to directly make HTTPS outcalls to Web2 servers and trustlessly receive a direct response,” explains Dieter Sommer, technical program manager at the DFINITY Foundation, which contributes R&D to the Internet Computer. “From the developer perspective, the experience is closer to the familiar Web2 programming paradigm without oracles.”
An HTTP API
The way that oracle projects like Chainlink, UMA (UMA), and API3 (API3) typically operate involves an off-chain authority that reads data from a particular external source and writes it to a unique smart contract to relay that data in response to on-chain queries. Smart contracts running on a blockchain can then be programmed to use the oracle as a substitute or translator for the off-chain data.
The disadvantages include the involvement of an intermediary that compounds the work of developers and increases the risk of service failure. Third parties can collude or conspire to manipulate data, and the oracles can also be hacked or compromised. Receiving and sending the data can take time, harming user experience, particularly in the case of decentralized oracles that need to aggregate data from redundant sources and coordinate consensus. Chainlink is particularly dominant in the oracle space, being blockchain agnostic, but using the oracle network is also expensive for developers.
The introduction of a direct HTTP API for smart contracts opens a path for Web3 to seamlessly interoperate with established HTTP-based services and data sources, creating an array of application possibilities that can efficiently leverage a vast amount of Web2 data. The HTTPS outcalls feature is newly incorporated into the Internet Computer Protocol stack, with a focus on its consensus layer. Internet Computer smart contracts are deployed on subnet blockchains, with each canister’s code and state replicated across all of a given subnet’s nodes.
Most blockchains rely on oracles to intermediate HTTP requests because replication and consensus make it impractical to interoperate directly with HTTP. Replicas would make identical calls to an external service and potential variations in timestamps or IDs would generate different responses, making it impossible for them to reach agreement.
“The IC protocol stack can conceptually be simply seen as a communication pipe between the canister and the conventional HTTP server that makes sure that the HTTP response makes it through consensus and all honest replicas receive the exact same response in execution,” the documentation explains.
Each of a subnet’s nodes running a given smart contract on the Internet Computer requests the URL, and if enough of them receive the same response, it is returned to the requesting canister. If the replica responses differ — e.g., because of contained timestamps, different ordering of response values, or different overlapping sets of response values — developers can simply code a transformation function. This standardizes the responses by eliminating parts that are unnecessary (such as timestamps or identifiers), sorting values, or trimming response values — basically fulfilling whatever relevant transformation makes the responses of all replicas equal in order to finally arrive at consensus.
“The canister smart contract itself is the HTTP client, so we give it the best possible flexibility and power to deliver the capability of integrating Web3 with Web2,” says Sommer. “With less cost and fewer trust assumptions, dapp developers can prioritize their product and business logic, and users get stronger security and less latency.”
After Oracles
Reducing external dependencies and intermediaries is a central principle behind blockchain, and is key to increasing the resilience and security of decentralized systems. Eliminating the need for smart contracts to rely on oracles can help developers more easily build and deploy a wide range of decentralized applications relating to finance, insurance, communications, sports, and logistics. One can imagine querying servers for Internet of Things (IoT) sensor data, as well as connecting to traditional email services and sending chat messages via message gateways.
While achieving such real-world use cases is essential to the development of Web3, the effort today requires developers and users alike to trust oracles, relying on trusted intermediaries in order to access off-chain data to allow a Web3 dapp or blockchain-based service to operate. Compromised data or an exploit of the oracle by hackers — or simply a mundane, conventional tech outage — can destabilize the entire smart contract and dependent smart contracts, thereby presenting a single point of failure. Developers and users will expect better experiences as Web3 continues to mature, and this underscores the need for a better approach to integrations.
Eliminating the complexity of current oracle solutions for off-chain data access could have the effect of speeding Web3 development and user onboarding. HTTPS outcalls on the Internet Computer show how developers can now create products and dapps that directly interact with conventional servers and enterprise infrastructure, effectively blurring the line between Web2 and Web3 while positioning smart contracts to truly eat the world.
