Proto-danksharding (EIP-4844)

Proto-danksharding represents a pivotal development in the evolution of Ethereum, promising to significantly enhance its scalability and efficiency. This intermediate step towards full danksharding aims to make transactions on Layer 2 cheaper and scale Ethereum to over 100,000 transactions per second. This article delves into what proto-danksharding is, how it works, its development history, and its implications for the future of Ethereum.

Thanks for reading Crypto Njal’s Substack! Subscribe for free to receive new posts and support my work.

Introduction to proto-danksharding

Proto-danksharding, also known as EIP-4844, introduces a novel approach for rollups to add data to Ethereum blocks more cheaply. This method is named after two researchers, Protolambda and Dankrad Feist, who proposed it. Historically, rollups posted their transactions in CALLDATA, which is processed by all Ethereum nodes and stored on-chain indefinitely, making it expensive. Proto-danksharding introduces data blobs, a transformative element to enhance Ethereum's scalability and efficiency. These blobs provide a dedicated space for rollups to store transaction data temporarily, reducing the need to post data in the more expensive and permanent CALLDATA format. Each blob is a chunk of data attached to Ethereum blocks, designed to be cost-effective and ephemeral, automatically expiring after approximately 18 days. The use of blobs significantly lowers the cost for rollups to operate, leading to cheaper transactions for end users. This approach allows Ethereum to handle a much higher transaction volume by offloading data storage from the main chain, thereby increasing throughput and reducing congestion on the network.

Proto-danksharding operates by allowing rollups to post transaction data in the form of blobs, which are not directly accessible by the Ethereum Virtual Machine (EVM). These blobs are verified through a commitment scheme involving polynomial functions. Rollups package transaction data into blobs and post a "commitment" to this data. The data in the blob is fitted to a polynomial function. For instance, a simple function like f(x) = 2x - 1 can be evaluated at different points, e.g., x=1, x=2, x=3. A prover evaluates the same polynomial function at specific points to verify the data. Any alteration in the data will result in different values, indicating a mismatch. This verification process is wrapped in cryptographic functions to ensure security. In contrast to traditional sharding, which divides the blockchain into separate segments, proto-danksharding introduces a novel approach for rollups to add data to Ethereum blocks more cheaply, significantly enhancing scalability and efficiency.

Sharding and proto-danksharding represent distinct approaches to enhancing scalability in the Ethereum network. Sharding, a traditional concept, involves partitioning the blockchain into smaller, independent segments known as shards, each capable of processing transactions in parallel. This aims to alleviate network congestion and boost throughput by distributing the workload across multiple shards. In contrast, proto-danksharding introduces a novel strategy focused on optimizing data storage rather than partitioning the blockchain. It introduces temporary storage units called blobs, which efficiently store large amounts of data off-chain, thereby reducing the burden on the main blockchain and improving scalability. While sharding focuses on dividing the blockchain into separate segments, proto-danksharding targets data storage optimization, making it an interim solution paving the way for future scalability improvements.

Development History

The concept of sharding to improve blockchain scalability dates back to 2016, with Ethereum developers seeking ways to divide the blockchain into smaller, more manageable parts. However, traditional sharding posed numerous challenges, including potential security risks and complex data migration processes. In response to these issues, Dankrad Feist, a researcher at the Ethereum Foundation, proposed an innovative approach focusing on sharding data storage rather than the entire blockchain state. This method, known as danksharding, aimed to optimize data availability and shard block production without overloading validators.

Feist’s ideas gained traction, and in collaboration with fellow Ethereum researcher Diederik Loerakker, also known as Protolambda, they developed an interim solution to pave the way for full danksharding. This solution, EIP-4844, introduced proto-danksharding. The proposal was named after the two researchers and outlined a method to add blob-carrying transactions to the Ethereum network. These blobs are large chunks of data stored temporarily, significantly reducing costs for layer 2 solutions like rollups, which bundle transactions off-chain before posting them to Ethereum.

Proto-danksharding formally began its development phase around 2022, gaining momentum as Ethereum transitioned from Proof of Work (PoW) to Proof of Stake (PoS) with the Ethereum 2.0 upgrade. This transition, while crucial for addressing scalability and transaction speed, did not sufficiently reduce gas fees, prompting the need for further innovations like proto-danksharding. Feist and Protolambda’s EIP-4844 was widely accepted by the Ethereum community due to its practicality and potential to drastically lower rollup fees.

A critical aspect of proto-danksharding is the introduction of data blobs. These blobs are attached to blocks and store data off-chain, which is not immediately necessary for the Ethereum Virtual Machine (EVM). They are automatically deleted after a set period, reducing the storage burden on Ethereum nodes and lowering transaction costs. This temporary nature of blobs distinguishes them from traditional CALLDATA, which remains on-chain permanently and is more expensive.

To ensure the integrity and verification of these blobs, the developers utilized a cryptographic scheme known as KZG (Kate-Zaverucha-Goldberg) commitments. This method involves fitting a polynomial function to the data blobs and generating a small cryptographic commitment that can be easily verified. The KZG ceremony, conducted in 2023, played a crucial role in this process. Tens of thousands of participants from the Ethereum community contributed to generating a secret random string of numbers necessary for the KZG scheme, ensuring its security and reliability.

The culmination of proto-danksharding’s development came with its implementation in the Dencun upgrade, a hard fork scheduled for late 2023. The Dencun upgrade, a portmanteau of the simultaneous "Cancun" and "Deneb" upgrades on Ethereum’s execution and consensus layers, respectively, included several other Ethereum Improvement Proposals (EIPs) alongside EIP-4844. These proposals aimed to further enhance data storage efficiency and reduce fees, collectively pushing Ethereum towards greater scalability.

On March 13, 2024, the Dencun upgrade was successfully deployed on Ethereum’s mainnet, marking the official launch of proto-danksharding. This milestone was soon followed by real-world testing and stress scenarios, such as the emergence of Ethscriptions and blobscriptions, which tested the new data blob market. Despite initial spikes in blob fees, the system demonstrated its effectiveness in providing cheaper data storage compared to the previous CALLDATA method.

Implications for Ethereum

At its core, proto-danksharding introduces a revolutionary approach to data storage and transaction processing on Ethereum. By leveraging temporary data blobs, the protocol significantly reduces the cost of storing data on-chain, particularly for layer 2 solutions like rollups. This dramatic reduction in transaction fees makes Ethereum more accessible to users and developers, opening doors to a multitude of use cases and applications previously hindered by high gas costs.

One of the most immediate implications of proto-danksharding is its impact on Ethereum’s scalability. With the ability to process transactions more efficiently and cost-effectively, Ethereum can accommodate a higher volume of transactions without experiencing congestion or exorbitant fees. This scalability boost is essential for Ethereum’s continued growth and relevance in an increasingly competitive blockchain landscape.

Furthermore, proto-danksharding fosters innovation within the Ethereum ecosystem by lowering barriers to entry for developers and entrepreneurs. With reduced transaction costs, developers can experiment more freely, creating decentralized applications (dApps) and smart contracts that leverage Ethereum’s robust infrastructure without being constrained by high gas fees. This influx of creativity and innovation has the potential to drive Ethereum’s ecosystem forward, fueling its evolution into a thriving hub of decentralized finance (DeFi), non-fungible tokens (NFTs), and other groundbreaking technologies.

From an economic perspective, proto-danksharding has profound implications for Ethereum’s token economy and ecosystem dynamics. Lower transaction fees make Ethereum more attractive for a wider range of users, including retail investors, traders, and institutions. This increased demand for Ethereum tokens could potentially drive up their value, benefiting existing holders and incentivizing further investment in the platform. Additionally, lower transaction costs make microtransactions and everyday use cases more feasible on Ethereum, potentially expanding its utility beyond niche markets into mainstream adoption.

Moreover, proto-danksharding reinforces Ethereum’s position as a leader in blockchain technology and innovation. By addressing longstanding scalability challenges and demonstrating the platform’s ability to evolve and adapt, Ethereum solidifies its status as a foundational pillar of the decentralized web. This confidence in Ethereum’s capabilities attracts talent, capital, and partnerships, further accelerating its growth and development.

However, despite its promise and potential, proto-danksharding is not without its challenges and considerations. As with any major protocol upgrade, there are technical complexities, security risks, and potential unforeseen consequences that must be carefully navigated. Additionally, the transition to proto-danksharding requires coordination and collaboration among Ethereum stakeholders, including developers, miners, validators, and users, to ensure a smooth and successful implementation.

Conclusion

Proto-danksharding marks a significant milestone in Ethereum's quest for scalability and efficiency. By introducing data blobs and a robust verification mechanism, it promises to reduce costs and increase transaction throughput, paving the way for a more scalable and accessible Ethereum network. As the Ethereum community continues to innovate and build upon these advancements, the future of decentralized applications and blockchain technology looks brighter than ever.