When individuals consider the issue of storage, they tend to picture extremely huge files such as videos or datasets. In practical uses, however, particularly of the consumer applications, the converse is often true, very large quantities of very small files. NFT metadata, profile pictures, chat messages, receipts, logs, thumbnails, and configuration files - all over.
Most decentralized storage systems consider each file as an object. They all require their storage operation, their metadata and their cost. Perhaps the overhead is not important in a single file, but as the number of small files increase (thousands of files per app, millions of files per app) the cost increases exponentially, and the performance reduces exponentially. This renders most Web3 applications costly to operate or compelled to revert to central databases.
Walrus deals with this issue in a pragmatic manner with Quilt. In place of keeping all the small files in different storage units, Walrus enables the multiple storage of many small files into one storage unit, and yet has the capability of accessing individual files individually when required. This makes storage predictable and inexpensive drastically cutting the overhead.
The point here is effectiveness and no dispensability. Developers do not have to create own batching logic and have to deal with sophisticated indexing systems. Walrus does this on the protocol level so that the apps can scale and storage is not a bottleneck.
Example:
A collection of NFTs of 10,000 NFTs should have at minimum 20,000 small files (images + metadata). In the absence of batching, this is very costly and time-consuming. Quilt allows the grouping of those files effectively, making them cost-effective and large collections practical on decentralized infrastructure.
