Filecoin Deep Dive 2: Market Actors and Participation

Filecoin Oct 05, 2020

Last month, we discussed why we think Filecoin is a big milestone in the evolution of the Internet and outlined four key reasons why Filecoin can succeed, namely:

  1. Significantly lowering data storage costs
  2. Properly designed incentives
  3. Resilience and censorship resistance
  4. The founding team’s experience at Protocol Labs

This week, we sat down with the Filecoin team to discuss the project in greater detail and present a more comprehensive look at the three main market participants in the Filecoin economy: clients, storage miners, and retrieval miners. Let’s dive in:

1. According to Filecoin’s whitepaper, there are three main participants in the decentralized storage network: clients, storage miners, and retrieval miners. Can you define these terms and explain how these participants are distinct from one another?

The clients’ role in the Filecoin economy is straightforward, but critical: they use Filecoin tokens to pay for the storage and retrieval of files.

Storage miners are the core service providers on Filecoin. Since Filecoin is a global network that everyone can use, the storage mining demographic is diverse – including individuals, and various organizations and companies. Storage miners both compete and collaborate in the Filecoin Network; they compete with each other for storage deals and block rewards, and they collaborate with each other by growing the Filecoin Economy and maintaining the blockchain.

Retrieval miners are rewarded for delivering content quickly. Similar to a hyper-local CDN, the retrieval market is for miners who have low-latency, high-bandwidth connections close to end-users. These well-placed miners earn filecoin for providing files and data at the fastest speeds.

2. The Filecoin network is made up of two markets: the storage market and the retrieval market. To start, can you explain how the storage market works, the participants involved, how they interact with each other, including  the processing of payments, and how these interactions are then settled on the blockchain?

Filecoin’s storage marketplace takes advantage of the world’s underused storage assets. With Filecoin, people anywhere in the world can offer their unused storage space to anyone else. This creates a unique sharing economy that has the power to unlock untold amounts of storage and drive an unprecedented economy of scale for pricing storage solutions.

Storage on the Filecoin network decouples hard drive space from many of the commonly bundled additional services provided by other cloud storage providers. The protocol enables buyers and sellers to trade in storage space directly. As a result, clients and miners are free to negotiate prices and features. Filecoin provides the secure network and token that clients and miners can use to enter storage contracts in a peer-to-peer fashion. Unlike traditional storage providers, Filecoin bundles one unique service: cryptographically verifying, every single day, that your data is still there. This verification allows anyone to offer storage in the marketplace while still enabling clients to trust that their data will be stored on the network.

At a practical level, clients choose from among the many active miners on the network and propose a storage deal for a specified price, duration, and amount of data. If the storage miner accepts the deal, the client transfers the file and submits a payment for the agreed amount. The payment is made available to the storage miner incrementally over the duration of the deal. For highly important data, clients may also choose to require deal collateral from the miner, which creates an additional incentive to store the data securely.

3. Following up on that question, can you explore those same points for the retrieval market?

A miner can participate in the retrieval contracts, supplying clients with a specified file in exchange for filecoin. This incentivizes well-placed nodes with high-throughput, high-bandwidth connections to participate in the network, promoting the widespread and rapid distribution of files – especially those that command high demand.

Retrieval deals are conducted through payment channels, allowing private, highly scalable off-chain payments directly between parties. Payment channels may be settled via on-chain transactions at any time.

4. How do clients who want to use Filecoin to store sensitive information ensure that their data is being stored privately?

Filecoin can store any type of data, encrypted or unencrypted. There are many use cases that require the ability to serve unencrypted data – for example, websites, videos, or widely-available scientific research. Clients storing sensitive information should encrypt it before storage.

5. How do clients ensure that their data is being stored properly?

Filecoin is set up from day one to store the world’s most important data reliably and efficiently. To enhance reliability, the protocol gives clients unlimited flexibility to store redundant copies of files with different miners, and to verify that unique copies are actually being stored. Unlike centralized cloud storage services, which back up data in ways clients can’t change or verify, Filecoin allows clients to easily express their own preferences for reliability and cost.

To guarantee that data is being stored properly, clients can simply observe the sectors into which their deals were sealed. The Filecoin blockchain’s proof-of-spacetime automatically tests miners once per day to ensure that the data has been continuously stored. If the sector enters a faulted state, it indicates that there may be an issue with the underlying storage.

6. Storage and retrieval miners clearly play vital roles on the network. Digging into that deeper, how does storage mining work on Filecoin and what is the mining cycle for storage miners specifically?

The role of storage miners is to keep files on behalf of the Filecoin network. Storage miners must cryptographically prove that they are honoring their pledge to store these files – this is achieved via the mechanisms of Proof-of-Replication (PoRep) and Proof-of-Spacetime (PoSt). Pledging storage to the Filecoin network itself requires filecoin; these are used as collateral to ensure that storage miners uphold their contractual obligations.

In the Filecoin network, data is stored in fixed-size sectors. Generally, storage miners fill these sectors with data stored on behalf of clients, who contract storage miner services for a particular length of time via deals. However, storage miners are not forced into making deals; if a storage miner doesn’t find any of the available deal proposals appealing, they can alternatively make capacity commitments, filling sectors with arbitrary data. This allows them to provably demonstrate that they are reserving space on behalf of the network. If desired, sectors created to serve as capacity commitments can later be “upgraded” to provide the contracted storage for future deals.

Once a sector has been filled, PoRep sees storage miners seal the sector – sealing is a computation-intensive process that results in a unique representation of the data (the original representation can subsequently be reconstructed by unsealing). Once data is sealed, storage miners will: 1) generate a proof; 2) run a SNARK on the proof to compress it; and finally, 3) submit the result of the compression to the blockchain as a certification of the storage commitment. Storage reserved for the network through this process is termed pledged storage.

After PoRep has been completed, storage miners must continuously prove that they are still storing the data they pledged to store. This is accomplished via PoSt, a procedure in which storage miners are issued a cryptographic challenge that can only be correctly answered by consulting a sealed sector directly. The storage miner must respond to this challenge within strict time limits; the computational difficulty of sealing ensures that storage miners must maintain ready access to and integrity of the sealed sector.

In Filecoin, PoSt manifests in two distinct challenges: WindowPoSt and WinningPoSt.

WindowPoSt is the mechanism by which the commitments made by storage miners are audited. It sees each 24-hour period broken down into a series of windows. Correspondingly, each storage miner’s set of pledged sectors is partitioned into subsets, one subset for each window. Within a given window, each storage miner must submit a PoSt for each sector in their respective subset. This requires ready access to each of the challenged sectors, and will result in a SNARK-compressed proof published to the blockchain as a message in a block. In this way, every sector of pledged storage is audited at least once in any 24-hour period, and a permanent, verifiable, and public record attesting to each storage miner’s continued commitment is kept.

The Filecoin network expects constant availability of stored files. Failing to submit WindowPoSt for a sector will result in a fault, and the storage miner supplying the sector will be slashed – that is, a portion of their collateral will be forfeited, and their storage power will see a reduction. Storage miners will have a limited period of time to recover from faults before they are considered to have abandoned their storage commitment altogether. Should the need arise, storage miners will also have the ability to preemptively issue a declared fault, which will result in reduced penalties, but which still must be addressed within a reasonable timeframe.

WinningPoSt is the mechanism by which storage miners are rewarded for their contributions. In the Filecoin network, time is discretized into a series of epochs – the blockchain’s height corresponds to the number of elapsed epochs. At the beginning of each epoch, a small number of storage miners are elected to mine new blocks (Filecoin utilizes tipsets, which permit multiple blocks to be mined at the same height). Each elected miner who successfully creates a block is granted filecoin, as well as the opportunity to charge other nodes fees to include messages in the block.

A storage miner’s probability of being elected corresponds to their storage power. In a process similar to that underlying WindowPoSt, storage miners are tasked with submitting a compressed proof of storage for a specified sector before the epoch concludes. Storage miners who fail to complete WinningPoSt in the necessary window will forfeit the opportunity to mine a block, but will not otherwise incur penalties for their failure to do so.

7. Similarly, how does retrieval mining work and what is the mining cycle for retrieval miners?

Retrieval mining is somewhat more straightforward because transactions happen instantaneously. A client requests a specific file from a miner. The miner may need to unseal the sector containing the file, or she may have cached an unsealed copy in anticipation of retrieval requests. The client and miner will create a payment channel or reuse an existing one, and the client will incrementally pay the miner for the file as it’s transferred.

In the early days of the network, we expect most retrieval to be performed by the storage miners who are responsible for the underlying files. There are already several groups working to further separate storage and retrieval roles, and allow for discovery and replication of content hosted on Filecoin by retrieval-only miners.

Legal Notice

This blog post is being distributed by Amalgamated Token Services Inc., dba “CoinList,” or one of its subsidiaries. This blog post and use of the CoinList website is subject to certain disclosures, restrictions and risks, available here.

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.