The problem Deduplication layer on Filecoin solves

The pricing models of web2 storage providers like Dropbox or Goolge drive charge a user based on per GB basis. Imagine a scenario where you and your friend both subscribed to Dropbox and storing the same file on respective Dropboxes. In this case, both of you will get charged from the Dropbox. Although, Filecoin is a web3 solution, it is still following the pricing models of web2 storage providers.
In our solution, if multiple users are storing the same file on Filecoin, they all can share the cost of storing the file and there will be only copy (excluding fault-tolerance requirement) of that file exsists in the network.

Challenges we ran into

We have faced multiple issues:

• Market API Abstratction
  We initially though of using the mock deals provided by Market API, however we wanted to store in actual Filecoin testnet rather tham simulating it through mocks.
• No real-time storage deals on testnet
  We wanted fto ind and have deal with storage provider on testnet, but no such thing existed. Therefore, we migrated to using web3.storage for storing the files on Filecoin.
• Limited documention for making storage deal on either mainnet or testnet
  Migrated to web3.storage
• Different CIDs
  The CIDs produced by IPFS and web3.storage are different. We tired with different versions of CIDs, including changing the encoding functions but still not able to figure out why it is happening. We migrated to using only the CID provided by web3.storage.