How to really do decentralized storage good？
Centralized storage
Centralized storage can be simply understood as existing cloud storage, which is a cloud computing system with data storage and management as its core. Simply put cloud storage is to share storage resources into a storage space that users can access at any time and place through any networkable device.After our daily use, as Dropbox, iCloud, Googe Drive, etc. are centralized cloud storage. Users store photos and other data on cloud services, and actually store the data on these centralized servers.
Centralized cloud storage has become the future development trend of storage technology. Its necessity is manifested on the one hand because of the popularity of the Internet, a large amount of multimedia information has created a large amount of unstructured data; on the other hand, many companies’ business data, personal photos and Data such as music needs to exist in the network so that it can be accessed at any time.
However, from the perspective of the personal cloud storage market, the market space is much huge. Such a large number of users also put forward higher requirements for network disk service providers. Only by forming a technological leading edge, providing more personalized services, and a funded project can finally win in the market competition.
Disadvantages of centralized storage
Despite the advantages of convenience and speed in traditional centralized cloud storage, there are still many risks and hidden dangers in its own mechanism and storage method:
1、 Privacy Problem
Centralized storage users can quickly upload audio, video and other files from the device to the network disk at any time according to their own preferences. This not only saves space on the mobile device, but also accesses the contents of the network disk very quickly when needed. But in fact, the administrator of the network disk can directly view and delete the files uploaded by the user from the platform of the server. In view of this management mechanism, the privacy of the user is liable to leak. Companies that have a large amount of data need a large amount of AI neural network training data in order to take the steps of the AI era. Huge commercial profit will drive large companies to use the data stored in the personal cloud disk to used in AI neural network training, thereby enhancing large companies AI competitiveness.
2、Server security is not high
Centralized storage servers have long been the primary target of hacking because of their special storage methods. A large amount of user data is stored on the server, and information hijacking of large user group services is an important source of revenue for hackers, which means that the security of the server directly affects the security of user data. The data of many large companies on the dark network are on sale, indicating that large companies cannot fully guarantee the security of data.
3、 Risk of termination of operation
In the current Internet environment, cloud storage service providers provide cloud storage services for the public. The annual investment is more than 500 million dollar, and the cloud storage profit model provided by private companies is still unclear. The investment in centralized storage is huge, but the low cost of user demand does not necessarily maintain the company’s operations for a long time. Therefore, service providers may also shut down the service due to poor management.
Decentralized storage
Decentralized storage distributes data across multiple network nodes, similar to the distributed ledger technology of blockchains. Storage vendors (managed hosts) store customer data based on contracts and periodically prove that they can continue to provide storage services until the contract expires.
Users can also rent out the storage space of their own computer without using the remaining disk, and at the same time get a certain token as a reward, while those with Token can rent the hard disk storage space of other users, these users from all over the world can put their own remaining disk space constitutes a decentralized network, and these spare disk space becomes the node of the decentralized network. The data will be cut into small pieces that are encrypted and distributed across many nodes.
Advantages of decentralized storage
1、 Cost down
Decentralized storage can take advantage of the sharing economy. It mainly uses the remaining disk space and unused user upload bandwidth. If you can make full use of these resources, you can save the owner of the storage resources and save. construction cost of centralized storage.
2、 Privacy enhancement
The data is divided into small pieces, which are encrypted and stored in a large number of nodes. This avoids the centralized risk of centralized storage. Even if a piece of data is leaked, it is only part of the data.
3、 Transmission faster
During the downloading process, the file segments are reorganized, and the file segments are derived from multiple storage space providers. Multi-point downloads make decentralized storage much faster than centralized storage. In addition, centralized storage will only deploy a few core big storage providers. Decentralized storage, everywhere is a storage providers(miners), using the nearest transmission, the speed will be faster.
4、 Blockchain smart contract
Through smart contracts, programs can determine usage and reward resource providers.
5、 Token Economic Revitalization Storage Market
Token incentives make more users willing to contribute their own idle resources, thus amplifying the storage capacity of the entire market.
Difficulties and challenges of decentralized cloud storage
1、 Stability problem
Because decentralized storage needs to store data segments in different nodes, when there are fewer users, the insufficient backup of the segments will result in the loss of data, and the centralized storage has been developed for many years, and the services provided are more mature and stable.
2、 Token cheating problem
Since decentralized storage is based on the blockchain-based Token economic model, storage providers (miners) benefit from providing services, which is the economic basis of the entire decentralized system. How to ensure that this mechanism is only fair and reliable, maybe It is a huge difficulty and challenge for a malicious miner to exploit the vulnerability to get illegal gains.
3、Cold start problem
Centralized cloud storage is a company that operates storage space and has the flexibility to choose to increase or decrease servers based on the amount of storage space actually used. For decentralized storage, it is a difficult point, because there is no company to operate the storage space, relying on Token incentives to encourage storage suppliers (miners) to provide services, there will be a balance between supply and demand in the sharing economy. That is to say, in the early stage, maybe because free users is using it, the miners are unwilling to provide services (mining) because they cannot obtain sufficient stable income. To solve this problem, we must design a good Token economic incentive model.
The key to decentralized storage
The key to doing decentralized storage lies in 2 points, 1. The advantages of decentralized storage, and the shortcomings of decentralized storage. So what if you do that?
1、QoS(Quality of Service)
1）QoS is about provisioning network services to the level of guarantee required by the application layer services.
2）QoS mechanisms provide means for ensuring network resources required by applications when delivered to achieve the expected level of user QoE(Quality of Experience).
2、 QoE(Quality of Experience)
The degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state.
What are the specific metrics of QoS and QoE? As mentioned earlier, decentralized storage has the advantage of being cheap enough, fast enough, and secure enough, and QoS is to quantify and optimize these metrics until they eventually go far beyond the QoS of centralized cloud storage.
The existing decentralized storage public chain projects, the main difficulty lies in how to decenter, how to achieve consensus, how to do rigorous mathematical proof to prevent cheating miners, etc. (especially FileCoin, look at FileCoin’s whitepaper and other papers, Almost 80% of the time is explaining how to prove), completely ignore the importance of Qos.
Qos is not ready at the beginning, and needs to be gradually optimized during the development of the project, and it will be improved step by step. So in the early days we had to ensure that the project was able to be upgraded flexibly, because we needed to try to adjust the algorithm.
3、Excellent economic model
As mentioned earlier, one of the difficulties and challenges of decentralized storage is stability. When the storage provider (miners) is unstable, file may lose, and the availability of the entire platform is low (SLA like amazon S3, also Qos. One of the indicators). A good economic incentive model can make storage suppliers (miners) stable, eliminate and punish unstable miners. As long as storage suppliers (miners) are stable, the availability of services (SLA) will become very high.
In addition to affecting the stability of storage providers (miners), the economic model will also encourage new storage providers (miners) to join or participate in order to address the aforementioned balance of demand and supply in sharing economy.
The economic model is the same as the quality of service (Qos). It is not done right from the beginning. It needs to be gradually optimized during the development of the project and changed step by step. So in the early days we had to ensure that the project could be upgraded flexibly because it was necessary to try to adjust the economic incentive strategy.
4、Early anti-cheating is difficult
……
5、Flexible and easy to upgrade architecture and governance mechanisms are very important
Since both quality of service (Qos) and economic models require an iterative process, it is important to build a flexible and easily scalable architecture.
Decentralized storage is not used for projects such as bitcoin, ethereum, etc., because they are projects that write all information to the blockchain. Decentralized storage only has incentives and contracts to write to the blockchain, and the actual stored files do not need to be written to the blockchain. So there is no need to ask for a step to go to the center.
I think the proper path to decentralized storage is to strengthen the center, then the weak center, and finally the center. The center here is centered only for users and storage providers (miners), doing scheduling and indexing, similar to Bittorrent Tracker, so that it can quickly iterate P2P algorithms and economic model strategies. After a little time of verification, you can turn the center here into a multi-center, and invite honest and powerful nodes to join, called a coalition chain. Finally, we prove that the algorithm can be both efficient and safe, from the weak center of the alliance system to the decentralization stage where everyone can participate.
Because of the above considerations:
For this reason, I found Bill and started up the PPIO decentralized storage public chain project together. Our goal is to develop a decentralized storage public chain that prioritizes quality of service(QoS) and good ecosystem. Later, I will gradually write some articles about PPIO chains to illustrate the characteristics and principles of PPIO, please look forward to it.
If you have an exchange about blockchain learning, you can contact me in the following ways:
my WeChat ID：elninowang, note the note: Blockchain learning
Github: https://github.com/omnigeeker
Telegram: @elninowang
Twitter: @elninowang
Medium: https://medium.com/@elninowang_cn
Steemit: https://steemit.com/@elninowang
Zhihu: https://www.zhihu.com/people/elninowang/activities
Jianshu: https://www.jianshu.com/u/103321b574ba