A distributed storage method of balancing video resources

doi:10.7523/j.issn.2095-6134.2016.05.017

Abstract

Abstract:

With the increase of internet video contents, the method of distributively storing these videos has attracted much attention. In the distributed storage system, storage balance will make a system more robust. However traditional distributed storage systems always adjust the storage imbalance after it happens, which might cause more IO cost. This study focuses on distributed video file storage and proposes a kind of storage system based on Hash and Bloom Filter, namely HBF. It balances the storage when a file is being stored into the system. The system contains different nodes and the files are distributively stored on these nodes. The ability to easily add and remove nodes makes capacity of the system more scalable. In addition, improvments in storage balance have also been achieved, which keeps the usage of nodes storage relatively consistent. The experiments indicate that HBF achieves a good balance of nodes storage as well as the high performance of the distributed video files storage system. Thus the load balance could be greatly improved, and meanwhile the ultilization of resources would be more reasonable.

Key words: video resources, distributed storage, storage balance, high performance

CLC Number:

TP393

HOU Jinzhong, ZHANG Libo, LUO Tiejian. A distributed storage method of balancing video resources[J]. , 2016, 33(5): 686-692.

References

[1] Wang Z, Luo T. Intelligent video content routing in a direct access network[C]//Symposium on Web Society. 2011:147-152.
[2] Hou J, Luo T, Wang Z, et al. An intelligent media delivery prototype system with low response time[C]//Advances in Swarm and Computational Intelligence. Springer International Publishing, 2015:253-264.
[3] Bloom B H. Space/time trade-offs in hash coding with allowable errors[J]. Communications of the ACM, 2010, 13(7):422-426.
[4] Qiao Y, Li T, Chen S. Fast bloom filters and their generalization[J]. IEEE Transactions on Parallel & Distributed Systems, 2014, 25(1):93-103.
[5] Karger D, Lehman E, Leighton T, et al. Consistent hashing and random trees:distributed caching protocols for relieving hot spots on the World Wide Web[C]//Proceedings of the Twenty-Ninth Annual ACM Symposium on the Theory of Computing. 1997:654-663.
[6] Zhao N, Wan J, Wang J, et al. GreenCHT:a power-proportional replication scheme for consistent hashing based key value storage systems[C]//Mass Storage Systems and Technologies (MSST), 201531st Symposium on. 2015:1-6.
[7] Maccormick J, Murphy N, Ramasubramanian V, et al. Kinesis:a new approach to replica placement in distributed storage systems[J]. ACM Transactions on Storage (TOS), 2009, 4(4):1-28.
[8] Weil S A, Brandt S A, Miller E L, et al. CRUSH:controlled, scalable, decentralized placement of replicated data[C]//SC 2006 Conference, Proceedings of the ACM/IEEE. IEEE, 2006:31-43.
[9] Sun B J, Wu K J. Research on cloud computing application in the peer-to-peer based video-on-demand systems[C]//Intelligent Systems and Applications (ISA), 20113rd International Workshop on. IEEE, 2011:1-4.
[10] 王君君. 网络文件的分布式存储设计与实现[D]. 济南:山东大学,2015.
[11] Decandia G, Hastorun D, Jampani M, et al. Dynamo:amazon's highly available key-value store[J]. ACM Sigops Operating Systems Review, 2007, 41(6):205-220.
[12] Wang Z, Luo C, Luo T, et al. A bloom filter-based index for distributed storage systems[C]//Distributed Computing and Artificial Intelligence, 12th International Conference. Springer International Publishing, 2015:293-301.