收稿日期: 2016-01-25
修回日期: 2016-03-18
网络出版日期: 2016-09-15
基金资助
中国科学院设备共享管理系统优化项目（Y42901VED2）资助
A distributed storage method of balancing video resources
Received date: 2016-01-25
Revised date: 2016-03-18
Online published: 2016-09-15
随着互联网视频内容日益增多，视频资源的分布式存储受到关注.在分布式存储系统中，如果文件能均衡存储在各个节点会使系统更加健壮，而传统的分布式存储系统通常在失衡发生后再调整，带来了较多的IO开销.对分布式视频文件存储进行研究，提出一种利用哈希和Bloom Filter的高性能存储系统HBF，在文件存入系统时即进行存储平衡.系统具备多个节点，文件分散保存在不同的节点上，系统通过增加或删除节点使容量具备可伸缩性，而且在存储平衡方面进行了改进，使存储节点之间的存储使用量保持相对一致.实验证明，HBF使分布式视频文件存储系统具有高性能并兼顾节点存储平衡，有利于负载均衡和资源的合理利用.
侯金钟 , 张立波 , 罗铁坚 . 一种均衡视频资源的分布存储方法[J]. 中国科学院大学学报, 2016 , 33(5) : 686 -692 . DOI: 10.7523/j.issn.2095-6134.2016.05.017
With the increase of internet video contents, the method of distributively storing these videos has attracted much attention. In the distributed storage system, storage balance will make a system more robust. However traditional distributed storage systems always adjust the storage imbalance after it happens, which might cause more IO cost. This study focuses on distributed video file storage and proposes a kind of storage system based on Hash and Bloom Filter, namely HBF. It balances the storage when a file is being stored into the system. The system contains different nodes and the files are distributively stored on these nodes. The ability to easily add and remove nodes makes capacity of the system more scalable. In addition, improvments in storage balance have also been achieved, which keeps the usage of nodes storage relatively consistent. The experiments indicate that HBF achieves a good balance of nodes storage as well as the high performance of the distributed video files storage system. Thus the load balance could be greatly improved, and meanwhile the ultilization of resources would be more reasonable.
Key words： video resources; distributed storage; storage balance; high performance
[1] Wang Z, Luo T. Intelligent video content routing in a direct access network[C]//Symposium on Web Society. 2011:147-152.
[2] Hou J, Luo T, Wang Z, et al. An intelligent media delivery prototype system with low response time[C]//Advances in Swarm and Computational Intelligence. Springer International Publishing, 2015:253-264.
[3] Bloom B H. Space/time trade-offs in hash coding with allowable errors[J]. Communications of the ACM, 2010, 13(7):422-426.
[4] Qiao Y, Li T, Chen S. Fast bloom filters and their generalization[J]. IEEE Transactions on Parallel & Distributed Systems, 2014, 25(1):93-103.
[5] Karger D, Lehman E, Leighton T, et al. Consistent hashing and random trees:distributed caching protocols for relieving hot spots on the World Wide Web[C]//Proceedings of the Twenty-Ninth Annual ACM Symposium on the Theory of Computing. 1997:654-663.
[6] Zhao N, Wan J, Wang J, et al. GreenCHT:a power-proportional replication scheme for consistent hashing based key value storage systems[C]//Mass Storage Systems and Technologies (MSST), 201531st Symposium on. 2015:1-6.
[7] Maccormick J, Murphy N, Ramasubramanian V, et al. Kinesis:a new approach to replica placement in distributed storage systems[J]. ACM Transactions on Storage (TOS), 2009, 4(4):1-28.
[8] Weil S A, Brandt S A, Miller E L, et al. CRUSH:controlled, scalable, decentralized placement of replicated data[C]//SC 2006 Conference, Proceedings of the ACM/IEEE. IEEE, 2006:31-43.
[9] Sun B J, Wu K J. Research on cloud computing application in the peer-to-peer based video-on-demand systems[C]//Intelligent Systems and Applications (ISA), 20113rd International Workshop on. IEEE, 2011:1-4.
[10] 王君君. 网络文件的分布式存储设计与实现[D]. 济南:山东大学,2015.
[11] Decandia G, Hastorun D, Jampani M, et al. Dynamo:amazon's highly available key-value store[J]. ACM Sigops Operating Systems Review, 2007, 41(6):205-220.
[12] Wang Z, Luo C, Luo T, et al. A bloom filter-based index for distributed storage systems[C]//Distributed Computing and Artificial Intelligence, 12th International Conference. Springer International Publishing, 2015:293-301.
/
| 〈 |
|
〉 |
