用户态并行文件系统性能优化

doi:10.7523/j.ucas.2020.0027

中国科学院大学学报 ›› 2022, Vol. 39 ›› Issue (2): 275-282.DOI: 10.7523/j.ucas.2020.0027

• 电子信息与计算机科学 • 上一篇

用户态并行文件系统性能优化

邓通亮^1,2,3, 陈宸^1,2,3, 殷树¹

1 上海科技大学信息科学与技术学院, 上海 201210;
2 中国科学院上海微系统与信息技术研究所, 上海 200050;
3 中国科学院大学, 北京 100049

收稿日期:2020-03-23 修回日期:2020-05-11 发布日期:2021-06-01
通讯作者: 殷树
基金资助:
国家自然科学基金（61402158）资助

On the efficiency of user-level parallel file systems

DENG Tongliang^1,2,3, CHEN Chen^1,2,3, YIN Shu¹

1 School of Information Science & Technology, ShanghaiTech University, Shanghai 201210, China;
2 Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai 200050, China;
3 University of Chinese Academy of Sciences, Beijing 100049, China

Received:2020-03-23 Revised:2020-05-11 Published:2021-06-01

摘要/Abstract

摘要： 用户态文件系统框架（filesystem in user space，FUSE）因其相对简单快捷的开发特性被广泛应用于新型文件系统的开发过程中。但是FUSE框架在处理应用的I/O请求时存在多次用户态和内核态切换、上下文切换以及额外的内存拷贝操作，造成一定的性能开销。为减小FUSE对I/O性能的影响，提出一种绕过FUSE的方案。该方案利用动态链接技术，通过将I/O请求的接收和转发功能实现在用户空间的方式消除FUSE框架的接入。该方案针对并行日志文件系统进行了应用实现和测试，结果表明，该方案对于大块读操作的性能提升最大可达131%，对小块写操作的性能最大提升5倍左右。

关键词: 文件系统, 存储系统, FUSE, 并行文件系统

Abstract: A common and easier way to deploy parallel filesystem is using the user-space file system framework (FUSE). However, it serves as an I/O interposition layer which crosses user and kernel space that may introduce significant overhead due to the user-kernel mode switches, context switches and additional memory copies. This paper designs a method based on dynamic linking techniques to bypass FUSE, and implements the interposition layer in user space. This method is implemented and evaluated in parallel log-structured file system which is a popular parallel file system for checkpointing. The experiments and evaluation show that it can improve 131% of the read performance when the transfer block size is large, and improve 5 times write performance at small transfer block size and guarantee no significant write performance downgrade in other scenarios.

Key words: file system, storage system, FUSE, parallel file system

中图分类号:

TP302

邓通亮, 陈宸, 殷树. 用户态并行文件系统性能优化[J]. 中国科学院大学学报, 2022, 39(2): 275-282.

DENG Tongliang, CHEN Chen, YIN Shu. On the efficiency of user-level parallel file systems[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(2): 275-282.

参考文献

[1] Bland B. Titan-early experience with the titan system at oak ridge national laboratory[C]//2012 SC Companion: High Performance Computing, Networking Storage and Analysis. November 10-16, 2012, Salt Lake City, UT, USA. IEEE Press, 2012: 2189-2211.DOI:10.1109/SC.Companion.2012.356.
[2] Livermore’s HPC Innovation Center. Catalyst[EB/OL]. (2017-10-08)[2018-01-01]. http://computation.llnl.gov/computers/catalyst.
[3] Oak Ridge National Laboratory. Summit[EB/OL]. (2018-07-30) [2019-08-07]. https://www.olcf.ornl.gov/summit/.
[4] Ross R B, Thakur R. PVFS: a parallel file system for Linux clusters[C]//Proceedings of the 4th annual Linux showcase and conference. CA, USA: USENIX Association, 2000: 391-430.
[5] Bent J, Gibson G, Grider G, et al. PLFS: a checkpoint filesystem for parallel applications[C]//Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, 2009: 1-12. DOI: 10.1145/1654081.
[6] Braam P. The Lustre Storage Architecture[EB/OL]. (2019-03-05) [2019-10-15]. http://lustre.org/.
[7] 周恩强, 董勇, 张伟,等. 对象存储并行文件系统小文件性能优化研究[J]. 计算机工程与科学, 2013, 35(12): 8-13.DOI:10.3969/j.issn.1007-130X.2013.12.002.
[8] 刘恋, 郑彪, 龚奕利. 分布式文件系统中元数据操作的优化[J]. 计算机应用, 2012, 32(12): 3271-3273.DOI:10.3724/SP.J.1087.2012.03271.
[9] Szeredi M. Fuse: File system in user space[EB/OL]. (2017-09-27) [2017-10-01]. http://fuse.sourceforge.net/.
[10] Rath N. List of fuse based file systems[EB/OL]. (2019-12-22) [2020-04-15]. https://github.com/libfuse/libfuse/wiki/Filesystems.
[11] Tarasov V, Gupta A, Sourav K, et al. Terra incognita: on the practicality of user-space file systems[C]//7th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 15). CA, USA: USENIX Association, 2015:15-15.
[12] Red Hat Inc. GlusterFS[EB/OL]. (2011-10-07) [2020-05-08]. https://www.gluster.org/.
[13] Hoskins M E. Sshfs: super easy file access over ssh[EB/OL]. (2006-04-28) [2020-05-08]. https://www.linuxjournal.com/article/8904.
[14] Vangoor B K R, Tarasov V, Zadok E. To fuse or not to fuse: performance of user-space file systems[C]//Proceedings of the 15th USENIX Conference on File and Storage Technologies, Santa Clara, CA, USA: USENIX Association, 2017:59-72.
[15] Lipp M, Schwarz M, Gruss D, et al. Meltdown: reading kernel memory from user space[C]//27th USENIX Security Symposium. Baltimore, MD, USA: USENIX Association, 2018: 973-990.
[16] Ishiguro S, Murakami J, Oyama Y, et al. Optimizing local file accesses for FUSE-based distributed storage[C]//2012 SC Companion: High Performance Computing, Networking Storage and Analysis. November 10-16,2012,Salt Lake City, UT, USA. IEEE, 2012: 760-765.DOI:10.1109/SC.Companion.2012.104.
[17] Zhu Y, Wang T, Mohror K, et al. Direct-FUSE: removing the middleman for high-performance FUSE file system support[C]//Proceedings of the 8th International Workshop on Runtime and Operating Systems for Supercomputers. 2018:1-8.DOI:10.1145/3217189.3217195.
[18] Ward L. The SYSIO library[EB/OL]. (2003-02-20) [2019-12-17]. https://sourceforge.net/projects/libsysio/.
[19] Rajgarhia A, Gehani A. Performance and extension of user space file systems[C]//Proceedings of the 2010 ACM Symposium on Applied Computing. 2010: 206-213.DOI:10.1145/1774088.1774130.
[20] 王卫锋, 杨林. 基于Hadoop的邮政寄递大数据分析系统设计与实现[J]. 中国科学院大学学报, 2017, 34(3): 395-400.DOI:10.7523/j.issn.2095-6134.2017.03.014.
[21] Zadok E, Nieh J. FiST: a language for stackable file systems[C]//Proceedings of the Annual Conference on USENIX Annual Technical Conference. CA, USA: USENIX Association, 2000:5-21.
[22] Li C P, Ding C, Shen K. Quantifying the cost of context switch[C]//Proceedings of the 2007 workshop on Experimental computer science. 2007: 2-5.DOI:10.1145/1281700.1281702.
[23] Egwutuoha I P, Levy D, Selic B, et al. A survey of fault tolerance mechanisms and checkpoint/restart implementations for high performance computing systems[J]. The Journal of Supercomputing, 2013, 65(3): 1302-1326.DOI:10.1007/S11227-013-0884-0.
[24] Rosenblum M, Ousterhout J K. The design and implementation of a log-structured file system[J]. ACM Transactions on Computer Systems, 1992, 10(1): 26-52.DOI:10.1145/146941.146943.
[25] Linux Programmer’s Manual. ld.so(8)[EB/OL]. (2020-04-30) [2020-05-08]. http://man7.org/linux/man-pages/man8/ld.so.8.html.
[26] Weil S A, Brandt S A, Miller E L, et al. Ceph: a scalable, high-performance distributed file system[C]//Proceedings of the 7th Symposium on Operating Systems Design and Implementation. CA, USA: USENIX Association, 2006: 307-320.
[27] Los Alamos National Laboratory. LANL fs test [EB/OL]. (2017-05-20) [2017-09-05]. https://github.com/fstest/fs_test.

用户态并行文件系统性能优化

On the efficiency of user-level parallel file systems

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	朱士坤, 李冬, 黄奎, 张宝贤. 一种高效的轻量级无线传感器网络文件系统设计与实现[J]. 中国科学院大学学报, 2014, 31(1): 130-134.
[2]	文晓阳，荆继武，刘鹏. 基于日志式文件系统的数据修复模型 [J]. 中国科学院大学学报, 2008, 26(6): 792-801.