用户态文件系统框架（filesystem in user space，FUSE）因其相对简单快捷的开发特性被广泛应用于新型文件系统的开发过程中。但是FUSE框架在处理应用的I/O请求时存在多次用户态和内核态切换、上下文切换以及额外的内存拷贝操作，造成一定的性能开销。为减小FUSE对I/O性能的影响，提出一种绕过FUSE的方案。该方案利用动态链接技术，通过将I/O请求的接收和转发功能实现在用户空间的方式消除FUSE框架的接入。该方案针对并行日志文件系统进行了应用实现和测试，结果表明，该方案对于大块读操作的性能提升最大可达131%，对小块写操作的性能最大提升5倍左右。
A common and easier way to deploy parallel filesystem is using the user-space file system framework (FUSE). However, it serves as an I/O interposition layer which crosses user and kernel space that may introduce significant overhead due to the user-kernel mode switches, context switches and additional memory copies. This paper designs a method based on dynamic linking techniques to bypass FUSE, and implements the interposition layer in user space. This method is implemented and evaluated in parallel log-structured file system which is a popular parallel file system for checkpointing. The experiments and evaluation show that it can improve 131% of the read performance when the transfer block size is large, and improve 5 times write performance at small transfer block size and guarantee no significant write performance downgrade in other scenarios.
[1] Bland B. Titan-early experience with the titan system at oak ridge national laboratory[C]//2012 SC Companion: High Performance Computing, Networking Storage and Analysis. November 10-16, 2012, Salt Lake City, UT, USA. IEEE Press, 2012: 2189-2211.DOI:10.1109/SC.Companion.2012.356.
[2] Livermore’s HPC Innovation Center. Catalyst[EB/OL]. (2017-10-08)[2018-01-01]. http://computation.llnl.gov/computers/catalyst.
[3] Oak Ridge National Laboratory. Summit[EB/OL]. (2018-07-30) [2019-08-07]. https://www.olcf.ornl.gov/summit/.
[4] Ross R B, Thakur R. PVFS: a parallel file system for Linux clusters[C]//Proceedings of the 4th annual Linux showcase and conference. CA, USA: USENIX Association, 2000: 391-430.
[5] Bent J, Gibson G, Grider G, et al. PLFS: a checkpoint filesystem for parallel applications[C]//Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, 2009: 1-12. DOI: 10.1145/1654081.
[6] Braam P. The Lustre Storage Architecture[EB/OL]. (2019-03-05) [2019-10-15]. http://lustre.org/.
[7] 周恩强, 董勇, 张伟,等. 对象存储并行文件系统小文件性能优化研究[J]. 计算机工程与科学, 2013, 35(12): 8-13.DOI:10.3969/j.issn.1007-130X.2013.12.002.
[8] 刘恋, 郑彪, 龚奕利. 分布式文件系统中元数据操作的优化[J]. 计算机应用, 2012, 32(12): 3271-3273.DOI:10.3724/SP.J.1087.2012.03271.
[9] Szeredi M. Fuse: File system in user space[EB/OL]. (2017-09-27) [2017-10-01]. http://fuse.sourceforge.net/.
[10] Rath N. List of fuse based file systems[EB/OL]. (2019-12-22) [2020-04-15]. https://github.com/libfuse/libfuse/wiki/Filesystems.
[11] Tarasov V, Gupta A, Sourav K, et al. Terra incognita: on the practicality of user-space file systems[C]//7th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 15). CA, USA: USENIX Association, 2015:15-15.
[12] Red Hat Inc. GlusterFS[EB/OL]. (2011-10-07) [2020-05-08]. https://www.gluster.org/.
[13] Hoskins M E. Sshfs: super easy file access over ssh[EB/OL]. (2006-04-28) [2020-05-08]. https://www.linuxjournal.com/article/8904.
[14] Vangoor B K R, Tarasov V, Zadok E. To fuse or not to fuse: performance of user-space file systems[C]//Proceedings of the 15th USENIX Conference on File and Storage Technologies, Santa Clara, CA, USA: USENIX Association, 2017:59-72.
[15] Lipp M, Schwarz M, Gruss D, et al. Meltdown: reading kernel memory from user space[C]//27th USENIX Security Symposium. Baltimore, MD, USA: USENIX Association, 2018: 973-990.
[16] Ishiguro S, Murakami J, Oyama Y, et al. Optimizing local file accesses for FUSE-based distributed storage[C]//2012 SC Companion: High Performance Computing, Networking Storage and Analysis. November 10-16,2012,Salt Lake City, UT, USA. IEEE, 2012: 760-765.DOI:10.1109/SC.Companion.2012.104.
[17] Zhu Y, Wang T, Mohror K, et al. Direct-FUSE: removing the middleman for high-performance FUSE file system support[C]//Proceedings of the 8th International Workshop on Runtime and Operating Systems for Supercomputers. 2018:1-8.DOI:10.1145/3217189.3217195.
[18] Ward L. The SYSIO library[EB/OL]. (2003-02-20) [2019-12-17]. https://sourceforge.net/projects/libsysio/.
[19] Rajgarhia A, Gehani A. Performance and extension of user space file systems[C]//Proceedings of the 2010 ACM Symposium on Applied Computing. 2010: 206-213.DOI:10.1145/1774088.1774130.
[20] 王卫锋, 杨林. 基于Hadoop的邮政寄递大数据分析系统设计与实现[J]. 中国科学院大学学报, 2017, 34(3): 395-400.DOI:10.7523/j.issn.2095-6134.2017.03.014.
[21] Zadok E, Nieh J. FiST: a language for stackable file systems[C]//Proceedings of the Annual Conference on USENIX Annual Technical Conference. CA, USA: USENIX Association, 2000:5-21.
[22] Li C P, Ding C, Shen K. Quantifying the cost of context switch[C]//Proceedings of the 2007 workshop on Experimental computer science. 2007: 2-5.DOI:10.1145/1281700.1281702.
[23] Egwutuoha I P, Levy D, Selic B, et al. A survey of fault tolerance mechanisms and checkpoint/restart implementations for high performance computing systems[J]. The Journal of Supercomputing, 2013, 65(3): 1302-1326.DOI:10.1007/S11227-013-0884-0.
[24] Rosenblum M, Ousterhout J K. The design and implementation of a log-structured file system[J]. ACM Transactions on Computer Systems, 1992, 10(1): 26-52.DOI:10.1145/146941.146943.
[25] Linux Programmer’s Manual. ld.so(8)[EB/OL]. (2020-04-30) [2020-05-08]. http://man7.org/linux/man-pages/man8/ld.so.8.html.
[26] Weil S A, Brandt S A, Miller E L, et al. Ceph: a scalable, high-performance distributed file system[C]//Proceedings of the 7th Symposium on Operating Systems Design and Implementation. CA, USA: USENIX Association, 2006: 307-320.
[27] Los Alamos National Laboratory. LANL fs test [EB/OL]. (2017-05-20) [2017-09-05]. https://github.com/fstest/fs_test.
