Detection of the malicious code injection by hooking system calls in kernel mode

doi:10.7523/j.issn.2095-6134.2010.5.018

›› 2010, Vol. 27 ›› Issue (5): 695-703.DOI: 10.7523/j.issn.2095-6134.2010.5.018

• Research Articles • Previous Articles Next Articles

Detection of the malicious code injection by hooking system calls in kernel mode

LI Wei¹, SU Pu-Rui²

1. Graduate University of the Chinese Academy of Sciences, Beijing 100049, China;
2. Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

Received:2009-12-24 Revised:2010-03-09 Online:2010-09-15

Abstract

Abstract:

Based on detailed analyses of all the methods about runtime process injection and hooking techniques in Windows operating system, we propose a method for dynamically detecting malicious code using the kernel-mode driver. It is implemented as a driver that is able to dynamically monitor every process, report attacks to the user accurately, and enhance overall system security.The experimental results show that this method achieves satisfactory detection effects in performance and detection.

Key words: Hook technology, system service descriptor table(SSDT), system service table(SST)

CLC Number:

TP309.5

LI Wei, SU Pu-Rui. Detection of the malicious code injection by hooking system calls in kernel mode[J]. , 2010, 27(5): 695-703.

References

[1] Nguyen, Reiher N, Kuenning P, et al. Detecting insider threats by monitoring system call activity . Information Assurance Workshop, IEEE Systems, Man and Cybernetics Society, 2003: 45-52.

[2] Iyer A, Ngo H Q. Towards a theory of insider threat assessment //Proceedings of the 2005 International Conference on Dependable Systems and Networks, 2005:108-117.

[3] Liu A, Martin C, Hetherington T, et al. A comparison of system call feature representations for insider threat detection . Information Assurance Workshop, IEEE Systems, Man and Cybernetics Society, 2005: 340-347.

[4] Yariv K. API Spying Techniques for Windows 9x, NT and 2000 . 2000 . http://www.internals.com/articles/apispy/apispy.htm.

[5] Jeffrey R. Windows核心编程
[M]. 北京:机械工业出版社,2000.

[6] Ivo I. API hooking revealed . 2002 . http://www.codeproject.com/system/hooksys.asp.

[7] Jeffrey R. Load your 32-bit DLL into another processs address space using INJLIB
[J]. Microsoft Systems Journal, 1994, 9(5).

[8] Keith B. Windows安全性编程
[M]. 北京:中国电力出版社, 2004.

[9] Keith B. Security Briefs . Microsoft Systems Journal, 1999, 14(8) .http://www.microsoft.com/msj/0899/security/security0899.aspx.

[10] Robert K. Three ways to Inject Your Code into Another Process . (2006-07-02) http://www.codeguru.com/Cpp/W-P/system/processesmodules/article.php/c5767.

[11] Rattle. Using process infection to bypass Windows software firewalls phrack , 2004, 11: 62-0x0d .http://www.phrack.org/show.php?p=62&a=13.

[12] Matt P. Learn system-level Win32 coding techniques by writing and API spy program
[J]. Microsoft systems Journal, 1994,9(12).

[13] Matt P. Under the Hood . Microsoft Systems Journal, 1997,12(9) .http://www.microsoft.com/msj/0997/hood0997.aspx.

[14] Holy_F. Technics of hooking API functions on Windows . 2002 .http://www.hxdef.org.

[15] Crazyload. Playing with Windows/dev/(k)mem
[J]. Phrack, 2002,0x0b:p59-0x10.

[16] Hoglund G, Butler J. Rootkits-Windows内核的安全防护
[M]. 北京:清华大学出版社, 2007.

[17] Tan CK. Defeating Kernel Native API Hookers by Direct Service dispatch Table Restoration . Special Interest Group in Securtiy and Information Integrity(SIG^2), 2004-07-08 http://www.security.org.sg.

Detection of the malicious code injection by hooking system calls in kernel mode

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 2

Recommended Articles

Metrics

Comments

[1]	LI Wei, SU Pu-Rui, SHI Yun-Feng. A technique for detecting malicious documents based on calculation of vector spaces [J]. , 2010, 27(2): 267-274.
[2]	PAN Jian-Feng, LIU Shou-Qun, XI Hong-Sheng, TAN Xiao-Bin. A method for hidden malcode anomaly detection using dynamic control-flow analysis [J]. , 2010, 27(1): 138-143.