In order to solve the problem that traditional signature-based malware detection systems are inefficacious in detecting new malware,a practical malware detection system is constructed to find out new malware. Application programming interface( API) call sequence is introduced to capture activities of a program in this system. After that,based on variable-length n-gram,API call order can be extracted from API call sequence as the malicious behavior feature of a software. Compared with traditional methods,which use fixed-length n-gram,the solution can find more new malware. The experimental results show that the presented approach improves the accuracy of malware detection.