Hello friendly Developers!

I would like to propose that clamwin actually scan a processes' memory in addition to looking up the running modules and passing them to the file scanner.

After taking a quick peek at the clamwin 0.88.5 binary in IDA, I see that you already have the functionality to enumerate all the modules running in memory, which is fantastic. Now if it is possible to build on top of that something similar to the following over-simplification:

. Enumerate all modules (done?)
. Obtain the name (done?)
. Obtain the module base address and size of in-memory image.
. Allocate a buffer equal to the size of image.
. ReadProcessMemory() the image into the buffer.
. Realign section raw data offsets. (does rebuildpe() do this?)
. Pass this buffer to scanbuff()

Entrypoint, Section, and "*" based extended format signatures should work normally provided the raw data offsets are fixed. The absolute and eof based offsets won't work due to mis-alignment between the data in the on-disk file compared to the in-memory image. (a small loss, in my opinion, compared to the gain of being able to see unpacked images in memory)

Maybe even add a new db specific to clamwin memory detection?

What do you think?

currently it doesn't need different signatures since it scans the disk copy of the memory modules, in almost all cases this is enough because
windows cannot remove/modify/rename loaded modules. The only exception would be a kernel mode rootkit or something like, but at this
point we are in user space, the kernel stuff could fake us anyway.
Next release will have kill/unload (for exe or dll) of infected modules before removing/moving

Right, that's why I mentioned doing both still.

Unfortunately, there are a lot of files that clamav can not detect with its existing signatures because the guys doing the sig's put them on stuff like arbitrary encrypted/compressed bytes in the data section of an executable. (why not just do a hash??)

I've seen a few pieces of malware packed with Upack which clamav detects. Unfortunately Upack uses the LZMA algorithm and from what I can tell, the compressed data is nearly completely altered with very minimal changes to the pre-compressed data. The interesting thing about this that you can change useless bytes in an executable, repack it with Upack, and the data on disk with the exception of the decompression routine is almost entirely different, giving you a psuedo "poly" that is now different from what the sig writer has seen and ready for distribution very quickly with no code changes. Whereas the decompressed code stays running in memory and is almost byte-for-byte identical. Guess what I would prefer to place a signature on...

Now in the instance that clamwin could read the memory of a process and scan it for signatures, those of us in the know, could place signatures on unpacked bytes rather than the junk that is available on disk. Being that those signatures would be useless to the general clamav engine running on non-windows platforms (which the malware cannot unpack itself and be available for memory dumping), hence the need for a db that is applicable to "unpacked" malware running on windows. Afaik clamav's unpacking capabilities are weak at best.



Removing malware from a system and expecting it to remain functional is a pretty difficult thing to do, especially when you consider things that inject DLLs into winlogon. Good luck unloading a DLL from that process without the box blue screening.

ClamWin is still relatively new in the antivirus field, and it is a work in progress. The developers have the opportunity to make it more relevant to current malware--rather than grafting something onto an older program like some of the commercial competition. They are tied to ClamAV for the engine and signature database, but ClamAV is doing a pretty good job with limited resources, and the signature updates are very frequent.

I recently read a review of ClamAV by a computer magazine writer, who was surprised by the high level of some of the code, considering it was a non-commercial/open source effort. ClamAV has said that it will include some advanced heuristics in the software within the next year or so. I see they have Swizzor in their signatures, which is sort of a model for some virus writers now, but I don't see them doing much with rootkits yet. They have to concentrate their limited resources upon the largest threats. At some point the ClamWin developers may want to develop some anti-malware techniques of their own, such as behavior blocking, that ClamAV doesn't, but it's going to be interesting to see what happens.

Regards,

This kind of scanning would really take a lot of time, perhaps not impossible, also I'll need to known exactly how libclamav unpacks pe before scanning, the unpacked code should be similar to in memory code. Also note that the unpackers in clamav-devel are more than the current stable, this functionality is still developed a lot by clamav developers.
Killing processes is not a big deal if they are directly malware, unloading dlls is not so safe, I'm thinking to add a different option for dll unload,
my way to unload dll should be safe as long the code in the dll is not being executed, but since we cannot easily known, it will still be unsafe,
but as last resort to cleanup a "pesty" system, I think it worth to have such an option

I looked at the code in the most recent snapshot of libclamav (pe.c, 09 Nov 2006) and it unpacks certain packers. libclamav tries to determine the presense of them in the cli_scanpe() function. It looks for byte signatures of SUE, UPX, FSG, Petite, Y0da's crypter, WWPack, and NsPack.

Some of the more common ones that are not on the list, Upack, Mew, pecompact, aspack, pepack, shrinker, telock, asprotect, etc.

You are right about the unpacked code being similar to that in memory, but only for the packers that clamav supports. For unsupported, it is still compressed/encrypted on disk and entirely different from what would be running in memory.

Scanning memory will slow scan times down, but it opens the door for greatly enhanced detection on Windows for those users that have an active infection.

Even if clamwin cannot terminate/remove the malware, I'm sure the user would greatly appreciate the warning that it has been detected running on the system.

I'm not sure I'm following your assertion about "grafting." Memory scanning is incredibly useful, and using the pre-existing clamav scanning engine, one can write signatures on unpacked malware. That is a "good thing."


Engine yes, signature database no. It would be extremely easy to add an additional "clamwin.cvd" to which clamav signature writers or others could contribute updates.

For example: I have a few variants of a password stealing trojan generally known as "Sinowal" of which some UPX'd and not packed variants are detected by clamav. (https://www.microsoft.com/security/encyclopedia/details.aspx?name=Win32%2fSinowal)

However those which are not detected use nspack, some use aspack, and some use a custom crypter which decrypts and jumps to the entrypoint of UPX scrambler 2.0, none of which clamav supports.

These files in memory are almost binary identical, but on disk... they are completely different, a signature writers nightmare.


Here's a quick sample from the two unpacked files in memory, file1 being nspack, and file2 being aspack:
FILE1:

FILE2:


This code appears to be at the tail end of end of a "switch/multiple if" statement where it determines the name of the file it is going to drop via CreateFileA/WriteFile.

Drop this tasty signature in clamwin.ndb:

ClamWin.Sinowal-A:1:*:5068{4}68{4}FF35{4}EB2083F9050F858C0200008B45FC83C0025068{4}68{4}FF35{4}FFD6


run "sigtool --build clamwin.cvd" on it and you would have a definition set that, provided that clamwin could scan memory, would detect this on the users box and warn them and/or remove it.

I wasn't particularly thinking of memory scanning when I mentioned ClamWin was a work in progress and didn't have to "graft" something--merely that the developers have the opportunity to make an antivirus program that is more relevant to current malware (built to earn dollars instead of gain publicity) than some of the legacy antiviral software. From my perspective as a single "user" of antiviral software, I think memory scanning is great if it can be done simply and leave enough resources available to run the computer with. Just because we have all this computer/hard drive memory, we don't have to use it all, and I hope the developers keep this in mind.

From my single user perspective again, it appears to me that a lot of effort goes into unpacking archived/compressed programs and that separate code routines must be written for each type of packer. Do you really need to do this? A virus doesn't usually do any harm until it is unpacked and put in memory or put into an active directory on a hard drive. At that point, the antivirus software (if it's resident) can kick in and look at the signature on hard drive or in memory, and I don't think it would need an unpacking routine for that. Of course, ClamWin being an on-demand scanner only, it would be a little late if the viral code went active before the next scan.

I guess this sounds like "behavior blocking," but if the scanner is quick enough, that what it does.

I also think updating the signature database yourself is great if you can do it, but I don't think that is something the average user wants to do.

Regards,

@b0ne
unfortunately the libclamav pe scan is strictly file desc based we only have in mem data, I think that finding the OEP in process memory shouldn't be a big deal, but without a major rewrite of libclamav the only solution would be write some data from OEP to disk then call scanpe.
Your considerations are very interesting, we may rewrite a different scanning routing that scans not packed pe from a memory stream, but currently is not so easy to add such code in libclamav and I should also learn more about PE structure, perhaps I also need this knowledge to make other stuff (there are no c/c++ libs around for resource replacing)

did you ever thinked about joining clamwin development?

I'm just playing with upx and just found that libclamav unpacks the exe and writes it to a temp file, with full pe header, similar data I get dumping the memory module from base address, the pe is similar but not equal, in both cases I get unpacked data but offset are different,
also memory dump of an upx compressed modules still have upx sections and libclamav still think that is upx compressed.
libclamav rewrite pe header with some custom data, it also add a text "!This file was created by ClamAV for internal use and should not be run...ClamAV - A GPL virus scanner - https://www.clamav.net", then it calls magic_scandesc that may lead in a infinite loop.
I think that by crafting dumped pe libclamav can scan the like other pe modules, so it doesn't need different kind of signatures, just some additional one for executable that cannot be unpacked by the internal upacker

also it would be usefull to have a smart way to understand if the executable is packed or not before dumping and scanning just to speedup a bit, the scan should be made to the on disk module and in memory, by detecting a non packed executable we can avoid unneeded scans

@b0ne
can you check:
https://oss.netfarm.it/clamav/files/clamav-pedump.7z https://oss.netfarm.it/clamav/files/clamav-pedump.7z
actually it dumps in-memory pe and then scans it, after that if no virus are found it tries
disk based (I think the reverse would be better)

you can avoid building a cvd file and
use directly -d file.nbd
it will go faster

the code is unconditional so it's a bit slow, but just to see if the idea is working

Hi Sherpya. I did manage to make it work, but it took a little trickery.

There are two problems, one of which you encountered already.

1. The first problem is passing unpacked memory-dumps of executables that use a packer that clamav recognizes. For example, the UPX'd file you dumped from memory and scanned.

- To prevent that problem clamwin must skip dumping the memory of any executable that libclamav is going to attempt to unpack itself. There are two ways that I can see accomplishing this, first the more elegant, and second the quick hackery.

-- clamav developers would need to be convinced to seperate and modularize PE recognition functions from the scanning functions. Currently, all of that code is contained within one giant function cli_scanpe(). libclamav would be more useful to have MZ/PE/Packer determination code into the cli_filetype() function and expand the enums to include CL_TYPE_MSEXE, CL_TYPE_UPXEXE, CL_TYPE_FSGEXE, etc. That way one could call cl_filetype2() to see if the file is a recognized packer or not.

-- the hackery option would be to strip down cli_scanpe() removing everything but broken PE header determination, and packer recognition, returning what type of clamav-supported packer was found if any inside of clamscan. You can see in the debug print statements where they finish determination and open/write(ndesc) a file out to be scanned.

2. The second problem is making the section headers within the pe-header have the correct "Pointer To Raw Data" value. The .text section of an exe on disk typically starts at file position 1024. However, each section is memory-page-aligned, so when you do ReadProcessMemory, the .text section typically starts at file position 4096 in the dumped file. (pe-header @ 0x400000, .text @ 0x401000+virtual size, .data @ 40?000+virtual size ).

- The easiest way to fix this is to take the "Virtual Address" value (RVA) of the section and over write the "PointerToRaw" value with that RVA. This will make that section point to the correct data on disk. You could probably borrow the cli_peheader() function from libclamav to obtain the structs that contain these values. (see pe.h and execs.h) There is also "PeLib" which is a C++ pe-header library, but it is released under the zlib/libpng license. I'm not sure how compatible that is with the GPL. I think you can make an exception for it though?

If you want a quick fix just to get it working, there is a DLL included with the program that I use quite frequently called PE-Tools. The DLL is called RebPE32.dll, and there is a function "DumpFix" exported from it that accepts the same buffer you passed to ReadProcessMemory(). After you execute DumpFix on the buffer, you can then write it to disk and it will work.

https://www.cloverbasin.com/files/clamscan_example.zip https://www.cloverbasin.com/files/clamscan_example.zip

This zip file contains all the files I used to produce a working detection.

FULLSCAN.NDB
Run mewpad.exe, and use the following command line: clamscan --memory --show-progress -d fullscan.ndb

fullscan.ndb will detect notepad_WinXP,SP1.exe, mewpad.dmp-unfixed.exe, and mewpad.dmp.exe

SECTION.NDB
Dump "mewpad.exe" after running it using PE-Tools or use the RebPE32.dll + DumpFix() function.
I used the following command line: clamscan -d section.ndb \path\to\mew*.exe

section.ndb will detect notepad_WinXP,SP1.exe and mewpad.dmp.exe


db\fullscan.ndb contains a signature to match WinMain() anywhere in a PE.
db\section.ndb contains a signature to match WinMain() at an exact offset from section.

DumpFix\RebPE32.dll the dumpfix function is contained in RebPE32.dll

files\notepad_WinXP,SP1.exe original windows xp service pack1 notepad.exe

files\mewpad.dmp-unfixed.exe dumped mew11 packed notepad.exe, section headers _NOT_ 'DumpFix'd.
files\mewpad.dmp.exe dumped mew11 packed notepad.exe, section headers 'DumpFix'd.
files\mewpad.exe mew11 packed exe

The ClamAV people seem to have their own agenda, so I don't believe you will get any help from them. They have their own agenda, and besides, they've got their own version of Clam for Windows that is several months behind where it should be.

Regards,

b0ne I think a "clean" and working rebuild is not needed here I think rearranging pe structure should be enough,
the code to dump pe is in, with minor changes I should be able to dump something usefull like at least the first code that is executed (right now it dumps all pe module, but it could be optimized by dumping only some parts). An ac-scan will have enough data to match the correct signature

I would happy to have a fast way to detect if a module is packed or not,
the idea is:
- normal scan on file on disk
if not detected
- check if packed, dump it, fixup, then re-pass to scanner

libclamav detection code is a bit incomplete:

a good starting point but doesn't covers all packers
then


clamav-devel added md5 hashing of sections (it's really usefull??)
and contains a bit smarter algo but more complex, it also seams to handle 64bit pe
the rest of the code is the same of clamav-stable except that it detects more unpackers

heuristic way can be an option like peid that calculates bytes entropy
ida perhaps detects a packed exe in a different way
it says that import segment seams to be destroyed and I have no clue about doing this check

another way can be looking at imported functions, a packed exe always imports GetProcAddress and similar
but some packers also retain some of original exe imports

I don't understood what you mean with WinMain() code matching, are you talking about to find winmain() then pick signature from there?
And finally clamscan would need different signatures and not different "kind" of signatures
clamav docs about making signatures are incomplete, the only example is a md5 match that is not really usefull
also it needs a signing server to sign the archiver and there are no docs around about it

@GuitarBob
what are you talking about?