ClamWin Free Antivirus Forum Index
ClamWin Free Antivirus
Support and Discussion Forums
Reply to topic
Talking About...
GuitarBob


Joined: 09 Jul 2006
Posts: 4376
Location: USA
Reply with quote
Well, there was a mention in a previous post that there were two ways to handle a problem--getting ClamAV to change some of its code or do it the quick way. My remark addressed the possibility of getting ClamAV's help--based on some of my communications with them.

Regards,
View user's profileSend private message
sherpya


Joined: 22 Mar 2006
Posts: 898
Location: Italy
Reply with quote
clamav doesn't support natively win32 so it's very difficult to ask help in adding windows specific code ;(
View user's profileSend private message
Re: Memory Scanning Et Al
b0ne


Joined: 26 Oct 2006
Posts: 174
Reply with quote
GuitarBob wrote:
From my single user perspective again, it appears to me that a lot of effort goes into unpacking archived/compressed programs and that separate code routines must be written for each type of packer. Do you really need to do this? A virus doesn't usually do any harm until it is unpacked and put in memory or put into an active directory on a hard drive.


Close, but not exactly. UPX is kind of like a self extracting Zip file, except the contents of the zip file are never written to disk.

Here's something of a diagram of how it works:

Under normal conditions the executable looks something like this when it is in memory:

(hexadecimal memory addresses)
MY.EXE
Code:

----------------------------------
0x400000     PE-HEADER
----------------------------------
  x400010     ExecutableSize
  x400014     Base Address = 0x400000
  x400018     Code Section = 0x1000 (RVA)
  x400020     Data Section = 0x5000 (RVA)
  x400024     Resource Section = 0x7000 (RVA)
  x400028     Main() = 0x1456 (RVA)

----------------------------------
0x401000     CODE SECTION
----------------------------------
  x401456     main()
  x402513     DownloadMalware( variable MalwareURL )
  x402642     WriteMalwareToDisk( variable FileNameToWrite )
  x402891     LaunchMalware()
----------------------------------
0x405000     DATA SECTION
----------------------------------
  x405121     Variable MalwareURL = "http://127.0.0.1/badstuff/hehehe.exe"
  x405321     Variable FileNameToWrite = "svchost.exe"
----------------------------------
0x407000     RESOURCES
----------------------------------


What a signature writer would do is create a signature on the MalwareURL, or perhaps the DownloadMalware() function, and say that the signature for DownloadMalware() resides at S0+0x2513 (section 0, "code section, plus 0x2513 bytes).


When you UPX compress "MY.EXE", it essentially "Zips" all of the sections of MY.EXE so that the stuff you place signatures on, is now garbled junk. When ClamAV goes to scan S0+2513 for the DownloadMalware() signature, it instead encounters "junk", and it is generally pretty useless to write signatures on the "junk."

When you run a UPX'd exe, there is a new "UpxMain()" function that gets executed which UnZips MY.EXE into it's proper places in memory, then JUMPS to the original main() commonly known as the OEP or original entrypoint. No file ever gets put on the system. The UPX'd exe looks like this:

MY.EXE UPX
Code:

----------------------------------
0x400000     PE-HEADER
----------------------------------
  x400010     ExecutableSize
  x400014     Base Address = 0x400000
  x400018     Code Section = 0x1000 (RVA)
  x400020     Data Section = 0x5000 (RVA)
  x400024     Resource Section = 0x7000 (RVA)
  x400028     UPX "ZIP" Data = 0x9000 (RVA)
  x40002C     UpxMain() = 0xB342 (RVA)

----------------------------------
0x401000     CODE SECTION
----------------------------------
  x401456     EMPTY
  x402513     EMPTY
  x402642     EMPTY
  x402891     EMPTY
----------------------------------
0x405000     DATA SECTION
----------------------------------
  x405121     EMPTY
  x405321     EMPTY
----------------------------------
0x409000     UPX ZIPPED DATA SECTION
----------------------------------
  x409456     COMPRESSED main()
  x409513     COMPRESSED DownloadMalware()
  x409642     COMPRESSED WriteMalwareToDisk()
  x409891     COMPRESSED LaunchMalware()
  x40A121     COMPRESSED Variable MalwareURL = "http://127.0.0.1/badstuff/hehehe.exe"
  x40A321     COMPRESSED Variable FileNameToWrite = "svchost.exe"

  x40B324     UpxMain()
  x40B325     decompress "main()"
  x40B326     copy "main() to original location, 0x401456
  x40B327     decompress "DownloadMalware()"
  x40B328     copy "DownloadMalware() to original location
                    .... decompress and copy the rest of the compressed items
  x40B401     JUMP to original "main()" @ 401456
View user's profileSend private message
b0ne


Joined: 26 Oct 2006
Posts: 174
Reply with quote
Quote:
clamav-devel added md5 hashing of sections (it's really usefull??) and contains a bit smarter algo but more complex, it also seams to handle 64bit pe the rest of the code is the same of clamav-stable except that it detects more unpackers


MD5 hashing of sections is really only useful for malware that doesn't bother to pack their executables, but does silly XOR/simple in place math to obfuscate their strings in the data section. It is also not hard to enlarge the CODE sections beyond the needed bytes for actual code to add hash breaking padding. So, maybe useful... I have yet to see it used in a real implementation.

Quote:

heuristic way can be an option like peid that calculates bytes entropy
ida perhaps detects a packed exe in a different way
it says that import segment seams to be destroyed and I have no clue about doing this check


PeID does several interesting things, they calculate entropy on "some" of the executable, but not all. I read a forum post a while ago, but I don't recall the specifics. The PeID developer is fairly tight lipped about that stuff. Next is the entry-point byte scanner which is also useful for detecting packers.

I believe IDA/Ollydbg use a similar technique to see warn if the file is packed or not. I think it is mostly due to Import Table of the executable having few numbers of api's like LoadLibrary/GetProcAddress only, etc. Also if the entrypoint resides outside of the code section.

I think a really quick way to determine if the file is packed or not is to hash the first or last 32 to 128 bytes of the typically non-writable sections of an executable in memory and on disk to see if they match. Code and resource sections should be fairly constant, if one or more of them don't match, dump it?

Quote:
I don't understood what you mean with WinMain() code matching, are you talking about to find winmain() then pick signature from there?


I placed the signature on the WinMain() function inside of notepad.exe to detect the version of notepad I put in the zip.

Quote:
And finally clamscan would need different signatures and not different "kind" of signatures clamav docs about making signatures are incomplete, the only example is a md5 match that is not really usefull also it needs a signing server to sign the archiver and there are no docs around about it


http://www.clamav.net/doc/0.81rc1/signatures.pdf http://www.clamav.net/doc/0.81rc1/signatures.pdf
The above URL is the document I followed to create the "new database" (.ndb) signatures that are present in fullscan.ndb and section.ndb. Since they are not "official" clamav signature databases, they would not neccesarily have to be signed, but I don't know enough about that whole process yet to figure it out.
View user's profileSend private message
sherpya


Joined: 22 Mar 2006
Posts: 898
Location: Italy
Reply with quote
I've made some readings around internet, the most interesting suggestions are:
- IAT is in non-standard section
- IMAGE_IMPORT_DIRECTORY inconsistencies
- Strings
- Very few Imports
- Differences between section’s VirtualSizeand SizeofRawData
- Non standard NumberOfRvaAndSizes

and in some other place, zero sized sections with code

so considering that I don't have idea about the standard section of IAT
I think the best matching rule is imports

Also your suggestion about hashing some pe code is interesting, it should be tuned a bit, but can be a good
method

about signing, libclamav can load single files in a cvd, I could even make a different format for signature archives and sign them by myself, so I think this "secret" it's not really a problem
View user's profileSend private message
sherpya


Joined: 22 Mar 2006
Posts: 898
Location: Italy
Reply with quote
hmm not so easy in memory modules are _very_ different from disk and I'm not able to find the correct part of binary data to check for
I hope lord pe deluxe doesn't make some strange things on dumped modules, I'm lazy Smile I should use the code I added in clamav as standalone process dumper
View user's profileSend private message
sherpya


Joined: 22 Mar 2006
Posts: 898
Location: Italy
Reply with quote
ok I've added an heuristic pe packed detection, it's based on:
- zero values in IAT
- size of any section is zero

I'm working on an entropy check, I've made the code but it's still inefficient on raw executable I need to check
on section code parts

the IAT has some issues but right now the "false positives" are not so much:
- ntdll.dll - not really a standard pe - I can make some checks for kernel mode stuff
- mingw executables - I may add specific signature checks
- dotnet cil - same there
- resources only dll - even here a check can be added

so the code needs to be improved, but it's easy to check the benefits.
i.e. standard upx code has 1 section with raw size to 0
changing raw size of this section to something different than zero
fools the libclamav unpacker but the IAT check still detects as packed data

mew files are detected as packed data, then the memory process is dumped

b0ne if you want to make some tests, binaries are here:
http://oss.netfarm.it/clamav/files/clamav-pedump.7z http://oss.netfarm.it/clamav/files/clamav-pedump.7z
while source of scanmem is here:
http://svn.sourceforge.net/viewvc/clamwin/trunk/clamav-release/contrib/msvc/src/scanmem.c?view=log http://svn.sourceforge.net/viewvc/clamwin/trunk/clamav-release/contrib/msvc/src/scanmem.c?view=log

happy hacking !!!
View user's profileSend private message
PE Files
GuitarBob


Joined: 09 Jul 2006
Posts: 4376
Location: USA
Reply with quote
Perhaps the link below might help in your checking re: PE files

http://www.windowsitlibrary.com/Content/356/11/1.html

Regards,
View user's profileSend private message
sherpya


Joined: 22 Mar 2006
Posts: 898
Location: Italy
Reply with quote
@GuitarBob
interesting docs

I've added a basic entropy checker and a signature matcher for mingw that lowers entropy value,
still need some check to see if I get false negatives and I'm still unsure if mingw sig matcher is needed or
not not many files on windows installations are compiled by mingw
resource only dll should be whitelisted by the entropy
I'll not care about kernel mode stuff, only ntdll.dll is matched,
still need to known how to understand if a file is a .net assembly
the Unix "file" utility is able to detect it but I've not understood the way
View user's profileSend private message
Heuristic D
GuitarBob


Joined: 09 Jul 2006
Posts: 4376
Location: USA
Reply with quote
"ok I've added an heuristic pe packed detection"

That sounds like a good idea. Could you use a similar technique for other situations--even files on the hard drive? I guess every file has a header, doesn't it? If you could, it might reduce the need to get deeply into a file in scanning for a virus. As for false positives, how about checking the file creation date or date of last change and compare with most recent ClamWin scan date? If there's been no change, it's probably okay. I'm wondering mainly about files on disk.

Regards,
View user's profileSend private message
b0ne


Joined: 26 Oct 2006
Posts: 174
Reply with quote
sherpya wrote:

b0ne if you want to make some tests, binaries are here:


I will give it a try. I have a simple MessageBoxA program that I will create a signature for and pack it with all the packers I have available to see investigate the import table functionality.
View user's profileSend private message
sherpya


Joined: 22 Mar 2006
Posts: 898
Location: Italy
Reply with quote
I've moved my stuff into a separate program:
http://oss.netfarm.it/clamav/files/exeScanner.exe http://oss.netfarm.it/clamav/files/exeScanner.exe
I've also added a signature check (for packed or whitelist)
if someone want to test it, it's a command line util, usage: exeScanner executable/dll
it also prints the first 16 bytes of the EP, the same data is checked against signatures list

some of false positives are fixed:
- ntdll.dll and kernel mode stuff - need to check some of pe headers [todo]
- resources only dll - partially fixed by entropy calculation, can be improved [todo]
- dotnet code - whitelisted by signature match [needs more checks]
- mingw executables - whitelisted by signature match [needs more checks]
View user's profileSend private message
Test Of Memory Scanning Improvement
GuitarBob


Joined: 09 Jul 2006
Posts: 4376
Location: USA
Reply with quote
When run, I get the following message "failed to create FileA failed 3"

Regards,
View user's profileSend private message
Memory Scanning Improvement
GuitarBob


Joined: 09 Jul 2006
Posts: 4376
Location: USA
Reply with quote
Forget my previous post--that was when running the file all by itself. When added to the command line in ClamWin, it ran okay with several programs in memory--no error message.

Regards,
View user's profileSend private message
sherpya


Joined: 22 Mar 2006
Posts: 898
Location: Italy
Reply with quote
@GuitarBob
exeScanner is a standalone program ;P
View user's profileSend private message
Memory Scanning Improvement
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
All times are GMT  
Page 2 of 4  

  
  
 Reply to topic