WORKS - Recognition of a packed file works fine, I didn't test it much.

PARTIAL - Dumping of the executable from memory works fine... however....the file is still out of alignment. Each section of the dumped file needs it's PE header PointerToRawData reset to match the RVA value of the section.

WORKS - Scanning of the dump works fine, except when I use an exact offset, because the PointerToRawData is not at the proper value, fixing the PE headers will fix this.

I created a simple MessageBoxA program, compiled it, and packed it with each of the following packers. ExeScanner recognized all of the files as packed. Here's the list:

Sample_aspack.exe
sample_diminisher.exe
sample_eXPressor.exe
sample_fsg.exe
sample_mew.exe
sample_neolite.exe
sample_nspack23.exe
sample_nspack36.exe
sample_pecompact277.exe
sample_pepack.exe
sample_pespin1304.exe
sample_pslist_petite.exe
sample_pslist_shrinker.exe
sample_sdprotector.exe
sample_upack.exe
sample_yodascrypter13.exe


Where did these signatures come from?

do u known how to align the file?
signatures are made by hand by looking at some samples,
the scanner can detect file as packed also without matching a sig

b0ne if you want to look at the code, it's here now:
https://clamwin.svn.sourceforge.net/viewvc/clamwin/trunk/clamav-release/contrib/msvc/src/exeScanner.c?revision=881&view=markup https://clamwin.svn.sourceforge.net/viewvc/clamwin/trunk/clamav-release/contrib/msvc/src/exeScanner.c?revision=881&view=markup

I don't know how much of it I can explain if you aren't familiar with PE-header structures. Here is an excellent tutorial, pay specific attention to "Tutorial 5: Section Table" https://spiff.tripnet.se/~iczelion/tutorials.html https://spiff.tripnet.se/~iczelion/tutorials.html

Here's my attempted explanation.

The PE-Viewer tool that I use on a regular basis is called PE-View: https://www.magma.ca/~wjr/PEview.zip https://www.magma.ca/~wjr/PEview.zip I'm going to use files that I already posted here as examples. https://www.cloverbasin.com/files/clamscan_example.zip https://www.cloverbasin.com/files/clamscan_example.zip

I'm going to paste standard output from the tool GT2 https://philip.helger.com/gt/program.php?tool=gt2 https://philip.helger.com/gt/program.php?tool=gt2 which has a /peall switch to dump out pe header structures used for the examples below.

EXAMPLE: mewpad.dmp-unfixed.exe
This executable was dumped from memory using PE-Tools without fixing the alignment. (Essentially GetModuleSize and BaseAddress, which are passed to ReadProcessMemory, then passed to WriteFile, how you are doing it now.)

The MEW section is where the original NOTEPAD code is eventually written. "OuUS‰O" is the name (non-ascii hex bytes mis-interpreted) of the section where the MEW decompressor resides with the packed data.

When the image is loaded by the windows loader, the "OuUS" section is read from disk at position 0x200 and written to the following memory address represented by this math: image base address + section RVA (relative virtual address). This works out to be 0x1000000h + 0x14000h = 0x1014000h.

The compressed bytes of the "OuUS" section in the packed/compressed file are stored at 0x200h. However, after we dump the image from memory to disk using ReadProcessMemory, these bytes are at file position 0x14000h (because the section is loaded in memory at base + 0x14000h and there are now 0x13FFF bytes before these bytes). The pe-header contained within the dump from ReadProcessMemory still says the section is stored on disk at 0x200h. This is wrong. We need to correct this 0x200 value (PointerToRawData) to match the RVA value, 0x14000.

We need to do this for all of the other sections listed in the file as well. Each sections PointerToRawData value must match the in memory RVA. (ReadProcessMemory creates a mirror of the IN MEMORY image to disk, so the sections reside at IN MEMORY locations, not at the locations described by the original PE-header.)


EXAMPLE: mewpad.dmp.exe
This executable was dumped using PE-Tools, but has the sections realigned. (I mentioned this before in a previous post about RebPe32.dll, and "DumpFix" function contained within that DLL.)

Here is what the sections look like after they have their PointerToRawData value's re-assigned to the section RVA's.


See how the MEW section is at RVA 0x1000h AND the PointerToRawData (phys off) value is 1000h? The Pe-header for that section now point to where the data resides in memory and on disk after a memory dump(!) rather than where it is was previous compressed and/or non-existant on disk. The image in memory is significantly different from on disk BEFORE it is decompressed and written to where all the data is supposed to exist.

You can see section alignment illustrated in the following screenshot after MEW has decompressed notepad.exe:

it doesn't seams so easy, how can I get the oep? address of entry point still refers the stub EP
e.g. I have a simple msgbox prog compiled with mingw
EP sig is 5589E58B0DD05040

dumped (and packed) file has EP @ 10fc (0x400 in file) in the first
section that has:
vsize 0x430
voff 0x1000
rawsize 0x600
rawoff 0x400

the upx EP sig is 0x60 0xbe (first 2 bytes)
I get it @ 0x7420 (0x620 in file)
some values in the pe header are still wrong like sizeofrawdata

the eop of dumped file is in file @ 0x1000
I have same value for UPX0 voffset and UPX1 virtualsize (it's related?)

so first of all there is a way to have 0x1000 by reading dumped exe headers?
then I may only need to change the AddressOfEntryPoint to match the oep
this should be made using relative virtual address that point to eop
right now I still need to read docs to make raw -> rva, but this shouldn't be so diffcult.
it may be usefull also to correct size and offset of each section, but even there I dunno where to find/calc them,
but I would to fix at least entry point (or you think other fixes are mandatory?)
I'm still new in PE analysis I'm adding code step by step while I'm understanding PE structure and similar info

You can't get the OEP without being familiar with the packer, but that is OK.



Without correcting the values, the sections are misaligned which makes scanning them useless because no offsets will match SECTION+OFFSET signatures.


There is some code in clamav which brings back section and RVA's in PE.C that we could steal, there is also a C++ library called PeLib that could be used. It has functions for performing pe-header manipulations.


It's easy to automatically find the entrypoint on UPX executables, but others it is not so easy, you would have to have a custom routine for each packer.


_The_ most important step is to fix the sections PointerToRawData to the sections RVA value so that the pe-header says the data resides on disk where it really is located.

ok I've seen PELib it seams intresting but is cpp I cannot use cpp code in clamav, the code is not so simple to rip since it uses template, anyway
do you known a simple way to fix RVA values in sections?
Also I can get the eip on the upx packed files by searching for:
61 popa
E9 889BFFFF jmp exe.004010FC
and reading the address,
currently I have no clue to add it as ep, since I only known the offset in the file and also it uses strange values in jmp:
I have @0040756F
JMP da2.004010FC

E9 889BFFFF
where e9 is jmp, ffff should mean that the address is relative the pe image
but 889b is not anyway an offset from 004010FC to 0040756F
I suspect I can calculate it when I'll have new section.

right now I have to reconstruct sections with correct values.
I'm looking at clamav upx unpacker but win32 unpacker code in clamav
is almost unreadable since some stuff is a rude conversion from asm to c

Then I have the way to find oep offset on disk (for now only upx, but other packers can be added too),
but I need to known how to calculate the ep rva
I have the raw ep at 0x1000 but clamav dumped file has:
EP rva 0x10fc and EP raw 38c that currently points to eop
(In my programs, mingw compiled the ep signature starts with 5589

you seams to known enough about pe and coding, your help would be appreciated

Finally I think that most common packers can be supported since I can check for "last_jmp" signature if possible,
this will off course not work with advanced packers, but there are many hacked upx packed executables around
and all of upx derivates should have 61 E9 889BFFFF as last jump code

Yes, hopefully if we can steal libclamav's code.

In pe.h, there is this function:


There is a struct here called "cli_exe_info", found in execs.h:


The cli_exe_section struct is also in execs.h:



It seems this cli_peheader() function obtains the information we need for each section? All we have to do is read these values from the file, and write over raw with rva and rsz with vsz. This forces the pe-header to match the positions and sizes from memory.


As for the jump question stuff:



889bffff is a little endian negative offset. the human readable value is FFFF9B88.

I don't remember the exact way the CPU handles the hex math of the negative value, but if FFFFFFFF is -1, then FFFF9B88 is that many values less than 0. (6478h) x86 offsets are calculated from the address AFTER the last byte of the last operand. The jmp instruction is 5 bytes long, so the address which it calculates from is 407574. (407574 - 6478) = 4010FC.



The OEP or just the EP? I wouldn't bother with the OEPs. The EP is already an RVA so I'm not sure why you need to calculate anything for that? Scanning for packer signatures?

Before calculating the on disk position of the dumped EP, you need to know which section the EP's RVA will fall within. Using libclamav names (ep >= rva && ep < (rva + vsz)).

To calculate the EP on disk you could do something like this: (presuming the sections are aligned where the section on disk values match the in memory values) EPondisk = (rva - ep) + raw. rva should be the same value as raw if we've aligned the file.


I am a reverser, I understand how code works in ASM the best, but have yet to spend much time learning c/c++ syntax to be of much help. I guess I should probably get better at coding eh?


Being that the code is now unpacked, all signatures are just calculated offsets to a position in the file based on the file base position 0, end of file, section, or entrypoint. Changing the EP value for only some dumps means signature writers have to be aware of when that change happens and create a sig from OEP rather than EP. To be consistant, it seems it would be wise to just leave the EP from the packed file in place.

hmm I forgot to byteswap the value, negative hex are handled using two's complement, like
~x + 1
so
~0xffff9b88 + 1 = 0x00006478

I would to find the oep to have directly the signature of the unpacked exe not the signature of the unpacker,
I can do packer signature by scanning on the packed file, anyway what are you saying is reasonable,
leaving ep as is can be universal while oep is not always possible to find

you said we need to fix section alignments but right now I dunno how to pick them from memory, the pe header should be
unchanged or the unpacker stubs needs always to make some change to make the exe work?
Finally yes, I may be more able to code in c, but your PE executable knowledge is for sure better than mine, so
you can help me a lot.
The problem is that we need additional signature of unpacked malwares, and I doubt clamav team will add sigs for
in-mem dumps, also aCaB the main devel that made unpackers code and main work on scanpe stuff doesn't like me
too much for some posts I made on the clamav ml

What does the OEP signature do for us? Writing signatures on it is usually unsafe unless it is a very specific malware entrypoint, not c runtime startup code that calls main(), or WinMain().


We don't need to get it from memory, we get them from the packed pe-header on disk.

My previous post explains the structure members that are returned by the function cli_peheader() function from libclamav, and what calculations to be made to align the pe-header of the dump to match where the data is in memory. These structure members are: rva, vsz, raw, rsz. Relative Virtual Address, Virtual Size, Raw offset on disk, raw size.

Every section needs to have the following:
RAW equal to RVA
Raw Size equal to Virtual Size

The dump will be aligned.




Perhaps if we contact each other off the forum to discuss the problem a little more it would be more effective?



Some signatures from clamav that are placed on unpacked data would still work nicely when malwares change packers from UPX to one clamav does not support.

But yes, clamwin would still need a new database for signatures that are created for dumps. Also people to write signatures for the that database. I would help, but do not nearly have the time to work on every malware sample. I am unfamiliar with the clamav signature creators, would they communicate and work with me to provide such usefulness for end-user benefit?


Uh oh! clamav is GPL, the point of it is so other people can change it and reuse code? What is he mad about?

do you mean something like:

UNFIXED:



and FIXED:



I've roughly made:

but I'm not sure about it, also look at the EP(raw) it becomes = to EP(rva)

I've also looked at cli_peheader() it does nothing of interesting, section data is copied to peinfo sections
and the only thing that it does is EP offset calculation on disk

I've made updated bins @ https://oss.netfarm.it/clamav/ https://oss.netfarm.it/clamav/
don't pick devel version but 0.88.6
I'm still unsure about some files.
--debug to see the exeScanner output --leave-temps leaves dumped files in temp

I think you might be interested in the following link to a recent Black Hat presentation on runtime packers by a guy from F-Secure:

https://www.blackhat.com/presentations/bh-usa-06/BH-US-06-Morgenstern.pdf

Hope it gives you some good info--at least for the future.

Regards,

If you're really interested, there's a better paper on them by paul craig from ruxcon 2006, it explains a lot more of how they work rather than "packer detection is a problem."