Wednesday, July 28, 2010

Analysing Encrypted PDF Files for malware

I recently came across a PDF 306d7e608a52121aa4508e9901e4072e which on Virus Total only has 7/42 (16.67%) detection.

The sample uses PDF Version 4 standard encryption - AES V2 with a 128 bit key which leads us to wonder if some AV vendors are not handling the decryption to peak inside at the PDF content.

PDFs can be encrypted, with a key calculated from values in the PDF so it can be opened and viewed, compliant PDF readers will dissallow certain permissions such as copy/paste or printing (some documents can have a user password to open the file, but we won't get into brute forcing that type here). While obtaining the key is a relatively trivial calculation, the encryption level is quite good and will bypass typical NIDS and apparently some AV products.

Some of the resources we can use to figure out the PDF encryption key generation is the ISO 32000-1 document on the PDF structure which is freely available from Adobe.

For older PDF Version 1/2 encryption I found this resource. But this particular PDF has the /V 4, for the latest version where it's own filter can be specified, in this case we also have /Standard and /AESV2, so the PDF specs has the solution:

ISO 32000-1 2008 Encryption Key Algorithm 2:

PDF uses a standard user (/U) password padding. Now the document will have some content in the /U user password field, but you should first try the standard padding as the full password as most documents do not require a user password to view them.

Step A:
PDF password padding / default user password in algorithm: 28BF4E5E4E758A4164004E56FFFA01082E2E00B6D0683E802F0CA9FE6453697A

Step B:
Build a string with 28BF4E5E4E758A4164004E56FFFA01082E2E00B6D0683E802F0CA9FE6453697A to use in a md5 hash.

Step C:
Get the /O owner password - original is 30E714592E1FF1FE5C285A601B413C71CAF3B989576B72AC51371CF782A7F32F0B, however, there's a escape character 5C that needs to be stripped, giving 30E714592E1FF1FE285A601B413C71CAF3B989576B72AC51371CF782A7F32F0B, append that to the value above.

Step D:
"Convert the integer value of the P entry to a 32-bit unsigned binary number and pass these bytes to the MD5 hash function, low-order byte first."
The /P value is -3392, we're using PHP which doesn't have an unsigned int, so we'll have to do the math ourselves instead of a sprintf:
pow(2, 32) + $p_value = fffff2c0, when set to low order first we get c0f2ffff to append to our hashing string above.

Step E:
Append the first field of the document ID to the hash string: ID[] - we will append F247C9CF56A1484FB2C4F8C200FA269F to our hash.

Step F:

Step G:
Our final value is
"28BF4E5E4E758A4164004E56FFFA01082E2E00B6D0683E802F0CA9FE6453697A30E714592E1FF1FE285A601B413C71CAF3B989576B72AC51371CF782A7F32F0Bc0f2ffffF247C9CF56A1484FB2C4F8C200FA269F" (converted to data) and hashed with md5 is 26612c183ad8994a9600ada07e1299f2.

Step H:
Hash the hash 50 times:

step h 0 md5(26612c183ad8994a9600ada07e1299f2) = e62c3762cfab0ba2d3bfb201f3c3d263
step h 1 md5(e62c3762cfab0ba2d3bfb201f3c3d263) = 954a6bc4674dd8d93f836d9ad0e8f5c9
step h 2 md5(954a6bc4674dd8d93f836d9ad0e8f5c9) = d6a528e6572312c3f5c9b5cbe12aa343
step h 3 md5(d6a528e6572312c3f5c9b5cbe12aa343) = 9b6d871ae2f747f6cc7491ac24302c78
step h 4 md5(9b6d871ae2f747f6cc7491ac24302c78) = 188a87ddb2b9b939ed9d101cc3c09a7e
step h 5 md5(188a87ddb2b9b939ed9d101cc3c09a7e) = f495bdd808b927fe35d644e8f2971bbc
step h 6 md5(f495bdd808b927fe35d644e8f2971bbc) = ec8d5f00274875134a6a100e0429cdff
step h 7 md5(ec8d5f00274875134a6a100e0429cdff) = 830844b7c7b6276c062854ad1edebfdc
step h 8 md5(830844b7c7b6276c062854ad1edebfdc) = d446741e4b5bca850bc9645a7ee3b045
step h 9 md5(d446741e4b5bca850bc9645a7ee3b045) = 73c86126cf89e96b105caf5de627a018
step h 10 md5(73c86126cf89e96b105caf5de627a018) = 192744b0bade31e7e46cc8f5e4697fb4
step h 11 md5(192744b0bade31e7e46cc8f5e4697fb4) =
step h 12 md5(b04e410c2c5f327650053c942c4b8fe7) = 038e96e7289bddcfa4f8a96e0a3ca1f7
step h 13 md5(038e96e7289bddcfa4f8a96e0a3ca1f7) = fd0ff1f4284e7aa9e9b11ddaf7e69350
step h 14 md5(fd0ff1f4284e7aa9e9b11ddaf7e69350) = cf2f8e2c3a57260bde21a28adf1d7c05
step h 15 md5(cf2f8e2c3a57260bde21a28adf1d7c05) = 4e7a8c7572985f90d0e903a4d9dd957d
step h 16 md5(4e7a8c7572985f90d0e903a4d9dd957d) = 871eff58e5ebc751fba8ad222ba97181
step h 17 md5(871eff58e5ebc751fba8ad222ba97181) = 1bd8326a037a5c8f3d8624123d80d7d2
step h 18 md5(1bd8326a037a5c8f3d8624123d80d7d2) = 509f4173ba89b3d8a8b30bff75604ab5
step h 19 md5(509f4173ba89b3d8a8b30bff75604ab5) = 9c0153ee14f16a3ab2f6932a51ed347a
step h 20 md5(9c0153ee14f16a3ab2f6932a51ed347a) = 12931d398d6d1abd44e49207684fc840
step h 21 md5(12931d398d6d1abd44e49207684fc840) = 7b9e11e1ad4f1a42ccf6f5a5f604df2d
step h 22 md5(7b9e11e1ad4f1a42ccf6f5a5f604df2d) =
step h 23 md5(6a02459568ce254fb717983ef92b173d) = 59a73d2798871876343d7eea5a723102
step h 24 md5(59a73d2798871876343d7eea5a723102) = 00b1aca101d3a7641b4f98c1f65c9ded
step h 25 md5(00b1aca101d3a7641b4f98c1f65c9ded) = 9ae87a76ba1af438af75ba2011727827
step h 26 md5(9ae87a76ba1af438af75ba2011727827) = 0a8090edc5aebc1521046d3f2c8913a7
step h 27 md5(0a8090edc5aebc1521046d3f2c8913a7) = 87c38f682ecaa9eb4bc4a9adb5d92552
step h 28 md5(87c38f682ecaa9eb4bc4a9adb5d92552) = fe140891a75402580507fc8f8db6ec17
step h 29 md5(fe140891a75402580507fc8f8db6ec17) =
step h 30 md5(2cf639b306532be03a1481ad104aa3e2) = eace2ceeb9e7c8e5900cb3b1ba4e3904
step h 31 md5(eace2ceeb9e7c8e5900cb3b1ba4e3904) = 46695a7f26b3a1874e17d6bf9d044a1b
step h 32 md5(46695a7f26b3a1874e17d6bf9d044a1b) = 775aaaaf5d8ed731400e777d47a9f730
step h 33 md5(775aaaaf5d8ed731400e777d47a9f730) = 32c27b94921ef5f21c4538c6043a3635
step h 34 md5(32c27b94921ef5f21c4538c6043a3635) = 1710f9661da8b8c843bf20100b5cd58a
step h 35 md5(1710f9661da8b8c843bf20100b5cd58a) = 32dd4dfb99e429399540e460664343e3
step h 36 md5(32dd4dfb99e429399540e460664343e3) = 628d80b8dce1084b816dad348de26b4d
step h 37 md5(628d80b8dce1084b816dad348de26b4d) = 0d484724b53c014bf044e7ab5fac4414

step h 38 md5(0d484724b53c014bf044e7ab5fac4414) = aa561b8f3211da0994b790eba2eedd11
step h 39 md5(aa561b8f3211da0994b790eba2eedd11) = 0969b6eb986d13e5965e57b818320227
step h 40 md5(0969b6eb986d13e5965e57b818320227) = aca2a9b85c975ca6d8b2ff1c848e93a0
step h 41 md5(aca2a9b85c975ca6d8b2ff1c848e93a0) = 60f6b7bd36240d64ea47d507bbba0241
step h 42 md5(60f6b7bd36240d64ea47d507bbba0241) = 17eb81278c5287ec1f85e6b6a4d584c2
step h 43 md5(17eb81278c5287ec1f85e6b6a4d584c2) = ca5756e27675308b3de57cf1695e7c8a
step h 44 md5(ca5756e27675308b3de57cf1695e7c8a) = 7353248cb20ab14cea5eee3674a7eafb
step h 45 md5(7353248cb20ab14cea5eee3674a7eafb) = 0eac9698646535097bc8a5c66a02a921
step h 46 md5(0eac9698646535097bc8a5c66a02a921) =
step h 47 md5(210e8578482403a49e414904980ec15a) = 7cf558c4d6af2b36cbc52b5381b98e85
step h 48 md5(7cf558c4d6af2b36cbc52b5381b98e85) = 53dede5bec03d50f3441b31c955f2121
step h 49 md5(53dede5bec03d50f3441b31c955f2121) = f5834bfee58498420a2128ea4edcd346
step h final hash f5834bfee58498420a2128ea4edcd346

Step I:
Use the full 128 bit length, so final key is f5834bfee58498420a2128ea4edcd346.

Using the key:

ISO 32000-1 Algorithm 1: Encryption of data using the RC4 or AES algorithms
Step A:
Looking at the image above, we get the object number of 47 and generation number 0.
As hex we get object number as 3 bytes 000038 and generation number as 2 bytes 0000, low order becomes:

Step B:
md5 (f5834bfee58498420a2128ea4edcd346 + 3800000000 + 73416C54) (73416C54 is hard coded padding for any object)
Result is 9edcf26e53cf4d286c6aad5f0b4821dd

Step C:
Read in object form stream to enstream, trim \x0a and \x0d
$decrypted = mcrypt_decrypt(
substr($d, 16),
substr($d, 0, 16)

And if there's a Filter/FlateDecode:
gzinflate($decrypted) (you may need to chomp the first one or two bytes to get the gzinflate to work.)

And that's it, a quick and easy how to decrypt PDF malware.

Tuesday, July 13, 2010

New blog

Welcome to our new blog, hopefully we'll have some insight on malware analysis related topics and trends. For now, check out our web-based shellcode analysis and unpacker tool at