Compression and Encryption: The ZIP Years.

In a comment to my last post Compression and Encryption, nhinkle asked:

Do you know then how encrypted ZIP files work? Encryption seems to be built into many encryption formats like zip, rar, 7z, etc. Do these usually compress and then encrypt, or somehow do both at once?

Well, ZIP handles this in its own special way. First let’s look at how a ZIP file is made up. A ZIP file consists of one or more ‘file entries’ – blocks of data that make up the actual content of the zip file, followed by a final ‘central directory’:

As you can see each file in the ZIP file has its own local header which contains the information about how the file is compressed. This allows each file in the ZIP file to be compressed in a different way – from “Store” (no compression – ideal for adding pre-compressed files) right up to the maximum and slowest compression available.

Encryption information is also stored in the file header (the decryption password and the encryption schema). This allows different encryption to be used for different files in the zip file. You could, for instance, have a ZIP file which has some files publicly readable, plus some files which can only be extracted with a password, and even some other files which require yet another password:

$ unzip test.zip
Archive:  test.zip
creating: Architecture/
[test.zip] Architecture/img14.jpg password:
inflating: Architecture/img14.jpg
inflating: Architecture/img18.jpg
inflating: Architecture/img16.jpg
inflating: Architecture/img17.jpg
inflating: Architecture/img15.jpg
inflating: Architecture/img13.jpg
creating: Characters/
[test.zip] Characters/img22.jpg password:
inflating: Characters/img22.jpg
inflating: Characters/img24.jpg
inflating: Characters/img20.jpg
inflating: Characters/img23.jpg
inflating: Characters/img19.jpg
inflating: Characters/img21.jpg
creating: Landscapes/
inflating: Landscapes/img8.jpg
inflating: Landscapes/img7.jpg
inflating: Landscapes/img11.jpg
inflating: Landscapes/img12.jpg
inflating: Landscapes/img9.jpg
inflating: Landscapes/img10.jpg
$

In the above example the files coloured red are encrypted with one password, the blue ones with another password, and the green ones aren’t encrypted at all.

So what does this mean as far as the order of encryption and compression goes? Well, it doesn’t actually tell us anything about that, but we know from my previous blog post that if compression is to have any chance at all it has to be done before the encryption.

But what about the security of this arrangement? Well, it means:

• Only the file blocks are encrypted
• This leaves the central directory unencrypted
• Anyone can read the directory structure and know the names of the files
• This leaves the ZIP file open to plain text brute force attacks.

You might think this is a very serious and fundamental flaw in the system. Well, you’d be right. That is why alot of modern ZIP implementations (such as 7z) give you the option of encrypting the central directory as well. It does mean that you have to have 1 master password for the entire ZIP file, so you lose the ability to have some of the file encrypted and some not, but it does make the whole thing more secure.

So, to sum it up:

Incoming data to the ZIP file is first compressed, then encrypted. A local file header is added to the start of this encrypted data describing how the file is encrypted and compressed. The data is then fed into the ZIP file and a pointer to that data’s location is added to the Central Directory. As an optional extra the Central Directory is then encrypted as well.