View Single Post
Neobeo

JCF Member

Joined: Sep 2002

Posts: 409

Neobeo is an asset to this forumNeobeo is an asset to this forum

Apr 2, 2006, 01:46 PM
Neobeo is offline
Reply With Quote
JJ2 File Format Specifications

Edit: I just slipped this tileset extractor on top so people can see it.

Well, I was working on a file-format entry for the ERE, but the formatting never came out the way I wanted it. Seeing as I'm in somewhat of a rush, I've decided to dump my article here, in the hopes that someone can move it to ERE or some other public article.

Zlib Compression in JJ2
This article will give you a crash course to understanding zlib, how it can be compressed / decompressed and how it is implemented in JJ2. You need either zlib.lib and zlib.h if you’re programming for C++, or zlib.dll if you’re programming for Visual Basic. Armed with this knowledge alone, it is possible to decrypt JJ2 files into their raw formats, and then figure out the formats from there.

Declaring zlib functions
Just about every programming language requires you to declare a function before you can use it. Notes about declares: The compress function will not be used at all, and is only provided for closure. This is because it is superseded by compress2, which allows compression at higher levels. Also, if you do not care about the integrity of your files, you can ignore the crc32 function and skip all crc checks. Without further ado:

Declares in C++
Code:
#include <zlib.h>
Declares in VB
Don’t forget to add "Private"s in front of them if you’re going to use them in your forms.
Code:
Declare Function compress Lib "zlib.dll" (ByRef dest As Any, ByRef destLen As Any, ByRef src As Any, ByVal srcLen As Long) As Long
Declare Function compress2 Lib "zlib.dll" (ByRef dest As Any, ByRef destLen As Any, ByRef src As Any, ByVal srcLen As Long, ByVal Level As Integer) As Long
Declare Function uncompress Lib "zlib.dll" (ByRef dest As Any, ByRef destLen As Any, ByRef src As Any, ByVal srcLen As Long) As Long
Declare Function crc32 Lib "zlib.dll" (ByVal crc As Long, ByRef buffer As Any, ByVal bufferLen As Long) As Long
How compression is used in JJ2 files
As a bare minimum, all zlib-compressed files in JJ2 files should at least provide the compressed and uncompressed sizes in its header. For example, consider an example zlib section of a file which goes:

0000
0010
0020
0030
0040

44 00 00 00 4F 00 00 00 78 DA E3 60 80 00 17 73
69 30 ED F0 53 98 41 E0 97 30 03 23 90 DD 92 53
C6 78 E7 87 30 C3 7B 31 47 06 35 89 03 FC 20 F9
04 0A DC DB 3C 07 B0 5A 01 55 39 06 95 04 25 86
2B 40 1A 0C 9E 30 00 00 2B 02 0E A6

D...O...X..`...^
%0..S.A..0.#...S
.x..0.{1G.5... .
....<..Z.U9...%.
+@...0...Q.[


In this self-made example, the first four bytes (which reads 68 in hex) represents the size of the compressed stream (the input data). The next four bytes (which reads 79 in hex) represents the size of the uncompressed stream (the output data). Armed with that data, we now know that we will be decompressing a 68-byte buffer into a 79-byte one. Note that zlib streams starts with a "78 DA" when compressed at the maximum level, and this can be very useful for skimming through large files to find out where a zlib stream begins or how many zlib streams there are.

In case you are already confused, a zlib stream is basically an array of bytes which contains compressed data, and is very unfriendly to the human eye. Fortunately, there is a very simple function which can be used to decompress a zlib stream into a buffer. This function is uncompress.

Using uncompress in C++
For this short snippet we will assume that you have the above "example" file stored as "tutorial.dat". This assumes you are writing a console program and have already #included anything necessary.

Code:
char *in;
char *out;
int CompressedSize;
int UncompressedSize;

int main() {
	FILE *fi = fopen("tutorial.dat", "rb"); //Opens tutorial.dat for binary reading
	fread(&CompressedSize, 4, 1, fi);       //Reads the first four bytes and stores it
	fread(&UncompressedSize, 4, 1, fi);     //Reads the next four bytes and stores it

	in = malloc(CompressedSize); //Allocates a size of CompressedSize for input stream
	fread(in, 1, CompressedSize, fi); //Reads CompressedSize bytes from the file
	fclose(fi); //We finished reading the file, no need to use it anymore

	out = malloc(UncompressedSize); //Allocates a size of UncompressedSize for output
	uncompress(out, &UncompressedSize, in, CompressedSize); //Decompresses, YAY :)

	//Here "out" contains the uncompressed data, so do whatever you want with it.
	//For simplicity, the above example is actually a string, so print it to screen:
	printf(out);
}
That wasn’t too bad was it? Once you can understand the concepts of malloc and uncompress, you can decompress just about every file format in JJ2. Don’t forget to free what you malloc though. Note that we pass a pointer to the second argument of uncompress, which will return with the uncompressed size of the buffer. Assuming you already had the right size in the first place, this value will not change. Also, the return value of uncompress determines whether the operation was a success or not. However, to keep things simple, we assume that our zlib stream is valid, and will not error-handle it. Note: I would usually prefer to use gzread(look it up on google), but chose uncompress because it is compatible with VB.

Using uncompress in VB
Here we use the same assumptions: a file tutorial.dat with containing that binary data.

Code:
Dim inBuffer() As Byte
Dim outBuffer() As Byte
Dim CompressedSize As Long
Dim UncompressedSize As Long

Sub Main()
	Open "tutorial.dat" For Binary Access Read As #1 'Opens tutorial.dat for reading
	Get #1, , CompressedSize   'Reads the first four bytes, stores in CompressedSize
	Get #1, , UncompressedSize 'Reads the next four bytes, stores in UncompressedSize

	ReDim inBuffer(CompressedSize - 1) 'Create buffer of CompressedSize bytes
	Get #1, , inBuffer 'Reads bytes into inBuffer
	Close #1 'Finish reading from file, no need to use it anymore

	ReDim outBuffer(UncompressedSize - 1) 'Create buffer of UncompressedSize bytes
	Uncompress outBuffer(0), UncompressedSize, inBuffer(0), CompressedSize 'YAY :)

	'Here "outBuffer" contains the uncompressed data, so do whatever you want with it.
End Sub
The VB function actually looks slightly harder than the C++ version. One thing to note is that ReDim uses "x-1" bytes when you want to allocate x bytes, because the array goes from buffer(0) to buffer(x-1).
__________________
<TABLE border=1><TR><TD>Facts:
Jazz Sprite Dynamite (JSD)
Tileset Extractor
Neobeo's Firetruck

</TD><TD>Myths:
Jazz Creation Station Plus (JCS+) - 10%
Coming soon - a dedicated server! - 25%
Jazz Sprite Dynamite v2 (JSDv2) - 2%
Another generic single-player level - 0%
</TD></TR></TABLE>

Last edited by Neobeo; Apr 3, 2006 at 12:25 AM.