View Single Post
Neobeo

JCF Member

Joined: Sep 2002

Posts: 409

Neobeo is an asset to this forumNeobeo is an asset to this forum

Apr 2, 2006, 01:48 PM
Neobeo is offline
Reply With Quote
Compressing files
Overall, uncompressing shouldn’t be too difficult, since you already know both the exact compressed AND decompressed size. Trying to compress data however, is more difficult, because you don’t know how big the new zlib stream will be. However, the zlib manual states that the compressed size of a buffer will be at most (1.01 * size + 12) bytes big, regardless of compression level. Here we will use compress2, using a compression level of 9 (maximum).

Using compress2 in C++
For this short snippet we will assume that the in buffer already contains a string called "Let us...". We will compress this into zlib and turn it into a file that is similar to JJ2's file formats.

Code:
//char *in;
char in[] = "Let us attempt to compress this string";
char *out;
int CompressedSize;
int UncompressedSize;

int main() {
	UncompressedSize = sizeof(in);
	CompressedSize = (int)((float)(UncompressedSize * 1.01) + 12); //Allocate space

	out = malloc(CompressedSize); //Allocates a size of UncompressedSize for output
	compress2(out, &CompressedSize, in, UncompressedSize, 9); //Compresses, YAY :)
	//Note that CompressedSize now holds the real uncompressed size

	//Here "out" contains the compressed data, so do whatever you want with it.
	//For our example, we will write this to testcompress.dat
	FILE *fo = fopen("testcompress.dat", "wb"); //Opens testcompress.dat for writing
	fwrite(out, 4, 1, &CompressedSize);   //Writes the compressed size
	fwrite(out, 4, 1, &UncompressedSize);   //Writes the uncompressed size
	fwrite(out, 1, CompressedSize, fo); //Writes compressed stream to file
	fclose(out); //We finished writing to the file, no need to use it anymore
}
Again, if you are implementing this, remember to #include, and don’t forget to free what you malloc. If you are fussy and want your out buffer to be the right size, use realloc to adjust it.

Using compress2 in VB
Same thing as above, string to zlib, write to testcompress.dat.

Code:
'Dim inBuffer() As Byte
Dim inBuffer As String
Dim outBuffer() As Byte
Dim CompressedSize As Long
Dim UncompressedSize As Long

Sub Main()
	inBuffer = "Let us attempt to compress this string"
	UncompressedSize = Len(inBuffer) + 1 'Add a null-char to the end of the string
	CompressedSize = UncompressedSize * 1.01 + 12 'Sets maximum output size

	ReDim outBuffer(CompressedSize - 1) 'Allocates output space
	compress2 outBuffer(0), CompressedSize, inBuffer, UncompressedSize, 9 'YAY :)
	ReDim Preserve outBuffer(CompressedSize - 1) 'Adjusts output space

	'Here "outBuffer" contains the uncompressed data, so do whatever you want with it.
	'For out example, we’ll just do something simple - dump it into testcompress.dat
	Open "testcompress.dat" For Binary Access Write As #1 'Opens file for writing
	Put #1, , CompressedSize   'Writes CompressedSize to the first four bytes
	Put #1, , UncompressedSize 'Writes UncompressedSize to the next four bytes
	Put #1, , outBuffer        'Writes the output buffer to the file
	Close #1 'Finish writing to file, no need to use it anymore
End Sub
Again, the Visual Basic function looks slightly harder than C++. (inserts rant about how everyone using VB should start using VC++) Only thing to note is the use of ReDim Preserve. Remember how the UncompressedSize value changes after going through compress2? We adjust the buffer so that it fits exactly the compressed stream, and prevent any problems with Put #1, , outBuffer.

CRC Checking

Finally we get to the crc32 stage. All (or most) of the JJ2 files contain an internal (and some even external) CRC check to verify that the file is not corrupted in any way. Seeing how most programmers are lazy, they will probably skip the CRC mechanism and just assume that file is valid, but it will be included here just for completeness anyway.

Declare Function crc32 Lib "zlib.dll" (ByVal crc As Long, ByRef buffer As Any, ByVal bufferLen As Long) As Long

or, in C++ format,

uLong crc32 (uLong crc, const Bytef *buf, uInt len);


As you can see, the crc32 function takes 3 arguments. The first one is a previous crc value, in case we do the CRC check in chunks. For simplicity, we will always use CRC on the whole buffer at once, which means we leave CRC as 0. buf/buffer is the buffer, and len/bufferLen is the length of the buffer. If you’ve fully understood the uncompress and compress2 functions then there is no need for an example here, since the buffer and length arguments are used here in exactly the same way as before.

Final notes/other stuff
None of the above code has been tested. They were jotted down directly on notepad and have never gone through any stage of compilation. If there are any bugs within them, report them here. The posts below will give info regarding specific file format headers, but this information alone should already be adventurous enough. ;D
__________________
<TABLE border=1><TR><TD>Facts:
Jazz Sprite Dynamite (JSD)
Tileset Extractor
Neobeo's Firetruck

</TD><TD>Myths:
Jazz Creation Station Plus (JCS+) - 10%
Coming soon - a dedicated server! - 25%
Jazz Sprite Dynamite v2 (JSDv2) - 2%
Another generic single-player level - 0%
</TD></TR></TABLE>