Author
|
Topic: Bypassing compression? (Read 1 times)
|
elixirnova
Guest
|
 |
« on: May 28, 2008, 01:44:12 am » |
|
I was wondering if anyone would guide me in the right direction.
I was thinking about making a decompresser/re compressor for editing the compressed data in megaman X3. But after thinking a bit it surely will not be that easy! The length of the data will most definitely change when the graphics or data is edited and then re compressed. It may not even fit in its previous location! There lies the problem!
The only idea that pops into my mind is expanding the ROM and changing the pointers to point to a larger area for the data to be stored. Also, if I am going to do this would it just be easier to bypass the compression routine as well and store all the data uncompressed in the expanded ROM and empty area? Is that even possible?
If that is what I need to do how do I go about doing it? I'm guessing I would number one need to change the pointer table to point to a location at the end of the ROM once I expand it and give more room to the compressed data. Or is it possible to go to the beginning of the compression routine and write my own routine in its place to simply load uncompressed data instead of compressed data?
Also, is expanding the ROM past some point make it crash easily or anything?
|
|
|
|
|
Ryusui
Guest
|
|
« Reply #1 on: May 28, 2008, 02:27:53 am » |
|
There's a simple way to determine if a ROM can be expanded.
Do the pointers for the data you want to move include a bank byte, or not? In the former case, ROM expansion is easy: all you have to do is update the pointers so they look in the right bank. In the latter case, it's impossible without some ASM hacking. The Breath of Fire 2 retranslation, it should be noted, features several such hacks.
A SNES ROM can be expanded up to 4MB in size. Note that some games managed to fudge their way around this with special hardware in the game cartridge, but as far as you're concerned, it's an ironclad upper bound. Crashing shouldn't be a problem, unless you really don't know what you're doing and have to write a routine to handle loading 24-bit pointers regardless. ^_^; (Also, if your recompressor is faulty, corrupt data might cause the game's decompression routines to crash.)
|
|
|
|
|
elixirnova
Guest
|
|
« Reply #2 on: May 28, 2008, 03:05:18 am » |
|
So since the pointers have the bank in them it should be easy! I am hacking MMX3 by the way.
Got any ideas on bypassing the compression and just loading normal snes gfx? I will try to read up on it tomorrow on my own, maybe figure something out.
Edit: By the way I just decompressed about all the GFX and in all I think they take up about 1mb! Which would put the ROM right around 3mb probably!
|
|
|
|
« Last Edit: May 28, 2008, 03:21:20 am by elixirnova »
|
|
|
|
Nightcrawler
Guest
|
|
« Reply #3 on: May 28, 2008, 07:25:10 am » |
|
You might want to try this. I'm assuming you must have missed this in your search through the documents section... http://www.romhacking.net/docs/381/ |
|
|
|
|
DaMarsMan
Guest
|
|
« Reply #4 on: May 28, 2008, 12:05:28 pm » |
|
You're going in the right direction. Working with expanded stuff like this can be a lot easier if you have lots of room.
|
|
|
|
|
HyperHacker
Guest
|
|
« Reply #5 on: May 28, 2008, 11:30:47 pm » |
|
A SNES ROM can be expanded up to 4MB in size. Note that some games managed to fudge their way around this with special hardware in the game cartridge, but as far as you're concerned, it's an ironclad upper bound. Doesn't Lunar Magic have an option to expand a ROM to 6MB by just rearranging banks and changing the header? Can that not be used on all games? I don't know SNES very well, just thought I'd throw that out there.  |
|
|
|
|
elixirnova
Guest
|
|
« Reply #6 on: May 29, 2008, 02:46:05 pm » |
|
Wow that chat log is pretty complex, but so far I have expanded the rom to 4MB (anything past wont work, and I am using Lunar Expand) I placed the title screen GFX decompressed at the beginning of the end of the rom (in the expanded area) and right now I just traced the decompression routine and found out where the CPU jumps to the decompression routine. I am a bit unsure about the pointer table though because at some point the game used it to store the location and "length"(used to decompress the block of gfx) of each compressed gfx block.. I guess I gotta read the asm @_@ some more.
I think I'm getting close though. Supposedly all that I need to do now is find the point just before it decompresses the gfx and replace it with a routine that just takes the gfx and stores it straight to VRAM?
Thinking about it now... Instead of loading the compressed data to RAM and having it be decompressed and sent to the VRAM the log says I should have it load uncompressed graphics straight to VRAM. Guess I have quite a bit more to learn because I don't have a clue how to do that! I think it is explained more in the log. Maybe I can figure this out soon! It will sure make hacking and making an editor TONS easier...
|
|
|
|
|
Tauwasser
Guest
|
|
« Reply #7 on: May 29, 2008, 11:47:36 pm » |
|
Well, as I see it, it always depends on what you want to achieve. How many gfx do you have to edit and put uncompressed in the rom? If it`s quite a few, you might be better off with writing a decompressor/recompressor and work with that instead of changing ever little occurrence of the decompression routine call to an uncompressed version (which might bring additional trouble, vblank issues and so on).
Also, it lacks quite a bit of professionalism in my opinion, but that`s that :rolleyes:
cYa,
Tauwasser
|
|
|
|
|
elixirnova
Guest
|
|
« Reply #8 on: May 30, 2008, 12:01:32 am » |
|
Well if I leave the graphics compressed in the rom, then I have the problem of not knowing if it will become significantly larger when I let someone edit them. On that note every graphic for levels and enemies and items are compressed. I suppose another route might be to simply change where the compressed graphic pointer table points to. I could say increase the room each block of graphics has before it overwrites the next block of data? I suppose that sounds like a lot easier option without removing the compression routines. On a side note I found out that the actual title screen for Mega Man X3 in compressed graphic block is "Mega Man X2" and a separate block "3" is stored... I have a feeling this means X, X2, X3 all have the same inner working in alot of ways... I'll try to expand the room for compressed graphics and maybe move some of them to the end of the file in the new expanded area...  :banghead: :banghead: EDIT: Wow, I'm pretty sure moving banks and editing the pointers is just as hard as redoing everything. I suppose I should just write a little program to do all this work for me... Moving 221 blocks of graphics around and rewriting their pointers is gonna take alot of time! I think it's taking around 1-2minutes to move each one, and that is at top efficiency by hand! I do admit it is well worth it since I will feel more comfortable about not having any glitches when I reinsert edited graphics. I'm definitely going with writing a program now that I think of how easy it is!! :thumbsup: I decided to multiple the compressed graphics blocks by 1.5 (so giving them x1.5 the space before the next block starts) in case when the graphics are edited they take up more space when reinserted. I'm no expert at compression sizes but I highly doubt any of this is compressed passed twice it's beginning size, but what do I know? If I do go with 1.5 times length It will take up about 1.6mb of data as opposed to about 1mb of compressed gfx. So that leaves .5mb at the end of the rom if I want to add anything later, maybe some more graphics or an asm hack? Also I will now have the 1mb in the middle of the rom to play wiht now that there are no gfx there! So that still leaves me with 1.5mb of expansion to work with. Not so sure that more levels are needed in a Megaman X game, but maybe different weapon asm or hacking upgrades or I dunno... |
|
|
|
« Last Edit: May 30, 2008, 02:22:16 am by elixirnova »
|
|
|
|
Ryusui
Guest
|
|
« Reply #9 on: May 30, 2008, 01:19:02 am » |
|
Wow, I'm pretty sure moving banks and editing the pointers is just as hard as redoing everything. I suppose I should just write a little program to do all this work for me... Moving 221 blocks of graphics around and rewriting their pointers is gonna take alot of time! I think it's taking around 1-2minutes to move each one, and that is at top efficiency by hand! I do admit it is well worth it since I will feel more comfortable about not having any glitches when I reinsert edited graphics.
It pays to write your own custom utils. Sylvanian Melodies would have been a pain in the ass if I had to insert all those recompressed graphics by hand: my app not only compresses, but batch-reinserts them. My compression routine isn't perfect, but I had plenty of leeway. |
|
|
|
|
elixirnova
Guest
|
|
« Reply #10 on: May 30, 2008, 10:59:20 pm » |
|
Well so far a success!
I wrote a utility to expand megaman x3 to 4mb and move all the compressed blocks of GFX to the end of the rom and expanded the room for each block so recompression/insertion will be no problem.
I am just about done writing a recompression routine too.
Thanks for all the help!
|
|
|
|
|
Ryusui
Guest
|
|
« Reply #11 on: June 01, 2008, 08:25:07 pm » |
|
Recompression is the hard part. ^_^; It's easy to write a decompressor - the game provides you its own blueprint for that - but getting a decent recompressor up and running takes time and effort.
|
|
|
|
|
elixirnova
Guest
|
|
« Reply #12 on: June 01, 2008, 10:44:35 pm » |
|
I just was starting to think about how to do this... and I cannot IMAGINE how on earth I begin... I realize there is no easy way to begin though with reverse engineering anything... But knowing what it looks like in the end and the beginning will help alot! So I decided to start with reading up on 4bpp format. I found this table explaining 4bpp... Colors Per Tile - 0-15
Space Used - 4 bits per pixel. 32 bytes for a 8x8 tile.
[r0, bp1], [r0, bp2], [r1, bp1], [r1, bp2], [r2, bp1], [r2, bp2], [r3, bp1], [r3, bp2]
[r4, bp1], [r4, bp2], [r5, bp1], [r5, bp2], [r6, bp1], [r6, bp2], [r7, bp1], [r7, bp2]
[r0, bp3], [r0, bp4], [r1, bp3], [r1, bp4], [r2, bp3], [r2, bp4], [r3, bp3], [r3, bp4]
[r4, bp3], [r4, bp4], [r5, bp3], [r5, bp4], [r6, bp3], [r6, bp4], [r7, bp3], [r7, bp4]
I'm thinking it pretty much stacks 2 bits together for each row and then stacks on two more bits to each row in order... I'm no good with wording but it helps me understand what is going on! So that is not too bad, but I suppose the next step is to really try to understand the decompression routine that Euclid wrote based on the ROM's routine. #include <stdio.h>
unsigned char readBuf[0xFFFF]; //
unsigned char writeBuf[0xFFFF];
unsigned int length = 0x2600; // Length is based on pointer table's data
void decompress()
{
unsigned int i = 0, j = 0;
unsigned char codeByte = readBuf;
unsigned int codeByteBitTest = 7;
unsigned int count = 0;
i++;
while (count < length)
{
if ((codeByte & (1 << codeByteBitTest)) != 0)
{
//read length
unsigned char nextByte = readBuf;
unsigned int len = nextByte /4;
unsigned int start_pos;
unsigned int k, oldJ;
i++;
count += len;
start_pos = (nextByte & 3) * 256 + readBuf;
i++;
//copy bytes
oldJ = j;
for (k = 0; len != 0; k++)
{
writeBuf[j] = writeBuf[(oldJ - start_pos) + k];
j++;
len--;
}
}
else
{
//read one byte
writeBuf[j] = readBuf;
i++;
j++;
count++;
}
if (codeByteBitTest == 0)
{
codeByte = readBuf;
i++;
codeByteBitTest = 8;
}
codeByteBitTest--;
}
}
int _tmain(int argc, _TCHAR* argv[])
{
FILE* fp = fopen("mmx3.smc","rb");
FILE* wp = fopen("out.bin","w");
fseek(fp,0xD030C,0);
fread(readBuf,sizeof(unsigned char),0xFFFF,fp);
fclose(fp);
decompress();
fwrite(writeBuf,sizeof(unsigned char),0xFFFF,wp);
fclose(wp);
return 0;
} Aside from probably decompressing a few bytes by hand and maybe making an 8x8 tile worth of data. Is there any better ides to go about deriving a re compression routine? I went through a couple bytes with it, but still cannot say that I understand maybe mapping out the routine with bits instead of hex bytes would help me to actually see it better? |
|
|
|
« Last Edit: June 02, 2008, 12:07:19 am by elixirnova »
|
|
|
|
Ryusui
Guest
|
|
« Reply #13 on: June 02, 2008, 01:47:17 am » |
|
Here's your mistake: you let someone else write your decompressor. To write a recompressor you need more than source code; you need to have a hands-on grasp of the compression format, and the only way to get this is to write the decompressor yourself.
|
|
|
|
|
tomaitheous
Guest
|
|
« Reply #14 on: June 02, 2008, 09:09:09 am » |
|
Are you trying to write a LZSS compressor?
|
|
|
|
|
|
RHDN Forum Archive
