Author
|
Topic: Japanese Version of FF7 character encoding? (Read 2018 times)
|
Cyberman
Guest
|
 |
« Reply #15 on: January 18, 2007, 11:21:38 pm » |
|
Alright, so what you're really doing is fixing up the code and everything to improve FFVII and hopefully any other game that Sqaure made with the same or similar code... So it'd possible to improve FFVIII as well, correct? The real improvement might be that it runs and isn't stuck in 16bpp mode under windows (PC version bug) eventually it will be mutli resolutioned as well. (IE you are not stuck with 320x240 full screen 640x480 windowed etc.) These are things a bit down the road however. Right now it only displays the field location and the walkmesh, with a view port window around the walkmesh actor. As for enhancement that will be left to other people. You have to remember the DATA and figures etc. in FF7 are trademarked and copyrighted. So MODIFICATIONS are right out for the makers of the engine. You can for example have a Tifa with blue jeans model, modify the engine to load in extra models and poof. There you are. We can't do that, think carefully about what you want to do first, I always say. We've too much hard work at stake for doing something stupid like distributing enhanced models etc. on trademarked and copyrighted material that our engine runs. Making new content for the game is questionable legally for us. (Chrono Resurection ring a bell?) Or am I, again, confused? maybe you are ... and maybe you aren't?  Also, and this might be a silly question but I feel inclined to ask it any way, this engine will only apply to PC games that use this engine? So, only the PC ports of FFVII and FFVIII will be able to use them? Actually the other way around, currently it does not work with the PC media at all, it currently only works with the Playstation version. This is because the majority of coders have the Playstation version. The seperate systems data sets aren't all that different it turns out though, so none of the developers are really concerned about this. It's just another thing that has to be done eventually to us. Cyb Random trivia: Q: Why do the compressed files all have there sizes stored at there begining? A: Square did not use the ISO_9660 system to read these files as the bios routines to use these were not thread safe. They wrote there own special drivers to access the data on the discs so that animation with FMV's and audio would actually work. This also means that if you alter a file too much on the CD you will crash there engine and the game. |
|
|
|
|
Starscream
Guest
|
|
« Reply #16 on: January 19, 2007, 12:12:16 pm » |
|
Interesting trivia, FF7 was coded in C++. Another interesting bit of trivia, FF7's original complete PS1 source code and data, is LOST (IE Square ... managed to loose it after it went gold). Square now is much more rigorous in there method of handling there most valuable assets.  Cyb But they still do have the PC-source I assume? Otherwise, would that be also an explanation for their refusal to remake the game? Where did you get that information from? Sorry, 3 questions in a row |
|
|
|
|
Gemini
Guest
|
|
« Reply #17 on: January 19, 2007, 01:26:07 pm » |
|
Random trivia:
Q: Why do the compressed files all have there sizes stored at there begining?
A: Square did not use the ISO_9660 system to read these files as the bios routines to use these were not thread safe. They wrote there own special drivers to access the data on the discs so that animation with FMV's and audio would actually work. This also means that if you alter a file too much on the CD you will crash there engine and the game.
Pretty much wrong.  They didn't use the internal game TOC, yeah, but only for some folders, like FIELD, not to mention that compressed files store their uncompressed size at the beginning, which is used only during the decompression procedure. As for the TOC matter, they didn't use it for the FIELD folder because Psx Cd drivers can't access more than 30 files per directory, so Square used a custom file system for over-riding that limit. Many games, like Grandia or Persona 2 (both episodes), use that trick. |
|
|
|
|
Cyberman
Guest
|
|
« Reply #18 on: January 19, 2007, 05:05:11 pm » |
|
Pretty much wrong. They didn't use the internal game TOC, yeah, but only for some folders, like FIELD, not to mention that compressed files store their uncompressed size at the beginning, which is used only during the decompression procedure. As for the TOC matter, they didn't use it for the FIELD folder because Psx Cd drivers can't access more than 30 files per directory, so Square used a custom file system for over-riding that limit. Many games, like Grandia or Persona 2 (both episodes), use that trick. so lets go over which ones use this. \\ENEMY1 \\ENEMY2 \\ENEMY3 \\ENEMY4 \\ENEMY5 \\ENEMY6 \\FIELD \\STAGE1 \\STAGE2 Lets take... \\FIELD\\FIELD.TDB now first UINT32 reads 13627 the file size in windows reads 14K now lets decompress it .... data size is now 66117. I would definately have to say, this assertion that it's the uncompressed file size, to be false. It's the actual file size. Yes I do know how to decode that particular file correctly. You will find this is exactly the same with ALL files in \\FIELD and \\ENEMYX \\STAGEX I know this, because I've been messing with them for the last 4 years. As for the why, heh! You could be right! This doesn't explain the multithreaded nature of the game and it's loading effacacy, however I'm darn certain that's NOT the uncompressed size (or it would have been a godsend for me decoding those damned files). Instead they are compressed using LZS that first number is likely for there kernal level routines to sequentially read the data correctly into memory. But they still do have the PC-source I assume? Otherwise, would that be also an explanation for their refusal to remake the game? Where did you get that information from? Sorry, 3 questions in a row Well the PC source is EIDOS baby as far as I know. Also they did have a pre production version it took Square a while to find the final data set sent out. The final source code for the playstation version is still a mystery as far as I know. There refusal to remake the game is quite obvious why, it would have to be a new game. A remake is relatively easy if the original data is highly intact and available along with the complete source. It would cost too much to rework the entire game from scratch and have it match the original game. You'll notice there other (earlier) games took almost no time to remake. This is because they modularized the games originally so it was pretty easy to port them. They also had all there data intact. Apparently they just had to make a PS1 engine to run there data and any FMV's they needed. You should look at this reference I gave a while back. Thinking upon how much work it was for us to do, they would have only a somewhat better clue, and a much tighter deadline I would have nixed they idea myself. FF8 and FF9 are better candidates for remakes but they weren't as popular. Wow this thread has had everything but what I posted for! LOL. Does anyone have the FF7 Japanese Encoding Tables they are willing to share?  (It appears people do not want to share this information for some reason). Please? Cyb |
|
|
|
|
Gemini
Guest
|
|
« Reply #19 on: January 19, 2007, 05:52:21 pm » |
|
Right so lets go over which ones use this.
\\ENEMY1 \\ENEMY2 \\ENEMY3 \\ENEMY4 \\ENEMY5 \\ENEMY6
\\FIELD
\\STAGE1 \\STAGE2
Lets take... \\FIELD\\FIELD.TDB now
first UINT32 reads 13627 the file size in windows reads 14K
now lets decompress it .... data size is now 66117. I would definately have to say, this assertion that it's the uncompressed file size, to be false. It's the actual file size. I guess I was wrong and mixed information from different Square games. Yeah, it's the compressed size-4. Anyway, your information is still wrong, considering that value is not used for loading data, infact size indexes are somewhere else. For the FIELD directory they can be found in FIELD\\FIELD.BIN, a gzipped file (it doesn't contain the compressed size at the beginning, as you can see), and the INIT folder has INIT\\YAMADA.BIN (uncompressed, only 80 bytes after all ). Can't remember the other folders, if there is any other using indexes. |
|
|
|
|
Cyberman
Guest
|
|
« Reply #20 on: January 19, 2007, 06:43:37 pm » |
|
I guess I was wrong and mixed information from different Square games. Yeah, it's the compressed size-4. Anyway, your information is still wrong, considering that value is not used for loading data, Well I can be wrong LOL. I never thought of FIELD.BIN... eeeee! Something I don't know. Alright panic as.. barely there. It's good to know, I wonder if other bits are floating about. Hmmm. infact size indexes are somewhere else. For the FIELD directory they can be found in FIELD\\FIELD.BIN, a gzipped file (it doesn't contain the compressed size at the beginning, as you can see), and the INIT folder has INIT\\YAMADA.BIN (uncompressed, only 80 bytes after all ). Can't remember the other folders, if there is any other using indexes. Ahhh... hmmm that's weird.. Yamada.bin only has a few indexs. What do they refer too? As there are only 3 files in init. Thanks hmmm well all things considered.. I wish my PS1 emulation CDROM driver worked would have saved me a few headaches with some of the data. What about in \\MAGIC\\ZENBAKU.BIN? That appears to be the only other one I can find that might resemble said info. Let me decompress FIELD.BIN. Hmmm none of the #'s are coming out right.  First the file size is 85435 for it. So I uncompressed it without skipping the first 4 bytes. Came out to a size of 3786 which is wrong. Right.. so I went back and skipped the first 4 bytes assuming that it was some sort of uint32 number (uncompressed size.. et al) size is now 5950. The decompression code is VERY acurate (it decompresses all valid streams) so it's definately not that. Is this a RAW file the first Uint32 looks kind of suspicious. Are you sure this isn't like Kernel.BIN and SCENE.BIN in format? Cyb Well it looks like there are some weirdo problems I can get size wise more to decompress if I skip the first 4 DWORDS in the file but the data is nonsense too me when viewed in my hexeditor. |
|
|
|
« Last Edit: January 19, 2007, 06:54:12 pm by Cyberman »
|
|
|
|
Gemini
Guest
|
|
« Reply #21 on: January 19, 2007, 06:57:10 pm » |
|
Ahhh... hmmm that's weird.. Yamada.bin only has a few indexs. What do they refer too? As there are only 3 files in init. In order: 1) INIT\\YAMADA.BIN 2) INIT\\WINDOW.BIN 3) INIT\\KERNEL.BIN 4) BATTLE\\BROM.X 5) BATTLE\\TITLE.BIN 6) BATTLE\\BATTLE.X 7) BATTLE\\BATINI.X 8) BATTLE\\SCENE.BIN 9) BATTLE\\BATRES.X 10) BATTLE\\CO.BIN Are you sure this isn't like Kernel.BIN and SCENE.BIN in format?
Cyb
Can't remember what's the format, but I'm sure it's gzip compressed. Remove the first 8 bytes and then decompress it with Winrar or whatever you prefer. |
|
|
|
|
Moulinoski
Guest
|
|
« Reply #22 on: January 20, 2007, 09:48:26 pm » |
|
Wow this thread has had everything but what I posted for! LOL. Does anyone have the FF7 Japanese Encoding Tables they are willing to share? (It appears people do not want to share this information for some reason). Please? Cyb I'm nearly romhacking retarded, so I'm afraid I cannot help you. I would like to, if I knew what to do... Alright, so what you're really doing is fixing up the code and everything to improve FFVII and hopefully any other game that Sqaure made with the same or similar code... So it'd possible to improve FFVIII as well, correct? The real improvement might be that it runs and isn't stuck in 16bpp mode under windows (PC version bug) eventually it will be mutli resolutioned as well. (IE you are not stuck with 320x240 full screen 640x480 windowed etc.) These are things a bit down the road however. Right now it only displays the field location and the walkmesh, with a view port window around the walkmesh actor. As for enhancement that will be left to other people. You have to remember the DATA and figures etc. in FF7 are trademarked and copyrighted. So MODIFICATIONS are right out for the makers of the engine. You can for example have a Tifa with blue jeans model, modify the engine to load in extra models and poof. There you are. We can't do that, think carefully about what you want to do first, I always say. We've too much hard work at stake for doing something stupid like distributing enhanced models etc. on trademarked and copyrighted material that our engine runs. Making new content for the game is questionable legally for us. (Chrono Resurection ring a bell?) Yes, it does... >_> Also, and this might be a silly question but I feel inclined to ask it any way, this engine will only apply to PC games that use this engine? So, only the PC ports of FFVII and FFVIII will be able to use them? Actually the other way around, currently it does not work with the PC media at all, it currently only works with the Playstation version. This is because the majority of coders have the Playstation version. The seperate systems data sets aren't all that different it turns out though, so none of the developers are really concerned about this. It's just another thing that has to be done eventually to us. Cyb Ah, I see... So this engine could apply to all PSX (original) FF's... plus some of Square's other games, like Chrono Cross? And what's the thing you'll eventually have to do? That sentence... I dunno, it doesn't connect... Do you mean work the data sets to match the PC ones? I'm sorry for all the trouble I'm causing you, I'm just curious... |
|
|
|
|
Cyberman
Guest
|
|
« Reply #23 on: January 20, 2007, 11:40:51 pm » |
|
Gemini
Thanks for the list. I've been plink with SCENE.BIN... what a pain that is LOL.
It doesn't extract right for me.
Moulinoski
Well in simple fun exciting and weird terms. Run the LPG archive and data for the PC.
I've had my fill with SCENE.BIN today so I'm going to twiddle with the movies now. And fiddle with PHP while I'm at it.
Thank goodness I'm going to deal with an ISO image first.
Cyb
|
|
|
|
|
Gemini
Guest
|
|
« Reply #24 on: January 21, 2007, 05:15:54 am » |
|
Gemini
Thanks for the list. I've been plink with SCENE.BIN... what a pain that is LOL.
It doesn't extract right for me. I had a program to extract it properly, although you need to edit KERNEL.BIN to rebuild it completely. Can't remember which segment contained all pointers to each chunk of data. :/ |
|
|
|
|
Moulinoski
Guest
|
|
« Reply #25 on: January 21, 2007, 10:29:29 am » |
|
Moulinoski
Well in simple fun exciting and weird terms. Run the LPG archive and data for the PC.
I've had my fill with SCENE.BIN today so I'm going to twiddle with the movies now. And fiddle with PHP while I'm at it.
Thank goodness I'm going to deal with an ISO image first.
Cyb
I'm gonna leave or else I'll risk being thought of as a retard, lol. O.O |
|
|
|
|
Cyberman
Guest
|
|
« Reply #26 on: January 21, 2007, 12:58:40 pm » |
|
I had a program to extract it properly, although you need to edit KERNEL.BIN to rebuild it completely. Can't remember which segment contained all pointers to each chunk of data. :/ I'm writting classes to extract data for Q-gears battle module. I've not considered editing the data as yet. Sadly I'm positive I'm screwing on the extraction in someway. From the wiki information SCENE.BIN consists of 8K blocks of data. The begining of each block has a list of 16 pointers. Each pointer is to a 'file' chunk inside the block. Each file chunk is LZS compressed. There are 256 file chunks. Each file chunk contains 4 battle scenes. This makes for 1024 battle scenes. Of course I'm having weird problems (gee par for the course these days SIGH). (mutter) I think I'll just uncompress and pad each file to like 2K boundries. That will make things clearer I guess. I'm gonna leave or else I'll risk being thought of as a retard, lol. O.O LGP archives are the format of the PC data, basically they repackaged the entire game into these archives, instead of having large tracks of files they all got packed into big archives for the hard disk. Ditto for the movies and other media. Much of the data is identical to the PS1 version so accessing it, isn't a big deal. The big differences are things like battle models, field models and movies. The bulk is pretty much identical. Even the animation format. Cyb |
|
|
|
|
Moulinoski
Guest
|
|
« Reply #27 on: January 22, 2007, 07:47:58 pm » |
|
I'm gonna leave or else I'll risk being thought of as a retard, lol. O.O LGP archives are the format of the PC data, basically they repackaged the entire game into these archives, instead of having large tracks of files they all got packed into big archives for the hard disk. Ditto for the movies and other media. Much of the data is identical to the PS1 version so accessing it, isn't a big deal. The big differences are things like battle models, field models and movies. The bulk is pretty much identical. Even the animation format. Cyb Ah, I get it. Kinda like a .zip file... Regarding the models, isn't that because the PC version had supposed better graphics? I dunno, I've never been able to get my hands on a PC version. |
|
|
|
|
Cyberman
Guest
|
|
« Reply #28 on: January 22, 2007, 10:12:43 pm » |
|
Ah, I get it. Kinda like a .zip file... Well archive with LZS compressed files I guess Regarding the models, isn't that because the PC version had supposed better graphics? I dunno, I've never been able to get my hands on a PC version. As far as I can tell they are both the same in terms of graphic information however the PS1 will look better on the PC using PS1 emulation because it's displayed in 32 bits instead of the PC versuibs 16 bit mode. The models use colored vertices instead of textured models. This makes them very scaleable and look quite good at hih resolutions. The only textures applied to the model are the eyes and some of the face for the characters. Only a few of the enemies use all the texture space. You can see a lot of details such as Barrets tattoe or on cloud his belt buckle in the PS1 version. Scene.BIn is not being friendly.. Oh well I can mercilessly abuse something else Cyb |
|
|
|
|
Cyberman
Guest
|
|
« Reply #29 on: January 29, 2007, 06:16:45 pm » |
|
Right so I've been messing with this and concluded that my decompression routines might be on the blink. I had someone I know decompress them (his first 2 attempts were... odd but lets not go there), anyhow the length of the records do not match. WEEE this is anoying but I'll live. I guess I had better figure out what the deal is with the decompression of the files. The records are supposed to be 7808 bytes in length, obviously that's not what I was getting Cyb |
|
|
|
|
|
