+  RHDN Forum Archive
|-+  Romhacking
| |-+  ROM Hacking Discussion
| | |-+  Romhacking Data Collection
Pages: [1]
Author Topic: Romhacking Data Collection  (Read 2 times)
Neil
Guest
« on: November 23, 2008, 12:20:48 pm »

I had an epiphany last night, or at least what might be a good idea that I have absolutely no clue what to do with or if there would be any interest in such a thing.

Currently, what happens with romhacking projects is that the individual hacker spends a lot of time investigating and reverse engineering to get their project done, and then the data vanishes. Usually. Mostly due to the fact that the notes are messy and disorganized and the time spent cleaning it up is seen as not worth it by the author. There are some exceptions. Specialized sites like the various super mario world hacking site and goldeneye/rareware community sites serve as data storehouses, and data crystal collects some (usually barebones) information on a lot of titles, but the main problem is a lack of participation and a lack of standardization. Most people scribble down address ranges with a few scattered notes and that's about it. These notes are usually discarded or archived on backups and forgotten, and rarely if ever shared. It occurred to me that what was currently going on was a giant waste of time. It also occurred to me that the solutions were already coded and available and immediately useful after a slight bit of tweaking. It occurred to me that what the scene was really missing was metadata. Seriously. If while we were working on projects we were entering data into some sort of slightly customized metadata creation utility, cataloging ranges and data formats, it'd all be handy and standardized and would make the question about whether or not you wanted to share your data and take completely out of the equation if you wanted to spend the time cleaning up your illegible notes. Best of all, your metadata would be XML, so converting it into something that could be thrown up on datacrystal would be a no brainer.

Or something.
Black_Phantom
Guest
« Reply #1 on: November 23, 2008, 01:38:36 pm »

That sounds like a pretty good idea to me as long it is standardized like you said and not lots of variations on the same thing. It will save late comers some of the tedious work of trying to reverse engineer some game that isn't so popular, aswell as encouraging people to share what they've found.

By the way, what's going on with Data Crystal? It seems to be non-existent now.
GenoBlast
Guest
« Reply #2 on: November 23, 2008, 02:08:18 pm »

Quote from: Black_Phantom on November 23, 2008, 01:38:36 pm
By the way, what's going on with Data Crystal? It seems to be non-existent now.
Hmm... strange. I think I was just on that site the other day, though, so it's not down forever.
Tauwasser
Guest
« Reply #3 on: November 23, 2008, 02:28:57 pm »

Funny thing... I had the same idea this weekend, partly because I'm currently reorganizing my stuff. However, it would need to be viewable in a really good application, so you can have overlaps, mark rom, ram, etc as necessary. Also, for all the asm coders out there, it would probably make sense when it would include various registers and you could check how they are changed, or what they are initialized with. Like, a = Demon number, bc = xy offset etc. And after that maybe c set when a = 0 etc.

So afterall it would probably have to do a lot with coding and then nobody might use it...

cYa,

Tauwasser
Vanya
Guest
« Reply #4 on: November 23, 2008, 03:02:14 pm »

I'm keep notes during my work on MM5, having a standardized way of publishing those notes would be great! ^_^
Black_Phantom
Guest
« Reply #5 on: November 23, 2008, 03:14:56 pm »

Quote from: Killa B on November 23, 2008, 02:08:18 pm
Quote from: Black_Phantom on November 23, 2008, 01:38:36 pm
By the way, what's going on with Data Crystal? It seems to be non-existent now.
Hmm... strange. I think I was just on that site the other day, though, so it's not down forever.

Smiley I hope so.

Also
Quote from: Tauwasser on November 23, 2008, 02:28:57 pm
So afterall it would probably have to do a lot with coding and then nobody might use it...

The first application might be best written in something like Python or Java so it's cross-platform. Then some faster native versions could be made. I noticed alot of people migrating to Linux recently, I'm thinking of doing the same.
DarthNemesis
Guest
« Reply #6 on: November 23, 2008, 03:26:44 pm »

How would you facilitate collaboration? Something like a wiki would be useful there, but then you wouldn't have the interface to work with...
Kitsune Sniper
Guest
« Reply #7 on: November 23, 2008, 03:35:57 pm »

I'm planning on uploading all my notes and such to my site sometime in the future, sort of like the bonus materials in games (or what some people would call 'omake'.) I especially want to upload the full scripts to the games I've translated. How's that sound?
Next gen Cowboy
Guest
« Reply #8 on: November 23, 2008, 04:01:22 pm »

Notes are a difficult thing to deal with. Personally I write my notes down (on paper) and copy what I think needs be saved. The problem arises in that, instead of having one set of notes to work from, basically this means two sets of notes have to be kept. One your actual notes, and two "cleaned up" versions to be submitted.

Is their a standard note format we should all use? When I write down addresses for character portraits lets say, I just mark it as "psicture 495 Mark" or whatever the case may be.

The last problem, and this one seems most prevalent, is that how many people actually want to share notes? I honestly do not know, perhaps the lack of notes has always been due to a lack format, or place for them to be hosted, and held. On the other hand it may just be people do not want their notes floating around.
KingMike
Guest
« Reply #9 on: November 23, 2008, 04:39:03 pm »

While random offset lists would help, I also have lots of notes within snippings of ASM trace logs.
That could be a bit of a pain to sift through logs dozens of megabytes long.
Next gen Cowboy
Guest
« Reply #10 on: November 23, 2008, 04:42:39 pm »

I had not even thought about tracing, breakpoints. or anything else that would be a pain to reorganize.

How would we organize something like this anyway?
ecst
Guest
« Reply #11 on: November 23, 2008, 09:07:20 pm »

Interesting idea, just a few comments:

Usual coding style for different consoles differs greatly, with developers of one platform coding in pure assembler while others utilizing compilers. For most SNES games, variables are essentially stored globally at predefined constant addresses, with catalogization being easy to achieve. Games for other (more modern) consoles might just have heap-like structure of dynamic objects with few constant global addresses. One then might want to catalogize the recursive structure of certain data objects, but how do you specify data structures in a formal and universal way? The list could go on, and there are many different types of things catalogizable for different games. How can you provide a universal format for doing all this? There is an unavoidable conflict between structuredness, completeness, and universality.

For each game there usually is a bunch of roms, consider for example all the official translations. Many people do their analyzing and hacking work for their localized version of choice. Often, especially if the original developers used linker tools or else, assembler code routines end up at different places, and sometimes even the addresses of global variables change. This means that address information and trace logs in general only hold valid for some specific version of a game. Doing the same analyzing work for each localized version of game is of course a waste of resources. Can a unified format account for differences between various versions of a game? Or is it better to just stick to the original (most of the time Japanese) release?

Because of the usual lazyness or lack of time, it would be a difficult time convincing sufficiently many people to adapt to this standardized format (even if some kind of portable magic graphical utility were avaliable). Would it not be easier to just constantly remind people to write clear, structured text files and upload them? Or to just upload them (better than nothing), and later have someone else messing with the game do the cleanup work? A well written plain text file can be the most informative thing on earth. There are numerous successful examples, just look at the documents archive on this site.
seirj
Guest
« Reply #12 on: November 23, 2008, 09:33:03 pm »

Maybe if such a thing were hosted on a site like this that has an active community it wouldn't be neglected like other such projects.
Pages: [1]  


Powered by SMF 1.1.4 | SMF © 2006-2007, Simple Machines LLC