+  RHDN Forum Archive
|-+  Romhacking
| |-+  ROM Hacking Discussion
| | |-+  New patching format, UPS, debuts today
Pages: 1 2 3 [4] 5 6 ... 9
Author Topic: New patching format, UPS, debuts today  (Read 5 times)
Shadowsithe
Guest
« Reply #45 on: April 01, 2008, 11:05:32 am »

I think by multiple input he means things like PC games.
Nightcrawler
Guest
« Reply #46 on: April 01, 2008, 12:17:04 pm »

Quote from: byuu on April 01, 2008, 10:09:16 am
Quote
UPS is OK so far, but kind of crippled from success without any containers available allowing handling of multi-file patching and/or dealing with headers on certain platforms.

We're working on it. UPS had to come before NINJA3. We chose not to add system-specific coding to UPS for the obvious reason that it would add infinite complexity to the UPS spec itself. I mean, how would you handle systems coming out in the future, for instance?

I understand why it is the way it is and agree with doing it that way. I've been an advocate of the simplest solution possible that meets the major shortcomings of what came before it. I'm just pointing out that without an established container format (also facing the same obstacles for acceptance) it's only half of what it could be. I'd like to see it be all of what it could be and I'm sure you would too. It would be nice to have a be all end all (for the most part) for the majority of patching needs in the community. Yeah, you'll never satisfy everybody, but if you've got UPS and you've got some containers for the most common needs (say multi-file, header support, etc..), we'll be well on our way because there's be few situations where UPS wouldn't apply.

Quote
Quote
It's no longer like ogg/vorbis if the container isn't an open standard as well.
It'll be an open standard.

Oh? Any information on this? If it's open, it would probably be a smart idea to discuss it openly. Wink Perhaps that's best put in another topic.


CRC32 check bypass option:

This seems like a good idea for a workaround for stack patching, WIP patching,  or other situations where you may purposely want to patch disregarding the CRC32s. I know you said you were going to do this. I'd also suggest making this option accessible from the GUI somehow as well as the command line. The same features should be available whether you use the GUI or command line in the patcher.
Lenophis
Guest
« Reply #47 on: April 01, 2008, 12:47:37 pm »

Quote from: byuu on April 01, 2008, 10:09:16 am
We're going to make SNES emus require that UPS patches are made against unheadered ROMs for soft patching.
Angry This is getting dangerously close to telling others how they should handle their roms. Despite your feelings or my feelings or anybody else's towards it, who are you (or anybody else for that matter) to say how end user x is supposed to store their roms?

Quote from: byuu on April 01, 2008, 10:09:16 am
It seems that everyone wants specific features, but a lot of them add too much complexity and/or are mutually exclusive to each other.
Keeping it simple is the best solution. Besides, with the source out, and you already saying "if you want to change it, call it something else entirely," what else can you do? Wink
kazuya
Guest
« Reply #48 on: April 01, 2008, 01:14:46 pm »

I had a few questions about the UPS utility but I think they can be awnserd if I read through some of the posts again
creaothceann
Guest
« Reply #49 on: April 01, 2008, 01:19:56 pm »

Quote from: Lenophis on April 01, 2008, 12:47:37 pm
Quote from: byuu on April 01, 2008, 10:09:16 am
We're going to make SNES emus require that UPS patches are made against unheadered ROMs for soft patching.

This is getting dangerously close to telling others how they should handle their roms. Despite your feelings or my feelings or anybody else's towards it, who are you (or anybody else for that matter) to say how end user x is supposed to store their roms?

byuu and Nach are in the unique position of being emulator authors. If someone can solve the current header mess, it's them.

A replacement header file could be supplied in the container together with the UPS file. Maybe even two UPS files - one for the header, one for the ROM.
BRPXQZME
Guest
« Reply #50 on: April 01, 2008, 02:00:41 pm »

Quote from: Lenophis on April 01, 2008, 12:47:37 pm
Quote from: byuu on April 01, 2008, 10:09:16 am
We're going to make SNES emus require that UPS patches are made against unheadered ROMs for soft patching.
Angry This is getting dangerously close to telling others how they should handle their roms. Despite your feelings or my feelings or anybody else's towards it, who are you (or anybody else for that matter) to say how end user x is supposed to store their roms?
Hang on to yer knickers! This isn’t telling you how to store ROMs, this is telling you how to handle patches.

This is only requiring the emulators (which have to sort out the header mess already anyway) to patch headerlessly; thus, feel free to use a UPS patch with a headered ROM via an emulator, but don’t make a patch against one.
byuu
Guest
« Reply #51 on: April 01, 2008, 03:14:25 pm »

Right, bsnes internally removes a header if it exists. I'll be passing this headerless data on to patch against UPS.

Although honestly, if I had the clout of ZSNES -- I'd kill all interleave formats, remove all ROM extensions sans .sfc (no more .smc, .swc, .fig, .078, .ufo, .gd7, .bin, .001, etc etc,) and refuse to load images with copier headers. I'd take it further and make a signed certificate system (validating the MD5SUM hash and PCB code for proper memory mapping), where games without one would show a red X in the status bar to indicate that the games were not verified.

Sadly, because I don't have that level of notability, there's only so little I can do. You guys may think I am being overbearing now, but you'll wish someone had done this fifty years from now when nobody can ascertain whether a given dump was the original, unmodified one or not anymore.

You honestly don't want to make a UPS patch against a headered ROM anyway. Since the match needs to be perfect due to XOR, you're forcing people to have an obscure copier-specific header just to patch a ROM. Which is absurd.

Headers are completely and utterly worthless for emulators. If you want to discuss NSRT headers, we'll do that in another topic.
Nightcrawler
Guest
« Reply #52 on: April 01, 2008, 04:35:27 pm »

If you were in charge of ZSNES, you'd have reduced it's userbase to a handful and killed it's clout entirely. Tongue
Vystrix Nexoth
Guest
« Reply #53 on: April 02, 2008, 11:59:49 am »

You're willing to hand off compression duties to external compressors (zip, 7zip, ...) (which I agree is the right call). Why not patch verification too? zip and 7zip have checksumming functionality.

Quote
it supports infinitely sized files, no 16MB limit

Right on. *patches /dev/zero* Wink

Quote from: The Disch
I rather like the idea of being able to have segmented patches.

XRIF can do that, such as to have bugfixes each separated from each other and individually selectable. (It does this by basically drawing lines in the sand between each one.) They can even be given short labels ("fix HEL2 bug", "fix LOCK bug", ...)


Quote
It's just a replacement to IPS

That's the problem. It only serves the same target domain as IPS: simple, unambitious patches. The problem with this, I think, is that that domain does not completely encompass the sort of hacking done today (or even back in the day). Put another way: you're trying to do X better, but these days many of us need X+1.
For example, many/most patches that would expand the data will do so by inserting data into the middle of the ROM, not appending them to the end. As you yourself have admitted, UPS "will always fall short" at that, while a more powerful patching format can handle it in stride.


So UPS is useful only for the simplest cases. Heavier formats such as XRIF are suitable for many more cases. Consider, for example, the idea of game-specific patch generators that could analyze the game's changed data more intelligently and write a patch that implements those changes more flexibly, more efficiently, and more appropriately than IPS/UPS. The patches they create would be applyable by any patcher supporting the features it uses, so no need for game-specific patchers too.

I'll be the first to admit that not having all features be universally required is a deficiency, though I've tried to make XRIF handle it as gracefully as it can. Also, once the features are implemented, they are de-facto required: "well, patcher X supports that stuff!"... that is, it'll put pressure on others to all support the same things. I'm working on a new implementation now that, if completed, will support more features than previous versions I've made, bringing that goal closer to reality.
Gemini
Guest
« Reply #54 on: April 02, 2008, 12:38:17 pm »

Quote from: Vystrix Nexoth on April 02, 2008, 11:59:49 am
The patches they create would be applyable by any patcher supporting the features it uses, so no need for game-specific patchers too.
I do not agree on the game-specific part. While some things can be done with powerful comparison algorithms, some can't. Example: movie subtitling, audio conversion+replacement, and such can't be handled efficiently by any generic patching format or patcher. A movie patch that takes only 20kb with Pixel's cd-tool could take 300 MB with a generic comparison algorithm that can't manage a file format in the most efficient way, no matter how powerful or versatile it is.
byuu
Guest
« Reply #55 on: April 02, 2008, 12:58:30 pm »

Quote
You're willing to hand off compression duties to external compressors (zip, 7zip, ...) (which I agree is the right call). Why not patch verification too?

I've already answered this. I don't intend to debate four bytes for several pages -- you can't make everyone happy. Further, the spec is out and finalized. It can't be changed.

Quote
So UPS is useful only for the simplest cases. Heavier formats such as XRIF are suitable for many more cases.

Sigh, I didn't want to compare our formats. I'd have preferred to just let the community decide. But since you insist ...

I'll agree that a more complex algorithm can have more uses, that much is obvious. However ... why are ~98% of patches in IPS format, and ~2% in PPF (*)? xdelta, bsdiff and XRIF make up less than a percentage of available patches, despite each being around how long now?
(* ~90% of these numbers were made up on the spot to prove a general point, not to argue semantics.)

Their strengths are their weaknesses. They're way, way too complex to implement. Writing the spec out on paper is the easy part. Even with XRIF, the commands may be simple enough, but implementing algorithms to search for where data was inserted / removed from a file is a very complex task. Different patchers will vary wildly in their success.

These formats are all too complex for the average person to implement. Nobody wants to read a 120kb HTML document, nor a ~600kb RFC specification. So people stick with IPS. Lots of tools, lots of emulator soft-patching support.

You can argue that "only one good patch creator needs to be made", or that "we can make one good library for everyone to use", but it doesn't really work that way. People like being able to implement their own stuff. And IPS is simple enough for them to do that.

They will make their own libraries, whether you like it or not. And when they implement such a complex spec, they will screw it up somewhere. This is inevitable. Then you'll have something like a webpage or a Java applet, that works in one program but not in another. "Oh, you have to use LolcatDiff to patch this game, BobcatDiff won't work."

A UPS patch can only exist in one single form, period. If it's even one bit different from the reference patcher, then it fails to be a valid patch. There's zero ambiguity, zero room for differences between the same patch file. Not even IPS can say this.

---

So then, are you fundamentally opposed to at least replacing IPS with a better IPS? That doesn't replace XRIF with UPS, it replaces IPS with UPS. I don't think anyone wants IPS around anymore.

Multi-patching still works, it just requires that patches be applied in a certain order. And that guarantees you that the end result is patched correctly. There's no room for "well I hope this is patched right" anymore. No more bug reports because someone applied a newer patch on top of an older patched game.

And we're bundling the platform-specific logic into NINJA, something that will be changeable in the future. So Nightcrawler's primary concern will be addressed -- albeit not immediately.

And you'll always have XRIF for areas where it's appropriate. Eg inserting data right in the middle of a file. That obviously will never fly with patching game code or ROMs, but it'll be quite useful for eg CD-based game patching. At least, you will once your specification is finalized.
« Last Edit: April 02, 2008, 01:11:50 pm by byuu »
Talbain
Guest
« Reply #56 on: April 02, 2008, 01:20:07 pm »

I'm pretty much all for getting rid of IPS.  It's clunky, strange to use, and confusing.  byuu's UPS patcher is much simpler, and less ambiguous.  As an end-user, I can say that I support any attempt to simplify the patching process.  He's also addressing the only problem I've found so far, that of using the same-named file as input/output.  Plus, he's attempting to address the complex algorithm problem as well, but it's understandable that it will take longer as it is... well, complex.  The practicality of UPS is what will make it better than IPS; that's what an end-user wants from a program.
Roth
Guest
« Reply #57 on: April 02, 2008, 02:01:35 pm »

So I'm sitting here wondering, should we be having multiple files in our archives for different users/ease of use? Presently I have only an IPS file for my only existing hack-thing-a-ma-bob; should I throw a UPS in there as well? As well as other patching formats that have been mentioned? Just UPS, to start getting away from IPS? Do the latter, but include other patching formats? Do this ->  :banghead: ?

About UPS... even though I know little of what I'm talking about, it seems that alot of things mentioned would be program specific. I mean, the core foundation of each of these programs is to patch a file/create a patch from a file. Checksums and other things like checking for same filename output, fail safes to protect the user, etc, are program(mer) specific, not really part of a spec. Am I missing something here as to why there is so much talk of extras? It seems to me if you want extras, the original UPS program is open source, so you can just make an improved program, while the spec itself, which apparently was first started in discussion over a year ago, will work just fine.

Again, I'm totally new to this, so something might be over my head that has been posted earlier, and I just didn't understand.
creaothceann
Guest
« Reply #58 on: April 02, 2008, 02:17:38 pm »

Quote from: Roth on April 02, 2008, 02:01:35 pm
So I'm sitting here wondering, should we be having multiple files in our archives for different users/ease of use? Presently I have only an IPS file for my only existing hack-thing-a-ma-bob; should I throw a UPS in there as well? As well as other patching formats that have been mentioned? Just UPS, to start getting away from IPS? Do the latter, but include other patching formats?

Just use UPS and provide a link to byuu's or other's patchers. Hardpatching is not much different from IPS.

Softpatching emulators will be available in the near future, at least bsnes and ZSNES, maybe SNES9x and SNESGT too. Until then you could provide an IPS file too, if there's demand.
Darkdata
Guest
« Reply #59 on: April 02, 2008, 04:23:40 pm »

Just thought I should bring up something that does require stack patching, and is used by a large group of users in the community.

Whether or not said goup is liked, I will leave up to you.


Thank you
Staff Note - Fixed URL. You're welcome. -Neil

Author Note: Fixed bad spelling. Next up grammour!
« Last Edit: April 10, 2008, 04:52:50 pm by Darkdata »
Pages: 1 2 3 [4] 5 6 ... 9  


Powered by SMF 1.1.4 | SMF © 2006-2007, Simple Machines LLC