I don't think that would be overkill, especially if all it involves is enabling the user to select an encoding type.

  WinHex lets you open up the RAM, which has GOTTA be useful for something!  What sucks is they won't let you save more than 200kB, and it seems to complain if you copy more than 16MB as hex string (to use in another editor). 
  What I want to do is find the VRAM of a game or emulator and extract the 3D data so I can search for geometric primitives.  For example I can convert a set of bytes into a point cloud and see if it correlates to the scene in the game, like an incomplete 3D screenshot.  Then if it works I can analyze the data for triangle meshes.  It'll be a lot of trial&error, but do you think I can do this? 

You should probably compile a list of all planned features so people can quickly see what's already been suggested and what you plan to have in it.

I've been shamed into an update, here.   Since the topic was safe from the great board prune, I didn't feel the urge to bump it.  Updates will become more frequent when I've reached the point of screenshots and alphas.

Lots of pieces are rather solid already, but it's an extremely ambitious project.  Therefore, it will take a while before it's reached a stage that I deem usable.  I was planning to use DockPanel Suite for Visual Studio-style docking, but I'm not sure that it's solid enough for my purposes.  Something about it rubs me the wrong way.  Besides, it's got a lot of Windows-specific code, which interferes with the cross-platform goals.

Currently I'm stalled at the "writing the custom hex editor controls" stage.  It's planned to be extensible enough that you can line up multiple arbitrary data representations and offset formats on the hex editor, so this is one stage where I don't want to skimp.  I'm kind of lost as to the best way to turn a click into a location in the background for data selection and carat positioning, so I'm stepping back for the moment to do other things.

I've got another project in the works that I'll probably showcase in a month or so.  It's a tool used to track time on various projects.  Most useful for billing and reporting purposes, although it's useful just to see where your time goes.  Anyone who's ever had to use ERP software will probably find this software very useful.  I'm putting transparent forms in this project, so once the feature is finalized I'll repurpose it for the hex editor.  I'm also trying a different approach to some XML stuff that I might end up using.

Sometimes it's easier to just write the features into separate apps first, and then make libraries out of them.

There was a bunch of features I was going to suggest, but it seems like they've all been mentioned already. (woot)

This looks like it'll be a really cool project once it gets going. I'm really looking forward to a release. I've assuming that since it'll be written for Mono that you'll be able to write plugins in any supported language. I'm partial to Python, so not having to learn C# and simply use IronPython instead would be a great plus.

The one thing I've really missed is the address support Disch suggested. The other ideas floating around so far are all great as well. One thing I'd like, not sure if it's been mentioned (I read the thread really quickly) is the ability to map some keys (like the F keys or something) to run commands, so you can run your emulator, or a game specific utility or whatever, in a quick and easy way. I got this idea from various game editors that have a configurable "run in emulator" command.

In all honesty, this is looking less and less like a hex editor and more like an IDE. Which is sweet, so more power to ya. In light of that, I've got a suggestion you may want to consider. It may be out in left field, but here goes.

With all the advanced text support that you're planning on adding, I'd suggest splitting the hex editor and the text editor into two seperate plug-ins. You could have them display side by side like a usual hex editor by default, but if they where actually seperate, then you could close the hex part and just use it like a text editor when not working on binary data.

The idea of allowing plug-ins to manipulate reads and writes in between the raw data and the display is brilliant. Besides the aformentioned support for compressed sections (Which, BTW, I think you should allow for whole files. You may think it's a bad idea. I kinda agree, but it sounds like the infrastructure will be there, so why not let the user decide?), there a hundred other things you could do with it.

People have mentioned disassembly. This could actually be pretty easy with the system you're setting up. (I'm not saying you'd have to write it yourself, just that if I understand your ideas correctly, it'd be easy for someone else to write.) Basically, what a disssasmbly view would do is get the raw binary data from the main data interface (or some other plug-in, who knows, why restrict it) and then disssamble it and display the output using the standard text display plug-in. If someone was really ambitious, they could even add write support to it, so it'd do assembly too. (That'd make ASM hacks a whole new level of easy. Ok, not easy, but easier.)

I think the automatic pointer stuff could work in a similar fashion. The plug-in would somehow plug into the data layer and monitor an area of data in one place that's being pointed to by other data elsewhere. When the data changes, it simply writes to the pointer table. With a sophisticated configuration system, it would be able to handle practically any sort of pointer scheme without being platform or (even worse, game) specific. (Of course a few presets could be handy too.)

The great thing about this architecture is that the rest of the program doesn't have to know or care anything about pointers (or ASM, or whatever). The text widget just edits the text like usual. The pointer plug-in updates the pointers in the background, and the hex widget shows the changes. It's almost like magic.

All of this brings me to the one feature I'd suggest. You've mentioned color coding and such, but I say take it a step further and impliment full syntax hilighting. It sounds complicated at first, but it's really not that bad. All you have to do is take the hilighting support you're already planning and add a pattern language to it. I'd suggest using regular expressions to avoid reinventing the wheel. (Of course simply shell style glob patterns (* and friends) are an option too. Perhaps offer both, gobbing for the average Joe and REs for the power users.)

While syntax hilighting might seem like overkill, especially as the main goal is a hex editor, not a code editor (however, I suggest you're headed for a great plaform for both), it would actually have uses while hex editing too. Say, for instance that you're editing some text that also contains some control characters. You could set up the hilighting to make them stand out, automatically. For instance, if 0xFF is the end of string char, you could have it be red, and if the other control chars are all in the range 0x00 - 0x1F, you could hilight them yellow.

If you made it generic enough, it could be quite simple and very powerful at the same time. Basic users could do things like what I mentioned with the control characters, while more advanced hackers could write a full blown hilighting scheme for 6502 assembler code.

Anyway, that's my two gil, I hope you think about it. If you deliver all the features you've planned, plus a bit of support for plain text editing, I'll probably use your program all the time.

Cygnus now has a FREE EDITION (caps theirs). you may want to check it out to see how it implements certain things... at least, i find it very very nice, especially when dealing with japanese (e.g. searching for shift-jis in the file, and pasting text into the search area from an input program). it doesn't have many features at all, but what it does, it does well. (that also means it should be quick to check out, i think... for the free edition, anyway.)