Projects
Text Hooker
last updated: 6:13 PM 4/15/2006
Summary
last updated: 6:13 PM 4/15/2006
This programs adds functionality to FCEudxSP that
allows the user to highlight text from the game screen and copy it into a text
window. Its intended purpose is to allow people with minimal rom hacking
knowledge (enough to know how to make a table) to play through Japanese games
without needing a full-blown translation.
I decided to create this so that I could play through some Japanese RPGs. Usually, if I can figure out what the menus say, that will be enough to get through the game (or at least far enough to realize that it's not worth playing).
Full feature list:
I decided to create this so that I could play through some Japanese RPGs. Usually, if I can figure out what the menus say, that will be enough to get through the game (or at least far enough to realize that it's not worth playing).
Full feature list:
- Display active parts of nametable with correct pallette *done*
- Allow tile-by-tile selection *done*
- Allow user to save/load selections *done*
- Allow user to load table files *done*
- Allow user to save table files *done*
- Allow user to hook text from the game into a text window *done*
- Handle tenten and maru marks *done*
- Translate hooked text via internet translators *done enough*
- Automatically hook/translate selected text as it changes *not done*
- Allow user to define custom definitions *done*
News
last updated: 6:13 PM 4/15/2006
6:13 PM 4/15/2006
Why hello there website! No, I did not feel like updating you. But here I am and there you are. And hey, here's some files. I've been done with this version for nearly a month now. I've been lagging behind writing my documentation. But, I think it's good enough for a release now. Check the files section!
1:11 AM 8/30/2005
I started up work on the whole Babelfish thing again. I had planned to break down the massive function into smaller parts before attempting to debug it. In reality, I ended up dashing that and debugging it anyway.
My first problem was a stupid one on my part. I was telling babelfish that I was sending four bytes more than I actually was, so it would sit until it times out, waiting for those other four bytes. So, by simply subtracting four from a value, I got it to successfully send the data.
My second problem was getting the sjis hex string into an actual string. The simple solution ended up being the strtol function and casting to an int. So, after putting way too much work into this single function, I finally have the proper engrish that babelfish is returning being displayed in the window.
There's a bit more I'd like to do with babelfish, but it hardly seems worth it due to its horrible translation quality. It has the added advantage of being able to convert into languages other than English, but that's about it. I'll probably get around to allowing it to do that with a later release of the program.
For now, I'm going to focus on the other major features that I haven't done yet. This includes being able to save selections and known word substitution.
8:10 PM 7/26/2005
I tried for a while to get Google's translator to work, but it's no good. I looked up all the internet translators that I could find and they all wanted UTF-8. It dawned on me then they're using UTF-8 because they have to handle so many languages. It would be dumb of them not to use unicode. Excite.co.jp uses shift-jis because it only does english/japanese translations.
Unfortunately, this means I'll have to either settle for the single translation, or find a way of converting between character sets. I gave the latter a shot. Some brief Googling pointed me to the iconv library. I've only been able to find one example program using it. The rest of the results are for php. This sucks because I cannot get this thing to work. It doesn't even seem to fail correctly. I get stuck with no output, no error number, no nothing.
Another forseeable option is to code my own utf8/sjis converter. It doesn't seem at all difficult to make, just really hard to tell when it's done correctly. You simply check what range of values the two bytes are in, and then add or subtract an offset associated with that range. The impossible part for me will be making sure all of the kanji convert properly. This isn't too much of an issue for nes games, but it would bug me anyway.
Idealy, I would like to have iconv working, since I would know for sure that the conversion is accurate. However, it doesn't seem as if that's going to work at all for me. I get this feeling that even if I got it to work, it wouldn't end up doing what I expected it to do. Maybe it's just sour grapes, I don't know.
10:46 PM 7/24/2005
Did some bugfixing and some translating. The bugfixes were small things like being able to set which scanline to update on, how often to update the display (the sound would occasionally skip when just playing the game with the Text Hooker open). Both of those were things I glossed over when I was starting since I didn't really know how to program in C for Windows. I barely do now, but I can get by now just by editing the code that's already there. I also had to fix the highlighting feature since I added support for choosing the scanline. Because of the way I implemented this, I had to change the highlighting from turning the tiles reddish, to inverting the tile's pallette. It's still very usuable, I think. I guess if there's a lot of gray, it wouldn't invert too noticabely, but even with Metal Slader Glory (which uses a gray background) I was able to make out the selection.
The translation feature is working now through excite.co.jp. I have yet to toy around with Google's form, but I'm hoping that will work with Shift-JIS as well. It really helps to have two translations to compare. Of course, when neither one makes sense, you still aren't any better off than just seeing the Japanese! Oh well, nothing I can do for those instances.
1:44 PM 7/23/2005
The features list should give you a good idea of what I have accomplished so far. The part I'm currently working on is the translation features. I have made programs in perl, php, and vb that query websites and scrub their responses for particular content, but I have never had to do it with C++. So the methodology was already pretty well known, it was just a matter of learning how to use winsock.
I spent a night going over and altering the code from a winsock tutorial and managed to learn enough about it to get by. I was able to query Babelfish with some Japanese or English text and get a translated response. It wasn't until I ported that code into the Text Hooker that I realized my current problem.
Normally, when you send a Japanese string to Babelfish, it will return the response in English, but with a few kana still left in there. The results window in the Text Hooker was not displaying these characters. I checked the results in the window with the results from the actual Babelfish website and they were different. I was able to determine that the couple of gibberish characters that were showing up were indeed supposed to be kana. However, it also showed me that the translation result was quite different. Similar, but different, I would say.
I checked Babelfish's source code to see what kind of Japanese encoding it was expecting. It wanted UTF-8. The Text Hooker was sending it, what I believe now to be, Shift-JIS. To verify that FireFox was actually sending it UTF-8 instead of Shift-JIS, I installed the Live HTTP Headers extension and looked at the request headers that were being sent. They were indeed in UTF-8.
The solution became apparent. I would have to convert the Shift-JIS text into UTF-8, query Babelfish, and then convert the response back into Shift-JIS. I am not about to do this, nor am I looking to bulk up the binaries by including some pre-existing language conversion libraries. The other solution, one which seems way too obvious in hindsight, is that I should simply not use Babelfish for the translations.
I know of two other internet translators that support Japanese text: google and excite.co.jp. The first, Google, is sendind its data in UTF-8. This isn't too surprising since it uses the same software as Babelfish. However, there is something in the request header that specifies which kind of encoding to use. I might be able to change that to Shift-JIS and have it work like I need it to. The second, excite.co.jp, is already using Shift-JIS. This one uses different software than Babelfish and Google, so I would like to implement this one as well so that the user can get a second engrishy opinion.
Why hello there website! No, I did not feel like updating you. But here I am and there you are. And hey, here's some files. I've been done with this version for nearly a month now. I've been lagging behind writing my documentation. But, I think it's good enough for a release now. Check the files section!
1:11 AM 8/30/2005
I started up work on the whole Babelfish thing again. I had planned to break down the massive function into smaller parts before attempting to debug it. In reality, I ended up dashing that and debugging it anyway.
My first problem was a stupid one on my part. I was telling babelfish that I was sending four bytes more than I actually was, so it would sit until it times out, waiting for those other four bytes. So, by simply subtracting four from a value, I got it to successfully send the data.
My second problem was getting the sjis hex string into an actual string. The simple solution ended up being the strtol function and casting to an int. So, after putting way too much work into this single function, I finally have the proper engrish that babelfish is returning being displayed in the window.
There's a bit more I'd like to do with babelfish, but it hardly seems worth it due to its horrible translation quality. It has the added advantage of being able to convert into languages other than English, but that's about it. I'll probably get around to allowing it to do that with a later release of the program.
For now, I'm going to focus on the other major features that I haven't done yet. This includes being able to save selections and known word substitution.
8:10 PM 7/26/2005
I tried for a while to get Google's translator to work, but it's no good. I looked up all the internet translators that I could find and they all wanted UTF-8. It dawned on me then they're using UTF-8 because they have to handle so many languages. It would be dumb of them not to use unicode. Excite.co.jp uses shift-jis because it only does english/japanese translations.
Unfortunately, this means I'll have to either settle for the single translation, or find a way of converting between character sets. I gave the latter a shot. Some brief Googling pointed me to the iconv library. I've only been able to find one example program using it. The rest of the results are for php. This sucks because I cannot get this thing to work. It doesn't even seem to fail correctly. I get stuck with no output, no error number, no nothing.
Another forseeable option is to code my own utf8/sjis converter. It doesn't seem at all difficult to make, just really hard to tell when it's done correctly. You simply check what range of values the two bytes are in, and then add or subtract an offset associated with that range. The impossible part for me will be making sure all of the kanji convert properly. This isn't too much of an issue for nes games, but it would bug me anyway.
Idealy, I would like to have iconv working, since I would know for sure that the conversion is accurate. However, it doesn't seem as if that's going to work at all for me. I get this feeling that even if I got it to work, it wouldn't end up doing what I expected it to do. Maybe it's just sour grapes, I don't know.
10:46 PM 7/24/2005
Did some bugfixing and some translating. The bugfixes were small things like being able to set which scanline to update on, how often to update the display (the sound would occasionally skip when just playing the game with the Text Hooker open). Both of those were things I glossed over when I was starting since I didn't really know how to program in C for Windows. I barely do now, but I can get by now just by editing the code that's already there. I also had to fix the highlighting feature since I added support for choosing the scanline. Because of the way I implemented this, I had to change the highlighting from turning the tiles reddish, to inverting the tile's pallette. It's still very usuable, I think. I guess if there's a lot of gray, it wouldn't invert too noticabely, but even with Metal Slader Glory (which uses a gray background) I was able to make out the selection.
The translation feature is working now through excite.co.jp. I have yet to toy around with Google's form, but I'm hoping that will work with Shift-JIS as well. It really helps to have two translations to compare. Of course, when neither one makes sense, you still aren't any better off than just seeing the Japanese! Oh well, nothing I can do for those instances.
1:44 PM 7/23/2005
The features list should give you a good idea of what I have accomplished so far. The part I'm currently working on is the translation features. I have made programs in perl, php, and vb that query websites and scrub their responses for particular content, but I have never had to do it with C++. So the methodology was already pretty well known, it was just a matter of learning how to use winsock.
I spent a night going over and altering the code from a winsock tutorial and managed to learn enough about it to get by. I was able to query Babelfish with some Japanese or English text and get a translated response. It wasn't until I ported that code into the Text Hooker that I realized my current problem.
Normally, when you send a Japanese string to Babelfish, it will return the response in English, but with a few kana still left in there. The results window in the Text Hooker was not displaying these characters. I checked the results in the window with the results from the actual Babelfish website and they were different. I was able to determine that the couple of gibberish characters that were showing up were indeed supposed to be kana. However, it also showed me that the translation result was quite different. Similar, but different, I would say.
I checked Babelfish's source code to see what kind of Japanese encoding it was expecting. It wanted UTF-8. The Text Hooker was sending it, what I believe now to be, Shift-JIS. To verify that FireFox was actually sending it UTF-8 instead of Shift-JIS, I installed the Live HTTP Headers extension and looked at the request headers that were being sent. They were indeed in UTF-8.
The solution became apparent. I would have to convert the Shift-JIS text into UTF-8, query Babelfish, and then convert the response back into Shift-JIS. I am not about to do this, nor am I looking to bulk up the binaries by including some pre-existing language conversion libraries. The other solution, one which seems way too obvious in hindsight, is that I should simply not use Babelfish for the translations.
I know of two other internet translators that support Japanese text: google and excite.co.jp. The first, Google, is sendind its data in UTF-8. This isn't too surprising since it uses the same software as Babelfish. However, there is something in the request header that specifies which kind of encoding to use. I might be able to change that to Shift-JIS and have it work like I need it to. The second, excite.co.jp, is already using Shift-JIS. This one uses different software than Babelfish and Google, so I would like to implement this one as well so that the user can get a second engrishy opinion.
Downloads
last updated: 6:13 PM 4/15/2006
The Text Hooker is part of my FCEUXDSP fork, FCEUXDSP CE:
fceuxdspce-1_0.zip (win32 binaries)
fceuxdspce-1_0s.zip (source code)
fceuxdspce-1_0.zip (win32 binaries)
fceuxdspce-1_0s.zip (source code)
Screenshots
last updated: 1:44 PM 7/23/2005
Here's a shot from the current GUI:
The GUI, as of release 1.0
This is the first batch of screenshots I made to show off the text selection and hooking features:
The GUI, as of release 1.0
This is the first batch of screenshots I made to show off the text selection and hooking features:
All unique content is owned by UglyJoe. Please do not
distribute or reproduce any part of this site without my permission.
Valid XHTML and Valid CSS.