Well I've been at this for a while and managed to get CDMage to display a new folder containing a single file. A scan for corruption reports no errors. Niether the binary file nor the directory itself is located within 150 sectors of any XA files nor are they anywhere near either end of the disc. (Not that any of that would stop CDMage). Yet when I try to import data into my newly created file CDMage throws this error:
  ' error importing file "C:\\path\\file.ext" '

The same error occurs regardless whether the directory record for the file is a straight copy from another directory or if I alter the LBA to point to null sectors.

That error is too vague, What am I missing?

I never got that error but i can think of:
-your image file "write protected" or opened in another program at the same time that lock the file
-some strange caracters in the directory or file name (like japanese characters)
-if my memory isn't wrong, only CdMage beta 5 can import files
-wrong block size

Oh...i forgot the most probable cause of that error: a corrupt image :p
Personally i never had problems with CdMage.
Anyway,  there is a little tool to expand the place in a TOC and u could try if he corrects the error after u created the xtra directory, download it from http://www.sadnescity.it/tools/hosted/TOC_Changer_v1.2_by_Phoenix.zip
Can we know what game are u working on and what file u want to exchange??

I see that u can read Italian and actually the tool i gave u was programmed for the Italian translation of Vagrant Story :p
Check this site for maybe some usefull info's: http://www.sadnescity.it/traduzioni/vs/vs.php.
Here is a little extract of that site:
Vagrant Story, a differenza di Xenogears e Chrono Cross, usa una normalissima TOC (Table of Contents) per accedere ai file presenti sul disco. Questo, dunque, fa sì che sia possibile gestire senza troppi problemi tutti i file attraverso Windows e grazie ad appositi programmi per la modifica delle iso. Almeno in questo, una volta tanto, abbiamo avuto vita facile.

I insist in saying that u have a corrupted image or something like that. For now i stay at work, I will try to make an image of VS and try to exchange some of those files and tell u the result later.

For the automation of this, i can't help too much because I don't have any programming skills but if I had to choose between the different docs u listed, i would choose the italian one because Sephiroth 1311 has a long Psx translation history (I had forgotten that i was mentioned in that doc ).

Oh, I should mention: the CD-Tool site is long dead and buried. I do have the last known Windows release though, which I should probably upload to here sometime. There were Linux builds as well, but I'm too lazy to go looking for those.

@Rocket Science: If I could get a copy of that tool from you it would be an enormous help.

The topic is how to insert a newly created file into the File System of a CD Image with emphasis placed on correctness and abiding by the relevant standards, ISO 9660 and it's extensions the "Ranbow Books". Particularly the Yellow Book which I already have a hard copy of but also the Green Book which must be bought. I don't have a copy of this so I am missing vital information regarding Mode 2 Form 1 / Form 2 standards. CDMage is used only to verify that what is being done is being done correctly. Here are the steps I performed to create my new directory...



[Step 1] Create a new directory record.
This step comes first because you need prior information of the directory before you can add a reference to it in the root directory record. To create a new directory it is much easier to use an existing directory as a template. I chose BATTLE as a template because it has only three files, thats more than enough. the Logical Block Address for BATTLE is 0xC0 convert it to an offset in the CD image with this formula:
  image offset = LBA * 2352; // 0x6E400 = 0xC0 * 0x930

Open the CD Image in a hexeditor and go to this offset (0x6E400). Then Copy everything from the subheader to the end of the directory records. It's important to include the subheader of the sector in the copy paste operation. I copied everything from 0x6E410 to 0x6E52B inclusive. That's exactly 0x12C bytes copied.

Now to find some empty sectors to use for our new directory and the files it will contain. I wrote a tool that does this for me. Sectors from LBA 0xC1 to 0x44C are all unused. I chose to use the LBA 0x100 for my directory. so convert it to a file offset:
  image offset = LBA * 2352; // 0x93000 = 0x100 * 0x930

Go to that address (0x93000) in the CD Image and paste in the previously copied directory record making sure to overwrite exactly 0x12C bytes of data including the sub header. Now the new directory is a carbon copy of the BATTLE directory.

for the time being the only change necessary to the new directory is to correct the LBA number of the current directory. You will find this field at offset 0x9301A to 0x93022 inclusive. Currently it is still pointing at the sector asigned to the BATTLE directory ie LBA 0x000000C0. The ISO 9660 standard requires that this field be represented as a both endian dword so correct the field respecting these requirements...
  0009301A: C0 00 00 00 00 00 00 C0 = LBA BATTLE
  0009301A: 00 01 00 00 00 00 01 00 = LBA New directory


[Step 2] Add a reference in the ROOT directory record.
PSX CD Images require that all directories should be placed only in the root directory. As a result all PSX CD's have a tree structure no more than one level deep. The ISO 9660 standard requires that directories be organized alphabetically. Filenames can only contain the following characters:
  a-characters: "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_!"%&'()*+,-./:;<=>?"

Directory names are somewhat more restrictive, just follow the conventions used by the other directories. For simplicity sake the name of this directory should appear last in the root directory record. For this reason I use the name "XMAP" for the name of the new directory. So off we go to the sector containing the ROOT directory record. You will find it at LBA 0x16 which is converted to offset 0xCA20 in the image. Go to that offset.

Again the first record is the current directory. it spans bytes 0xCA38 to 0xCA67 inclusive this record is describing the current ROOT directory. The next record is the parent directory. The ROOT directory has itself as its own parent. The parent record spans the bytes from 0xCA68 to 0xCA97 and should exactly match the record for the current directory.

Next record is the BATTLE directory. this record already describes our new directory with the exceptions of the name, the length of the name, and that the LBA asigned to the new directory should be 0x100 and not 0xC0. The first byte of the BATTLE record is located at 0xCA98. the value of this byte is 0x36 this value is the size of the record. To save time just copy 0x36 bytes from 0xCA98 to 0xCACD inclusive. This is the entire directory record.

Now where do we paste these bytes to? Well the safest way to do this is to use the size of the records to skip over each record. skip over the BATTLE record and you will find the size of the next record, skip over that record and you are again pointing at the size of the next record. Do this recursively until you reach the end of the records.

Just for safety sake the offset that should be used to paste in the new record is 0xCE52 and the new record should include offset 0xCA87 as its last byte.

First you will need to correct the LBA from 0xC0 to 0x100.
  0000CE54: C0 00 00 00 00 00 00 C0 = LBA BATTLE
  0000CE54: 00 01 00 00 00 00 01 00 = LBA XMAP

Next overwrite the name of the directory with XMAP. You will notice that there are 2 extra bytes left over. They need to be stripped out and the size of the record will need to be adjusted. Go to offset 0xCE76 and start overwriting the bytes found there with the value from the byte exactly 2 bytes ahead. Do this until you have reached the end of the record. This has effectively stripped out the extra bytes. The size of the name of the directory will need modifying as well you will find this at offset 0xCE72 change it from 0x06 to 0x04.

Now to correct the size of the record. Go back to offset 0xCE52 and overwrite the size of the record from 0x36 to 0x34. Bear in mind that ISO 9660 requires that the size of the record must be an even number. If it is not an even number then the name of the directory must be padded with zeros such that the size of the record is an even number. The value at offset 0xCE72 (the length of the name of the directory) should include any zero padding bytes.

At this point the file system contained in the image will be partially parsed (incorrectly) by some tools including ISO Buster. Any tool that parses this image should be used with caution these tools don't know what they are doing, If all you want is for your image to be capable of working with emulators fire away. This is exactly where cdprog stops with regards to the images file system. But there is more work that needs to be done to meet the required standards.


[Step 3] Correcting the Path Table Entries
This step is a breeze compared to what has gone before. There are 4 path table records. The first two are designed to cater to systems that use the little endian byte order. The next two cater for systems that use the big endian byte order. There are two copies of each type of path table for verification purposes. You need to add new entries to each of the 4 path table records.

The first path table is located at LBA 0x12 (offset 0xA560). This is a little endian path table. The purpose of a path table is to speed up file access by maintaining a list of all directories contained in the image. without the binary files cluttering things up. A path table record is much more simple than a directory record. I will use MIPS convention to describe the structure since the target is PSX which is a system with a 32bit word a 16bit half word and an 8bit byte.

byte\tLength of Directory Identifier 
byte\tExtended Attribute Record Length 
word\tLBA of the directory.
half\tDirectory number of parent directory (Must be 0x0001 for PSX, the ROOT directory)
byte\tDirectory Identifier (name) in d-characters.

The name of the directory is padded with a zero if the Length of Directory Identifier field is odd, no padding is used otherwise. This means that each table entry will always start on an even byte number. d-characters means that only characters belonging to the sub set of ANSI that follows must be used:
  d-characters: "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_"

Armed with this info we can build a new path table record to describe our new directory.
lenDir\tbyte 04
lenXA\tbyte 00
LBADir\tword 00000100
Parent\thalf 0001
idName\tbyte "XMAP"

so our little endian records will look like this
04 00 00 01 00 00 01 00 58 4D 41 50

and our big endian records will look like this
04 00 00 00 01 00 00 01 58 4D 41 50

insert the little endian record to offset 0xA642 leaving the zero byte padding in front of it as it is a part of the TITLE record. This is the first path table taken care of. Then insert the same little endian record to offset 0xAF72, That's the second path table taken care of. We are done with the little endian path tables now to correct the big endian path tables. Insert the data from the big endian record to offset 0xB8A2 and also to offset 0xC1D2.



[Step 4] Correcting the Primary Volume Descriptor
There is a field in the Primary Volume Descriptor that states the size of the Path Table in bytes. Before we modified the Path Tables the size was 0xCA after modifying the size grew to 0xD6. This small adjustment is all that is required. This field can be located at offset 0x939C of the image. It is stored as a both endian dword.
  0000939C: CA 00 00 00 00 00 00 CA = lenPathTable
  0000939C: D6 00 00 00 00 00 00 D6 = lenPathTable

After this adjustment has been made save the image. Without repairing the EDC and ECC check that the file system is correctly parsed by the lesser CD Image tools such as MagicISO, ISOBuster, etc. These tools should display the file system including the new directory "XMAP" along with it's contents, which are simply a carbon copy of the contents of the "BATTLE" directory. Try opening the CD Image in CDMage however and an error is thrown saying:
"Data track has errors in the file system,
 It's content may be displayed incorrectly.
 Continue parsing?"

If you choose to continue parsing you will notice that the changes you made to the file system are not reflected in the file system that CDMage is displaying. In fact CDMage has taken it upon itself to repair the corrupt sectors. You now need to repair the EDC and ECC of the sectors that you editted. You have two options. Use ECC regen or use CDMages in built option "Rebuild Sector Fields". If you want to use the CDMage meathod here's how to do it:

1] Right click on any file and select "Browse File", This will display the Browser window, keep the browser window open for the remainder of the process.
2] From the menu bar select "Action" -> "Scan For Corruption", This will display a DialogBox the only button of interest is "Scan". Double-click this button to begin the scan. You will soon see the sectors that you edited in the Browser window. Wait for the scan to complete. CDMage should display a MessageBox stating:
   "Image has 6 corrupt sector(s)."
3] Select all 6 sectors. Right click on any one of the corrupt sectors and select the option "Rebuild Sector Fields..." This will display a new DialogBox make sure that all of the CheckBoxes have been ticked then press "OK"
4] Close CDMage. The changes will be applied automatically on closure. The file system has now been modified to the extent which my knowledge of the standard will allow.

The next time you open the CD Image in CDMage all of the sectors will have their EDC and ECC repaired. However CDMage will still report errors in the file system. Despite these as yet unknown file system errors CDMage will display the new file structure with the "XMAP" directory and its contents. (Which are still simply links to the files contained in the "BATTLE" directory). You can extract or import these files. They already exist in the image. You can rename them. Renaming the files wont break anything. They still extract and import correctly. But if you try to resize the files (beyond the sector boundary ie beyond the scope of the italian doc), or relocate the files to new sectors, the files will be broken.

This is a screenshot of a successful result of importing a file into the image. The file being imported is actually overwriting an existing file which was linked to. The existing file was originally contained in the "BATTLE" directory...


Now the files within the "XMAP" directory have been renamed, resized and have had unused sectors asigned to them. The ability to insert files has been broken but file extraction still works...


The problem is it works when the file being inserted already exists within the image. It breaks when the file is asigned to sectors that have not been formatted to be used for data. My problem is how to format the sectors correctly.

Yes, I mean the sub header. I have been playing around with it and it is this sub header for the sectors asigned to the binary files that is causing the import error. The only information I have regarding this sub header is from ECC Regens help file which does not contain enough information to know what I'm doing here.

Apart from this there is still an unknown error with the file system being detected by CDMage. This is more worrying because I have some idea of what to do to make the sectors work. But this error has me at a loss, it could be any number of things causing this.

Edit : also that information was obtained from a reliable source. PSX standards demand a file system no deeper than one level. Even if some developers chose not to respect that.

Thanks Gemini, seem's to be working correctly. Still getting file system errors but I think I can figure that one out with time.

Anybody got more detailed information regarding the subheader?