Improved Track Decoding

20 February 2002

Today the track-decoding handler was improved, since it would have been too slow for the increasing number of known disk format descriptors. The old handler would have eaten tens of megabytes for one track and we would have ultra-slow execution caused by the frequent allocation/de-allocation of memory. Note: This is completely different to sync scanning and pre-processing mentioned in previous WIP’s.

We needed to decode a block of data by its descriptor and linked descriptors (i.e. data descriptors, see the 6th Feb 2002 WIP), for each bit decoded on a track where the continuous unmapped area would permit the block to exist. The bits may and are restricted by several conditions - like pre-filtering by sync-sets, etc., but there is still a lot of decoding to do. During block decoding, a score is assigned to each block that indicates how well it matches with the expected data types, values and other criteria.

We collect this data for each block for a track, and then give an overall score to this track. Finally the highest scoring track format is selected and tested for unmapped data, which of course will lower the score.

Now we have a track in a form that should have all, or at least most, of its data decoded, checked for integrity, and a percentage value indicating the confidence of the track matching the selected format. This can then be manually reviewed and quite possibly a derivative format created if needed.

Speed. Running this software for hours/days is not really welcome just for one game! It can be easily seen that the decoding process always requires two versions of a block being decoded in the root decoding level. After a new block is decoded at a different bit position, its score should be compared against the existing one. If it is higher, the decoding should be kept and the old one removed. This is very slow.

Double buffering - somewhat similar to game screen updates. One buffer holds the existing result data, while the other one is used for decoding. If the new block scores better, the active buffer is simply changed to the one containing the new block, and the comparator block placeholder is assigned for the next decoding if there is any more to do. If the decoder score is lower, we just try the next position to decode. Once a block is fully decoded for each possible position, only then do we keep the result data as the final data.

The previous solution was nice, and conventional, and was very slow. This new one is remarkably fast. Note: The processing of each track must be continually kept to only a few seconds - multiply that with 168 (84 cylinders * 2 sides) to see why!