Data Integrity and Commodore Documentation Errors

24 February 2002

Added automatic support for checksum dependencies/nested checksums, i.e. checksums that must be calculated in a specific order to give the correct result. This feature is used by mixed format data blocks, such as an AmigaDOS block containing an Atari block.

AmigaDOS style parities are now supported. These give genuine format scores, although the severity of the error is not quite balanced yet.

The following state for tracks in the analyser are now the following:

• “not tested” - a track that is not analysed yet
• “unknown” - an unknown format
• track type “rejected” - manually selected format, rejected by analyzer
• track type “changed” - track analysed, but format is manually overridden
• track type “manually selected” - but unanalysed track
• track type “nn.nnn%” - analysed track, with a integrity score

[This article has been moved here.]

[This article has been moved here.]

We almost got a heart attack due to the number of bad Amiga dumps in games with AmigaDOS tracks. However many hours of debugging showed that, unlike the official Commodore documentation, there is no relationship whatsoever in 2 values in the block headers. Thus many otherwise good tracks scored 95.454%, since the relationship was checked for validity.

We are thinking about a decent way to still check the value properly, without hacks or removing the misleading check. [sarcasm]Well done Commodore, for your excellent documentation - as always[/sarcasm]. The person responsible for the documentation obviously had no idea what these values were for. It was probably not used properly outside the Kickstart anyway.

A fix will be documented as soon as we come up with something reasonable.

The 95.454% integrity score, is caused by having the same data corrupted, just one byte, in the block header.

We got suspicious after reverse engineering a few dumps as they loaded the offending tracks with no problems. Also seeing 95.454% several hundred times in a row makes you wonder... After this only a few hours of debugging was needed to see where the decoder finds failing data.

Actually, the analyser is extremely good in restoring a complete track from a partially good read. It has restored tracks that the original game loaders thought were bad - and that is original genuine data!

Fully checking the value to be correct requires a solution; this is the catch 22 of ensuring we have 100% correct dumps. With this in mind the analyser descriptor has now been changed to a range check.

Actually the mystery value meant the number of blocks until the track gap, including the one where the value is read from, and it has no relationship whatsoever with anything else. Unfortunately the gap can only be found after all the other blocks are mapped, and thus it is impossible to tell positively if the value is good or not before the track is fully analysed. Only then do we know for sure where the gap is. A third step may be added later to check the offending value after a full analysation.