KryoFlux - Exporters, Analysation & System Disk Image Support

2010-01-14

Made the base export system fairly flexible, although the framework code is now fairly complex, adding a new format should require minimal code.

It is possible now to describe a geometry for the format to be output by the exporter. The geometry supports fixed sized data (cutting or padding as necessary) as well as data size supplied by other subsystems. It is also possible to define data dependencies, to make the presence of a track image in a disk image optional depending on whether the track itself contains any data, or a defined set of other tracks contain any data.

For example, most people would want a fixed 80 track ADF image of a standard AmigaDOS disk, or a 81/82 track image only if track 80 or 81 actually contains legible data - and limit the number of tracks to the minimum required to represent what is on the disk. This can be described as a disk geometry with fixed 80 tracks; with the presence of track 80 if either track 80 or 81 contains data; and track 81 if it contains any data. In this example even if track 80 is empty an ADF image requires all tracks present, and none missing, so if it has data only on track 81 both track 80 and 81 must be present in the disk image.

Taking this further, a wiped Amiga disk will give a zero-sized ADF by design.

It is also possible to adjust the tracks in a disk image to an arbitrary uniform length. For example, a disk may contain 9 sectors on some track or 10 on others, and the target format only supports a fixed uniform size for all tracks - but that size can vary. In this case, after having analysed the entire image, it is possible to tell the smallest track size that would be able to hold all sectors on the disk without data loss and adjust all tracks accordingly.

For all this to work, the export system holds the disk image and active geometry settings in memory; track image data can be changed any time until the disk image is closed. Once the disk image is closed tracks are committed to the output image as per the predefined geometry and taking track dependencies into account.

It may not be obvious from the previous section, but one point is that it is perfectly possible for any number of exporters to be active at the same time.

For example:

• Dump an Amiga disk and output it as DRAFT, CT RAW and ADF at the same time, or a C64 disk as D64 and DRAFT, etc. All done using the main data stream being dumped from the KryoFlux hardware. No need re-dump the disk or use other converter tools, or even read the same track twice for different formats.

• Dump a dual format disk (Amiga/ST in this example, but anything is possible) to .ADF and .ST formats at the same time keeping only the data that is relevant for each image/platform in each image, plus write a DRAFT file at the same time containing all the flux reversal data.

This functionality could even be spun out to different threads to speed up the whole process if needed, though it’s probably fast enough.

Made a simple format analyser framework. It’s not intended to replace CTA (which uses a very complex system of probabilities, finding similarities, scripting, etc.), it’s just good enough to add support for various platforms, e.g. try to read and generate track data for a well known format such as the one used by AmigaDOS, generic MFM PC FDCs or C64.

For our purposes, only a DRAFT-like low level dump will do to get the data we need. However, we appreciate that most people will be interested in the common sector image formats currently usable in emulators.

The relationship and limits between the various subsystems is as follows:

• The CellAnalyzer understands and supports all functionality related to analysing the flux reversal periods present in a stream, and is also able to convert to bit cells and/or bit streams, simulating the behaviour of the relevant parts of an FDC.
• The FormatAnalyzer supports analysing data creating the FDC and/or the OS supplied track data required in a format. It is possible to bypass it for formats that only need lower level data output by the CellAnalyzer, e.g. flux reversal periods for DRAFT, or bit cells for CT RAW.
• The Export System converts the track data to the desired output format and disk geometry supplied.

As you can see it is possible to have the exact same data generated by the CellAnalyzer and/or the FormatAnalyzer to be represented by different types of disk images (even at the same time) as their functions and data are not coupled in any way.

When doing output to multiple image types, results can also be shared between the systems as needed. For example, there is no need to repeat cell band analysis or create a bit cell stream if it’s already been done and the target system uses the same parameters. For example, ST and Amiga are both 300 RPM. Only steps that differ for a target platform need to be re-done on the flux or track image already present in memory.

Added an AmigaDOS to ADF format analyser.

It supports detecting of the cell size and switching to DD or HD ADF images or even detecting a DD only image that’s been written to an HD disk.

It builds the correct track data as much as is possible; alterations of the standard AmigaDOS blocks are marked as warnings; errors such as header or data EDC problems as hard errors.

It should be able to read all normal disks as well as protected disks that vary the standard parameters - although many of those would fail to work in ADF format if they actually check for the presence of the altered parameters, since a standard ADF only holds a simple sector content dump.

Support in the host for logical and physical tracks. For example, a Commodore 64 flippy disk has an offset where say track 0 (as far as imager software concerned) is a fixed logical point, but physically the drive attached for dumping may have to dump a different track to read the expected result on side 1 (e.g. negative track numbers). It is wrong to mix up physical device requirements (such as seeking to negative, below track 0 tracks) with disk geometry, so these two had to be separated.

The host now reads all physical tracks required by the logical disk geometry defined to have all the real (logical) track data present. Further processing works using the logical tracks, as how the data was physically read is of no concern, but what it logically represents is. For this reason, if stream (test) file output is enabled, KryoFlux streams are saved with their logical positions from now; when reading stream files physical track layout is ignored - it was organised to be logically correct at the time of output. The same is true about any other analysation process that uses any kind of track information; they all work with the logical track number.

By iterating through physical track numbers and matching with logical track numbers and geometry, it is possible to still read and process a disk quickly even if it uses a different physical track zero point for side 1 without moving the head back and forth to read the same logical track, or having to dumping the entire side 0 first then side 1. The latter would take more time due to the required time to move and settle the head and is more than just switching disk sides and wait for the head to load.

Track limits of the KryoFlux device now get reconfigured according to the physical/logical track relationship specified.

Error reporting is very detailed now, and all output is user configurable. Any problem can be easily identified with the subsystem where it happened. For example the ADOS/ADF converter sets the error data, and that is used by other subsystems, e.g. print error messages to the console or even retry dumping the data from the KryoFlux board and attached drive.

The console output now shows the format type and any error information. If you see OK, it’s good, if < error> then something is bad and it’s now retrying the read. You see a “read failed” message if it has given up trying (max retries will be configurable by the user, with a default of 5). If you see an error, then OK for the same track then a redump succeeded. If output shows < unformatted> if the data is not meaningful for the selected format. It may or many not be formatted using another format instead.

You can also see the number of sectors (expected), the number of bad sectors, and the number of missing sectors.

It would be even nicer to have detailed reports on each sector, listed with errors or warnings about modified format parameters displayed, but at least the output is very informative now.