Data storage

Here I will talk about data storage on microcomputers, generally and Atari ST related - considering low level of informations circulating around on diverse forums and WEBsites for Atari.

_Data storage: it means permanent storing of any binary data on some internal or external medium, without need for electric power. And of course possibility for easy retrieving, loading in micro.
In 1985, when Atari ST arrived on market, floppy disks were best medium for data storage, considering price, speed, requirements for distributing new SW.
So, Atari ST is equipped with floppy drive, necessary HW for interfacing it with core, and of course support in OS. Even name of OS - in case of Atari it is GEMDOS is based on word 'Disk'. GEMDOS is very similar to much more spread DOS (also Disk OS), and supports same floppy disk format/system: FAT12.
But Atari did more than floppy support: there is ACSI interface, intended for hard disk usage in first place. It is fast for it's time (up to 1250Kbytes/sec on ST), and with original addressing capable to access max 1GB capacity. Since hard disks were pretty expensive in those years, not much people had them, and SW publishers, especially game publishers did not support installing of SW on hard disks.

_Loading stored binary data: it means practically loading required data on exact memory location, and nothing more.
This is point where is most of misunderstandings. But first, little theory about: purpose of DOS is to make data storage easy for humans. Therefore we will not load some data by giving command like: " load file #34972 to RAM address 50000 " . So, DOS uses filenames, directories for organizing mass of files. Then, DOS takes care about RAM usage etc. All it is not simple, and filesystem handle takes aprox. 35 % of GEMDOS.
Atari people talks about incompatibility of some hard disk interfaces, and even worse things: that some types of interfaces slowdown machine, require long driver SW (much RAM occupied) etc. All this is just lack of knowledge, or prejudices.
In reality type of (hard disk) interface is not relevant. Atari ST has only one built in - ACSI. But it has big flaws, especially today: there is no disk directly attachable on (not counting couple of still working Megafiles). In meantime IDE hard disks, CD ROM drives became dominant on market, so normal is that diverse IDE interfaces for Atari machines arrived. Even in 'official' ones: ST Book and Falcon. But Falcon was end of Atari 16/32bit micro era. Despite it, some new interfaces, drivers, supporting SW was made in following years, and is still made - in very limited amount.
Back to desinformations about slowdowns caused by not ACSI interfaces: some 'experts' talking that only the DMA (direct memory access) is good for hard disks, and that PIO (parallel Input/Output) is bad, because causes slowdown generally. All it is pretty wrong, and shows that they have no clue about how things work.
In real work hard disk IF operates only when some data access is requested. Then in 99% cases all what machine dooes is waiting for data transfer finish. After data transfer is done, all goes further same as there is no any hard disk IF in machine. So, it is irrelevant how data transfer self occurs - DMA or PIO. In case of Atari ST PIO is actually faster, because CPU can move datas faster than DMA chip. And even with IDE we have option for DMA in machines with blitter. Then speed may go up to 1900Kbytes/sec (ST) ! Only exception where DMA could be really better is multitasking (not much usable on 68000 systems) - because then disk access of some background process will not slowdown so much active process as with PIO. But it is really not something problematic as hard disk access is usually very short, under 1 second. Plus, all it is controlled by OS, which may distribute CPU time so, that background hard disk access will not use 100 % CPU time.

_Compatibility: One of main arguments against non-ACSI hard disk interfaces on Ataris is that they are "not compatible" . This is pretty pathetic, together with claims that only good things are what Atari made, recommended. We know that best hard disk IFs are made by 3-rd party as ICD for instance. And they used some new solutions, not following Atari. Actually, biggest problem with Atari ST machines is lack of expansion bus, connector. It means that adding some IDE IF requires opening machine and soldering a lot. Except if you have Mega ST or Paskud. It is the reason why diverse ACSI-SCSI and now Satandisk are so popular - just connect adapter and use it... Just to mention that there was even IDE CD ROM IF for cartridge port available from ASH.
In reality all possible incompatibility is SW nature: starting with inability to boot from hard disk. What is case if you have TOS older than 2.06 and IDE hard disk. But people made so much diverse TOS patches, so adding IDE boot to some older TOS is made too. Paskud is special case: it boots with SW in cartridge EPROMs, as that IF is on cartridge for ST cartridge port.

Everything after booting from ACSI port (except FAT16 filesys. handling) is not handled with TOS in fact.

Because booting from ACSI is all what all TOS versions have with hard disks. There is FAT16 support too in all TOS versions, but first we must to access partitions, reserve some RAM for buffers etc. All that job is on hard disk driver. Actually, hard disk driver itself, if made well, and written in ASM takes very little RAM - 4-10KB only. Part of driver which installs, recognises partitions is not needed later. More space we need for buffers, 32KB minimum for partitions up to 512MB. IDE drives of 10GB can work perfect with 40KB RAM reserved for driver, buffers, tables.

The _Real compatibility problems with hard disks on Atari ST: Actually, we could say that there is no compatibility problem - simple, because old SW is not intended for hard disk usage. People just wants too much. That's all :-) .
Well, I don't think that it is too much to want to run some game from hard disk in 2009. Then, by Atari we have a lot of incompatibilities even when run SW from floppies, mostly with TOS 2.06.
So, most of so called hard disk incompatibility is just because SW is intended to run from floppies. Starting with simple thing that launcher of PRG will seek files on floppy A, and not in current DIR. Or what is case with many games: loading data with direct floppy access, bypassing GEMDOS filesystem.
All in all, 'incompatibility' has nothing with hard disk IF type.
Generally, if something works from ACSI attached hard disk, it will work from IDE, Paskud too. Possible problems are related with hard disk driver SW, and not with HW. And those driver SW problems may be: too much low RAM occupation, Timer C sensitivity (HDDriver will fail if Timer C is stopped). Obsolete driver, not working with newer hard disks (Hushi, CBHD).

Just to add here for completeness that there is a few, only couple SW which has own hard disk driver, for ACSI port. But I even don't know which are they - after some discussion we decided that Notator works not only from ACSI drives, as some people suggested.

_SCSI vs IDE : SCSI is for sure more advanced than IDE. But in small, home systems with 1-2 drives SCSI advanced features are not used. IDE is good enough for. And IDE is developed a lot. Today, SCSI and IDE is aimed for complete different usage, and price difference is huge. It has sense only to buy some older, second hand SCSI - new drives are very expensive. It was not so before 15-20 years, when SCSI was only slightly expensiver, and we had same base drives with SCSI and IDE interface. I read even in late 90-es about that IDE has no perspective and that SCSI will soon prevail as IDE has many problems with bus compatibility for instance. But integrated EIDE controllers on motherboards resolved all problems with speed and compatibility. Modern IDE drives can over 80 Mbytes/sec.
In case of Atari ST machines simple IDE IF attached directly to bus is good enough because we can not achieve some bigger speeds due to machine's slow RAM. Only drawback is timing problem with some CF cards. Solution would be ACSI-IDE adapter - it would solve timing and attaching/building problems at once. Not to mention possibility to attach CD/DVD drives too...

P. Putnik, June 2009.