I could say that all it started when I started programming Atari ST floppy related SW - in 1987-88. First program was COPYACC, done in pure ASM, with low level FDC access, mostly for speed. This is relevant because floppy and ACSI port use same DMA chip in ST .

In 1992 I bought for some 100 DEM used 40 MB Seagate IDE hard disk with plan to attach it to Atari. Inspiration came from PC HW book, this page:

      
 

 Well, those with some knowledge may conclude a lot from above schematic.  What is essence is that IDE (AT-bus in those times) works on very simple principle:  CPU adressing some I/O port - in this case IDE port and basically accessing it as RAM .
With write can set mode, write address of sector(s), count - what want to read into RAM of computer or to write from it to disk. Of course, there is command to start some operation too.  After start of sector read command, driver SW needs to wait that data is ready (signal) and then can read whole sector in 256 steps - for 512 bytes, since it's 16 bit wide. SW needs self to address RAM and increment it, while IDE port for data is same all time - disk will supply data in order.

On HW side, IDE adapter can be very simple in 16-bit computer. I did put my design from 1992 in about 1998 online, and was one of the firsts.  Later made IDE adapter for Sinclair Spectrum, what was more complicated since it is 8-bit . And disks of that time could not work in 8-bit mode.
In beginning I used diverse existing hard disk driver SW as driver. And mostly partitions under 32 MB size - that is important factor , more about little later ...

Started work on own driver SW around 1998 - first version was so called CHS IDE driver - good for smaller capacity drives.
I hade some 60 MB conner, 160 MB IBM 2.5 inch drives in my 520 ST expanded to 2.5 MB .

Then was some pause in my Atari activity, but I went back around 2006 . When good and fast enough emulators for Windows appeared. 

I joined atari-forum (AF in further text), Atariage, Atarimania .  Made some floppy SW, but this time for Windows (FloImg) .
Then wanted to make some special IDE adapter - for ST cartridge port. And I wanted that it be fast, faster than Paskud, for instance.
1 thread from that time:  Thread named 'Some Idea'
Was not pleasure, and as it was usual, I learned that best is if I do as much is possible alone - what usually meant 100% of work :-)
But I then invented complete new way of fast cartridge port write. And said there that will not publish it. Partially because was disappointed by people's reactions and attitude.

Fast IDE cartridge adapter  CATA design started, and I had working prototype in month-two. As usual, SW part took more time than HW part.
More details:  Astide
At middle of page ...  Note that it is old content, in meantime it is improved, and read speed is much better.

Later CATA is basically same considering HW, but there is new, superfast mode - speed is 3.6 MB/sec with 8 MHz Atari ST(E) .
Price for it is little intervention in machine - need to cut 1 R/W signal, and lead out 2 wires, so can invert that signal when needed.
Without it, max read speed is as usual by IDE adapters - 1.6 MB/sec, little more if using blitter .

I could now write in short principle of work, but it will only happen if someone specificaly asks for that.


TOS and work with hard disks.
 As we know, TOS is old creation, from era when hard disks were expensive and rare, but there is support for them starting from first TOS 1.00 .
Things were similar like with floppies - using standard format, but not all details were PC (DOS) compatible. So, there is FAT16, with Intel byte order (low Endian) for compatibility, but it is compatible only for partitions under 32 MB size. Larger ones, called BigDOS FAT16, or by Atari AHDI something like BigGEM . And that is part what certainly Atari and DR (Digital Research) did not well.
In short: Instead 32-bit sector addressing TOS stays at 16-bit one, and result (better said drawback) is that then needs to use so called large sectors to be able to access partitions over 32 MB. That means need for larger buffers, slower and less efficient work. The reason was most likely not so good C compiler used, and lack of motivation of programmers.  And if it can be justified for early TOS versions, no excuse for why it was not improved in later ones. Even Falcon TOS 4.xx is same in it.  And it is possible to do it better - myself corrected TOS 1.04 and 1.62 with real FAT16 support - where of course 32-bit sector addressing is used, + many other corrections. 2 months of work.

My article from 2014 about TOS and hard disks

And something interesting, what I seen after googling for my own WEBpage:
Type in "atari tos fat16 bigdos" - and first 3 hits will be from my site. No, I did not pay anything to Google for that :-)


To be continued ....