This is attempt to make almost complete user guide to people equipped with some mass storage - in first place hard disks, Flash cards via adapters, on their Atari ST(E) machines and folowers. Will focus on up/to date (it is end of year 2009) solutions in HW and SW. Will describe things without too much technical details, for average users, as much it is possible. But some elementary knowledge is necessary.

Update at Oktober 2014, with new experiences, discoverings:
Currently many Atari people using TOS/DOS compatible partitioning because it allows easy file transfer Atari-PC (Mac). It works best with some Linux. In case of Windows  there are 2 problem types:  Windows XP and followers access only first partition on removable drives, so on now most used Flash cards. There are some solutions to override that limit, as CFAdisk driver, or Dummydisk. There is page here about it, I will put link here after updating that page.
Another problem are Windows long  file names (LFN) - it creates additional records in directories, for which TOS is not ready, and diverse errors may appear, starting with incorrect informations about used partititon space, inability to delete some files, directories, or even crash. And it is even worse with newer Windowses, Service packs. For instance Windows XP with SP3 will create by default always when creating new directory the System Recovery directory . And it is not easy to remove. Talking about case, when user installed solution to override first partition only limit with Flash cards.
 Please read this article further, and will see some lower hints about avoiding LFN on Atari disks, cards.
I solved filtering of LFN in my latest driver update .
There are plans to make special LFN clean up SW too .

TOS 1.02 specific problems:  most known is that TOS 1.02 can handle partitions of max 256 MB  , and that work is unreliable. That it needs Folder100 for better work.  I did some tests with 250 MB partitions filled over 60%, and must say that it is practically useless. You will soon see some directories shown as empty, while there are files in, then some partitions as empty, incorrect infos.
 
As we may see, there are even not enough digits to print values over 99 MB. But that's smallest problem.  Even with Folder100 it will lose access to most of data when partition(s) contain many files. I guess that when it was tested, only very small drives were available (in range of 20-40 MB).  Then, pressing Escape in Desktop with opened partition window will refresh only displaying, but not rereads disk, as is in other TOS versions. All in all, any serious user, gamer must update TOS 1.02 to 1.04 or 2.06 (what is little harder) .

Displaying of hidden directories: TOS 1.02 and 2.06 will show hidden directories in Desktop, while 1.04, 1.06/1.62 not.

Some things can be corrected with patching TOS. I did LFN filtering in TOS 1.04 and 2.06, and that was not hard.  On request I can suply updates, or updated TOS ROM images.


_{Basic facts}:  Part of TOS responsible for work with files, directories on disks is called GEMDOS. But we will refer here mostly generally as TOS. Important is to know that we have some limitations depending on TOS version used:

Above screenshot is taken from Hddriver 7.80 .  At top are maximal partition sizes.  Low are less known  partition count limits. But it needs some further explanation:  limit of 14 partitons (C-P) for TOS (any version) is actually limit of installed logical drives. If we attach 2 disks, and  total count of partitions on them is above 14 we are in trouble:  whether some partitions will be inaccessible, or we will have crashes, 'Folder100' error messages - when access some logical drive over P:  . There stays at bottom that no upper limit for drive's capacity .... Hmm.. - not sure what author wanted to say, but there are such limits:

_{Drive/media capacity limits}:
  Not the best formulation - we should talk about max. accessible capacities.  It depends mostly on used HW - ACSI/SCSI adapter for instance, but from SW too.
With most of ACSI adapters for ST serie that limit is 1GB. For instance with built in ACSI/SCSI adapter in Mega STE or STacy. With IDE adapters no HW capacity limit in reality (it is 2 power 48 sectors with latest ATA specs).
  Best ACSI/SCSI adapters from ICD can over 1GB, and new UltraSatan too. 

_{SW capacity limits}:
1.. Hard disk driver limits:  older drivers may have some special limits, but we will not discuss them here. Basically, if driver is intended for ordinary ACSI (adapters) it has same limit as HW - 1GB. If is ICD command capable limit is 2048 GB, at least in theory, as nobody tested with so large SCSI drives. 2048 GB is actually limit of used 32-bit partition tables.
IDE drivers may have limit of 128MB (older ones even less). Latest drivers are capable for much more (using 48-bit LBA), but all it is really above space needed by Atari SW, not to mention limits caused by max. partition sizes and count (see pic above).

2. Simple calculation says:  max 14 partitions mult. by max 512MB per part. means  max 7 GB  usable per media/drive. (TOS > 1.02) .
 I think that this is very relevant point.  Do we need more at all ? Is there Atari SW what needs more space ?
I'm  sure that all Atari SW, capable to run from hard disks (even with help of some utils (as ImgRunner)) can fit on some 2-4GB space.  Of course, some people tends to copy everything, without checking is it usable really. Then may run out of even 7 GB. But I consider it as not racional usage. See later: about data safety, backups - related with data transfer too.

Here I will not go about limits when using some more advanced OS - as Mint or Magic.


_{The filesystem}:
Filesystem is responsible for data accessing, storing, manipulating in form acceptable for humans. Therefore it operates with named files, directories, instead of numeric locations on disks. TOS uses filesystem referred as FAT16 , what is basically same as used by DOS.  FAT16 means some limitations too, but they are not really relevant for average user.
What is interesting, especially now is compatibility with modern PC OS-es as Windows (XP). Linux .  Of course, because of easy data transfer in relation Atari PC.
We are now in pretty comfortable situation:  connecting some SD or CF card to PC via USB card reader and copying data from/onto. Then attach it to Atari and using SW, datas. Unfortunately, it goes not so simple. Or better said, it will not work unless we know some facts (aka 'know what doing'), and using some required SW, special partitioning, drivers.
First problem is that official disk partitioning scheme by Atari is different than by DOS. It means that PC with DOS, Windows will not recognise any partition, file on such disks. In further text I will refer on such disks/medias as AHDI partitioned.
Second, bigger problem is that even FAT16 used by Atari is not same as FAT16 used by DOS. Difference appears by partitions over 32MB (BIGDOS or BGM partitions). Short said, Atari uses so called large sectors to achieve work with larger partitions, while DOS uses multisector clusters. In practice it means that data transfer will not work (or needs special SW).  Or that only data located in first 32MB of partition is accessible on Atari .  Then, writing files on such DOS partition with Atari will sooner or later cause data loss, FAT, partition damage.
Fortunately, there is a way to make TOS-DOS compatible FAT16 partitions. It is possible with Hddriver - max. 1 such partition per drive. With mine (PPTOSDOS) driver - max 14 such partitions, max 512MB/partition. In both cases medias/disks are directly accessible under DOS, Windows, Linux. On Atari, usable only with their drivers.

_{Problems under Windows XP}:
1. When attaching some Flash card with TOS/DOS compatible or even pure DOS  partitions only first is accessible. The reason is that XP mounts only first partition from removable medias/disks, from some reason.  There is a solution: WinXP multiple parts. on removables

2. Problems with long filenames (LFN) :  TOS is not ready for LFN . If such files are on disk, diverse problems may appear:  errors by copying files, impossible to delete directories because of some weird named files in, wrong "Bytes used" reported in Disk Information etc.
To avoid problems, never copy files with LFN to Atari disk/media. Probably best is to do all transfers with Total Commander, proper set for it:



Unchecking third box will result with easier recognition of LFN files. Then rename them to 8.3 filenames before copying to Atari disk.
Or may uncheck second box (Use long names), then for sure will not copy with LFN.

Other thing to care about is deleting files/directories under Win XP: it moves to Recycle Bin by default, what means again troubles with LFN.
To avoid creation of LFN when deleting press Shift when deleting - stays for Explorer and Total Commander. Or better:



Uncheck box at bottom (F8/Del ...) . Then no need to hold Shift when deleting with Total Commander.

How LFN looks with some disk editor - CHKDSK3, DIR Cleanup :



Unfortunately, CHKDSK3 will not clean out LFN extra fields - those where Att is 00F. It is visible that we have some garbage by file Sizes .
I don't know about Atari SW capable to remove those extra fields. But it is easy on PC - just rename LFN to 8.3 or delete. (not in Recycle Bin).

For experts:  may fix on Atari. Need to put (hex) $E5 to locations $80 and $A0 - then extra fields (where attribute is 000F) will become 'deleted files' and not garbage. Problem solved.
With Sector Editing in CHKDSK3. I tried some other disk editors, but experienced diverse problems, so CHKDSK3 seems as best solution, despite not to good user interface.




Partitioning:
  2 partitioning types are usable for Ataris: above mentioned AHDI (called TOS too) and DOS type. Despite AHDI is official Atari type, it is not recommended now. The reason is that is not compatible with DOS, so data transfer with PC is troublesome. Use it only if not plan to attach Atari disks/cards to PC.
DOS partitioning is supported by best drivers as ICD and Hddriver (not by AHDI), but only Hddriver from that 2 offers TOS/DOS compatible, with above mentioned 1 part. limit.

Using pure DOS partitioned disks on Ataris:
It is possible with above mentioned drivers, with mine too (link ). But must whether work with non-BIGDOS partitions, so with max 32MB size, whether to use program named BigDOS with larger ones.  BigDOS is nice SW, allows work with partitions up to 2GB, filters out LFN. But must be loaded from AUTO folder, it disables partitions under 32MB and some SW (mostly games) doesn't like it.



To ensure correct work under TOS we need to care about some things by partioning on PC:
Must maintain 2 sectors/cluster by FAT16 under 32 MB - see picture above.  By smaller drives is best to not use BigDOS, and creating only partitions under 32MB.

Always create 1 primary (first) partition, and others as logical drives in extended partition. FAT, (and not FAT32 or NTFS),  size 2 GB max.

BigDOS (if needed) must be copied on first partition (C) while it is empty, to ensure that will be placed at beginning, and of course in AUTO folder.


XHDI :
Is additional communication SW interface made in early 90-es, to override some limitations of AHDI and PUN. But it is actually not used by TOS SW, except few, rare cases. Installing XHDI has sense if using Mint for instance. Even BigDOS works fine without XHDI. My recommendation is: use it only if must. Why spending  RAM space to something not used at all ?


TOS versions, possible problems :
At top are some limitations depending from used TOS version. But oldest TOS versions as 1.00 and 1.02 may have additional problems. 1.00 is absolutely not recommended, and many SW will not work with, not to mention slow work etc. 1.02 is better, but still not good enough. Users with those TOS versions need to upgrade to at least 1.04. It is minimum for decent work, data safety. Latest ST(E) TOS version is 2.06, and offers many improvements over 1.04. But is not really good for gamers. And upgrade to TOS 2.06 on some ST may be not simple. Atari realised some patches as FOLDR100.PRG, needed by work with many files, directories - at price of some RAM . Not much people will need it.

Common mistakes of users :
Same as it is with Windows systems - people think that best is to install, activate everything. But result is usually just slow(er) work and less free RAM, conflicts.
And by TOS very often problems with older SW. So, avoid ACCessories, resident programs, diverse drivers. Use them only if must.
Best is to have some boot manager with which may easy change boot configs. Or selectable boot partition.

Disk utility SW :

There is a lot of diverse disk utility for TOS - copy, disk editing,  peformance increasing, etc.  SW. Unfortunately, most of is very old and not capable to handle larger partitions for instance.

Kobold 2 is champ among copy programs. It uses intelligent caching, filters out LFN. And it seems that works well with BGM partitions :



Kobold2 copied folder with 114 files in 22 secs, while Desktop copy of same took 2 min 24 secs. It is with slower disk/driver + verify on. With faster disk(s) and verify off can even much faster.

Some new 'discovery' - Sept. 28  2010 : Kobold2 works well with partitions over P :



15-th partition Q is displayed so in Desktop (TOS 2.06) . But often TOS will crash instead opening window with content.

z

What is really on that partition is in right window of Kobold. And of course, you can copy there, from there . So, Kobold is very useful when attaching 2 or more drives, and total partition count is over 14. You can transfer files fast and reliable. After removing other drives, Q will became some lower letter, so can access everything on from TOS, Desktop. See video :





Most of disk editors has similar problems with large partitions, and will report invalid BPB, FAT size... For instance Knife. So far, I can recommend only CHKDSK3.


Data safety, backups :
Nothing is 100% reliable, safe. It stays especially for 20 year old computers. Therefore every should have backup copies of his relevant data, programs. PC may serve as such - on modern drives there is a lot of space. But better, more safe way is to keep backups on DVDs for instance. And it leads us back to data transfer, and why having DOS compatible partitioning is good, especially with modern CF, SD cards. Another benefit of DOS compatibility is data recovery. There is a lot of such SW for PC, and some are even free.


Finally, important note: you did read somewhere different statements than here. Well - WEB is full with outdated sites, incorrect statements, shallowness,  forums where self declared "experts" write everything...  What is written here is carefully checked, plus is result of many years experience. By hard disk driver  programmer  and old Atari user.

 Used literature, sites:  Atari ST ProfiBuch.  Dev-Docs

 





Related, recommended pages, sites:  New speed test PRG
......

If interested, Google little about FAT, FAT16, partitioning and similar.