Playing Atari ST games from hard drive - in details

and.b #%11011110,$ffff8007.w  *8MHz, STE bus
  You can use all above visible, modern storage devices on Atari ST serie, Falcon:  2.5 inch IDE hard disk, 3.5 inch IDE, SCSI with UV, Satandisk with SD card, Compact Flash card etc...
Adapters may be required for some combinations.

  As we know floppy disks were used as main storage and distribution medias in 80-es, so games for Atari ST machines were distributed practically exlusively on floppies.
Most of games fit on single floppy disk, but there is a lot of those which was on 2 or more.
  As mass storage became cheap and old machines are still in usage there is a need to play old games from hard disks or diverse Flash Cards, attachable on Atari machines via diverse interfaces.
But solving playing of games made to run from floppy disks,  from hard disks is not simple thing in most of cases.

The situations:
  Best case is when game is on regular floppy, without protection, so you can simple copying files to some DIR on hard disk, and run from there.
 Well, simple copying and successful running is not same:  in many cases it will not work.
     The problems:   1: game still wants to read files from floppy  or finds not it's files
                             2: game crashes after start or at some point

  Reason for first is usually that game uses absolute paths pointing to drive A instead relative ones. Solution is simple in some cases: just edit executable and remove A:\ before filenames.
  Similar case is when no drive letter, just backslash \  before filenames - it means that game will run only from ROOT dir of some partition. Cure is again editing of executable + directory names by need.
  I did so in couple cases as Star Wars or Defender of Crown. Editing may be done with some good hex editor as HxD (free).

fied1.png    Step 1:  select filename itself, together with terminating 0, after A:\  . Press CTRL-C (Copy)

fied2.png    Step 2:  select from begin ( A:\ ) . Same count of bytes ! HxD will warn if not same count.

fied3.png    Step 3:  press CTRL-V (Paste) . Last edited byte must be zero. Filelength must remain same.

  If you see that game reads floppy, and runs not, complains about missing main floppy or similar, there is some protection in question, most likely. Then solution may be to look for some cracked version.
  Or to adapt game by self - later more about it, for creative people.

  Usual reason for  second (crash, freezing and similar) is that game is made to run from low RAM, which is occupied with hard disk driver. It happens mostly by older games. But similar problem happens too because of  TOS versions. Later TOS versions use more (low) RAM, so game overwrites system what results usually in crash. It happens especially often by Falcon, which has largest low RAM usage.
 Solutions: running game from AUTO folder. Lowering hard disk driver cache, or using such which can run from high RAM. Using different TOS version.
  If nothing helps some patches must be done with game.
 Examples:  Flight Simulator II (Sublogic)  it is on unprotected floppy, and can be copied on hard drive simple, and will run from any DIR. However, it uses lower RAM, and may fail on Falcon or TOS 2.06.         Running from AUTO folder is a solution.

  There are no files visible on floppy, or is just few of them, short ones (Sundog for example) :  game is stored in non-standard way. Solution requires expert who can transfer disk content to file(s). Of course, many of such games is already filed.

  The problems with already filed, cracked games:  many of them can run from hard disk by simple copying in some DIR. But many will not run - from same reasons as described abowe by copyable originals.
 So, additional fixes, adaptions may be required.

  Doing adaptations :
   It is usually hard, and requires time, machine code knowledge, architecture (not that, but of Atari machines, TOS) knowledge. Fortunately, we have now a tool which makes it much easier: Steem Debugger. With it you have fine overview about what happening. May fast find problematic points in games and similar. Even cracking is much-much easier that on real STs.

 The situations and solutions from my experience (updated at Oktober 13 2009) :

 Games without files on floppies:  need to disassemble boot sector and see what it and further parts load, where. In simple case game will load whole content at once and starts. Such was V*rus. Additional problem may be copy or 'manual' (asking for some words from manual) protection, but I will not go here about it. Then need to make loader which will place whole content at right place in RAM and start game. Examples:  Carrier Command, Jimmy White Snooker.
  Harder is with those ones which load from floppy during gameplay.  As we don't want to bother with floppies must solve it somehow: usual way is to use RAMdisk - placing floppy content into RAM, and modifying game's loader so that it will read from RAMdisk instead floppy drive. It requires some more RAM, but it is usually not problem for hard disk owners. Examples: Xenon 2, Battle Command.
  Next one is when game is on 2 or more floppies. Then we can again use RAMdisk (and need more RAM), but with additional task: finding how game knows and asks which floppy is/to insert. It can be done unvisible for user, so game will never ask for floppy change (F1GP adaptation, for instance).
  RAMDisk is very fine solution - it is not sensitive on Timers, HW state, does not need TOS. But needs RAM. Actually, I think that if RAMDisk can fit in upper half of 1MB, it is good solution. If it must be larger, then is better to go on loading from hard drive solution. Then 512K games will be usable from hard drive on machines with 1MB. More about later, below.

 The hard nuts:
 One of worst cases is when game uses low RAM and TOS functions too. It results in non-work in higher TOS versions usually, even from floppy.

 Solution could be special RAMdisk: with bypassing TOS  filesystem handling, and direct file access for game files. I made small proggy which puts all files from current DIR in 1, RAMdisk content file, together with required file datas (same as in DOS directories). Then simple code can read files from RAM instead ffrom disks. Done with game Rock'n Roll. But by larger games it may need too much RAM.

 Even worse:
Game uses VDI, AES calls too. But is placed too low in RAM for higher TOS versions,
Example: Millennium 2.2 - works not under TOS 2.06 and higher. Running via AUTO folder solves it not - then no mouse cursor....

After failing with RAMdisk and filesystem bypass (it worked, but no keyboard input on TOS 2.xx and 4.xx ) I thinkered out very simple and effective solution:
Leaving hole in RAM before loading AES and Desktop and hard disk driver. In that case low area will be usable for many problematic games. Maybe loaders need to be little corrected.
How to make hole in low RAM?  Very simple: with program in AUTO folder or in case of hard disk - on bootable floppy. Program just reserves RAM up to 512KB pos, so driver, AES will load after it. Then game, which must run in low RAM (usually about pos. $12000 )  will not overwrite any system structures.
Latest thing - to use bootable floppy when have hard disk may sound stupid, but it is only solution at moment, in case of not mine hard disk drivers - it must be executed before hard disk driver boots. Latest versions of my hard disk drivers have option for Hole creatiuon during install (booting).
On page with adapted games is now Millennium 2.2 with Hole installing programs. By need we can use holemakers up to 1MB or even more...

  Problems with TOS 2.06 and later, and some games, not memory related:
ome games start, but stuck on same point - Flight of the Intruder, Space Harrier ... Problem is Timer C (200Hz timer) related. TOS 2.06 uses timer C by regular Traps, unlike older TOS versions. If it is stopped it does endless loop. So, if game stops Timer C, usually when stops sound output then it will stuck.
  I think that Atari is guilty for this problem - by changing Trap behavior. Unless they noticed somewhere that Timer C (200Hz) is necessary for Trap calls - before TOS 2.06 .
  Solution: restarting timer C regulary - in V-blank, before Trap calls. With it working of Space Harrier, Flight of the Intruder  and F16 Falcon is solved on TOS 2.06, Falcon.
 But not only TOS 2.06 needs running TImer C. Hard disk drivers as AHDI and HDDriver also need it. When game stops Timer C disk access will cause stucking. Examples: Defender 2, Car Vup  etc.  Solution is small TSR util, which will restart TImer C before any hard disk access and restore it's state when finishes. See for on page with Open Source SW here.

Running from AUTO folder: 
Running from AUTO folder on floppy is the usual and preferable way. But with hard disks it usually just complicates usage. If you have enough RAM, then in most of cases game will work with DESKTOP launch too. But as always, we have some exceptions:  FTL games as OIDS, Dungeon Master... Ultima III, IV . What I saw so far is:  VDI init - game performs VDI call  'Init workstation' at start. But it is destructive after AES init. Solution is to change that call to 'Init Virtual Workstation' - case of OIDS, Ultima III, IV, CHSB Utility disk. Dungeon Master uses all possible RAM , so the area intended for GEM too (area below Membot). If want to run it from Desktop need to reallocate game's buffers to different area (free RAM).

Falcon game compability:
  Usual reasons why some game fails on Falcon:
  1. using PSG shadow registers ($FF8804-6) - on Falcon it crashes.Solution is setting STE emul. mode on Bus control reg. In some cases sound will be
distorted after it, so need to change game code by using  not long writes to PSG regs.
  2.  game uses very low address space at $700, where PMMU table of 68030 is placed. Solution is moving table in upper RAM before gamestart.
  3.  self modifying code falls - solution: disable CPU cache(s)
  4.  game is too fast - solution: set CPU to 8MHz, cache(s) off .
Above situations can be solved by using known programs as STE2FALC or Backward. Or by making special FAlcon launcher of Game.

Less known possible problems:
   Screen disalign - happens by direct write in ST screen base registers ($FF8201.. ) . Update: it happens only if screenbase is on odd $100-ts address.
But not always. Case when it is not fixed: Parasol Stars. Game alternates between 2 screen bases, where one is on odd $100-t. Changing it would be too hard, as at many places in code it is referred. As error shows only in rare cases, no worth to bother, I guess.

  Exception handling code:  68030 uses different stackframe by exceptions. In case of Traps there is 2 byte more on stack top. So, code which deals with exceptions needs to be modified. As I see, game Simulcra heavily depends on exception stack layout, and fixing it for Falcon is almost hopeless.
 By interrupts there is even more difference. Some games use code for V-blank interrupt, and it fails because of different stack frame. Simplest solution is to use Vblqueue in such cases. (Sundog, Startrek).

 MFP issues:  as I see, MFP in ST and Falcon works not always as it should be by book. In some case false triggering occurs, for instance from serial port. That causes usually not responding IKBD. Solution against is to disable not needed interrupts as those from serial port. It was made in case of Damocles, Starglider 2, Paperboy etc. Problem happened by restroring gamestate on Falcon, and in rarer case on STs too.
 Some games write in MFP registers and then read back - needed as Timers may show written data not immediately. But on Falcon it may cause stucking, as some regs will never take written value (not gone deeper in that so far). It was so with Arnour-Geddon, and I just disabled that reading back and waiting.

 Timer A is little different on Falcon. It may cause again non-responsive keyboard, mouse by games using it. Example:  Starglider. Needed to rework laser sound playing routine for Falcon.

 I saw direct ROM jumps in couple cases. Of course, such games work not even on ST with some higher TOS version. Need to change code in such cases, what may be not simple...

 Not disk or memory related incompability:  see above: "Problems with TOS 2.06 and later, and some games, not memory related" .

Generally. there is 2 main way to make game 'compatible' :  1st is changing game code itself (patching it).   2nd is changing computer settings, environment (RAM layout, Traps, bus behavior, CPU settings etc...)
   Of course may be both. Changing of code may be done with launcher program, which can do some settings on machine (CPU speed, bus settings...) too .

Crappy programming:  there are some very strange ideas of  programmers in few games. As writing in GEMDOS RAM space by Skychase  and  Predator. It results usually with crash in different TOS version than programmers tested (from floppies too) . There are some TOS ROM depending rutines with assumed characteristics. But they fail on later TOS versions - IKBD read by Prince of Persia, Super Cauldron .

 Table with some RAM locations for diverse TOS versions.
 Row 'Desktop' shows where will some executable load when launch from Desktop. Row 'AUTO' shows where will load when run from AUTO folder. Of course, values are for clean machine, without any ACC, drivers etc. So, if you use hard disk driver, it will raise positions of load for AUTO and Desktop,
Values are for UK TOS versions, smaller diffs. are possible by other lang. versions, by Desktop load pos. If executable is TOS and not PRG loading locations are lower for some 15 KB, depending on TOS version.

TOS  1.00
TOS 1.04
TOS 2.06
TOS 4.02 (Falcon)
$33298  (Set to DE, ST low res.)

  Some further research shows that area between  "first adr. not used by OS" - actually by GEMDOS, (defined in OS header) and Membot remains unused when starting SW from AUTO folder.  It is over 24KB in case of TOS 2.06 ! That area is used by Desktop, regardless from RAM location where AES is loaded.
 I saw only 1 PRG what 'correctly'  uses that area:  Dungeon Master.

If we launch some SW from Desktop which uses Desktop workspace TOS will crash. Reason is that in workspace is stack and some variables used by Timers. Such problem happens when start some game made for very low RAM (Sapiens) in higher TOS version, or by Dungeon Master which is intended for AUTO or boot launch.  Solution is: change Timer C interrupt code to bare counting. If we want return to Desktop from PRG need to save Desktop workspace somewhere and restore it before exit from PRG.

Detailed RAM usage:
From bottom upwards, values are for UK TOS variants.
Space for:
At, by TOS 1.04
At, by TOS 2.06
Screen buffer, 32KB
32KB below Phystop
32KB below Phystop
User SP by SW start
Right below screen
Right below screen
Free space for SW

Desktop run,  PRG $12496 if clean
$18772 if clean

Desktop run,  TOS
$F096 if clean
$148F2 if clean

AES workspace, Desktop RSC
$A956 if clean
$CDBA if clean
Optional drivers, resident SW - boot, or AUTO
For instance hard disk drivers come here, by autoboot .
TPA begin
From here RAM alloc. goes in order from bottom upwards, by needed size.
Desktop workspace
Hardcoded in ROM. This area remains unused by AUTO start SW.
GEMDOS workspace, variables
$800 Hardcoded in ROM
Not used
$600 Here may come some short code, reset-proof init.
System variables
$400 $400 Hardcoded in ROM
CPU vectors
CPU specific

Possible conflicts:  We want to run some SW in low RAM space - because it is not relocatible or whatever reason. If area is below TPA begin for actual TOS then:  Timer C rutine writes in Desktop workspace. Must set SSP to different area. Every Trap #1 call will write in basepage area of last started PRG. Pointer to basepage of active process is defined in TOS header (offset 28) . By setting pointer to some 256 byte long unused space we will avoid possible crash. In case of Desktop launch basepage is relative high (may be over $18000).
If game uses GEMDOS workspace we can not use any TOS calls - game must take care about everything. It is OK in most of cases, and when we use only floppies. But when we want to access hard disk during gameplay then we need to solve couple problems:
1:  Replacing floppy accessing code with our, hard disk accessing code in games.
2:  Work of hard disk driver. It is usually just overwritten by game, as beeing placed in low RAM.
   Solutions for later are:  using RAMdisk instead disk access. ULS, what swaps TOS RAM space with game content by need. GAC - gamecache, where we have low level hard disk access and simple code using very little RAM.
   But really good solution is just done:  it is modified GEMDOS 0.15 (from TOS 1.04). Modified so, that not uses low RAM - workspace is moved in high RAM. So, there is no conflict.
So far, I made 3 variants. Variant 1 (D15R_NG1, ..F1) is for games sensitive on TOS version and loading all at once (or singleparted by adaptor/cracker). It runs in high RAM, and loads no hard disk drivers. Result is free low RAM, so gamestate saves will be no longer than 512KB for 512KB games. No need for bothering with AUTO folders. Used by Starglider, Skychase etc.

 Variant 2: (D15R_HH4, ..F4) is
for games sensitive on TOS version and loading datas during gameplay, but with correct relocatable TPA code. Benefit too is smaller statesave, as we use lower RAM. + Timer C problems gone. Hard disk driver is reloaded in high RAM. Examples:  Iron Lord, Rich for the Skies - work like charm on Falcon :-)

 Variant 3:
(D15R_H5, ..F5)  is for games not using TOS, but loading during play from floppies. It solves hard disk access with regular drivers by using TOS 1.04 GEMDOS filesystem code, placed in high RAM. Hard disk driver works also in High RAM. Benefits: fast work, as no need for huge RAM swaps. 1MB RAM is enough for 512KB games to run from hard drive + gamestate save/restore possibility. So far used by: Creatures, F1 GP, Armour-Geddon. More coming soon....

  As latest step in TOS modifying I see rework of GEMDOS FAT16 filesystem handler - with PC FAT16 compatible one. This will allow easier data transfer, larger partitions (up to 2GB) and less RAM usage with buffers.

Categorization of game sources (releases) beeing adapted:

Categorization is good to make things easier - we may use common code for several games, and changing only few parameters as filenames, loading and executing locations...
Simplest case for hard disk run is: singleparted, so need to load (+depack) and run only 1 file. There may be addit. screenshots as title pic(s), but it changes not launching self.
If game has more files then question is how it loads them - via TOS filesystem or with direct floppy access. In later case we need rework of. But problems may happen with filesystem access too - as game still may be for too low RAM, or because of changed Timers. Best is if game has TPA executable, then no RAM conflict.

Concrete game launchers with Gamex, sources:
Castle Master
  it is category:  M1TPL - 2 execs, 1MB min (from hard disk), TOS dependant, TPA, filesystem using.
Sideways  category: S5I - singleparted (by me), 512K, independent from TOS after load. It means that exec. is not relocatible too.
Starglider  category: S5O, with GOS install - singleparted (by me), 512K, little TOS dependant. Because problems with TOS 2.06 it is with GEMDOS 0.15 in RAM (GOS 1.00) .  Also, AUTOrun without resetting machine !

GXUTIL  Gamex utility source - devel. vers. 20 (V 0.6) .

Little supporting code:

 Small code for setting Mega STE to 16MHz, cache on. On other machines will do nothing 'bombastic' - so can place in any loader. Code is PC relative.

supv   *Must be executed in supervisor mode
* Setting 16MHz, cache on,  if Mega STE...
   move.l  sp,a3
   lea  busertr(pc),a1
   move.l   8.w,orgbue-busertr(a1)
   move.l  a1,8.w
   move.b #$77,$FFFF8E21.w   *16Mhz, cache on
*Will do bus error if not Mega STE

backorb move.l orgbue(pc),8.w

busertr  move.l a3,sp
  bra.s  backorb

orgbue  dc.l  0 ****

    PP , August 2008 - Oktober 2009.

Menu Properties Quick Reference