The Trampoline (Inside booting the Mac with a microscope)

[Protools5LEGuy]

To be more on topic I split some things

http://macos9lives.com/smforum/index.php/topic,3195.0.html

Let's keep "Anyone will to help create 9.2.3 for " for 9.2.3 developments

I beg you all to make a proper post on the trampoline as the Dream team post mentions because that was the only "relevant" info on that post.

Also, the new understandings on how you are disecting how OS9 boots deserve a own post.

Thanks for sharing all your valuable time with our community.

Perhaps I should start a new thread for all of our "theoretical" stuff?

Are you asking for an update post on where we are at and how we are doing this in which thread?

We were talking about the trampoline on "Anyone wanna help out 9.2.3?" And also on the "Dream Team" post. To focus our community to be on topic we thought it would be great to split some topics and merge all the post on unsupported machines with the same specs.

So, Elliot, Nanopico, iMic, Mac OS Plus, MacTron, cc333 and anyone willing, this is da place

[nanopico]

Got it.

[nanopico]

Here's what we know now (this may be across several other threads too, but here it is in one spot).

This should be pretty accurate based on documentation and experimentation, but this process may be modified as we find out more.
The order of a few items might be just slightly off, but overall it gives a good picture of the general order of things.

Outline of the boot process (this applies to new world machines. Old World are a bit different).

• Press The Power Button, Okay I think we all knew that part, but in case you didn't then now you know.
• The boot firmware embedded on the machine is executed first
  
  • Boot firmware interrogates and probes all devices on the motherboard and attached to any external ports.
  • While it finds hardware it builds the device tree. For each device it checks for firmware and properties. And adds this info to the device tree. This includes addresses of the device, interrupt data required to startup and configure the device and any fcode drivers for the device.
    The device tree is just a data structure in memory that is used during boot by the OS.  Manipulating the in memory version will only have effect until the machine is rebooted. (Thus the reason for the use of nvram scripts that have been used for things like changing cpu version numbers).
    
  • The firmware then looks to the boot environment variable to find the device and file to load and execute.
    
  • If you hold down the key combo for Open Firmware then rather than boot the firmware drops into the Client Interface program.
• In the case of Mac OS 9 the ROM file is a CHRP Script with a payload.  Once this is loaded Open Firmware executes the script
• The script sets up memory to hold the trampoline code and the parcels package, and sets variables for Open Firmware to know where to copy the code to and where to start execution
• At the end of the CHRP Script is an instruction for Open Firmware to start executing
• Open Firmware loads the programs to the requested location and jumps to the start point to start executing. (Keep in mind that Open Firmware is still exists at this point)
• The first thing to run is the trampoline code.  For the most part this is a dumb program (not that it isn't important, but more that it really doesn't know very little about what it's doing).
  
  • The trampoline code starts up an Open Firmware connection through the Client Interface API's. 
  • The trampoline code asks Open Firmware for the device tree and iterates through it and copies all node, device and property info into a flat structure that OS 9 will be able to use.
  • While the trampoline code is looping through the device tree it compares each nodes type, name property and compatibility property to the data in the parcels.  If there is a match it look into the parcel for instruction on what to add to the device (properties, drivers, configuration instruction).  This is where drivers that are required to be in place for early boot are loaded.
    Some of the things loaded are actually the libraries and routines that are needed very early that are needed to be loaded before the nano kernel can load them as they are used very early on for setting up other things.
    
  • The trampoline code turns off open firmware (thus the reason for copying the device tree)
  • The trampoline code does some page setup of memory, moves some things around, setups up the Virtual Memory.
  • The trampoline code  then jumps to the nanokernel in the loaded toolbox. And it's job is done.
• And now we are to the nanokernel (the actual OS is now starting up.
  
  • Happy Mac Icon displayed
  • The nanokernel looks to the boot block of the start up device to find the location of the System folder, name of the first application to launch (The finder), and the name of the debugger if there is one.
  • Font Manager and System Error Handler initialized.
  • Set up Disk cache and CPU Specific software patches are executed.
  • The 68k Emulator is started up.  Not surprisingly a lot of the low level OS stuff runs off this.
  • Some  more required hardware is setup and initialized (remember the trampoline installed some drivers needing to be there for boot?  That's because the OS doesn't yet have a way to load them)
  • The resource manager is started up
  • The Time Manager is started and initialized along with the NVRAMServices.  This is required for drivers to be installed and hardware started up.
  • Script systems are loaded and initialized and initialization resources are executed from the System File
  • Data structures are setup for internal use
  • The debugger if specified is loaded. so that early startup function can be debugged.This is loaded at the top of RAM usually.  When the memory manager starts, if it finds there is a debugger it leave's it's memory alone and doesn't manage it so it runs completely outside of the control of the OS.
    An interrupt and trap is set so that the debugger can be jumped too.
    
  • The nanokernel starts loading system extensions (including control panels). 
    These load very early so that they can patch other parts of the system before they start up.
  • Some more managers are started (not really sure at this point exactly what goes on so from here to the process manager may be a bit off)
  • Memory Manager Is loaded and started up.
    This one really got me at first. I figured this would be started much earlier, but according to Apple documentation it does appear to start fairly late.  If this is true then it will be easier to patch the system to get access to the full 2GB of ram installed on some machines.
    
  • Additional services and managers are started
  • The memory manager allocats the stack and heap for the process manager.
  • The process manager starts.  The process managers works pretty close with the memory manager at this point for loading programs allocating memory and executing programs
    All programs actually run from with in the memory allocated by the memory manager to the process manager.
    Programs are loaded at the top of available memory of the process manager.  So programs load from top to bottom in memory.
    
• Depending on configuration certian parts of AppleTalk related to printing are loaded
• The Finder (Or what ever was specified in the boot block) is loaded and started and you are ready to use the machine

[DieHard]

I have read almost every classic Mac OS startup sequence and they pale in comparison to yours !

Excellent breakdown, cleared up a few things I had in the totally wrong sequence, thanks so much

[nanopico]

I have read almost every classic Mac OS startup sequence and they pale in comparison to yours !

Excellent breakdown, cleared up a few things I had in the totally wrong sequence, thanks so much

What  were the things you thought happened in a different order?

Also some of the things may be a little out of sequence so I'm still looking to make this completely accurate. 

The only thing I don't understand yet is
A). What order do all the managers actually start in, I know pretty good some groups of them but I need to figure out the exact order (not that it's super important, but my brain needs to know).
B). How it determines the order to load extensions and control panels.


A lot of the stuff I was reading only focused on a parts of it. 
The details of the trampoline code is where most of the experimentation comes in.
The rest of it was learned from Inside Macintosh and Tech Notes from Apple.
Sadly it's not all in one place so it was a matter of taking notes as I read and then piecing it together as I went.  Also having a very food understanding of general operating system design doesn't hurt. 

[MacOS Plus]

The only thing I don't understand yet is
A). What order do all the managers actually start in, I know pretty good some groups of them but I need to figure out the exact order (not that it's super important, but my brain needs to know).
B). How it determines the order to load extensions and control panels.

  From what I remember about extension loading sequence, first there is a preferential file type property that can be set to make an extension load ahead of the default file type, after which everything else loads in alphabetical order.  Users would often insert a space at the front of an extension's filename in order to force it to the front of the alphabetical queue.  I can't remember the alternate file type off the top of my head, but I have changed it on some of my extensions in the past when inserting a space in the filename wasn't sufficient to give it first priority (usually CPU upgrade extensions where the vendor hadn't changed it themselves already, and also Medianet).  I'm fairly certain the control panels load after the extensions, again in alphabetical order.

[nanopico]

The only thing I don't understand yet is
A). What order do all the managers actually start in, I know pretty good some groups of them but I need to figure out the exact order (not that it's super important, but my brain needs to know).
B). How it determines the order to load extensions and control panels.

  From what I remember about extension loading sequence, first there is a preferential file type property that can be set to make an extension load ahead of the default file type, after which everything else loads in alphabetical order.  Users would often insert a space at the front of an extension's filename in order to force it to the front of the alphabetical queue.  I can't remember the alternate file type off the top of my head, but I have changed it on some of my extensions in the past when inserting a space in the filename wasn't sufficient to give it first priority (usually CPU upgrade extensions where the vendor hadn't changed it themselves already, and also Medianet).  I'm fairly certain the control panels load after the extensions, again in alphabetical order.

Thank you. Greatly helpful.  I do recall the documents on writing extensions differentiated between two type.  I will no go an a treasure hunt for that info.

[DieHard]

Users would often insert a space at the front of an extension's filename in order to force it to the front of the alphabetical queue.  I can't remember the alternate file type off the top of my head

Yes, we would often prioritize when getting conflicts by loading extensions (with a space)... 1) CPU (if applicable), 2) SCSI Cards, 3) Network Cards, Video... then rest although I think the Video extensions were already "tagged" as priority... been 20 years since hunting down extensions manually so it's hard to remember... this was the Mac OS 8.1 days; OS 9.2 is very rarely plagued by such issues. Also, "Color" labeling the initial OS load set (As I mentioned in previous posts), helped track down new extensions that may be problematic.

[nanopico]

I updated the order of things to what I have found.  Still a work in progress so expect it to change more as I find more info.

[nanopico]

For anyone interested in what the managers are that existing in the Macintosh OS here's a list. If you read all of Inside Macintosh you will be able to understand them all. You literally have to read all of Inside Macintosh to understand it since it's not all documented in one section.
Going through that you start to get an idea of the order that these are initialized and start to understand the boot process.

Typically a manager is a library of function and data structures.  When a manager is loaded there is an initialization routine that runs to setup the structures.  Any time one of it's functions are called it updates the structure accordingly.  There are some exception where some of them respond to an interrupt or trap and then perform it's duties, but other than that, they just sit there doing nothing until an application calls a method.  They are a lot like the background services that run in modern OS.

Listed in no particular order. I'm still trying to figure out the exact order these start in and what starts between them.

• Resource Manager
• Scrap Manager
• Help Manager
• List Manager
• Component Manager
• Translation Manager
• Desktop Manager
• Mixed Mode Manager
• Code Fragment Manager
• Exception Manager
• Device Manager
• Slot Manager
• SCSI Manager
• ADB Manager
• Power Manager
• Font Manager
• Script Manager
• Text Services Manager
• Dictionary Manager
• Sound Manager
• Sound Input Manager
• Speech Manager
• Image Compression Manager
• Process Manager
• Time Manager
• Verticle Retrace Manager
• Notification Manager
• Deferred Task Manager
• Segment Manager
• Shutdown Manager
• Gestalt Manager
• Trap Manager
• Start Manager
• Package Manager
• Memory Manager
• Virtual Memory Manager
• Event Manager/Toolbox Event Manager
• Menu Manager
• Window Manager
• Control Manager
• Dialog Manager
• Edition Manager
• Data Access Manager
• Printing Manager
• File Manager
• Alias Manager
• Disk Initialization Manager
• Digital Signature Manager
• Interprogram Messaging Manager
• Catalog Manager
• Authentication Manager

[MacTron]

Thanks for this really great explanation. 

[nanopico]

I've found some more documentation that brings more detail to the boot process. 
It looks like the memory manager and process manager start earlier than expected.
Either I interpreted earlier documentation wrong or I interpreted this new stuff wrong.
I will update the process once I figure out what is correct. 
Oh the joy's of work in progress.

[nanopico]

Steps 1 -7 outlined above are still accurate based on current knowledge.  But the remainder I believe is different so here is the updated order of steps from 8 on.
This is kind of a high level view and may get more and more detailed as we learn more.

• And now we are to the nanokernel (the actual OS is now starting up.
  
  • Happy Mac Icon displayed
  • The interrupt system is initialized
  • I/O Drivers initialized. This is not all devices just some required to boot.  One of them being memory
  • Memory Manager started up.
  • The nanokernel looks to the boot block of the start up device to find the location of the System folder, name of the first application to launch (The finder), and the name of the debugger if there is one.
    A copy of the boot block is stored as a resource in the System Folder.  The resource type is boot and resource id 0 is the boot block.
    When an OS 9 system folder is blessed that resource is copied to the boot block on disk.
    Most of what follows is considered part of the start manager.
    
  • System file is loaded.
  • The data fork of the System file contains the next level of boot code.  There is some relation between this and the resource id 1 of type boot in the system file
    
  • The 68k Emulator is started up.  Not surprisingly a lot of the low level OS stuff runs off this.
  • The resource manager is started up
  • Machine specific and CPU specific patches are loaded from resources in the system file. 
    This may be where the CPU version is checked to determine if the CPU is supported. It is the most logical spot for that code.
  • Set up Disk cache
  • ROM patches are located from resources in the system file.  The ROMversion is checked and patched based on it's version.
    This is probably more relevant to old world ROM's as the boot process is different for them.
    
  • ROM resource overrides are loaded.  These are stored in the ROv# resources in the System file
  • The Time Manager is started and initialized along with the NVRAMServices.  This is required for drivers to be installed and hardware started up.
  • Script systems are loaded and initialized and initialization resources are executed from the System File.
  • Data structures are setup for internal use
  • The debugger if specified is loaded. so that early startup function can be debugged.This is loaded at the top of RAM usually.
    The debugger is loaded and the memory it is loaded to is setup by the memory manager but somehow hidden from the system.
    An interrupt and trap is set so that the debugger can be jumped too.
    The debugger is not controlled by the process manager so that it can interrupt anything.
    The error handling in the kernel probably checks an entry in it's jump table to determine if it should drop into the debugger.
    
  • The nanokernel starts loading system extensions (including control panels). 
    Extensions are loaded in order by the file name alphabetically.
    Extensions can be in the root of the System Folder, or the Extensions folder.  I believe it actually loads the ones in the root before the ones in the Extensions folder.
    There are three types of resources loaded as extensions. Each extensions can reference further resources, allocate memory in the system stack and patch code.
    
    • PACK - ROM extensions.  These are different from system extensions are used to extend specific ROM code.
    • INIT - Your normal system extensions and control panels.
    • DRVR - these are for hardware devices.  But they can also be virtual software drivers that appear to the system as hardware.
  • Most other managers are loaded.
  • Process manager is started. It is initialized and treated as standard application except that is loaded in a slightly different order due to it's relationship with the System heap.
    
    • Memory manager loads code resources for the process manager
    • The heap is loaded first and initialized. (Generally the stack is loaded and intialized first).
      Part of the initialization is to grow the heap to the size referenced in a resource.  This resource is either in a resource in the System file or embedded in one of the code segments in the code resources for the process manager.
      This is the point where the 1.5 GB limit is enforced. At this point it may be that we find the resource for the process manager size and update it and we have all 2 GB of RAM.
    • The A5 world and stack pointer are loaded. The A5 world contains a bunch of global stuff that the OS uses and the application uses.  See the Memory book of Inside Macintosh page 1-12 for a better explanation of A5
    • The jump table is loaded. If your not familiar with this, it's the table of function definitions for the application. Since applications may not be loaded at the same location in memory, this table is used to map the function to it's actual memory address.  Each application has it's own jump table.
    • The stack is loaded and placed just below the A5 sections.
    • The the process manager starts running.
  • The process managers works pretty close with the memory manager and is almost a sub system of the memory manager
    All programs actually run from with in the memory allocated by the memory manager to the process manager.
    Programs are loaded at the top of available memory of the process manager.  So programs load from top to bottom in memory.
    
• Depending on configuration certian parts of AppleTalk related to printing are loaded
• The Finder (Or what ever was specified in the boot block) is loaded and started and you are ready to use the machine

Additional digging is going to happen to get an even better picture. 

[nanopico]

Additional info on the boot process.  Eln and I have been really ripping things apart.  Here is some info for those that like to geek out on this stuff.

This is the boot process I'm 100% sure of.

1. Power on the machine and the built in firmware ROM starts up hardware as much as it needs to to build a device tree in open firmware (OF for now)

2. OF locates and loads the boot file from disk (The Mac OS ROM file).

3. On loading the image to RAM the boot script is executed to further setup the environment for the trampoline.

4. The script allocates memory for the trampoline, parcels and the info pages that the kernel will use. 

5. The trampoline contains three segments. A code sgement, data segment and notes segment (which is pretty useless at the moment).
The code segment is loaded to physical/virtual address 0x200000 and the data segment at 0x100000

6. The trampoline starts executing at memory address 0x20F078


Now the exact order and detail is not 100% clear for the next parts.

7. Trampoline initiates a client interface to OF

8. Trampoline obtains handles to the do-translate, do-map, do-unmap, claim-mem, release-mem,claim-virt and release-virt methods of the mmu and mmc devices
It will use these methods to setup the memory mapping for the hardware in the device tree and to set it all up for the kernel.

9. Trampoline get's the address the parcels package is stored at

10. Memory will be claimed for a flattened directory tree structure (or whatever it is the kernel and os want's for hardware access)

11. The OF device tree will be crawled and compared to the parcels

12. Parcels that are to be added as nodes will be loaded into ram and mapped to the flattened structure

13. Parcels that add properties and code to existing nodes will get loaded into ram and mapped to the flattened structure

14. Some devices (primarly ATA controllers and intterupt controllers) have some info about them in the trampoline code and not in a parcel that details how to start them up.  This will need to be updated for the ATA controller in the xserve and then a parcel with an OS 9 ndrv may need to be added to provide boot support from the internal ATA controller

15. The various info structures  will be created and populated for the kernel. (config info, cpu info...)

16. (this step I know is 100% true)  The trampoline disables open firmware

17.  The nano kernel starts up and setups up more memory stuff and loads various pieces of code. It then updates the memory map to clober a bunch of the early boot stuff and relocate structures and code to the final address without having to actually move them around.

18. Then from their it's loading libraries and code fragments and the managers and such and then off to the finder and the users does stupid shit and watches porn or something.

So that's the low level boot process. A couple more interesting tidbits for those who care.

The chrp boot script does a bunch of memory setup and mapping for the info pages, trampoline and parcels. 
These parts are allocated the following amounts of ram.
Trampoline    24   4k pages  (98304 bytes)
Parcels         610 4K pages  (2498560 bytes)
Info Pages    638 4k pages   (2613248 bytes)

[nanopico]

Here is a little bit more on the trampoline code.
In case anyone wants to know.
The trampoline consists of at least 4568 lines of assembly code.  This is code that is verified as referenced and used.
There is another 11973 lines of code that I have not determined if is data or code.
At this time it appears that there are 118 methods in the code.  This has not been verified yet. Some of these methods would be debugging methods.

Now it's time to analyze every line of code by hand.

[evilmoo]

From an initial glance, it looks like an awful lot of the Trampoline code is debugging logs and messages.

[nanopico]

From an initial glance, it looks like an awful lot of the Trampoline code is debugging logs and messages.

You got it.

And a lot of loops

[powermax]

I'd like to offer a decompiled C source code of the OS9 trampoline I've made several years ago. I'm not sure which OS version it belongs to but we could figure it out.

It won't compile because several standard library functions this code depends on are missing. But it could be easily made compilable (we probably need to figure out how to compile PowerPC ELF binaries). The problem is that the original code bundles a custom lightweight version of the stdc library like bootX does. We need either to find a replacement for them or to write our own implementation or to find a way to avoid them.

Anyway, my code is heavily commented to help to understand what's going on. By reading this code, I was able to spot several bugs in the original code.

I could upload the code either in a private repository of mine and gain you an access to it or directly to your Gitlab repository. Just tell me what you prefer.

[Protools5LEGuy]

Welcome and thanks! Now the Dream Team is complete!

[ELN]

Christmas in August, huh? Great job! How about you post an archive on this site, and we'll see what we can do about getting it to build. Once that's done, I would absolutely love to have it in the repo.

The build system (new as of December) is MPW-based. I have concocted a slightly dodgy set of PPCAsm files that produce the Trampoline as an ELF *within* an XCOFF container. That XCOFF then gets slurped up by one of my gigantic array of postlinking tools. MPW is almost as flexible as a full Unix environment, so we'll be able to figure something out.

Can't wait to have a read!