Mac OS 9 Lives

Classic Mac OS Software => Hacking the System, Mac OS 9.3, and Beyond ! => Topic started by: Daniel on May 14, 2017, 11:34:37 AM

Title: Mac OS Rom Hacking
Post by: Daniel on May 14, 2017, 11:34:37 AM
Hello everyone,

I created a custom Mac OS ROM file and thought it might be useful. It does not have the Trampoline or Toolbox data in it. Instead, it loads them from a folder called 'Bootfiles' in the System folder it resides in. The Trampoline and Toolbox are in separate files and can be modified independently. It also loads and runs a Forth boot script from that folder before running the Trampoline.

To use it, just copy the Mac OS ROM file and the Bootfiles folder into your System Folder. It is probably best to save the old ROM file somewhere else and have a second System Folder that can be booted from to restore the first one. I have only tested this on one system. Use it at your own risk.

The Trampoline and Toolbox file are taken from a recent release of CDG5 (it even supports NKDebug if you have it). It probably only works on 9.2.2, but you can swap out the Trampoline and Toolbox for ones that have more compatibility (assuming that you can figure out how to create the separate Trampoline and Toolbox in the first place. It can be trivial or impossibly difficult depending on what tools you have available).

I apologize for the messiness of the custom ROM. I started out with a regular Mac OS ROM file and iteratively modified it until it did what I wanted it to.

I hope that the custom ROM is useful. I would appreciate any feedback on it.
Thank You,
Daniel

Edit:

The version in this message should probably not be downloaded. It is not compatible with every NewWorld Mac. The latest version is attached to another message in this topic. Please download that instead.
Title: Re: Mac OS Rom Hacking
Post by: ELN on May 14, 2017, 07:22:23 PM
Legend. Testing shortly.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 14, 2017, 07:35:34 PM
If you try to use the helper functions I made ( $$+ and load-from-path ) for other projects, be careful. load-from-path seems to only work if you use it in chrp files. This is presumably because init-program copies the data in <boot-script> over to another memory region that is not deallocated by release-load-area. If you use it with a forth or fcode script, calling it swaps the loaded data out from under the running program (this seems to not crash it, but makes it return to the program that called the script). ELF and XCOFF files are presumably safe to use load-from-path as well, but I lack the means to test it (also, ELF and XCOFF files are in native powerpc, so it might not make sense to call load-from-path in there).

Also, be careful editing the Mac OS ROM file. Your text editing program might change the file and creator codes after you save it. I am not sure if Simpletext or BBEdit do this, but I know for a fact that Textedit on 10.3.9 does.
Title: Re: Mac OS Rom Hacking
Post by: ELN on May 14, 2017, 08:15:53 PM
Neither QEMU nor my MDD seem to like the image very much. Any ideas? While I notice that the script mixes some LFs in with the usual CRs, Open Firmware usually doesn't mind that. Is the resource fork meant to be missing? The OS usually crashes pretty early if you do that.

But this is very cool, potentially handy for debugging, and easy to integrate into our build system. Which, by the way, does some funky Forth preprocessing that I'd really like your help with. Keen?
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 14, 2017, 08:26:33 PM
I had no idea that the resource fork was missing. The LFs are probably because I used Textedit on OSX to create the script. For testing the scripts, go into Open firmware and type in " boot [the device alias or path to the device]:[the partition number or ','][the path to the Mac OS ROM file or '\\:tbxi']" . This will show the debugging info for it (assuming it even gets that far). Please send me the entire log or the last line of text in the log. I will look over tomorrow. It is 11:25 PM for me at the moment. I would probably like to help.
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 15, 2017, 11:21:43 AM
If you try to use the helper functions I made ( $$+ and load-from-path ) for other projects, be careful. load-from-path seems to only work if you use it in chrp files. This is presumably because init-program copies the data in <boot-script> over to another memory region that is not deallocated by release-load-area. If you use it with a forth or fcode script, calling it swaps the loaded data out from under the running program (this seems to not crash it, but makes it return to the program that called the script). ELF and XCOFF files are presumably safe to use load-from-path as well, but I lack the means to test it (also, ELF and XCOFF files are in native powerpc, so it might not make sense to call load-from-path in there).
Pardon me if this is already known and stated and I am just confusing myself.

In the boot script, the stuff loading and claiming memory are for loading the elf file into and then loads it.  The init-program tells OF to actually reads the elf file and puts everything in the proper place.  The trampoline is a non-relocatable file so it will always be loaded at the same spot OF. Thus all the memory claiming and loading stuff.

The OF boot script and trampoline are very important to OS 9.  OS X pretty much doesn't care.  The boot script loads the xcoff file and it's done. No setup.  Straight to the kernel it is.  BootX is pretty much a dummy (I'm exaggerating just a small bit) program that loads the kernel and jumps to it's entry point. The  XNU kernel handles all initialization.  So the drivers for the OS are used during boot. The two are completely different, but it does demonstrate the dependencies of OF in OS 9.

Also just as another note.  Open Firmware can load four types of files for boot. 


Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 15, 2017, 01:44:09 PM
I was not sure if it was already known. It took me a bit of time to figure out that using load from a forth file is unsafe. It makes more sense to put this stuff out in the open to prevent unnecessary roadblocks for anyone trying to use this.

Also, I am rather new here so I do not know about everything that has been discussed.

From what I know, you can load a forth script and run it. The first character in the script has to be a '\' for it to be recognized. Apparantly you can also load a fcode script, but I do not know fcode.
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 15, 2017, 02:45:08 PM
It's all good. Input is always welcome. It can definitely give a different view point to solve problems.


I was not sure if it was already known. It took me a bit of time to figure out that using load from a forth file is unsafe. It makes more sense to put this stuff out in the open to prevent unnecessary roadblocks for anyone trying to use this.

Actually using load from a forth file/script is not unsafe.  This is actually a way to debug scripts.  There just certain things you need to be careful about. But that is true in any programming language (I personally love infinite loops that bypass certain OS memory control check that allow a program to consume all available RAM in under a second).

Also, I am rather new here so I do not know about everything that has been discussed.

That can work to every one's advantage sometimes since you may describe something already discussed in a different light which may provide an answer.

From what I know, you can load a forth script and run it. The first character in the script has to be a '\' for it to be recognized. Apparantly you can also load a fcode script, but I do not know fcode.

I didn't know anything about it starting with a '\' I have to go play now.  I know that \ is the root selector of the device tree and so not much can be done with the system until you select that root node.
Thinking about it now. I wonder what would happen if you loaded the Mac OS ROM, deleted the root node (and thus everything below it) and then hit go and see what happens.  I'm not about to try that one though.  I've created enough bricks for now doing these kind of things.

Oh yeah. Welcome!
Title: Re: Mac OS Rom Hacking
Post by: DieHard on May 15, 2017, 02:47:26 PM
Mac OS 9.3.1 release date.... soon  ;)
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 15, 2017, 04:52:02 PM
I believe that I have finally got it working  ;D
The problem was that I was using an undocumented method called " (load) " to load the Toolbox file (the parenthesis are part of the name). It seems to be only implemented on G3s, which is what I was doing the initial tests on. The problem with the regular function loading function " load " is that after loading a file it checks to see if it is a valid program file and unloads the data if it complains. The Toolbox is not a chrp, elf, xcoff, or forth program. To make it so I could use " load ", I had to add a backslash and a space ("\ ") to the front of the Toolbox (right before the "prcl"). This tricks Open Firmware into thinking that the Toolbox is a forth script to avoid "unrecognized Client Program format" errors. Any replacement Toolbox files will also need a "\ " prepended to them, so keep this in mind. The Trampoline does not need any modifications because it is already an elf file.

Now the ROM file works on several different systems. I have also tested unstuffing the file to make sure that Stuffit is not corrupting it somehow. Note that the Mac OS ROM file has no resource fork. At first this was an accident but I found that including one crashes the system during startup 100% of the time. It seems to be fine without one. It has worked on a G4 and several G3s.

To make it really easy to tell if it works, the secondary boot script now paints colored stripes across the screen a few seconds before the happy mac appears. This version of it works pretty much the same as the previous version, except it actually works on non G3s. You just stick Mac OS ROM and Bootfiles in your System Folder and away you go.

As to forth boot scripts, a lot of useful info can be found here:
https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2001.html (https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2001.html)
https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2004.html (https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2004.html)
The first link shows how to create and run forth scripts. The second one shows how to get an Open Firmware prompt over telnet. Telnet has been very useful for my work. I can control Open Firmware from my modern laptop and, because of Wifi, I don't even have to be in the same room as the machine I'm debugging as long as it has a wired connection to the network.

Have Fun,
Daniel
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 15, 2017, 06:02:01 PM
I believe that I have finally got it working  ;D
The problem was that I was using an undocumented method called " (load) " to load the Toolbox file (the parenthesis are part of the name). It seems to be only implemented on G3s, which is what I was doing the initial tests on. The problem with the regular function loading function " load " is that after loading a file it checks to see if it is a valid program file and unloads the data if it complains. The Toolbox is not a chrp, elf, xcoff, or forth program. To make it so I could use " load ", I had to add a backslash and a space ("\ ") to the front of the Toolbox (right before the "prcl"). This tricks Open Firmware into thinking that the Toolbox is a forth script so to avoid "unrecognized Client Program format" errors. Any replacement Toolbox files will also need a "\ " prepended to them, so keep this in mind. The Trampoline does not need any modifications because it is already an elf file.

Now the ROM file works on several different systems. I have also tested unstuffing the file to make sure that Stuffit is not corrupting it somehow. Note that the Mac OS ROM file has no resource fork. At first this was an accident but I found that including one crashes the system during startup 100% of the time. It seems to be fine without one. It has worked on a G4 and several G3s.

To make it really easy to tell if it works, the secondary boot script now paints colored stripes across the screen a few seconds before the happy mac appears. This version of it works pretty much the same as the previous version, except it actually works on non G3s. You just stick Mac OS ROM and Bootfiles in your System Folder and away you go.

As to forth boot scripts, a lot of useful info can be found here:
https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2001.html (https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2001.html)
https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2004.html (https://www.fenestrated.net/mirrors/Apple%20Technotes%20(As%20of%202002)/tn/tn2004.html)
The first link shows how to create and run forth scripts. The second one shows how to get an Open Firmware prompt over telnet. Telnet has been very useful for my work. I can control Open Firmware from my modern laptop and, because of Wifi, I don't even have to be in the same room as the machine I'm debugging as long as it has a wired connection to the network.

Have Fun,
Daniel

load problem solved. 
Besides telnet you can also output to the serial port or get input on it or both! Pretty much useless though on any new world stuff.  Have to go back to when there was a physical port. Or just use the modem port like I think the stealth port does to get a serial connection on the G4/G3 stuff.
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 15, 2017, 06:03:04 PM
Mac OS 9.3.1 release date.... soon  ;)

Oh sweet! I didn't know about 9.3.1!  Can't wait to see that one.
Title: Re: Mac OS Rom Hacking
Post by: DieHard on May 15, 2017, 06:08:18 PM
Mac OS 9.3.1 release date.... soon  ;)

Oh sweet! I didn't know about 9.3.1!  Can't wait to see that one.

I was counting on you for that release :(
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 15, 2017, 06:44:31 PM
Mac OS 9.3.1 release date.... soon  ;)

Oh sweet! I didn't know about 9.3.1!  Can't wait to see that one.

I was counting on you for that release :(

Oh.  Uhm...  well you see... 
Let's get 9.3.0 First
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 15, 2017, 07:09:51 PM
Hi again,
I made a slight modification to the custom rom. The previous version was not nearly dangerous enough. This iteration intercepts "quiese", the function that is called to shut down Open Firmware. This could go wrong in so many ways. Specifically, the quiesce function is intercepted so that it will load and run a forth script called "Quiesce-Intercepter" from the Bootfiles folder before actually shutting down Open Firmware. Quiesce is usally called near the end of a bootloader such as the Trampoline or BootX. By intercepting it, we will be able to see the internal structures of the bootloader at the time it tries to shut down Open Firmware.

The "Quiesce-Intercepter" has to be smaller than 4KB in this version (though you could change the limit to an arbitrary value by altering two numbers in the Mac OS ROM file). It is also not enabled unless the secondary boot script "Script" sets the value "Quiesce-Interceptor?" to true.

I have tested this on a G3 and a G4, though no guarantees of it working on anybody else's systems.

Also, can somebody add a "Mad Scientist" smiley face to the list of available emojis? I want to be able to use it in my posts...
Title: Re: Mac OS Rom Hacking
Post by: Protools5LEGuy on May 15, 2017, 07:18:11 PM


Also, can somebody add a "Mad Scientist" smiley face to the list of available emojis? I want to be able to use it in my posts...

(http://foohack.com/blog/wp-content/uploads/2008/01/victor-frankenstein.jpg)

Mac OS 9.3.1 release date.... soon  ;)

Oh sweet! I didn't know about 9.3.1!  Can't wait to see that one.

I was counting on you for that release :(

Oh.  Uhm...  well you see... 
Let's get 9.3.0 First

Shit. I was looking for 9.2.4 because 9.2.3 is already taken...  ;D
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 16, 2017, 05:24:20 AM
I propose a compromise in the naming schemes. How about we release 9.2.4 for compatablility with more devices and basic stability improvements and the like. We can save 9.3 for major modifications to the core system software(NanoKernal, System Folder, Finder, Etc.). It will probably take a while to get protected memory and multiprocessing working, so these should probably be saved for 9.3. G5 compatablilty and  removal of the 1.5 Gig limit could go in either. I am not sure exactly how much work those would take.
Title: Re: Mac OS Rom Hacking
Post by: MacTron on May 16, 2017, 07:06:55 AM
...
It will probably take a while to get protected memory and multiprocessing working, so these should probably be saved for 9.3. G5 compatablilty and  removal of the 1.5 Gig limit could go in either. I am not sure exactly how much work those would take.

Wow!, This sounds really promising, thank you guys :)
and keep the hard working !
Title: Re: Mac OS Rom Hacking
Post by: ELN on May 17, 2017, 02:17:21 AM
Looking good! I'm gearing up to give this another try!

A quick note on the resource fork thing. The resource fork of a Mac OS ROM file acts as a System Enabler. Several of its resources are there to patch over System file resources. That way, there was pretty much no need to update the 9.2.2 System as new Macs were released circa 2002.

The addition of a resource fork to this image crashes the system because of the presence of a cfrg resource, which contains the byte offsets of two PEF containers (NQDResidentCursor and ProcessMgrLib) in the data fork. (You might have these mixed in with the Parcels. Let me know if you want a clean Toolbox parcels file.) While PEF containers could live in resources, putting them in the data fork allowed them to be memory-mapped. So much for the Grand Unified Model, alas. The resource fork is necessary for iMic's extra hardware support to work, so consider leaving it in, minus the cfrg resource.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 17, 2017, 04:47:36 AM
I knew that some resources had to be referencing the parcels, I just had no idea which ones. Sending me a clean Parcles file would be great. Can you also send me the data fork files? Would I have to modify the cfrg to point to new offsets?

Also, what interesting features would you like to have added to the bootroom? I have already figured something out that lets you load the system folder from somewhere other than where the Mac OS ROM file is (non-interactively. You have to specify the path to the folder in a text file). I haven't posted it here quite yet. Any suggestions? I have no idea how useful my crazy improvements would actually be.  :)
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 17, 2017, 05:50:15 AM
I knew that some resources had to be referencing the parcels, I just had no idea which ones. Sending me a clean Parcles file would be great. Can you also send me the data fork files? Would I have to modify the cfrg to point to new offsets?

Also, what interesting features would you like to have added to the bootroom? I have already figured something out that lets you load the system folder from somewhere other than where the Mac OS ROM file is (non-interactively. You have to specify the path to the folder in a text file). I haven't posted it here quite yet. Any suggestions? I have no idea how useful my crazy improvements would actually be.  :)

The two PEF executables discussed are not parcels.  The are seperate for the parcels package.  They are actual code fragments.  If they are moved from their current offset, then yes the cfrg resource have to be updated.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 17, 2017, 06:10:17 AM
That makes sense. I am somewhat unfamiliar with the terminology.

Having to statically specify offsets into the Mac OS ROM file would be really inconvenient. Currently, I can just use BBEdit to add stuff to the CHRP script whenever I want to. Any offsets would have to be updated every time I modify the file. I technically can get around that, but it would be really hackish. I would have to use that system folder redirect I mentioned to load the system folder from a partition that has a valid Mac OS ROM file already in it. The chrp file would be run from the first Mac OS ROM file, but the System would load the fragments from another Mac OS ROM file.
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 17, 2017, 06:26:21 AM
That makes sense. I am somewhat unfamiliar with the terminology.

Having to statically specify offsets into the Mac OS ROM file would be really inconvenient. Currently, I can just use BBEdit to add stuff to the CHRP script whenever I want to. Any offsets would have to be updated every time I modify the file. I technically can get around that, but it would be really hackish. I would have to use that system folder redirect I mentioned to load the system folder from a partition that has a valid Mac OS ROM file already in it. The chrp file would be run from the first Mac OS ROM file, but the System would load the fragments from another Mac OS ROM file.

Yeah it's nice that everything is not static.  Move the parcels package, no problem, move the two code fragments, no problem.  Just references have to be updated. Anything less static would make it way slower, anything more static would make it a pain to maintain.

I still need to check out the scripts and stuff you have. Hopefully I can get that today.  Would have had time yesterday, but wedding anniversary was a bit more fun touring Paisley Park and such.
Title: Re: Mac OS Rom Hacking
Post by: ELN on May 18, 2017, 08:51:53 PM
Our build system does sets all the offsets for you when it rebuilds the parcels, trampoline, PEFs and forks.

Code: [Select]
#
# File: WrapMacOSROM
#
# Contains: This script wraps a Classic-style "MacOSROM" parcels blob
# in a CHRP booter along with a Forth script and a
# "Trampoline" ELF bootloader. It depends on ZeroPad, a file
# containing an ASCII EOF plus 0x4fff NULs, in the same dir.
# It also depends on :Tools:SixDigHex, which takes one arg.
# It ALSO depends on :Tools:SubstituteEquates, which deals
# in {Keys}.
#
# Change History (most recent first):
#
# 01/07/17 HQX Wrote script.


Set DirName "StreamEdit -e '¥,° Replace /[Â:]+°/ ¶"¶"'"

Set ScriptFile "{1}"
Set IconFile "{2}"
Set BadgeFile "{3}"
Set ElfFile "{4}"
Set ParcelsFile "{5}"
Set OutputFile "{6}"
Set ZeroPad "`Echo "{0}" | {DirName}`ZeroPad"

Set o1 "{TempFolder}cdg5BaseScript"
Set o2 "{TempFolder}cdg5PaddedScript"
Set o3 "{TempFolder}cdg5ShortenedScript"

Set WrapperSize `Count -c "{ZeroPad}"`
Set ElfSize `Count -c "{ElfFile}"`
Set ParcelsSize `Count -c "{ParcelsFile}"`
Set InfoSize `Evaluate {ElfSize} + {ParcelsSize}`

Set ElfOffset {WrapperSize}
Set ParcelsOffset `Evaluate {WrapperSize} + {ElfSize}`

Echo "<CHRP-BOOT>" > "{o1}"
Echo "<COMPATIBLE>" >> "{o1}"
Echo "MacRISC MacRISC3 MacRISC4" >> "{o1}"
Echo "</COMPATIBLE>" >> "{o1}"

Echo "<DESCRIPTION>" >> "{o1}"
Echo "MacROM for NewWorld." >> "{o1}"
Echo "</DESCRIPTION>" >> "{o1}"

Echo "<ICON" SIZE=64,64 COLOR-SPACE=3,3,2 ">" >> "{o1}"
Echo "<BITMAP>" >> "{o1}"
Catenate "{IconFile}" >> "{o1}"
Echo "</BITMAP>" >> "{o1}"
Echo "</ICON>" >> "{o1}"

Echo "<BOOT-SCRIPT>" >> "{o1}"
SubstituteEquates "{ScriptFile}" ¶
{FeatureSet} ¶
-d ElfOffset=`SixDigHex {ElfOffset}`
-d ElfSize=`SixDigHex {ElfSize}`
-d ParcelsOffset=`SixDigHex {ParcelsOffset}`
-d ParcelsSize=`SixDigHex {ParcelsSize}`
-d InfoSize=`SixDigHex {InfoSize}`
| StreamEdit -e "¥,° Replace -c ° />/ '&gt;'" -e "¥,° Replace -c ° /</ '&lt;'" ¶
>> "{o1}"
Echo "</BOOT-SCRIPT>" >> "{o1}"

Echo "<OS-BADGE-ICONS>" >> "{o1}"
Catenate "{BadgeFile}" >> "{o1}"
Echo "</OS-BADGE-ICONS>" >> "{o1}"
Echo "</CHRP-BOOT>" >> "{o1}"

Catenate                  "{o1}" "{ZeroPad}" > "{o2}"
FileDiv -b -n {WrapperSize}                     "{o2}" "{o3}"
Catenate                                              "{o3}01" "{ElfFile}" "{ParcelsFile}" > "{OutputFile}"

SetFile -t tbxi -c chrp "{OutputFile}"
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 19, 2017, 03:49:19 AM
I have not set up the cdg5 build system. Judging from your posts, it requires Docker, Make, QEMU, Python 3, and the Toolbox Toolbox you made. Everything has to be in the right folder and you have to cd to the folder containing the files you want to build. Is that correct?

Looking at the build script, it would be pretty easy to modify. Is the zeropad so you can avoid having to change the prcl resources?
Title: Re: Mac OS Rom Hacking
Post by: ELN on May 19, 2017, 05:46:13 AM
Nosir! Just MPW. Using Classic on Tiger lets you use git, which helps, but is not essential. Ignore the GitHub repos! It is now hosted privately in GitLab.

Old versions did use Docker (and used Docker to install Python and all the other stuff). Nanopico shamed me into porting it to MPW, so that was my summer project.

I'll be super happy to get you started. Just let me get tomorrow's exam out of the way.
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 19, 2017, 06:13:01 AM
Nanopico shamed me into porting it to MPW, so that was my summer project.
I'm not so sure I shamed you as much as I was a stubborn asshole who was working on OS 9 and wanted to us OS 9 to do the work.  Though I think it is better to be using MPW and ELN using it in OS X does make a nice blend of options, but the stubborn sadistic side of me uses only OS 9 and MPW. That's it. Well for me anyway.
There were many advantages to using MPW (not that it was easy to get everything moved there, thank ELN).
Native Mac tools that understand the mac system and formats with out questionable third party tools that sometimes don't work. I do have to say I miss having an editor with intelisense, but I'll live.  It was done before and it can be done now.

Is the zeropad so you can avoid having to change the prcl resources?

The parcels can be changed and modified if need be. The zero padding is just to ensure it starts on a location that is a multiple of I can't remember. Or maybe the Apple dev's had more debug stuff in there that got removed and they just felt like leaving the space empty? The script will adjust the PEF offsets as need be like ELN stated.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 19, 2017, 01:39:41 PM
I have a ton of questions about the parcels and the Trampoline.

What is the format of the Parcels from the Trampoline's perspective?

Does the order the Parcels are in matter?

How does the Trampoline respond to duplicate parcels? I have successfully managed to copy the parcels file into memory twice, so that the Trampoline would get a duplicate of every parcel. It booted fine. That simply tells me that it can handle duplicate parcels; it does not tell me how it handles them. Say, for example, we had a regular toolbox file with a replacement nanokernal tacked on to the end of it in memory(maybe it came from another file...). Which nanokernal would take precedence? I want to be able to have a primary parcels file in the Mac OS ROM and be able to override parcels in it by sticking replacements in another file(I am pretty sure that I can do that. I just can't test it at the moment).

Can one of you create a replacement nanokernal that puts an easily noticeable line in the NKLog or something? Please send it to me correctly formated so that it can just be appended to the Parcels and it will work.

As for the build system working in MPW, awesome!!!  :) :) :)
Title: Re: Mac OS Rom Hacking
Post by: ELN on May 19, 2017, 05:56:21 PM
Quick primer on the parcels.

Introduced with Mac OS ROM 2.x or 3.x, their only job is to make binaries available to the Trampoline bootloader at runtime. Most of the parcels are drivers, which the Trampoline actually squishes into the Open Firmware device tree, later to be discovered by the MacOS environment after the 68k boot code takes control. By loading only the parcels that match hardware, memory is saved. But the most important parcel is the "Mac OS ROM Parcel", which is just a compressed 4 MB ROM image, the same as any other Power Mac. While it has had its initialisation code ripped out, the ROM parcel still contains the NanoKernel and emulator, as well as a full 3 MB 68k ROM with the Toolbox, OS and many drivers. It is always loaded (with some free space scooped out of the middle, again to save memory).

By the way, you'll also find a raw parcels file somewhere in Classic Startup.app.

Because the 'prcl' blob is a reasonably complex structure, I have made a build system for it inspired by other Apple code. It is my second attempt, coming after all that toolboxtoolbox stuff. It works pretty darn well, too. Here is the Rez code that tells the ParcelLayout program which parcels to put into the structure. The structure is essentially a linked list of groups of binary blobs with metadata sprinkled in. The order is described below: mostly alphabetical, but with some special parcels coming first. The {string, string} pairs are {name, compatible} strings for the benefit of the Trampoline.

Make a GitLab account (this goes for anyone) and PM me the name. I'll add you to the repo as a developer, and show you around it. Among other useful tasks, it automates the process of doing a full Mac OS ROM rebuild every time you change a little thing in (say) the NanoKernel.

I'd be very surprised if the Trampoline was actually reading from two 'prcl' blobs consecutively. They need a bit more work to concatenate than that.

Code: [Select]
/*
File: Parcels.r

Contains: Parcel descriptions which are compiled and used by ParcelLayout

Change History (most recent first):

  1/9/16 HQX Updated with info from earlier Mac OS ROM versions
  1/8/16 HQX Painstakingly adapted from the output of a Python script
*/


#include "RomTypes.r"


/* The CodeRegister, CodePrepare, ROM and Checksum parcels are special, and go first */



resource 'prcl' (100, "CodePrepare") {
kPropertiesForSpecialNode,
{"CodePrepare Node Parcel"},
0x00020000,
{
"name",
0x00000000,
useCstring {"AAPL,CodePrepare"};

"AAPL,prepare_order",
0x00000000,
useCstringArray {{"Placeholder"}};

#if BlueBox
kUseResName,
0x00020094,
fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "InterruptTreeBlueBoxLib"};
#else
/* MotherBoardHAL was present in 6.1, absent in 7.5.1 */
//kUseResName,
//0x00020094,
//fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "MotherBoardHAL"};

kUseResName,
0x00020094,
fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "TimeManagerLib"};

kUseResName,
0x00020094,
fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "NVRAMServicesLib"};

kUseResName,
0x00020094,
fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "RTCServicesLib"};
#endif
};
};


resource 'prcl' (110, "CodeRegister") {
kPropertiesForSpecialNode,
{"CodeRegister Node Parcel"},
0x00010000,
{
"name",
0x00000000,
useCstring {"AAPL,CodeRegister"};

kUseResName,
0x00010094,
fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "NativePowerMgrLib"};

#if !BlueBox
/* AGPLib was absent in 7.5.1, present in 8.4 */
kUseResName,
0x00010094,
fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "AGPLib"};

/* EtherPrintfLib AND StartLib were absent in 3.8 and earlier, present in 6.1 */
kUseResName,
0x00010194,
fromResource {RsrcDir, "MiscLibs.rsrc", 'nlib', "EtherPrintfLib"};
#endif

kUseResName,
0x00010094,
fromResource {RsrcDir, "MiscLibs.rsrc", 'shlb', "StartLib"};
};
};


/* Naturally the most important parcel */

resource 'prcl' (200, "The Really Important Parcel") {
'rom ',
{"Mac OS ROM Parcel"},
0x00000000,
{
"",
0x00000005,
fromDataFork {ImageDir, "RiscMondo"};
};
};


/* I wonder what these calculated checksums are used for... */

resource 'prcl' (300, "Property Checksum") {
'psum',
{"Property Checksum"},
0x00000000,
{
"",
0x00000000,
useCstringArray {{
"name",
"model",
"compatible",
"device_type",
"reg",
"assigned-addresses",
"slot-names",
"vendor-id",
"device-id",
"class-code",
"devsel-speed",
"fast-back-to-back",
"bootpath",
"mac-address",
}};

"",
0x00000000,
useCstringArray {{
"boot-rom",
"macos",
}};

"",
0x00000000,
useCstringArray {{
"Placeholder",
}};

"",
0x00000000,
useCstringArray {{
"Placeholder",
}};

"",
0x00000000,
useCstringArray {{
"usb",
"ieee1394",
}};
};
};


/* Now the 'prop' parcels, which are used to modify actual devices in the tree */
/* (in alphabetical order) */

/* BlueBox uses: a "bal" (BlueAbstractionLayer) parcel
a MacOSROMFile-version parcel
all parcels containing a PowerMgrPlugin
*/


#if !BlueBox
resource 'prcl' (1000, "Network: apple21143") {
kPropertiesForExistingNode,
{"apple21143", "network"},
0x0000000c,
{
"lanLib,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "Network.rsrc", 'shlb', "apple21143"};
};
};
#endif


/* backlight driver was absent in 3.8, present in 6.1 */
#if !BlueBox
resource 'prcl' (1010, "Backlight") {
kPropertiesForExistingNode,
{"backlight", "backlight"},
0x00000009,
{
"driver,AAPL,MacOS,PowerPC",
0x00000000,

#if CusterBacklightParcel
fromResource {RsrcDir, "Display.rsrc", 'ndrv', "CusterBacklightDriver"};
#else
fromResource {RsrcDir, "Display.rsrc", 'ndrv', "BacklightDriver"};
#endif

"code,AAPL,MacOS,name",
0x00000002,
useCstring {"BacklightDriver"};
};
};
#endif


#if BlueBox
resource 'prcl' (1015, "BAL PowerMgrPlugin") {
kPropertiesForExistingNode,
{"bal", "power-mgmt"},
0x00000009,
{
"pef,AAPL,MacOS,PowerPC,register",
0x00000000,
fromResource {RsrcDir, "PowerManagement.rsrc", 'ndrv', "bal"};

"code,AAPL,MacOS,name",
0x00000002,
useCstring {"PowerMgrPlugin"};
};
};
#endif


/* bmac+ driver was absent in 7.5.1, present in 8.4 */
#if !BlueBox
resource 'prcl' (1020, "Network: bmac+") {
kPropertiesForExistingNode,
{"bmac+", "network"},
0x0000000c,
{
"lanLib,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "Network.rsrc", 'shlb', "bmac+"};
};
};
#endif


/* PCCard library was in "cardbus"-named parcel in 3.8,
/* moved down to "cardbus"-compat parcel in 6.1 */

#if !BlueBox
//resource 'prcl' (1025, "Old CardBus Parcel") {
// kPropertiesForExistingNode,
// {"cardbus", "cardbus"},
// 0x00000009,
// {
// kUseResName,
// 0x00010094,
// fromResource {RsrcDir, "PCCard.rsrc", 'nlib', "PCCard"};
// };
//};
#endif


/* chrp-named parcels were present in 3.8, absent in 6.1 */

#if !BlueBox
//resource 'prcl' (1026, "CHRP parallel?") {
// kPropertiesForExistingNode,
// {"chrp,ecp", "parallel"},
// 0x0000000c,
// {
// "driver,AAPL,MacOS,PowerPC",
// 0x00000006,
// fromResource {RsrcDir, "LegacyDrivers.rsrc", 'ndrv', "parallel"};
// };
//};
#endif


#if !BlueBox
//resource 'prcl' (1027, "chrp,ide/ide") {
// kPropertiesForExistingNode,
// 0x0000000c,
// {
// "driver,AAPL,MacOS,PowerPC",
// 0x00000006,
// fromResource {RsrcDir, "LegacyDrivers.rsrc", 'ndrv', "ide"};
// };
//};
#endif


#if !BlueBox
resource 'prcl' (1030) {
kPropertiesForExistingNode,
{"cmd646-ata", "ata"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "ATA.rsrc", 'ndrv', "cmd646-ata"};
};
};
#endif


/* cofb driver was absent in 3.8, present in 6.1 */
#if !BlueBox
resource 'prcl' (1040, "Framebuffer... generic?") {
kPropertiesForExistingNode,
{"cofb", "display"},
0x00000008,
{
"driver,AAPL,MacOS,PowerPC",
0x00000024,
fromResource {RsrcDir, "Display.rsrc", 'ndrv', "cofb"};
};
};
#endif


/* Present in both BlueBox and bare metal */
resource 'prcl' (1050, "CUDA PowerMgrPlugin") {
kPropertiesForExistingNode,
{"cuda", "via-cuda"},
0x0000000c,
{
"pef,AAPL,MacOS,PowerPC,register",
0x00000000,
fromResource {RsrcDir, "PowerManagement.rsrc", 'ndrv', "cuda"};

"code,AAPL,MacOS,name",
0x00000002,
useCstring {"PowerMgrPlugin"};
};
};


#if !BlueBox
resource 'prcl' (1060, "Network: bmac+") {
kPropertiesForExistingNode,
{"gmac", "network"},
0x0000000c,
{
"lanLib,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "Network.rsrc", 'shlb', "gmac"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1070, "Grackle... the G3 PCI controller?") {
kPropertiesForExistingNode,
{"grackle","pci"},
0x0000000c,
{
"pef,AAPL,MacOS,PowerPC,prepare",
0x00000016,
fromResource {RsrcDir, "PCI.rsrc", 'nlib', "Grackle_PCICyclesLib"};

"code,AAPL,MacOS,name",
0x00000000,
useCstring {"Grackle_PCICyclesLib"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1080, "ATA: Heathrow") {
kPropertiesForExistingNode,
{"heathrow-ata", "ide"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "ATA.rsrc", 'ndrv', "heathrow-ata"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1090, "ATA: Heathrow again!") {
kPropertiesForExistingNode,
{"heathrow-ata", "ata"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000002,
fromResource {RsrcDir, "ATA.rsrc", 'ndrv', "heathrow-ata"};
};
};
#endif


/* The Kauai driver is present in 9.1.1 and later, absent in 9.0.1 and earlier */
#if !BlueBox
resource 'prcl' (1100, "ATA: Kauai") {
kPropertiesForExistingNode,
{"kauai-ata", "ata"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "ATA.rsrc", 'ndrv', "kauai-ata"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1110, "ATA: KeyLargo") {
kPropertiesForExistingNode,
{"keylargo-ata", "ata"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "ATA.rsrc", 'ndrv', "keylargo-ata"};
};
};
#endif


/* The I2C driver is present in 9.6.1 and later, absent in 9.1.1 and earlier */
/* (See also uni-n-i2c below) */
#if !BlueBox
resource 'prcl' (1120, "I2C on KeyWest") {
kPropertiesForExistingNode,
{"keywest-i2c", "i2c"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "I2C.rsrc", 'ndrv', "i2c"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1130, "NVRAM on MacIO") {
kPropertiesForExistingNode,
{"mac-io", "nvram"},
0x0000000a,
{
"driver,AAPL,MacOS,PowerPC",
0x00000004,
fromResource {RsrcDir, "NVRAM.rsrc", 'ndrv', "mac-io"};
};
};
#endif


resource 'prcl' (1140, "File version") {
kPropertiesForExistingNode,
{"macos"},
0x00000001,
{
"MacOSROMFile-version",
0x00000000,

#if BlueBox
useCstring {"4.8GMc1"};
#else
useCstring {"10.2f1"};
#endif

};
};


#if !BlueBox
resource 'prcl' (1150, "'flash' NVRAM??") {
kPropertiesForExistingNode,
{"nvram,flash", "nvram"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000004,
fromResource {RsrcDir, "NVRAM.rsrc", 'ndrv', "flash"};
};
};
#endif


/* PCCard library was in a parcel called "cardbus" in 3.8 and earlier, moved here in 6.1 */
/* These two CardBus drivers were separate in 7.5.1 and earlier, merged in 8.4 */
#if !BlueBox
resource 'prcl' (1160, "CardBus 1") {
kPropertiesForExistingNode,
{"pci104c,ac1a", "cardbus"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000016, /* 0x14 in 7.5.1 and earlier; like this in 8.4 */
fromResource {RsrcDir, "PCCard.rsrc", 'ndrv', "cardbus"};

"PCCard",
0x00010094,
fromResource {RsrcDir, "PCCard.rsrc", 'nlib', "PCCard"};
};
};
#endif


/* This second CardBus parcel was absent in 6.1 and earlier, present in 7.5.1 */
#if !BlueBox
resource 'prcl' (1170, "CardBus 2") {
kPropertiesForExistingNode,
{"pci104c,ac50", "cardbus"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000016,
fromResource {RsrcDir, "PCCard.rsrc", 'ndrv', "cardbus"};

"PCCard",
0x00010094,
fromResource {RsrcDir, "PCCard.rsrc", 'nlib', "PCCard"};
};
};
#endif


/* FireWire parcel absent in 3.0, present in 3.7 */
#if !BlueBox
resource 'prcl' (1180, "FireWire") {
kPropertiesForExistingNode,
{"pciclass,0c0010", "ieee1394"},
0x0000020c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000004,
fromResource {RsrcDir, "FireWire.rsrc", 'ndrv', "firewire"};

kUseResName,
0x00010096,
fromResource {RsrcDir, "FireWire.rsrc", 'nlib', "FWServicesLib"};

kUseResName,
0x00010096,
fromResource {RsrcDir, "FireWire.rsrc", 'nlib', "DevNLib"};

kUseResName,
0x00010096,
fromResource {RsrcDir, "FireWire.rsrc", 'nlib', "DFMLib"};

kUseResName,
0x00010096,
fromResource {RsrcDir, "FireWire.rsrc", 'nlib', "GenericDriverFamily"};
};
};
#endif


resource 'prcl' (1190) {
kPropertiesForExistingNode,
{"pmu", "power-mgt"},
0x0000000c,
{
"pef,AAPL,MacOS,PowerPC,register",
0x00000000,
fromResource {RsrcDir, "PowerManagement.rsrc", 'ndrv', "pmu"};

"code,AAPL,MacOS,name",
0x00000002,
useCstring {"PowerMgrPlugin"};

"target-mode-capable",
0x00000002,
useCstring {"SCSI"};

/* This came before the one above in 3.8, in this position in 6.1. Wonder why... */
kUseResName,
0x00010096,
fromResource {RsrcDir, "PowerManagement.rsrc", 'nlib', "PMULib"};
};
};


/* The I2C driver is present in 9.6.1 and later, absent in 9.1.1 and earlier */
/* (See also keywest-i2c above) */
#if !BlueBox
resource 'prcl' (1200, "I2C on UniNorth") {
kPropertiesForExistingNode,
{"uni-n-i2c", "i2c"},
0x0000000c,
{
"driver,AAPL,MacOS,PowerPC",
0x00000006,
fromResource {RsrcDir, "I2C.rsrc", 'ndrv', "i2c"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1210) {
kPropertiesForExistingNode,
{"uni-north", "pci"},
0x0000000c,
{
"pef,AAPL,MacOS,PowerPC,prepare",
0x00000016,
fromResource {RsrcDir, "PCI.rsrc", 'nlib', "UniNorth_PCICyclesLib"};

"code,AAPL,MacOS,name",
0x00000002,
useCstring {"UniNorth_PCICyclesLib"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1220, "RTC on CUDA") {
kPropertiesForExistingNode,
{"via-cuda", "rtc"},
0x0000000a,
{
"driver,AAPL,MacOS,PowerPC",
0x00000004,
fromResource {RsrcDir, "RTC.rsrc", 'ndrv', "cuda"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1230, "NVRAM on PMU") {
kPropertiesForExistingNode,
{"via-pmu", "nvram"},
0x0000000a,
{
"driver,AAPL,MacOS,PowerPC",
0x00000004,
fromResource {RsrcDir, "NVRAM.rsrc", 'ndrv', "pmu"};
};
};
#endif


#if !BlueBox
resource 'prcl' (1240, "RTC on PMU") {
kPropertiesForExistingNode,
{"via-pmu", "rtc"},
0x0000000a,
{
"driver,AAPL,MacOS,PowerPC",
0x00000004,
fromResource {RsrcDir, "RTC.rsrc", 'ndrv', "pmu"};
};
};
#endif


/* The PMU2000 parcel was absent in 6.1 and earlier, present in 7.5.1 */
resource 'prcl' (1250) {
kPropertiesForExistingNode,
{"via-pmu-2000", "power-mgt"},
0x0000000c,
{
"pef,AAPL,MacOS,PowerPC,register",
0x00000000,
fromResource {RsrcDir, "PowerManagement.rsrc", 'ndrv', "pmu-2000"};

"code,AAPL,MacOS,name",
0x00000002,
useCstring {"PowerMgrPlugin"};

kUseResName,
0x00010096,
fromResource {RsrcDir, "PowerManagement.rsrc", 'nlib', "PMULib"};
};
};

resource 'prcl' (1260) {
kPropertiesForExistingNode,
{"via-pmu-99", "power-mgt"},
0x0000000c,
{
"pef,AAPL,MacOS,PowerPC,register",
0x00000000,
fromResource {RsrcDir, "PowerManagement.rsrc", 'ndrv', "pmu-99"};

"code,AAPL,MacOS,name",
0x00000002,
useCstring {"PowerMgrPlugin"};

kUseResName,
0x00010096,
fromResource {RsrcDir, "PowerManagement.rsrc", 'nlib', "PMULib"};
};
};
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 19, 2017, 06:22:45 PM
I have no idea if the Trampoline is actually reading from both parcels. I just know that the parcels file was read into memory twice and the Trampoline was told the offset and length of the double size parcel. For all I know, the parcels file might have a special "I'm done" signal at the end. I will set up a GitLab account soon.

Code: [Select]
" Toolbox" load-from-path
load-base 2 + constant parcels-offset
load-size 2 - 2 * constant parcels-size
parcels-size pagesz-1 + pagemask and constant parcels-pages
cr ." Starting toolbox allocation"
parcels-pages 400000 claim-mem constant rom-phys parcels-pages 1000 claim-virt constant rom-virt rom-phys rom-virt parcels-pages 10 do-map rom-phys rom-virt parcels-pages 10 do-map 
\ copy the compressed image
debug? if cr ." copying compressed ROM image" then
rom-virt parcels-pages 0 fill
parcels-offset rom-virt parcels-size 2 / move
parcels-offset rom-virt parcels-size 2 / + parcels-size 2 / move
debug? if cr ." MacOS-ROM phys,virt,size: " rom-phys u. ." , " rom-virt u. ." , " parcels-size u. then
\ create the actual property
debug? if cr ." finding/creating '/rom/macos' package" then
device-end 0 to my-self
" /rom" find-device
" macos" ['] find-device catch if 2drop new-device " macos" device-name finish-device then
" /rom/macos" find-device
debug? if cr ." creating 'AAPL,toolbox-parcels' property" then
rom-virt encode-int parcels-size encode-int encode+ " AAPL,toolbox-parcels" property
device-end

Edit:
Just to be clear, this code it part of the Custom ROM file I built. It will not work if you stick it in an ordinary ROM file.
Title: Re: Mac OS Rom Hacking
Post by: DieHard on May 21, 2017, 12:40:46 PM
I am so glad you guys are NOT personal messaging all these developments; I get a woody just reading this thread !

This is so amazing in the sense that we seem to be very close to reverse engineering and running a fully-working modified replacement ROM that conquer the world ! (Or at least replace the "Happy Face" with an OS9 Lives Logo upon boot)
 
Title: Re: Mac OS Rom Hacking
Post by: devils_advisor on May 21, 2017, 01:00:53 PM
I wish i could find the mentioned os7 sources. Supposedly somebody spread them but it seams to be a myth.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 21, 2017, 01:48:33 PM
The system 7 source code link is here:
http://www.toddp.com/classic/Software%20Install/Apple%20Source%20Code/System%207.1%20Source/Tools/ (http://www.toddp.com/classic/Software%20Install/Apple%20Source%20Code/System%207.1%20Source/Tools/)
I have had it on my bookmark bar in Safari since before I came to this forum.

As for developments via private messages, I have kind of moved to that a bit for working on CDG5. Sorry :(
Not all that much has happened since I joined as a developer on GitLab(it was only a few days ago). We did figure out that code like this will not work inside loops or conditionals.
Code: [Select]
dev /
I will try to post when we make major breakthroughs.
Title: Re: Mac OS Rom Hacking
Post by: MacOS Plus on May 21, 2017, 03:17:41 PM
I am so glad you guys are NOT personal messaging all these developments; I get a woody just reading this thread!

<chuckles> We're still a 'family website', right?
Title: Re: Mac OS Rom Hacking
Post by: DieHard on May 21, 2017, 08:31:18 PM
Quote
I will try to post when we make major breakthroughs.

As you wish :)   But many of us enjoy the details... not everything has to be a breakthrough. 

It's nice to see it build and we appreciate all the hard work that you guys have done
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 22, 2017, 06:09:29 AM
The system 7 source code link is here:
http://www.toddp.com/classic/Software%20Install/Apple%20Source%20Code/System%207.1%20Source/Tools/ (http://www.toddp.com/classic/Software%20Install/Apple%20Source%20Code/System%207.1%20Source/Tools/)
I have had it on my bookmark bar in Safari since before I came to this forum.

As for developments via private messages, I have kind of moved to that a bit for working on CDG5. Sorry :(
Not all that much has happened since I joined as a developer on GitLab(it was only a few days ago). We did figure out that code like this will not work inside loops or conditionals.
Code: [Select]
dev /
I will try to post when we make major breakthroughs.

I had that problem with selecting the cpu with the code I have to check the cpu version and change it if need be.
You have to use select-dev
Code: [Select]
" /" select-dev

Quote
I will try to post when we make major breakthroughs.

As you wish :)   But many of us enjoy the details... not everything has to be a breakthrough. 

It's nice to see it build and we appreciate all the hard work that you guys have done

I agree. I haven't had anything to report though.
I think the last thing I had to say, I already posted, but in case I didn't.

I had a theory and ELN is starting to confirm that it is very possible a solution.
The theory is that the 68K emulator and a cache file are mapped to 1.5 GB.  Funny how that is the limit.
The trampoline is what is doing that load and mapping. It is also responsible for generating a mapping of the usable RAM and handing off the total RAM (which in this case don't map).  The trampoline would need to be modified to map the emulator and cache to a higher memory location and pass the usable memory correctly to the kernel.
This would all be done in the trampoline. So reversing that now.

Again this is a theory that ELN is showing this is very much the problem.  We still could be wrong, but something this likely is worth chasing.

This part is not easy and will take time.  We have no existing code base or person who worked on it.  This means a complete reverse, which is a very manual process.  You almost have to literally walk through the code by hand one instruction at a time. Very time consuming. As various methods are identified and the functionality is determined, then it starts to get faster.  So start slow, get faster. Emulators can sort of help, but with out source to match to they are almost useless.  The role they would play would be to find the last instruction before the nanokernel's first instruction.  This would make it able to work backwards and forwards, but backwards won't get very far.  So this is something I'm working through and because it's a slow process, don't expect much yet.

If I make no sense, I'm sorry.
Title: Re: Mac OS Rom Hacking
Post by: WolfpackN64 on May 22, 2017, 07:27:44 AM
I had a theory and ELN is starting to confirm that it is very possible a solution.
The theory is that the 68K emulator and a cache file are mapped to 1.5 GB.  Funny how that is the limit.
The trampoline is what is doing that load and mapping. It is also responsible for generating a mapping of the usable RAM and handing off the total RAM (which in this case don't map).  The trampoline would need to be modified to map the emulator and cache to a higher memory location and pass the usable memory correctly to the kernel.
This would all be done in the trampoline. So reversing that now.

Again this is a theory that ELN is showing this is very much the problem.  We still could be wrong, but something this likely is worth chasing.

This part is not easy and will take time.  We have no existing code base or person who worked on it.  This means a complete reverse, which is a very manual process.  You almost have to literally walk through the code by hand one instruction at a time. Very time consuming. As various methods are identified and the functionality is determined, then it starts to get faster.  So start slow, get faster. Emulators can sort of help, but with out source to match to they are almost useless.  The role they would play would be to find the last instruction before the nanokernel's first instruction.  This would make it able to work backwards and forwards, but backwards won't get very far.  So this is something I'm working through and because it's a slow process, don't expect much yet.

If I make no sense, I'm sorry.

You guys are doing amazing work anyhow. I just wish I could throw some money your way.  :)
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 22, 2017, 12:07:15 PM
I had a theory and ELN is starting to confirm that it is very possible a solution.
The theory is that the 68K emulator and a cache file are mapped to 1.5 GB.  Funny how that is the limit.
The trampoline is what is doing that load and mapping. It is also responsible for generating a mapping of the usable RAM and handing off the total RAM (which in this case don't map).  The trampoline would need to be modified to map the emulator and cache to a higher memory location and pass the usable memory correctly to the kernel.
This would all be done in the trampoline. So reversing that now.

Again this is a theory that ELN is showing this is very much the problem.  We still could be wrong, but something this likely is worth chasing.

This part is not easy and will take time.  We have no existing code base or person who worked on it.  This means a complete reverse, which is a very manual process.  You almost have to literally walk through the code by hand one instruction at a time. Very time consuming. As various methods are identified and the functionality is determined, then it starts to get faster.  So start slow, get faster. Emulators can sort of help, but with out source to match to they are almost useless.  The role they would play would be to find the last instruction before the nanokernel's first instruction.  This would make it able to work backwards and forwards, but backwards won't get very far.  So this is something I'm working through and because it's a slow process, don't expect much yet.

If I make no sense, I'm sorry.

Nice. I didn't even have to suggest working on the Trampoline next :)
Is there any particular reason the Trampoline is in an Elf file? It is just code and data. It could probably be turned into an xcoff file. You would have to look at the relocation tables and stuff(I am unfamiliar on the exact terminology) to figure out how to arrange the data so that the code can get to it. You would also have to figure out if the method for finding the OF client interface is different in an xcoff. BootX seems like a good place to look for that(its source code is even available somewhere). Once we get the Trampoline in a format that can be built easily with MPW then we can start turning it into assembly(and possibly C or Pascal). I am afraid that I will not be of much help until then. I have essentially no experience with ppc assembly, elf files, or xcoff files :(
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 22, 2017, 12:55:10 PM
Nice. I didn't even have to suggest working on the Trampoline next :)
Is there any particular reason the Trampoline is in an Elf file? It is just code and data. It could probably be turned into an xcoff file. You would have to look at the relocation tables and stuff(I am unfamiliar on the exact terminology) to figure out how to arrange the data so that the code can get to it. You would also have to figure out if the method for finding the OF client interface is different in an xcoff. BootX seems like a good place to look for that(its source code is even available somewhere). Once we get the Trampoline in a format that can be built easily with MPW then we can start turning it into assembly(and possibly C or Pascal). I am afraid that I will not be of much help until then. I have essentially no experience with ppc assembly, elf files, or xcoff files :(


It's disassembled. Method boundaries are mostly identified.  Code functionality is be determined.  Specific methods that should exists are being looked for.  Loops and recursive methods are being determined for first round analysis as they will be the parcel processing and device tree walker.

ELF xcoff and raw are loadable by OF. Why ELF is beyond me.  The ELF is static and not relocatable.  There are no relocation tables. There is one data section and one code section.  The layout and memory of the loaded code is known.

It really is just a lot of manual work at this point since there is absolutely no code reference for this.

BootX does a lot less than the trampoline.  Most of what the trampoline does is built into XNU. You have to look at XNU init code, kexts and all the hardware frameworks to figure that out.  And it's not as fun as manual reversing.

Progress is being made.  I was going to be working on it a lot this past weekend, but with the past week of almost non-stop rain a lot of houses here have some mild to major flooding in basements.  I spent all weekend pumping water out of my basement. Thankfully the water never touched any of my mac's.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 22, 2017, 04:30:46 PM
Wow. That is way farther along than I had thought we were :)
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 23, 2017, 06:20:22 AM
Wow. That is way farther along than I had thought we were :)

Yeah.  Slow but moving. Not really much to say here  as that moves, unless people really want to hear method counts, method addresses and how many lines of code I have reviewed. If people want that, then cool I'll share it. Just would be a lot of info that may be useless because I would have no confidence in it until I get further. At that point I will share.

Sorry folks if you want more. Just ask. I will share all the mundane details.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 30, 2017, 01:58:31 PM
I was reading the revised edition of "Designing PCI Cards and Drivers" when I discovered this little snippet. It is part of a list describing how New World Macs are initialized. It is on page 92 (amazingly, page 92 of the pdf corresponds with the page numbered 92. This usually does not happen).
Quote
6. The Trampoline code decompresses the Mac OS ROM image, gathers information about the system from Open Firmware, creates data structures based on this information, terminates Open Firmware, and rearranges the contents of memory to an interim location in physical memory space.
7. The Trampoline code transfers control to the HardwareInit routine in the Mac OS ROM bootinfo file.
8. The HardwareInit routine copies data structures to their correct places in memory, and then calls the NanoKernel.
9. The NanoKernel fills in its data structures and then calls the 68K emulator.
So, have you found this HardwareInit routine yet? It looks interesting...
Title: Re: Mac OS Rom Hacking
Post by: IIO on May 30, 2017, 02:26:47 PM
daniel: what will happen if you boot another, non-prepared system folder with this rom file - or the other way round?
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 30, 2017, 02:47:55 PM
It depends. If you stick the custom rom file in a system folder by itself, it will just not work. The file I made stores the required data in the bootfiles folder. It would not even begin to boot up normally. If you somehow boot up a system folder that has the custom rom file and bootfiles folder with another rom file (maybe by having that rom file on a partition without a system file and finder), it will behave just as a normal system folder. There is nothing in the normal startup code that looks in the bootfiles folder, so it is just ignored. The rom file would normally be consulted for a few things, but it has no resource fork so those consultations fail. Mac OS 9 seems to be ok with that. You could have a system that absolutely has to have some of the resources (or datafork pefs) stored in a regular rom file, but I have not come across one of those systems yet and have no idea which ones they would be, if any. I hope that this answered your question.
Title: Re: Mac OS Rom Hacking
Post by: ELN on May 31, 2017, 01:25:26 AM
So, have you found this HardwareInit routine yet? It looks interesting...

Unfortunately not. The HardwareInit routine was originally at ROM+340000 but was removed in v2 of the NewWorld ROM (the first version to use parcels, also called NewNewWorld). The later Trampoline versions jump straight to the NanoKernel, at ROM+310000.
Title: Re: Mac OS Rom Hacking
Post by: Daniel on May 31, 2017, 03:46:25 AM
Fascinating. I thought it would be something like that. The book had no mention of Mac OS 9 in it. Still, they probably share a lot of the same code. It would be interesting to find a sequence of opcodes in one and search for it in the other. That might tell us where the pagemap stuff in the Trampoline is.
Title: Re: Mac OS Rom Hacking
Post by: nanopico on May 31, 2017, 06:02:35 AM
Fascinating. I thought it would be something like that. The book had no mention of Mac OS 9 in it. Still, they probably share a lot of the same code. It would be interesting to find a sequence of opcodes in one and search for it in the other. That might tell us where the pagemap stuff in the Trampoline is.

I may have a few points wrong here but it is pretty close.

A lot of the documentation in Inside Macintosh and some of the extension like you are reading stopped being updates during OS 8 and 7.
By 9 they had no interest because they knew OS X was coming. Most of the documentation for 9 that was different from Inside Macintosh was done through Tech Notes if that.
Finding some of that data can be sort of hard.
The last thing I found in regards to the new world rom's and open firmware was from the first version and quit a bit changed since then.
Title: Re: Mac OS Rom Hacking
Post by: nightstalkerpoet on January 06, 2018, 08:51:17 AM
Is it possible to simply write a new U3H_PCICyclesLib file to replace the others, based on CPC925? ala https://bugs.freedesktop.org/attachment.cgi?id=131346 ?

https://opensource.apple.com/source/

https://web.archive.org/web/20131106030015/http://dec8.info/Apple/macos8pdfs/

http://www.cpu.lu/~mlan/ftp/MacTech.pdf
Title: Re: Mac OS Rom Hacking
Post by: IIO on July 18, 2022, 01:41:02 AM
The system 7 source code link is here:
https://web.archive.org/web/20200224195214/http://www.toddp.com/classic/Software%20Install/Apple%20Source%20Code/System%207.1%20Source/Tools/
Title: Re: Mac OS Rom Hacking
Post by: zefrenchtoon on July 18, 2022, 11:20:20 AM
The system 7 source code link is here:
https://web.archive.org/web/20200224195214/http://www.toddp.com/classic/Software%20Install/Apple%20Source%20Code/System%207.1%20Source/Tools/

Very interesting but unfortunately, some files seems to be missing on the archive :(
Title: Re: Mac OS Rom Hacking
Post by: Daniel on July 18, 2022, 01:30:44 PM
The system 7 source code link is here:
https://web.archive.org/web/20200224195214/http://www.toddp.com/classic/Software%20Install/Apple%20Source%20Code/System%207.1%20Source/Tools/

Very interesting but unfortunately, some files seems to be missing on the archive :(

Elliot (ELN on this site) has the sources, and has made various attempts to update them to Mac OS 9 over the years. Here is the latest attempt https://github.com/elliotnunn/supermario (https://github.com/elliotnunn/supermario).

Title: Re: Mac OS Rom Hacking
Post by: IIO on July 18, 2022, 04:33:04 PM
oh, didnt check if it is complete. just noticed that the original is gone and thought i´d update the thread.

missing download links on archived sites is a pestilence.

can we post it here? i think it was on apple ftp for some years, too.