Disk Speed Upgrades (aka The Bootable PCI SATA & SSD thread)

[smilesdavis]

just when i found 2 sonnets after a year of search rofl

[dosdude1]

I suppose I should have posted in this thread instead, but yes indeed, I have successfully patched out the EEPROM ID check in the SeriTek/1S2 ROM! After doing so, though, I had intended to see if I could reduce the ROM enough to get it to fit onto a 128K EEPROM, but unfortunately there is just too much there to be able to reduce it and retain its full functionality. With that said, however, there is an older version of the ROM, v5.0.7, that apparently is small enough to fit on a 128K EEPROM. Of course it never actually worked with a 128K EEPROM installed, as the ID check prevented it, but with the appropriate patches, could be made to work no issues. The problem, though, is I haven't been able to find a copy of that version of the ROM anywhere. So if someone could dig up a copy, that would be AMAZING, and of course be the ultimate solution to flashing these cards without having to replace their EEPROM. My patched version of the SeriTek/1S2 v5.1.3 ROM is attached.

[joevt]

How much bigger than 128K is the patched ROM? Maybe you could add compression to make it fit (if it isn't already compressed). The lzss algorithm is probably simple enough to implement in Open Firmware. The Sonnet cards used this method.

[dosdude1]

How much bigger than 128K is the patched ROM? Maybe you could add compression to make it fit (if it isn't already compressed). The lzss algorithm is probably simple enough to implement in Open Firmware. The Sonnet cards used this method.

It comes out to around 138K. Most of the space is being taken up by the OS X NDRV, but that is already compressed (it's an mkext shoved in as binary data).

[joevt]

I would run it through a compressor to see if it can get to less than 128K anyway. Compare compressing with and without the mkext part.

Here's a lzss decompressor (C code) https://forums.macrumors.com/threads/question-how-powerful-of-a-graphics-card-will-work-in-a-beige-power-macintosh-g3.2303689/post-31602677
I used it for decompressing New World Mac Open Firmware.
I included a version that does compression and decompression but I haven't checked if the compression produces output that can be decompressed by the version that decompresses Mac New World Open Firmware. It shouldn't matter. The point is that the decompression algorithm is simple enough to implement in Open Firmware.

I would decompress to memory outside the Open Firmware dictionary since that is a limited resource. I would avoid using encode-bytes+ if that is using space in the dictionary. Maybe there's a fcode trick to include binary blobs larger than a string of 256 characters. I am thinking something like this:

Code: [Select]

" "(00012345)" \ encode the size of the binary blob in a small string here; the following code only works if the address of this string is pointing to this fcode
decompress \ this takes the address from the stack of the string, reads the 4 bytes representing the length of the binary blob below, adds an offset to skip this decompress call and skip the following branch fcode to get the address of the blob,
\ branch fcode goes here to jump over the following 12345 bytes of the binary blob.
« 12345 bytes of stuff goes here»
executedecompressedfcodeimage
To make this work will also require modification to the tokenizer to support arbitrary length binary blobs that will produce the fcode output described above.
Another problem is that this only works for blobs that are less than 32K since that is the limit of the branch fcode. In that case, do it without the branch, have the blob at the end of the uncompressed first stage fcode, include the offset of the binary blob in the string that encodes the length.
Actually, the Open Firmware spec says that strings are copied from fcode to a temporary buffer (str0 or str1 in Mac Open Firmware), therefore you can't get a pointer to fcode that way. You may just want to map the PCI ROM and get the binary blob from that. Or forget this and use encode-bytes+.

[ssp3]

It comes out to around 138K. Most of the space is being taken up by the OS X NDRV, but that is already compressed (it's an mkext shoved in as binary data).

Indeed, starting from around offset 0x21d70 it contains all zeroes.
I know nothing about ROM dissasembling, their structures etc. (I wish, I knew, since I'm trying to finalize my WD MyBook Studio Firewire jailbreak), but, can't the firmware, by someone skilled in the trade
a) be optimized at the machine code level manually, or/and
b) decompiled and then re-compiled with different optimization level (min size) or with different compiler
to get the final file size under 128k?

I've loaded the firmware in Hopper and, when looking at some of the procedures in pseudo-code mode and checking/unchecking 'remove potentially dead code' checkbox, there were some differences.

[joevt]

Indeed, starting from around offset 0x21d70 it contains all zeroes.
I know nothing about ROM dissasembling, their structures etc. (I wish, I knew, since I'm trying to finalize my WD MyBook Studio Firewire jailbreak), but, can't the firmware, by someone skilled in the trade
a) be optimized at the machine code level manually, or/and

manually is a lot of manual work.

b) decompiled and then re-compiled with different optimization level (min size) or with different compiler
to get the final file size under 128k?

You can decompile and re-compile fcode with just a couple clicks. There's no optimizing that can be done here except the tokenizer can choose to convert some 5 byte literals to single byte fcode (the numbers -1,0,1,2,3 for example). A smarter tokenizer could reuse duplicate field definitions - basically a field is an fcode that adds a number to a number on the stack. A 5+ field for one struct is exactly the same as a 5+ field for another struct. Also, a tokenizer could replace code that doesn't use fields with a field fcode - for example "100 +" could be replaced with a 100+ field fcode if it exists.

You can disassemble assembly code into something that can be reassembled with some effort. There's no optimization for assembly code.

Decompiling from assembly to C code is difficult. Tools like Hopper or IDA usually don't produce code that can be easily compiled that behaves exactly the same as the original. Optimizers can be applied to C code though.

I've loaded the firmware in Hopper and, when looking at some of the procedures in pseudo-code mode and checking/unchecking 'remove potentially dead code' checkbox, there were some differences.

What did you load in Hopper? The firmware for a Power Mac PCIe card contains a lot of Open Firmware code which Hopper knows nothing about. Did you extract the NDRV? Without extracting the NDRV it will contain 5 or 6 bytes of fcode for every 250+ bytes of PowerPC code. Hopper doesn't produce pseudo code for PowerPC code - are you looking at Intel code? The "remove potentially dead code" option might make the pseudo code easier to read - or it might make the pseudo code less accurate.

[Borgmac]

Quote from DieHard

Perhaps there is a kind soul that wants to make a small, but reasonable profit, and give us a price for cards that are modded and ready to go.  Everyone's time is worth money and many will not be able to make their own card, so it is critical that a member that has the time and resources will take this knowledge and help the masses.  Hopefully, this will make the precious OS9 bootable SATA cards in reach for all

I am not up to that level yet, but I have been working on the famous Chernobyl Quicksilver to find an easy way to install Linux and Flashrom to be able to flash the Sil3112 SATA cards on a PowerMac G4.
Starting from a very basic knowledge of UNIX, it took me a while to find an easy Linux setup without burning one CD for every Linux version.
I have found a solution, using the cheap IDE to SATA adapter for an Inland 120 GB SSD, but it is using an app (BalenaEtcher) requiring Mac OS 10.10 minimum
The next step is to install the Flashrom program, compatible with the Linux version installed.
Then I will need to find a way to copy directly the Linux CD to the SSD without the BalenaEtcher, which I know is possible
Will continue to work on that and flash my first card.
Then I could have some time available...

[ssp3]

What did you load in Hopper? The firmware for a Power Mac PCIe card contains a lot of Open Firmware code which Hopper knows nothing about. Did you extract the NDRV? Without extracting the NDRV it will contain 5 or 6 bytes of fcode for every 250+ bytes of PowerPC code. Hopper doesn't produce pseudo code for PowerPC code - are you looking at Intel code? The "remove potentially dead code" option might make the pseudo code easier to read - or it might make the pseudo code less accurate.

As I already said, I have no idea what kind of code is contained in 1s2 firmware. And it was my question to someone skilled in the trade, not a statement
I loaded it in Hopper just for kicks and saw that parts of it were recognized as Intel procedures, albeit with some strange looking instructions here and there.

[joevt]

I looked at the dosdude 1S2_512-patched.rom.

My tokenizer saves 199 bytes by converting numbers like 0,1,2,3 from 5 bytes to 1 byte. The fcode is 138343 bytes. 135.1 KiB.
Without the fcode that encodes the ndrv and mkext, the fcode is 10595 bytes. 10.3 KiB.
lzss cannot compress the mkext - it grows from 70069 to 75172 bytes.
lzss does a pretty good job on the ndrv - it shrinks from 54648 to 30527 bytes.
That's probably sufficient to get the size of the rom to less than 128 KiB (after you add forth code that does lzss decompression). You wouldn't have to compress all the fcode, or even the fcode that produces the ndrv. Instead compress the ndrv binary, include that as a series of encode-bytes+ in the forth code, then add forth code that uses the lzss decompress function on the result of that to create the blob for the ndrv property.

The lzss decompression function is less than 50 lines of C code so it should be pretty easy to convert it to forth.

[joevt]

A couple things: the fcode appears to be missing the fcode-end at the end of the rom.

Code: [Select]

" 5.1.3.1S2"
encode-string
" rom-revision"
property

fcode-end

pci-end

And this function is weird because the if ... then crosses a repeat. It's like the if is a conditional break or leave instruction.

Code: [Select]

: colon_definition_function_8a2 \ (8a2) [0b5 0b7]
get-msecs \ [125]
begin \ [0b1]
readstatusreg \ [859]
dup \ [047]
0<> \ [035]
over \ [048]
0ff \ [010]
<> \ [03d]
and \ [023]
swap \ [049]
0c0 \ [010]
and \ [023]
40 \ [010]
<> \ [03d]
and \ [023]
if \ (0x14) [014]
get-msecs \ [125]
over \ [048]
- \ [01f]
1388 \ [010]
< \ [03a]
while \ (0x6) [014]
repeat \ (0xffd5) [013 0b2]
then \ [0b2]
drop \ [046]
; \ [0c2]
I don't think it's a problem. It just looks weird. My tokenizer accepts this interpretation and produces the following fcode that results in the same interpretation

Code: [Select]

001418: b(:) \ [0x0b7]
001419:     get-msecs \ [0x125]
00141B:     b(<mark) \ [0x0b1]
00141C:         readstatusreg \ [0x859]
00141E:         dup \ [0x047]
00141F:         0<> \ [0x035]
001420:         over \ [0x048]
001421:         b(lit) \ [0x010] 0xff
001426:         <> \ [0x03d]
001427:         and \ [0x023]
001428:         swap \ [0x049]
001429:         b(lit) \ [0x010] 0xc0
00142E:         and \ [0x023]
00142F:         b(lit) \ [0x010] 0x40
001434:         <> \ [0x03d]
001435:         and \ [0x023]
001436:         b?branch \ [0x014] 0x14
001439:             get-msecs \ [0x125]
00143B:             over \ [0x048]
00143C:             - \ [0x01f]
00143D:             b(lit) \ [0x010] 0x1388
001442:             < \ [0x03a]
001443:             b?branch \ [0x014] 0x6
001446:                 bbranch \ [0x013] 0xffffffd5
001449:             b(>resolve) \ [0x0b2]
00144A:         b(>resolve) \ [0x0b2]
00144B:     drop \ [0x046]
00144C:     b(;) \ [0x0c2]


[dosdude1]

It comes out to around 138K. Most of the space is being taken up by the OS X NDRV, but that is already compressed (it's an mkext shoved in as binary data).

Indeed, starting from around offset 0x21d70 it contains all zeroes.
I know nothing about ROM dissasembling, their structures etc. (I wish, I knew, since I'm trying to finalize my WD MyBook Studio Firewire jailbreak), but, can't the firmware, by someone skilled in the trade
a) be optimized at the machine code level manually, or/and
b) decompiled and then re-compiled with different optimization level (min size) or with different compiler
to get the final file size under 128k?

I've loaded the firmware in Hopper and, when looking at some of the procedures in pseudo-code mode and checking/unchecking 'remove potentially dead code' checkbox, there were some differences.

It's not exactly that simple... There are two NDRVs in the ROM that need to be patched. One for OS X, and one for OS 9. There is no EEPROM ID check in the actual FCode itself. For the OS X one, the first thing you need to do is extract the binary blob into a raw binary file, which will then create an mkext file you can work with. Next, you'll need to unpack the mkext. You can do so using "mkextunpack" under OS X. This should be done under an older OS X version (I used 10.4 Tiger) on a PowerPC machine for best results, though for this step it doesn't really matter. Once you have the kexts extracted, NOW you can open them in IDA, or other disassembler of choice, and begin reverse-engineering. "FT_ATA_Sil3112.kext" is the one where the EEPROM ID check is implemented. You WILL need at least some knowledge of PowerPC assembly in order to make much sense of it, and make any changes. Once you determine and make desired binary patches, you will then need to reverse the extraction process... Use OS X 10.4 on PPC to create new mkext with modified kexts (you can use "kextcache"), and get the modified mkext back into the EEPROM -- That's the tricky part, which I had to make a few scripts for, use FCode tokenizer, etc.

The OS 9 binary is quite a bit easier... All you need to do for that one is extract the binary blob into a binary file, and start reverse-engineering... It is not compressed or packed in any way.

Lastly, the zeroes at the end of the ROM are not needed, they are just there from when the ROM image was dumped from a 512K EEPROM. They can simply be removed.

[dosdude1]

I looked at the dosdude 1S2_512-patched.rom.

My tokenizer saves 199 bytes by converting numbers like 0,1,2,3 from 5 bytes to 1 byte. The fcode is 138343 bytes. 135.1 KiB.
Without the fcode that encodes the ndrv and mkext, the fcode is 10595 bytes. 10.3 KiB.
lzss cannot compress the mkext - it grows from 70069 to 75172 bytes.
lzss does a pretty good job on the ndrv - it shrinks from 54648 to 30527 bytes.
That's probably sufficient to get the size of the rom to less than 128 KiB (after you add forth code that does lzss decompression). You wouldn't have to compress all the fcode, or even the fcode that produces the ndrv. Instead compress the ndrv binary, include that as a series of encode-bytes+ in the forth code, then add forth code that uses the lzss decompress function on the result of that to create the blob for the ndrv property.

The lzss decompression function is less than 50 lines of C code so it should be pretty easy to convert it to forth.

This is probably the best way to go... I'll start working on this and see if I can get that implemented.

[ssp3]

It's not exactly that simple...

Thanks for the insight. I learn something new every day 
P.S. Love your Mac OS X USB Drive Creator. Excellent tool!

[joevt]

It's not exactly that simple... There are two NDRVs in the ROM that need to be patched. One for OS X, and one for OS 9. There is no EEPROM ID check in the actual FCode itself. For the OS X one, the first thing you need to do is extract the binary blob into a raw binary file, which will then create an mkext file you can work with. Next, you'll need to unpack the mkext. You can do so using "mkextunpack" under OS X. This should be done under an older OS X version (I used 10.4 Tiger) on a PowerPC machine for best results, though for this step it doesn't really matter. Once you have the kexts extracted, NOW you can open them in IDA, or other disassembler of choice, and begin reverse-engineering. "FT_ATA_Sil3112.kext" is the one where the EEPROM ID check is implemented. You WILL need at least some knowledge of PowerPC assembly in order to make much sense of it, and make any changes. Once you determine and make desired binary patches, you will then need to reverse the extraction process... Use OS X 10.4 on PPC to create new mkext with modified kexts (you can use "kextcache"), and get the modified mkext back into the EEPROM -- That's the tricky part, which I had to make a few scripts for, use FCode tokenizer, etc.

The OS 9 binary is quite a bit easier... All you need to do for that one is extract the binary blob into a binary file, and start reverse-engineering... It is not compressed or packed in any way.

Lastly, the zeroes at the end of the ROM are not needed, they are just there from when the ROM image was dumped from a 512K EEPROM. They can simply be removed.

What's the point of an mkext in a PCI rom? If the Mac OS X kernel can use the mkext, it can also use the extracted kexts saved to disk. I suppose the mkext makes using read only media easier (can you connect sata CD or DVD's to the SATA card?). Also, without the mkext, you need to be able to copy the kexts to the drive. I suppose if you were using XPostFacto, the kexts could be copied to a helper disk (built in SCSI or ATA).

Anyway, if you think there are PCI cards with 64K ROMs, then removing the mkext would make the firmware fit in those cases.

[joevt]

This is probably the best way to go... I'll start working on this and see if I can get that implemented.

Maybe consider using dingusppc to test your decompression algorithm.

I use a script named run to start dingusppc:

Code: [Select]

#!/bin/bash
cd /Volumes/Work/Programming/dingusppc/joevt-dingusppc/build/bin
thewindow="$(osascript -e '
tell application "Terminal"
    set currenttab to do script ""
    tell currenttab
        tell current settings
            set custom title to "dingusppc Modem Port"
            set title displays device name to false
            set title displays shell path to false
            set title displays window size to false
            set title displays settings name to false
            set number of columns to 140
            set number of rows to 80
        end tell
        do script "clear; sleep 2; cd /Volumes/Work/Programming/dingusppc/joevt-dingusppc/build/bin; socat UNIX-CLIENT:dingussocket -,cs8,parenb=0,echo=0,icanon=0,isig=0,icrnl=0" in currenttab
    end tell
    return currenttab
end tell
')"

# 2.4
./dingusppc -d -b "/Volumes/Work/Open Firmware and Name Registry/ROM PowerPC Mac/ROM PM G3/ROMs/@FFC00000 len-400000" --serial_backend=socket --rambank1_size=256 --rambank2_size=256 --rambank3_size=256 --fdd_img=FATFLOPPY.img --pci_B1=Nv47

sleep 1
osascript -e 'tell app "Terminal" to close '"${thewindow#*of }"

Then I execute it using ./run
Include -d for debug. When you start dingusppc, it will show the debug prompt, then you can printenv and setenv and nvedit to modify Open Firmware settings. I set input and output device to ttya and auto-boot? to false so it will enter Open Firmware after you leave debug mode by typing go

Include --serial_backend=socket so that serial port will go to a socket (not available for Windows).

The run script uses AppleScript to automatically open that socket into a new Terminal.app window. Or you could manually enter the socat command in a new Terminal.app window that you create yourself.

Use my fork at https://github.com/joevt/dingusppc
Use the xcode-project fork.
It requires installing sdl2 which you can do with homebrew.

Create code for the sata pci card similar to promise20269.c   This implementation doesn't handle any of the registers but it is enough to load the rom. Since it doesn't handle the registers, the fcode takes a while to complete but it does complete. Another option is to just replace the rom (since I didn't include it in the fork) because you just want to test decompression of the ndrv.

To add a PCI card to slot C1, add to the command line:
 --pci_C1=PCIDeviceName

[ssp3]

Question - are there any tools available that can extract above mentioned mkext and ndrv or do I have to find them manually in hex editor? Preferably without compiling that requires capstone, macports, brew, High Sierra and all that kind of stuff  Asking out of pure curiosity.

[joevt]

Question - are there any tools available that can extract above mentioned mkext and ndrv or do I have to find them manually in hex editor? Preferably without compiling that requires capstone, macports, brew, High Sierra and all that kind of stuff  Asking out of pure curiosity.

It's a process. Use detok to convert a rom to text (or use my DumpPCIRom.sh script which uses detok and produces forth code that can be used with toke). Extract the lines that encode the ndrv or mkext properties, use search and replace to remove the non-hex stuff, use xxd to convert hex to binary, research mkext, google utilities that can build mkext or extract kexts.
 https://forums.macrumors.com/threads/question-how-powerful-of-a-graphics-card-will-work-in-a-beige-power-macintosh-g3.2303689/

[IIO]

What's the point of an mkext in a PCI rom?

i was thinking something similar.

in case it is impossible to fit everything onto the card, maybe it could make sense to provide the OSX functionality with a premade kext installer and have only the OS9 driver on the card?

the requirement of installing a driver for a card is not a big deal for the enduser.

[dosdude1]

What's the point of an mkext in a PCI rom?

i was thinking something similar.

in case it is impossible to fit everything onto the card, maybe it could make sense to provide the OSX functionality with a premade kext installer and have only the OS9 driver on the card?

the requirement of installing a driver for a card is not a big deal for the enduser.

That's definitely a possibility, though my goal was to retain the original ROM's functionality. If compressing the OS 9 NDRV doesn't end up working out, I'll just modify the ROM and rip out the OS X mkext. The kexts can indeed simply be installed into OS X as normal.

[ssp3]

It's a process. Use detok to convert a rom to text (or use my DumpPCIRom.sh script which uses detok and produces forth code that can be used with toke). Extract the lines that encode the ndrv or mkext properties, use search and replace to remove the non-hex stuff, use xxd to convert hex to binary, research mkext, google utilities that can build mkext or extract kexts.
 https://forums.macrumors.com/threads/question-how-powerful-of-a-graphics-card-will-work-in-a-beige-power-macintosh-g3.2303689/

Eek! That's way too much body movement for someone who just curious. (Disclaimer - I don't have the card and don't plan to use any).
EDIT. Found "FT_ATA_Sil3112.kext v.5.3.1" inside the InstallPPC.app that's on 5.3.1 firmware/software dmg, although its binary is 255k.

[refinery]

I would think the purpose of having the mkext would be for cases where somebody is trying to boot off the disk on the card... if you have no disk on the internal bus, how's it going to load a driver for the card from a disk it doesnt know how to access yet at that stage of booting?

[IIO]

I would think the purpose of having the mkext would be for cases where somebody is trying to boot off the disk on the card... if you have no disk on the internal bus, how's it going to load a driver for the card from a disk it doesnt know how to access yet at that stage of booting?

of course you are right, i forgot about that scenario. you wouldnt be able to boot into X from it.

[dosdude1]

I would think the purpose of having the mkext would be for cases where somebody is trying to boot off the disk on the card... if you have no disk on the internal bus, how's it going to load a driver for the card from a disk it doesnt know how to access yet at that stage of booting?

of course you are right, i forgot about that scenario. you wouldnt be able to boot into X from it.

Once you got the kext into the target install of OS X on the target drive, it will be able to boot from it just fine... Just a bit of a pain, having it embedded in the ROM is much more convenient.

[joevt]

I would think the purpose of having the mkext would be for cases where somebody is trying to boot off the disk on the card... if you have no disk on the internal bus, how's it going to load a driver for the card from a disk it doesnt know how to access yet at that stage of booting?

You are saying that Mac OS X can't boot from a disk unless there's a kext in firmware.

So how does Mac OS X boot from a built-in SCSI or ATA disk? Those do not have kexts in firmware.

I believe Open Firmware is used to read kexts or kext cache but you can check BootX source code to verify that.
 https://www.oreilly.com/library/view/mac-os-x/0596003560/ch02s01s01.html
 https://flylib.com/books/en/3.126.1.47/1/
 https://github.com/apple-oss-distributions/BootX/blob/814114e6a6cf10dfa512c520f5a1c1fb9c58a432/bootx.tproj/include.subproj/fs.h
BootX contains HFS+ file system driver that calls Open Firmware.

[ssp3]

Apparently, kextcache by default produces compressed mkexts. Probably that's the reason why lzss couldn't compress it further.
What happens if one makes uncompressed mkext and then compresses it with lzss? Any gain in size reduction?
(I can't test it myself, I have neither original 513 mkext, nor any 10.4/10.5 PPC machines).

From kextcache man:

-compressed
              Compress the mkext or prelinked kernel (enabled by default).
-uncompressed
              Do not compress the mkext or prelinked kernel.  If specified as
              the only other argument with -c, uncompresses an existing pre-
              linked kernel file in place.

Also, what about upx or mpress? Do they exist for PPC and, if so, can they be used on kexts?

[ssp3]

Once you got the kext into the target install of OS X on the target drive, it will be able to boot from it just fine... Just a bit of a pain, having it embedded in the ROM is much more convenient.

Re. embedded in ROM. There's this bit from SeriTek/1S2, version 5.3.1 install instructions:

WHEN TO UPGRADE FIRMWARE
If your current firmware is working for you, FirmTek does not
recommend updating the firmware.
<snip>
One exception to this rule: Leopard users will want to upgrade. The
new 5.3.1 firmware corrects a Leopard wake on sleep issue. However,
Tiger users that are not experiencing problems should stay with
their existing firmware.

So, to cover the Leopard wake on sleep issue, one has to be prepared to deal with 5.3.1 firmware, which is 175k in size compared to 138k of 5.1.3.

[peeperpc]

Once you got the kext into the target install of OS X on the target drive, it will be able to boot from it just fine... Just a bit of a pain, having it embedded in the ROM is much more convenient.

Re. embedded in ROM. There's this bit from SeriTek/1S2, version 5.3.1 install instructions:

WHEN TO UPGRADE FIRMWARE
If your current firmware is working for you, FirmTek does not
recommend updating the firmware.
<snip>
One exception to this rule: Leopard users will want to upgrade. The
new 5.3.1 firmware corrects a Leopard wake on sleep issue. However,
Tiger users that are not experiencing problems should stay with
their existing firmware.

So, to cover the Leopard wake on sleep issue, one has to be prepared to deal with 5.3.1 firmware, which is 175k in size compared to 138k of 5.1.3.

I tried 5.3.1 firmware for a day, with 10.5.8 as the main OS (MDD Mac).

- With 5.1.3, using a 2TB Seagate drive with an ATA/SATA adapter as the boot drive, it takes noticeably longer before the system finds the drive and starts booting from it. But with 5.3.1, the startup time is back to normal. Yes, the boot drive is on the internal ATA100 bus but somehow the firmware can still interfere.

- With 5.1.3, I can use smartmontools with the drive connected to the card. But with 5.3.1, I can't.

- I've had no sleep issue with 5.1.3 nor 5.3.1.

In the end, I reverted to 5.1.3. Since smartmontools is more important for me than shorter boot time.

[dosdude1]

I have another absolutely MASSIVE update for you all... I was able to implement an LZSS decompression algorithm in the ROM, and use it to decompress a stored LZSS-compressed version of the OS9 NDRV. As such, the ROM has been reduced in size to UNDER 128K! This means that you can now flash it onto cards WITHOUT having to replace the original 128k EEPROM! Of course this image also includes my patched OS9 and OS X NDRVs that remove the EEPROM ID check. The updated ROM image is attached, along with my custom Forth LZSS decompression implementation. Hope you all enjoy!

EDIT: Updated ROM padding to 128K for Flashrom support.

[ssp3]

Cool! 
Silly question - does compression impact data transfer rate anyhow?

[dosdude1]

Cool! 
Silly question - does compression impact data transfer rate anyhow?

Not at all, all it does is decompress the LZSS-compressed OS9 NDRV binary upon system power on, at that point it is loaded into memory and is then executed by the OS during boot.

[ssp3]

Not at all, all it does is decompress the LZSS-compressed OS9 NDRV binary upon system power on, at that point it is loaded into memory and is then executed by the OS during boot.

Thanks for the info! You did a great job and big favour to Mac users!


Off-topic. Are you familiar with ARM thumb mode firmware reversing? I'm thinking of liberating some WD Firewire external enclosure(s). I've been partially successful, but lacking further skills. If interested, let me know, I'll PM you.

[dosdude1]

Not at all, all it does is decompress the LZSS-compressed OS9 NDRV binary upon system power on, at that point it is loaded into memory and is then executed by the OS during boot.

Thanks for the info! You did a great job and big favour to Mac users!


Off-topic. Are you familiar with ARM thumb mode firmware reversing? I'm thinking of liberating some WD Firewire external enclosure(s). I've been partially successful, but lacking further skills. If interested, let me know, I'll PM you.

I've never worked with anything ARM, so I'm not immediately familiar. Though I could always take a look and figure it out.

[joevt]

I have another absolutely MASSIVE update for you all... I was able to implement an LZSS decompression algorithm in the ROM, and use it to decompress a stored LZSS-compressed version of the OS9 NDRV. As such, the ROM has been reduced in size to UNDER 128K! This means that you can now flash it onto cards WITHOUT having to replace the original 128k EEPROM! Of course this image also includes my patched OS9 and OS X NDRVs that remove the EEPROM ID check. The updated ROM image is attached, along with my custom Forth LZSS decompression implementation. Hope you all enjoy!

EDIT: Updated ROM padding to 128K for Flashrom support.

Which lzss compression algorithm did you use?

Last time I looked at your ndrv (earlier in this thread), it was 54648 bytes decompressed and 30527 bytes compressed, but the compressed version in this update is 31377 bytes. So you're either using a different compression or a different ndrv. I tried decompressing this using the C algorithm I was using before but that turns the 31377 bytes into 96180 which is way too large. Compressing that produces 31229 bytes - not matching the 31377 input bytes - which means my lzss is different than yours (the lzss can't be successful unless a round trip of decompression and compression produces output that matches the input).

Your decompression algorithm in fcode is 572 bytes. You can shrink that to 429 bytes (143 less bytes) if you use headerless instead of external. This doesn't matter in this case since you've achieved the goal of shrinking the rom to < 128K but it could be useful for other storage device roms or for creating a 64K rom.

[dosdude1]

I have another absolutely MASSIVE update for you all... I was able to implement an LZSS decompression algorithm in the ROM, and use it to decompress a stored LZSS-compressed version of the OS9 NDRV. As such, the ROM has been reduced in size to UNDER 128K! This means that you can now flash it onto cards WITHOUT having to replace the original 128k EEPROM! Of course this image also includes my patched OS9 and OS X NDRVs that remove the EEPROM ID check. The updated ROM image is attached, along with my custom Forth LZSS decompression implementation. Hope you all enjoy!

EDIT: Updated ROM padding to 128K for Flashrom support.

Which lzss compression algorithm did you use?

Last time I looked at your ndrv (earlier in this thread), it was 54648 bytes decompressed and 30527 bytes compressed, but the compressed version in this update is 31377 bytes. So you're either using a different compression or a different ndrv. I tried decompressing this using the C algorithm I was using before but that turns the 31377 bytes into 96180 which is way too large. Compressing that produces 31229 bytes - not matching the 31377 input bytes - which means my lzss is different than yours (the lzss can't be successful unless a round trip of decompression and compression produces output that matches the input).

Your decompression algorithm in fcode is 572 bytes. You can shrink that to 429 bytes (143 less bytes) if you use headerless instead of external. This doesn't matter in this case since you've achieved the goal of shrinking the rom to < 128K but it could be useful for other storage device roms or for creating a 64K rom.

Originally I was planning on using the C lzss implementation you use as it was, however when I was doing initial tests with it, I found that decompressing the NDRV after compressing it with the same implementation resulted in garbage. I fixed this by simply changing the following:

Code: [Select]

#define EI 11
#define EJ  5
#define F ((1 << EJ) + 1)

After making these changes, that implementation began working properly to compress and decompress the NDRV, but resulted in the slightly larger compressed file size. I then wrote the Forth implementation based on the decoding logic of that C implementation.

Also, I intentionally left the named FCode words in there as I thought it would help prevent a potential issue where tokenized FCode numbers may conflict with those already in the ROM (as I wasn't able to detokenize the ROM into a re-tokenizable format). I got around this issue by making a slight modification to the tokenizer source code, starting the FCode number at 0xA00 instead of 0x800, to ensure the generated FCode numbers would not conflict with those already there. So of course I could re-tokenize it headerless, but no need to bother since it already fits into the 128K ROM as it is.

[joevt]

Originally I was planning on using the C lzss implementation you use as it was, however when I was doing initial tests with it, I found that decompressing the NDRV after compressing it with the same implementation resulted in garbage.

Not sure why the decode didn't work for you. Did you try the C decode algorithm or only the fcode decode algorithm? I found that using the default C decode/encode, the decode of the encoded ndrv correctly produced bytes that match the original ndrv:

Code: [Select]

/Volumes/Work/Programming/lzss/lzss/lzss e driver,AAPL,MacOS,PowerPC.bin driver,AAPL,MacOS,PowerPC.lzss
text:  54648 bytes
code:  30527 bytes (55%)

/Volumes/Work/Programming/lzss/lzss/lzss d driver,AAPL,MacOS,PowerPC.lzss driver,AAPL,MacOS,PowerPC_reverse.bin

bbdiff driver,AAPL,MacOS,PowerPC.bin driver,AAPL,MacOS,PowerPC_reverse.bin
/Volumes/Work/Open Firmware and Name Registry/ROM Firmtek 1S2/dosdude/driver,AAPL,MacOS,PowerPC_reverse.bin and /Volumes/Work/Open Firmware and Name Registry/ROM Firmtek 1S2/dosdude/driver,AAPL,MacOS,PowerPC.bin are identical.

md5 driver,AAPL,MacOS,PowerPC.bin driver,AAPL,MacOS,PowerPC_reverse.bin
MD5 (driver,AAPL,MacOS,PowerPC.bin) = dc939ef1b4a4145f552c012be20211d7
MD5 (driver,AAPL,MacOS,PowerPC_reverse.bin) = dc939ef1b4a4145f552c012be20211d7


Also, I intentionally left the named FCode words in there as I thought it would help prevent a potential issue where tokenized FCode numbers may conflict with those already in the ROM

Fcode words get fcode numbers regardless if they are external or headers or headerless.

(as I wasn't able to detokenize the ROM into a re-tokenizable format).

You manually inserted the fcode of the compression algorithm and compressed ndrv into the fcode of the rom and then updated the checksum?

My DumpPCIRom.sh script can create tokenizable Forth text from fcode. (I need to add a small fix for non-PCI ROM fcode files, then I should make a GitHub repository). It's part of the process I used for modifying Nvidia GPU firmwares so they can work in Old World Macs for example.
 https://forums.macrumors.com/threads/question-how-powerful-of-a-graphics-card-will-work-in-a-beige-power-macintosh-g3.2303689/
It was also used nearly 20 years ago in the work for flashing PC Radeon graphics cards (7000, 8500, 9000, 9100) for Mac. Back then it was an MPW script.

I got around this issue by making a slight modification to the tokenizer source code, starting the FCode number at 0xA00 instead of 0x800, to ensure the generated FCode numbers would not conflict with those already there.

My tokenizer has a tokenizer command that you can put into the Forth text that changes the fcode number that will be used for the next word.

Code: [Select]

tokenizer[ a00 next-fcode ]tokenizer
I should fix my DumpPCIRom.sh script to add that line when it encounters a word definition that is not using the expected next fcode number as in the case with your compressed rom. next-fcode isn't usually necessary. It's there so that the tokenization can more closely match the original fcode since the names produced by DumpPCIRom.sh for headerless words includes the fcode number as part of the name like in this example:

Code: [Select]

: colon_definition_function_801         \ (801)     [0b5 0b7]
    my-space                            \           [103]
    9                                   \           [010]
    +                                   \           [01e]
    dup                                 \           [047]
    " config-b@"                        \           [012]
    $call-parent                        \           [209]
    5                                   \           [010]
    or                                  \           [024]
    swap                                \           [049]
    " config-b!"                        \           [012]
    $call-parent                        \           [209]
    ;                                   \           [0c2]


[V.Yakob]

EDIT: Updated ROM padding to 128K for Flashrom support.

Which computer do you use Flashrom on?
I don't have a PC, but I installed it via brew Flashrom on the PowerMac G4 MDD, and it seems that the PPC version does not support "--programmer satasii", despite the fact that it is stated in man.
I also installed Debian on this machine, but the result is the same.

In the near future I will try to upload this firmware through the programmer.

[joevt]

Which computer do you use Flashrom on?
I don't have a PC, but I installed it via brew Flashrom on the PowerMac G4 MDD, and it seems that the PPC version does not support "--programmer satasii", despite the fact that it is stated in man.
I also installed Debian on this machine, but the result is the same.

In the near future I will try to upload this firmware through the programmer.

For macOS, doesn't Flashrom require DirectHW to do PCI stuff or memory mapping stuff? I made a DirectHW for all macOS versions and architectures but I only tested the pci stuff with pciutils (lspci, setpci, pcitree.sh).
It might not have all the stuff required for satasii even for Intel macOS?
According to the flashrom makefile, satasii requires: DEPENDS_ON_RAW_MEM_ACCESS and DEPENDS_ON_LIBPCI
I think DirectHW.kext can do both of those but some other stuff might be required. I have not tried building or using flashrom.

[ssp3]

Can't you just rename the firmware file to ROMFILE.1S2 and replace with it the original that's inside the Mac "SeriTek1S2Flasher.app"?

[dosdude1]

EDIT: Updated ROM padding to 128K for Flashrom support.

Which computer do you use Flashrom on?
I don't have a PC, but I installed it via brew Flashrom on the PowerMac G4 MDD, and it seems that the PPC version does not support "--programmer satasii", despite the fact that it is stated in man.
I also installed Debian on this machine, but the result is the same.

In the near future I will try to upload this firmware through the programmer.

In order to use Flashrom, you must do so using a PC running Linux (though there may be a DOS version you can use as well, not sure). Flashrom under Mac OS does require DirectHW.kext, so I wouldn't bother with that as I don't know if DirectHW.kext can be compiled for PowerPC. I do also have this copy of Flashrom for PowerPC Linux; you could try that and see if that works: http://dosdude1.com/files/flashromppc.zip

[dosdude1]

Can't you just rename the firmware file to ROMFILE.1S2 and replace with it the original that's inside the Mac "SeriTek1S2Flasher.app"?

That application does not allow you to flash a "blank" card, or one that doesn't have one of the three "supported" 512K EEPROMs installed. I did already patch the OS9 version of this tool to be able to flash an unflashed card with any EEPROM, which I've attached here (already bundled with my compressed 128K ROM image), however in my testing I've found that sometimes the tool just locks up depending on the EEPROM you have installed. Definitely at least worth a try, though.

[V.Yakob]

I made a DirectHW for all macOS versions and architectures

Can you tell me how to compile a project to get kext? I only get a unix file.
I have very little experience in xcode, for all the time I collected 2-3 applications that I found on GitHub.

Can't you just rename the firmware file to ROMFILE.1S2 and replace with it the original that's inside the Mac "SeriTek1S2Flasher.app"?

The firmware update utility performs a check, and if the card is not the same, it simply does not display it.
I have a modified board, the utility displays it, but does not display the non-modified card.
I also tried to use the firmware utility from Wiebe -- sees the board, starts the process, but can't interact, the process fails.
Developers have always been cunning and defended themselves from people like us.

[V.Yakob]

I do also have this copy of Flashrom for PowerPC Linux; you could try that and see if that works: http://dosdude1.com/files/flashromppc.zip

I tried it, on 10.4 and Debian -- it doesn't work.
While I'm trying to build DirectHW.kext from the project joevt, but I can't do it yet.

[dosdude1]

Originally I was planning on using the C lzss implementation you use as it was, however when I was doing initial tests with it, I found that decompressing the NDRV after compressing it with the same implementation resulted in garbage.

Not sure why the decode didn't work for you. Did you try the C decode algorithm or only the fcode decode algorithm? I found that using the default C decode/encode, the decode of the encoded ndrv correctly produced bytes that match the original ndrv:

Code: [Select]

/Volumes/Work/Programming/lzss/lzss/lzss e driver,AAPL,MacOS,PowerPC.bin driver,AAPL,MacOS,PowerPC.lzss
text:  54648 bytes
code:  30527 bytes (55%)

/Volumes/Work/Programming/lzss/lzss/lzss d driver,AAPL,MacOS,PowerPC.lzss driver,AAPL,MacOS,PowerPC_reverse.bin

bbdiff driver,AAPL,MacOS,PowerPC.bin driver,AAPL,MacOS,PowerPC_reverse.bin
/Volumes/Work/Open Firmware and Name Registry/ROM Firmtek 1S2/dosdude/driver,AAPL,MacOS,PowerPC_reverse.bin and /Volumes/Work/Open Firmware and Name Registry/ROM Firmtek 1S2/dosdude/driver,AAPL,MacOS,PowerPC.bin are identical.

md5 driver,AAPL,MacOS,PowerPC.bin driver,AAPL,MacOS,PowerPC_reverse.bin
MD5 (driver,AAPL,MacOS,PowerPC.bin) = dc939ef1b4a4145f552c012be20211d7
MD5 (driver,AAPL,MacOS,PowerPC_reverse.bin) = dc939ef1b4a4145f552c012be20211d7


I just tested it again, and it definitely did work, so I must have just done something dumb I didn't realize. Guess I could try again with those parameters set back to their original values, but I don't see a need since it's already under 128K in size.

Also, I intentionally left the named FCode words in there as I thought it would help prevent a potential issue where tokenized FCode numbers may conflict with those already in the ROM

Fcode words get fcode numbers regardless if they are external or headers or headerless.

Yes, I quickly realized that LOL.

(as I wasn't able to detokenize the ROM into a re-tokenizable format).

You manually inserted the fcode of the compression algorithm and compressed ndrv into the fcode of the rom and then updated the checksum?

Correct, along with changing the size in the FCode header and the number of blocks in the PCI header. That was the only thing I could do not being able to re-tokenize the whole thing. I did try your DumpPCIRom.sh script, but it segfaulted when attempting to decode the original ROM when using that perl script, and produced too many untokenizable words when trying to decode it with the NDRVs stripped out (which apparently were what was causing the segfault).

I got around this issue by making a slight modification to the tokenizer source code, starting the FCode number at 0xA00 instead of 0x800, to ensure the generated FCode numbers would not conflict with those already there.

My tokenizer has a tokenizer command that you can put into the Forth text that changes the fcode number that will be used for the next word.

Code: [Select]

tokenizer[ a00 next-fcode ]tokenizer
I should fix my DumpPCIRom.sh script to add that line when it encounters a word definition that is not using the expected next fcode number as in the case with your compressed rom. next-fcode isn't usually necessary. It's there so that the tokenization can more closely match the original fcode since the names produced by DumpPCIRom.sh for headerless words includes the fcode number as part of the name like in this example:

Code: [Select]

: colon_definition_function_801         \ (801)     [0b5 0b7]
    my-space                            \           [103]
    9                                   \           [010]
    +                                   \           [01e]
    dup                                 \           [047]
    " config-b@"                        \           [012]
    $call-parent                        \           [209]
    5                                   \           [010]
    or                                  \           [024]
    swap                                \           [049]
    " config-b!"                        \           [012]
    $call-parent                        \           [209]
    ;                                   \           [0c2]


Ah, I didn't realize that.

[dosdude1]

I do also have this copy of Flashrom for PowerPC Linux; you could try that and see if that works: http://dosdude1.com/files/flashromppc.zip

I tried it, on 10.4 and Debian -- it doesn't work.
While I'm trying to build DirectHW.kext from the project joevt, but I can't do it yet.

Try my patched OS9 copy of the SeriTek updater utility posted above, it just may work. Of course ensure you're booted into Mac OS 9, not using Classic mode. I am still working on patching the OS X version, but it is significantly more difficult to do so.

[ssp3]

While I'm trying to build DirectHW.kext from the project joevt, but I can't do it yet.

Here, try this v1.6 (Intel/PPC). Just built it on Intel 10.6.8, xcode 3.2.6 with 10.4/10.5/10.6 sdk's. Still got two errors on build, but I don't want to dive deeper into it. Current master version produced even more errors, so I skipped it. I'm flying blind here, so it's not tested.

* Why don't you guys provide pre-built binaries on you githubs? Why do we all have to go thru all the pains? 

** BUILD SUCCEEDED **

cc DirectHW.c -dynamiclib -framework IOKit -o libDirectHW.dylib
In file included from DirectHW.c:19:
MacOSMacros.h:54:6: warning: #warning x86_64
MacOSMacros.h:70:6: warning: #warning SDK 10.6+
cc -static -c DirectHW.c -o libDirectHW.a
In file included from DirectHW.c:19:
MacOSMacros.h:54:6: warning: #warning x86_64
MacOSMacros.h:70:6: warning: #warning SDK 10.6+
mv libDirectHW.dylib build/Release/libDirectHW.dylib
mv: rename libDirectHW.dylib to build/Release/libDirectHW.dylib: No such file or directory
make[1]: *** [libs] Error 1
make: *** [directhw] Error 2

[V.Yakob]

Here, try this v1.6. Just built it on Intel 10.6.8, xcode 3.2.6 with 10.4/10.5/10.6 sdk's.

Cool, thanx, I don't have OS X 10.6, I tried building for Intel 13.2 and PPC 10.4.

Try my patched OS9 copy of the SeriTek updater utility posted above, it just may work. Of course ensure you're booted into Mac OS 9, not using Classic mode. I am still working on patching the OS X version, but it is significantly more difficult to do so.

I tried the utility, it works!!
But this board apparently requires additional modifications, because after flashing the PowerMac G4 MDD (2003) won't boot. The boot hangs on the gray screen, the USB does not work (there was a trembling on my back).
Power on -> bong -> a boot disk search icon appears -> grey screen.
If you pull out the board, the computer is working properly.

EPROM Chip -- 39SF010A
Voltage regulator (U1) -- AMS 2911 1.8 0525
Voltage regulator (U2) -- AMS 2911CD 3.3 0323

There are no other boards for testing yet.

[ssp3]

For what it's worth, there's this interesting part by (S)ATAman in Firmtek pirate thread:
http://macos9lives.com/smforum/index.php/topic,5268.msg38303.html#msg38303
Note that he mentions Sil3112DeviceNub.kext

And then there's a "InstallPPC.app" on SeriTek1S2_5.3.1.dmg, which has uncompressed
FT_ATA_Sil3112.kext and  Sil3112DeviceNub.kext
inside and which probably does just what (S)ATAman describes.
I guess, if one replaces the FT_ATA_Sil3112.kext with patched one (uncompressed) the app could be used to assist in flashing the cards.

First, we need to tell the system that there is an ATA device nub.
The simplest:

1) sudo cp -r  /Volumes/SeriTek2SE4Flash/Sil3112DeviceNub.kext /tmp
2) sudo cp -r /tmp/Sil3112DeviceNub.kext /System/Library/Extensions

Loading the kernel extension temporarily:
3) sudo cp -r /Volumes/SeriTek2SE4Flash/FT_ATA_Sil3132E.kext /tmp
4) sudo kextload /tmp/FT_ATA_Sil3132E.kext

After few seconds the Silicon Image 3124 controller is operational.
Do not attach any drives to it, proceed with flashing.

[ssp3]

I tried the utility, it works!!
But this board apparently requires additional modifications, because after flashing the PowerMac G4 MDD (2003) won't boot.

dosdude1, I think it would be cool if you could release the OS9 only stripped down version that does not use any kind of compression. One less variable in testing.

[dosdude1]

Here, try this v1.6. Just built it on Intel 10.6.8, xcode 3.2.6 with 10.4/10.5/10.6 sdk's.

Cool, thanx, I don't have OS X 10.6, I tried building for Intel 13.2 and PPC 10.4.

Try my patched OS9 copy of the SeriTek updater utility posted above, it just may work. Of course ensure you're booted into Mac OS 9, not using Classic mode. I am still working on patching the OS X version, but it is significantly more difficult to do so.

I tried the utility, it works!!
But this board apparently requires additional modifications, because after flashing the PowerMac G4 MDD (2003) won't boot. The boot hangs on the gray screen, the USB does not work (there was a trembling on my back).
Power on -> bong -> grey screen.
If you pull out the board, the computer is working properly.

EPROM Chip -- 39SF010A
Voltage regulator (U1) -- AMS 2911 1.8 0525
Voltage regulator (U2) -- AMS 2911CD 3.3 0323

There are no other boards for testing yet.

The voltage regulators could be causing this issue, apparently that can happen on some QuickSilver and/or MDD systems with some Sil3112 cards. Did you try this card in another machine? It does look like you were able to get it to boot in something, considering System Profiler sees it. Oh, and one other thing. Make sure you do NOT have two cards installed. The drivers in my ROM have been modified to remove the EEPROM ID check, and will conflict with another card if it still has the stock, unmodified ROM.

[dosdude1]

I tried the utility, it works!!
But this board apparently requires additional modifications, because after flashing the PowerMac G4 MDD (2003) won't boot.

dosdude1, I think it would be cool if you could release the OS9 only stripped down version that does not use any kind of compression. One less variable in testing.

I've already thoroughly tested the ROM, that isn't causing the issue. What could be causing an issue, though, is the voltage regulators on the card causing problems, which is a known issue with some cheap Sil3112 cards on later QuickSilver and MDD models. You have to modify one of the voltage regulators on the card to fix this issue. Another potential issue could be the OS9 utility not properly writing the ROM, but that wouldn't cause the system to stop booting.

[V.Yakob]

The voltage regulators could be causing this issue, apparently that can happen on some QuickSilver and/or MDD systems with some Sil3112 cards. Did you try this card in another machine? It does look like you were able to get it to boot in something, considering System Profiler sees it. Oh, and one other thing. Make sure you do NOT have two cards installed. The drivers in my ROM have been modified to remove the EEPROM ID check, and will conflict with another card if it still has the stock, unmodified ROM.

With this board, MDD was launched, the board was defined in OS9, as can be seen on the screen.
I did the flashed when 2 boards were installed: modified and non-modifyed.
Maybe the failure occurred because of this, or there may be a reason in the voltage regulators. In any case, this EPROM is now only to remove, for flashing by the programmer.
Someday later.

[joevt]

Can you tell me how to compile a project to get kext? I only get a unix file.
I have very little experience in xcode, for all the time I collected 2-3 applications that I found on GitHub.

In Xcode 3.2.6 and earlier, there's a menu where you select the target (KEXT, framework, library) and the type of build (Release, Debug).

Here, try this v1.6 (Intel/PPC). Just built it on Intel 10.6.8, xcode 3.2.6 with 10.4/10.5/10.6 sdk's. Still got two errors on build, but I don't want to dive deeper into it. Current master version produced even more errors, so I skipped it. I'm flying blind here, so it's not tested.

* Why don't you guys provide pre-built binaries on you githubs? Why do we all have to go thru all the pains? 

** BUILD SUCCEEDED **

cc DirectHW.c -dynamiclib -framework IOKit -o libDirectHW.dylib
In file included from DirectHW.c:19:
MacOSMacros.h:54:6: warning: #warning x86_64
MacOSMacros.h:70:6: warning: #warning SDK 10.6+
cc -static -c DirectHW.c -o libDirectHW.a
In file included from DirectHW.c:19:
MacOSMacros.h:54:6: warning: #warning x86_64
MacOSMacros.h:70:6: warning: #warning SDK 10.6+
mv libDirectHW.dylib build/Release/libDirectHW.dylib
mv: rename libDirectHW.dylib to build/Release/libDirectHW.dylib: No such file or directory
make[1]: *** [libs] Error 1
make: *** [directhw] Error 2

I didn't update the makefile since I only needed the kext which I build in Xcode instead of using make.
The xcodebuild command probably should be updated to specify the 10.6 project since the DirectHW folder contains two Xcode projects.
I think you need to change build to build/build10.6

The makefile installs the kext to /Library/Extensions/DirectHW.kext but that's not necessary for old versions of macOS. In old versions of macOS (10.8 and earlier) or new versions of macOS with SIP disabled (10.11 - 10.15) You can load it manually as long as it has proper file permissions. macOS 11 - 13 requires installing to /Library/Extensions and allowing the kext in Security preferences panel.

The makefile installs the framework, dylib, library, and object files to default locations on your system so that apps that don't compile or link the DirectHW.c file directly into their binary can use the code. DirectHW on PPC probably won't do anything for flashrom until you make changes and recompile flashrom.

Code: [Select]

pciutils testing:

                        arch    darwin  darwin2 darwin3 intel-conf1     How to load unsigned DirectHW.kext
10.4.11 Tiger           p       -       -       p       -               
10.5.8  Leopard         pP      -       -       pP      -               
10.4.11 Tiger             i     i       -       p i     pi               
10.5.8  Leopard           ix    ix      -       p ix    pix             
10.6.8  Snow Leopard       x    ix      -       p ix    pix             
10.7.5  Lion               x    ix      -         ix     ix             
10.8.5  Mountain Lion      x    ix      -         ix     ix             
10.9.5  Mavericks          x    ix      -         ix     ix             install to /System/Library/Extensions.
10.10.5 Yosemite           x    ix      ix        ix     ix             add "kext-dev-mode=1" to boot-args.
10.11.6 El Capitan         x    -       -         ix     ix             disable SIP.
10.12.6 Sierra             x    -       panic     ix     ix             disable SIP.
10.13.6 High Sierra        x    ix      ix        ix     ix             disable SIP.
10.14.6 Mojave             x    ix      ix        ix     ix             disable SIP.
10.15.7 Catalina           x     x       x         x      x             disable SIP.
11.7.2  Big Sur            x     x       x         x      x             disable SIP. Install to /Library/Extensions. Allow in "Security & Privacy" preferences panel. Restart.
12.6.2  Monterey           x     x       x         x      x             disable SIP. Install to /Library/Extensions. Allow in "Security & Privacy" preferences panel. Restart.
13.1    Ventura            x     x       x         x      x             disable SIP. Install to /Library/Extensions. Allow in "Security & Privacy" preferences panel. Restart.

                kernel          user
p = ppc         10.1            10.1
P = ppc64       -               10.5 SDK (only works on PPC Mac)
i = i386        10.4            10.4
x = x86_64      10.6            10.6
a = arm64       11              11

- for 10.1 to 10.3 : use GCC 3.3 and the oldest SDK.
- for 10.4 Xcode 2.2.x or 2.3, use GCC 4.0 and 10.4 SDK (not 10.4u).
- for 10.4 Xcode 2.4 or later, define KPI_10_4_0_PPC_COMPAT for PPC architecture, use 10.4u SDK for both PowerPC and Intel.
- for ppc64 user, 10.5 SDK has 64 bit libraries.

# darwin doesn't work from ppc userspace because AppleACPIPlatformExpert user client doesn't support cross endian.
# darwin doesn't work with ppc kernel since ppc doesn't have ACPI and doesn't use AppleACPIPlatformExpert.
# intel-conf1 method only supports registers 00-FF.
# both darwin3 and intel-conf1 require DirectHW.kext.
# darwin3 requires joevt/directhw.
# ppc userspace requires joevt/directhw.
# darwin and darwin2 require boot-args to contain "debug=".
# all require administrator rights.
# darwin2 requires OS X 10.10 or later.
# darwin and darwin2 don't seem to work in OS X 10.11; instead, use Darwin3 or intel-conf1 with DirectHW.kext.


Correct, along with changing the size in the FCode header and the number of blocks in the PCI header. That was the only thing I could do not being able to re-tokenize the whole thing. I did try your DumpPCIRom.sh script, but it segfaulted when attempting to decode the original ROM when using that perl script, and produced too many untokenizable words when trying to decode it with the NDRVs stripped out (which apparently were what was causing the segfault).

Maybe you need updated versions. What version of macOS were you using? I use it in Monterey currently. The script works with my version of detok.

[dosdude1]

Correct, along with changing the size in the FCode header and the number of blocks in the PCI header. That was the only thing I could do not being able to re-tokenize the whole thing. I did try your DumpPCIRom.sh script, but it segfaulted when attempting to decode the original ROM when using that perl script, and produced too many untokenizable words when trying to decode it with the NDRVs stripped out (which apparently were what was causing the segfault).

Maybe you need updated versions. What version of macOS were you using? I use it in Monterey currently. The script works with my version of detok.

Ah, that's probably why, then. I'm using OS X 10.9 Mavericks, with an unmodified version of Detok.

[ssp3]

I didn't update the makefile since I only needed the kext which I build in Xcode instead of using make.
The xcodebuild command probably should be updated to specify the 10.6 project since the DirectHW folder contains two Xcode projects. I think you need to change build to build/build10.6

First thing I did is yanked that other project and, yes, I used make. I did not use xcodebuild. For the kicks I tried to build it now directly in xcode and look what I got.
I overlooked those warnings in terminal before, they were probably there.

[V.Yakob]

That application does not allow you to flash a "blank" card, or one that doesn't have one of the three "supported" 512K EEPROMs installed. I did already patch the OS9 version of this tool to be able to flash an unflashed card with any EEPROM, which I've attached here (already bundled with my compressed 128K ROM image), however in my testing I've found that sometimes the tool just locks up depending on the EEPROM you have installed. Definitely at least worth a try, though.

But this board apparently requires additional modifications, because after flashing the PowerMac G4 MDD (2003) won't boot. The boot hangs on the gray screen, the USB does not work (there was a trembling on my back).

Another board, MDD booted with it, after flashing: Power on -> Bong -> the boot disk search icon appears -> gray screen. USB doesn't work, so I can't go to OpenFirware and see how the card is detected.
2 MDDs have the same behavior.
This time OS9 was booted from PATA HDD, the flashing was also performed without visible problems.
In general, for that "red" board, I will order EPROM chip 39SF040A, and flash the uncompressed firmware version, see what happens.
And with this "black" board (AM29F010B) I will do the same as last time: replacing the EPROM chip with PM39LV040 and transferring 0 ohms of the resistor.

Probably in a month.

P.S. It's good that I bought a programmer last month.

[dosdude1]

That application does not allow you to flash a "blank" card, or one that doesn't have one of the three "supported" 512K EEPROMs installed. I did already patch the OS9 version of this tool to be able to flash an unflashed card with any EEPROM, which I've attached here (already bundled with my compressed 128K ROM image), however in my testing I've found that sometimes the tool just locks up depending on the EEPROM you have installed. Definitely at least worth a try, though.

But this board apparently requires additional modifications, because after flashing the PowerMac G4 MDD (2003) won't boot. The boot hangs on the gray screen, the USB does not work (there was a trembling on my back).

Another board, MDD booted with it, after flashing: Power on -> Bong -> the boot disk search icon appears -> gray screen. USB doesn't work, so I can't go to OpenFirware and see how the card is detected.
2 MDDs have the same behavior.
This time OS9 was booted from PATA HDD, the flashing was also performed without visible problems.
In general, for that "red" board, I will order EPROM chip 39SF040A, and flash the uncompressed firmware version, see what happens.
And with this "black" board (AM29F010B) I will do the same as last time: replacing the EPROM chip with PM39LV040 and transferring 0 ohms of the resistor.

Probably in a month.

P.S. It's good that I bought a programmer last month.

It is the decompression routine that's causing the issue, I've been able to verify that on my own MDD. Unfortunately so far I've been unable to figure out exactly WHY it's causing this issue, so I'm working on that now still. You should be able to recover the card without soldering by using Flashrom on a PC, or using an older Mac, which does not exhibit this issue.

[V.Yakob]

You should be able to recover the card without soldering by using Flashrom on a PC, or using an older Mac, which does not exhibit this issue.

I don't have a PC. I don't want to buy an old system unit to restore the board.
There is also Quicksilver, but it is not yet possible to restore the power supply.
Only hardcore, only solder. 🤪

[dosdude1]

I FINALLY got it figured out, well, sort of... I couldn't figure out what in my implementation was causing the issue, so I decided to make a new implementation, based off a different C implementation along with a (broken) Forth implementation I came across. With this decompression routine implemented, the USB probing issue stopped happening on my MDD! So now, finally, we have a fully working ROM that fits onto a 128K EEPROM! I have attached this new ROM below, along with my patched copy of the OS 9 SeriTek flasher tool with said ROM embedded, as well as the Forth source of the new decompression implementation.

[ivanshpak]

I FINALLY got it figured out, well, sort of... I couldn't figure out what in my implementation was causing the issue, so I decided to make a new implementation, based off a different C implementation along with a (broken) Forth implementation I came across. With this decompression routine implemented, the USB probing issue stopped happening on my MDD! So now, finally, we have a fully working ROM that fits onto a 128K EEPROM! I have attached this new ROM below, along with my patched copy of the OS 9 SeriTek flasher tool with said ROM embedded, as well as the Forth source of the new decompression implementation.

Red PCB card Silicon Image via MDD 1.25 (FW 400) (5 Slot)
Prior Wiebe firmware
Boot into 9.2.2 (Maсtron image), with the SSD SanDisk Ultra 2 120GB

dosdude1 You are a god, you have made a huge contribution to our community! Where to donate!?

[AtariMan]

I've already flashed four boards.
All work in Sawtooth and MDD.
I can't believe this miracle!
Thank you very much for your work!!!

[FBz]

BIG Congratulations!!!
And mucho kudos to dosdude1 (and everyone else in this effort).

Now perhaps someone can / will translate this into a step-by-step, D-I-Y for the rest of us?
You know… the “recipe” for what is necessary - with easy to follow instructions.

AND while at it, how about some QuickBench and XBench results too?
(In various machines, booting OS 9 AND OS X.) Begin-new-thread?.

And quickly too, before the rest of the world finds out and these now
inexpensive cards exponentially rise in price. (aka: The Syntho 9600 Syndrome.)
*See also… the “G4 Mac Mini Madness”.

C’mon with it.

[ssp3]

And quickly too, before the rest of the world finds out and these now
inexpensive cards exponentially rise in price. (aka: The Syntho 9600 Syndrome.)
*See also… the “G4 Mac Mini Madness”.

Then don't hype it, don't run around bragging about it and don't make any clickbate Youtube videos.
Delete all praise posts that use red or oversized text 

Lay low! 

dosdude1 isn't making money with this stuff and we all know that he's good, right? Lets not make life easier for fleebay sharks and similar "businessman".
Mac users, who really need their cards flashed, sooner or later will find this thread.
Just my 5 cents.

[FBz]

And quickly too, before the rest of the world finds out and these now
inexpensive cards exponentially rise in price. (aka: The Syntho 9600 Syndrome.)
*See also… the “G4 Mac Mini Madness”.

Then don't hype it, don't run around bragging about it and don't make any clickbate Youtube videos.
Delete all praise posts that use red or oversized text 

Lay low! 

dosdude1 isn't making money with this stuff and we all know that he's good, right? Lets not make life easier for fleebay sharks and similar "businessman".
Mac users, who really need their cards flashed, sooner or later will find this thread.
Just my 5 cents.

Well, sooner or later the step-by-step process will out.
Whether it be here, 68kmla, MacRumors or wherever.
And while I completely understand the 5-cent viewpoint.
The D-I-Y will eventually be posted somewhere.

And not at all about bragging rights.
More like sharing with those that have been confronted
with the overpriced SeriTek (and other) card options for so very long.

 So here, or just from some “enterprising”  Fleabay offering first?
*Available public domain Forum "how-to" info - or opportunistic profiteer?

[IIO]

BIG Congratulations!!!

Now perhaps someone can / will translate this into a step-by-step, D-I-Y for the rest of us?
You know… the “recipe” for what is necessary - with easy to follow instructions.

the "which card(s) to buy" (for noobs) question from that other forum driver from last year is also still open.

which uh, reminds me on a certain project #someone wanted to start in november 22.

damn, i just killed the content of my boot drive on the main OS9 machine. lots of extra work in the next days.

[AtariMan]

Has anyone tried to flash Silicon Image SIL3114?
dosdude1 is this possible with your firmware?

[dosdude1]

Has anyone tried to flash Silicon Image SIL3114?
dosdude1 is this possible with your firmware?

I don't believe so, as that firmware is specifically designed for the Sil3112. You can definitely try it, but you'll probably have to flash it via Flashrom or using an EEPROM programmer, as the SeriTek/1S2 flashing tool probably won't detect it. Also, just note that the firmware I have posted is essentially unchanged from the stock SeriTek/1S2 firmware, the only differences are it has had the EEPROM ID checks patched out, and the OS 9 NDRV is compressed, allowing it to fit on a 128K EEPROM.

[AtariMan]

Thank you for the detailed answer,
I'll try to flash it on a PC using Flashrom in the evening.

[AtariMan]

Silicon Image SIL3114 does not work with SeriTek / 1S2 firmware,
but works without problems with WiebeSATA 3112 firmware.
Two out of four ports work.

[AtariMan]

Hi dosdude1.
I found the firmware for SIL3114,
maybe you will be interested to see and try to flash the card.
Perhaps all 4 ports will work.

https://www.siig.com/download/search?keyword=SC-SA4M12

[eastone]

There's also another firmware for sil3512 but there is a  problem how to extract it from the archive...
https://www.siig.com/pub/media/files/drivers/0005/siig3512101updriver1.sit

[redstudio]

let me understand well. does the flasher linked by dosdude1 make pci sata 3112 work on powermac? also os9? no modifications to the board are needed? this would be great! let me know, thanks

[dosdude1]

let me understand well. does the flasher linked by dosdude1 make pci sata 3112 work on powermac? also os9? no modifications to the board are needed? this would be great! let me know, thanks

That is exactly correct, though do be aware that the flashing tool may not work with the specific EEPROM on your card. In my testing, it worked with most EEPROMs, but not all. If it doesn't work, you can always flash using Flashrom on a PC. Still no card modification/soldering required. Once flashed, card will work and be bootable in both OS X and OS 9 (actually as old as System 7.5).

[redstudio]

unfortunately I don't have a card to test immediately.. I'll get a couple as soon as possible to try them.. if everything works you did a really great job Dosdude1.. you finally made it possible to pay little sata controller as PC users.. thanks a lot for your work!

[chrisNova777]

glad someone picked up the ball and ran with it    i thought SATAMAN was gonna sort this couple years back
i obsessed about this pretty hard back in mid 2010s it was frustating knowing it was possible... technical difficulties always get in the way, its great when the walls that block progress come crashing down, this is great news for alot of reasons

"you get a 1S2, and you get a 1S2!, and you get a 1S2!" -- oprah

just when i found 2 sonnets after a year of search rofl

lol