Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[UnexpectedBomb]

It's still very experimental, but at long last... myth has become reality.

Introducing the first-of-its-kind USB 2.0 mass-storage driver for Mac OS 9. It's rough around the edges and I'm putting it out as an early beta, but it works... and it's FAST.

As many of you know, OS 9 only ever supported USB 1.1, operating at ~1 MB/s in practice. This experimental app mounts a USB 2.0 (or 3.0!) flash drive on a desktop and reads it at ~20 MB/s and writes at ~13 MB/s on my MDD via USB 2.0 PCI card, achieving speeds roughly 10-20x faster than anything OS 9 was doing before. Files copy in the Finder like any other disk.

All that said, I must stress that this is an early beta; a proof that hi-speed USB on OS 9 is possible. It is NOT a polished plug-and-play driver (yet). There's no extension to install. The whole stack (a built-from-scratch EHCI host-controller driver, a software-synthesized root hub, and a block driver) lives inside one small launcher app. Because USB 2.0 PCI cards share their physical ports with the old USB 1.1 "companion" controllers, when you plug in the drive makes all the difference, so it works through a specific sequence:

1) Boot with the drive UNPLUGGED from the PCI card
2) Run the launcher app. It will claim the card's ports for the 2.0 EHCI controller
3) When it says "INSERT USB DRIVE NOW," it's time to plug that sucker in
4) It mounts at 2.0 speed, and the drive icon will show a little "2.0" text badge on it, so you can tell it apart from a plain 1.1 mount at a glance. (Also it will generate some harmless dev logs.)

You MUST leave the app running because it hosts the stack. Eject the drive in the Finder before unplugging, and reboot to mount another drive.

My exact test rig: G4 MDD (non-FW800), an NEC-chipset IOGEAR USB 2.0 PCI card, and a SanDisk Ultra USB 3.0 64GB flash drive. That's the only combination I've been able to test so far, and I really need your help here; when you test this app, please make sure to report back with specific model information the following:
- the model Mac you used
- the model USB PCI card you used
- the type of USB flash drive you used
- the read/write speeds you achieved

Features that DO NOT work yet (and I'd genuinely appreciate some assistance with):
- auto-mounting an attached drive at boot
- hot-plugging (swapping drives without a reboot)

Mac OS 9's USB stack does not transfer a mass-storage device nicely from the USB 1.1 companion controller over to EHCI; a drive present at boot gets claimed by 1.1 before my driver has a chance to load (when placed in Startup Items). See the technical documentation for an explanation as to why my efforts thus far to prevent this have failed. If anyone here knows the classic USB stack intervals well enough to explain how to a companion controller correctly release a port to EHCI, or why ExpertIdleTask won't return after a mass-storage re-probe of a device that won't mount, I'd be very grateful for the advice!

Thanks in advance for your patience with the hot-off-the-press code. This stack took MANY hours of dev work and I'm hopeful we have something genuinely useful here to build from. Bug reports and hardware compatibility reports are welcome. I'm hoping to eventually get this working with late-model G4's that have onboard USB 2.0 (G4 Mac Mini anyone?) as well as USB 2.0 CardBus adapters for PowerBooks. In the meantime, I'll continue working on the hot-plugging feature. Let me know how it all goes, and I look forward to the collaboration.

-UB

https://github.com/UnexpectedBomb/MacOS9-USB2-EHCI

[joevt]

You mention mass-storage. What about other types of USB devices? HID? Video? Audio?

Creating a USB 2.0 driver (UIM - USB Interface Module) was discussed at
 http://macos9lives.com/smforum/index.php?topic=7797.msg60153

A UIM ndrv is a shared library. Maybe it's possible to create a UIM shim that imports from the original UIM to intercept all the UIM calls? That might be useful for something. Logging or modification? This may require a replacement mechanism in the ndrv (I know that exists for graphics ndrv using kReplaceCommand in DoDriverIO).

What are the obstacles of making a UIM that lives in the system heap instead of an application process as you've done?

EHCI is mentioned in Open Firmware 4.6.7b15 and later, but there's no fcode image for PCI EHCI controllers (PCI class code 0C:03:20). There are fcode images for PCI OHCI controllers (PCI class code 0C:03:10) since Open Firmware 3.0.0.

Would it be possible to create a UIM for XHCI controllers? There exist PCI to PCIe bridges that can allow connecting a USB 3.x card. USB 3.x is full-duplex but PCI is not. I don't think that should matter.

[UnexpectedBomb]

Thanks for the great questions @joevt, I'll try to tackle them:

Other device classes (HID / video / audio): Mass-storage only for now. Haven't messed with any other types of USB devices, as flash drive functionality is my top priority. If I can get flash drives working reliably first, I'd then look into supporting other devices in future builds.

System heap vs. application process: I did attempt the system heap solution first, and the biggest problem here is task context. When the driver runs headless, its callback arrives nested inside the Finder's SystemTask, and pumping from there won't advance the Expert's hub enumeration, so the device never finishes resetting and mounting. Driven from a normal top-level process loop, the same UIM mounts fine. I tried two headless routes and both failed: a 68K WaitNextEvent trap patch (never fires, since the PPC Finder calls WaitNextEvent through InterfaceLib), and a Notification Manager re-arm that starves the live controller. So the app just supplies that top-level context; the code itself could live elsewhere. A UIM in the system heap would definitely be preferred eventually!

kReplaceCommand:
This is indeed a DoDriverIO mechanism for graphics ndrvs, but USB UIMs are Expert plugins with a numbered dispatch table, so interception would be a shared-library interpose or a dispatch-table hook rather than kReplaceCommand.

OF and EHCI fcode: Matches exactly what I see. OF creates the pciclass,0c0320 node (that's how I find the card and map its registers through the Name Registry), but there's no fcode driver, so I bring the controller up myself once the OS is running.

xHCI via a PCIe-to-PCI bridge: Should be technically possible. Agreed that full-duplex is a non-issue because across the bridge it's just PCI transactions, but PCI bandwidth (~133 MB/s at 32-bit/33 MHz) would cap everything. Would it be faster than 2.0? Yes, technically speaking, but nowhere near the expected USB 3 speeds. The xHCI driver itself would be a major effort, much bigger than EHCI. Ultimately, the minor gain in PCI-capped throughput over EHCI probably wouldn't be worth the trouble. The only thing that would maybe change this outlook for me is if we somehow got OS 9 booting natively on the Power Mac G5... probably a pipe dream. For now, USB 2.0 is still a game-changer.

Appreciate you taking the time to get into the weeds on this. If we ever achieve a mature, stable end-product here, it will be a the result of a lot of teamwork.

[joevt]

PCI speeds: See list of PCI speeds at https://forums.macrumors.com/threads/question-how-powerful-of-a-graphics-card-will-work-in-a-beige-power-macintosh-g3.2303689/

Blue & White G3 has a 32-bit 66 MHz slot (it's an Apple Grackle hack - it doesn't support 33 MHz like a normal 66 MHz slot would). I can use this with the StarTech PCI to PCIe bridge.

Macs since the Blue & White G3 have 64-bit 33 MHz slots but it's difficult to find a 64-bit PCI to PCIe bridge.

AGP to PCIe bridges that work beyond 66 MHz are even more difficult to get (if they ever existed outside of some graphics cards).

Having a PCI slot that is slower than the USB speed has been a problem even with Intel Macs.
- A Mac Pro 2008 has PCIe slots that don't support x2 link width. A USB 3.x card with a PCIe gen 2 x2 ASM1142 controller would run as PCIe gen 1 x1 (250 MB/s).
- A PCIe gen 2 x2 USB controller has a 1000 MB/s PCIe cap. A PCIe gen 3 x1 USB controller has a 984.6 MB/s PCIe cap. But USB 3.1 gen 2 is 1212 MB/s.

Other device classes: I thought maybe they would get supported for free since the UIM is at a level lower than the USB interfaces? I haven't looked too deeply at UIM code though.

What about USB 2.0 hubs? I think I have a USB 2.0 keyboard with USB ports somewhere.

[UnexpectedBomb]

Good point on the PCI speeds, I stand corrected. The B&W's 266 MB/s PCI speeds would certainly be significantly faster than USB 2.0. In theory, an xHCI controller behind a bridge could run several times faster than what EHCI is currently giving me. The problem still comes back to the xHCI driver itself, which would be brutal. For now, I think EHCI has a much better payoff for the effort when you consider all the G4's that have on-board USB 2.0 (Mini, iBook, aluminum PowerBook, late iMac and eMac) that would also benefit from a reliable OS 9 driver.

As far as additional device classes for free, that is the appeal of doing this at the UIM level, since the UIM sits below the interfaces. The one concern right now is that my UIM currently only runs the async schedule (control and bulk). Anything that requires interrupt transfers (HID) or isochronous (audio, video) needs me to implement the periodic schedule in the UIM first. Once that is done, those device classes will (hopefully) work. Mass storage is the tough one since I have to do a self-probe takeover to deal with a flaky composite driver.

Haven't tried any USB 2.0 hubs yet, although I think the USB ports in the back of my 20" aluminum Cinema Display would qualify. I'll report back once I try it. If you're able to successfully mount a drive and transfer data using this app, let me know your results!

[Windoze]

I also had a look at implementing a EHCI UIM 1 or 2 years ago.
After some reversing of the OS 9 USB stack I deemed it infeasable.

Problem is, that USB 2 changes more than just faster speed.
USB 2 allows bigger packet sizes, USB frames are split into microframes, there is the companion controller situation (EHCI can ONLY handle USB 2, for older devices it falls back to the OHCI controller. This gets even more complicated if you have an USB 1 device on a USB 2 Hub), etc.

All of this is not implemented in the rest of the OS 9 USB stack. You would have to replace almost all parts of it...
For example: You can look at the source of the early OS X USB stack to see that the OHCI UIM has to communicate with the EHCI UIM to coordinate the handoff. But the OS 9 version doesn't have that mechanism.

I think having a separate stack for fast mass storage is the only way this will ever work.

[UnexpectedBomb]

Hi @Windoze, I can see that you too have investigated this quite thoroughly. Indeed, the OS 9 stack has no understanding of high-speed, microframes, or the companion handoff, and certainly it has nothing like the OS X EHCI/OHCI coordination. A full USB 2.0 stack really would mean replacing most of it.

For now, my scope with this particular effort is different. A separate stack for mass storage is exactly what I built (about 20 MB/s on an MDD, which is blazing compared to USB 1.1), but I didn't have to replace almost all of the OS 9 stack to get there. I register as a UIM under the Expert, let Apple's hub driver handle enumeration, and take over only the bulk endpoints for the data path.

Regarding the companion handoff, of which I built a static version: on the G4 Mini's on-board controller (still working through this part), I claim the drive port and hand the keyboard and mouse ports back to the 1.1 companion via the port-owner bits. It's not as robust as OS X's dynamic coordination, but it is enough for one mass-storage device. I haven't yet messed with the whole "1.1-behind-2.0-hub" situation, which may be a tough nut to crack.

All that said... I think we're further along today than we've ever been in the Mac OS 9 USB 2.0 saga.

[Daniel]

For getting OHCI and EHCI to work together, you can just replace the UIM used by OS 9. There will be two copies of it, one in the USB extension(s) and one in the Mac OS ROM file. You can use https://github.com/elliotnunn/tbxi to edit the copy in the Mac OS ROM file.

[Jubadub]

Before anything else, thank you @UnexpectedBomb for implementing this! Indeed, it's been for decades one of those pain points for us Mac OS users to have USB 2.0 hardware, yet we could ever only operate it at USB 1.1 speeds at highest.

Thank you again for fixing that in the capacity you have!

Now, of course, like you and all others here, I do hope we somehow can figure out how to improve upon this existing implementation to address its current limitations. I'd also like to second @Daniel's suggestion to check out @ELN's tbxi project, since it makes it viable to edit and/or patch the Mac OS ROM file, inside of which USB drivers also reside (plus a lot else, like the 68040LC emulator, and even the Mac OS nanokernel). I believe this will be 100% required if we want to, say, boot Mac OS 9 from USB with USB 2.0 speeds. That'd be a big bonus!

Currently (and for the foreseeable future), I have no physical access to my various MDDs, being "forced" to live with "just" a Mac mini G4 1.5 GHz "silent upgrade" model, and a 15" PowerBook G4 DLSD (the only non-G5 PowerMac NOT to be able to boot Mac OS 9 so far, alongside its 17" sibling). So all I can test this on is the mini, which apparently is still a work-in-progress and won't yet work. But once that is working, I'll gladly line up for testing this!

Thank you again for taking this dive, it might just give us the momentum we need.

[joevt]

Does MacOS9-USB2-EHCI work correctly with multiple USB controllers? My B&W G3 has a USB 1.1 controller and a USB 2.0 controller. See attached dump-device-tree output.

The built-in USB is at:
/pci@80000000/pci-bridge@d/usb@6

The USB 2.0 controller should match with a USB 1.1 controller with the same device path but different function number, at least in this example:
/pci@80000000/pci-bridge@d/usb@2
/pci@80000000/pci-bridge@d/pci17fc,1688@2,3

When I tried connecting a USB device, the EHCILauncher app hanged.

The Retro68 console does not function as well as a CodeWarrior SIOUX console. The Retro68 console is missing window move and resize, menus, update handling, text selection, copy/paste.

[UnexpectedBomb]

Thanks @joevt for giving this a try! Including that device tree is super helpful.

Two things stand out. First, your USB 2.0 controller is an ALi/ULi chip (pci10b9,5239, the M5239 EHCI with its M5237 OHCI companion). I have only tested this driver against an NEC EHCI on a PCI card and the Intrepid on-board EHCI. ALi is untested, so a chipset-level incompatibility during EHCI bring-up is a real possibility.

Second, the interrupt is shared, but not the way it is on the on-board machines. Your EHCI (device 2) shares its INT line with the ALi OHCI companion on the same device, so my driver takes its "shared companion" path, which is the one that can have problems. But your keyboard and mouse are on a separate controller (the OPTi usb@6, on a different line), so that shared companion is idle. On a Mac Mini the shared companion is the keyboard/mouse controller, so your case is different.

So it is one of two things and I can't tell which without the log: an ALi-specific bring-up problem, or my shared-interrupt path locking up (the same class as the on-board bug). The EHCIUIM_init.log written next to the app would settle it, even a partial one after a reset. I am looking for whether it reaches the "SELFPROBE READ CAPACITY" line (points to the shared-IRQ path) or dies earlier during controller bring-up (points to the ALi chipset), and whether it logs "sharedCompanion 1".

Did you boot with the drive unplugged and insert it only at the "INSERT USB DRIVE NOW" prompt? Thanks again, this is helpful data.

[Jubadub]

I almost forgot, but I heard (I think from House of Moth's blog?) that the MDD, at least the relaunch 2003 model (if not all of them), actually shipped with USB 2.0 hardware/controllers onboard, but they somehow only run at 1.1 speeds by default, like even on OS X? Something that might be worth looking into. It's been half a decade or more since I heard of this.