I think it would be more interesting doing something similar to Classic but with Linux and a nanokernel shim (but at the same time MacOnLinux is there already)
Classic have had a lot of performance and compatibility related issues. I remember not being able to use DAW software under Classic at all. I doubt we could do it better than Apple...
I would claim that the performance issues related to Classic was more to do with the fact Classic was a second class citizen in the Mac OS X installation- if OS X prioritized the Classic process it probably wouldn't been that bad. (as well as the rootless mode, window doublebuffering hacks etc that basically only made it usable for word processors and a like applications that basically only used Toolbox (Quickdraw/Quicktime etc especially) calls for everything, a lot of applications on OS 9 however had their own implementations of blitting routines etc and those broke on Classic because of the stupid double buffering hack Apple used- which of course was necessary to allow windows to overlap without artifacts when run in a rootless window.
As for audio etc yes, the lack of hardware access of course caused issues there- but that's something that could have been solved if Apple wanted to (QEMU, VMWare etc supported it for ages) PCI passthrough etc is not a technical impossibility even without an IOMMU on the PCI buss, it's just not secure- but anyone who cares about security wouldn't run Mac OS 9 to start with.
Compatibility would be tricky regardless of solution, especially on everything but the first G5 (PCI-X; PCI-X can run in PCI compatibility mode as far as I know- so I think it's theoretically plausible to have it work with OS 9 with some efforts... but then again I never really looked into the specification at detail so it's mostly a guess), the PCIe on the other hand that the majority of the G5s used are completely different beast however and even if someone magically would manage to get PCIe working, it's not compatible with PCI so no PCI card drivers etc would work (which means no ASIO or similar).
While the alternative would be to leverage the device layer support on another kernel such as Linux would give a lot of this "for free", In fact MacOnLinux already has higher compatibility than Classic ever had- unfortunately MacOnLinux was never ported to the G5 (and I believe the original implementation is extremely dated so it's unlikely it would even work with recent linux kernels), I know there was efforts porting MacOnLinux to use KVM instead of it's own hypervisor implementation- but I guess that essentially would leave us up to the point we are right now- with QEMU and KVM, ie. The bootstrap/nanokernel is unable to boot due to supervisor differences on the G5.
Anyway it's just my two cents, getting the kernel started and Mac OS to boot at all is of course beneficial in all scenarios, regardless of what path one may take after that.