Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers

[IIO]

Because the Mac OS usage is calculated by the total ram - what the process manager has allocated to it.

allocated as the max, as long as not some other apps are asking for this RAM to use, then it is the min.

[DieHard]

This also gets around basic limitations in the Mac OS, like the 1.5 gig limit. A 500 meg Ephboot disk should let a 2 gig machine use all of its memory.

As we know, the apple RAM disk memory used pads itself to the System memory used and is thus, worthless in our quest for the missing .5 GB, but if the quote above is real that would be amazing... this means the wheel we are engineering has already been built.  We need to get this app. and test it NOW !

[nanopico]

Because the Mac OS usage is calculated by the total ram - what the process manager has allocated to it.

allocated as the max, as long as not some other apps are asking for this RAM to use, then it is the min.

I'm not sure I follow you, but The Mac OS Usage I am referring to is the amount of ram allocated to Mac OS when you view About This Computer.  It's not calculated by the actual usage.

This also gets around basic limitations in the Mac OS, like the 1.5 gig limit. A 500 meg Ephboot disk should let a 2 gig machine use all of its memory.

As we know, the apple RAM disk memory used pads itself to the System memory used and is thus, worthless in our quest for the missing .5 GB, but if the quote above is real that would be amazing... this means the wheel we are engineering has already been built.  We need to get this app. and test it NOW !

From what i have seen with this, it id ram disk that it mapped to look like a physical hard drive and the ram used by it is not visible to the OS.  The ram disk has to be used to boot the system.  So you in theory would get be able to use all the memory, but at least 512 MB would have to be a ram disk that the system boots from.   It's a rather neat system and I'm trying to wrap my head around it's workings to see what we can learn from it, but it doesn't seem to patch the system memory manager at all so we are still limited to 1.5 GB max allocation once booted.  But once booted the system will still load it's self now within that 1.5 GB and not outside it like it currently does.  So actually in theory, going this route can cause the usable memory to be less than not using this tool/utility/application.

[nanopico]

Knowledge attained regarding getting to 2GB.
Just a small update.
So I found some bit's poking around and reading documentation.
So what we know is that the process manager get's allocated 1.5GB of ram and the rest is the system.  This is calculated by the size of the actual memory minus that which is allocated to the process manager.
The process manager is installed at the top of RAM and it's heap grows down, just like every other application.
The system is placed at the bottom of RAM and it's heap grows up.
The reason for this is so that the system doesn't hit a hard limit if it needs to grow. So if it didn't get enough allocated early on and it grows it hits the bottom of RAM and that probably isn't good.
If the system heap needs to grow it takes it from the process manager by shrinking  it and then expanding the system heap.  When the system heap can shrink it does so and then grows the process manager's heap.  Generally you won't ever actually this though.  The system heap really only grows during boot while it's loading extensions.
What I didn't realize is that there is a dummy heap between the two of them. So at the end of the process manager heap is a heap header and then a small empty block.  This dummy heap is supposedly what's moved around to shift the memory between the two heaps.

Supposedly the dummy heap is there to hide some of the aspects of the process manager from things like macsbug.
So in theory if one could find the function location of the function that does the dynamic resizing then one could create an extension
that checks ram and runs that function and boom, you get use of all ram!

This is just a theory. Reality doesn't always align correctly.  It still doesn't explain why the process manager only get's 1.5 GB when more is available. 

[Metrophage]

Thanks for your work and sharing of these insights!

[MacTron]

This is just a theory. Reality doesn't always align correctly.  It still doesn't explain why the process manager only get's 1.5 GB when more is available.

The whole 32 bits memory space was designed when 1MB of RAM was a lot of RAM . This 1.5 GB limit was taken later, to avoid interfering with other addressable directions with others uses. ... Probably.

[nanopico]

This is just a theory. Reality doesn't always align correctly.  It still doesn't explain why the process manager only get's 1.5 GB when more is available.

The whole 32 bits memory space was designed when 1MB of RAM was a lot of RAM . This 1.5 GB limit was taken later, to avoid interfering with other addressable directions with others uses. ... Probably.

I would completely agree and believe this to be true as well. 
I was trying to figure out if the limit exists due to an addressing method where the page size is the limiting factor, but I have not found any indication of that or any bit pattern to support that yet.

[nanopico]

I've found something really really cool (well to me it is).
So the System Heap grows up and the Process Manager Heap grows down.  This would be because they share a boundary and the process manager will be less likely to have to have anything at the bottom of it's heap as it loads programs at the top and moves down and heaps in the applications also grow down (the system heap is the only one that grows up).
There are a set of function exported from the memory manager for shrinking and expanding the heap in both the system and process manager.
These are private methods as normal application should never call them. 
When the process manager is started it looks to a resource for it's size (just like any other application with it's minimum and preferred sizes).
Once the process manager is set up the system heap is grown to meet it and then they proceed to have a wonderful relationship where they have little application children until they are both terminated.  Ok jokes aside. 
So now that I know those functions I can start to look for the resource to set the size.  Modify that and if all my assumptions are correct, then we are in business for gaining back most of that 512 MB (except for that actually used by the system heap).

[mrhappy]

Once the process manager is set up the system heap is grown to meet it and then they proceed to have a wonderful relationship where they have little application children until they are both terminated.

now THAT I understand!!

[MacOS Plus]

  I seem to recall there being a physical performance limit of PC133 RAM that capped it at 1.5GB.  Do you think the OS limit might just be a legacy holdover of that, so maintaining compatibility with most of the G4 systems wasn't complicated by the arrival of the DDR-based systems?  In any other respect the cap would look arbitrary.

[nanopico]

  I seem to recall there being a physical performance limit of PC133 RAM that capped it at 1.5GB.  Do you think the OS limit might just be a legacy holdover of that, so maintaining compatibility with most of the G4 systems wasn't complicated by the arrival of the DDR-based systems?  In any other respect the cap would look arbitrary.

I'm not sure I have ever heard of PC133 having some sort of a a 1.5GB limit.
I know I've got an older server sitting my basement that used PC133 that oddly enough has 2 sticks of 1GB each in it with out issue.

I think the limit is arbitrary at this time, and more likely is some sort of hold over from bank switching and page size related to the pre G3 power macs that had 6 dimm slots.

[MacOS Plus]

  Yes, one way or another I'm sure it's a compatibility holdover.  I suppose we should keep in mind that we might break support in some machines by tinkering with this.  Not that it will be of concern to the majority of the users who would want to take advantage of removing the RAM cap since they are more than likely going to be using the later more powerful models anyway.  I'm more curious to know just from an information standpoint if this happens and what machines might be affected and why.

[nanopico]

  Yes, one way or another I'm sure it's a compatibility holdover.  I suppose we should keep in mind that we might break support in some machines by tinkering with this.  Not that it will be of concern to the majority of the users who would want to take advantage of removing the RAM cap since they are more than likely going to be using the later more powerful models anyway.  I'm more curious to know just from an information standpoint if this happens and what machines might be affected and why.

The breaking stuff is what concerns me or if it become unstable or something.
My intent is that if this get's figured out an extension can be made so that you only enable it if you need it and know it works for your setup.

[MacOS Plus]

  Yes, one way or another I'm sure it's a compatibility holdover.  I suppose we should keep in mind that we might break support in some machines by tinkering with this.  Not that it will be of concern to the majority of the users who would want to take advantage of removing the RAM cap since they are more than likely going to be using the later more powerful models anyway.  I'm more curious to know just from an information standpoint if this happens and what machines might be affected and why.

The breaking stuff is what concerns me or if it become unstable or something.
My intent is that if this get's figured out an extension can be made so that you only enable it if you need it and know it works for your setup.

  This is definitely the best approach.  I found mention of the PC133 issue I was talking about.  It's buried in the text of this web page:

http://etutorials.org/Misc/pc+hardware+tuning+acceleration/Chapter+4+Main+Components+and+the+Optimal+Choice/Motherboards/

  The relevant text is as follows - "PC133 specifications recommend a limit of three DIMM or six banks at 133 MHz (The maximum memory in this configuration is 1.5 GB)."  As I remember it you can run into timing and bandwidth issues if you try to physically go beyond this configuration.  It's not a guaranteed failure but was enough of a potential problem to be officially mentioned in the spec.  A lot of PC motherboards from the time period (when PC133 support was introduced) reduced their DIMM slot count from four to three to prevent the end user from making this mistake.  You could have more than 1.5GB reliably installed if you ran it at a 100MHz bus speed, which was possible in certain motherboards with switchable memory or shared memory/CPU bus speed settings in the BIOS.

  Obviously the arrival of DDR eliminated this issue, but by then it was probably deemed not worth fixing in OS 9 because it would have opened a huge can of worms.  I know that OS 9 can't even properly identify the type of some of my DDR DIMMs and erroneously claims they are PC133 in System Profiler, which is a very weird thing for it to do in a DDR-only architecture.  (It also claims that one of the stock Apple PC2100 DIMMs that came with one of my Xserves is actually "PC2600".)

  Have you ever wondered why the Powermac G4 "Digital Audio" and "Quicksilver" are the only G4 models with only three RAM slots?  Well they're the only ones built around PC133 memory.  They can't take more than 1.5GB RAM by design so they can't benefit from removing the software limitation.  The "AGP", "Gigabit Ethernet", "MDD" and "Xserve" will be the only machines we can realistically target for greater than 1.5GB.  I'm not sure what would have to be done to effectively test for good behaviour in these systems.  Some of the guys using a lot of software DAW processing might be able to push the limits.  I recall mention though that there is a 1GB-per-application maximum allocation unit in OS 9 also.

  One final note, the four RAM slots in the AGP and Gigabit Ethernet G4s likely were not ever intended for installing greater than 1.5GB.  The cost of larger DIMMs at the time, particularly if bought through Apple, was prohibitively expensive to many users.  The slots were there to make the most of lower capacity DIMMs.  This lets me make good use of the vast pile of 64MB, 128MB and 256MB DIMMs I kept around after they were pulled from other upgraded PC systems.  Most of my OS 9 and 8.6 machines don't get enough serious use to require a lot of RAM anyway.

[nanopico]

And I do believe we have a definitive answer!!!
Thank you!

Just to add to this as evidence.  The G4's had 4 PC100 slots up to the Digital Audio which used PC133 and had only 3 slots probably for the exact reason identified by MacOS plus. 

Documentation in Inside Macintosh hints that the OS on 32 bit platforms can support up to 4 GB of ram. (Yes I know that's the limit of the address space, but the document indicates most of that could be used by an application).

So the Digital Audio and Quick Silver machines have the physical limitation of 1.5 GB.

So it might actually be simple to get that extra 512MB back.
Would best be done as an extension and stress tested to ensure stability.

[MacTron]

I recall mention though that there is a 1GB-per-application maximum allocation unit in OS 9 also.

This is a Myth. May Be true for some apps but not in general. I have used Cubase with 1.4 Gb or RAM for years without issues:


All this memory is used by the 21 instruments loaded in 3 instances of Kompakt ( a software sampler).
I'm waiting to test nanopico's first work about this.  If I can have 512 mb more I could add 7 or 8 instruments more

[MacOS Plus]

  That's awesome, MacTron!  I'm actually quite interested in dedicating a machine to software sampler instruments.  This probably is just about the best task for pushing the limits of RAM and CPU.  I'll have to try this eventually with one of my three MDDs.  I could probably even MIDI-sync multiple machines if I want to really go nuts.  Did you have to manually allocate all that RAM in the app dialog box or does it just expand on its own as it needs it?

  I have a G4 Sawtooth, MDD, MDD FW800 and Xserve Tray-load, plus a Gigabit Ethernet motherboard I plan to get running also.  This means I should be able to test 2GB RAM on all possible G4 systems when the time comes.  (I guess except for the Xserve Slot-load which is still a work in progress for OS 9.  Not sure I'm interested enough to bother buying one of those.)

[DieHard]

Did you have to manually allocate all that RAM in the app dialog box or does it just expand on its own as it needs it?

CMD-I manually... like Mactron I have set mine very high for Cubase (in my tips I recommend 400,000), but I never tried going past 800,000 (.8GB).

Hmmmmm 

[nanopico]

Did a little further research last night.
I'm still fairly positive it's due to some hardware limitation at some point that just never got updated in code, but I don't think the actual limit is due to the PC133 limitation as this has been in there long before that.

[MacOS Plus]

  I just figure that with OS X well-established by the arrival of the MDD, Apple saw no incentive to build in a workaround even if it was possible in OS 9.  Most users didn't need more than 1.5GB of RAM and the benefit would be available to a relatively limited number of machines anyone would have still been willing to spend money on upgrading.  The permanent limitation of the PC133-based machines and the commitment to the transition to OS X would have made cutting off development on the issue an easy financial decision for Apple.

  I agree that the PC133 models shouldn't have any bearing on the OS code as it stands.  It's unlikely any change you make to the code could exclusively break support for those machines.