Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: 1 2 3 4 [5] 6 7 8 9   Go Down

Author Topic: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers  (Read 129820 times)

nanopico

  • Moderator
  • 512 MB
  • *****
  • Posts: 769
Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
« Reply #80 on: November 02, 2016, 09:05:36 PM »

OMG

Basic memory map for any one that cares. 
Assume 2 GB of ram installed.
Assume your system heap with your extensions and all that fun jazz takes 32 MB.
System zones and heaps set up at address 0 and use up to 32 MB
Process manager zone created at 1.5 GB and grows down to meet the system. (1.5 GB - 32 GB)
About this mac should now show show 32 MB for Mac OS and 1.5 GB - 32 MB for free space.
But it shows 1.5 GB - 32 MB as free space and then 32 MB + 512 MB for the Mac OS.
Alright back to the memory map (it's generalized here, but if you want more details I can post the full map of my system)

  • 0 - 32 MB system
  • 32 MB - 1.5 GB Process Manager
  • 1.5 GB - 3.75 GB not accessible. Can not read or write to anything in that range
  • 3.75 GB - 4 GB readable and writeable and if you do either your system will crash as this appears to be mappings for the hardware addresses and registers

Point is to fix this bug ( do consider this a bug ) The early boot that sets up the MMU in the cpu needs to map out the additional 512 MB so it's accessible and the process manager needs to be told to start at 2 GB and grow down. Since everything is relocatable with in the process manager (if it wasn't we'd all be stuck with like 32 mb of RAM) and that 512 MB of missing memory isn't currently mapped, this should cause almost no problems going forward.

So on to the two things that need to be found to fix this.
Logged
If it ain't broke, don't fix it, or break it so you can fix it!

nanopico

  • Moderator
  • 512 MB
  • *****
  • Posts: 769
Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
« Reply #81 on: November 03, 2016, 08:02:39 AM »

Mild update.

Eln kindly pointed out to me in our most recent conversation there are two things mapped just over the 1.5GB mark. I had forgotten about this so it changes things, but only slightly.  So we are looking into how to move those and then see what happens with the creation of the process manager and possible having to update one value there.
So good news is....    This is becoming more of a reality
Logged
If it ain't broke, don't fix it, or break it so you can fix it!

ELN

  • 256 MB
  • *****
  • Posts: 295
  • new to the forums
Re: 1.5GB Limit (some answers)
« Reply #82 on: November 13, 2016, 06:20:44 AM »

but a few guys scratchin at notepad on a forums site can do it when the real pros couldnt? :) i dunno guys...

We are disassembling and patching the Nanokernel.

https://github.com/elliotnunn/cdg5/blob/master/kernel-disasm-script.py
Logged

Protools5LEGuy

  • Staff Member
  • 2048 MB
  • ******
  • Posts: 2790
Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
« Reply #83 on: November 15, 2016, 08:16:28 AM »

We are very pleased to have the Dream Team among us!

The troll has been kicked out.

99% Mac OS 9 Lives! users love every little update on these topics.
Logged
Looking for MacOS 9.2.4

nanopico

  • Moderator
  • 512 MB
  • *****
  • Posts: 769
Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
« Reply #84 on: November 16, 2016, 06:32:31 AM »

    We are very pleased to have the Dream Team among us!

    The troll has been kicked out.

    99% Mac OS 9 Lives! users love every little update on these topics.

    Updates with progress towards the goal may slow for a bit.  We are rewriting some tools so that we can do the entire process in OS 9.
    Yes we are crazy enough to do the whole thing using MPW.  So while we retool we might not see much in the way of a functional result.

    And if any one would like to help out, I am looking to verify the memory map as we believe it to be.
    Even though I am testing and working on this on 23 different Macs I am looking for anyone who want's to help as I have several that are same model and there are still others I don't have. I'd like to get details from a variety of machines with and with out 2GB installed.

    So here are the instructions.
    • Install Macsbug
            
      • Download macsbug
      • Put it in your system folder
      • Reboot
         
    •       Press command + the power button (both at the same time).  This gets you into macsbug
         
    •       type

            log memoryMap
         
    •       Type the following

            dm 5FFFe000 FF0

            It will pause for a moment at that point type

            command + space
         
    • once the last command finishes type
            
               dm 68000000 ffff0
            
            again when it pauses type

                             command + space
         
    • type

            log
         
    • type the letter g and your good to go

    You can now uninstall macsbug by deleting it from the system folder and rebooting

    You do need to hit enter after every command (I'm hoping that is obvious to everyone, but who knows)

    You will now have a file called memoryMap on your desktop. If you would be so kind as to send that file to me, that would be awesome.

    I may ask again for some of you to help in verifing the hardware pci range mapping, but that is a bit harder to put into easy commands as each machine has different hardware so the ranges are different and it's a lot of trial and error to get the range.
    [/list]
    Logged
    If it ain't broke, don't fix it, or break it so you can fix it!

    Protools5LEGuy

    • Staff Member
    • 2048 MB
    • ******
    • Posts: 2790
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #85 on: November 16, 2016, 08:17:55 PM »


    Yes we are crazy enough to do the whole thing using MPW.  So while we retool we might not see much in the way of a functional result.

    So here are the instructions.

       Install Macsbug
          
       

    Is this version?

    http://macos9lives.com/smforum/index.php/topic,3565.0.html

    http://macintoshgarden.org/apps/macsbug-663
    Logged
    Looking for MacOS 9.2.4

    nanopico

    • Moderator
    • 512 MB
    • *****
    • Posts: 769
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #86 on: November 17, 2016, 05:58:28 AM »


    Yes we are crazy enough to do the whole thing using MPW.  So while we retool we might not see much in the way of a functional result.

    So here are the instructions.

       Install Macsbug
          
       

    Is this version?

    http://macos9lives.com/smforum/index.php/topic,3565.0.html

    http://macintoshgarden.org/apps/macsbug-663

    Yes that one.
    Sorry I should have gave a link to that.
    If any instructions are not clear to anyone, please let me know and I'll update them.
    I don't want anyone to feel they can't help.
    Logged
    If it ain't broke, don't fix it, or break it so you can fix it!

    LillyOmegaWolf

    • 16 MB
    • ***
    • Posts: 18
    • OS9 and SL forever :)
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #87 on: March 06, 2017, 11:09:21 PM »

    Still need someone to trial it too? Or am I a bit late to the party?
    I would love to get the full 2gb of ram working on my G4, if only because I enjoy tweaking and pushing limits :)  so Im happy to copy a new install onto a spare drive and test if you still need testers.
    Logged
    Filmmaker, FCPX editor, Hackintosh user and classic Mac enthusiast.

    InspectorG

    • 8 MB
    • **
    • Posts: 12
    • Total Mac Addict
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #88 on: March 06, 2017, 11:47:25 PM »

    Still need someone to trial it too? Or am I a bit late to the party?
    I would love to get the full 2gb of ram working on my G4, if only because I enjoy tweaking and pushing limits :)  so Im happy to copy a new install onto a spare drive and test if you still need testers.

    Same here!
    Logged

    nanopico

    • Moderator
    • 512 MB
    • *****
    • Posts: 769
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #89 on: March 07, 2017, 04:59:25 AM »

    Still need someone to trial it too? Or am I a bit late to the party?
    I would love to get the full 2gb of ram working on my G4, if only because I enjoy tweaking and pushing limits :)  so Im happy to copy a new install onto a spare drive and test if you still need testers.

    Same here!

    There isn't anything to test at the moment.
    The primary factor limiting the memory is thought to have been identified, but not confirmed as work has to be done to move some things in the OS.
    As of yesterday I am not working on this for an unknown length of time.  ELN has picked up on this and is running with it along with another fine guy, I'm not sure if it's cool to identify him so right now I'm just going to call him awesome dude.
    All is still moving forward, but don't expect to hear anything from me for god knows when.
    Logged
    If it ain't broke, don't fix it, or break it so you can fix it!

    LillyOmegaWolf

    • 16 MB
    • ***
    • Posts: 18
    • OS9 and SL forever :)
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #90 on: March 07, 2017, 02:07:11 PM »

    Well you have put a lot of work into this. So I can understand a break, especially if theres two awesome dudes who have taken this up.
    Logged
    Filmmaker, FCPX editor, Hackintosh user and classic Mac enthusiast.

    ELN

    • 256 MB
    • *****
    • Posts: 295
    • new to the forums
    First limit-breaking ROM: testers needed
    « Reply #91 on: May 24, 2017, 06:18:07 AM »

    All right, here goes. In the spirit of working in the open:

    I think I have broken the 1.5 GB limit...

    ...by 64 MB.

    So now it's the 1.5625 GB limit. But it's definitely a start. First, some explanation.

    Before the Trampoline (an ELF-contained bootloader inside the Mac OS ROM file) hands control to the NanoKernel, it populates some data structures that describe many levels of the Mac OS environment. The NanoKernel pays the most attention to ConfigInfo, a structure that was formerly part of the RISC ROM but became a dynamic structure on NewWorld Macs. One of the jobs of ConfigInfo is to tell the NanoKernel how to lay out the address space of the blue task. ConfigInfo instructs the NanoKernel to make the main area, starting at address zero, as large as possible given the RAM available. If "Virtual Memory" is enabled then this area will be expanded later in the boot process, at the command of the 68k boot code. But ConfigInfo also instructs the NanoKernel to create these small special-purpose areas:
    Code: [Select]
    5fffe000-5fffefff InfoRecord page: contains pointers to structures shared by kernel and userspace
    64000000-6417ffff Open Firmware device tree, and Mac II-style "Universal" structures
    68fef000-68feffff ConfigInfo
    68ffe000-68ffefff Kernel Data Page (kernel actually expands this to ~68ff5000-68ffefff)
    68fff000-68ffffff Emulator Data Page

    Other areas in the 6 segment are configured differently. The addresses listed might be familiar to anyone who's looked at the NanoKernel log before. But notice in the attached screenshot that the 5fffe000 area is missing, and a 6fffe000 area is present instead. That is because I have inserted code into the NanoKernel that reads the Trampoline's PageMap and edits it. For the curious, here is that code:

    Code: [Select]
    macro
    PutPMDT &w1, &w2
    lisori r31, &w1
    stwx r31, r20, r21
    addi r21, r21, 4
    lisori r31, &w2
    stwx r31, r20, r21
    addi r21, r21, 4
    endm


    macro
    StartSeg &s
    stw r21, NKConfigurationInfo.SegMap32SupInit + 8*&s(r3)
    stw r21, NKConfigurationInfo.SegMap32UsrInit + 8*&s(r3)
    stw r21, NKConfigurationInfo.SegMap32CPUInit + 8*&s(r3)
    stw r21, NKConfigurationInfo.SegMap32OvlInit + 8*&s(r3)
    endm

    macro
    SpecialArea &a
    stw r21, NKConfigurationInfo.PageMap&a.Offset(r3)
    endm


    if RedRover & !Vanilla

    ; INIT
    lwz r20, NKConfigurationInfo.PageMapInitOffset(r3)
    add r20, r20, r3
    li r21, 0

    lwz r29, NKConfigurationInfo.SegMaps + 8*6(r3)
    add r29, r29, r20 ; r20 being the base of the pagemap
    subi r29, r29, 8

    @nt6 lwzu r28, 8(r29)
    srwi r28, r28, 28
    cmpwi r28, 4
    bne @nt6
    lwz r28, 0(r29)
    lwz r29, 4(r29)
    ; Now r28/r29 are the 64000000 entry!
    ; (Important because its physical address and size vary)


    ; Vanilla page table (probably doesn't vary much between NW Macs)

    ; StartSeg 0x0
    ; PutPMDT 0x0000ffff, 0x00000a00
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;
    ; StartSeg 0x1
    ; PutPMDT 0x0000ffff, 0x00000a00
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;
    ; StartSeg 0x2
    ; PutPMDT 0x0000ffff, 0x00000a00
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;
    ; StartSeg 0x3
    ; PutPMDT 0x0000ffff, 0x00000a00
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;
    ; StartSeg 0x4
    ; PutPMDT 0x0000ffff, 0x00000a00
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;
    ; StartSeg 0x5
    ; PutPMDT 0x0000fffd, 0x00000a00
    ; PutPMDT 0x0000fffd, 0x00000a00
    ; SpecialArea IRP
    ; PutPMDT 0xfffe0000, 0x00000012
    ; PutPMDT 0x0000ffff, 0x00000a00
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;
    ; StartSeg 0x6
    ; stwx r28, r20, r21
    ; addi r21, r21, 4
    ; stwx r29, r20, r21
    ; addi r21, r21, 4
    ; PutPMDT 0x8fef0000, 0x00003013
    ; SpecialArea KDP
    ; PutPMDT 0x8ffe0000, 0x00000011
    ; SpecialArea EDP
    ; PutPMDT 0x8fff0000, 0x00000012
    ; PutPMDT 0x0000ffff, 0x00000a01
    ; PutPMDT 0x0000ffff, 0x00000a01
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;
    ; StartSeg 0x7
    ; PutPMDT 0x0000ffff, 0x00000a01
    ; PutPMDT 0x0000ffff, 0x00000a01
    ; PutPMDT 0x0000ffff, 0x00000a00
    ;


    ; Don't forget to patch logical address fields:
    lisori r31, 0x6fffe000
    stw r31, NKConfigurationInfo.LA_InfoRecord(r3)


    StartSeg 0x0
    PutPMDT 0x0000ffff, 0x00000a00
    PutPMDT 0x0000ffff, 0x00000a00

    StartSeg 0x1
    PutPMDT 0x0000ffff, 0x00000a00
    PutPMDT 0x0000ffff, 0x00000a00

    StartSeg 0x2
    PutPMDT 0x0000ffff, 0x00000a00
    PutPMDT 0x0000ffff, 0x00000a00

    StartSeg 0x3
    PutPMDT 0x0000ffff, 0x00000a00
    PutPMDT 0x0000ffff, 0x00000a00

    StartSeg 0x4
    PutPMDT 0x0000ffff, 0x00000a00
    PutPMDT 0x0000ffff, 0x00000a00

    StartSeg 0x5
    PutPMDT 0x0000ffff, 0x00000a00
    PutPMDT 0x0000ffff, 0x00000a00

    StartSeg 0x6
    PutPMDT 0x00003fff, 0x00000a00
    PutPMDT 0x00003fff, 0x00000a00

    stwx r28, r20, r21 ; Name Registry
    addi r21, r21, 4
    stwx r29, r20, r21
    addi r21, r21, 4

    PutPMDT 0x8fef0000, 0x00003013 ; ConfigInfo

    SpecialArea KDP
    PutPMDT 0x8ffe0000, 0x00000011
    SpecialArea EDP
    PutPMDT 0x8fff0000, 0x00000012
    SpecialArea IRP
    PutPMDT 0xfffe0000, 0x00000012

    PutPMDT 0x0000ffff, 0x00000a01
    PutPMDT 0x0000ffff, 0x00000a01
    PutPMDT 0x0000ffff, 0x00000a00

    StartSeg 0x7
    PutPMDT 0x0000ffff, 0x00000a01
    PutPMDT 0x0000ffff, 0x00000a01
    PutPMDT 0x0000ffff, 0x00000a00


    ; Later parts of the table
    ; (definitely vary between machines)

    ; StartSeg 0x8
    ; PutPMDT 0x0000ffff, 0x8000003a
    ; PutPMDT 0x0000ffff, 0x80000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ; StartSeg 0x9
    ; PutPMDT 0x0000000f, 0x9000003a
    ; PutPMDT 0x0020001f, 0x9002003a
    ; PutPMDT 0x80007fff, 0x98000032
    ; PutPMDT 0x0000ffff, 0x90000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ; StartSeg 0xa
    ; PutPMDT 0x0000ffff, 0xa0000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ; StartSeg 0xb
    ; PutPMDT 0x0000ffff, 0xb0000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ; StartSeg 0xc
    ; PutPMDT 0x0000ffff, 0xc0000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ; StartSeg 0xd
    ; PutPMDT 0x0000ffff, 0xd0000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ; StartSeg 0xe
    ; PutPMDT 0x0000ffff, 0xe0000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ; StartSeg 0xf
    ; PutPMDT 0x0000ffff, 0xf000003a
    ; PutPMDT 0x0000ffff, 0xf0000a01
    ; PutPMDT 0x0000ffff, 0x00000a00

    ;stw r21, NKConfigurationInfo.PageMapInitSize(r3)
    li r20, 0
    li r21, 0
    li r31, 0

    endif

    The effect of the patch is to move the InfoRecord page from the 5 segment to the equivalent location in the 6 segment, and to expand the blue area up to the bottom of the next obstacle, at 64000000. This is a difference of 64 MB.

    On its own, this patch crashes my Power Mac (even though it only has 256 MB). This is because the 68k Emulator, which lives in ROM just a few KB north of the NanoKernel, contains a hardcoded constant, 5fffe000, which it uses to fetch the LogicalMemorySize from the NanoKernel's SystemInfo structure. I changed that, and all was well. (Hopefully there are no more subtle crashes waiting to strike. Well-written software should use the NKLocateInfoRecord kernel call instead of hardcoding that location.)

    Interestingly, the patch alone did not crash QEMU. QEMU is currently my only way to start Mac OS 9 with the full 2 GB of RAM. (By the way, anyone want to send me 2 GB that will work in my FW800 MDD? I'm hopeless with hardware.) QEMU does not start the Dynamic Recompiling function of the emulator, which is the part that contains the hardcoded constant. I am not yet sure why.

    So that's one down and

    So, I need testers with 2 GB Macs running Mac OS 9.2.2. Attached is a Mac OS ROM file and a copy of NKLogReader. First, dump the log output of your normal kernel. Then, install the patched Mac OS ROM file. If you get a crash on boot, please take a picture of the kernel log. If it boots up, please get a screenshot of the Finder About window and a kernel dump.

    Soon we might bust the 1.5 GB limit completely. Good luck, and have fun.
    Logged

    Protools5LEGuy

    • Staff Member
    • 2048 MB
    • ******
    • Posts: 2790
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #92 on: May 24, 2017, 08:00:47 AM »

    Are this 64Mb able to be used as a RAM disk?
    Logged
    Looking for MacOS 9.2.4

    ELN

    • 256 MB
    • *****
    • Posts: 295
    • new to the forums
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #93 on: May 24, 2017, 08:50:32 AM »

    The system should see it as normal, usable RAM.
    Logged

    MacOS Plus

    • 256 MB
    • *****
    • Posts: 418
    • The 9serve Lives!
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #94 on: May 24, 2017, 09:28:21 AM »

      Not smooth sailing so far.  One of the DIMMs I borrowed from an Xserve to max out the RAM in my MDDs caused random hard freezes.  The usual moral of the story applies - if possible stick to a fully matched set of DIMMs!  Completing a matched set fixed that issue.

      I had a bit of trouble with saving the attached file.  Classilla doesn't use the correct file name for the archive, no big deal, but I don't think the ROM file came out right.  A fresh 9.2.2 boot drive wasn't recognized as bootable at startup with the replacement ROM.  Also I had to pull NKLogReader from your original post about it because it wasn't present in the current attachment.  I was able to run NKLogReader when booted from the original ROM and then save the log to a file.  The modified ROM will have to wait until this issue is sorted out.

      Once I can get 'step one' working, could you apply the "unsupported G4s" patch so I can try this on FW800 and Xserve?
    Logged

    MacTron

    • Staff Member
    • 2048 MB
    • ******
    • Posts: 2116
    • keep it simple
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #95 on: May 24, 2017, 10:29:41 AM »

    Great!!!  ;D

    All right, here goes. In the spirit of working in the open:

    I think I have broken the 1.5 GB limit...

    ...by 64 MB.
    ... 64MB...
    ... this is a small step for a man... but a big step for ...
     ;D ;D ;D ;D

    Logged
    Please don't PM about things that are not private.

    ELN

    • 256 MB
    • *****
    • Posts: 295
    • new to the forums
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #96 on: May 24, 2017, 10:32:51 AM »

    I am surprised to hear that! Sorry you've had trouble. I tried downloading the archive from the site. It worked fine for me, log dumper and all.

    Here's a new archive. I rebuilt the image with some of the changes I've made in the last couple of hours, and compressed it with a different version of Stuffit. You should get an extra 128 MB recognised now.

    My builds should actually boot fine on an "unsupported" G4. (But I reflashed my MDD back in 2015, so I haven't been testing that.)

    Thanks for trying it out already! Let me know how this goes.
    Logged

    ELN

    • 256 MB
    • *****
    • Posts: 295
    • new to the forums
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #97 on: May 24, 2017, 10:36:22 AM »

    Thanks, MacTron! Could I trouble you for the contents of your kernel log?
    Logged

    MacTron

    • Staff Member
    • 2048 MB
    • ******
    • Posts: 2116
    • keep it simple
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #98 on: May 24, 2017, 12:23:45 PM »

    Thanks, MacTron! Could I trouble you for the contents of your kernel log?

    No problem but I have to solve an issue with the "Second limit-breaking ROM" because it cant startup the MDD I was using for testing, and in this machine I only have installed one System folder  :-[ ... LOL
    Logged
    Please don't PM about things that are not private.

    MacTron

    • Staff Member
    • 2048 MB
    • ******
    • Posts: 2116
    • keep it simple
    Re: Mac OS 9.2.2 Memory Limit of 1.5 GB... Some Answers
    « Reply #99 on: May 24, 2017, 01:26:46 PM »

    Attached is a screenshot of the crash I obtain when booting with "Second limit-breaking ROM".
    Logged
    Please don't PM about things that are not private.
    Pages: 1 2 3 4 [5] 6 7 8 9   Go Up

    Recent Topics