MacinAI Local: Building a Model-Agnostic LLM Inference Engine for Mac OS 9

By: Alex Hoopes | Published: March 19, 2026

How I built a complete AI platform (custom C89 inference engine, BPE tokenizer, AltiVec SIMD optimization, and a Python export pipeline) that runs GPT-2, TinyLlama, Qwen, and any HuggingFace model on a 2002 PowerBook G4.

Technical Reference

Source Language: C89 (ANSI C), compiled with CodeWarrior Pro 5

Target OS: System 7.5.3 through Mac OS 9.2.2

Target CPUs: Motorola 68000, 68030, 68040; PowerPC G3, G4

Supported Model Architectures:

    LLaMA-family: RMSNorm + SwiGLU + RoPE (LLaMA, Mistral, Qwen, Gemma, TinyLlama, SmolLM, StableLM)
    GPT-2-family: LayerNorm + GeLU + learned pos (GPT-2, OPT, Pythia, GPT-J, Falcon, Phi)

Quantization: Float32, Q8_0 (per-group int8, block size 32)

Custom Model: ~100M parameters, 1.1GB Macintosh training corpus, 5,800+ SFT instruction pairs, DPO refinement

Inference Speed (PowerBook G4 1GHz, Q8): 2.66 tok/sec (0.38s/token) for custom model, 0.63 tok/sec for Qwen 0.5B

Memory Usage (100M Q8 model): ~124MB total (100MB weights, 23MB KV cache, 1MB overhead)

AltiVec Speedup: 6.3x over scalar baseline (demo build), up to 7.3x measured on earlier build

BPE Vocabulary: 8,205 tokens (8,192 BPE + 13 special + command tokens)

This is MacinAI local, a vintage Macintosh AI inference program capable of running a 1.1 billion parameter language model locally on a PowerBook G4 (1GHz, 1GB RAM, Mac OS 9.2) - no internet, no server, no cloud. Just a Mac from 2002 and CDs.

Timestamps:
00:00 - Introduction
01:10 - MacinAI Tool Model (94M param)
04:10 - GPT-2 Model (124M param)
05:10 - SmolLM Model (360M param)
06:45 - Qwen 2.5 Model (500M param)
10:30 - TinyLlama (1.1B param)
33:45 - Conclusion

The inference engine is written entirely in C89 using the Mac Toolbox API, with AltiVec SIMD acceleration for the heavy matrix math. The TinyLlama 1.1B model requires 1.2 GB of weights but the machine only has 1 GB of RAM, so the engine pages transformer layers from the hard drive on demand - 14 layers in memory, 8 streamed from disk per token.

Models demonstrated:

    MacinAI Tool 94M (custom) - 2.86 tok/s
    GPT-2 124M - 2.36 tok/s
    SmolLM 360M Instruct - ~1.5 tok/s
    Qwen2.5 0.5B Instruct - ~2 tok/s
    TinyLlama 1.1B Chat - ~0.1 tok/s (with disk paging)


Able to be shipped on a 2-disc CD set with a custom installer - the first LLM ever distributed on CD (if people actually want it).

Export script supports any HuggingFace model (LLaMA-family, GPT-2-family) to the custom .bin format.

Hardware: PowerBook G4 Titanium 1GHz, 1GB RAM, Mac OS 9.2.2
Detailed implementation and downloads: https://oldapplestuff.com/blog/MacinAI-Local/

#MARCHintosh #RetroComputing #MacOS9 #AI #LLM

TetOS is a custom version of the classic MacOS that uses custom resources and ResEdit to retheme the entire os,
Based on the custom MacOS shown in the music video for Machine Love by Jamie Paige
All assets were custom made by me from scratch
For more information on the technical aspect of how to do this please read my write up over at TinkerDifferent