Mac OS 9 Lives

Classic Mac OS Software => Internet & Communications => Topic started by: blinkfox on October 26, 2020, 10:09:52 AM

Title: TLS1.1+ support for Classilla with MITMproxy
Post by: blinkfox on October 26, 2020, 10:09:52 AM
Hey ya'll.

I recently aquired a Mac Mini G4 that I've been using for running classic MacOS 9.2.2 and have been enjoying the experience immensely.

As most of you know we have a fantastic browser available to us, Classilla, but browsing the modern web with Classilla has two major drawbacks:

1) The rendering side, where modern versions of CSS won't render correctly and Javascript code execution is slow and error prone due to the now ancient Javascript engine used.

2) The security side, where it's impossible to establish https connections to the vast majority of websites out there since TLS 1.0 is the latest encryption protocol Classilla supports and most sites choose not to support encryption protocols of that era.

Although I can't really help with the first class of problems, I do have a solution to the second one.

I'm able to connect to all https enabled websites using Classilla by setting up a proxy server (of sorts) on a Raspberry Pi I had laying around and routing all the web traffic from Classilla through the proxy.

For https sites the proxy server sets up a connection to the website with a modern encryption protocol (TLSv1.1+) and then establishes a legacy protocol connection (TLSv1) back to Classilla with a mirror of the upstream certificate, but signed by CA certificate the proxy server generates on first launch.
That CA certificate has to be imported into Classilla.

The proxy server I'm using is MITMproxy, running on Ubuntu 20.04 on Raspberry Pi 4.

Installation:
root@ubuntu:~# apt install mitmproxy

Running:
root@ubuntu:~# SECLEVEL=1 mitmproxy --showhost --set ssl_version_client=all --set ssl_version_server=secure --set upstream_cert=true --ssl-insecure

Now that the proxy server is running, we need to import the CA certificate MITMproxy uses to sign certificates into Classilla. You can find the certificate in /root/.mitmproxy/mitmproxy-ca-cert.pem and you'll need to copy that to the Mac.

Then you open Classilla and select Edit → Preferences → Advanced → Certificates → Authorities → Import — and browse to the local directory that has a copy of mitmproxy-ca-cert.pem and import the certificate. A popup window should appear from Classilla where you need to select the checkbox for "Trust this CA to identify web sites" and then click on the OK button.

Once you've imported the CA certificate from MITMproxy you need to configure Classilla to route it's web traffic through the proxy server, the Classilla preference window should still be open from the last step so select Advanced → Proxies from the preference navigation panel and specify the IP address and port of the proxy server for both HTTP Proxy and SSL Proxy. MITMproxy uses port 8080 by default.

Finally, we'll restart Classilla and assuming you've configured everything correctly, you should now be able to connect to websites that are using modern encryption protocols without problems.

(https://i.imgur.com/nDNC7pR.jpeg)

Full size screenshot: https://i.imgur.com/ODg2RRW.jpeg (https://i.imgur.com/ODg2RRW.jpeg)

Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: PowerFinnrich on October 28, 2020, 10:40:19 AM
...and the difference to the WRP Proxy is,
that Classilla displays actual websites and not only a PNG or GIF picture?
That would be a great step forward, I'm going to try it out.
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: IIO on October 29, 2020, 04:06:58 AM
you basically download on another machine and then browse from there. this is basically what also happens in the internet anyway, if you download apple.de/index.htm the content is also coming from your gateway in you neighbourhood and not from munich or cupertino.

i just dont know what happens where there is a download link which is also https? ;) would you solve it the same way?
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: blinkfox on October 29, 2020, 05:01:08 PM
...and the difference to the WRP Proxy is, that Classilla displays actual websites and not only a PNG or GIF picture?

That's correct, there is no remote rendering taking place. Just the native Classilla engine.


you basically download on another machine and then browse from there. this is basically what also happens in the internet anyway, if you download apple.de/index.htm the content is also coming from your gateway in you neighbourhood and not from munich or cupertino.
i just dont know what happens where there is a download link which is also https? ;) would you solve it the same way?

Normal, proxy-less internet traffic always connects you (the client) directly to the web server. If the server is on the internet then your packets will travel trough various routers along the way until they reach their destination (the web server) but in essence you are speaking directly to the web server.

Normal proxy servers can sit as intermediaries between you and the web server, so instead of you sending requests directly to the web server, you send them to the proxy server you have configured, and the proxy server makes requests to the web server on your behalf and then sends you the response (and keeps a cached copy to speed up subsequent visits). For SSL connections things are bit different. The proxy server has a specific way to connect you directly to the web server so you'll get the correct and valid SSL certificate the web server has configured. This requires that your browser and web server can agree on an encryption protocol to communicate with each other, something Classilla has had real problems with in recent years as website operators have opted not to support the very old TLS 1.0 standard, which is the latest encryption standard Classilla supports.

And finally, we get to proxy servers that can intercept SSL connections. MITMproxy is one of these proxies.
These proxies handle the SSL connections to the web server over the latest encryption protocols and then create a SSL connection back to the client with a legacy encryption protocol Classilla supports. This requires the proxy server to present a self-signed copy of the SSL certificate back to Classilla so it's important to add the CA certificate  from the proxy to Classilla's certificate store or you'll get SSL connection warning on every site you visit.
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: Milko on November 04, 2020, 02:29:36 PM
This looks fantastic. There are tons of sites that render usably and sometimes almost flawlessly in Classilla but can't be loaded anymore because of the lack of TLS 1.1+ support. Is there a way to do it in other operating systems like OS X?
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: blinkfox on November 08, 2020, 07:46:16 AM
This looks fantastic. There are tons of sites that render usably and sometimes almost flawlessly in Classilla but can't be loaded anymore because of the lack of TLS 1.1+ support. Is there a way to do it in other operating systems like OS X?

There sure is! MITMproxy[1] is written in Python and is available for Linux, macOS and Windows.

I just tried the macOS port and it works flawlessly. You'll first need to install HomeBrew[2] and then you can install it through HomeBrew:
$ brew install mitmproxy

and then run it with:
$ mitmproxy --showhost --set ssl_version_client=all --set ssl_version_server=secure --set upstream_cert=true --ssl-insecure

Don't forget to import the CA certificate from MITMproxy into Classilla as per the original post.

I'll leave the Windows port as an exercise for the reader.

[1] https://mitmproxy.org
[2] https://brew.sh
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: blinkfox on November 08, 2020, 08:03:51 AM
This might be slightly off-topic but I have stopped using Classilla as a daily browser and instead am using Firefox via a VNC session running on the Raspberry Pi.

My experience was that even though I could establish a connection to any website (via MITMproxy) rendering those websites sometimes caused the browser to lock-up and made the entire OS freeze for up to a couple of minutes. For me personally, that was unacceptable.

So I sincerely hope that this guide is helpful to some of you that might only visit sites you know that are compatible with Classilla's rendering and JavaScript engine, or are willing to accept that some sites might randomly cause your computer to crash. For everyone else I'd rather recommend going the VNC browser route.

Just for fun, here's the difference between Classilla and Firefox v82 rendering the Web Standards Project's complience test cases Acid2 (from 2005) and Acid3 (from 2008). It's just a hot mess.

https://en.wikipedia.org/wiki/Acid2
https://en.wikipedia.org/wiki/Acid3
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: Syntho on November 18, 2020, 10:15:37 AM
Any chance of getting this to work on a Synology NAS?
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: blinkfox on April 18, 2021, 05:26:12 PM
There is a much, much easier way of getting modern TLS support with Classilla as of version 9.3.4b with the Crypto Ancienne library and the carl proxy server.

https://github.com/classilla/cryanc/blob/main/README.md (https://github.com/classilla/cryanc/blob/main/README.md)
https://www.floodgap.com/software/classilla/carl.html (https://www.floodgap.com/software/classilla/carl.html)

Here's a quick way of getting carl up and running on a Raspberry Pi 4 running Ubuntu 20.04 TLS

If you don't have the tools required for building from source code you'll need to install the build-essential meta-package that will install everything you'll need.
Code: [Select]
$ sudo apt install build-essential
Grab the source code from GitHub and build it.
Code: [Select]
$ git clone https://github.com/classilla/cryanc.git
$ cd cryanc
$ gcc -O3 -o carl carl.c
$ sudo cp carl /usr/bin/

Install the xinetd package and configure a service for carl
Code: [Select]
$ sudo apt install xinetd
$ sudo systemctl enable xinetd
$ sudo rm -f /etc/xinetd.d/*

Add the following to /etc/services via your favourite text editor
Code: [Select]
carl 8765/tcp
Add the following to /etc/xinetd.d/carl via your favourite text editor
Code: [Select]
service carl
{
disable = no
socket_type = stream
protocol = tcp
port = 8765
wait = no
user = root
server = /usr/bin/carl
server_args = -p
}

We're done. Time to start up xinetd and then all we need to do is configure Classilla to use the proxy server.
Code: [Select]
$ sudo systemctl start xinetd

Configure Classilla

Type about:config into the address bar and change the setting "network.http.proxy.use-http-proxy-for-https" to true.

Then all we need to do is open up Classilla Preferences → Advanced → Proxies and specify the IP address and port of the proxy server for the SSL Proxy. We've configured carl to use port 8765 in this example.

And that's it. Classilla can now browse websites that use a modern version of TLS.
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: Mat on April 18, 2021, 10:44:27 PM
There sure is! MITMproxy[1] is written in Python
As we have Python in an early version for Mac OS 9, is some coder around who can have a look how much effort it would need to port it to Mac OS 9 itselve?
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: bfcastello on April 22, 2021, 09:41:59 PM
There is a much, much easier way of getting modern TLS support with Classilla as of version 9.3.4b with the Crypto Ancienne library and the carl proxy server.

https://github.com/classilla/cryanc/blob/main/README.md (https://github.com/classilla/cryanc/blob/main/README.md)
https://www.floodgap.com/software/classilla/carl.html (https://www.floodgap.com/software/classilla/carl.html)

Here's a quick way of getting carl up and running on a Raspberry Pi 4 running Ubuntu 20.04 TLS

If you don't have the tools required for building from source code you'll need to install the build-essential meta-package that will install everything you'll need.
Code: [Select]
$ sudo apt install build-essential
Grab the source code from GitHub and build it.
Code: [Select]
$ git clone https://github.com/classilla/cryanc.git
$ cd cryanc
$ gcc -O3 -o carl carl.c
$ sudo cp carl /usr/bin/

Install the xinetd package and configure a service for carl
Code: [Select]
$ sudo apt install xinetd
$ sudo systemctl enable xinetd
$ sudo rm -f /etc/xinetd.d/*

Add the following to /etc/services via your favourite text editor
Code: [Select]
carl 8765/tcp
Add the following to /etc/xinetd.d/carl via your favourite text editor
Code: [Select]
service carl
{
disable = no
socket_type = stream
protocol = tcp
port = 8765
wait = no
user = root
server = /usr/bin/carl
server_args = -p
}

We're done. Time to start up xinetd and then all we need to do is configure Classilla to use the proxy server.
Code: [Select]
$ sudo systemctl start xinetd

Configure Classilla

Type about:config into the address bar and change the setting "network.http.proxy.use-http-proxy-for-https" to true.

Then all we need to do is open up Classilla Preferences → Advanced → Proxies and specify the IP address and port of the proxy server for the SSL Proxy. We've configured carl to use port 8765 in this example.

And that's it. Classilla can now browse websites that use a modern version of TLS.

Thank you! I configured it with my Raspberry Pi 3B and its working. I am testing it using my OS 9.2.2 virtual machine, installed on UTM for iOS and my iPad Pro. Works flawlessly. I wonder if it works with other browsers and other vintage OS or just with Classilla.
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: MacHead on June 13, 2021, 03:43:23 PM
Thanks for this blinkfox. I had trouble setting up carl on my FreeNAS server, but MITM was incredibly easy to install and get up and running. Browsing in Classilla is actually doable now.

I feel like I'm back in 2003 again.
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: bfcastello on October 31, 2021, 06:03:02 PM
There is a much, much easier way of getting modern TLS support with Classilla as of version 9.3.4b with the Crypto Ancienne library and the carl proxy server.

https://github.com/classilla/cryanc/blob/main/README.md (https://github.com/classilla/cryanc/blob/main/README.md)
https://www.floodgap.com/software/classilla/carl.html (https://www.floodgap.com/software/classilla/carl.html)

Here's a quick way of getting carl up and running on a Raspberry Pi 4 running Ubuntu 20.04 TLS

If you don't have the tools required for building from source code you'll need to install the build-essential meta-package that will install everything you'll need.
Code: [Select]
$ sudo apt install build-essential
Grab the source code from GitHub and build it.
Code: [Select]
$ git clone https://github.com/classilla/cryanc.git
$ cd cryanc
$ gcc -O3 -o carl carl.c
$ sudo cp carl /usr/bin/

Install the xinetd package and configure a service for carl
Code: [Select]
$ sudo apt install xinetd
$ sudo systemctl enable xinetd
$ sudo rm -f /etc/xinetd.d/*

Add the following to /etc/services via your favourite text editor
Code: [Select]
carl 8765/tcp
Add the following to /etc/xinetd.d/carl via your favourite text editor
Code: [Select]
service carl
{
disable = no
socket_type = stream
protocol = tcp
port = 8765
wait = no
user = root
server = /usr/bin/carl
server_args = -p
}

We're done. Time to start up xinetd and then all we need to do is configure Classilla to use the proxy server.
Code: [Select]
$ sudo systemctl start xinetd

Configure Classilla

Type about:config into the address bar and change the setting "network.http.proxy.use-http-proxy-for-https" to true.

Then all we need to do is open up Classilla Preferences → Advanced → Proxies and specify the IP address and port of the proxy server for the SSL Proxy. We've configured carl to use port 8765 in this example.

And that's it. Classilla can now browse websites that use a modern version of TLS.

Hi, I noticed that with these instructions it's possible to compile carl on a modern macOS, I did it on macOS Monterey (despite some warnings along the way from the compiler).

However since macOS Monterey does not provide a xinetd (nor a micro_inetd) service, I don't know what is the similar way/service in modern macOS to configure and run carl on startup?
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: blinkfox on November 22, 2021, 11:26:34 AM
However since macOS Monterey does not provide a xinetd (nor a micro_inetd) service, I don't know what is the similar way/service in modern macOS to configure and run carl on startup?

The LaunchD system in macOS can be used to run services that require (x)inetd.

Create the file /Library/LaunchDaemons/org.acme.carl_legacy_proxy.plist with the following content:

Code: [Select]
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>org.acme.carl_legacy_proxy</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/sbin/carl</string>
        <string>-p</string>
    </array>
    <key>inetdCompatibility</key>
    <dict>
        <key>Wait</key>
        <false/>
    </dict>
    <key>Sockets</key>
    <dict>
        <key>Listeners</key>
        <dict>
            <key>SockServiceName</key>
            <string>ultraseek-http</string>
            <key>SockType</key>
            <string>stream</string>
            <key>SockFamily</key>
            <string>IPv4</string>
        </dict>
    </dict>
</dict>
</plist>

Then load the new service (as root) from your Terminal:
Code: [Select]
# launchctl load /Library/LaunchDaemons/org.acme.carl_legacy_proxy.plist

And that's it, the Carl proxy service is running on port 8765 and will start automatically on boot.

To uninstall just unload the service and remove the .plist file.

Code: [Select]
# launchctl unload /Library/LaunchDaemons/org.acme.carl_legacy_proxy.plist
# rm -f /Library/LaunchDaemons/org.acme.carl_legacy_proxy.plist
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: blinkfox on November 22, 2021, 11:30:35 AM
Configure Classilla

Type about:config into the address bar and change the setting "network.http.proxy.use-http-proxy-for-https" to true.

Just a small addendum to the Classilla configuration, if you experience a 10 second delay on every request going through the proxy server you should see if setting "network.http.proxy.version" to "1.0" helps. (I think it defaults to 1.1)
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: bfcastello on November 22, 2021, 11:57:40 AM
This is great. I'm gonna try it later. Having Carl on my main MacBook which is used as a host for a guest OS9 VM, totally negates the need for a Raspberry Pi here in my room or on the road with me. Thanks!

Think I'm gonna use the space saved in my pocket (without the Pi3) for a box of bubble gums :P lol.
Title: Re: TLS1.1+ support for Classilla with MITMproxy
Post by: bfcastello on December 22, 2021, 07:28:47 PM
Configure Classilla

Type about:config into the address bar and change the setting "network.http.proxy.use-http-proxy-for-https" to true.

Just a small addendum to the Classilla configuration, if you experience a 10 second delay on every request going through the proxy server you should see if setting "network.http.proxy.version" to "1.0" helps. (I think it defaults to 1.1)

Right, I compiled it on my M1 Mac running macOS Monterey, it gave me 11 warnings but compiled something that apparently is OK.

I moved it to /usr/local/bin and tested with $ cryanc -v which returned me the version, so appears to be OK.

I then did a copy/paste of your launchd service plist as instructed and loaded it. I think that one thing may be missing - the port? With the Pi3 I was using 8765. It doesn't seem to be working when I fire up macOS 9 and run Classilla after updating the machine ip.


Nevermind. I figured out what was wrong. Your plist points to /usr/local/sbin instead of /usr/local/bin, I fixed it. All working wonderfully now. Thank you!