I'll do a summary next, but the short version is that both temperatures and noise are down compared to stock.
First however there is the side panel that needed some holes to facilitate the removal of hot air from the case. Going by
this example I thought basically any holes at all would go a long way towards solving the problem of heat buildup in the case. I came up with a design that I thought would work:

To test, I took an old scuffed side panel and started to drill some holes in it. I tried it, but straight away it became clear that just a few holes were not going to cut it: temperatures went up to almost what they were with a closed side panel. So, I drilled some more holes in my test panel:

It doesn't look like much, but the total surface area of the holes is about the same as in my design, so it should be good to test. As a side note, the panel is very easy to work on, as can be seen from the tiny threads of material left between the holes on the second row from the bottom. Polycarbonate is quite elastic, and does not break or crack easily.
There was a clear improvement, with temperatures only going up 3-4 degrees compared to with the side panel off:

To clarify: I started up the system and did some stress tests, during which I put on the side panel. Temperatures go up straight away as you can see. Then, after leaving it on overnight, I did some more stress tests, during which I took the side panel off again, resulting in temperatures dropping again.
Clearly, more holes are better, so first I enlarged the hole I cut in the side of the case:

...and also stuck the wifi antenna out of the way. Then, I added another row of holes to the design, and increased the diameter of all of them by 1 mm:

This increased the total surface area by 39%, which I hoped would be enough. After a few hours of careful drilling, I ended up with this:

Given that most of it was done with an electric hand drill, I'm not too unhappy with the result:

One more, with the panel in place:

So did it work? Thankfully, yes it did. After some more fiddling with cable routing, temperatures are now between 53 and 55 degrees, so not as low as without the side panel, but low enough for me. I expect they will drop some more over time as the thermal compound sets or cures or whatever.