Thank you for your replies
I forgot to mention that I tried with another personality card - also with same issue.
The solution here and now is that I plugged in a USB DAC and use that as my sound card. This, however, works pretty well.
From what I understand of your ideas and the things I could stumble upon on Google, the ROM perhaps saved the sample rate wrongly although showing it as 44.1. I did find 2 people with exact same problem. One person decided in the end to change mother board and the other one got lucky with just changing personality card, but eventually the problem came back and therefore it was concluded that the personality card slot was flaky and that the weight on this slot is kinda too heavy and may eventually not make good enough contact.
A friend of mine also suggested to set the rate at 22khz and reboot, then set it again at 44.1khz and reboot again. Haven't tried it yet, but will do that later. The idea was to "reset" the ROM.
Can it really be true that it's possible via the OS, be it 9 og X, to save something in the hardware and thereby make such a fault?