Mac OS 9 Lives

Classic Mac OS Software => Application Development & Programming => Topic started by: Knezzen on October 20, 2018, 01:58:47 PM

Title: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on October 20, 2018, 01:58:47 PM
Hi!

Just finished backporting MacTorrent to REALbasic 5.5.5 just to be able to develop the application in Mac OS 9 and not just use it there.

Now to the problem. The back ported code leaks terribly when ran (after the application is built etc).
It uses up as much memory as the torrent data that is written to disk is big, so the problem lays somewhere in the part of the code that manages the torrent data (I think).
The frustrating part is that the code runs well without memory leaks in REALbasic 2005 and newer (without rewriting anything), so it's really hard to find where the memory leak comes from. REALbasic 2005 and newer probably uses some other kind of garbage collecting that is more effective. Something that RB 5.5.5 doesn't do.

Would be wonderful if some REALbasic guru could lend me a helping hand on the matter. I want to get new versions of MacTorrent out the door so to say, and I want to do that using RB 5.5.5.

So... Help! :)
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on October 31, 2018, 02:27:43 AM
In REALbasic 5.5.5 a socket can continue to exist even when the connection is closed and all references to the socket are gone. (The garbage collection is implemented as reference counted pointers, so as soon as there are no references to your object anymore, it's automatically deleted, except for sockets.) See the documentation about sockets. This is the only way that it can cause memory leaks.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on October 31, 2018, 05:28:29 AM
Thank you very much for your input! Will look into this tonight.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on November 09, 2018, 09:36:16 AM
There's also a way in which YOU can cause a memory leak, but since your program used to work normally in REALbasic 2005, this probably didn't happen in REALbasic 5.5.5.

Code: [Select]
App.Open:
Sub Open()
  dim a as Thing
  dim b as Thing
  dim c as Thing
 
  a=new Thing
  b=new Thing
  c=new Thing
 
  a.m_next=b
  b.m_next=a
  c.m_next=c
End Sub

App.Close:
Sub Close()
  #if DebugBuild then
    DebugDumpObjects("Objects.txt")
  #endif
End Sub
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on November 09, 2018, 09:36:59 AM
There are also so-called memory leaks that can resolve themselves:

Code: [Select]
App.Open:
Sub Open()
  dim a as TimerThing
 
  a=new TimerThing
  a.m_next=a
  a.mode=2
  a.period=1000
  a.enabled=true
End Sub

App.Close:
Sub Close()
  #if DebugBuild then
    DebugDumpObjects("Objects.txt")
  #endif
End Sub

TimerThing.Destructor:
Sub Destructor()
  m_next=nil // place here breakpoint.
End Sub

TimerThing.Action:
Sub Action()
  m_seconds=m_seconds+1
  if m_seconds=10 then
    m_next=nil
  end
End Sub
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Naiw on November 15, 2018, 11:48:23 AM
There's also a way in which YOU can cause a memory leak, but since your program used to work normally in REALbasic 2005, this probably didn't happen in REALbasic 5.5.5.

Code: [Select]
App.Open:
Sub Open()
  dim a as Thing
  dim b as Thing
  dim c as Thing
 
  a=new Thing
  b=new Thing
  c=new Thing
 
  a.m_next=b
  b.m_next=a
  c.m_next=c
End Sub

App.Close:
Sub Close()
  #if DebugBuild then
    DebugDumpObjects("Objects.txt")
  #endif
End Sub

Ie the kind of memory leaks you expect from reference counting.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on November 16, 2018, 08:01:02 AM
Did you try DebugDumpObjects? It should tell you the name of the class of the lost objects. Then you have an idea where to search.

You can also try to make a class dependency graph. This should be a directed graph. If  you have circular references then you should find a loop in the graph.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on January 07, 2019, 12:32:27 PM
I'm very thankful for your imput, but I havent had time to look into this anymore YET. Hopefully there will be some time avaliable this week for some more bug searching :)
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on January 09, 2019, 07:30:43 AM
Is the torrent data a built-in type like string or is it something from a plugin?
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on March 07, 2019, 08:49:49 AM
Is the torrent data a built-in type like string or is it something from a plugin?
A built in type.

Did you try DebugDumpObjects?

I did and got quite a list from it:
Code: [Select]
973619 used in heap1
901 objects
Picture
String
String
String
String
String
String
String
String
Dictionary
InternetHeaders
String
BinaryStream
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
String
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentManagerListeningSocket
TorrentManagerTimer
String
String
String
String
Dictionary
InternetHeaders
TorrentTrackerSocket
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
FolderItem
TorrentFile
TextEncoding
Torrent
FolderItem
WebListeningManager
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
QuitMenuItem
AppleMenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuBar1
MDIWindow
App

I quit the application after downloading 50% of a 500mb torrent. So the leak is quite immence.
But as you mentioned, I do have somewhere to look for looping references.

You can also try to make a class dependency graph. This should be a directed graph. If  you have circular references then you should find a loop in the graph.

How do I do this? Do you have any awesome code examples?

Thanks again for all your help. You're the king! :)
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on March 08, 2019, 07:40:31 AM
I'm surprised that it shows only one class that you defined. Which classes have a data member of type TorrentPeer?

Here's a good example of a class dependency graph:
http://www.cs.sjsu.edu/~pearce/modules/lectures/ood/metrics/Dependency.htm
At the bottom you see a diagram with a loop (B, C and E).
That's what you are expected to find.

I suspect that you derived a class X from socket. TorrentPeer points to X and X points to TorrentPeer. X doesn't show up in the list of memory leaks because a socket is an exception to the rule. The other leaked objects are linked by TorrentPeer and X.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on March 12, 2019, 01:46:16 PM
So I think I found the source of the terrible memory leak. When I backported this piece of code from RB 2006R3 I probably did something horrible, because from what I understand the nested loop in the code below never died, causing an infinite loop.

This piece of code is from the RB2006R3 project file. The problem is that the "exit for" syntax doesn't exist in RB 5.5.5.
OS923: Do you have any tip on how to rewrite this in RB 5.5.5 to properly kill the loop?

Code: [Select]
  dim ii,x as integer
  dim b as MemoryBlock
  dim a as integer
  dim tests(8) as integer
 
  tests(7) = 1 //00000001
  tests(6) = 2 //00000010
  tests(5) = 4 //00000100
  tests(4) = 8 //00001000
  tests(3) = 16 //00010000
  tests(2) = 32 //00100000
  tests(1) = 64 // 01000000
  tests(0) = 128 //10000000
 
  b = NewMemoryBlock(ceil(ubound(pieces)/8)+1)
 
  for ii = 0 to ceil(ubound(pieces)/8)
   
    a = 0
    for x = 0 to 7
      if (ii*8+x) < ubound(pieces) then
        if (pieces(ii*8+x).remaining = 0) then
          a = BitWise.BitOr(a,tests(x))
        end if
      else
        exit for x
        exit for ii
      end if
    next
    b.byte(ii) = a
   
  next
 
  return b
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on March 13, 2019, 06:49:11 AM
If this causes the memory leak, then "pieces", whatever that may be, should show up in the list made by DebugDumpObjects, except if they have a circular reference.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on March 13, 2019, 07:04:12 AM
Code: [Select]
a=a+tests(x)will be faster than
Code: [Select]
a = BitWise.BitOr(a,tests(x))
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on March 13, 2019, 08:14:39 AM
Code: [Select]
a=a+tests(x)will be faster than
Code: [Select]
a = BitWise.BitOr(a,tests(x))

Thanks! Do you have any idea on how to cleanly exit the loop(s) since I can't do "exit for x" and "exit for ii" in RB 5.5.5?
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on March 14, 2019, 01:29:49 PM
Do you have any idea on how to cleanly exit the loop(s) since I can't do "exit for x" and "exit for ii" in RB 5.5.5?

Figured out this one myself. Another question. I'm trying to sort an integer with another integer. In newer versions of RealBasic I can use the "sortwith" syntax, but RB 5.5.5 doesn't have that.

The syntax I used to use was "Integer1.SortWith Integer2". This doesn't work anymore for obvious reasons.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on March 15, 2019, 05:03:38 AM
Code: [Select]
a=a+tests(x)will be faster than
Code: [Select]
a = BitWise.BitOr(a,tests(x))

Thanks! Do you have any idea on how to cleanly exit the loop(s) since I can't do "exit for x" and "exit for ii" in RB 5.5.5?
If the "pieces" array changes while your code is executed in a thread, then you can exit a loop by assigning one or two large values, otherwise exit will never happen and then you can write it like in the attachment. I tried it and it works.

You can also import BitSet with a declare statement. This is a slow function in C++, but it might be OK for a REALbasic program.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on March 15, 2019, 05:12:01 AM
Another question. I'm trying to sort an integer with another integer. In newer versions of RealBasic I can use the "sortwith" syntax, but RB 5.5.5 doesn't have that.

The syntax I used to use was "Integer1.SortWith Integer2". This doesn't work anymore for obvious reasons.
From the REALbasic 5.5.5 Language Reference:
Quote
Notes The Sort method works with Integer, string, single, and double arrays only. It accepts
only one-dimensional arrays.
The fastest solution is probably to define a class X that contains integers i1 and i2 and a "next" of type X. For every i make a new X, place Integer1(i) in i1 and Integer2(i) in i2 and set next to nil and then append it to an array. Now perform mergesort on the array.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on March 18, 2019, 03:30:56 AM
It has to be b.byte(i)=a in my example.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on March 18, 2019, 03:34:28 AM
You can also write a plugin to extend MemoryBlock with BitSet, BitClr etc.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on September 18, 2019, 12:54:36 AM
Still struggling with this. If anyone would like to have a look at my RB project and code, they are more than welcome.
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: Knezzen on September 18, 2019, 08:30:32 AM
OS923: I can't reply to your PM. Look over your settings and hit me up with another message and I'll try again :)

Thanks!
Title: Re: Terrible memory leak in REALbasic 5.5.5
Post by: OS923 on September 20, 2019, 09:04:12 AM
Done.