Author Topic: Terrible memory leak in REALbasic 5.5.5 (Read 11264 times)

Knezzen · « **on:** October 20, 2018, 01:58:47 PM »

Hi!

Just finished backporting MacTorrent to REALbasic 5.5.5 just to be able to develop the application in Mac OS 9 and not just use it there.

Now to the problem. The back ported code leaks terribly when ran (after the application is built etc).
It uses up as much memory as the torrent data that is written to disk is big, so the problem lays somewhere in the part of the code that manages the torrent data (I think).
The frustrating part is that the code runs well without memory leaks in REALbasic 2005 and newer (without rewriting anything), so it's really hard to find where the memory leak comes from. REALbasic 2005 and newer probably uses some other kind of garbage collecting that is more effective. Something that RB 5.5.5 doesn't do.

Would be wonderful if some REALbasic guru could lend me a helping hand on the matter. I want to get new versions of MacTorrent out the door so to say, and I want to do that using RB 5.5.5.

So... Help!

OS923 · « **Reply #1 on:** October 31, 2018, 02:27:43 AM »

In REALbasic 5.5.5 a socket can continue to exist even when the connection is closed and all references to the socket are gone. (The garbage collection is implemented as reference counted pointers, so as soon as there are no references to your object anymore, it's automatically deleted, except for sockets.) See the documentation about sockets. This is the only way that it can cause memory leaks.

Knezzen · « **Reply #2 on:** October 31, 2018, 05:28:29 AM »

Thank you very much for your input! Will look into this tonight.

OS923 · « **Reply #3 on:** November 09, 2018, 09:36:16 AM »

There's also a way in which YOU can cause a memory leak, but since your program used to work normally in REALbasic 2005, this probably didn't happen in REALbasic 5.5.5.

Code: [Select]

App.Open:
Sub Open()
  dim a as Thing
  dim b as Thing
  dim c as Thing
  
  a=new Thing
  b=new Thing
  c=new Thing
  
  a.m_next=b
  b.m_next=a
  c.m_next=c
End Sub

App.Close:
Sub Close()
  #if DebugBuild then 
    DebugDumpObjects("Objects.txt")
  #endif
End Sub

OS923 · « **Reply #4 on:** November 09, 2018, 09:36:59 AM »

There are also so-called memory leaks that can resolve themselves:

Code: [Select]

App.Open:
Sub Open()
  dim a as TimerThing
  
  a=new TimerThing
  a.m_next=a
  a.mode=2
  a.period=1000
  a.enabled=true
End Sub

App.Close:
Sub Close()
  #if DebugBuild then 
    DebugDumpObjects("Objects.txt")
  #endif
End Sub

TimerThing.Destructor:
Sub Destructor()
  m_next=nil // place here breakpoint.
End Sub

TimerThing.Action:
Sub Action()
  m_seconds=m_seconds+1
  if m_seconds=10 then
    m_next=nil
  end
End Sub

Naiw · « **Reply #5 on:** November 15, 2018, 11:48:23 AM »

Quote from: OS923 on November 09, 2018, 09:36:16 AM

There's also a way in which YOU can cause a memory leak, but since your program used to work normally in REALbasic 2005, this probably didn't happen in REALbasic 5.5.5.

Code: [Select]
App.Open: Sub Open() dim a as Thing dim b as Thing dim c as Thing a=new Thing b=new Thing c=new Thing a.m_next=b b.m_next=a c.m_next=c End Sub App.Close: Sub Close() #if DebugBuild then DebugDumpObjects("Objects.txt") #endif End Sub

Ie the kind of memory leaks you expect from reference counting.

OS923 · « **Reply #6 on:** November 16, 2018, 08:01:02 AM »

Did you try DebugDumpObjects? It should tell you the name of the class of the lost objects. Then you have an idea where to search.

You can also try to make a class dependency graph. This should be a directed graph. If you have circular references then you should find a loop in the graph.

Knezzen · « **Reply #7 on:** January 07, 2019, 12:32:27 PM »

I'm very thankful for your imput, but I havent had time to look into this anymore YET. Hopefully there will be some time avaliable this week for some more bug searching

OS923 · « **Reply #8 on:** January 09, 2019, 07:30:43 AM »

Is the torrent data a built-in type like string or is it something from a plugin?

Knezzen · « **Reply #9 on:** March 07, 2019, 08:49:49 AM »

Quote from: OS923 on January 09, 2019, 07:30:43 AM

Is the torrent data a built-in type like string or is it something from a plugin?

A built in type.

Quote from: OS923 on November 16, 2018, 08:01:02 AM

Did you try DebugDumpObjects?

I did and got quite a list from it:

Code: [Select]

973619 used in heap1
901 objects
Picture
String
String
String
String
String
String
String
String
Dictionary
InternetHeaders
String
BinaryStream
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
String
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentPeer
TorrentManagerListeningSocket
TorrentManagerTimer
String
String
String
String
Dictionary
InternetHeaders
TorrentTrackerSocket
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
Piece
FolderItem
TorrentFile
TextEncoding
Torrent
FolderItem
WebListeningManager
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuItem
QuitMenuItem
AppleMenuItem
MenuItem
MenuItem
MenuItem
MenuItem
MenuBar1
MDIWindow
App

I quit the application after downloading 50% of a 500mb torrent. So the leak is quite immence.
But as you mentioned, I do have somewhere to look for looping references.

Quote from: OS923 on November 16, 2018, 08:01:02 AM

You can also try to make a class dependency graph. This should be a directed graph. If you have circular references then you should find a loop in the graph.

How do I do this? Do you have any awesome code examples?

Thanks again for all your help. You're the king!

OS923 · « **Reply #10 on:** March 08, 2019, 07:40:31 AM »

I'm surprised that it shows only one class that you defined. Which classes have a data member of type TorrentPeer?

Here's a good example of a class dependency graph:
http://www.cs.sjsu.edu/~pearce/modules/lectures/ood/metrics/Dependency.htm
At the bottom you see a diagram with a loop (B, C and E).
That's what you are expected to find.

I suspect that you derived a class X from socket. TorrentPeer points to X and X points to TorrentPeer. X doesn't show up in the list of memory leaks because a socket is an exception to the rule. The other leaked objects are linked by TorrentPeer and X.

Knezzen · « **Reply #11 on:** March 12, 2019, 01:46:16 PM »

So I think I found the source of the terrible memory leak. When I backported this piece of code from RB 2006R3 I probably did something horrible, because from what I understand the nested loop in the code below never died, causing an infinite loop.

This piece of code is from the RB2006R3 project file. The problem is that the "exit for" syntax doesn't exist in RB 5.5.5.
OS923: Do you have any tip on how to rewrite this in RB 5.5.5 to properly kill the loop?

Code: [Select]

  dim ii,x as integer
  dim b as MemoryBlock
  dim a as integer
  dim tests(8) as integer
  
  tests(7) = 1 //00000001
  tests(6) = 2 //00000010
  tests(5) = 4 //00000100
  tests(4) = 8 //00001000
  tests(3) = 16 //00010000
  tests(2) = 32 //00100000
  tests(1) = 64 // 01000000
  tests(0) = 128 //10000000
  
  b = NewMemoryBlock(ceil(ubound(pieces)/8)+1)
  
  for ii = 0 to ceil(ubound(pieces)/8)
    
    a = 0
    for x = 0 to 7
      if (ii*8+x) < ubound(pieces) then
        if (pieces(ii*8+x).remaining = 0) then
          a = BitWise.BitOr(a,tests(x))
        end if
      else
        exit for x
        exit for ii
      end if
    next
    b.byte(ii) = a
    
  next
  
  return b

OS923 · « **Reply #12 on:** March 13, 2019, 06:49:11 AM »

If this causes the memory leak, then "pieces", whatever that may be, should show up in the list made by DebugDumpObjects, except if they have a circular reference.

OS923 · « **Reply #13 on:** March 13, 2019, 07:04:12 AM »

Code: [Select]

a=a+tests(x)will be faster than

Code: [Select]

a = BitWise.BitOr(a,tests(x))

Knezzen · « **Reply #14 on:** March 13, 2019, 08:14:39 AM »

Quote from: OS923 on March 13, 2019, 07:04:12 AM

Code: [Select]
a=a+tests(x)will be faster than
Code: [Select]
a = BitWise.BitOr(a,tests(x))

Thanks! Do you have any idea on how to cleanly exit the loop(s) since I can't do "exit for x" and "exit for ii" in RB 5.5.5?

Knezzen · « **Reply #15 on:** March 14, 2019, 01:29:49 PM »

Quote from: Knezzen on March 13, 2019, 08:14:39 AM

Do you have any idea on how to cleanly exit the loop(s) since I can't do "exit for x" and "exit for ii" in RB 5.5.5?

Figured out this one myself. Another question. I'm trying to sort an integer with another integer. In newer versions of RealBasic I can use the "sortwith" syntax, but RB 5.5.5 doesn't have that.

The syntax I used to use was "Integer1.SortWith Integer2". This doesn't work anymore for obvious reasons.

OS923 · « **Reply #16 on:** March 15, 2019, 05:03:38 AM »

Quote from: Knezzen on March 13, 2019, 08:14:39 AM

Quote from: OS923 on March 13, 2019, 07:04:12 AM
Code: [Select]
a=a+tests(x)will be faster than
Code: [Select]
a = BitWise.BitOr(a,tests(x))

Thanks! Do you have any idea on how to cleanly exit the loop(s) since I can't do "exit for x" and "exit for ii" in RB 5.5.5?

If the "pieces" array changes while your code is executed in a thread, then you can exit a loop by assigning one or two large values, otherwise exit will never happen and then you can write it like in the attachment. I tried it and it works.

You can also import BitSet with a declare statement. This is a slow function in C++, but it might be OK for a REALbasic program.

OS923 · « **Reply #17 on:** March 15, 2019, 05:12:01 AM »

Quote from: Knezzen on March 14, 2019, 01:29:49 PM

Another question. I'm trying to sort an integer with another integer. In newer versions of RealBasic I can use the "sortwith" syntax, but RB 5.5.5 doesn't have that.

The syntax I used to use was "Integer1.SortWith Integer2". This doesn't work anymore for obvious reasons.

From the REALbasic 5.5.5 Language Reference:

Quote

Notes The Sort method works with Integer, string, single, and double arrays only. It accepts
only one-dimensional arrays.

The fastest solution is probably to define a class X that contains integers i1 and i2 and a "next" of type X. For every i make a new X, place Integer1(i) in i1 and Integer2(i) in i2 and set next to nil and then append it to an array. Now perform mergesort on the array.

OS923 · « **Reply #18 on:** March 18, 2019, 03:30:56 AM »

It has to be b.byte(i)=a in my example.

OS923 · « **Reply #19 on:** March 18, 2019, 03:34:28 AM »

You can also write a plugin to extend MemoryBlock with BitSet, BitClr etc.

[News, Information & Feedback]	What's your opinion on the "like" button? by ssp3	Today at 03:55:18 PM
[Software]	New version of the Hotline Connect client HotStuff released! by Protools5LEGuy	Today at 03:22:37 PM
[News, Information & Feedback]	254 guests 4 users online. by Protools5LEGuy	Today at 03:16:47 PM
[Software]	OS 9 ChatGPT client by Protools5LEGuy	Today at 02:38:52 PM
[News, Information & Feedback]	Thoughts on the future of Mac OS 9 Lives by Knezzen	Today at 12:47:09 PM
[New Member Welcome]	G'day folks :)) by Knezzen	Today at 12:36:51 PM
[Community Marketplace]	Elusive EVB-002-3 "Bribge" IDE/SATA Adapter offered to members @ $3.99 ! by aBc	Today at 09:48:05 AM
[Community Marketplace]	FS/FA Mac mini G4 Upgraded with SSD & Pre-installed Mac OS 9 by davecom	Yesterday at 05:07:31 PM
[Video Cards, Monitors & Displays]	Issues with GeForce 4MX - iMac G4 - M6498 (EMC 1956) by Greystash	Yesterday at 05:06:30 PM
[Software]	Blokz RAVE released by MacTron	Yesterday at 12:40:26 PM
[Logic Audio by Emagic]	XSKey Emulator for Logic 5 & 6 by AtariMan	Yesterday at 12:23:22 PM
[Hardware]	Mac mini G4 Screws by davecom	Yesterday at 11:55:05 AM

News:

Author Topic: Terrible memory leak in REALbasic 5.5.5 (Read 11264 times)