Posted by Tonic in SWF Studio V3 on Jul 22 2008, 10:04 am

Hello,

i'm french, so excuse me if I make fault. :)

I develop an application who use the FileSys plugin, for read, and write datas in xml.

I use regulary the read and write methods, and I was able to notice a leak of memory.

I have create an application for test.

The application launches a reading and a writing in a file every 500 milliseconds.


<?xml version="1.0" encoding="utf-8"?>
<mx:Application 
   xmlns:mx="http://www.adobe.com/2006/mxml" 
   layout="absolute" 
   xmlns:debug="com.yelloware.debug.*"
   creationComplete="init()"
>   
   <mx:Script>
      <![CDATA[
         import flash.utils.setInterval;
         
         private var index : Number;
         
         private function init():void
         {
            ssCore.init();
            index = 0;
            setInterval(addText,500);   
         }
         
         private function addText():void
         {
            var filePath : String = "monFichier.xml";
            var exist : Object = ssCore.FileSys.fileExists({path:filePath},{sync:true});         
            var str : String = "";
            var xml : XML;
            
            if(exist.result=="TRUE")
            {
               var strXML : Object = ssCore.FileSys.readFile({path:filePath},{sync:true});   
               str = strXML.result;
               str += "\t<value index=\""+index+"\">index_"+index+"</value>\n";               
            }else
            {
               str = "<?xml version=\"1.0\" encoding=\"utf-8\"?>";
               str += "\t<value index=\""+index+"\">index_"+index+"</value>\n";
            }         
            
            index++;            
            
            var response : Object = ssCore.FileSys.writeToFile({path:filePath,data:str},{sync:true});            
         }
      ]]>
   </mx:Script>   
</mx:Application>


Result of the test:

00 min - Memory : 32 Mb
05 min - Memory : 54 Mb
10 min - Memory : 120 Mb
15 min - Memory : 210 Mb
20 min - Memory : 330 Mb
25 min - Memory : 470 Mb
30 min - Memory : 650 Mb !

We can see that the application use a lot of memory after few minutes.
(Processor use only 4/5 % of ressurce)

I use not correctly the methods read and write?
Has it a means to close the file or free the memory?

Thanks :)


Posted by northcode in SWF Studio V3 on Jul 22 2008, 10:21 am

We'll have a look at this, but I wouldn't be using XML to log data because it forces you to read and write your entire file instead of doing something smarter like using FileSys.appendToFile. I would switch to a text file or CSV file for logging data or keep the XML in memory and write it out once at the end.

Does the memory get freed up if you minimize and restore your application? If so you're seeing Flash garbage collection issues. Grant Skinner has a good article on Resource Management in AS3 that says you can force Flash to perform garbage collection like this in AS3:


try {
   new LocalConnection.connect('foo');
   new LocalConnection.connect('foo');
}
catch(e:*) {}


Posted by Tonic in SWF Studio V3 on Jul 22 2008, 10:35 am

Thank you for your response. :)

I have seen the solution of LocalConnection to force the garbage collector, but the resources are not freed.

I have tested with "delete" on objects properties, but it's not change.

For the test, you don't have mean an xml file, you can test with a text file, there is the same problem.

Thank you for your tests. :)


Posted by Tonic in SWF Studio V3 on Jul 23 2008, 06:55 am

Hello,

have you tested this code?

thanks :)


Posted by Tonic in SWF Studio V3 on Jul 24 2008, 03:58 am

Up :)


Posted by northcode in SWF Studio V3 on Jul 24 2008, 09:21 am

Calm down, we haven't forgotten, it's on the to-do list :)


Posted by northcode in SWF Studio V3 on Jul 24 2008, 03:33 pm

Okay, we've done some tests and have more info. It's not a problem with any particular FileSys function, but with synchronous calls to plugins that return a lot of data. The memory is leaked faster when bigger chunks of data are returned so it's easier to see, but a little memory will be leaked on each call. In most cases the amount of memory leaked wouldn't be enough to notice (which is why we didn't) but with high frequency requests for lots of data (like you're doing) it can grow quickly. I'm going to create a problem report for this one right now.

If you can change the format of the data you're storing from XML to plain text or CSV data so that you only ever call FileSys.appendToFile, your memory usage will drop dramatically. It's the frequent calls to FileSys.readFile that are consuming all the memory. I'm not recommending that as a permanent solution, but as a workaround until we can plug the leak. However, avoiding all that I/O is also just a better general solution.

problem report: Memory leaked by synchronous plugin calls [165]

edit: added link to problem report


Posted by Tonic in SWF Studio V3 on Jul 25 2008, 04:39 am

Thank you very much for your answer.

Indeed, I made this test, with an asynchronous call of everything methods, and the memory does not move.

- Synchronus : Number of recursion = 3244

00 min - Memory : 32 Mb
05 min - Memory : 54 Mb
10 min - Memory : 120 Mb
15 min - Memory : 210 Mb
20 min - Memory : 330 Mb
25 min - Memory : 470 Mb
30 min - Memory : 650 Mb :( :(

- Asynchronus : Number of recursion = 3407

00 min - Memory : 32 Mb
05 min - Memory : 36 Mb
10 min - Memory : 38 Mb
15 min - Memory : 41 Mb
20 min - Memory : 45 Mb
25 min - Memory : 47 Mb
30 min - Memory : 51 Mb :) :)

Code for asynchronus method :


<?xml version="1.0" encoding="utf-8"?>
<mx:Application 
   xmlns:mx="http://www.adobe.com/2006/mxml" 
   layout="absolute" 
   xmlns:debug="com.yelloware.debug.*"
   creationComplete="init()"
xmlns:controls="com.yelloware.ui.controls.*">
   
   <mx:Script>
      <![CDATA[
         import com.yelloware.system.file.File;
         import mx.utils.ObjectUtil;
         import northcode.utils.ObjectUtils;
         import flash.utils.setTimeout;
         import com.yelloware.debug.Debug;
         import flash.utils.setInterval;
         
         private var index : Number;         
         private var canWrite : Boolean = true;   
         private var filePath : String = "monFichier.xml";
            
         private function init():void
         {
            ssCore.init();
            index = 0;
            setInterval(addText,500);            
         }
         
         private function addText():void
         {
            if(canWrite)
            {
               var exist : Object = ssCore.FileSys.fileExists({path:filePath},{callback:existHandler});
               canWrite = false;
            }else
            {
               Debug.print("On a du sauter un interval : "+index);
            }         
         }
         
         private function existHandler(returnObj : Object, callbackObj : Object, errorObj : Object):void
         {
            var str : String = "";
            
            if(returnObj.result=="TRUE")
            {
               ssCore.FileSys.readFile({path:filePath},{callback:readHandler});   
               
            }else
            {
               str = "<?xml version=\"1.0\" encoding=\"utf-8\"?>";
               str += "\t<value index=\""+index+"\">index_"+index+"</value>\n";
               ssCore.FileSys.writeToFile({path:filePath,data:str},{callback:writeHandler});
            }         
            
            index++;            
         }
         
         private function readHandler(returnObj : Object, callbackObj : Object, errorObj : Object):void
         {
            var str : String = "";
            str = returnObj.result;
            str += "\t<value index=\""+index+"\">index_"+index+"</value>\n";
            ssCore.FileSys.writeToFile({path:filePath,data:str},{callback:writeHandler});
         }
         
         private function writeHandler(returnObj : Object, callbackObj : Object, errorObj : Object):void
         {
            canWrite = true;
         }
         
         
      ]]>
   </mx:Script>   
   <debug:Debug width="100%" height="100%"/>
</mx:Application>


I am going to use this system for my recording.

Still thank you. :clap:


Posted by northcode in SWF Studio V3 on Jul 28 2008, 08:30 pm

We've got a fix for this problem now, it will be part of the next release. I just created a quick test where I repeatedly read a 25MB file every 100ms. I let it run for several minutes and there is no more memory leak. Before the fix, this test chewed up virtual memory at an alarming pace!

Memory leaked by synchronous plugin calls [165]