ElkArte Community

Elk Development => Bug Reports => Exterminated Bugs => Topic started by: badmonkey on August 26, 2017, 05:15:25 pm

Title: Out of memory on manage attachments page
Post by: badmonkey on August 26, 2017, 05:15:25 pm
One of my sites has 150,000 plus attachments in 5 directories.  It runs OOM when trying to reach attachment settings or folders in the ACP.  Could that be addressed here?
Title: Re: Out of memory on manage attachments page
Post by: emanuele on August 27, 2017, 05:06:12 am
@badmonkey that sounds odd, I'm not sure why it should have such a problem. :-\
Title: Re: Out of memory on manage attachments page
Post by: badmonkey on August 27, 2017, 02:28:14 pm
If it's unusual not sure how to troubleshoot it?  Raising the php memory limit to 2G allows the interface to work.  If it is indeed unusual, does anyone else using elk have attachment directories as large?
Title: Re: Out of memory on manage attachments page
Post by: Ruth on August 27, 2017, 02:54:28 pm
Not this large, Badmonkey.  We have 66.790 attachments with 3.827.119 KB
Title: Re: Out of memory on manage attachments page
Post by: emanuele on August 27, 2017, 03:46:23 pm
I split the topic so the discussion is easier to follow.

On which page exactly goes out of memory? (Just to be sure.)
Title: Re: Out of memory on manage attachments page
Post by: badmonkey on August 27, 2017, 07:02:19 pm
Quote from: emanuele – I split the topic so the discussion is easier to follow.

On which page exactly goes out of memory? (Just to be sure.)
 Thanks for addressing it eman.   ;)



ACP > Attachment Directories

and

ACP > Attachment Settings > Manage Attachment Directories

both result in 500 errors due to OOM.  I said 2G memory limit will cover it, and it will, but don't know the EXACT limit that will work.   :-X



Title: Re: Out of memory on manage attachments page
Post by: Spuds on August 27, 2017, 07:23:45 pm
Is that 1.0 or 1.1 (I'm not sure its any different but just want to be sure)
Title: Re: Out of memory on manage attachments page
Post by: live627 on August 27, 2017, 07:31:10 pm
Could possibly be this code

Code: [Select]
	// Count the files with a glob, easier and less time consuming
$num_files = count(glob($dir . '/*.elk'));

EDIT: Tested with 180k files in one dir, down to 7MB memory from 40+MB. In ./sources/subs/ManageAttachments.subs.php, find the above code and replace with
Code: [Select]
	// Count the files with an iterator for a smaller memory footprint.
$num_files = count(new GlobIterator($dir . '/*.elk'));

EDIT2: This is 1.0.
Title: Re: Out of memory on manage attachments page
Post by: badmonkey on August 27, 2017, 07:48:46 pm
The forum in question uses 1.1.   8)
Title: Re: Out of memory on manage attachments page
Post by: live627 on August 27, 2017, 08:34:32 pm
https://github.com/elkarte/Elkarte/blob/c58a9e6ecf9ea836d77d02a2ff458b7413b755bf/sources/subs/ManageAttachments.subs.php#L1282

Code: [Select]
	$num_files = count(iterator_to_array($glob));

replace wth

Code: [Select]
	$num_files = $glob->count();
Title: Re: Out of memory on manage attachments page
Post by: Frenzie on August 28, 2017, 09:00:03 am
It looks like you can't backport that fix to 1.0.x because it requires PHP 5.3 or higher. :)
Title: Re: Out of memory on manage attachments page
Post by: Spuds on August 28, 2017, 09:03:47 am
Thanks for the debug  :D ... I added this to the 1.1 tracker so we don't miss this in the final.
Title: Re: Out of memory on manage attachments page
Post by: emanuele on August 28, 2017, 09:18:18 am
Thanks Spuds!
I wanted to do it this morning but I was already a bit late. O:-)