ElkArte Community

Elk Development => Feature Discussion => Topic started by: emanuele on August 26, 2017, 06:38:36 am

Title: A smarter way to handle directories (and URLs)
Post by: emanuele on August 26, 2017, 06:38:36 am
Okay, based on server transfers experiences read here and there, I tend to think the current way we are doing it is overkill.

Recap: at the moment we have a very flexible system that has the advantage of letting anyone define where almost each and every bit of ElkArte goes (e.g. "main board", sources, themes, smiley, attachments, maybe something else).
Of course, with great flexibility comes great responsibility (and complexity): each and every time any admin moves to a different server he has to re-configure at least... from 6 to 10 directories.

So, my proposal would be:

and that's all that comes to my mind right now, there may be more.
Similar approach would be for URLs: we should set the "main" URL for themes and then deal with relative ones. And smiley, being a child of the default theme, would inherit the same URL.

What do you think?
Title: Re: A smarter way to handle directories (and URLs)
Post by: Frenzie on August 26, 2017, 06:54:41 am
Quote(maybe with the option to have theme-specific smiley sets, that is likely more useful than let the users pick their preferred set)
I don't particularly care where smilies are stored, but some people definitely appreciate being able to pick theme and smiley set separately.
Title: Re: A smarter way to handle directories (and URLs)
Post by: Ruth on August 26, 2017, 07:00:01 am
I don't understand everything you have written here, Emanuele - but in summary it sounds pretty good to me.

Sorry,  O:-)  if my question is too stupid:

What would be with very old and migrated forums like ours? It is now more than 11 years old - and for example the attachments from our first 3 years are somehow "different" to newer ones.

Will this work for us, what you have written here?
Or is it a way to make things easier, specially for migrated forums?
Title: Re: A smarter way to handle directories (and URLs)
Post by: Spuds on August 26, 2017, 08:33:26 am
Don't forget about avatars ... the custom dir stuff should be removed and simply have a sane directory spot for them. 

Agree on the overkill of the attachments, all the date, year, month separation seemed good on the surface but its really a maintenance and site move problem, even repair_settings does not deal with it 100% properly.  I'd say a base dir and then separate by year ... drop the size of directory or number of files, most of that was probably for fat32 file systems.
Title: Re: A smarter way to handle directories (and URLs)
Post by: Frenzie on August 26, 2017, 08:54:54 am
If memory doesn't betray me then performance might start to degrade on ext2 when you start exceeding 100,000 files or so.[1] But please don't pin me down on that. :P FTP implications might be more relevant.

@kode54 is an admin on hydrogenaudio, correct? How are things with the attachments over there?
And FAT32 can't store more than what, 60,000? Who the heck wouldn't use NTFS for a Windows server even 15 years ago? >_>
Title: Re: A smarter way to handle directories (and URLs)
Post by: Spuds on August 26, 2017, 09:43:08 am
Last I heard kode54 had a parting of the ways with hydrogenaudio

No harm in leaving the settings, they are just ones I've never had reason to use, but then I only deal with around 10k files in each "year" sub-dir.   You may be  more likely to reach a performance issue with avatars since they are all in one directory and are fetched more frequently since they are shown in a lot of areas.
Title: Re: A smarter way to handle directories (and URLs)
Post by: Frenzie on August 26, 2017, 11:18:39 am
Oh, that's too bad. It's a site I sometimes like to check before buying music on Qobuz or CD.[1] But I don't think I have an account even though I've probably glanced at it on and off since '04.
Stuff like http://dr.loudness-war.info/ and https://hydrogenaud.io/index.php/topic,89818.0.html tl;dr don't buy UMG (which owns Decca, DG, and EMI)
Title: Re: A smarter way to handle directories (and URLs)
Post by: emanuele on August 26, 2017, 12:55:31 pm
I don't know the specifics of the file systems, I remember few times FTP hitting a 10k limit that was annoying as hell (i.e. it was not possible to see more than 10k files in the client).

Either by year, of by simple number it would be fine with me (I would probably prefer number though).

Quote from: Spuds – Don't forget about avatars ... the custom dir stuff should be removed and simply have a sane directory spot for them. 
Right, that too.
Title: Re: A smarter way to handle directories (and URLs)
Post by: Frenzie on August 26, 2017, 02:52:50 pm
By year is useful information all by itself without needing anything else, but it's only a very mild preference.
Title: SPLIT: Out of memory on manage attachments page
Post by: emanuele on August 27, 2017, 03:44:29 pm
The discussion about OOM started by badmonkey has been moved to Bug Reports (http://www.elkarte.net/community/index.php?board=2.0) - http://www.elkarte.net/community/index.php?topic=4637.0
Title: Re: A smarter way to handle directories (and URLs)
Post by: emanuele on August 31, 2017, 05:31:02 am
Quote from: Ruth – What would be with very old and migrated forums like ours? It is now more than 11 years old - and for example the attachments from our first 3 years are somehow "different" to newer ones.

Will this work for us, what you have written here?
Or is it a way to make things easier, specially for migrated forums?
I missed that one: well, upgrades should deal with all this, so for those running a forum it should be... nothing to change.
Maybe, smiley would need to be reinstalled (or something like that) to adapt to the new position, but nothing too big (hopefully).
Title: Re: A smarter way to handle directories (and URLs)
Post by: radu81 on August 31, 2017, 09:21:41 am
I store attachments by year and month and I like this option. I still have to set automatic backups also for files and not only for database. IIRC Spuds writed a script to do automatic backups, and having attachments divided by month could be useful for this purpose. While this could be useful for backups, I have to admit I always had some problems restoring my forum to other host or other location. Last time I tried to set a test forum I failed... I know @emanuele‍ I still have to send you the login details to have a look at that install  :-[
Title: Re: A smarter way to handle directories (and URLs)
Post by: Jorin on August 31, 2017, 09:33:51 am
You store the forum attachments by year and month? Sounds interesting, but how does it work?
Title: Re: A smarter way to handle directories (and URLs)
Post by: radu81 on August 31, 2017, 09:55:29 am
there is a setting in Admin -> Forum -> Attachments and avtars -> Attachment Settings
Title: Re: A smarter way to handle directories (and URLs)
Post by: ahrasis on August 31, 2017, 06:23:30 pm
I don't really care that much so long all transitions from the old version are smooth, efficient and fully covered in all possible aspects.

Don't forget about CDN. I hope that that all default static contents for Elkarte may be served from a CDN as an option in the future.