ElkArte Community

Elk Development => Feature Discussion => Topic started by: kucing on September 03, 2013, 03:58:09 pm

Title: CDN Support
Post by: kucing on September 03, 2013, 03:58:09 pm
Is there any plan for supporting CDN?

The free CDNs : CloudFlare and Incapsula.
The paying CDNs : MaxCDN, CloudFront and CDN77.

Or I just misunderstood about how to setup CDN in CMSes (not limited to ElkArte)?
Title: Re: CDN Support
Post by: Trekkie101 on September 03, 2013, 04:49:33 pm
They work a few stages back at the DNS stage, by default, this software will work with ut.
Title: Re: CDN Support
Post by: Spuds on September 03, 2013, 04:50:21 pm
The only issue that I know of with CDN's is that the $_SERVER['REMOTE_ADDR'] will come in as the IP if the CDN for everyone, so that makes IP tracking / banning / geo location etc mostly unusable ...

CloudFlare sends in the original IP in $SERVER['HTTP_CF_CONNECTING_IP'] (but you should not just use that when found, you need to make sure that the REMOTE_ADDR one is from a CloudFlare range)

Will we support that, or is that addon territory?   I'm not sure ... I'm thinking addon as I'd imagine each CDN will have its own IP needs so trying to address that in core would be a source of continual maintenance.
Title: Re: CDN Support
Post by: kucing on September 03, 2013, 05:14:09 pm

@Trekkie101 and @Spuds : Yeah, that is how CloudFlare and Incapsula do their CDN setup, and I failed badly when trying to use them (not on the same time).

When I'm using MaxCDN there is a difference in how to setup, I don't need to change my NS to them. I just need to add one or more CNAME record to them. I found easier that way.

I agree with you @Spuds this is addon material, because not everyone will use CDN. But I believe for the free ones (CloudFlare and Incapsula) many would request them in the future.

Thanks for hearing me out.
Title: Re: CDN Support
Post by: Pandos on September 03, 2013, 05:34:58 pm
There are more issues with CDN'S if the real IP is not properly passed through the server or CDN:

Session: Users will always get: s"ession expired".
Attachements: Upload due to session problem impossible (same error message)
Search: by Google PSS search is messed up and throws error 414: URL is too long (due to given parameters)

Quick and dirty solution:
add the following to index.php:
Code: [Select]
$_SERVER['REMOTE_ADDR'] = $_SERVER['HTTP_X_FORWARDED_FOR'];

Search: only solution to set the search to subdomain which is not delivered by CDN.
Title: Re: CDN Support
Post by: emanuele on December 08, 2013, 10:16:41 am
/me tracks this to remember to check if there is a way to hook and change what is necessary without having to touch the code.
Title: Re: CDN Support
Post by: ahrasis on August 24, 2017, 04:38:08 am
I am extending the discussion about CDN since the topic is already opened in here. Basically in EU Cookie Notification (http://www.elkarte.net/community/index.php?topic=910.msg33360#msg33360) thread, I mentioned about testing this two months ago.  ;)

I was basically thinking if all EA updated (default) js and css can be put into a CDN like what was used by Insites Cookie Consent (https://cookieconsent.insites.com/download/) which is basically using a combination from their github at https://github.com/cdnjs/cdnjs and call them via cloudflare via this code:
Code: [Select]
<link rel="stylesheet" type="text/css" href="//cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.css" />
<script src="//cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.js"></script>
My testing was different however, in the sense that I were creating https://cdn.sch.my where all images, js and css files for https://elkarte.sch.my were supposedly kept but actually in the same path but only using different domain name. My test went missing but I think recreating them is not that difficult since I was just using the repair_settings.php to do all that replacement together with multi-tenancy addon.[1]

At the end, I am not sure why I was doing all these at the first place but may be I'll recall them when I started to do it again.  :D
I think there could be minor changes which I might have forgotten. O:-)
Title: Re: CDN Support
Post by: emanuele on August 25, 2017, 06:23:24 pm
The only issue that I know of with CDN's is that the $_SERVER['REMOTE_ADDR'] will come in as the IP if the CDN for everyone, so that makes IP tracking / banning / geo location etc mostly unusable ...

CloudFlare sends in the original IP in $SERVER['HTTP_CF_CONNECTING_IP'] (but you should not just use that when found, you need to make sure that the REMOTE_ADDR one is from a CloudFlare range)

Will we support that, or is that addon territory?   I'm not sure ... I'm thinking addon as I'd imagine each CDN will have its own IP needs so trying to address that in core would be a source of continual maintenance.
If the various CDN always use the $SERVER variable, we could just add a setting to specify the index holding the IP that we should use.
The only part I'm not sure about, is if having it in $modSettings is early enough to deal with out internals or we need to know the index before that.
Title: Re: CDN Support
Post by: ahrasis on August 26, 2017, 02:26:30 am
It is being tracked for years. Hopefully, this could be resolved / added as features.
Title: Re: CDN Support
Post by: emanuele on August 26, 2017, 09:06:25 am
Well, my proposal stands only if the condition is valid. Now someone has to verify it. xD
Title: Re: CDN Support
Post by: Spuds on August 26, 2017, 02:53:46 pm
I'm not sure if all of the services send the original IP in the connection.

Doing some more looking, Cloudflare and Incapsula set $_SERVER['HTTP_X_FORWARDED_FOR'] which seems to be the industry standard for these services.  So a simple setting to enable CDN may be simply along the lines of.
Code: [Select]
if (isset($_SERVER['HTTP_X_FORWARDED_FOR']))
{
$_SERVER['REMOTE_ADDR'] = $_SERVER['HTTP_X_FORWARDED_FOR'];
}

That is somewhat unique to CDN's where you change your DNS to point to them and they choose the assets.  If you are using, I'll say traditional CDN, such as cloudfront (amazon) then its not a problem since you are simply pointing certain assets (js/css/etc) to be fetched from the CDN URL. 

For a traditional setup we would need the ability to point JS, CSS, Theme assests (images), avatars, and maybe attachment URL's, for the borg cloud style CDN simply a check the HTTP_X_FORWARDED_FOR header maybe all that is needed.
Title: Re: CDN Support
Post by: ahrasis on August 27, 2017, 01:14:29 am
Haha... I cannot verify anything regarding this. I don't even know how it is done to to be done properly.
Title: Re: CDN Support
Post by: emanuele on August 27, 2017, 11:07:02 am
Nothing to verify, my proposal doesn't work because the request cleaning comes before having modSettings populated, so it is too early.
Title: Re: CDN Support
Post by: kucing on September 23, 2017, 01:01:52 am
If using CloudFlare ( I do) I just ignore CloudFlare IPs: https://www.cloudflare.com/ips/ and take the needed header: https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-Cloudflare-handle-HTTP-Request-headers-

But if using conventional CDN (pull zone) like KeyCDN/MaxCDN I think Elkarte doesn't support it yet?
Title: Re: CDN Support
Post by: ahrasis on September 23, 2017, 05:28:57 am
Still I do not understand this CDN that much. But as said:
I was basically thinking if all EA updated (default) js and css can be put into a CDN like what was used by Insites Cookie Consent (https://cookieconsent.insites.com/download/) which is basically using a combination from their github at https://github.com/cdnjs/cdnjs and call them via cloudflare via this code:
Code: [Select]
<link rel="stylesheet" type="text/css" href="//cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.css" />
<script src="//cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.js"></script>
I was thinking if we are to test this cdn idea, we could participate and create a branch of elkarte default static files in there?

Just a thought...
Title: Re: CDN Support
Post by: Spuds on September 23, 2017, 02:38:19 pm
That's correct .. I do have an un-released addon which points asset (JS/CSS/Images/etc) to a CDN of your choice.  I use it for Amazon's CDN cloudfront service.   Should work with other conventional CDN's as well.  

Looks like MaxCDN has been soaked up by StackPath CDN and its, IMO, a bit pricey for what most forums use (looks like min $20 a month?)
Title: Re: CDN Support
Post by: kucing on September 23, 2017, 02:51:59 pm
Looks like MaxCDN has been soaked up by StackPath CDN and its, IMO, a bit pricey for what most forums use (looks like min $20 a month?)

I have a free sponsored account from 4  years ago. :P Consuming +- 100GB per month, but only for one domain.

if you want a cheap CDN I can recommend BunnyCDN: https://bunnycdn.com/ for me its the keycdn successor.
Title: Re: CDN Support
Post by: ahrasis on January 06, 2018, 04:57:25 am
That's correct .. I do have an un-released addon which points asset (JS/CSS/Images/etc) to a CDN of your choice.  I use it for Amazon's CDN cloudfront service.   Should work with other conventional CDN's as well.
Is the addon ready for testing @Spuds? I have been reading about this RawGit (https://rawgit.com/) and it seems good to be made as an additional options for those who are trying to implement CDN for their static contents. I'd like to test it with this RawGit using ElkArte github repo as it also uses StackPath CDN. I currently testing my own created CDN, sort of. [1]
I am testing my own cdn at https://cdn.sch.my via my ISPConfig Control Panel and pointing it to the same path of my master domain with MultiTenancy addon (via $buffer).
Title: Re: CDN Support
Post by: Spuds on April 01, 2018, 03:05:53 am
Had not seen the rawgit before ... very interesting !

I'll have to hunt around for the CDN code I have, I'm not sure I ever packaged it up for use  :-*
Title: Re: CDN Support
Post by: ahrasis on May 11, 2018, 04:50:41 pm
I am currently using repair_settings.php  to change to a cdn url but currently only avatar, smileys and images folder are covered. So now I am looking for a way to change css and script to use the cdn url as well.

How do I change the settings for these two? Can our repair_settings.php be modified to include both of their url? Or do I need to use an addon?