Skip to main content
Topic: Large file upload fails at 33% (Read 1108 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Large file upload fails at 33%

I have set max attachment file sizes to »no limit» but for some reason when I try to upload 386.79 MB zip file as attachment it seems to stop at 33%. Server guy says there is no limits on the serverside with PHP and such as I understood those can affect beyond EA ... what can cause this? To me it seems like some limit somewhere, since it stops at 33% of that big file every time, but don't have a clue where ...
Stop Whining!

Re: Large file upload fails at 33%

Reply #1

Php as well as apache or nginx can have time limits. Nginx may have it's own file size limit. Server guy should look at those. 

Re: Large file upload fails at 33%

Reply #2

Quote from: badmonkey – Php as well as apache or nginx can have time limits. Nginx may have it's own file size limit. Server guy should look at those.

Yes. And he stated that there are no limits, like I stated in the OP ... that is why this is mystery.
Stop Whining!

Re: Large file upload fails at 33%

Reply #3

Quote from: Vague Whiner – I have set max attachment file sizes to »no limit» but for some reason when I try to upload 386.79 MB zip file as attachment it seems to stop at 33%. Server guy says there is no limits on the serverside with PHP and such as I understood those can affect beyond EA ... what can cause this? To me it seems like some limit somewhere, since it stops at 33% of that big file every time, but don't have a clue where ...

What program/browser are you using to do the file transfer? Can you enable logging for the transfer?
Silly question perhaps, but file type isn't restricted in the attachment, is it?

// Deep inside every dilemma lies a solution that involves explosives //

Re: Large file upload fails at 33%

Reply #4

Given 33% of that file is right around 128MB it would think its a limitation either at the PHP level or the Web server (Nginx, Apache,  Other) level.

You can check in the ElkArte admin panel what your PHP values are (support and more by PHP version).  You need to check the values of: upload_max_filesize = ??? and post_max_size = ??? 

You can also check that its not a timeout issue, although since you consistently get 33% its not likely a timeout issue (yet ;) ).  Those values to check are max_input_time = ??? and max_execution_time = ???  That file of 386MB = 3088Mbits so if the timeouts are set to 300 seconds you would then need a minimum upload speed of 3088/300 = 10.3mbps to get the file there.

If those all look good then you would next need to check your server settings.  For example on Nginx it would be client_max_body_size ???; found in the http or server section of the main or site config.  And for web server timeout it would be fastcgi_read_timeout ???; probably in a location directive or even a fastcgi.conf file, all depends on how the server is set up.  I don't think there is an Apache directive for this, but then again I don't use it enough these days to be sure.

Re: Large file upload fails at 33%

Reply #5

Quote from: Spuds – Given 33% of that file is right around 128MB it would think its a limitation either at the PHP level or the Web server (Nginx, Apache,  Other) level.

Yes. I thought so but since I was told there is no limits in there it really baffled me ...

Quote from: Spuds – You can check in the ElkArte admin panel what your PHP values are (support and more by PHP version).  You need to check the values of: upload_max_filesize = ??? and post_max_size = ??? 

But apparently there is some limits after all:

Code: [Select]
upload_max_filesize	512M	512M
post_max_size 0 0

Not sure what that 0 indicates on post_max_size though ...

Quote from: Spuds – You can also check that its not a timeout issue, although since you consistently get 33% its not likely a timeout issue (yet ;) ).  Those values to check are max_input_time = ??? and max_execution_time = ???  That file of 386MB = 3088Mbits so if the timeouts are set to 300 seconds you would then need a minimum upload speed of 3088/300 = 10.3mbps to get the file there.

Hmmm ... since transfer speeds have been varying from 200k/s to 1M/s and all have stopped at 33% I don't think this timeout really plays a role here :)

But ...

Code: [Select]
max_input_time	300	300
max_execution_time 180 180

Still when transfer speed have been very different I don't really see timeout to be the culprit here ...

Quote from: Spuds – If those all look good then you would next need to check your server settings.  For example on Nginx it would be client_max_body_size ???; found in the http or server section of the main or site config.  And for web server timeout it would be fastcgi_read_timeout ???; probably in a location directive or even a fastcgi.conf file, all depends on how the server is set up.  I don't think there is an Apache directive for this, but then again I don't use it enough these days to be sure.

On the server apache is running and I was told that there shouldn't be any limiting happening over that side ... But I have to ask my friend to make sure there really is nothing that does this kind of thing. Peculiar thing that is really.

Quote from: Steeley – What program/browser are you using to do the file transfer? Can you enable logging for the transfer?

Doesn't matter, have tried with several browsers ...

Quote from: Steeley – Silly question perhaps, but file type isn't restricted in the attachment, is it?

No .zip is not restricted :)
Stop Whining!

Re: Large file upload fails at 33%

Reply #6

Quote from: Vague Whiner –
Code: [Select]
upload_max_filesize	512M	512M
post_max_size 0
The 0 means "no-limit" in most cases, but the manual also states
Quote will not disable the limit when the content type is application/x-www-form-urlencoded or is not registered with PHP. 

Those settings would normally allow multiple 512M files to be uploaded (or until you timeout).  Not sure that is relevant on the upload, can't remember and I'm not at a place where I can look.  You can try setting post_max_size to 512M or 1G and be sure its not the problem.

If it was a timeout it would happen after 3 or 5 mins based on those settings, but if it was that not sure why it would always be 33%.

Also, be sure to check that you do not have any limits set in your Admin Panel attachment settings.  In that area there are two to validate Max attachment space per post and Max size per attachment Normally those would prevent you from even starting an upload but still good to check.


Re: Large file upload fails at 33%

Reply #7

Quote from: Vague Whiner –

Quote from: Steeley – What program/browser are you using to do the file transfer? Can you enable logging for the transfer?

Doesn't matter, have tried with several browsers ...


Au Contraire Mssr..- a log file may indeed matter if you want to know what's punting your upload.. and what program you are using may help determine how to enable such transfer logging..

// Deep inside every dilemma lies a solution that involves explosives //

Re: Large file upload fails at 33%

Reply #8

Quote from: Steeley – Au Contraire Mssr..- a log file may indeed matter if you want to know what's punting your upload.. and what program you are using may help determine how to enable such transfer logging..

And how does client software logs tell what does stop the upload? Maybe I am missing something logical here ...
Stop Whining!

Re: Large file upload fails at 33%

Reply #9

Quote from: Vague Whiner –
Quote from: Steeley – Au Contraire Mssr..- a log file may indeed matter if you want to know what's punting your upload.. and what program you are using may help determine how to enable such transfer logging..

And how does client software logs tell what does stop the upload? Maybe I am missing something logical here ...

Logging often will point to the culprit, which can be more effective than just poking around in settings hoping to find something that looks like it might be the problem. 

Edge, Chrome, Firefox, etc. have "developer tools" built in that can log what is happening locally and between the browser and the server. I'll defer to others with more expertise than I (such as @Spuds ) for guidance on setting up and activating the logging (and interpreting the results).
But first, we need to know what software you are using.

Here's a couple intro links as an example

Depending on what exactly you are looking for, there are a number of options to zero in on the code, traffic messages, etc.. to see where and maybe why things fall apart..


Can be really handy when the host says "it ain't me" and you have a log file that suggests "Oh yes it is.." (and conversely, avoid the embarrassment if the host is right..)
Last Edit: February 28, 2023, 03:14:14 pm by Steeley

// Deep inside every dilemma lies a solution that involves explosives //

Re: Large file upload fails at 33%

Reply #10

Quote from: Steeley – Can be really handy when the host says "it ain't me" and you have a log file that suggests "Oh yes it is.." (and conversely, avoid the embarrassment if the host is right..)

Well ... »Host» in this case is server run by my friend ...

Anyway ... I did reproduce the attachment upload fail logging on, but I cannot really make anything from the log file. What should I be looking for in there?

Used https://netlog-viewer.appspot.com/ to view the file ...

Got 10MB worth of json that makes no sense to me :D
Stop Whining!

Re: Large file upload fails at 33%

Reply #11

Quote from: Vague Whiner –
Quote from: Steeley – Can be really handy when the host says "it ain't me" and you have a log file that suggests "Oh yes it is.." (and conversely, avoid the embarrassment if the host is right..)

Well ... »Host» in this case is server run by my friend ...

Anyway ... I did reproduce the attachment upload fail logging on, but I cannot really make anything from the log file. What should I be looking for in there?

Used https://netlog-viewer.appspot.com/ to view the file ...

Got 10MB worth of json that makes no sense to me :D

Probably wouldn't make much sense to me either, but I'll bet there's a few here that wouldn't mind casting a critical eye on it just to see what can be seen.  I assume you have that file saved and can shuffle it off to the curious and technically adept upon request. 

There's an app you can download (catapult net log viewer) that will take that json file and wrap an interface on it so that it can be "organized" but that still leaves you head scratching if you're not familiar with the forum code and server APIs, whereas someone who is probably already has that or similar and more readily achieve an "ah HA!" with it.  Or, they may suggest using the Browser-native DevTool to extract a different log in a different format.

I'm not that guy.  So as we say in the aviation world: "Go to position and hold", and someone hopefully will get back to ya' and "guide you in"..

// Deep inside every dilemma lies a solution that involves explosives //

Re: Large file upload fails at 33%

Reply #12

Quote from: Steeley –
There's an app you can download (catapult net log viewer) that will take that json file and wrap an interface on it so that it can be "organized" but that still leaves you head scratching if you're not familiar with the forum code and server APIs, whereas someone who is probably already has that or similar and more readily achieve an "ah HA!" with it.  Or, they may suggest using the Browser-native DevTool to extract a different log in a different format.


The one I used (and linked in my earlier reply :) to take a look into the JSON file is the one you refer here ... anyway that file did not really reveal anything to my eyes ... here is the last of the HTTP2 SESSION event that is about the attachment upload:

Code: [Select]
t=174865 [st=171951]    HTTP2_SESSION_RECV_WINDOW_UPDATE
                        --> delta = 114625
                        --> stream_id = 1
t=174865 [st=171951]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16304
                        --> window_size = 2016418147
t=174865 [st=171951]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16304
                        --> stream_id = 1
t=174866 [st=171952]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016401772
t=174866 [st=171952]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174866 [st=171952]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016385397
t=174866 [st=171952]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174866 [st=171952]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016369022
t=174866 [st=171952]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174867 [st=171953]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016352647
t=174867 [st=171953]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174867 [st=171953]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016336272
t=174867 [st=171953]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174868 [st=171954]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016319897
t=174868 [st=171954]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174868 [st=171954]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -71
                        --> window_size = 2016319826
t=174868 [st=171954]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 71
                        --> stream_id = 1
t=174868 [st=171954]    HTTP2_SESSION_STREAM_STALLED_BY_STREAM_SEND_WINDOW
                        --> stream_id = 1
t=174972 [st=172058]    HTTP2_SESSION_RECV_WINDOW_UPDATE
                        --> delta = 81875
                        --> stream_id = 1
t=174972 [st=172058]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16304
                        --> window_size = 2016303522
t=174972 [st=172058]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16304
                        --> stream_id = 1
t=174972 [st=172058]    HTTP2_SESSION_RECV_HEADERS
                        --> fin = false
                        --> :status: 500
                            date: Tue, 28 Feb 2023 21:09:09 GMT
                            server: Apache/2.4.37 (FreeBSD)
                            content-length: 529
                            content-type: text/html; charset=iso-8859-1
                        --> stream_id = 1
t=174972 [st=172058]    HTTP2_SESSION_RECV_DATA
                        --> fin = false
                        --> size = 529
                        --> stream_id = 1
t=174972 [st=172058]    HTTP2_SESSION_UPDATE_RECV_WINDOW
                        --> delta = -529
                        --> window_size = 15728111
t=174972 [st=172058]    HTTP2_SESSION_RECV_DATA
                        --> fin = true
                        --> size = 0
                        --> stream_id = 1
t=174973 [st=172059]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016287147
t=174973 [st=172059]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174973 [st=172059]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016270772
t=174973 [st=172059]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174974 [st=172060]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016254397
t=174974 [st=172060]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174974 [st=172060]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -16375
                        --> window_size = 2016238022
t=174974 [st=172060]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 16375
                        --> stream_id = 1
t=174975 [st=172061]    HTTP2_SESSION_UPDATE_SEND_WINDOW
                        --> delta = -71
                        --> window_size = 2016237951
t=174975 [st=172061]    HTTP2_SESSION_SEND_DATA
                        --> fin = false
                        --> size = 71
                        --> stream_id = 1
t=174975 [st=172061]    HTTP2_SESSION_STREAM_STALLED_BY_STREAM_SEND_WINDOW
                        --> stream_id = 1
t=174976 [st=172062]    HTTP2_SESSION_RECV_RST_STREAM
                        --> error_code = "0 (NO_ERROR)"
                        --> stream_id = 1
t=174979 [st=172065]    HTTP2_SESSION_UPDATE_RECV_WINDOW
                        --> delta = 529
                        --> window_size = 15728640
t=174979 [st=172065]    HTTP2_SESSION_SEND_WINDOW_UPDATE
                        --> delta = 529
                        --> stream_id = 0
t=174980 [st=172066]

Only thing that pop out is that
Code: [Select]
--> :status: 500
which tells me that there is internal server error, nothing more ...

Other than that there is nothing that indicates anything ... but maybe I am not just seeing something :D
Stop Whining!

Re: Large file upload fails at 33%

Reply #13

Quote from: Vague Whiner –
Other than that there is nothing that indicates anything ... but maybe I am not just seeing something :D

Yes, it looks like it started going slow and then fizzled out all together which rules out most forum admin settings methinks ... but there's a lot of stuff going on that is not in that output.  The Browser DevTools (In Edge, pressing F12 opens the door) have the ability to capture and expose many other wonderous things. When I do it I feel like a 5 year old peeking into my grandfather's work shop and seeing all kinds of really awesome machines and bizarre looking tools hanging on pegboards with no idea what they do or how to use them to make a toy box or a rocking horse.

Over the years I became pretty good at making toy boxes and rocking horses (building homes, etc. although I admit I'm more of a framer than a carpenter - airplanes, cars and radios are what really piqued my interest).  I may not be brilliant, but I was trainable.  Maybe I'd be more help to you if computers and the internet were a thing back in the late 50's to capture a young boy's fancy.  I discovered I was doomed in the 7th grade when my math teacher tried to tell me that letters could be numbers greater than 9.  That was a hex on me

The best I can do if someone more practiced in advanced binary electronics doesn't step into this thread is suggest cutting a couple slots into that file and make an ashtray out of it..  :rolleyes:
 

Last Edit: March 01, 2023, 12:09:05 pm by Steeley

// Deep inside every dilemma lies a solution that involves explosives //

Re: Large file upload fails at 33%

Reply #14

Quote from: Steeley – The best I can do if someone more practiced in advanced binary electronics doesn't step into this thread is suggest cutting a couple slots into that file and make an ashtray out of it..  :rolleyes:

Well ... one can always workaround these things :) Like in this instance I can upload the ZIP-file to the server, then upload small ZIP-file as attachment and then replace that with the big one ... but that is just a workaround. Would be nice to find out what gives with this thing, don't you think?

Anyway, to be using anything edgy I need to find one M$ hole in the wall machine, me thinks ... which I do not have :)

I have pushed this matter with server guy again and he will make sure there is nothing beyond EA that is causing this ... and as we might already know at this point, it most likely is something there.

I will report back when I know more ...
Stop Whining!