Large file upload fails at 33% February 28, 2023, 06:24:09 am I have set max attachment file sizes to »no limit» but for some reason when I try to upload 386.79 MB zip file as attachment it seems to stop at 33%. Server guy says there is no limits on the serverside with PHP and such as I understood those can affect beyond EA ... what can cause this? To me it seems like some limit somewhere, since it stops at 33% of that big file every time, but don't have a clue where ...
Re: Large file upload fails at 33% Reply #1 – February 28, 2023, 07:21:54 am Php as well as apache or nginx can have time limits. Nginx may have it's own file size limit. Server guy should look at those.
Re: Large file upload fails at 33% Reply #2 – February 28, 2023, 09:03:10 am Quote from: badmonkey – February 28, 2023, 07:21:54 amPhp as well as apache or nginx can have time limits. Nginx may have it's own file size limit. Server guy should look at those. Yes. And he stated that there are no limits, like I stated in the OP ... that is why this is mystery.
Re: Large file upload fails at 33% Reply #3 – February 28, 2023, 10:20:35 am Quote from: Vague Whiner – February 28, 2023, 06:24:09 amI have set max attachment file sizes to »no limit» but for some reason when I try to upload 386.79 MB zip file as attachment it seems to stop at 33%. Server guy says there is no limits on the serverside with PHP and such as I understood those can affect beyond EA ... what can cause this? To me it seems like some limit somewhere, since it stops at 33% of that big file every time, but don't have a clue where ...What program/browser are you using to do the file transfer? Can you enable logging for the transfer? Silly question perhaps, but file type isn't restricted in the attachment, is it?
Re: Large file upload fails at 33% Reply #4 – February 28, 2023, 10:33:26 am Given 33% of that file is right around 128MB it would think its a limitation either at the PHP level or the Web server (Nginx, Apache, Other) level.You can check in the ElkArte admin panel what your PHP values are (support and more by PHP version). You need to check the values of: upload_max_filesize = ??? and post_max_size = ??? You can also check that its not a timeout issue, although since you consistently get 33% its not likely a timeout issue (yet ). Those values to check are max_input_time = ??? and max_execution_time = ??? That file of 386MB = 3088Mbits so if the timeouts are set to 300 seconds you would then need a minimum upload speed of 3088/300 = 10.3mbps to get the file there.If those all look good then you would next need to check your server settings. For example on Nginx it would be client_max_body_size ???; found in the http or server section of the main or site config. And for web server timeout it would be fastcgi_read_timeout ???; probably in a location directive or even a fastcgi.conf file, all depends on how the server is set up. I don't think there is an Apache directive for this, but then again I don't use it enough these days to be sure.
Re: Large file upload fails at 33% Reply #5 – February 28, 2023, 10:52:32 am Quote from: Spuds – February 28, 2023, 10:33:26 amGiven 33% of that file is right around 128MB it would think its a limitation either at the PHP level or the Web server (Nginx, Apache, Other) level.Yes. I thought so but since I was told there is no limits in there it really baffled me ...Quote from: Spuds – February 28, 2023, 10:33:26 amYou can check in the ElkArte admin panel what your PHP values are (support and more by PHP version). You need to check the values of: upload_max_filesize = ??? and post_max_size = ??? But apparently there is some limits after all:Code: [Select]upload_max_filesize 512M 512Mpost_max_size 0 0Not sure what that 0 indicates on post_max_size though ...Quote from: Spuds – February 28, 2023, 10:33:26 amYou can also check that its not a timeout issue, although since you consistently get 33% its not likely a timeout issue (yet ). Those values to check are max_input_time = ??? and max_execution_time = ??? That file of 386MB = 3088Mbits so if the timeouts are set to 300 seconds you would then need a minimum upload speed of 3088/300 = 10.3mbps to get the file there.Hmmm ... since transfer speeds have been varying from 200k/s to 1M/s and all have stopped at 33% I don't think this timeout really plays a role here But ...Code: [Select]max_input_time 300 300max_execution_time 180 180Still when transfer speed have been very different I don't really see timeout to be the culprit here ... Quote from: Spuds – February 28, 2023, 10:33:26 amIf those all look good then you would next need to check your server settings. For example on Nginx it would be client_max_body_size ???; found in the http or server section of the main or site config. And for web server timeout it would be fastcgi_read_timeout ???; probably in a location directive or even a fastcgi.conf file, all depends on how the server is set up. I don't think there is an Apache directive for this, but then again I don't use it enough these days to be sure.On the server apache is running and I was told that there shouldn't be any limiting happening over that side ... But I have to ask my friend to make sure there really is nothing that does this kind of thing. Peculiar thing that is really.Quote from: Steeley – February 28, 2023, 10:20:35 amWhat program/browser are you using to do the file transfer? Can you enable logging for the transfer? Doesn't matter, have tried with several browsers ... Quote from: Steeley – February 28, 2023, 10:20:35 amSilly question perhaps, but file type isn't restricted in the attachment, is it?No .zip is not restricted
Re: Large file upload fails at 33% Reply #6 – February 28, 2023, 12:06:21 pm Quote from: Vague Whiner – February 28, 2023, 10:52:32 amCode: [Select]upload_max_filesize 512M 512Mpost_max_size 0 The 0 means "no-limit" in most cases, but the manual also states Quote will not disable the limit when the content type is application/x-www-form-urlencoded or is not registered with PHP. Those settings would normally allow multiple 512M files to be uploaded (or until you timeout). Not sure that is relevant on the upload, can't remember and I'm not at a place where I can look. You can try setting post_max_size to 512M or 1G and be sure its not the problem. If it was a timeout it would happen after 3 or 5 mins based on those settings, but if it was that not sure why it would always be 33%.Also, be sure to check that you do not have any limits set in your Admin Panel attachment settings. In that area there are two to validate Max attachment space per post and Max size per attachment Normally those would prevent you from even starting an upload but still good to check.
Re: Large file upload fails at 33% Reply #7 – February 28, 2023, 12:23:30 pm Quote from: Vague Whiner – February 28, 2023, 10:52:32 amQuote from: Steeley – February 28, 2023, 10:20:35 amWhat program/browser are you using to do the file transfer? Can you enable logging for the transfer? Doesn't matter, have tried with several browsers ... Au Contraire Mssr..- a log file may indeed matter if you want to know what's punting your upload.. and what program you are using may help determine how to enable such transfer logging..
Re: Large file upload fails at 33% Reply #8 – February 28, 2023, 12:47:58 pm Quote from: Steeley – February 28, 2023, 12:23:30 pmAu Contraire Mssr..- a log file may indeed matter if you want to know what's punting your upload.. and what program you are using may help determine how to enable such transfer logging.. And how does client software logs tell what does stop the upload? Maybe I am missing something logical here ...
Re: Large file upload fails at 33% Reply #9 – February 28, 2023, 01:40:34 pm Quote from: Vague Whiner – February 28, 2023, 12:47:58 pmQuote from: Steeley – February 28, 2023, 12:23:30 pmAu Contraire Mssr..- a log file may indeed matter if you want to know what's punting your upload.. and what program you are using may help determine how to enable such transfer logging.. And how does client software logs tell what does stop the upload? Maybe I am missing something logical here ...Logging often will point to the culprit, which can be more effective than just poking around in settings hoping to find something that looks like it might be the problem. Edge, Chrome, Firefox, etc. have "developer tools" built in that can log what is happening locally and between the browser and the server. I'll defer to others with more expertise than I (such as @Spuds ) for guidance on setting up and activating the logging (and interpreting the results). But first, we need to know what software you are using.Here's a couple intro links as an examplehttps://textslashplain.com/2020/01/17/capture-network-logs-from-edge-and-chrome/https://learn.microsoft.com/en-us/microsoft-edge/devtools-guide-chromium/network/Depending on what exactly you are looking for, there are a number of options to zero in on the code, traffic messages, etc.. to see where and maybe why things fall apart..Can be really handy when the host says "it ain't me" and you have a log file that suggests "Oh yes it is.." (and conversely, avoid the embarrassment if the host is right..)
Re: Large file upload fails at 33% Reply #10 – February 28, 2023, 04:25:28 pm Quote from: Steeley – February 28, 2023, 01:40:34 pmCan be really handy when the host says "it ain't me" and you have a log file that suggests "Oh yes it is.." (and conversely, avoid the embarrassment if the host is right..)Well ... »Host» in this case is server run by my friend ...Anyway ... I did reproduce the attachment upload fail logging on, but I cannot really make anything from the log file. What should I be looking for in there?Used https://netlog-viewer.appspot.com/ to view the file ...Got 10MB worth of json that makes no sense to me
Re: Large file upload fails at 33% Reply #11 – February 28, 2023, 06:01:07 pm Quote from: Vague Whiner – February 28, 2023, 04:25:28 pmQuote from: Steeley – February 28, 2023, 01:40:34 pmCan be really handy when the host says "it ain't me" and you have a log file that suggests "Oh yes it is.." (and conversely, avoid the embarrassment if the host is right..)Well ... »Host» in this case is server run by my friend ...Anyway ... I did reproduce the attachment upload fail logging on, but I cannot really make anything from the log file. What should I be looking for in there?Used https://netlog-viewer.appspot.com/ to view the file ...Got 10MB worth of json that makes no sense to me Probably wouldn't make much sense to me either, but I'll bet there's a few here that wouldn't mind casting a critical eye on it just to see what can be seen. I assume you have that file saved and can shuffle it off to the curious and technically adept upon request. There's an app you can download (catapult net log viewer) that will take that json file and wrap an interface on it so that it can be "organized" but that still leaves you head scratching if you're not familiar with the forum code and server APIs, whereas someone who is probably already has that or similar and more readily achieve an "ah HA!" with it. Or, they may suggest using the Browser-native DevTool to extract a different log in a different format.I'm not that guy. So as we say in the aviation world: "Go to position and hold", and someone hopefully will get back to ya' and "guide you in"..
Re: Large file upload fails at 33% Reply #12 – March 01, 2023, 06:01:54 am Quote from: Steeley – February 28, 2023, 06:01:07 pmThere's an app you can download (catapult net log viewer) that will take that json file and wrap an interface on it so that it can be "organized" but that still leaves you head scratching if you're not familiar with the forum code and server APIs, whereas someone who is probably already has that or similar and more readily achieve an "ah HA!" with it. Or, they may suggest using the Browser-native DevTool to extract a different log in a different format.The one I used (and linked in my earlier reply to take a look into the JSON file is the one you refer here ... anyway that file did not really reveal anything to my eyes ... here is the last of the HTTP2 SESSION event that is about the attachment upload:Code: [Select]t=174865 [st=171951] HTTP2_SESSION_RECV_WINDOW_UPDATE --> delta = 114625 --> stream_id = 1t=174865 [st=171951] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16304 --> window_size = 2016418147t=174865 [st=171951] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16304 --> stream_id = 1t=174866 [st=171952] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016401772t=174866 [st=171952] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174866 [st=171952] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016385397t=174866 [st=171952] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174866 [st=171952] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016369022t=174866 [st=171952] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174867 [st=171953] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016352647t=174867 [st=171953] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174867 [st=171953] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016336272t=174867 [st=171953] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174868 [st=171954] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016319897t=174868 [st=171954] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174868 [st=171954] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -71 --> window_size = 2016319826t=174868 [st=171954] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 71 --> stream_id = 1t=174868 [st=171954] HTTP2_SESSION_STREAM_STALLED_BY_STREAM_SEND_WINDOW --> stream_id = 1t=174972 [st=172058] HTTP2_SESSION_RECV_WINDOW_UPDATE --> delta = 81875 --> stream_id = 1t=174972 [st=172058] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16304 --> window_size = 2016303522t=174972 [st=172058] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16304 --> stream_id = 1t=174972 [st=172058] HTTP2_SESSION_RECV_HEADERS --> fin = false --> :status: 500 date: Tue, 28 Feb 2023 21:09:09 GMT server: Apache/2.4.37 (FreeBSD) content-length: 529 content-type: text/html; charset=iso-8859-1 --> stream_id = 1t=174972 [st=172058] HTTP2_SESSION_RECV_DATA --> fin = false --> size = 529 --> stream_id = 1t=174972 [st=172058] HTTP2_SESSION_UPDATE_RECV_WINDOW --> delta = -529 --> window_size = 15728111t=174972 [st=172058] HTTP2_SESSION_RECV_DATA --> fin = true --> size = 0 --> stream_id = 1t=174973 [st=172059] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016287147t=174973 [st=172059] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174973 [st=172059] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016270772t=174973 [st=172059] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174974 [st=172060] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016254397t=174974 [st=172060] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174974 [st=172060] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -16375 --> window_size = 2016238022t=174974 [st=172060] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 16375 --> stream_id = 1t=174975 [st=172061] HTTP2_SESSION_UPDATE_SEND_WINDOW --> delta = -71 --> window_size = 2016237951t=174975 [st=172061] HTTP2_SESSION_SEND_DATA --> fin = false --> size = 71 --> stream_id = 1t=174975 [st=172061] HTTP2_SESSION_STREAM_STALLED_BY_STREAM_SEND_WINDOW --> stream_id = 1t=174976 [st=172062] HTTP2_SESSION_RECV_RST_STREAM --> error_code = "0 (NO_ERROR)" --> stream_id = 1t=174979 [st=172065] HTTP2_SESSION_UPDATE_RECV_WINDOW --> delta = 529 --> window_size = 15728640t=174979 [st=172065] HTTP2_SESSION_SEND_WINDOW_UPDATE --> delta = 529 --> stream_id = 0t=174980 [st=172066]Only thing that pop out is that Code: [Select]--> :status: 500 which tells me that there is internal server error, nothing more ...Other than that there is nothing that indicates anything ... but maybe I am not just seeing something
Re: Large file upload fails at 33% Reply #13 – March 01, 2023, 11:10:09 am Quote from: Vague Whiner – March 01, 2023, 06:01:54 amOther than that there is nothing that indicates anything ... but maybe I am not just seeing something Yes, it looks like it started going slow and then fizzled out all together which rules out most forum admin settings methinks ... but there's a lot of stuff going on that is not in that output. The Browser DevTools (In Edge, pressing F12 opens the door) have the ability to capture and expose many other wonderous things. When I do it I feel like a 5 year old peeking into my grandfather's work shop and seeing all kinds of really awesome machines and bizarre looking tools hanging on pegboards with no idea what they do or how to use them to make a toy box or a rocking horse.Over the years I became pretty good at making toy boxes and rocking horses (building homes, etc. although I admit I'm more of a framer than a carpenter - airplanes, cars and radios are what really piqued my interest). I may not be brilliant, but I was trainable. Maybe I'd be more help to you if computers and the internet were a thing back in the late 50's to capture a young boy's fancy. I discovered I was doomed in the 7th grade when my math teacher tried to tell me that letters could be numbers greater than 9. That was a hex on me. The best I can do if someone more practiced in advanced binary electronics doesn't step into this thread is suggest cutting a couple slots into that file and make an ashtray out of it..
Re: Large file upload fails at 33% Reply #14 – March 01, 2023, 04:48:14 pm Quote from: Steeley – March 01, 2023, 11:10:09 amThe best I can do if someone more practiced in advanced binary electronics doesn't step into this thread is suggest cutting a couple slots into that file and make an ashtray out of it.. Well ... one can always workaround these things Like in this instance I can upload the ZIP-file to the server, then upload small ZIP-file as attachment and then replace that with the big one ... but that is just a workaround. Would be nice to find out what gives with this thing, don't you think?Anyway, to be using anything edgy I need to find one M$ hole in the wall machine, me thinks ... which I do not have I have pushed this matter with server guy again and he will make sure there is nothing beyond EA that is causing this ... and as we might already know at this point, it most likely is something there.I will report back when I know more ...