European Experts Exchange (EEE.org) :: Free Experts PHP MySql Oracle Microsoft Web SQL Delphi


Web :: General :: Apache, downloading large files (100Mb and more)
By: 7workers	Date: 29/01/2009 11:47:48	Points: 20	Status: Answered Quality : Excellent
Have problems downloading large files, it just stucks at 70-80%. But not for all of file but for most. Nothing in error log, access_log looks fine pretty much, except the point that atual length of files is more then in logs: XX.XX.XX.XX - - [25/Jan/2009:12:37:56 +0000] "GET /store/pub/.ibiovthvddylypix/You-Are-God-In-Heaven-Keys.zip HTTP/1.1" 200 13445 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5" XX.XX.XX.XX - - [25/Jan/2009:17:02:49 +0000] "GET /store/pub/.pqbvmyjbceyfcstc/You-Are-God-In-Heaven-Keys.zip HTTP/1.1" 206 18793489 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5" XX.XX.XX.XX - - [26/Jan/2009:09:25:27 +0000] "GET /store/pub/.vuqpgmnpfgudqwce/You-Are-God-In-Heaven-Keys.zip HTTP/1.1" 200 238016129 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5" the actual size is 277326092 when downloading from the same server (wget http://localhost/....) everything works users reported the if they pause downloading and resume again (for example in firefox) it works fine. Hosting tech support can't figure out what's going on. There is nothing in firewall etc. apache config: Timeout 20 KeepAlive On MaxKeepAliveRequests 300 KeepAliveTimeout 20 <IfModule prefork.c> StartServers 1 MinSpareServers 1 MaxSpareServers 3 ServerLimit 50 MaxClients 50 MaxRequestsPerChild 4000 </IfModule> <IfModule worker.c> StartServers 2 MaxClients 100 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule> ... LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_balancer_module modules/mod_proxy_balancer.so LoadModule proxy_ftp_module modules/mod_proxy_ftp.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_connect_module modules/mod_proxy_connect.so LoadModule cache_module modules/mod_cache.so LoadModule mem_cache_module modules/mod_mem_cache.so ... Please, give me ANY ideas, i need to sort this out 100%
By: VGR	Date: 30/01/2009 08:59:51		Type : Comment
hello, Preliminary remark : I upload files up to 1GB to my Apache servers (even on windows), so it can't be that complicated ;-) First let's not argue on the HTTP 206 "Partial Content" that you have seen in your access logs : for me, it means the user suspended the download. Then, I will give you the information you need. The problem doesn't lie with the httpd.conf configuration that you've provided. It more lies with the front-end language (if any) configuration, eg php.ini if PHP. As you've not told you used PHP or the like, I will assume only Apache is in the way to the actual file. Then, it gets simplier (in a way) The things you've to check in the httpd.conf are : Timeout 20 It means Apache will die on the HTTP request if it gets stuck more than 20 seconds, including when accepting your upload file into the temporary directory before re-sending the data (in your case, downloading may include also time spent in this state, but I'm no specialist). I think 20 is a bit too low. It doesn't cost a lot to change this, so personally I run with Timeout 600 Now, if you have a PHP script serving the file to the clients' browsers, then there ARE things to check in php.ini for large file transfers, but I'll speak of this later, depending on your feedback. HTH

By: 7workers	Date: 30/01/2009 09:38:25		Type : Comment
Well, first of all i have problems DOWNloading files, not uploading. And PHP is not used when downloading file. I just put file into folder and download it directly, without PHP. Timeout... not sure. This mean each child will be hanging around for a log time. Having big load on server i can run out of RAM

By: VGR	Date: 30/01/2009 14:59:11		Type : Comment
well, when you DOWNload files through an HTTP link, you pass through the web server anyway (even a [bad] file:// access passes through, IMHO) you may encounter memory limits or timeouts. IIS also has problems like this (see http://forums.iis.net/p/1154459/1891494.aspx) The fact that you succeed doing a wget on localhost may be a confirmation . Have you tried logging into a remote server and doing there a wget to your URI ? It may also be because of the SOCKS on the client, as when using passive FTP and failing over 20 seconds (see microsoft bug)

By: 7workers	Date: 30/01/2009 19:59:59		Type : Comment
Yes, i did it - logged into remote server and tried to wget (two different servers) - same issue. It looks like the only one workaround is to pause downloading and resume it. Also, FTP works fine with downloading same files.

By: VGR	Date: 31/01/2009 12:35:31		Type : Comment
the only thing that comes to my mind is the SOCKS V5 story on the client... Is your problem reproductible for all clients, all the time ? this is quite difficult to diagnose. Try to find a clitne that has patched his SOCKS and see what happens... Sorry not to be of much help.

By: 7workers	Date: 31/01/2009 16:16:18		Type : Comment
I don't really know what you mean by "patched his SOCKS" (my socks has no holes! ) LOL. The issue appears on windows and linux machines, different linux (redhat, centos, fedora) and windows vista. So i think this is not client issue. And i don't really expect much of help... I started to hope on miracle :)

By: VGR	Date: 01/02/2009 22:40:29		Type : Answer
it may be a connection failure, a bug in the driver (client or server) especially with Centos or red hat [quite old, aren't they?] and Vista [ a bit too green, isn't it ?]. Just to demonstrate you're not alone with such a strange "download gets interrupted for large files" problem : this link or that one googling a bit, it may also be a firewall issue, or a browser problem (Internet Exploder) or a cache problem (Mozilla Firefox, Internet Exploder). A suggestion ? Tell your clients to use a download manager, including the one of Firefox, they'll save their time...

By: OpConsole	Date: 14/10/2010 08:37:21		Type : Comment
don't forget to close the Question after a while. You've the choice between three levels of satisfaction before clicking on the "A" button top right of the chosen Answer(s) thx

By: OpConsole	Date: 03/04/2011 19:06:04		Type : Comment
force close

Do register to be able to answer

©2010 These pages are served without commercial sponsorship. (No popup ads, etc...). Bandwidth abuse increases hosting cost forcing sponsorship or shutdown. This server aggressively defends against automated copying for any reason including offline viewing, duplication, etc... Please respect this requirement and DO NOT RIP THIS SITE.
Please DO link to this page! Post to Plurk Post to Yahoo Buzz Post to Delicious Post to Digg Post to MySpace Post to Ping.fm Post to Reddit Post to StumbleUpon

Contact

page generated in 115.721940 milliseconds

Why Google AdSense ads ?

New Improved Search!