473,706 Members | 2,324 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

downloading a large file!

Hi All,

Quick question, in my program I need to download a large file (100MB)
file. I would like to know how can I do this in segments so that if the
download fails, I will able to download from the the place where the
download fails.

Thanks.

Jan 17 '06 #1
10 5489
Hi,

"DBC User" <db*****@gmail. com> wrote in message
news:11******** *************@g 44g2000cwa.goog legroups.com...
Hi All,

Quick question, in my program I need to download a large file (100MB)
file. I would like to know how can I do this in segments so that if the
download fails, I will able to download from the the place where the
download fails.


For this to work you need to control both ends of the transmition, do you?
if so you would have to send in the request the offset from the beginning of
the file, after that is as usual, get the bytes, and append them to the
file.
--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation
Jan 17 '06 #2
I believe you can use HTTP for this. The 1.1 specification has a
provision in it (in the form of a header I believe) which you can set to
indicate where to start the download from. This way, you can indicate which
byte to start at, and then just append to your file.

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m

"DBC User" <db*****@gmail. com> wrote in message
news:11******** *************@g 44g2000cwa.goog legroups.com...
Hi All,

Quick question, in my program I need to download a large file (100MB)
file. I would like to know how can I do this in segments so that if the
download fails, I will able to download from the the place where the
download fails.

Thanks.

Jan 17 '06 #3
Hello Nicholas Paldino [.NET/C# MVP],
I believe you can use HTTP for this. The 1.1 specification has a
provision in it (in the form of a header I believe) which you can set
to indicate where to start the download from. This way, you can
indicate which byte to start at, and then just append to your file.

Hope this helps.


This header is called "Range". Using it, one can specify one or multiple
byte segments to be retrieved.

Cheers,
--
Joerg Jooss
ne********@joer gjooss.de
Jan 17 '06 #4
It should be noted that you cannot assume that all servers will allow
this. So in that case you'll need to restart the entire download if the
server you are connecting to doesn't support Range.

--
Benny Raymond
http://bloatedcowsoftware.com

Joerg Jooss wrote:
Hello Nicholas Paldino [.NET/C# MVP],
I believe you can use HTTP for this. The 1.1 specification has a
provision in it (in the form of a header I believe) which you can set
to indicate where to start the download from. This way, you can
indicate which byte to start at, and then just append to your file.

Hope this helps.

This header is called "Range". Using it, one can specify one or multiple
byte segments to be retrieved.

Cheers,

Jan 17 '06 #5
Hello Benny,
It should be noted that you cannot assume that all servers will allow
this. So in that case you'll need to restart the entire download if
the server you are connecting to doesn't support Range.


That's correct. A server that supports Range requests should advertise this
capability using the "Accept-Ranges" header.

Cheers,
--
Joerg Jooss
ne********@joer gjooss.de
Jan 17 '06 #6
let me see if I understood this correct

I guess I need to have control over both side and I do. (I am writing a
tray item do this task). From the client side, I call a http page to do
the download (Are we taking file copy over http??) as the file copy is
running I keep track of the offset. If the download breaks in the
middle (that is no response from the server or any other failure), I
re-request the file copy, during the request, I will pass the offset in
the http (how can I do that). To make this happen, I have to have the
download server configured to handle accept-ranges (Where do I do
that??).

Thanks.

Jan 17 '06 #7
Hello DBC,
let me see if I understood this correct

I guess I need to have control over both side and I do. (I am writing
a tray item do this task).
Not really. You just need to identify those servers that accept Range requests.
If a server doesn't support Ranges, you have to repeat the entire download
from scratch.
From the client side, I call a http page to
do the download (Are we taking file copy over http??) as the file copy
is running I keep track of the offset.
Yes, HTTP. I assumed that sort of blindly after Nicolas' reply.
If the download breaks in the
middle (that is no response from the server or any other failure), I
re-request the file copy, during the request, I will pass the offset
in the http (how can I do that).
Add a Range header to your HttpWebRequest by calling AddRange().
To make this happen, I have to have
the download server configured to handle accept-ranges (Where do I do
that??).


That depends on the server or the web application (if there's one involved).
What are using? IIS? Apache?

Cheers,
--
Joerg Jooss
ne********@joer gjooss.de
Jan 17 '06 #8
I am developing this application using .Net C# 2005 and my IIS is
Windows. As I mentioned, I have a tray icon process which runs in the
background and when it sees a request to download, it will start
downloading based on what I find here. I have 6 servers which are
dedicated to do downloading. All are load balanced.

Jan 18 '06 #9
Hi Jorge,
"Not really. You just need to identify those servers that accept Range
requests.
If a server doesn't support Ranges, you have to repeat the entire
download
from scratch. "

In here how can you make the server to support ranges?

Thanks a lot for all the explanation.

Jan 18 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
3224
by: ASP Spam Fighter | last post by:
Hello all, I don't know how to get around this one... If anybody can help me with this problem, I would appreciate it very much. I've been trying to send a (large) file to the browser via a download page that uses a file download component. The file downloads fine, except that Internet Explorer refuses to browse to a different page while the file is downloading! I don't understand why it does that and I don't know how to fix this...
0
1375
by: sales | last post by:
If you are having difficulty downloading large files, please check out www.Downloads4Dialups.com. Mention this newsgroup in the "Special Instructions" window on the Shipping Form, and receive a 20% discount. Help with downloading large files
5
6892
by: kevin | last post by:
Hi, Any help with this would be really appreciated! I'm trying to download a file from a remote server. The access permissions is okay but the problem I'm facing is that the file is getting downloaded before the Save As dialogue appears. As we will be downloading some large files, it's unpractical to have this. So far the code I have to download the file is
3
1360
by: blantz | last post by:
I am using Forms Authentication to force a login to a support folder, where there are large .exe and .zip files for download. In my web.config file, i am blocking these file extensions with a httpHandler <httpHandlers> <add verb="*" path="*.exe" type="MyHandler.NewHandler,httpHandler"/> <add verb="*" path="*.zip" type="MyHandler.NewHandler,httpHandler"/> </httpHandlers>
10
5505
by: mwt | last post by:
This code works fine to download files from the web and write them to the local drive: import urllib f = urllib.urlopen("http://www.python.org/blah/blah.zip") g = f.read() file = open("blah.zip", "wb") file.write(g) file.close()
4
2013
by: PiotrKolodziej | last post by:
hi I have a thread that downloades a file. Problem is : Not all files are beeing downloaded. I observed that only the small files are beeing downloaded correctly. I also cant download two files in a row. I got to Exit and run the program again to even start downloading. The second problem i interprett as iam not closing something ( but don't know what ). For the first problem i have no idea what is wrong.Here is the download thread Maybe...
1
4986
by: DBC User | last post by:
Hello, I would like to copy a very large file (80MB) from the web, which is the best approach for downloading such a large file and also how can I show how much percent completed to user. I know I can use webdownload and also before start downloading, the can use the HEAD to get the size and then check the file size very 10 sec and show the progress (one of the approach). But for sure in .Net there should be a better way to do this....
7
2492
by: Steve | last post by:
Hello, I am a beginner/intermediate c++ coder. I have a program that I am writing and would like to be able to distribute to some of my friends. The problem with this is that the data files required by it are rather large, and most of them have email accounts with filesize limits. What I want to do is upload the files they'll need to my webpage and then have the program itself check to see if the files are present in the directories (I...
6
3764
by: cyusman | last post by:
Hi, We have just moved our application to a new webfarm server environment which utilizing hardware load balancing, SSL off-loading and HTTP compression off-loading.My application is running on .NET 1.1, IIS 6, Win2003. Now we are having problem when trying to download word documents located in the file server. It often says Internet Explorer cannot download <filenamefrom <servername>. Sometimes we are able to view
35
9357
by: keerthyragavendran | last post by:
hi i'm downloading a single file using multiple threads... how can i specify a particular range of bytes alone from a single large file... for example say if i need only bytes ranging from 500000 to 3200000 of a file whose size is say 20MB... how do i request a download which starts directly at 500000th byte... thank u cheers
0
9285
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8993
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7906
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6614
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5938
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4448
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
3147
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2500
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2094
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.