I wrote an application which uses WebRequest to access several thousand small
graphics file on an on-line server, uses GetResponseStream and Image.FromStream
to copy their contents, and then saves the copies to a local hard disk.
It seemed "prudent" to me to insert a wait state after each access to a file on
the server like :
System.Threading.Thread.Sleep(250);
Testing using this scenario on fifty or so of the images went ok, but I am
wondering, before I "turn the application loose," if there are any "best
practices" regarding multiple accesses to a server like this. If I were using an
AxWebBrowser, I could be waiting on the DocumentComplete event, but the
performance of WebRequest seems great, and I like the idea of doing everything
without the overhead of the webbrowser control, and any "visual presentation" of
the process or results other than a progress bar.
Any hints, tips, ideas, pointers on creating robust (and "polite") large-scale
access to on-line image resources will be very appreciated.
thanks !
Bill Woodruff
dotScience
Chiang Mai, Thailand 1 4380
Bill Woodruff wrote: I wrote an application which uses WebRequest to access several thousand small graphics file on an on-line server, uses GetResponseStream and Image.FromStream to copy their contents, and then saves the copies to a local hard disk.
It seemed "prudent" to me to insert a wait state after each access to a file on the server like :
System.Threading.Thread.Sleep(250);
Testing using this scenario on fifty or so of the images went ok, but I am wondering, before I "turn the application loose," if there are any "best practices" regarding multiple accesses to a server like this. If I were using an AxWebBrowser, I could be waiting on the DocumentComplete event, but the performance of WebRequest seems great, and I like the idea of doing everything without the overhead of the webbrowser control, and any "visual presentation" of the process or results other than a progress bar.
Any hints, tips, ideas, pointers on creating robust (and "polite") large-scale access to on-line image resources will be very appreciated.
You didn't say how many requests are sent in parallel, or whether you're
using HTTP 1.1 persistent connections. In case of persistent connections,
the 250 ms delay seems to be a waste of time to me -- there's no risk of
port exhaustion.
The HTTP spec says:
"Clients that use persistent connections SHOULD limit the number of
simultaneous connections that they maintain to a given server. A single-user
client SHOULD NOT maintain more than 2 connections with any server or
proxy."
This is also the default behaviour for WebRequest instances.
Cheers,
--
Joerg Jooss www.joergjooss.de ne**@joergjooss.de This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Cenray |
last post by:
How to generate a web request from a Winform Application using a WebService?
In a C# Windows Application, when I try to genrate a web request using the
following code, it is woking fine
...
|
by: Vasu |
last post by:
Hi,
I have a requirement to download a file from the web
site using a client tool.
Iam writing a C# program to download using WebRequest,
HttpRequest, WebResponse and
so on. The problem...
|
by: thomas peter |
last post by:
I am building a precache engine... one that request over 100 pages on an
remote server to cache them remotely...
can i use the HttpWebRequest and WebResponse classes for this? or must i use
the...
|
by: Peter Qian |
last post by:
Hi,
I'm working on a windows form based program that can log into a web
service (Apache based, https is used for auth). I was able to post the login
data and obtain a sessionID. However I'm not...
|
by: |
last post by:
Hello,
I would like to do the following from a asp.net button click:
<form method="POST"
action="https://www.1234.com/trans_center/gateway/direct.cgi">
<input type="hidden" name="Merchant"...
|
by: Mark Waser |
last post by:
Hi all,
I'm trying to post multipart/form-data to a web page but seem to have
run into a wall. I'm familiar with RFC 1867 and have done this before (with
AOLServer and Tcl) but just can't seem...
|
by: STEVE.KING |
last post by:
Greetings:
I need to download an executable and some other binary files (a C++
DLL) from a virtual directory in my WinForms application. No-Touch
Deployment manages my .NET code fine but...
|
by: shankararaman.s |
last post by:
Hi,
I am trying to develop an interface which will fetch all my Yahoo
mails. I am not able to sign in to yahoo by posting the form with my
username & password. Please find my code below and...
|
by: rlueneberg |
last post by:
I am totally confused. Can someone please illuminate what is going on
under the hood in this piece of code from John Lewis.
My main confusion is how the cookieContainer can be passed to the...
|
by: phil.jacobs |
last post by:
Hi there,
I am very stuck and have tried everything on this one!
I am trying to get to an https website and cannot work out how to
logon. When I log on manually, I have to enter username and...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
|
by: ryjfgjl |
last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
| | |