473,385 Members | 1,472 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Crawler and connection reset by peer exception

2
I made a python program that crawls a major website and collects a bunch of data. I have found that I get "connection reset by peer" issues on almost every page I crawl. I keep a log of every page, which gives me this error, so I can recrawl that page, but is there a way to not get that error?
Aug 19 '08 #1
0 1575

Sign in to post your reply or Sign up for a free account.

Similar topics

4
by: Donnal Walter | last post by:
On Windows XP I am able to connect to a remote telnet server from the command prompt using: telnet nnn.nnn.nnn.nnn 23 where nnn.nnn.nnn.nnn is the IP address of the host. But using telnetlib,...
2
by: Pandis Ippokratis | last post by:
Hi, I have the following problem: I am implementing a server in Java and I use jdbc to connect to an Oracle 9.0.2 db. All seem to work fine, but when I intensionally cause a network failure...
3
by: mirandacascade | last post by:
This may be more of a socket question than a python question; not sure. Using this code to instantiate/connect/set options connectionHandle = socket.socket(socket.AF_INET, socket.SOCK_STREAM)...
4
by: Joe Lester | last post by:
I'm seeing this message a couple times per day in my postgres log: 2004-04-20 14:47:46 LOG: could not receive data from client: Connection reset by peer What does it mean? I've seen in the...
3
by: Van_Gogh | last post by:
Hi, I am learning how to use the smtplib module, but am having some very early problems, maybe because I don't understand it. So, am I correct that by following the example in the Python: >>>...
0
by: xristoph | last post by:
Hello everyone! I am trying to set up a broker-worker system (one broker, multiple and heterogeneous workers) in python for a local network (though possibly also over inet). I am using...
14
by: ahlongxp | last post by:
Hi, everyone, I'm implementing a simple client/server protocol. Now I've got a situation: client will send server command,header paires and optionally body. server checks headers and decides...
5
by: chrispoliquin | last post by:
Hi, I have a small Python script to fetch some pages from the internet. There are a lot of pages and I am looping through them and then downloading the page using urlretrieve() in the urllib...
2
by: gigs | last post by:
I connect to web site with httplib.HTTPConnection. after some time i get this error: 104 "connection reset by peer". What exception i should use to catche this error thx!
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.