I have a test script below which I use to fetch urls into strings,
either over https or http. When over https, I use m2crypto.urllib and
when over http I use the standard urllib. Whenever, I import sockets
and setdefaulttimeo ut, however, using m2crypto.urllib tends to cause a
http.BadStatusL ine to be raised, even if the timeout is set to be very
large. All of the documents in the test script can be accessed
publicly.
Any ideas? Is there a better/easier way to get https docs in python?
Thanks,
JDH
import urllib, socket
from cStringIO import StringIO
from M2Crypto import Rand, SSL, m2urllib
#comment out this line and the script generally works, but without it
#my zope process, which is using this code, hangs.
socket.setdefau lttimeout(200)
def url_to_string(s ource):
"""
get url as string, for https and http
"""
if source.startswi th('https:'):
sh = StringIO()
url = m2urllib.FancyU RLopener()
url.addheader(' Connection', 'close')
u = url.open(source )
while 1:
data = u.read()
if not data: break
sh.write(data)
return sh.getvalue()
else:
return urllib.urlopen( source).read()
if __name__=='__ma in__':
s1 = url_to_string(' https://crcdocs.bsd.uch icago.edu/crcdocs/Files/informatics.doc ')
s2 = url_to_string(' http://yahoo.com')
s3 = url_to_string(' https://crcdocs.bsd.uch icago.edu/crcdocs/Files/facepage.doc')
print len(s1), len(s2), len(s3) 1 2487
John Hunter <jd******@ace.b sd.uchicago.edu > writes:
[...] Any ideas? Is there a better/easier way to get https docs in python?
[...]
Python 2.3 has https support built-in even on Windows.
John This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Gary Feldman |
last post by:
I think I've found a deficiency in the design of urllib related to https.
In order to complete an https connection, it appears that URLOpener and
hence FancyURLOpener require the key and cert files. Or at least, it's not
clear from the description of socket.ssl what it does if they're omitted.
However, urlopen has no way to specify such things. Nor should it - for
typical uses, a person simply trying to retrieve data from an ssl site...
|
by: Carl Waldbieser |
last post by:
I tried to adapt the instructions for building the M2Crypto module
(http://sandbox.rulemaker.net/ngps/m2/INSTALL.html) to build a version
compatible with Python2.3, but I've had some mixed results. I actually got
everything to build and install, but when I try to import M2Crypto-- well,
here is a sample session:
>>> import M2Crypto
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File...
|
by: Dilip |
last post by:
Gurus
I didn't personally encounter these problems but some friends of mine
who are doing some deep heavy-metal C#/Networking application have run
into these problems. Does anybody have any idea if these are
addressed for a future release?
thanks
--Dilip
|
by: Jaap Spies |
last post by:
Hi,
Running Fedora Core 4: Python 2.4.3 and Python 2.4.1.
I'm getting:
IOError: (2, 'No such file or directory')
all the time.
Trying to track down this problem:
Python 2.4.1 (#1, May 16 2005, 15:19:29)
|
by: kgrafals |
last post by:
Hi,
I'm just trying to read from a webpage with urllib but I'm getting
IOErrors. This is my code:
import urllib
sock = urllib.urlopen("http://www.google.com/")
and this is the error:
| |
by: John Nagle |
last post by:
I've been running M2Crypto successfully using Python 2.4 on Windows 2000,
and now I'm trying to get it to work on Python 2.3.4 on Linux.
Attempting to initialize a context results in
Traceback (most recent call last):
File "/www/htdocs/sitetruth.com/cgi/ratingdetails.cgi", line 46, in ?
DetailsPageBuilder.detailspage(kdbfile,ktemplatefile,url) # check and
display domain or URL as web page
File "./sitetruth/DetailsPageBuilder.py", line...
|
by: John Nagle |
last post by:
Here are three network-related exceptions. These
were caught by "except" with no exception type, because
none of the more specific exceptions matched. This
is what a traceback produced:
1. File "D:\Python24\lib\socket.py", line 295, in read
data = self._sock.recv(recv_size)
timeout: timed out
|
by: Heikki Toivonen |
last post by:
M2Crypto has some old code that gets and sets socket timeouts in
http://svn.osafoundation.org/m2crypto/trunk/M2Crypto/SSL/Connection.py,
for example:
def get_socket_read_timeout(self):
return
timeout.struct_to_timeout(self.socket.getsockopt(socket.SOL_SOCKET,
socket.SO_RCVTIMEO, timeout.struct_size()))
The helper timeout module is here:
|
by: Heikki Toivonen |
last post by:
The 0.19.1 release fixes the build when OpenSSL has been configured
without EC support, thanks to Miloslav Trmac.
M2Crypto is the most complete Python wrapper for OpenSSL featuring RSA,
DSA, DH, HMACs, message digests, symmetric ciphers (including AES); SSL
functionality to implement clients and servers; HTTPS extensions to
Python's httplib, urllib, and xmlrpclib; unforgeable HMAC'ing
AuthCookies for web session management; FTP/TLS...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |