473,237 Members | 1,067 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,237 software developers and data experts.

Using urllib2 to get info from webpage

Thekid
145 100+
I'm trying to familiarize myself with urllib2 and what I want to try is to take a puzzle from a website, complete it and then submit it back to the website. So far this is what I have but I do have some ideas for other parts of it. For now I'm just working on getting the puzzle from the page. This will print out the entire page. BeautifulSoup and html5lib were suggested to me and I briefly looked at them but wanted to know if there's a simplier way of grabbing info from a site.

Expand|Select|Wrap|Line Numbers
  1. import urllib2
  2. req = urllib2.Request('http://www.cryptograms.org')
  3. response = urllib2.urlopen(req)
  4. the_page = response.read()
  5. print the_page
  6.  
Oct 26 '07 #1
0 1320

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Andre Bocchini | last post by:
I'm having some trouble using proxy authentication. I can't figure out how to authenticate with a Squid proxy. I know for a fact the proxy is using Basic instead of Digest for the authentication....
0
by: Lee, Dustin | last post by:
I'm using python2.2.2 on redhat linux 7.3 Here's a snippet of what I'm trying to run: # set up proxy import urllib2 proxy_support =...
2
by: John F Dutcher | last post by:
Can anyone comment on why the code shown in the Python error is in some way incorrect...or is there a problem with Python on my hoster's site ?? The highlites don't seem to show here...but line...
0
by: Matt | last post by:
I'm trying to get the HTML data off of a webpage. Let's say for the sake of argument it's the python homepage. I've googled around and found some examples that people said worked. Here's what...
2
by: Joshua Ginsberg | last post by:
I'm a bit baffled by something... In a script I wrote, I have defined a function that runs urllib2.urlopen() on a urllib2.Request object and returns the file-like object. The code that calls...
11
by: Johnny Lee | last post by:
Hi, I was using urllib to grab urls from web. here is the work flow of my program: 1. Get base url and max number of urls from user 2. Call filter to validate the base url 3. Read the source...
1
by: onceuponapriori | last post by:
Greetings gents. I'm a Railser working on a django app that needs to do some scraping to gather its data. I need to programatically access a site that requires a username and password. Once I...
3
by: Magnus.Moraberg | last post by:
Hi, I wish to extract all the words on a set of webpages and store them in a large dictionary. I then wish to procuce a list with the most common words for the language under consideration. So,...
1
by: Larry Hale | last post by:
Since it seems I have a "unique" problem, I wonder if anyone could point me in the general/right direction for tracking down the issue and resolving it myself. See my prior post @...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.