By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
448,787 Members | 1,250 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 448,787 IT Pros & Developers. It's quick & easy.


P: 103
I do not understand what additional parameters i have to pas to urllib.request to download a full webpage.


For instance, if you open
on the right bottom you will find "usage examples".

If you try to save wabpage and than reopen it:
file:///C:/Users/Gintare/Downloads/ htm
the "usage examples" will be displayed.

But if you use:
Expand|Select|Wrap|Line Numbers
  1. res=""
  2. resp = urllib.request.urlopen(res)
  3. html =             
  4. soup=BeautifulSoup(html)                          shtml=BeautifulSoup.prettify(soup)                           fFI=open('requested_file.html','w',encoding='utf-8')
  5. fFI.writelines(shtml)
  6. fFI.close()
The 'requested_file.html' will not have the section "usage examples".

How to download full webpage with the section "usage examples" using urllib.request?

2) How to retrieve other examples, which normally are accessible by pressing hyperlink "next". There are more than 40 examples for this word, but hey are displayed only in portions by 4.
Mar 28 '15 #1
Share this Question
Share on Google+
2 Replies

Expert Mod 2.5K+
P: 3,503
Have you tried simply using:

Expand|Select|Wrap|Line Numbers
  1. urllib.urlopen
Take out the 'request' portion and try that instead.
Apr 7 '15 #2

P: 103
I am getting error:
AttributeError at /search1

'module' object has no attribute 'urlopen'

Request Method: GET
Request URL:
Django Version: 1.6.2
Exception Type: AttributeError
Exception Value:

on line 1000: resp = urllib.urlopen(res)
Apr 7 '15 #3

Post your reply

Sign in to post your reply or Sign up for a free account.