By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,050 Members | 1,020 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,050 IT Pros & Developers. It's quick & easy.

liburl cant load webpage with Javascript

P: n/a
Hi,

I want do use liburl to scan a webpage which is only accessible from within
my LAN environment. While mozilla manages to load the target URL properly
neither wget, nor liburl or liburl2 does.
I had a closer look at the html source and discovered a lot of Javascript,
including Cookies.

My suspicion is that the Javascript code needs to be executed for the page
to work properly. Also I don't know how liburl deals with Cookies, but
since they are handled by the Javascript in the source code they are
probably not considered at all.

In any case I get an IOError: connection refused, Error Code 111.

Does anyone know a way out of this?

Thanks for any hints,
Ciao
Uwe
Jul 18 '05 #1
Share this Question
Share on Google+
2 Replies


P: n/a
Uwe Mayer <me*****@hadiko.de> wrote in message news:<c8*********@news2.rz.uni-karlsruhe.de>...
Hi,

I want do use liburl to scan a webpage which is only accessible from within
my LAN environment. While mozilla manages to load the target URL properly
neither wget, nor liburl or liburl2 does.
I had a closer look at the html source and discovered a lot of Javascript,
including Cookies.

My suspicion is that the Javascript code needs to be executed for the page
to work properly. Also I don't know how liburl deals with Cookies, but
since they are handled by the Javascript in the source code they are
probably not considered at all.

In any case I get an IOError: connection refused, Error Code 111.

Does anyone know a way out of this?

Thanks for any hints,
Ciao
Uwe


Mozilla is a web browser, and it implements cookies, DOM for HTML
pages, and a Javascript interpreter with objects representing browser
automation.
It's unlikely and inappropriate for low level HTTP implementations
like wget and liburl to have that kind of support for advanced web
features; maybe you can support cookies and Javascript in your
application.

In the specific case of "IOError: connection refused, Error Code 111",
however, the failure seems to happen at a lower protocol level: wrong
host names or port numbers, unavailable servers and maybe proxy
authentication requirements are the usual causes of refused
connections.

Lorenzo Gatti
Jul 18 '05 #2

P: n/a
ga***@dsdata.it (Lorenzo Gatti) writes:
Uwe Mayer <me*****@hadiko.de> wrote in message news:<c8*********@news2.rz.uni-karlsruhe.de>... [...]
I had a closer look at the html source and discovered a lot of Javascript,
including Cookies.

[...] Mozilla is a web browser, and it implements cookies, DOM for HTML
pages, and a Javascript interpreter with objects representing browser
automation.
It's unlikely and inappropriate for low level HTTP implementations
like wget and liburl to have that kind of support for advanced web [...]

JavaScript support is rare, but many libraries and tools support
cookies (including wget and my library, ClientCookie -- essentially a
drop-in replacement for urllib2). For JS, see my FAQ here (under
"Embedded script is messing up my web-scraping. What do I do?"):

http://wwwsearch.sourceforge.net/bits/GeneralFAQ.html

In the specific case of "IOError: connection refused, Error Code 111",
however, the failure seems to happen at a lower protocol level: wrong

[...]

Right.
John
Jul 18 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.