By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,768 Members | 2,054 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,768 IT Pros & Developers. It's quick & easy.

Python program that validates an url against w3c markup validator

P: n/a
I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.

Can anyone help me with this? or at least give me a hint?

Thank you so much.

Nov 29 '06 #1
Share this Question
Share on Google+
6 Replies


P: n/a
urls = [("/tmp/validate1.html", "http://www.theage.com.au")]

for url, outputfile in urls:
commandString = 'wget -o %s http://validator.w3.org/check?uri=%s' %
(outputfile, url)
system.exec(commandString)

is one easy way.

yaru22 wrote:
I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.

Can anyone help me with this? or at least give me a hint?

Thank you so much.
Nov 29 '06 #2

P: n/a
yaru22 wrote:
I was looking at the python library and thought urllib or urllib2
may be used to make this program work.

But I don't know how to send my urls to the w3c validator and get
the result.
Another great alternative is using the Twisted framework:

<http://twistedmatrix.com/projects/co...ntation/howto/
index.html>
<http://twistedmatrix.com/documents/current/api/
twisted.web2.client.http.HTTPClientProtocol.html>

Once you get familiar with the style of Twisted code, it's quite
easy to many write clients and servers because most protocols are
already implemented and must only be adapted to your needs.

Regards,
Björn

--
BOFH excuse #40:

not enough memory, go get system upgrade

Nov 29 '06 #3

P: n/a
yaru22 wrote:
I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.
this should get you going, I think:
>>import urllib
uri = "http://www.python.org"
f = urllib.urlopen("http://validator.w3.org/check?uri=" + uri)
print f.headers
Date: Wed, 29 Nov 2006 06:52:33 GMT
Server: Apache/2.0.54 (Debian GNU/Linux) mod_perl/1.999.21 Perl/v5.8.4
Content-Language: en
X-W3C-Validator-Recursion: 1
X-W3C-Validator-Status: Valid
X-W3C-Validator-Errors: 0
Connection: close
Content-Type: text/html; charset=utf-8
>>print f.headers["x-w3c-validator-status")
Valid
>>uri = "http://www.cnn.com"
f = urllib.urlopen("http://validator.w3.org/check?uri=" + uri)
print f.headers["x-w3c-validator-status"]
Invalid
>>print f.headers["x-w3c-validator-errors"]
39

</F>

Nov 29 '06 #4

P: n/a
Fredrik Lundh wrote:
>>f = urllib.urlopen("http://validator.w3.org/check?uri=" + uri)
>>print f.headers
here's the specification, btw:

http://validator.w3.org/docs/api.html

</F>

Nov 29 '06 #5

P: n/a
At Wednesday 29/11/2006 02:29, yaru22 wrote:
>I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.
Why not use the standalone validator? You can download and install it.
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
Nov 29 '06 #6

P: n/a
In article <11*********************@j44g2000cwa.googlegroups. com>,
"yaru22" <ya****@gmail.comwrote:
I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.

Can anyone help me with this? or at least give me a hint?

Thank you so much.
This article might be of help:
http://www.standards-schmandards.com/2005/massvalidate

"Periodically you may want to make sure your entire website validates.
This can be a hassle if your site is big. In this article we introduce a
few python scripts which will help us do mass validation from a list of
links. We will also modify the W3C validator to work the way we want."

You might also be interested in my validating spider (see my sig) which
will validate an entire site for you but that won't teach you a thing
about Python programming. ;)

--
Philip
http://NikitaTheSpider.com/
Whole-site HTML validation, link checking and more
Nov 29 '06 #7

This discussion thread is closed

Replies have been disabled for this discussion.