Python program that validates an url against w3c markup validator

[yaru22]

I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.

Can anyone help me with this? or at least give me a hint?

Thank you so much.

[tleeuwenburg]

urls = [("/tmp/validate1.html", "http://www.theage.com.au")]

for url, outputfile in urls:
commandString = 'wget -o %s http://validator.w3.org/check?uri=%s' %
(outputfile, url)
system.exec(commandString)

is one easy way.

yaru22 wrote:

I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.

Can anyone help me with this? or at least give me a hint?

Thank you so much.

[Bjoern Schliessmann]

yaru22 wrote:

I was looking at the python library and thought urllib or urllib2
may be used to make this program work.

But I don't know how to send my urls to the w3c validator and get
the result.

Another great alternative is using the Twisted framework:

<http://twistedmatrix.com/projects/co...ntation/howto/
index.html>
<http://twistedmatrix.com/documents/current/api/
twisted.web2.client.http.HTTPClientProtocol.html>

Once you get familiar with the style of Twisted code, it's quite
easy to many write clients and servers because most protocols are
already implemented and must only be adapted to your needs.

Regards,
Björn

--
BOFH excuse #40:

not enough memory, go get system upgrade

[Fredrik Lundh]

yaru22 wrote:

I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.

this should get you going, I think:

>>import urllib
uri = "http://www.python.org"
f = urllib.urlopen("http://validator.w3.org/check?uri=" + uri)
print f.headers

Date: Wed, 29 Nov 2006 06:52:33 GMT
Server: Apache/2.0.54 (Debian GNU/Linux) mod_perl/1.999.21 Perl/v5.8.4
Content-Language: en
X-W3C-Validator-Recursion: 1
X-W3C-Validator-Status: Valid
X-W3C-Validator-Errors: 0
Connection: close
Content-Type: text/html; charset=utf-8

>>print f.headers["x-w3c-validator-status")

Valid

>>uri = "http://www.cnn.com"
f = urllib.urlopen("http://validator.w3.org/check?uri=" + uri)
print f.headers["x-w3c-validator-status"]

Invalid

>>print f.headers["x-w3c-validator-errors"]

39

</F>

[Fredrik Lundh]

Fredrik Lundh wrote:

>>f = urllib.urlopen("http://validator.w3.org/check?uri=" + uri)
>>print f.headers

here's the specification, btw:

http://validator.w3.org/docs/api.html

</F>

[Gabriel Genellina]

At Wednesday 29/11/2006 02:29, yaru22 wrote:

>I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

Why not use the standalone validator? You can download and install it.
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

[Nikita the Spider]

In article <11*********************@j44g2000cwa.googlegroups. com>,
"yaru22" <ya****@gmail.comwrote:

I'd like to create a program that validates bunch of urls against the
w3c markup validator (http://validator.w3.org/) and store the result in
a file.

Since I don't know network programming, I have no idea how to start
coding this program.

I was looking at the python library and thought urllib or urllib2 may
be used to make this program work.

But I don't know how to send my urls to the w3c validator and get the
result.

Can anyone help me with this? or at least give me a hint?

Thank you so much.

This article might be of help:
http://www.standards-schmandards.com/2005/massvalidate

"Periodically you may want to make sure your entire website validates.
This can be a hassle if your site is big. In this article we introduce a
few python scripts which will help us do mass validation from a list of
links. We will also modify the W3C validator to work the way we want."

You might also be interested in my validating spider (see my sig) which
will validate an entire site for you but that won't teach you a thing
about Python programming. ;)

--
Philip
http://NikitaTheSpider.com/
Whole-site HTML validation, link checking and more