473,322 Members | 1,523 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Trimming X/HTML files

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dear,

I need to parse XHTML/HTML files in all ways :
~ _ Removing comments and javascripts is a first issue
~ _ Retrieving the list of fields to submit is my following item (todo)

Any idea where I could find this already made ... ?

\T,
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFC6CAqqN0SJr+xLBURAkc5AKCAb5pdsUZjCezr8bcpd2 GTJ9FueACg7raK
pMU5Y4Z7PaPGfGZkJ/wCDpw=
=iM7V
-----END PGP SIGNATURE-----
Jul 28 '05 #1
2 1410
Thomas SMETS wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dear,

I need to parse XHTML/HTML files in all ways :
~ _ Removing comments and javascripts is a first issue
~ _ Retrieving the list of fields to submit is my following item (todo)

Any idea where I could find this already made ... ?


You could try XIST (http://www.livinglogic.de/Python/xist).

Removing comments and javascripts works like this:

---
from ll.xist import xsc, parsers
from ll.xist.ns import html

e = parsers.parseURL("http://www.python.org/", tidy=True)

def removestuff(node, converter):
if isinstance(node, xsc.Comment):
node = xsc.Null
elif isinstance(node, html.script) and \
(unicode(node["type"]) == u"text/javascript" or \
unicode(node["language"]) == u"Javascript" \
):
node = xsc.Null
return node

e = e.mapped(removestuff)

print e.asBytes()
---

Retrieving the list of fields from all forms on a page might look like this:

---
from ll.xist import xsc, parsers, xfind
from ll.xist.ns import html

e = parsers.parseURL("http://www.python.org/", tidy=True)

for form in e//html.form:
print "Fields for %s" % form["action"]
for field in form//xfind.is_(html.input, html.textarea):
if "id" in field.attrs:
print "\t%s" % field["id"]
else:
print "\t%s" % field["name"]
---

This prints:

Fields for http://www.google.com/search
q
domains
sitesearch
sourceid
submit

Hope that helps!

Bye,
Walter Dörwald
Jul 28 '05 #2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
The regular expression remove script out of an HTML/XHTML file is simple
enough but raises a major performance issue....

The following regular expression :
r'(<script(\s*\S+\s*)+</script>)'
takes ages to complete in python on simple HTML file more than 3 minutes
of CPU time on a 150 lines HTML file. In jython it just never completes
but returns a painfull RunTimeException : maximum number of ??? reached.

Is the only way out dealing with strings and "match" instead of regular
expression ?
More over Jython is not yet 2.3 compliant, hence advanced features of
2.3 regular expression are not yet available !

\T,


Thomas SMETS wrote:
|
| Dear,
|
| I need to parse XHTML/HTML files in all ways :
| ~ _ Removing comments and javascripts is a first issue
| ~ _ Retrieving the list of fields to submit is my following item (todo)
|
| Any idea where I could find this already made ... ?
|
| \T,
|
|

- --
Thomas SMETS
Bruxelles
@ : du**************@yahoo.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD4DBQFC7OkTqN0SJr+xLBURAuTYAKDLxLv+hpnSrZ6uowOmUc zVxgxLqwCYhfJ3
fwjPZzg88gh3lNY8jkG3SA==
=urIC
-----END PGP SIGNATURE-----
Jul 31 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Simon | last post by:
Hi there. I seem to be a flurry of questions here recently, thanks for you help all, but ... got another question here. I am storing URL in a database for a 'sort of' directory type website. ...
6
by: Generic Usenet Account | last post by:
I have worked out the following implementation for trimming the leading and trailing whitespace characters of a string (I am surprised not to see these as member functions). Am I doing it...
12
by: Stefan Weiss | last post by:
Hi. (this is somewhat similar to yesterday's thread about empty links) I noticed that Tidy issues warnings whenever it encounters empty tags, and strips those tags if cleanup was requested....
13
by: john_g83 | last post by:
have a bit of c code that is ment to take a string (that may or may not have spaces before or after the string) i.e. " stuff ", and trims off the whitespace before and after. Code: char *trim...
4
by: Chris | last post by:
Hi, I have the follwing url http://localhost/testsite/test/webform1.aspx?cat=dog Is there a function available in the request.url object to return only the webform1.aspx or even...
0
by: Paul | last post by:
On my local site, I have a folder that is security trimmed, so that only members of a Role can see it after they register and log on (I set the memberships). All works fine locally. However,...
3
by: Enyi | last post by:
Situation : Need to get a list of files in a directory, but I am currently getting the whole path of the file (Example: "C:\A Folder\Another Folder\file.txt"). What I want is just the filename and...
7
by: veg_all | last post by:
I am looking for a simple way to keep my log files from growing too large. Basically I would want something that truncates the off first 25 kb of a 100kb log file. I could do something with reading...
1
by: Harald Korneliussen | last post by:
Can anyone tell me of a program that does this? When working on large applications that promiscously include large header files, it would be nice to have an automatic way of figuing out which...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.