473,556 Members | 2,558 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Trimming X/HTML files

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dear,

I need to parse XHTML/HTML files in all ways :
~ _ Removing comments and javascripts is a first issue
~ _ Retrieving the list of fields to submit is my following item (todo)

Any idea where I could find this already made ... ?

\T,
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFC6CAqqN0 SJr+xLBURAkc5AK CAb5pdsUZjCezr8 bcpd2GTJ9FueACg 7raK
pMU5Y4Z7PaPGfGZ kJ/wCDpw=
=iM7V
-----END PGP SIGNATURE-----
Jul 28 '05 #1
2 1423
Thomas SMETS wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dear,

I need to parse XHTML/HTML files in all ways :
~ _ Removing comments and javascripts is a first issue
~ _ Retrieving the list of fields to submit is my following item (todo)

Any idea where I could find this already made ... ?


You could try XIST (http://www.livinglogic.de/Python/xist).

Removing comments and javascripts works like this:

---
from ll.xist import xsc, parsers
from ll.xist.ns import html

e = parsers.parseUR L("http://www.python.org/", tidy=True)

def removestuff(nod e, converter):
if isinstance(node , xsc.Comment):
node = xsc.Null
elif isinstance(node , html.script) and \
(unicode(node["type"]) == u"text/javascript" or \
unicode(node["language"]) == u"Javascript " \
):
node = xsc.Null
return node

e = e.mapped(remove stuff)

print e.asBytes()
---

Retrieving the list of fields from all forms on a page might look like this:

---
from ll.xist import xsc, parsers, xfind
from ll.xist.ns import html

e = parsers.parseUR L("http://www.python.org/", tidy=True)

for form in e//html.form:
print "Fields for %s" % form["action"]
for field in form//xfind.is_(html. input, html.textarea):
if "id" in field.attrs:
print "\t%s" % field["id"]
else:
print "\t%s" % field["name"]
---

This prints:

Fields for http://www.google.com/search
q
domains
sitesearch
sourceid
submit

Hope that helps!

Bye,
Walter Dörwald
Jul 28 '05 #2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
The regular expression remove script out of an HTML/XHTML file is simple
enough but raises a major performance issue....

The following regular expression :
r'(<script(\s*\ S+\s*)+</script>)'
takes ages to complete in python on simple HTML file more than 3 minutes
of CPU time on a 150 lines HTML file. In jython it just never completes
but returns a painfull RunTimeExceptio n : maximum number of ??? reached.

Is the only way out dealing with strings and "match" instead of regular
expression ?
More over Jython is not yet 2.3 compliant, hence advanced features of
2.3 regular expression are not yet available !

\T,


Thomas SMETS wrote:
|
| Dear,
|
| I need to parse XHTML/HTML files in all ways :
| ~ _ Removing comments and javascripts is a first issue
| ~ _ Retrieving the list of fields to submit is my following item (todo)
|
| Any idea where I could find this already made ... ?
|
| \T,
|
|

- --
Thomas SMETS
Bruxelles
@ : du************* *@yahoo.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD4DBQFC7OkTqN0 SJr+xLBURAuTYAK DLxLv+hpnSrZ6uo wOmUczVxgxLqwCY hfJ3
fwjPZzg88gh3lNY 8jkG3SA==
=urIC
-----END PGP SIGNATURE-----
Jul 31 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
1370
by: Simon | last post by:
Hi there. I seem to be a flurry of questions here recently, thanks for you help all, but ... got another question here. I am storing URL in a database for a 'sort of' directory type website. The <a href> is built from the recordset of URL's which is all working no problem, easy stuff.
6
4824
by: Generic Usenet Account | last post by:
I have worked out the following implementation for trimming the leading and trailing whitespace characters of a string (I am surprised not to see these as member functions). Am I doing it correctly, or is there a better way to do it? Thanks, Gus //////////// Source code begin /////////////////////
12
17085
by: Stefan Weiss | last post by:
Hi. (this is somewhat similar to yesterday's thread about empty links) I noticed that Tidy issues warnings whenever it encounters empty tags, and strips those tags if cleanup was requested. This is okay in some cases (such as <tbody>), but problematic for other tags (such as <option>). Some tags (td, th, ...) do not produce warnings when...
13
3654
by: john_g83 | last post by:
have a bit of c code that is ment to take a string (that may or may not have spaces before or after the string) i.e. " stuff ", and trims off the whitespace before and after. Code: char *trim (char *str, char ch) { char *first, *last; int count;
4
1183
by: Chris | last post by:
Hi, I have the follwing url http://localhost/testsite/test/webform1.aspx?cat=dog Is there a function available in the request.url object to return only the webform1.aspx or even /webform1.aspx? Thanks
0
1202
by: Paul | last post by:
On my local site, I have a folder that is security trimmed, so that only members of a Role can see it after they register and log on (I set the memberships). All works fine locally. However, when I deployed to my ISP, the security trimming does not work. Here is the relevant code. I can add users just fine, they show up in the ISP SQL...
3
1952
by: Enyi | last post by:
Situation : Need to get a list of files in a directory, but I am currently getting the whole path of the file (Example: "C:\A Folder\Another Folder\file.txt"). What I want is just the filename and extension. So from above example, all I actually want is "file.txt". Language: VB.NET 2005 Current Code: Imports...
7
1827
by: veg_all | last post by:
I am looking for a simple way to keep my log files from growing too large. Basically I would want something that truncates the off first 25 kb of a 100kb log file. I could do something with reading in the file twice, first to determine number of lines. Then a second time to write a copy of the reduced log file. Any simpler workarounds?
1
1296
by: Harald Korneliussen | last post by:
Can anyone tell me of a program that does this? When working on large applications that promiscously include large header files, it would be nice to have an automatic way of figuing out which aren't needed any longer. Any IDEs? Or could someone give me a lint?
0
7547
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
6178
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5452
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5171
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3597
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3576
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2040
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1161
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
865
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.