473,654 Members | 3,089 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Good HTML Parser

Can anyone recommend a good HTML/XHTML parser, similar to
HTMLParser.HTML Parser or htmllib.HTMLPar ser, but able to intelligently
know that certain tags, like <br>, are implicitly closed? I need to
iterate through the entire DOM, building up a DOM path, but the stdlib
parsers aren't calling handle_endtag() for any implicitly closed tags.
I looked at BeautifulSoup, but it only seems to work by first parsing
the entire document, then allowing you to query the document
afterwards. I need something like a SAX parser.
Jul 17 '08 #1
2 1583
Chris wrote:
Can anyone recommend a good HTML/XHTML parser, similar to
HTMLParser.HTML Parser or htmllib.HTMLPar ser, but able to intelligently
know that certain tags, like <br>, are implicitly closed? I need to
iterate through the entire DOM, building up a DOM path, but the stdlib
parsers aren't calling handle_endtag() for any implicitly closed tags.
I looked at BeautifulSoup, but it only seems to work by first parsing
the entire document, then allowing you to query the document
afterwards. I need something like a SAX parser.
This isn't possible. Your own example of arbitrarily closeable Tags needs
context that just a SAX-like parser can't provide.

I suggest you use BeautifulSoup, and if you must create your own
event-generation around that which you can attach consumers to.

Diez
Jul 17 '08 #2
Chris wrote:
Can anyone recommend a good HTML/XHTML parser, similar to
HTMLParser.HTML Parser or htmllib.HTMLPar ser, but able to intelligently
know that certain tags, like <br>, are implicitly closed? I need to
iterate through the entire DOM, building up a DOM path, but the stdlib
parsers aren't calling handle_endtag() for any implicitly closed tags.
I looked at BeautifulSoup, but it only seems to work by first parsing
the entire document, then allowing you to query the document
afterwards. I need something like a SAX parser.
Try lxml.html. It's very memory friendly and extremely fast, so you may end up
without any reason to use SAX anymore.

http://codespeak.net/lxml/

Stefan
Jul 17 '08 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
2588
by: YoBro | last post by:
Hi I have used some of this code from the PHP manual, but I am bloody hopeless with regular expressions. Was hoping somebody could offer a hand. The output of this will put the name of a form field beside name. I want to get the following but not sure how to modify the code below. 1. Field Name (to appear beside NAME:) 2. Field Type (to appear beside TYPE:)
3
3499
by: Mitchua | last post by:
When I run the well quoted line: my $ascii = HTML::FormatText->new->format(HTML::Parse::parse_html($html)); to remove HTML tags from an html document, it replaces all tables with "". Is there a quick and easy way to get the table content parsed too? Thanks a lot, Mitchua
0
2715
by: Himanshu Garg | last post by:
Hello, I am using HTML::Parser to extract text from html pages from http://bbc.co.uk/urdu/ However the encoding of the input text seems to change to some unknown encoding in the output. The program is given below. The HTML is in a string to keep the example simple. The same problem appears with HTML in a file.
3
3121
by: Himanshu Garg | last post by:
Hello, I am trying to pinpoint an apparent bug in HTML::Parser. The encoding of the text seems to change incorrectly if the locale isn't set properly. However Parser.pm in the directory (/usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi/HTML/) doesn't seem to contain the "real" parsing statements.
6
5458
by: Els | last post by:
If I use <? include "file.html"; ?> in the html of my document, do I _have_ to change the extension of that document to .php, or would it still work and be valid if I let it remain .html? -- Els Mente humana é como pára-quedas; funciona melhor aberta.
3
685
by: SEGACO | last post by:
Hi, Can someone tell me if C# includes something to parse HTML? Thanks.
2
2660
by: David Virgil Hobbs | last post by:
Loading text strings containing HTML code into an HTML parser in a Javascript/Jscript I would like to know, how one would go about loading a text string containing HTML code, so as to be able to use javascript or Jscript to work with the HTML code in the text string, in the same way that one works with XML code in a text string using the XML parser. If I was able to load the text string containing the HTML code succesfully, I would be...
0
1991
by: june | last post by:
Hi, I have a big problem with parsing HTML into a XHTML using Cberneko to validate the html. First I tried to work with a HTML-File. This solutions works fine: String aHTMLFile = "file:\\C:/work/Eclipse3.1.1/html-file.html"; org.xml.sax.InputSource pSource = new InputSource(aHTMLFile);
4
1917
by: Jackie | last post by:
Hi, all, I want to get the information of the professors (name,title) from the following link: "http://www.economics.utoronto.ca/index.php/index/person/faculty/" Ideally, I'd like to have a output file where each line is one Prof, including his name and title. In practice, I use the CSV module.
0
8379
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
1
8494
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8596
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6162
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5627
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4150
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4297
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2719
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
1597
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.