python/xpath question...

bruce

for guys with python/xpath expertise..

i'm playing with xpath.. and i'm trying to solve an issue...

i have the following kind of situation where i'm trying to get certain data.

i have a bunch of tr/td...

i can create an xpath, that gets me all of the tr.. i only want to get the
sibling tr up until i hit a 'tr' that has a 'th' anybody have an idea as to
how this query might be created?..
the idea would be to start at the "Summer B", to skip the 1st "tr", to get
the next "tr"s until you get to the next "Summer" section...

sample data.....

<tr<Th colspan=14 class="soc_comment"Summer B </th</tr>

<tr>
<td nowrap valign="bottom" class="colhelp">
<a href="#">Course
Course
 Course number and suffix, if applicable.
 C = combined lecture and lab course
 L = laboratory course
</a></td>
</tr>

<tr>
<td valign="top" nowrap><a href="javascript:crsdescunderpop('AST1002');">AST
1002</a></td>
</tr>
<tr>
<td valign="top" nowrap><a
href="javascript:crsdescunderpop('AST1022L');">AST 1022L</a></td>
</tr>
<tr>
<td valign="top" nowrap><a
href="javascript:crsdescunderpop('AST1022L');">AST 1022L</a></td>
</tr>
<tr>
<td valign="top" nowrap><a
href="javascript:crsdescunderpop('AST1022L');">AST 1022L</a></td>
</tr>
<tr<Th colspan=14 class="soc_comment"Summer C </th</tr>

<tr>
<td nowrap valign="bottom" class="colhelp">
<a href="#">Course
..
..
..

thanks...

-bruce

Jul 6 '06 #1

Subscribe Post Reply

2465

Stefan Behnel

bruce wrote:

for guys with python/xpath expertise..

i'm playing with xpath.. and i'm trying to solve an issue...

i have the following kind of situation where i'm trying to get certain data.

i have a bunch of tr/td...

i can create an xpath, that gets me all of the tr.. i only want to get the
sibling tr up until i hit a 'tr' that has a 'th' anybody have an idea as to
how this query might be created?..

I'm not quite sure how this is supposed to be related to Python, but if you're
trying to find a sibling, what about using the "sibling" axis in XPath?

Stefan

Jul 6 '06 #2

John J. Lee

(Damn gmane's authorizor, I think I lost four postings because the
auth messages went to my work email address (and I thought the
authorization was supposed to be one-time only per group anyway??). I
deleted them as spam since I hadn't posted from there for days :-(
Grrr. At least I could reconstruct this one...)

"bruce" <be*******@earthlink.netwrites:

for guys with python/xpath expertise..

i'm playing with xpath.. and i'm trying to solve an issue...

i have the following kind of situation where i'm trying to get certain data.

i have a bunch of tr/td...

i can create an xpath, that gets me all of the tr.. i only want to get the
sibling tr up until i hit a 'tr' that has a 'th' anybody have an idea as to
how this query might be created?..

[...]

((//tr/th)[2]/../following-sibling::tr/td/..)[count(.|((//tr/th)[3]/../preceding-sibling::*))=count((//tr/th)[3]/../preceding-sibling::*)]
which makes use of the following idiom for writing an intersection:

$set1[count(.|$set2)=count($set2)]
and gets the second group in the sequence you describe. IMHO, this
illustrates what happens when XPath is pushed too far ;-) I don't see
an easier way, but perhaps I missed one.

Example code:

(Note that the expression used here doesn't get any trailing group of
tr elements if there's no terminating tr/th -- that fits your
specification, but may not be what you really wanted. To fix that,
meditate on the above expression for an hour or two <0.8 wink>.)

#---------------------------------------------------------
def xpath(path, source):
import StringIO
import pprint
from lxml import etree
f = StringIO.StringIO(source)
tree = etree.parse(f)
r = tree.xpath(path)
#return "\n".join(etree.tostring(el) for el in r)
return pprint.pformat([etree.tostring(el) for el in r])

simple = """\
<html>
<tr><th>A</th></tr>
<tr><td>B</td></tr>
<tr><td>C</td></tr>
<tr><th>D</th></tr>
<tr><td>E</td></tr>
<tr><td>F</td></tr>
<tr><th>G</th></tr>
<tr><td>H</td></tr>
<tr><td>I</td></tr>
</html>
"""

for i in range(3):
expr = '((//tr/th)[%s]/../following-sibling::tr/td/..)[count(.|((//tr/th)[%s]/../preceding-sibling::*))=count((//tr/th)[%s]/../preceding-sibling::*)]' % (i+1, i+2, i+2)
print "---------------------"
print xpath(expr, simple)
#---------------------------------------------------------
john[0]$ tst.py
---------------------
['<tr><td>B</td></tr>\n', '<tr><td>C</td></tr>\n']
---------------------
['<tr><td>E</td></tr>\n', '<tr><td>F</td></tr>\n']
---------------------
[]
Knowing what you're doing, though, you'd probably be better off with
BeautifulSoup than XPath. Also note that mechanize (which I know
you're using) only supports BeautifulSoup 2 at present. You can't use
BeautifulSoup 3 yet (I hope to fix that 'RSN').
John

Jul 9 '06 #3

John J. Lee

Stefan Behnel <st******************@web.dewrites:
[...]

I'm not quite sure how this is supposed to be related to Python, but if you're
trying to find a sibling, what about using the "sibling" axis in XPath?

<nit>
There's no "sibling" axis in XPath. I'm sure you meant
"following-sibling" and/or "preceding-sibling".
</nit>
John

Jul 9 '06 #4

Similar topics

XPath and XQuery in Python?

by: Nelson Minar | last post by:

Could someone help me get started using XPath or XQuery in Python? I'm overwhelmed by all the various options and am lacking guidance on what the simplest way to go is. What library do I need to...

Python

Python xml.dom, help reading attribute data

by: Thierry Lam | last post by:

Let's say I have the following xml tag: <para role="success">1</para> I can't figure out what kind of python xml.dom codes I should invoke to read the data 1? Any help please? Thanks...

Python

xpath support in python 2.4

by: And80 | last post by:

Hi, I would like to use xpath modules in python2.4.... In my local machine I am running python2.3.5 and on the server I run python2.4. I have seen that while on my computer i am able to import...

Python

python/libxml2dom questions

by: bruce | last post by:

hi paul... in playing around with the test python app (see below) i've got a couple of basic questions. i can't seem to find the answers via google, and when i've looked in the libxml2dom stuff...

Python

RE: python screen scraping/parsing

by: bruce | last post by:

Hi Paul... Thanks for the reply. Came to the same conclusion a few minutes before I saw your email. Another question: tr=d.xpath(foo) gets me an array of nodes.

Python

python - firefox dom/xpath question/issue

by: bruce | last post by:

Hi. Got a test web page, that basically has two "<html" tags in it. Examining the page via Firefox/Dom Inspector, I can create a test xpath query "/html/body/form" which gets the target form for...

Python

python mechanize/libxml2dom question

by: bruce | last post by:

hi... i've got the following situation, with the following test url: "http://schedule.psu.edu/soc/fall/Alloz/a-c/acctg.html#". i can generate a list of the tables i want for the courses on the...

Python

python/xpath question..

by: bruce | last post by:

morning.... i apologize up front as this is really more of an xpath question.. in my python, i'm using the xpath function to iterate/parse some html. i can do something like ...

Python

Re: python/xpath question..

by: John Krukoff | last post by:

On Wed, 2008-09-03 at 13:36 -0700, bruce wrote: Well, you could just do the test (and the count!) in the xpath expression: count( //tr/td ) It sounds like you're not familiar with xpath? I...

Python

Easy Steps to Fix "Canon Printer Won't Connect to WiFi Network"

by: taylorcarr | last post by:

A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...

General

Basic Javascript concepts

by: aa123db | last post by:

Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...

Javascript

Merging data from multiple Excel files

by: ryjfgjl | last post by:

In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...

Data Management

Migrating Website to Cloud - Emmanuel Katto

by: emmanuelkatto | last post by:

Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel

General

Navigating the Data Structures and Algorithms (DSA)

by: BarryA | last post by:

What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...

Algorithms / Advanced Math

Looking to do Android software development, any suggestions? Is flutter better?

by: nemocccc | last post by:

hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?

General

What is ONU?

by: marktang | last post by:

ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...

General

Changing the language in Windows 10

by: Hystou | last post by:

Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...

Windows Server

Maximizing Business Potential: The Nexus of Website Design and Digital Marketing

by: jinu1996 | last post by:

In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

Online Marketing