Hi,
I am using the HTMLParser to parse a web page, part of the routine I need
to write (I am new to Python) involves looking for a particular tag and
once I know the start and the end of the tag then to assign all the data
in between the tags to a variable, this is easy if the tag starts and ends
on the same line however how would I go about doing it if its split over
two or more lines?
Thanks
R 10 2804
Rigga wrote: I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
Perhaps you should glue the whole text as a one long line?
--
Pawel Kraszewski FreeBSD/Linux
E-Mail/Jabber Phone ICQ GG Pa************* *@wp.pl +48 604 777447 45615564 69381
Rigga wrote: Hi,
I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
Thanks
R
Don't re-invent the wheel, http://www.crummy.com/software/BeautifulSoup/
--
Nigel Rowe
A pox upon the spammers that make me write my address like..
rho (snail) swiftdsl (stop) com (stop) au
On Sun, 20 Jun 2004 21:03:34 +1000, Nigel Rowe wrote: Rigga wrote:
Hi,
I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
Thanks
R
Don't re-invent the wheel, http://www.crummy.com/software/BeautifulSoup/
I want to do it manually as it will help with my understanding of Python,
any ideas how I go about it?
Rigga wrote: Hi,
I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
What difference does it make that the text is spread over more than one
line? Just collect the data in handle_data.
--
Regards,
Diez B. Roggisch
Rigga wrote: On Sun, 20 Jun 2004 21:03:34 +1000, Nigel Rowe wrote:Don't re-invent the wheel, http://www.crummy.com/software/BeautifulSoup/
I want to do it manually as it will help with my understanding of Python, any ideas how I go about it?
Wouldn't it help you a lot more then, to figure it out on your
own? ;-)
(If you are really looking for help improving your understanding
of Python, reading the source for BeautifulSoup and figuring out
how *it* does it will probably get you farther than figuring it
out yourself. If, on the other hand, it's a better understanding
of *programming* that you are after, then doing it yourself is
the best bet... IMHO )
-Peter
"Rigga" <Ri***@hasnomai l.com> wrote in message
news:pa******** *************** *****@hasnomail .com... Hi,
I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
Thanks
Depending on exactly what I want to do, I frequently use <file>.read()
to pick up the entire file in one string, rather than <file>.readline s() to
create a list of strings. It works quite well when what I need to do
can be served by regexs (which is not always the case.)
John Roth R
Rigga <Ri***@hasnomai l.com> writes: I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
I often have variants of this problem too. The simplest way to make it
work is to read all the HTML in at once with a single call to
file.read(), and then use a regular expression. Note that you probably
don't need re.MULTILINE, although you should take a look at what it
means in the docs just to know.
This works fine as long as you expect your files to be relatively
small (under a meg or so).
On Sun, 20 Jun 2004 17:22:53 +0000, Nelson Minar wrote: Rigga <Ri***@hasnomai l.com> writes: I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
I often have variants of this problem too. The simplest way to make it work is to read all the HTML in at once with a single call to file.read(), and then use a regular expression. Note that you probably don't need re.MULTILINE, although you should take a look at what it means in the docs just to know.
This works fine as long as you expect your files to be relatively small (under a meg or so).
Im reading the entire file in to a variable at the moment and passing it
through HTMLParser. I have ran in to another problem that I am having a
hard time working out, my data is in this format:
<TD><SPAN class=qv id=EmployeeNo
title="Employee Number">123456</SPAN></TD></TR>
Some times the data is spread over 3 lines like:
<TD><SPAN class=qv id=BusinessName
title="Business Name">Some Shady Business
Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>
The data I need to get is the data enclosed in quotes after the word
title= and data after the > and before the </SPAN, in the case aove would
be: Some Shady Business
Group Ltd.
Running the file through HTMLParser I discovered that the title= part
and the data part I need is contained in a list therefore I have done this:
snippet of my code:
class MyHTMLParser(HT MLParser):
def handle_starttag (self, tag, attrs):
print "Encountere d the beginning of a %s tag" % tag
def handle_data(sel f, data):
if "title=" in data:
print "found title"
However I can not work out how to search through the data (which is in a
list) to pull out the data I need.
Sorry if this is a dumb question but hey I am learning!
Many thanks
Rigga
Rigga <Ri***@hasnomai l.com> wrote: On Sun, 20 Jun 2004 17:22:53 +0000, Nelson Minar wrote:
Rigga <Ri***@hasnomai l.com> writes: I am using the HTMLParser to parse a web page, part of the routine I need to write (I am new to Python) involves looking for a particular tag and once I know the start and the end of the tag then to assign all the data in between the tags to a variable, this is easy if the tag starts and ends on the same line however how would I go about doing it if its split over two or more lines?
I often have variants of this problem too. The simplest way to make it work is to read all the HTML in at once with a single call to file.read(), and then use a regular expression. Note that you probably don't need re.MULTILINE, although you should take a look at what it means in the docs just to know.
This works fine as long as you expect your files to be relatively small (under a meg or so).
Im reading the entire file in to a variable at the moment and passing it through HTMLParser. I have ran in to another problem that I am having a hard time working out, my data is in this format:
<TD><SPAN class=qv id=EmployeeNo title="Employee Number">123456</SPAN></TD></TR>
Some times the data is spread over 3 lines like:
<TD><SPAN class=qv id=BusinessName title="Business Name">Some Shady Business Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>
The data I need to get is the data enclosed in quotes after the word title= and data after the > and before the </SPAN, in the case aove would be: Some Shady Business Group Ltd.
Approach:
1. Extract '<SPAN ([^>]*)>([^<]*)</SPAN>' which is
<SPAN class=qv id=BusinessName
title="Business Name">Some Shady Business
Group Ltd.</SPAN>
with parenthized groups giving
submatch[1]='class=qv id=BusinessName \ntitle="Busine ss Name"'
submatch[2]='Some Shady Business\nGroup Ltd.'
2. Split submatch[1] into
class=qv
id=BusinessName
title="Business Name"
Homework:
Write a Python script.
Bash solution:
First, you need my patched Bash which can be found at http://freshmeat.net/projects/bashdiff/
You need to patch the Bash shell, and compile. It has many Python
features, particularly regex and array. Shell solution is
text='<TD><SPAN class=qv id=BusinessName
title="Business Name">Some Shady Business
Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>'
newf () { # Usage: newf match submatch1 submatch2
eval $2 # --> class, id, title
echo $title > title
echo $3 > name
}
x=()
array -e '<SPAN ([^>]*)>([^<]*)</SPAN>' -E newf x "$text"
cat title
cat name
I can explain the steps, that it's rather long. :-)
--
William Park, Open Geometry Consulting, <op**********@y ahoo.ca>
No, I will not fix your computer! I'll reformat your harddisk, though. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Daniel Hill |
last post by:
OK, I have very, VERY basic background knowledge of VB6
and have now upgraded to VB.NET and now I'm struggling to
bring up the forms I want.
What I am looking to do is to have a click a command
button bring up a form, and to have which form is brought
up determined by which item is selected in the
accompanying combo box.
I've gathered that you have to declare the item as a
|
by: Michael Hill |
last post by:
Hi, folks.
I am writing a Javascript program that accepts (x, y) data pairs from
a text box and then analyzes that data in various ways. This is my
first time using text area boxes; in the past, I have used individual
entry fields for each variable. I would now like to use text area
boxes to simplify the data entry (this way, data can be produced by
another program--FORTRAN, "C", etc.--but analyzed online, so long as
it is first...
|
by: Chris |
last post by:
Hi
I can use a text file as a datasource but am unable to get the datatable to
see the text file as having multiple columns. Everything gets
put into the first column in the datatable. Sample of code and text file
included.
Please help
Regards
|
by: MLH |
last post by:
Am trying to import 20,000+ lines of text in a file
FTP'd from a UNIX platform to windows via FTP
session in a DOS box. About 2000 records have
multiple lines in them separated by CRLF's.
That's not the idea. Each line was supposed to
be a separate record. I don't know why the 2000
records having MULTIPLE lines of text are so
stubborn. I've run UNIX2DOS utility against the
raw text file - no help. I know that UNIX and MAC
|
by: Edson Peacock |
last post by:
I have a report with sub reports, one of the subreports have 12 text boxes
that are 2" high and I want them all to grow if one goes to 3" high.
If anyone has any suggestions they are very much appreciated.
Thanks
| |
by: DH |
last post by:
Hi,
I'm trying to strip the html and other useless junk from a html page..
Id like to create something like an automated text editor, where it
takes the keywords from a txt file and removes them from the html page
(replace the words in the html page with blank space) I'm new to python
and could use a little push in the right direction, any ideas on how to
implement this?
Thanks!
|
by: Ed |
last post by:
Hi,
Could you tell me what the equivalent of this perl code would be in C
please?
print EOT<<
I want
to print over
multiple lines in c
without doing lots of printfs
|
by: watsod1 |
last post by:
Hello,
This is my first post, Hello to all. This also a test post to make sure that I am doing the right thing and following rules etc.
I have been searching for a way to make the treeview display multiple lines of text on each treeview node (treenode).
After searching for answers and not finding any, I stumbled on the solution in the msdn help files (search for TreeView.DrawNode Event). It has source code that shows how to display...
|
by: kronecker |
last post by:
I am trying to delete multiple lines in a text file using the
following
Private Sub Read_TextFile()
Dim objReader As StreamReader
Dim strfull, strContents, strContentsold, strContentsnew As
String
objReader = New StreamReader("C:\answer.txt")
'Clear the Text Box1
TextBox1.Clear()
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |