SGML parsing tags and leeping track

[hapaboy2059]

Hello,

I need help in using sgmlparser to parse a html file and keep track of
the number of times each tag is being used.

In the end of this program I need to print out the number of times each
tag was seen(presumably any type of tag can be used) and the linked
text.

I need help in getting past the first steps. I already have this basic
program to return hyperlinks. I cant seem to understand how to parse
any tag and keep track of it to print it out at a later time....

very frustrated and help is appreciated!!!! !

--------------------------------------------------------------------------
import sgmllib, urllib

class HtmParser(sgmll ib.SGMLParser):
def __init__(self, verbose=0):
"Initialise an object, passing 'verbose' to the superclass."

sgmllib.SGMLPar ser.__init__(se lf, verbose)
self.hyperlinks = []
self.descriptio ns = []
self.inside_a_e lement = 0

def start_a(self, attributes):
"Process a hyperlink and its 'attributes'."

for name, value in attributes:
if name == "href":
self.hyperlinks .append(value)

def get_hyperlinks( self):
"Return the list of hyperlinks."

return self.hyperlinks
parser = HtmParser()

inptAdrs = raw_input('Plea se input the absolute path to the url\n')
print 'you entered: ', inptAdrs

content = urllib.urlopen( inptAdrs)

bufff = content.read()
print 'Statistics for ', inptAdrs

print 'There is', len(bufff), 'characters in the web page'

parser.feed(buf ff)
print parser.get_hype rlinks()
parser.close()
---------------------------------------------------------------------------------

any help is much appreciated

[hapaboy2059]

could i make a global variable and keep track of each tag count?

Also how would i make a list or dictionary of tags that is found?
how can i handle any tag that is given?

[Heiko Wundram]

Am Dienstag 02 Mai 2006 20:38 schrieb ha*********@gma il.com:

could i make a global variable and keep track of each tag count?

Also how would i make a list or dictionary of tags that is found?
how can i handle any tag that is given?

The following snippet does what you want:

from sgmllib import SGMLParser

class MyParser(SGMLPa rser):

def __init__(self):
SGMLParser.__in it__(self)
self.tagcount = {}
self.links = set()

# Tag count handling
# ------------------

def handle_starttag (self,tag,metho d,args):
self.tagcount[tag] = self.tagcount.g et(tag,0) + 1
method(args)

def unknown_startta g(self,tag,args ):
self.tagcount[tag] = self.tagcount.g et(tag,0) + 1

# Argument handling
# -----------------

def start_a(self,ar gs):
self.links.upda te([value for name, value in args if name == "href"])

parser = MyParser()
parser.feed(fil e("test.html"). read()) # Insert your data source here...
parser.close()

print parser.tagcount
print parser.links

See the documentation for sgmllib for more info on handle_starttag (whose
logic might just as well have been implemented in start_a, but if you want
argument handling for more tags, it's best to keep it at this one central
place) and unknown_startta g.

--- Heiko.

[Heiko Wundram]

Am Dienstag 02 Mai 2006 20:38 schrieb ha*********@gma il.com:

could i make a global variable and keep track of each tag count?

Also how would i make a list or dictionary of tags that is found?
how can i handle any tag that is given?

The following snippet does what you want:

from sgmllib import SGMLParser

class MyParser(SGMLPa rser):

def __init__(self):
SGMLParser.__in it__(self)
self.tagcount = {}
self.links = set()

# Tag count handling
# ------------------

def handle_starttag (self,tag,metho d,args):
self.tagcount[tag] = self.tagcount.g et(tag,0) + 1
method(args)

def unknown_startta g(self,tag,args ):
self.tagcount[tag] = self.tagcount.g et(tag,0) + 1

# Argument handling
# -----------------

def start_a(self,ar gs):
self.links.upda te([value for name, value in args if name == "href"])

parser = MyParser()
parser.feed(fil e("test.html"). read()) # Insert your data source here...
parser.close()

print parser.tagcount
print parser.links

See the documentation for sgmllib for more info on handle_starttag (whose
logic might just as well have been implemented in start_a, but if you want
argument handling for more tags, it's best to keep it at this one central
place) and unknown_startta g.

--- Heiko.