470,634 Members | 2,092 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,634 developers. It's quick & easy.

parsing a site/page that uses/calls javascript functions...


I've got a couple of test apps that I use to parse/test different html
webpages. However, I'm now looking at how to parse a given site/page that
uses javascript calls to dynamically create/display the resulting HTML.

I can see the HTML is the Browser page if I manually select the btn that
invokes the javascript function, but I have no idea how to create an app
that can effectively parse the page.

My test apps use python, along with mechanize/browser/urllib. I've seen
sites/docs that discuss selenium, spidermonkey, etc... If possible, I'm
trying to find a complete example (that walks through how to setup the
environment, to how to finally extract the DOM elements of a given
javascript page), or I'm looking to find someone I can work with, to create
a complete example that can then be posted to the 'net.

I'd really rather have a headless browser solution, as my overall goal is to
run a parsing/crawling over a number of pages that utilize javascript..

Pointers, thoughts, comments, etc will be greatly appreciated.


Sep 28 '08 #1
0 879

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Ralph Krausse | last post: by
9 posts views Thread by RiGGa | last post: by
1 post views Thread by Fastly | last post: by
3 posts views Thread by Dennis M. Marks | last post: by
2 posts views Thread by laredotornado | last post: by
9 posts views Thread by paul | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.