473,548 Members | 2,633 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Python spider than can handle Ajax

1 New Member
Hi,

I'm looking for a python spider module that can handle ajax components. The ajax generates links that the spider should continue crawling.

If anyone has heard of something like that, I'd appreciate the help.

Cheers
Sharon
Oct 29 '08 #1
1 2516
kudos
127 Recognized Expert New Member
The easiest approach is to simulate the browsing with a webbrowser and extracting information from that. There is some javascript <-> python stuff too, for instance something called : python-spidermonkey

-kudos


Hi,

I'm looking for a python spider module that can handle ajax components. The ajax generates links that the spider should continue crawling.

If anyone has heard of something like that, I'd appreciate the help.

Cheers
Sharon
Oct 30 '08 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

65
6668
by: Anthony_Barker | last post by:
I have been reading a book about the evolution of the Basic programming language. The author states that Basic - particularly Microsoft's version is full of compromises which crept in along the language's 30+ year evolution. What to you think python largest compromises are? The three that come to my mind are significant whitespace,...
5
2013
by: Magnus Lyck? | last post by:
Something really strange is happening to me (sometimes). I'm using Python 2.3.2 on NT 4.0 as well as win32all-157, adodbapi and db_row. During a recursive call to a method, it seems Python messes up its variable bindings once in a while. Suddenly, one of several local variables gets rebound to the object it was bound to one step up in the...
42
3645
by: Bicho Verde | last post by:
I have now free time and money to do what I want :-) I have some basic skills in programming (C, Pascal, Macromedia Actionscript) but don't know exactly what to do in the world of programming. And also I don't know exactly why would I learn Python rather than C#, C++ or Perl. Basicaly I don't know where to start, if there is much to do or if...
0
2070
by: dtsearch | last post by:
New release expands-through a .NET Spider API, to Linux, and to OpenOffice-dtSearch's ability to index over a terabyte of text in a single index, with indexed search time typically less than a second BETHESDA, MD (January 10, 2006) dtSearch Corp., a leading supplier of enterprise and developer text retrieval software, announces Version 7.2...
37
2439
by: John Salerno | last post by:
I contacted my domain host about how Python is implemented on their server, and got this response: ------------------- Hello John, Please be informed that the implementation of python in our server is through mod_python integration with the apache. These are the steps needed for you to be able to run .py script directly
28
2605
by: H J van Rooyen | last post by:
Hi, I want to write a small system that is transaction based. I want to split the GUI front end data entry away from the file handling and record keeping. Now it seems almost trivially easy using the sockets module to communicate between machines on the same LAN, so that I want to do the record keeping on one machine.
2
2128
by: Shadab Sayani | last post by:
Hi , We have a project where I need to read files store them in database in the backend.We have done this in python.Now we decided to use Ajax technique for user interface.For that we found that GWT is one of the best toolkits.Now I got a doubt can I interface GWT with python. Thanks , Shadab.
3
1654
by: Victor | last post by:
hi I just get a problem. in my site all my menu is using Ajax(like the componentart menu). but the website is heavily rely on the web search engine to promote(like Google bot). I think most of the web search engine use a web spider to go through your site. It seems the Ajax menu will screw up this. Is anyway to solve this issue(keep the Ajax...
4
4075
by: sonich | last post by:
I need simple web crawler, I found Ruya, but it's seems not currently maintained. Does anybody know good web crawler on python or with python interface?
0
7707
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7951
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
0
7803
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6036
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5362
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
3495
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
1926
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1051
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
751
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.