473,473 Members | 2,226 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

webcrawler (bot) in javascript?

I have a few programs that search an entire site for a keyword, and I
run
them from my machine: "Light Web Searcher" and "Teleport Pro" are
just
a few. My objective is to create a bare-bones software application
identical to the functions available in the aforementioned pieces of
software.

What language would I be able to accomplish that with?
Thanks,
Danny
Feb 20 '08 #1
3 2445
wrote on 20 feb 2008 in comp.lang.javascript:
I have a few programs that search an entire site for a keyword, and I
run
them from my machine: "Light Web Searcher" and "Teleport Pro" are
just
a few. My objective is to create a bare-bones software application
identical to the functions available in the aforementioned pieces of
software.

What language would I be able to accomplish that with?
The Q is mainly off topic,
you can only ask if it can be done using Javascript.

Answer: Yes.

--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)
Feb 20 '08 #2
On Feb 20, 2:18*pm, "Evertjan." <exjxw.hannivo...@interxnl.netwrote:
*wrote on 20 feb 2008 in comp.lang.javascript:
I have a few programs that search an entire site for a keyword, and I
run
them from my machine: "Light Web Searcher" and "Teleport Pro" are
just
a few. My objective is to create a bare-bones software application
identical to the functions available in the aforementioned pieces of
software.
What language would I be able to accomplish that with?

The Q is mainly off topic,
you can only ask if it can be done using Javascript.

Answer: Yes.

--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)
Great. Can you get me started on writing a web crawler (bot) in
Javascript. What tools do I need?
Feb 28 '08 #3
On Feb 28, 10:57 am, bdy120...@gmail.com wrote:
On Feb 20, 2:18 pm, "Evertjan." <exjxw.hannivo...@interxnl.netwrote:
wrote on 20 feb 2008 in comp.lang.javascript:
I have a few programs that search an entire site for a keyword, and I
run
them from my machine: "Light Web Searcher" and "Teleport Pro" are
just
a few. My objective is to create a bare-bones software application
identical to the functions available in the aforementioned pieces of
software.
What language would I be able to accomplish that with?
The Q is mainly off topic,
you can only ask if it can be done using Javascript.
Answer: Yes.
--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

Great.
The answer may be yes, but JavaScript would still be a poor choice. A
web crawler requires a lot of bandwidth and lots of text parsing. A
server running Java or C++ would do far better than your web browser
running JavaScript.
Can you get me started on writing a web crawler (bot) in
Javascript. What tools do I need?
Your brain and a text editor. :)
Feb 28 '08 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

14
by: Ludwig77 | last post by:
I read that there are some tags that can be entered in a web page's meta tags in order to prevent web bot searching and indexing of the web page for search engines. What is the tagging that I...
4
by: Mark | last post by:
Hi, I am writing some code to log vistors to my website. I notice in some statistics pacakages that they are able to differentiate between normal visitors and search bots. Does anyone know how...
2
by: David Starr | last post by:
Well their bot is BANNED on my domain together with the yahoo,msn and other idiots who have bots and i haven't seen the google bot for a few weeks now,thank goodness.i hope it is something to do...
4
by: jerry.parasol | last post by:
Hello All, Does anyone know which is the best IRC bot written in PHP? I'm kind of tired of spending hours on eggdrop.conf and learning tcl... I've seen http://www.redelmstudios.com/hotnsour/...
4
by: gloomer | last post by:
Hey guys, I've really got into python, but I'm still noobish at it. I have an idea how my Python bot would work. But I can't find the right python code for it. print "Hi,i'm a bot" if input...
3
by: HornyAZNBoy | last post by:
Hello. I have been learning VB6 for a month and I need help Making a Bot of a game called MapleStory. A Bot is a program that plays for you when you are away or asleep. Here is an ezample of a...
7
Thekid
by: Thekid | last post by:
How can I go about writing a simple IRC bot? I read one thread in this forum about an IRC bot but I couldn't get it to work. I've looked at a few like eggdrop, supybot and phenny but I want to try a...
0
by: pr501869 | last post by:
ok i do not know if anyone can help me but here goes. i have aquired a bot ..IRCza bot and whyen i try to refine it's responses it tells me something about its botmaster and in PM i say i am you...
0
by: bakermi | last post by:
Hello, I have coded an IRC bot in Python. Each inbound packet is parsed, and once the bot decides whether it is a command directed at the bot or not, it will either discard the packet or make a...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
1
muto222
php
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.