By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,077 Members | 2,121 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,077 IT Pros & Developers. It's quick & easy.

web crawlers in java

P: 1
can ne body give me an insight on java cralers and what it takes to code web crawlers???
Dec 13 '09 #1
Share this Question
Share on Google+
1 Reply


100+
P: 539
know what data you are going to extract
make an algorithm of it
beware of Object manipulations( specially for a 24/7 System, it consumes huge memory and increasing if not handled properly )

on threads, be careful to handle passed-by-reference instances

Select what database system you are going to use for storage on your extracted data

good luck
Dec 13 '09 #2

Post your reply

Sign in to post your reply or Sign up for a free account.