473,395 Members | 1,496 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,395 software developers and data experts.

Possible to "scape" a website?

bbu
Hello,
I need to collect data from a website, everyday. I get an email everyday with 6 to 10 links that is can pick form to get to them. I need to automate as much as possible.

My ideas so far: use an Outlook2000 macro
to automatically open the email when it comes in.
to send the links to a threaded web service
the web service thread
opens the link
opens a notepad of "the source"
scans the source for keywords and
puts the strings into an array
Will this work?
Is this the best way?
Thanks
BB
Nov 21 '05 #1
1 1024
Nak
Hi there,

http://www.members.lycos.co.uk/nickp.../soft-wtr2.htm

The above link has source of mine for an applicaton that scans HTML for
links, it would be easily modified to do as you suggest. As for opening
Outlook, that shouldn't be too difficult, you could even create a kind of
"Send to" plug-in that will enable you to open the file. One thing you
might want to think about is the "Referer" of the web page, this should be
set before attempting to download the content as you *might* encounter
problems. Getting the referer shouldn't be that difficult, the homepage of
the people who send you the email might be enough?

Nick.

"bbu" <bb*@socal.rr.com> wrote in message
news:HZ*******************@twister.socal.rr.com...
Hello,
I need to collect data from a website, everyday. I get an email everyday
with 6 to 10 links that is can pick form to get to them. I need to automate
as much as possible.

My ideas so far: use an Outlook2000 macro
to automatically open the email when it
comes in.
to send the links to a threaded web service
the web service thread

opens the link

opens a notepad of "the source"

scans the source for keywords and

puts the strings into an array
Will this work?
Is this the best way?
Thanks
BB
Nov 21 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

83
by: liketofindoutwhy | last post by:
I am learning more and more Prototype and Script.aculo.us and got the Bungee book... and wonder if I should get some books on jQuery (jQuery in Action, and Learning jQuery) and start learning about...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.