473,406 Members | 2,954 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,406 software developers and data experts.

Screen Scraping for Modern Applications?

Hello,

I'm very new to python and still familiarizing myself with the
language, sorry if the post seems moronic or simple.

For a side project I'm working on I need to be able to scrape a modern
computer desktop. Is there any basic material already available to do
this? I'd rather not need to write my own to interact with hardware.

Thanks!

Jun 12 '06 #1
7 3532
I just learned about a python library called BeautifulSoup in a thread
responding to my Southwest Airlines script that I wrote and posted on
the Python cookbook web site. I wrote my script using the provided
HTMLParser class. BeautifulSoup could have saved me some time perhaps.
Take a look. Feel free to browse my code for an example of how to use
HTMLParser for a moderately involved project.

BeautifulSoup link:
http://www.crummy.com/software/BeautifulSoup/

My Southwest Boarding pass script link:
http://aspn.activestate.com/ASPN/Coo.../Recipe/496790

Enjoy.
- Ken
lj*****@gmail.com wrote:
Hello,

I'm very new to python and still familiarizing myself with the
language, sorry if the post seems moronic or simple.

For a side project I'm working on I need to be able to scrape a modern
computer desktop. Is there any basic material already available to do
this? I'd rather not need to write my own to interact with hardware.

Thanks!


Jun 12 '06 #2
In article <11*********************@h76g2000cwa.googlegroups. com>,
<lj*****@gmail.com> wrote:
Jun 12 '06 #3

Cameron Laird wrote:
In article <11*********************@h76g2000cwa.googlegroups. com>,
<lj*****@gmail.com> wrote:
.
.
.
For a side project I'm working on I need to be able to scrape a modern
computer desktop. Is there any basic material already available to do
this? I'd rather not need to write my own to interact with hardware.

.
.
.
Oh, no.

Very hard. Very, very hard.

At least as you've expressed it here, this is an exceptionally
difficult task. What does "scrape" mean to you? Do you want
to reproduce the display remotely, or extract text, or some other
possibility? Does "modern computer" mean Windows to you?


Scrape means simply scraping pixel colors from locations on the screen.
I'll worry about assembling it into meaningful information.

Previously, I used Java, and it has a very handy built in class called
Robot that can, amongst other things, report the color of on screen
pixels. Thusly, I assumed the task could be accomplished reasonably
easily.

Jun 12 '06 #4
<lj*****@gmail.com> wrote in message
news:11**********************@c74g2000cwc.googlegr oups.com...


Scrape means simply scraping pixel colors from locations on the screen.
I'll worry about assembling it into meaningful information.


I've used pywinauto to interact with a Flash animation panel, running within
an Internet Explorer browser. By "interact" I mean moving and clicking the
mouse, and capturing images, which can then be processed to access
individual pixels.

-- Paul
Jun 12 '06 #5
lj*****@gmail.com wrote:
Scrape means simply scraping pixel colors from locations on the screen.
I'll worry about assembling it into meaningful information.


import ImageGrab
im = ImageGrab.grab()
v = im.getpixel((x, y))

requires:

http://www.pythonware.com/products/pil/

</F>

Jun 12 '06 #6
> > Scrape means simply scraping pixel colors from locations on the screen.
I'll worry about assembling it into meaningful information.


import ImageGrab
im = ImageGrab.grab()
v = im.getpixel((x, y))

requires:

http://www.pythonware.com/products/pil/

</F>


##
# (New in 1.1.3) The <b>ImageGrab</b> module can be used to copy
# the contents of the screen to a PIL image memory.
# <p>
# The current version works on Windows only.</p>
#

How about doing the same on linux?
Jun 12 '06 #7
In article <11**********************@c74g2000cwc.googlegroups .com>,
<lj*****@gmail.com> wrote:
Jun 12 '06 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Jonathan Epstein | last post by:
I would like to perform a more classical type of "screen scraping" than what most people now associate with this term. I only want to find all the text on the current screen, and obtain associated...
4
by: Roland Hall | last post by:
Am I correct in assuming screen scraping is just the response text sent to the browser? If so, would that mean that this could not be screen scraped? function moi() { var tag = '<a href='; var...
0
by: Robert Martinez | last post by:
I've seen a lot about screen scraping with .NET, mostly in VB.net. I have been able to convert most of it over, but it is still just very basic stuff. Can someone help direct me toward some good...
3
by: Jim Giblin | last post by:
I need to scrape specific information from another website, specifically the prices of precious metals from several different vendors. While I will credit the vendors as the data source, I do not...
4
by: rachel | last post by:
Hello, I am currently contracted out by a real estate agent. He has a page that he has created himself that has a list of homes.. their images and data in html format. He wants me to take...
0
by: bruce | last post by:
Hi.. I'm not that familiar with Pythin, but I wasn wondering if there are any XPath/Python Gurus that I might be able to talk to regarding screen scraping applications... Thanks -Bruce...
6
by: Christopher Glenn | last post by:
I have very basic html skills. My friend who has a wide screen monitor and is using IE7 sent me a jpg screen shot of my home page. I have attached this jpg, but I recall a while back that...
4
by: different.engine | last post by:
Folks: I am screen scraping a large volume of data from Yahoo Finance each evening, and parsing with Beautiful Soup. I was wondering if anyone could give me some pointers on how to make it...
3
by: WFDGW2 | last post by:
I want to write or obtain C++ code that will scrape text from a dialog box within a poker client, and then record that text somewhere else. What do I do? Thanks.
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.