473,325 Members | 2,342 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,325 software developers and data experts.

Screen Scraping

17
I have been trying to find a sample of this on the internet and i cannot.

Scenario: I have an Oracle program, it has a logon screen with username and password, 2nd screen i need to enter a "1" to get to the screen i am looking for.

From there, i just want to scrape all or some of the information from that screen into my Access. I won't get access to the tables directly otherwise that is the route i would have taken.
Can anyone help or point me to a tutorial on this type of stuff?

Thanks, Much appreciated.

Ashe
Jan 28 '08 #1
3 8308
NeoPa
32,556 Expert Mod 16PB
It appears to me that screen scraping is a facility not native either to Access or VBA. As such I don't feel this is the right place for this question.
Jan 28 '08 #2
Here is some VBA code that will grab an open IE Browser window on your desktop and return the document so that it can be scraped. Use the Tagnames and Innertext fields. You will need to handle frames on the page specially.
This should get you going


Expand|Select|Wrap|Line Numbers
  1. Sub TstdcIEURL()
  2. Dim myObj As Object, ObjDoc As Object
  3. Dim myTagName As String, myInnerText As String
  4.     'Set myObj = dcIEURL("replace with desired IE domain name", "optional parameter, replace with Location name")
  5.     Set ObjDoc = myObj.Document
  6.     For i = 1 To ObjDoc.all.length
  7.         myTagName = ObjDoc.all.Item(i).tagName
  8.         myInnerText = ObjDoc.all.Item(i).innerText
  9.         Debug.Print myTagName, myInnerText
  10.         Stop
  11.     Next i
  12.     Stop
  13. End Sub
  14. Public Function dcIEURL(zDomainName, Optional zLocName As String) As Object
  15. 'zDomainName is the IE window with data you want to screen scrape, LocName is optional and a further qualification of the window data
  16. On Error GoTo ErrHandler
  17. Dim objShellWins As SHDocVw.ShellWindows
  18. Dim ObjIE As SHDocVw.InternetExplorer
  19. Dim ObjDoc As Object
  20. Dim myInnerText As String
  21. Dim myNode As String, AccountID As String
  22. Dim i As Integer, myLeft As String, myRight As String
  23. Dim strOut As String, mySearch As String
  24. Dim intFree As Integer, myURL As String
  25. Dim myURLDomainInspecting As String, myURLDomainSeeking As String
  26. Dim myLocNameInspecting As String, myLocNameSeeking As String
  27. Dim myIE As Variant
  28.  
  29.  
  30.     ' Instantiate
  31.     Set objShellWins = New SHDocVw.ShellWindows
  32.     ' There might be multiple IE windows open
  33.     If objShellWins.Count > 0 Then
  34.         myURLDomainSeeking = zDomainName
  35.         If IsMissing(zLocName) Then
  36.             myLocNameSeeking = ""
  37.         Else
  38.             myLocNameSeeking = zLocName
  39.         End If
  40.         For Each ObjIE In objShellWins
  41.             With ObjIE
  42.                 ' Try to locate the browser with a specific address in it's AddressBar.
  43.                 myURLDomainInspecting = GetDomain(.LocationURL)
  44.                 myLocNameInspecting = ObjIE.LocationName
  45.                 Select Case myURLDomainInspecting
  46.                     Case Is = myURLDomainSeeking    'Domains are equal here
  47.                         ' URL points to one passed to this function
  48.                         If myLocNameSeeking <> "" Then   'Bypass if no loc name to find
  49.                             Select Case myLocNameInspecting
  50.                                 Case Is = ""            'LocName not present, use URL
  51.                                     Set myIE = ObjIE
  52.                                     Exit For
  53.                                 Case Is = myLocNameSeeking
  54.                                     Set myIE = ObjIE
  55.                                     Exit For
  56.                             End Select
  57.                         Else
  58.                             Set myIE = ObjIE
  59.                             Exit For
  60.                         End If
  61.                     Case Else
  62.                         If Left(myURLDomainSeeking, 6) = "rodweb" Then
  63.                             If Left(myURLDomainInspecting, 6) = "rodweb" Then
  64.                                 Set myIE = ObjIE
  65.                                 Exit For
  66.                             End If
  67.                         End If
  68.                 End Select
  69.             End With
  70.         Next
  71.     End If
  72. If IsObject(myIE) = False Then
  73.     Set myIE = CreateObject("InternetExplorer.Application")
  74.     myIE.Width = 1200
  75.     myIE.Height = WinHeight
  76.     myIE.Left = 1
  77.     myIE.Top = 1
  78.     myIE.Visible = True
  79. End If
  80. Set dcIEURL = myIE
  81. ExitHere:
  82.     On Error Resume Next
  83.     Close #intFree
  84.     Set ObjDoc = Nothing
  85.     Set ObjIE = Nothing
  86.     Set objShellWins = Nothing
  87.     Exit Function
  88. ErrHandler:
  89.     With Err
  90.         MsgBox "Error: " & .Number & vbCrLf & .Description, _
  91.             vbCritical Or vbOKOnly, .Source
  92.     End With
  93.     Resume ExitHere
  94.  
  95. End Function
  96. Sub TstdcIEURL()
  97. Dim myObj As Object, ObjDoc As Object
  98. Dim myTagName As String, myInnerText As String
  99.     'Set myObj = dcIEURL("replace with desired IE domain name", "optional parameter, replace with Location name")
  100.     Set myObj = dcIEURL("ecf.nceb.uscourts.gov")
  101.     Set ObjDoc = myObj.Document
  102.     For i = 1 To ObjDoc.all.length
  103.         myTagName = ObjDoc.all.Item(i).tagName
  104.         myInnerText = ObjDoc.all.Item(i).innerText
  105.         Debug.Print myTagName, myInnerText
  106.         Stop
  107.     Next i
  108.     Stop
  109. End Sub
  110. Public Function dcIEURL(zDomainName, Optional zLocName As String) As Object
  111. 'zDomainName is the IE window with data you want to screen scrape, LocName is optional and a further qualification of the window data
  112. On Error GoTo ErrHandler
  113. Dim objShellWins As SHDocVw.ShellWindows
  114. Dim ObjIE As SHDocVw.InternetExplorer
  115. Dim ObjDoc As Object
  116. Dim myInnerText As String
  117. Dim myNode As String, AccountID As String
  118. Dim i As Integer, myLeft As String, myRight As String
  119. Dim strOut As String, mySearch As String
  120. Dim intFree As Integer, myURL As String
  121. Dim myURLDomainInspecting As String, myURLDomainSeeking As String
  122. Dim myLocNameInspecting As String, myLocNameSeeking As String
  123. Dim myIE As Variant
  124.  
  125.  
  126.     ' Instantiate
  127.     Set objShellWins = New SHDocVw.ShellWindows
  128.     ' There might be multiple IE windows open
  129.     If objShellWins.Count > 0 Then
  130.         myURLDomainSeeking = zDomainName
  131.         If IsMissing(zLocName) Then
  132.             myLocNameSeeking = ""
  133.         Else
  134.             myLocNameSeeking = zLocName
  135.         End If
  136.         For Each ObjIE In objShellWins
  137.             With ObjIE
  138.                 ' Try to locate the browser with a specific address in it's AddressBar.
  139.                 myURLDomainInspecting = GetDomain(.LocationURL)
  140.                 myLocNameInspecting = ObjIE.LocationName
  141.                 Select Case myURLDomainInspecting
  142.                     Case Is = myURLDomainSeeking    'Domains are equal here
  143.                         ' URL points to one passed to this function
  144.                         If myLocNameSeeking <> "" Then   'Bypass if no loc name to find
  145.                             Select Case myLocNameInspecting
  146.                                 Case Is = ""            'LocName not present, use URL
  147.                                     Set myIE = ObjIE
  148.                                     Exit For
  149.                                 Case Is = myLocNameSeeking
  150.                                     Set myIE = ObjIE
  151.                                     Exit For
  152.                             End Select
  153.                         Else
  154.                             Set myIE = ObjIE
  155.                             Exit For
  156.                         End If
  157.                     Case Else
  158.                         If Left(myURLDomainSeeking, 6) = "rodweb" Then
  159.                             If Left(myURLDomainInspecting, 6) = "rodweb" Then
  160.                                 Set myIE = ObjIE
  161.                                 Exit For
  162.                             End If
  163.                         End If
  164.                 End Select
  165.             End With
  166.         Next
  167.     End If
  168. If IsObject(myIE) = False Then
  169.     Set myIE = CreateObject("InternetExplorer.Application")
  170.     myIE.Width = 1200
  171.     myIE.Height = WinHeight
  172.     myIE.Left = 1
  173.     myIE.Top = 1
  174.     myIE.Visible = True
  175. End If
  176. Set dcIEURL = myIE
  177. ExitHere:
  178.     On Error Resume Next
  179.     Close #intFree
  180.     Set ObjDoc = Nothing
  181.     Set ObjIE = Nothing
  182.     Set objShellWins = Nothing
  183.     Exit Function
  184. ErrHandler:
  185.     With Err
  186.         MsgBox "Error: " & .Number & vbCrLf & .Description, _
  187.             vbCritical Or vbOKOnly, .Source
  188.     End With
  189.     Resume ExitHere
  190.  
  191. End Function
  192.  
Nov 3 '08 #3
RedSon
5,000 Expert 4TB
You can just as easily get this information from copying and pasting the open browser window.

Screen scraping is generally frowned upon since you are using another companies data for your own purposes. As long as you are using it for academic purposes then it shouldn't be a problem.
Nov 5 '08 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Jonathan Epstein | last post by:
I would like to perform a more classical type of "screen scraping" than what most people now associate with this term. I only want to find all the text on the current screen, and obtain associated...
4
by: Roland Hall | last post by:
Am I correct in assuming screen scraping is just the response text sent to the browser? If so, would that mean that this could not be screen scraped? function moi() { var tag = '<a href='; var...
2
by: Me | last post by:
I am dealing with a poorly written windows application that does not contain an API. I would like to use C# to run a predetermied set of steps in the application and scrape the resulting data...
0
by: Robert Martinez | last post by:
I've seen a lot about screen scraping with .NET, mostly in VB.net. I have been able to convert most of it over, but it is still just very basic stuff. Can someone help direct me toward some good...
3
by: _eee_ | last post by:
Does anyone know of a simple code module that can do screen scraping, including simulating user-entered pushbuttons, etc. I can get the first screen on a website with HttpWebRequest, but I need...
3
by: Jim Giblin | last post by:
I need to scrape specific information from another website, specifically the prices of precious metals from several different vendors. While I will credit the vendors as the data source, I do not...
1
by: niv | last post by:
Hello, I would like to screen scrape certain parts of a webpage...how can I do this in asp.net For instance.... a stockticker thats embeded on a webpage.. I dont want the entire page.. I...
4
by: rachel | last post by:
Hello, I am currently contracted out by a real estate agent. He has a page that he has created himself that has a list of homes.. their images and data in html format. He wants me to take...
4
by: different.engine | last post by:
Folks: I am screen scraping a large volume of data from Yahoo Finance each evening, and parsing with Beautiful Soup. I was wondering if anyone could give me some pointers on how to make it...
3
by: WFDGW2 | last post by:
I want to write or obtain C++ code that will scrape text from a dialog box within a poker client, and then record that text somewhere else. What do I do? Thanks.
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.