473,326 Members | 2,255 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Convert Google Finance data to Array

270 100+
Hello,
I'm trying to extract data from Google finance csv file using javasript.


http://finance.google.com/finance/hi...=25&output=csv


however the csv file has 6 columns (Date,Open,High,Low,Close,Volume),
and i want only two columns (Date,Close) in array from it.

Following is the example of csv

Expand|Select|Wrap|Line Numbers
  1. Date,Open,High,Low,Close,Volume
  2. 5-May-09,399.98,405.00,397.25,402.99,2401096
  3. 4-May-09,398.17,402.40,394.79,401.98,3204843
  4. 1-May-09,395.03,397.59,391.55,393.69,2428611
  5. 30-Apr-09,395.76,403.75,394.80,395.97,4361069 
I wrote following code according to http://bytes.com/topic/javascript/an...ascript-arrays
but i'm getting errors in the csvArray() method, 'undefined' is null or not an object.

Expand|Select|Wrap|Line Numbers
  1. <html>
  2. <head>
  3. <title>- Another CSV parser -</title>
  4. <script type="text/javascript" src="jquery.js"></script>
  5. <script type="text/javascript">
  6.     function loadData(){
  7.         $.get("http://finance.google.com/finance/historical",
  8.           { 
  9.                 q:'GOOG',
  10.                 startdate:'May 1 2009',
  11.                 enddate:'May 5 2009',
  12.                 start:'225',
  13.                 num:'25',
  14.                 output:'csv'
  15.           },
  16.           function(data)
  17.           {
  18.              // Writing output into div
  19.              $("#csvDiv").html(data);
  20.              $("#csvDiv").hide();
  21.              ConvertToArray(data);
  22.           }
  23.         );
  24.     }
  25.     function csvArray(csv){
  26.         var i= 0;var A= csv.split(/\s*\n\s*/);
  27.         while(A[i++]) A[i]= A[i].split(/ *, */);
  28.         return A;
  29.     }
  30.     function ConvertToArray(data){
  31.         var myStr=$("#csvDiv").html();
  32.         var myArr=csvArray(myStr);
  33.         //ignore first 6 lines
  34.         var i=6;
  35.         for (i; i<myArr.length; i++) {
  36.             document.writeln(myArr[i]+'<br />');
  37.         }
  38.     }
  39. </script>
  40. </head>
  41. <body>
  42. <div id="csvDiv">
  43.  
  44. </div>
  45. <script type="text/javascript">
  46.     loadData();
  47. </script>
  48. </body>
  49. </html>
regards,
Nitin Sawant
May 20 '09 #1
13 11653
NitinSawant
270 100+
I'm also unable to replace newline character with comma

Expand|Select|Wrap|Line Numbers
  1. myArr=myArr.replace("\n",",");
  2.  
May 20 '09 #2
RamananKalirajan
608 512MB
Hi I am litlle bit new to jquery. But i got some doubts/not clear with your code..


Expand|Select|Wrap|Line Numbers
  1. function(data)
  2.          {
  3.             // Writing output into div
  4.             $("#csvDiv").html(data);
  5.             $("#csvDiv").hide();
  6.             ConvertToArray(data);
  7.        }
In the above mentioned function, from were u are getting the value for the variable data? when i alerted that value its seems o be a html file. But it returns empty value. My suggestion is that u get an undefined error in the function csvArray() as it gets null value.

Regards
Ramanan Kalirajan
May 20 '09 #3
NitinSawant
270 100+
Hello Raman,
Thanks for your reply,

$.get() function returns the variable data,
There is no problem retrieving csv string,
You can see the retrieved data by commenting $("#csvDiv").hide();

Manipulating csv string data is quite difficult for me.
I want to convert it to javascript array,

regards,
Nitin Sawant
May 20 '09 #4
acoder
16,027 Expert Mod 8TB
Check that myStr is defined.

When using the newline character in a string, you need to escape it:
Expand|Select|Wrap|Line Numbers
  1. myArr=myArr.replace("\\n",",");
May 20 '09 #5
NitinSawant
270 100+
Hello,
Thanks for your reply,

Still it doesn't work, i think google people have inserted some secret character instead of newline in the csv file.

Expand|Select|Wrap|Line Numbers
  1. <html>
  2. <head>
  3. <title>- Another CSV parser -</title>
  4. <script type="text/javascript" src="flotr/jquery.js"></script>
  5. <script type="text/javascript">
  6.     function loadData(){
  7.         $.get("http://finance.google.com/finance/historical",
  8.           { 
  9.                 q:'GOOG',
  10.                 startdate:'May 1 2009',
  11.                 enddate:'May 5 2009',
  12.                 start:'225',
  13.                 num:'25',
  14.                 output:'csv'
  15.           },
  16.           function(data)
  17.           {
  18.              // Writing output into div
  19.              $("#csvDiv").html(data);
  20.              $("#csvDiv").hide();
  21.              ConvertToArray(data);
  22.           }
  23.         );
  24.     }
  25.  
  26.     function ConvertToArray(data){
  27.         var myArr=$("#csvDiv").html();
  28.         myArr=myArr.replace('\\n',',');//this line doesn't work
  29.         myArr=myArr.split(',');
  30.         document.writeln("array length is: "+myArr.length+"<br/>");
  31.  
  32.         //show all elements of array
  33.         var i=0;
  34.         for (i; i<myArr.length; i++) {
  35.             document.writeln(myArr[i]+'<br />');
  36.         }
  37.     }
  38. </script>
  39. </head>
  40. <body>
  41. <div id="csvDiv">
  42.  
  43. </div>
  44. <script type="text/javascript">
  45.     loadData();
  46. </script>
  47. <!--
  48.     http://finance.google.com/finance/historical?q=GOOG&startdate=May+1%2C+2009&enddate=May+5%2C+2009&start=225&num=25&output=csv 
  49. -->
  50. </body>
  51. </html>
It doesn't separate the stock volume and date

Expand|Select|Wrap|Line Numbers
  1. Date
  2. Open
  3. High
  4. Low
  5. Close
  6. Volume 5-May-09
  7. 399.98
  8. 405.00
  9. 397.25
  10. 402.99
  11. 2401096 4-May-09
  12. 398.17
  13. 402.40
  14. 394.79
  15. 401.98
  16. 3204843 1-May-09
  17. 395.03
  18. 397.59
  19. 391.55
  20. 393.69
  21. 2428611 30-Apr-09
  22. 395.76
  23. 403.75
  24. 394.80
  25. 395.97
  26. 4361069
May 20 '09 #6
acoder
16,027 Expert Mod 8TB
It's probably because the script is in another domain, not yours. Use server-side code to get it from your domain or use JSON/script tags.
May 20 '09 #7
NitinSawant
270 100+
@acoder
I'm able to access data but unable to manipulate it,

There must be some problem with csv file or google has locked it so that no one can retrieve it
May 20 '09 #8
acoder
16,027 Expert Mod 8TB
In your original code, on line 31, what's the value of myStr?
May 20 '09 #9
NitinSawant
270 100+
value of myStr is the content of csv file returned from Google Finance site:

Expand|Select|Wrap|Line Numbers
  1. Date,Open,High,Low,Close,Volume 5-May-09,399.98,405.00,397.25,402.99,2401096 4-May-09,398.17,402.40,394.79,401.98,3204843 1-May-09,395.03,397.59,391.55,393.69,2428611 30-Apr-09,395.76,403.75,394.80,395.97,4361069
May 20 '09 #10
acoder
16,027 Expert Mod 8TB
Instead of using the HTML from the csv Div, use the 'data' directly:
Expand|Select|Wrap|Line Numbers
  1. var myArr=csvArray(data);
I'm not sure how you're getting the data if it's from a different domain. Are you testing locally?
May 21 '09 #11
NitinSawant
270 100+
Thanks a million @acoder

when I was testing it locally, it was working but
when i put it on server it was not working.

there is something called as cross site restriction with javascript, thats why not working

now i resolved the error,

its becoming too complicated in javascript so i did it through screen scraping in php. now its working as desired http://nitin.tech4you.org/retrieve.php

Expand|Select|Wrap|Line Numbers
  1. <?php
  2. /*
  3. * example.php
  4. * class_http.php example usage
  5. * Author: Troy Wolf (troy@troywolf.com)
  6. * Comments: Please be a good neighbor when screen-scraping. Don't write code
  7.             that will needlessly make hits to third-party websites. Use
  8.             class_http's caching feature whenever possible. It is designed to
  9.             make you a good neighbor!
  10. */
  11.  
  12. /*
  13. Include the http class. Modify path according to where you put the class
  14. file.
  15. */
  16. require_once(dirname(__FILE__).'/class_http.php');
  17.  
  18. /* First, instantiate a new http object. */
  19. $h = new http();
  20.  
  21. /*
  22. Where do you want to store your cache files?
  23. Default is current dir. You can set it here, or hard-code in the class.
  24. You must end this value with a "/".
  25. */
  26. //$h->dir = "/public_html/"; 
  27.  
  28.  
  29. $currentDate= date("d-M-y",time());
  30.  
  31. /*
  32. Screen-scrape the Google home page without caching.
  33. */
  34. if (!$h->fetch('http://finance.google.com/finance/historical?q=GOOG&startdate=May+1%2C+2009&enddate='.$currentDate.'&start=225&num=25&output=csv',60)) {
  35.   /*
  36.   The class has a 'log' property that contains a log of events. This log is
  37.   useful for testing and debugging.
  38.   */
  39.   echo "<h2>There is a problem with the http request!</h2>";
  40.   echo $h->log;
  41.   exit();
  42. }
  43.  
  44. /* Echo out the body content fetched from the url. */
  45. $content= $h->body;
  46.  
  47. //replace all newlines
  48. $content=str_replace("\n",",",$content);
  49. echo $content;
  50.  
  51. /* If you just want to know the HTTP status code: */
  52. //echo "Status: ".$h->status;
  53.  
  54. /* If you are interested in seeing all the response headers: */
  55. //echo "<pre>".$h->header."</pre>";
  56.  
  57.  
  58. ?>
  59.  
Thanks again @Raman, @Acoder

warm regards,
Nitin Sawant
May 21 '09 #12
acoder
16,027 Expert Mod 8TB
That explains it. Thanks for posting your solution.
May 21 '09 #13
There is a powerful Google Finance scraper which can scrape companies, exchanges and price information large scaled.
It's a free PHP project at http://scrape-google-finance.compunect.com

Should help if you need to go for larger amounts of data.
Apr 2 '14 #14

Sign in to post your reply or Sign up for a free account.

Similar topics

5
by: Karthik | last post by:
Hello, How can I convert a BSTR data type to std::string??? Thanks a much! Karthik
1
by: vsk | last post by:
Hai, I need to know how to convert an unsigned char array into hexstring. can anyone help me in this regard?. Thanks.
4
by: kvicky | last post by:
I have multiline textbox and I am trying to do a search functionality based on the input entered in this Textbox. I am able to convert into a string array if the search components are separated by...
2
by: Bilwin | last post by:
Hi friends, How can i convert StringTokenizer to string array? friends please reply me..
5
by: Maxim | last post by:
Hi all, I'm calling a COM Interface method that returnes SafeArray wrapped into variant. Is it possible to convert it to managed array? Because working with SAFEARRAY directly is a bit...
2
by: probashi | last post by:
I would like to add Chart Like Google Finance into my ASPX pages. How involved this will be !!! Thanks.
0
by: mwenz | last post by:
I am trying to update an Access table using OLEDB in VB.Net 2005. I can add rows but I cannot update them. Code to instantiate the Access database and table... Dim conn As New...
4
by: Man4ish | last post by:
HI , I am trying to convert string into char array of characters.but facing problem. #include <iostream> #include <string> using namespace std; int main() { string t="1,2,3,4,5,6";
12
by: Peter | last post by:
Trying to convert string to byte array. the following code returns byte array of {107, 62, 194, 139, 64} how can I convert this string to a byte array of {107, 62, 139, 65} ...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.