By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,341 Members | 1,709 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,341 IT Pros & Developers. It's quick & easy.

Convert Google Finance data to Array

100+
P: 270
Hello,
I'm trying to extract data from Google finance csv file using javasript.


http://finance.google.com/finance/hi...=25&output=csv


however the csv file has 6 columns (Date,Open,High,Low,Close,Volume),
and i want only two columns (Date,Close) in array from it.

Following is the example of csv

Expand|Select|Wrap|Line Numbers
  1. Date,Open,High,Low,Close,Volume
  2. 5-May-09,399.98,405.00,397.25,402.99,2401096
  3. 4-May-09,398.17,402.40,394.79,401.98,3204843
  4. 1-May-09,395.03,397.59,391.55,393.69,2428611
  5. 30-Apr-09,395.76,403.75,394.80,395.97,4361069 
I wrote following code according to http://bytes.com/topic/javascript/an...ascript-arrays
but i'm getting errors in the csvArray() method, 'undefined' is null or not an object.

Expand|Select|Wrap|Line Numbers
  1. <html>
  2. <head>
  3. <title>- Another CSV parser -</title>
  4. <script type="text/javascript" src="jquery.js"></script>
  5. <script type="text/javascript">
  6.     function loadData(){
  7.         $.get("http://finance.google.com/finance/historical",
  8.           { 
  9.                 q:'GOOG',
  10.                 startdate:'May 1 2009',
  11.                 enddate:'May 5 2009',
  12.                 start:'225',
  13.                 num:'25',
  14.                 output:'csv'
  15.           },
  16.           function(data)
  17.           {
  18.              // Writing output into div
  19.              $("#csvDiv").html(data);
  20.              $("#csvDiv").hide();
  21.              ConvertToArray(data);
  22.           }
  23.         );
  24.     }
  25.     function csvArray(csv){
  26.         var i= 0;var A= csv.split(/\s*\n\s*/);
  27.         while(A[i++]) A[i]= A[i].split(/ *, */);
  28.         return A;
  29.     }
  30.     function ConvertToArray(data){
  31.         var myStr=$("#csvDiv").html();
  32.         var myArr=csvArray(myStr);
  33.         //ignore first 6 lines
  34.         var i=6;
  35.         for (i; i<myArr.length; i++) {
  36.             document.writeln(myArr[i]+'<br />');
  37.         }
  38.     }
  39. </script>
  40. </head>
  41. <body>
  42. <div id="csvDiv">
  43.  
  44. </div>
  45. <script type="text/javascript">
  46.     loadData();
  47. </script>
  48. </body>
  49. </html>
regards,
Nitin Sawant
May 20 '09 #1
Share this Question
Share on Google+
13 Replies


100+
P: 270
I'm also unable to replace newline character with comma

Expand|Select|Wrap|Line Numbers
  1. myArr=myArr.replace("\n",",");
  2.  
May 20 '09 #2

RamananKalirajan
100+
P: 607
Hi I am litlle bit new to jquery. But i got some doubts/not clear with your code..


Expand|Select|Wrap|Line Numbers
  1. function(data)
  2.          {
  3.             // Writing output into div
  4.             $("#csvDiv").html(data);
  5.             $("#csvDiv").hide();
  6.             ConvertToArray(data);
  7.        }
In the above mentioned function, from were u are getting the value for the variable data? when i alerted that value its seems o be a html file. But it returns empty value. My suggestion is that u get an undefined error in the function csvArray() as it gets null value.

Regards
Ramanan Kalirajan
May 20 '09 #3

100+
P: 270
Hello Raman,
Thanks for your reply,

$.get() function returns the variable data,
There is no problem retrieving csv string,
You can see the retrieved data by commenting $("#csvDiv").hide();

Manipulating csv string data is quite difficult for me.
I want to convert it to javascript array,

regards,
Nitin Sawant
May 20 '09 #4

acoder
Expert Mod 15k+
P: 16,027
Check that myStr is defined.

When using the newline character in a string, you need to escape it:
Expand|Select|Wrap|Line Numbers
  1. myArr=myArr.replace("\\n",",");
May 20 '09 #5

100+
P: 270
Hello,
Thanks for your reply,

Still it doesn't work, i think google people have inserted some secret character instead of newline in the csv file.

Expand|Select|Wrap|Line Numbers
  1. <html>
  2. <head>
  3. <title>- Another CSV parser -</title>
  4. <script type="text/javascript" src="flotr/jquery.js"></script>
  5. <script type="text/javascript">
  6.     function loadData(){
  7.         $.get("http://finance.google.com/finance/historical",
  8.           { 
  9.                 q:'GOOG',
  10.                 startdate:'May 1 2009',
  11.                 enddate:'May 5 2009',
  12.                 start:'225',
  13.                 num:'25',
  14.                 output:'csv'
  15.           },
  16.           function(data)
  17.           {
  18.              // Writing output into div
  19.              $("#csvDiv").html(data);
  20.              $("#csvDiv").hide();
  21.              ConvertToArray(data);
  22.           }
  23.         );
  24.     }
  25.  
  26.     function ConvertToArray(data){
  27.         var myArr=$("#csvDiv").html();
  28.         myArr=myArr.replace('\\n',',');//this line doesn't work
  29.         myArr=myArr.split(',');
  30.         document.writeln("array length is: "+myArr.length+"<br/>");
  31.  
  32.         //show all elements of array
  33.         var i=0;
  34.         for (i; i<myArr.length; i++) {
  35.             document.writeln(myArr[i]+'<br />');
  36.         }
  37.     }
  38. </script>
  39. </head>
  40. <body>
  41. <div id="csvDiv">
  42.  
  43. </div>
  44. <script type="text/javascript">
  45.     loadData();
  46. </script>
  47. <!--
  48.     http://finance.google.com/finance/historical?q=GOOG&startdate=May+1%2C+2009&enddate=May+5%2C+2009&start=225&num=25&output=csv 
  49. -->
  50. </body>
  51. </html>
It doesn't separate the stock volume and date

Expand|Select|Wrap|Line Numbers
  1. Date
  2. Open
  3. High
  4. Low
  5. Close
  6. Volume 5-May-09
  7. 399.98
  8. 405.00
  9. 397.25
  10. 402.99
  11. 2401096 4-May-09
  12. 398.17
  13. 402.40
  14. 394.79
  15. 401.98
  16. 3204843 1-May-09
  17. 395.03
  18. 397.59
  19. 391.55
  20. 393.69
  21. 2428611 30-Apr-09
  22. 395.76
  23. 403.75
  24. 394.80
  25. 395.97
  26. 4361069
May 20 '09 #6

acoder
Expert Mod 15k+
P: 16,027
It's probably because the script is in another domain, not yours. Use server-side code to get it from your domain or use JSON/script tags.
May 20 '09 #7

100+
P: 270
@acoder
I'm able to access data but unable to manipulate it,

There must be some problem with csv file or google has locked it so that no one can retrieve it
May 20 '09 #8

acoder
Expert Mod 15k+
P: 16,027
In your original code, on line 31, what's the value of myStr?
May 20 '09 #9

100+
P: 270
value of myStr is the content of csv file returned from Google Finance site:

Expand|Select|Wrap|Line Numbers
  1. Date,Open,High,Low,Close,Volume 5-May-09,399.98,405.00,397.25,402.99,2401096 4-May-09,398.17,402.40,394.79,401.98,3204843 1-May-09,395.03,397.59,391.55,393.69,2428611 30-Apr-09,395.76,403.75,394.80,395.97,4361069
May 20 '09 #10

acoder
Expert Mod 15k+
P: 16,027
Instead of using the HTML from the csv Div, use the 'data' directly:
Expand|Select|Wrap|Line Numbers
  1. var myArr=csvArray(data);
I'm not sure how you're getting the data if it's from a different domain. Are you testing locally?
May 21 '09 #11

100+
P: 270
Thanks a million @acoder

when I was testing it locally, it was working but
when i put it on server it was not working.

there is something called as cross site restriction with javascript, thats why not working

now i resolved the error,

its becoming too complicated in javascript so i did it through screen scraping in php. now its working as desired http://nitin.tech4you.org/retrieve.php

Expand|Select|Wrap|Line Numbers
  1. <?php
  2. /*
  3. * example.php
  4. * class_http.php example usage
  5. * Author: Troy Wolf (troy@troywolf.com)
  6. * Comments: Please be a good neighbor when screen-scraping. Don't write code
  7.             that will needlessly make hits to third-party websites. Use
  8.             class_http's caching feature whenever possible. It is designed to
  9.             make you a good neighbor!
  10. */
  11.  
  12. /*
  13. Include the http class. Modify path according to where you put the class
  14. file.
  15. */
  16. require_once(dirname(__FILE__).'/class_http.php');
  17.  
  18. /* First, instantiate a new http object. */
  19. $h = new http();
  20.  
  21. /*
  22. Where do you want to store your cache files?
  23. Default is current dir. You can set it here, or hard-code in the class.
  24. You must end this value with a "/".
  25. */
  26. //$h->dir = "/public_html/"; 
  27.  
  28.  
  29. $currentDate= date("d-M-y",time());
  30.  
  31. /*
  32. Screen-scrape the Google home page without caching.
  33. */
  34. if (!$h->fetch('http://finance.google.com/finance/historical?q=GOOG&startdate=May+1%2C+2009&enddate='.$currentDate.'&start=225&num=25&output=csv',60)) {
  35.   /*
  36.   The class has a 'log' property that contains a log of events. This log is
  37.   useful for testing and debugging.
  38.   */
  39.   echo "<h2>There is a problem with the http request!</h2>";
  40.   echo $h->log;
  41.   exit();
  42. }
  43.  
  44. /* Echo out the body content fetched from the url. */
  45. $content= $h->body;
  46.  
  47. //replace all newlines
  48. $content=str_replace("\n",",",$content);
  49. echo $content;
  50.  
  51. /* If you just want to know the HTTP status code: */
  52. //echo "Status: ".$h->status;
  53.  
  54. /* If you are interested in seeing all the response headers: */
  55. //echo "<pre>".$h->header."</pre>";
  56.  
  57.  
  58. ?>
  59.  
Thanks again @Raman, @Acoder

warm regards,
Nitin Sawant
May 21 '09 #12

acoder
Expert Mod 15k+
P: 16,027
That explains it. Thanks for posting your solution.
May 21 '09 #13

P: 1
There is a powerful Google Finance scraper which can scrape companies, exchanges and price information large scaled.
It's a free PHP project at http://scrape-google-finance.compunect.com

Should help if you need to go for larger amounts of data.
Apr 2 '14 #14

Post your reply

Sign in to post your reply or Sign up for a free account.