473,603 Members | 2,652 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Best way to parse CSV data?

Hello

I googled for samples, but I'd like to make sure there's no simpler
way to parse tab-separated data read from a web page, before I can
save them into a database:

<?php
$input = file_get_conten ts ("http://server/data.php");

//item1<TAB>item2 <CRLF>
$contents = explode("\r\n", $input);

foreach ($contents as $row) {
$line = explode("\t",$r ow);
foreach($line as $col) {
//Here, will save each col into database
print $col;
}
}
?>

Thanks for any tip.
Jun 2 '08 #1
9 3090
Gilles Ganault wrote:
Hello

I googled for samples, but I'd like to make sure there's no simpler
way to parse tab-separated data read from a web page, before I can
save them into a database:

<?php
$input = file_get_conten ts ("http://server/data.php");

//item1<TAB>item2 <CRLF>
$contents = explode("\r\n", $input);
You can use
$contents = file("http://server/data.php");

to get the data into an array
foreach ($contents as $row) {
$line = explode("\t",$r ow);
foreach($line as $col) {
//Here, will save each col into database
print $col;
}
}
?>

Thanks for any tip.

Works as well as anything else.
--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===

Jun 2 '08 #2
On Wed, 14 May 2008 17:29:17 -0400, Jerry Stuckle
<js*******@attg lobal.netwrote:
> foreach ($contents as $row) {
$line = explode("\t",$r ow);
foreach($line as $col) {
//Here, will save each col into database
print $col;
}
}
?>

Works as well as anything else.
I ended up simplifying it a bit:

$input = file("http://server/data.php");
foreach ($input as $row) {
//Final CRLF is INDISPENSABLE!
preg_match("|(. +?)\t(.+?)\r\n| ",$row,$matches );

//Any way to display the output of prepare() just to check?
$sql = "INSERT INTO mytable (id, name) VALUES (?,?)";
$insert = $dbh->prepare($sql );
$insert->execute(array( $matches[1],$matches[2]));
}

I did have a problem with regexing data from file(): Without "\r\n",
the regex doesn't work as planned.

Also, I went through the PDO documentation, but didn't find a method
to display the output of prepare(), which would be convenient for
debugging purposes.

Thanks.
Jun 2 '08 #3
Gilles Ganault wrote:
On Wed, 14 May 2008 17:29:17 -0400, Jerry Stuckle
<js*******@attg lobal.netwrote:
>> foreach ($contents as $row) {
$line = explode("\t",$r ow);
foreach($line as $col) {
//Here, will save each col into database
print $col;
}
}
?>
Works as well as anything else.

I ended up simplifying it a bit:

$input = file("http://server/data.php");
foreach ($input as $row) {
//Final CRLF is INDISPENSABLE!
preg_match("|(. +?)\t(.+?)\r\n| ",$row,$matches );

//Any way to display the output of prepare() just to check?
$sql = "INSERT INTO mytable (id, name) VALUES (?,?)";
$insert = $dbh->prepare($sql );
$insert->execute(array( $matches[1],$matches[2]));
}

I did have a problem with regexing data from file(): Without "\r\n",
the regex doesn't work as planned.

Also, I went through the PDO documentation, but didn't find a method
to display the output of prepare(), which would be convenient for
debugging purposes.

Thanks.
Yep, if you're going to use a regex, you need the \r\n. But personally
I prefer explode().

And sorry, there's no way to display the output of prepare(). It's out
in the database.

--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===

Jun 2 '08 #4
Gilles Ganault wrote:
Hello

I googled for samples, but I'd like to make sure there's no simpler
way to parse tab-separated data read from a web page, before I can
save them into a database:

<?php
$input = file_get_conten ts ("http://server/data.php");

//item1<TAB>item2 <CRLF>
$contents = explode("\r\n", $input);

foreach ($contents as $row) {
$line = explode("\t",$r ow);
foreach($line as $col) {
//Here, will save each col into database
print $col;
}
}
?>

Thanks for any tip.

Since your goal is to save it to a db (mysql? oracle?) why not use the
tools provided with the db engine and save yourself a ton of headaches
trying to reinvent the wheel...

mysql - search for mysql load file
oracle - search for sql*loader
Access - add it as an external table.
Jun 2 '08 #5
On Wed, 14 May 2008 22:46:58 +0200, Gilles Ganault <no****@nospam. com
wrote:
Hello

I googled for samples, but I'd like to make sure there's no simpler
way to parse tab-separated data read from a web page, before I can
save them into a database:

<?php
$input = file_get_conten ts ("http://server/data.php");

//item1<TAB>item2 <CRLF>
$contents = explode("\r\n", $input);

foreach ($contents as $row) {
$line = explode("\t",$r ow);
foreach($line as $col) {
//Here, will save each col into database
print $col;
}
}
?>

Thanks for any tip.
<?php
$input = fopen("http://server/data.php",'r');
$result = array();
while($roo = fgetcsv($result ,0,"\t")) $result[] = $row;
var_dump($resul t)
?>

Or, for a database like MySQL, look into LOAD DATA INFILE syntax.
--
Rik Wasmus
[SPAM] Now temporarily looking for some smaller PHP/MySQL projects/work to
fund a self developed bigger project, mail me at rik at rwasmus.nl. [/SPAM]
Jun 2 '08 #6
On Wed, 14 May 2008 20:14:09 -0400, Jerry Stuckle
<js*******@attg lobal.netwrote:
>And sorry, there's no way to display the output of prepare(). It's out
in the database.
Too bad. Thanks.
Jun 2 '08 #7
On Wed, 14 May 2008 22:35:11 -0500, Michael Austin
<ma*****@firstd basource.comwro te:
>Since your goal is to save it to a db (mysql? oracle?) why not use the
tools provided with the db engine and save yourself a ton of headaches
trying to reinvent the wheel...
Forgot to mention the source data is off on a remote server to which I
have no access, hence my need to read data from its web page output,
parse it, before putting data into a second database. Thanks.
Jun 2 '08 #8
On Thu, 15 May 2008 07:46:10 +0200, "Rik Wasmus"
<lu************ @hotmail.comwro te:
(snip)

Thanks!
Jun 2 '08 #9
On May 15, 1:21 pm, Gilles Ganault <nos...@nospam. comwrote:
On Wed, 14 May 2008 22:35:11 -0500, Michael Austin

<maus...@firstd basource.comwro te:
Since your goal is to save it to a db (mysql? oracle?) why not use the
tools provided with the db engine and save yourself a ton of headaches
trying to reinvent the wheel...

Forgot to mention the source data is off on a remote server to which I
have no access, hence my need to read data from its web page output,
parse it, before putting data into a second database. Thanks.
There is no documented standard for 'CSV' (even though it has a
registered mime type) - so each solution needs to be tailored to the
source.

I'm no regex guru but your approach does not seem to allow for escaped
or embedded newlines within field data.

C.
Jun 2 '08 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

19
2450
by: Peter A. Schott | last post by:
I've got a file that seems to come across more like a dictionary from what I can tell. Something like the following format: ###,1,val_1,2,val_2,3,val_3,5,val_5,10,val_10 ###,1,val_1,2,val_2,3,val_3,5,val_5,11,val_11,25,val_25,967,val_967 In other words, different layouts (defined mostly by what is in val_1, val_2, val_3). The ,#, fields indicate what "field" from our mainframe the corresponding value
12
7705
by: jacob nikom | last post by:
Hi, I would like to store XML files in MySQL. What is the best solution: 1. Convert it to string and store it as CLOB/text 2. Serialize it and store as byte array 3. Flatten it out and create one column per element, each column is VARCHAR Does MySQL has anything special for XML data? Is there any software which helps to store XML data in MySQL
2
4406
by: Jim Williams | last post by:
**** Post for FREE via your newsreader at post.usenet.com **** I'm working on a 3d engine system. I need to save/load data in the systems objects to/from XML files. When I say objects, I mean C++ objects. (These generally correspond to scene objects, but that consideration isn't important as far as loading/saving goes.) I think I have a reasonable grasp of XML, it's method of data representation, and parsing XML files.
3
6746
by: Andy Fish | last post by:
Hi, I have a webpage with some javascript that constructs some data structures, some of which contain nested structures or arrays of other structures. It's the same order of complexity as a web.xml file, say. I need to post the whole lot to the server and I'm not sure which is the best way to do it. It has to be browser-independent. The only ways I can think of are:
19
3208
by: Johnny Google | last post by:
Here is an example of the type of data from a file I will have: Apple,4322,3435,4653,6543,4652 Banana,6934,5423,6753,6531 Carrot,3454,4534,3434,1111,9120,5453 Cheese,4411,5522,6622,6641 The first position is the info (the product) I want to retreive for the corresponding code. Assuming that the codes are unique for each product and all code data is on one line.
0
4212
by: Anonieko Ramos | last post by:
ASP.NET Forms Authentication Best Practices Dr. Dobb's Journal February 2004 Protecting user information is critical By Douglas Reilly Douglas is the author of Designing Microsoft ASP.NET Applications and owner of Access Microsystems. Doug can be reached at doug@accessmicrosystems.com. --------------------------------------------------------------------------------
1
1124
by: phl | last post by:
Hi, I have some data in an xml file which I need to use in my web form. The data in the xml file needs to be shown in maybe a repeater, grid or data control. I will need to parse the data make sense of it and maybe place it in a data structure. Once I have a meaningful representation of this data(the format is not exactly how i want it), I would like to cache it. This data does not change often. So my questions are: - Would it be...
29
2882
by: gs | last post by:
let say I have to deal with various date format and I am give format string from one of the following dd/mm/yyyy mm/dd/yyyy dd/mmm/yyyy mmm/dd/yyyy dd/mm/yy mm/dd/yy dd/mmm/yy mmm/dd/yy
19
2126
by: Steve | last post by:
I have to create 2 strings and then parse one string out to save the data into the database. My first string looks like this: John|Smith|Jsmith@yahoo.com,Greg|Henry|gh@aol.com,Kelly|Smith|ks@gmail.com, I then need to parse out that string to seperate them by the commas so I see this John|Smith|Jsmith@yahoo.com
6
1165
dennison
by: dennison | last post by:
I am trying to write code for a class that has a couple of attributes that will can only be known upon creation. Because I think accessor functions (e.g. getSomething()) are outdate, I declared my attributes as "public readonly" in order to protect the attributes from getting modified yet still accessible by the parent class. I am not convinced that this is the best practice for this. Implementing properties seem like a bit of a waste of time to...
0
7997
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8406
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8065
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8274
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
6737
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5441
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
3906
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
3954
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2433
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.