473,842 Members | 1,928 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to read csv file that has ',' within data?

13 New Member
I am new to Perl programming and I faced a problem now.

I need to process a csv file and as for now, if the string is like '123' or 'asd', I could do it without any problem because it is commas inbetween data. But if I want to process like 'asd,asd,asd', I do not know how to read the whole string with the commas because as of now, I use split ',' to get data from different field of the csv. please help!!!
Jul 30 '08 #1
8 4677
410 Recognized Expert Contributor
I believe you have tried writing the code. But instead of putting it in plain words, it would have been better if you posted the piece of script that you tried and the sample data.
Suppose, your data is as below:
Expand|Select|Wrap|Line Numbers
  1. 'asd,asd,asd','123','123'
  2. 'asd','dgf','pqr'
One way of looking at the requirement is, you would essentially need to take out the string within single quotes (' ') rather than that delimited by comma(,). You can acheive this through regex.
The code:
Expand|Select|Wrap|Line Numbers
  1. use strict;
  2. open (IN,'input_file') or die "$!"; 
  3. my @data;
  4.  while (<IN>) {
  5. push @data, $1 while(/('.+?')/g); #get the string surrounded with single-quotes
  6. }
  7. print "$_\n" foreach(@data);
will produce the output:
Expand|Select|Wrap|Line Numbers
  1. 'asd,asd,asd'
  2. '123'
  3. '123'
  4. 'asd'
  5. 'asd'
  6. 'asd'
Jul 30 '08 #2
4,059 Recognized Expert Specialist
In a properly formatted CSV file, some of the data will be quoted and some will not be. The Text::CSV_XS module can correctly parse a CSV file with embedded commas as long as that particular field is quoted.
Jul 30 '08 #3
13 New Member
Thanks for the prompt reply.

My data was input through excel then save under csv. In the excel, i gt few fields... Lets say is "abc,def,ghi",1 23,asd

I tried this

Expand|Select|Wrap|Line Numbers
  1. my $csv = "test.csv";
  2. my @data = read_file($csv);
  4. foreach($a=1; $a<@data.""; $a++)
  5. {
  6.     print @data[$a];
  7. }
  9. sub read_file{
  10.     open(F, $_[0]) || die("Can't open file $_[0]!");
  11.     my @data = <F>;
  12.     close F;
  13.     return @data;
  14. }
the output is "abc,def,ghi",1 23,asd

What i 1 is that if i to assign "abc,def,gh i" to $name, how do i get the data within the double quote and also how to assign the 123 and asd which doesnt have the double quote as output to be $contact_no and $address respectively?

I tried using the delimited comma, but within the double quote, there are 2 commas, as a result, i will have 5 individual data...
Jul 30 '08 #4
4,059 Recognized Expert Specialist
I still recommend you use Text::CSV_XS but here is a way to do it without the module:

Expand|Select|Wrap|Line Numbers
  1. while (<DATA>) {
  2.     my @new  = ();
  3.     push(@new, $+) while $_ =~ m{
  4.         # the first part groups the phrase inside the quotes.
  5.         # see explanation of this pattern in MRE
  6.         "([^\"\\]*(?:\\.[^\"\\]*)*)",?
  7.            |  ([^,]+),?
  8.            | ,
  9.        }gx;
  10.        push(@new, undef) if substr($_, -1,1) eq ',';
  11.        print "$_\n" for @new;
  12. }  
  14. __DATA__
  15. "abc,def,ghi",123,asd
The solution is from the Perl CookBook which got the regexp from "Mastering Regular Expressions". Don't ask me to explain the regexp, it confuses the hell out of me.
Jul 30 '08 #5
13 New Member
Thanks alot kevin.

But, if i want to use the module, do you mind if you actually show me an example? thanks alot in advance... :p
Jul 30 '08 #6
4,059 Recognized Expert Specialist
The modules documentation has examples. You can find the documentation on CPAN or google for Text::CSV_XS.
Jul 30 '08 #7
13 New Member
Just for information, may i know are you from Singapore?
Jul 30 '08 #8
4,059 Recognized Expert Specialist
Just for information, may i know are you from Singapore?
If you are asking me, no, I am not from Singapore. And I have never been there.
Jul 30 '08 #9

Sign in to post your reply or Sign up for a free account.

Similar topics

by: Karen Grube | last post by:
Hi! I hate to bother you all with this, but I don't know how best to approach a particular task. Here's the deal: Once a month I personally (that is, in my own personal inbox on my company's email server) receive an email from one of our vendors to which is attached a text file, a comma-delimited data file containing the details of our monthly
by: Magix | last post by:
Hi, I have these string data: str_data1, str_data2, str_data3, which capture some value after a routine process A. Then I would like to write (append) these 3 string values into a text file each time after routine process A, the text file is named "mytext.dat" in following format with "#####" as separator. The maximum entries of them is 5. When reaching the fifth entry, it will delete the very first entry.
by: JG | last post by:
Does anyone know a standard (or supported on Linux, Mac, Win32) way to clear a read stream buffer (standard ANSI C file stream)? I would even settle for a platform specific way of doing it. And no, I know I can use direct low level I/O or non-buffered to do reads, but for my app, I need the buffering. I can implement myself, but this is not optimal. Example, I open a read only file using fopen(). I periodically know
by: QQ | last post by:
I am trying to read data from Excel files I use the following codes, however, it didn't work Can anyone tell me what the format excel files have, what I should do to read it from C? Thanks a lot! #include <stdio.h>
by: Yogi_Bear_79 | last post by:
pardon my ignorance as I am a self-taught hobbyist programmer. I am curious after reading up on SharpZipLib. Can I embed a zipped txt file in my program? Then either read from within the zip file or unzip and read it? I currently have an embedded text file that contains a list that is read into an array. I'm always looking to save space. And I could reduce my file size 75% if it was zipped! I have looked at the SharpZipLib web site,...
by: Karen Grube | last post by:
Hi! I hate to bother you all with this, but I don't know how best to approach a particular task. Here's the deal: Once a month I receive in my own inbox on my company's Outlook Exchange email server an email from one of our vendors to which is attached a text file (a comma-delimited data file containing the details of our monthly
by: james.burrows | last post by:
I'm looking for a quick way to read lines of data from a text file into memory. Each line will contain e.g. a book entry: - id,name,price,category,author,description I could have over 20000 of these lines, and I'd like a function that reads in the list of books grabbing only the id and name fields. Then I'd like another function that when passed an id will grab all the details from that entry.
by: Pim75 | last post by:
Hello, I have to read a XML file in ASP and save the values in a database. I can get this work, but I cannot read some nested nodes of the xml file. This is a part of the XML file: <Interface> <Product> <CategoryFeatureGroup ID="622" No="1"> <FeatureGroup ID="0">
by: drharris | last post by:
First, please forgive my newness to XML. I've used it to serialize/ deserialize objects, exporting and importing datasets, and other such things that pretty much automate reading in the file. I've done extensive googling, and most examples people give are so simplistic it makes me want to cry. Most are one level deep and utilize XMLDocument or other in-memory processes, or do things like: while(reader.Read()) { //do things I'm not going...
by: =?Utf-8?B?c2lwcHl1Y29ubg==?= | last post by:
Hi Is there any way to Read an INIFile from a string or Stream instead of a physical file ??? I want to read the INIFile into a string then store in a db but when I read the string from the db or save string to the db I don't want to have to copy the string to a file to use the WritePrivateProfileString and GetPrivateProfileString. Is there a way around this instead of writing my own class to operate on a string or stream ???
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.