473,378 Members | 1,492 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,378 software developers and data experts.

dealing with commas in fields of csv file

I am trying to write a csv file to a database table, but the problem is
that several fields have commas within them, so the code is reading the
commas as the end of fields. My code is below. How do I get around
this?

Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();

StreamReader sr = new
StreamReader("\\\\devext02\\Xerox_Upload\\archive\ \" + strFileName +
".csv");
string fullFileStr = sr.ReadToEnd();
sr.Close();
sr.Dispose();

string[] lines = fullFileStr.Split('\n');
DataTable dt = new DataTable();

string[] sArr = lines[0].Split(',');

foreach (string s in sArr)
{
dt.Columns.Add(new DataColumn());
}

DataRow row;
string finalLine = "";

foreach (string line in lines)
{
row = dt.NewRow();
finalLine = line.Replace(Convert.ToString('\r'), "");
row.ItemArray = finalLine.Split(',');
dt.Rows.Add(row);
}

SqlConnection objConnection = new
SqlConnection(ConfigurationManager.ConnectionStrin gs["ConnectionString"]
ConnectionString);

System.Data.SqlClient.SqlBulkCopy bc = new
System.Data.SqlClient.SqlBulkCopy(objConnection,
SqlBulkCopyOptions.TableLock, null);

bc.BatchSize = dt.Rows.Count;
objConnection.Open();
bc.DestinationTableName = "UploadDataStaging";
bc.WriteToServer(dt);
objConnection.Close();
bc.Close();
TimeSpan ts = stopWatch.Elapsed;
stopWatch.Stop();

*** Sent via Developersdex http://www.developersdex.com ***
Sep 6 '06 #1
3 11284
Mike,

If it is a CSV file, instead of trying to parse it yourself, why not use
the text provider for OLEDB and use the classes in the System.Data.OleDb
namespace to access the contents of the file as a data set?
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"Mike P" <mi*******@gmail.comwrote in message
news:ug**************@TK2MSFTNGP03.phx.gbl...
>I am trying to write a csv file to a database table, but the problem is
that several fields have commas within them, so the code is reading the
commas as the end of fields. My code is below. How do I get around
this?

Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();

StreamReader sr = new
StreamReader("\\\\devext02\\Xerox_Upload\\archive\ \" + strFileName +
".csv");
string fullFileStr = sr.ReadToEnd();
sr.Close();
sr.Dispose();

string[] lines = fullFileStr.Split('\n');
DataTable dt = new DataTable();

string[] sArr = lines[0].Split(',');

foreach (string s in sArr)
{
dt.Columns.Add(new DataColumn());
}

DataRow row;
string finalLine = "";

foreach (string line in lines)
{
row = dt.NewRow();
finalLine = line.Replace(Convert.ToString('\r'), "");
row.ItemArray = finalLine.Split(',');
dt.Rows.Add(row);
}

SqlConnection objConnection = new
SqlConnection(ConfigurationManager.ConnectionStrin gs["ConnectionString"]
ConnectionString);

System.Data.SqlClient.SqlBulkCopy bc = new
System.Data.SqlClient.SqlBulkCopy(objConnection,
SqlBulkCopyOptions.TableLock, null);

bc.BatchSize = dt.Rows.Count;
objConnection.Open();
bc.DestinationTableName = "UploadDataStaging";
bc.WriteToServer(dt);
objConnection.Close();
bc.Close();
TimeSpan ts = stopWatch.Elapsed;
stopWatch.Stop();

*** Sent via Developersdex http://www.developersdex.com ***

Sep 6 '06 #2
Hi,

A " character is used to enclose a field that contains a comma as part of
hte values.

The easiest solution can be or either use OleDb data provider or go to
opennetcf.org and download theirs, you will have the source code that you
can browse/modify at will

--
--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation


"Mike P" <mi*******@gmail.comwrote in message
news:ug**************@TK2MSFTNGP03.phx.gbl...
>I am trying to write a csv file to a database table, but the problem is
that several fields have commas within them, so the code is reading the
commas as the end of fields. My code is below. How do I get around
this?

Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();

StreamReader sr = new
StreamReader("\\\\devext02\\Xerox_Upload\\archive\ \" + strFileName +
".csv");
string fullFileStr = sr.ReadToEnd();
sr.Close();
sr.Dispose();

string[] lines = fullFileStr.Split('\n');
DataTable dt = new DataTable();

string[] sArr = lines[0].Split(',');

foreach (string s in sArr)
{
dt.Columns.Add(new DataColumn());
}

DataRow row;
string finalLine = "";

foreach (string line in lines)
{
row = dt.NewRow();
finalLine = line.Replace(Convert.ToString('\r'), "");
row.ItemArray = finalLine.Split(',');
dt.Rows.Add(row);
}

SqlConnection objConnection = new
SqlConnection(ConfigurationManager.ConnectionStrin gs["ConnectionString"]
ConnectionString);

System.Data.SqlClient.SqlBulkCopy bc = new
System.Data.SqlClient.SqlBulkCopy(objConnection,
SqlBulkCopyOptions.TableLock, null);

bc.BatchSize = dt.Rows.Count;
objConnection.Open();
bc.DestinationTableName = "UploadDataStaging";
bc.WriteToServer(dt);
objConnection.Close();
bc.Close();
TimeSpan ts = stopWatch.Elapsed;
stopWatch.Stop();

*** Sent via Developersdex http://www.developersdex.com ***

Sep 6 '06 #3
Nicholas,

Do you have an example of using this method?
Thanks,

Mike

*** Sent via Developersdex http://www.developersdex.com ***
Sep 7 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: AES | last post by:
Encountered a URL containing a comma the other day -- the first time I've ever noticed that, so far as I can recall. It worked fine, however, and I gather commas are legal in URLs. Out of...
6
by: dixie | last post by:
I have a text field on a form which has names with a comma between them like this: 'J. Smith, A. Jones, A. Man, J. Johns'. I am trying to find a procedure that will count the number of people in...
4
by: striker | last post by:
I have a comma delimited text file that has multiple instances of multiple commas. Each file will contain approximatley 300 lines. For example: one, two, three,,,,four,five,,,,six one, two,...
14
by: Adrienne Boswell | last post by:
Although this is a client side issue, I am also posting to asp.general in case there is someway to do this only server side (which I would prefer). Here's my form: <form method="post"...
4
by: riccrom123 | last post by:
Hi, I have a csv file where the commas are skipped by the characther \ for example field1,field2,field3\,field3,field4 I would like to extract the fields and obtain field1 field2...
1
by: vdesio | last post by:
I am new to ASP. I am using the free ezscheduler asp program to create a calendar for my website. The idea is for employees to schedule their work shifts online. I have modified some of the...
2
gregerly
by: gregerly | last post by:
I'm having an issue parsing a CSV file in which some fields contain commas. My CSV looks something like this: 123,asdflkj23klj,"Sept. 22, 2008",some field,one more field,"Aug 1,1983" To parse...
9
by: Bruce | last post by:
I'm outputting form content into a csv file. If a comma is used in one of the fields, however, it will interpret to go to next column. Is there a workaround? Thanks. $fp = fopen('my.csv','a');...
4
by: E11esar | last post by:
Hi there. This could be a curious one. Has anybody come across a solution to remove stray commas that appear within strings in a CSV file please? In effect I have many address fields that are...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.