Problem Parsing CSV File

[pindoriya1]

Hi,

m using split function to parse the csv file ..... i m getting problem in one line of file which looks like this :

"BLUEUSBXM","X-MICRO (XBT-DG5R (C) R1) BLUETOOTH USB DONGLE PLUS, CLASS 1",360,0.00,"","",0.00,"",0,0.00,"","T1","",5.58," 4000",8.50,0.00,"02/10/2006",0.00,"16/04/2007"


now the problem is it parses first field properly.
BUT
Not parsing second field.....
avars = split(line,",")
should give:
avars(1) = "X-MICRO (XBT-DG5R (C) R1) BLUETOOTH USB DONGLE PLUS, CLASS 1"

instead of that it gives me:
avars(1) = "X-MICRO (XBT-DG5R (C) R1) BLUETOOTH USB DONGLE PLUS"

how can i solve this problem....

Thanks
Pindoriya1

[DrBunchman]

The Split function is working correctly - it is splitting the file every time it finds a comma and there is a comma after the word PLUS.

To solve this you could make sure that all the fields that are entered into the CSV file have any comma's removed or you could parse the file in a different way.

Dr B

[pindoriya1]

The Split function is working correctly - it is splitting the file every time it finds a comma and there is a comma after the word PLUS.

To solve this you could make sure that all the fields that are entered into the CSV file have any comma's removed or you could parse the file in a different way.

Dr B

Thanks I know that that is why i want solution for that.....
actually the file comes from account software SO EITHER we change in the software OR we solve this problem once and for alll the time we get such feeds form anywhere.....

i prefer second way, as rather than telling accountign people to change name.... i want solution which i can use everywhere i get this situation................

YOU SUGGEST I COULD PARSE FILE IN DIFFERENT WAY....
so can you suggest me one or two of such ways....

thank you
pindoriya1

[DrBunchman]

The only thing I can think of that you might be able to do here is to loop through the string checking each character to see whether it's a comma. If it is a comma then check the next character to see whether it is a space - if so then it's going to be a comma inside one of your fields rather than one of your delimiters so you can ignore it.

It's not foolproof as somebody might enter some text which had a comma without a space after it but it will yield better results than the existing method.

Dr B

[jhardman]

best method: in the app that saves the CSV file, replace all commas with ",". You will also need to replace all instances in the existing csv file where there is a comma that isn't used as a delimiter with "," This will display correctly, can even be copied and pasted correctly, and the people who use it will never know the difference.

Second method: Try DrB's solution, but I would be happier if you added a trim() call to the original function that saves data to make sure no fields begin with a space, but this is still far from fool-proof. And if you are going to alter the app that saves the data, you might as well use the first solution.

Third method: Get a list of all products that have a comma, before you parse the .csv file, replace those texts with a version that doesn't contain commas. I would only recommend this if you have absolutely no access to the app that saves the csv file and you have a definite list of texts that have commas.

Let me know if this helps.

Jared

[danp129]

use regular expressions to parse CSV strings

[jhardman]

best method: in the app that saves the CSV file, replace all commas with ",". You will also need to replace all instances in the existing csv file where there is a comma that isn't used as a delimiter with "," This will display correctly, can even be copied and pasted correctly, and the people who use it will never know the difference.

and I spent all that time typing in the character code for a comma... I didn't notice that Bytes.com just replaced my code with commas. the code should be an ampersand followed by "#44;".

Jared

[jhardman]

use regular expressions to parse CSV strings

yes, I agree. Regex would be a good way to do the replaces.

Jared

[DrBunchman]

Of course, didn't think of that.

Good idea.

[danp129]

As an alternative to regexp's (due to the complexity of writing them) you could also use an ODBC connection with the Microsoft Text Driver to read the CSV file and select a specific record or recurse through all of them.

Provider=MSDASQL; Driver={Microsoft Text Driver (*.txt; *.csv)}; DBQ=C:\path\;

In this case, the c:\path\ folder would be equivilant to a "database" and the filename would be the table. So if you have "c:\path\mydata.csv" then your select statement would be something like "Select * from mydata.csv".

There may be some limitations using this method, such as limited to 255 character length per field or maybe there is not a limit but the driver may only look at the first 10 rows to decide what the max length of a field is w/o editing the registry or creating an .ini file for the driver. You may also not be able to join "tables" (files) like a normal database, I haven't used csv files for quite some time so I can't say.

You can find some examples searching for: "Microsoft Text Driver" asp

[jhardman]

As an alternative to regexp's (due to the complexity of writing them) you could also use an ODBC connection with the Microsoft Text Driver to read the CSV file and select a specific record or recurse through all of them.

That's a good solution, but I don't think it would get around the problem of commas (which are reserved characters in csv files) being inserted by the user. Maybe one of us should test?

Jared

[danp129]

Commas in part of the text field are completely valid for CSV so long as a text qualifier (ie the double quotes) is used.

[danp129]

So if you have "c:\path\mydata.csv" then your select statement would be something like "Select * from mydata.csv".

Just noticed a typo, should be .... something like "Select * from mydata" w/o the .csv