By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,624 Members | 1,737 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,624 IT Pros & Developers. It's quick & easy.

Removing duplicate rows in a textfile then add to datatable.

P: 82
Hi, I am looking for advice on the best way to achieve my current goal.
I am reading a textfile and extracting information from each line. For example string1 and string2. I am then writing these string to a datatable.
However, sometimes there will be repeats in string1 and string2.
If there are repeated lines I would like to be able to delete the repeating line and add a counter to find the total number of repeating lines.
so if a string repeates 3 times I would like there to be a column with a 3 displayed next to the line in my datatable.
If anyone has any suggestion on the best approach or some example code I would very much appreciate this.

Best,
Starlight849
Aug 16 '10 #1

✓ answered by Joseph Martell

The dictionary collection maintains a unique set of keys. In other words, the dictionary collection will not allow you to add the same key more than once.

In your example, your logic would look something like this:

Expand|Select|Wrap|Line Numbers
  1. Dim myLines As New System.Collections.Generic.Dictionary(Of String, Integer)()
  2. Dim key As String = "Mango Banana"
  3. If (myLines.ContainsKey(key)) Then
  4.     myLines(key) += 1
  5. Else
  6.     myLines(key) = 1
  7. End If
  8.  

Share this Question
Share on Google+
4 Replies


Joseph Martell
Expert 100+
P: 196
Take a look at the dictionary collection. This data structure may be what you are looking for. You have to set a key and a value. In your case, the key would be a string and the value would be an integer to count the number of occurrences:
Expand|Select|Wrap|Line Numbers
  1. Dim myLines As New System.Collections.Generic.Dictionary(Of String, Integer)()
  2.  
Aug 17 '10 #2

P: 82
I will look into this but I'm comparing two strings. If string 2 matches string 1 more than once then I delete the repeating strings in both. I then add a counter to see how many times the matches appear.

Example:
string1 string2
apple orange
apple banana
apple pear
mango banana
mango banana

I only want to delete the second occurence of mango banana. I then need a total count of mango banana (2).

I've added the strings to a datatable and am going to loop through the datatable to do this search. If anyone has any example about comparing and declaring a previous and current row in a datatable I would appreciate it.
Aug 17 '10 #3

Joseph Martell
Expert 100+
P: 196
The dictionary collection maintains a unique set of keys. In other words, the dictionary collection will not allow you to add the same key more than once.

In your example, your logic would look something like this:

Expand|Select|Wrap|Line Numbers
  1. Dim myLines As New System.Collections.Generic.Dictionary(Of String, Integer)()
  2. Dim key As String = "Mango Banana"
  3. If (myLines.ContainsKey(key)) Then
  4.     myLines(key) += 1
  5. Else
  6.     myLines(key) = 1
  7. End If
  8.  
Aug 17 '10 #4

P: 82
Ok, I figured it out. Thanks for your help, Joseph.
Aug 19 '10 #5

Post your reply

Sign in to post your reply or Sign up for a free account.