473,854 Members | 1,756 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Removing duplicate entries in a csv file using a python script

4 New Member
I m a beginner to python. Could you tell me how should i proceed to remove duplicate rows in a csv file
Nov 19 '07
10 41023
sathish119
4 New Member
I am in Python 2.3. Define a comparison function to pass to the list sort method:
Expand|Select|Wrap|Line Numbers
  1. def comp581(a, b):
  2.     x = cmp(a[5], b[5])
  3.     if not x:
  4.         y = cmp(a[8], b[8])
  5.         if not y:
  6.             return cmp(a[1], b[1])
  7.         return y
  8.     return x
  9.  
  10. yourList.sort(comp581)
In Python 2.4:
Expand|Select|Wrap|Line Numbers
  1. yourList.sort(key=lambda i: (i[5], i[8], i[1]))
thanks a lot. im able to sort the list now. (the version im using is 2.5.1 - i used the 'lambda' functionality)
Nov 26 '07 #11

Sign in to post your reply or Sign up for a free account.

Similar topics

3
3194
by: SeeBelow | last post by:
I have not been able to find a simple way to have a python script execute an MS-DOS program repeatedly. execv(), for example, won't run it more than once. I have been using many lines of a .bat file, like so: .. .. .. del xyz myexec 1 2
4
4392
by: sri2097 | last post by:
Hi all, I'm storing number of dictionary values into a file using the 'cPickle' module and then am retrieving it. The following is the code for it - # Code for storing the values in the file import cPickle book = {raw_input("Name: "): } file_object = file(database, 'w+') cPickle.dump(book, file_object)
10
14281
by: learnerofpython | last post by:
I have a cygwin application which i have invoked using python script like: os.system("path of the exe file") How can i enter text in the cygwin( which has appeared )using the python script itself???
0
1240
by: psbasha | last post by:
Hi, Is there any module available to write and read a XML file using Python? Links for the material will help me. Thanks in advance PSB
12
20809
by: joestevens232 | last post by:
Hello Im having problems figuring out how to remove the duplicate entries in an array...Write a program that accepts a sequence of integers (some of which may repeat) as input into an array. Write a function that processes the array so that any duplicate values are eliminated. Write an output function that prints out the values of the array. You can assume that there are no more than 20 integers in the input. But there may be less. Zero...
4
6234
by: AshishMishra16 | last post by:
HI friends, I am using the Flex to upload files to server. I m getting all the details about the file, but I m not able to upload it to Server. Here is the code i m using for both flex & for Struts: import java.io.File; import java.io.PrintWriter; import java.util.ArrayList; import java.util.Enumeration; import java.util.Iterator;
2
9309
by: Viewer T. | last post by:
I am trying to write a script that deletes certain files based on certain criteria. What I am trying to do is to automate the process of deleting certain malware files that disguise themselves as system files and hidden files. When I use os.remove() after importing the os module is raises a Windows Error: Access denied probably because the files have disguised themselves as system files. Is there anway to get adequate privileges under...
1
2303
by: deardp | last post by:
hi : i need to convert an image (ex: jpeg or gif) into binary form and generating the image back to the original one using python script(it was some thing like decoding and encoding). pls pass me if u have any code. regards Durga
1
2231
by: Arulmanoj | last post by:
Hi, I have to write the content from one csv file to another csv file using VB Script. The scource content file contains 5 columns and the destination file contains six column as below. Source File: Coulmn1 Coulmn3 Coulmn4 Coulmn5 Coulmn6 abc abc abc abc abc Destination File: Coulmn1 Coulmn3 Coulmn3 Coulmn4 Coulmn5 Coulmn6
2
2682
by: Tatiana Louisa | last post by:
Hi, I am a bit new in python. Would be nice to get some help. I would like to remove the first line of 100 files in a folder. How can i proceed to write a python script. I can do it for a single file, but dont know how can I modify my script to run it for a folder containing 100 files with .txt extension. Tatiana
0
9752
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
1
10766
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10373
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7921
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7083
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5753
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5946
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4565
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4167
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.