473,320 Members | 2,122 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Removing similar lines and repeats

gi|282558029|gene000415 lga:LGAS_0495 1e-87 100....
gi|282558029|gene000415 lga:LGAS_0495 1e-87 189...
gi|282558029|gene000415 lmo:lmo0426 5e-06 29....
How to compare lines for similar starting string. I want only first line out of many such lines with similar starting string.
How to do that?
Jan 17 '12 #1
3 1608
use below script for remove duplicate line in file
$ftmp = 'Name.txt';
my %match = ();
{
local @ARGV = ($ftmp);
local $^I = '.tmp';
while(<>)
{
$match{$_}++;
next if $match{$_} > 1;
print;
}
}
Jan 23 '12 #2
Dear sathishkumar88,
I want to remove different lines having similar starting word. not completely identical lines....thanks for the reply.
Jan 23 '12 #3
please use this one
$ftmp = 'Name.txt';
my %match = ();
{
local @ARGV = ($ftmp);
local $^I = '.tmp';
while (<>)
{
$row=$_;
if($row=~m/([^<]*?)\s/is)
{
$word=$1;
$match{$word}++;
next if $match{$word} > 1;
print;
}

}
}
Jan 24 '12 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

11
by: rajarao | last post by:
hi I want to remove the content embedded in <script> and </script> tags submitted via text box. My java script should remove the content embedded between <script> and </script> tag. my current...
3
by: Adam | last post by:
Hello, I have a 2 column file that looks like this: VALUE1 | VALUE2 12 | 7 10 | 7 9 | 8 10 | 8 11 | 8 6 | 9 8 | 9
5
by: nuffnough | last post by:
This is python 2.4.3 on WinXP under PythonWin. I have a config file with many blank lines and many other lines that I don't need. read the file in, splitlines to make a list, then run a loop...
10
by: Backwards | last post by:
Hello all, I'll start by explaining what my app does so not to confuss you when i ask my question. ☺ I have a VB.Net 2.0 app that starts a process (process.start ...) and passes a prameter...
1
by: galewind | last post by:
Hi, I have exported a report to text file using the Outputto action in a macro. There is no problem in the format in the text file except that there is a blank line every 2/3 records. How to remove...
27
by: Neil | last post by:
Hello all! I wrote program with a array of pointers, and I suspect they are pointing at each other in the Do ...While loop. Something is messed up with the increment variable word. A program...
1
by: derik | last post by:
I hav a text file with N number of lines . i am able to read the file till the end .. in between many lines are starting with #. these lines starting with # are to be omitted. pleast help me in...
1
by: devguru | last post by:
I have an interesting script to write. The idea is to "rollup" rows that only differ by one column and concatenate the column value. For example: xxxx a xxxxxxxxxxxx yyyyyyy zzzzzzzzzzzzzzz...
5
momotaro
by: momotaro | last post by:
this program is to double space the lines but it works perfectly but there is two "unusfull lines in my point of view since i tried every possible version to see if the any porblem will hapen if...
4
by: starlight849 | last post by:
Hi, I am looking for advice on the best way to achieve my current goal. I am reading a textfile and extracting information from each line. For example string1 and string2. I am then writing these...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.