473,597 Members | 2,145 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Removing similar lines and repeats

2 New Member
gi|282558029|ge ne000415 lga:LGAS_0495 1e-87 100....
gi|282558029|ge ne000415 lga:LGAS_0495 1e-87 189...
gi|282558029|ge ne000415 lmo:lmo0426 5e-06 29....
How to compare lines for similar starting string. I want only first line out of many such lines with similar starting string.
How to do that?
Jan 17 '12 #1
3 1619
sathishkumar88
4 New Member
use below script for remove duplicate line in file
$ftmp = 'Name.txt';
my %match = ();
{
local @ARGV = ($ftmp);
local $^I = '.tmp';
while(<>)
{
$match{$_}++;
next if $match{$_} > 1;
print;
}
}
Jan 23 '12 #2
perlboy
2 New Member
Dear sathishkumar88,
I want to remove different lines having similar starting word. not completely identical lines....thanks for the reply.
Jan 23 '12 #3
sathishkumar88
4 New Member
please use this one
$ftmp = 'Name.txt';
my %match = ();
{
local @ARGV = ($ftmp);
local $^I = '.tmp';
while (<>)
{
$row=$_;
if($row=~m/([^<]*?)\s/is)
{
$word=$1;
$match{$word}++ ;
next if $match{$word} > 1;
print;
}

}
}
Jan 24 '12 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

11
3052
by: rajarao | last post by:
hi I want to remove the content embedded in <script> and </script> tags submitted via text box. My java script should remove the content embedded between <script> and </script> tag. my current code is function RemoveHTMLScript(strText) { var regEx = /<script\w*<\/script>/g
3
1823
by: Adam | last post by:
Hello, I have a 2 column file that looks like this: VALUE1 | VALUE2 12 | 7 10 | 7 9 | 8 10 | 8 11 | 8 6 | 9 8 | 9
5
1999
by: nuffnough | last post by:
This is python 2.4.3 on WinXP under PythonWin. I have a config file with many blank lines and many other lines that I don't need. read the file in, splitlines to make a list, then run a loop that looks like this:
10
6771
by: Backwards | last post by:
Hello all, I'll start by explaining what my app does so not to confuss you when i ask my question. ☺ I have a VB.Net 2.0 app that starts a process (process.start ...) and passes a prameter through from a combo box. The combo box items are made up of IP address and computer host name. Anything a user places in this combo box it writes this to a txt file called history.txt
1
3052
by: galewind | last post by:
Hi, I have exported a report to text file using the Outputto action in a macro. There is no problem in the format in the text file except that there is a blank line every 2/3 records. How to remove these lines??
27
2162
by: Neil | last post by:
Hello all! I wrote program with a array of pointers, and I suspect they are pointing at each other in the Do ...While loop. Something is messed up with the increment variable word. A program clip of what I'm talking about. #include <stdio.h> #include <string.h>
1
1488
by: derik | last post by:
I hav a text file with N number of lines . i am able to read the file till the end .. in between many lines are starting with #. these lines starting with # are to be omitted. pleast help me in removing the lines starting with #
1
1999
by: devguru | last post by:
I have an interesting script to write. The idea is to "rollup" rows that only differ by one column and concatenate the column value. For example: xxxx a xxxxxxxxxxxx yyyyyyy zzzzzzzzzzzzzzz xxxx b xxxxxxxxxxxx yyyyyyy zzzzzzzzzzzzzzz 1xxx a xxxxxxxxxxxx yyyyyyy zzzzzzzzzzzzzzz 1xxx b xxxxxxxxxxxx yyyyyyy zzzzzzzzzzzzzzz 2xxx a xxxxxxxxxxxx yyyyyyy zzzzzzzzzzzzzzz 2xxx b xxxxxxxxxxxx yyyyyyy zzzzzzzzzzzzzzz
5
1439
momotaro
by: momotaro | last post by:
this program is to double space the lines but it works perfectly but there is two "unusfull lines in my point of view since i tried every possible version to see if the any porblem will hapen if omited but there was none...could we just take them out?" this is the code: #include <stdio.h> #include <string.h> #define LINE_LEN 80
4
1978
by: starlight849 | last post by:
Hi, I am looking for advice on the best way to achieve my current goal. I am reading a textfile and extracting information from each line. For example string1 and string2. I am then writing these string to a datatable. However, sometimes there will be repeats in string1 and string2. If there are repeated lines I would like to be able to delete the repeating line and add a counter to find the total number of repeating lines. so if a string...
0
7965
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8380
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8258
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
5847
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5426
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
3881
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
3923
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1493
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1231
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.