473,219 Members | 2,017 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,219 software developers and data experts.

Delete a line from a text file

Hello

I'm trying to get a single line removed from a text file using a search
pattern (pretty simple: if line contains "NODE1") -> remove line). To
achieve this I's like to operate with only the original file and no
temp file since the file is 1. on a network resource 2. pretty big 3.
accessed by plenty of clients. Copying the file or reading everything
into an array and then writing it back using the StreamWriter would
lock the file for an unacceptable period of time.
using a database is not an option :o(

The file is ment for logging a progam execution and contains the
following:

NODE USER TIMESTAP VERSION LOCALREG
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

As an example i'd like to remove all lines containing "NODE1" giving
me:

NODE USER TIMESTAP VERSION LOCALREG
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

Any help would be greatly appreciated....

Tom

Jun 23 '06 #1
6 41873
Tom,

Doing it in-file is not possible. You can't shorten a file (not by any
means I know).

Rather, what you have to do is resort to creating a new file, and then
overwriting the original file with that.

So, that being said, you would read through your file, and as you find
the records you want to keep, you would write them to the new file.

Then, when you have the new, shorter file, delete the old file, and
rename the new file.

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"tomtown.net" <to*******@gmail.com> wrote in message
news:11**********************@m73g2000cwd.googlegr oups.com...
Hello

I'm trying to get a single line removed from a text file using a search
pattern (pretty simple: if line contains "NODE1") -> remove line). To
achieve this I's like to operate with only the original file and no
temp file since the file is 1. on a network resource 2. pretty big 3.
accessed by plenty of clients. Copying the file or reading everything
into an array and then writing it back using the StreamWriter would
lock the file for an unacceptable period of time.
using a database is not an option :o(

The file is ment for logging a progam execution and contains the
following:

NODE USER TIMESTAP VERSION LOCALREG
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

As an example i'd like to remove all lines containing "NODE1" giving
me:

NODE USER TIMESTAP VERSION LOCALREG
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

Any help would be greatly appreciated....

Tom

Jun 23 '06 #2
Hello tomtown.net,

Well, first off, you don't *remove* anything from a file... any file. You
can *add* (append) to a file or or delete ALL the contents of a file and
re-write it. If you can modify the consuming application (provided people
aren't opening the file directly in notepad or some crazy shit like that)
then you may want to consider an index file. Using an index you could easily
mark records as deleted or to be ignored.

Unfortunately you can't treat the file with an ODBC connection/command because
the ODBC text file driver does not support the DELETE command.

-Boo
Hello

I'm trying to get a single line removed from a text file using a
search
pattern (pretty simple: if line contains "NODE1") -> remove line). To
achieve this I's like to operate with only the original file and no
temp file since the file is 1. on a network resource 2. pretty big 3.
accessed by plenty of clients. Copying the file or reading everything
into an array and then writing it back using the StreamWriter would
lock the file for an unacceptable period of time.
using a database is not an option :o(
The file is ment for logging a progam execution and contains the
following:

NODE USER TIMESTAP VERSION LOCALREG
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
As an example i'd like to remove all lines containing "NODE1" giving
me:

NODE USER TIMESTAP VERSION LOCALREG
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
Any help would be greatly appreciated....

Tom

Jun 23 '06 #3

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard.caspershouse.com> wrote in
message news:uk**************@TK2MSFTNGP03.phx.gbl...
Tom,

Doing it in-file is not possible. You can't shorten a file (not by any
means I know).
SetEndOfFile

Rather, what you have to do is resort to creating a new file, and then
overwriting the original file with that.

So, that being said, you would read through your file, and as you find
the records you want to keep, you would write them to the new file.
Correct. But, you can do that in-place, as far as the file is concerned.

Think of the way memmove works.

How about the following algorithm? Note that ptrin and ptrout can be
pointers, indexes, or whatever your language prefers.

char[] buffer1 = allocate 2k;
char[] buffer2 = allocate 1k;
read from file at [buffer1 + 0, buffer1 + 2k)

ptrin = buffer1
ptrout = buffer2

flag removeit = false
until (reached end of input file) do
if ptrin reached buffer1 + 1k then
move [buffer1 + 1k ... buffer1 + 2k) to buffer1 + 0
ptrin -= 1k
read from file at [buffer1 + 1k, buffer + 2k)
endif
if ptrout reached buffer2 + 1k then
write to file [buffer2 + 0, buffer2 + 1k)
ptrout = buffer2
endif
if removeit then
if value at ptrin is newline then
removeit = false
endif
else
if value at ptrin is "NODE1" then
removeit = true
else
move value at ptrin to ptrout
advance ptrout
endif
endif

advance ptrin
loop
write to file [buffer2, ptrout)
set end of file

Then, when you have the new, shorter file, delete the old file, and
rename the new file.

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"tomtown.net" <to*******@gmail.com> wrote in message
news:11**********************@m73g2000cwd.googlegr oups.com...
Hello

I'm trying to get a single line removed from a text file using a search
pattern (pretty simple: if line contains "NODE1") -> remove line). To
achieve this I's like to operate with only the original file and no
temp file since the file is 1. on a network resource 2. pretty big 3.
accessed by plenty of clients. Copying the file or reading everything
into an array and then writing it back using the StreamWriter would
lock the file for an unacceptable period of time.
using a database is not an option :o(

The file is ment for logging a progam execution and contains the
following:

NODE USER TIMESTAP VERSION LOCALREG
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

As an example i'd like to remove all lines containing "NODE1" giving
me:

NODE USER TIMESTAP VERSION LOCALREG
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

Any help would be greatly appreciated....

Tom


Jun 23 '06 #4

"Ben Voigt" <rb*@nospam.nospam> wrote in message
news:uR****************@TK2MSFTNGP04.phx.gbl...

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard.caspershouse.com> wrote
in message news:uk**************@TK2MSFTNGP03.phx.gbl...
Tom,

Doing it in-file is not possible. You can't shorten a file (not by
any means I know).
SetEndOfFile

Rather, what you have to do is resort to creating a new file, and then
overwriting the original file with that.

So, that being said, you would read through your file, and as you find
the records you want to keep, you would write them to the new file.


Correct. But, you can do that in-place, as far as the file is concerned.

Think of the way memmove works.

How about the following algorithm? Note that ptrin and ptrout can be
pointers, indexes, or whatever your language prefers.


More notes:
* Of course you must keep the read and write offsets in the file separate.
* You will need to lock the file while checking the file length, but you can
release the lock immediately.
* Run this on the file server if at all possible, to avoid the roundtrips
and minimize the length of time locked.
* For the last block (partial block), keep the file locked, so that you can
truncate the file before some client extends it more.
* You can pause and resume anytime, just by saving your pointers, because
the clients are writing to a different area of the file. This means if you
know the start the file is already processed, you can skip to the new data.
* This algorithm will actually only remove from the word "NODE1" to the end
of the line. If the keyword does not appear at the start of the line, some
of the line will remain. To remove the entire line, you can save the
pointer/index each time you hit a newline, and move back to writing at that
location. Note that this affects your in-memory pointer and potentially
also your file write pointer.

char[] buffer1 = allocate 2k;
char[] buffer2 = allocate 1k;
read from file at [buffer1 + 0, buffer1 + 2k)

ptrin = buffer1
ptrout = buffer2

flag removeit = false
until (reached end of input file) do
if ptrin reached buffer1 + 1k then
move [buffer1 + 1k ... buffer1 + 2k) to buffer1 + 0
ptrin -= 1k
read from file at [buffer1 + 1k, buffer + 2k)
endif
if ptrout reached buffer2 + 1k then
write to file [buffer2 + 0, buffer2 + 1k)
ptrout = buffer2
endif
if removeit then
if value at ptrin is newline then
removeit = false
endif
else
if value at ptrin is "NODE1" then
removeit = true
else
move value at ptrin to ptrout
advance ptrout
endif
endif

advance ptrin
loop
write to file [buffer2, ptrout)
set end of file

Then, when you have the new, shorter file, delete the old file, and
rename the new file.

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"tomtown.net" <to*******@gmail.com> wrote in message
news:11**********************@m73g2000cwd.googlegr oups.com...
Hello

I'm trying to get a single line removed from a text file using a search
pattern (pretty simple: if line contains "NODE1") -> remove line). To
achieve this I's like to operate with only the original file and no
temp file since the file is 1. on a network resource 2. pretty big 3.
accessed by plenty of clients. Copying the file or reading everything
into an array and then writing it back using the StreamWriter would
lock the file for an unacceptable period of time.
using a database is not an option :o(

The file is ment for logging a progam execution and contains the
following:

NODE USER TIMESTAP VERSION LOCALREG
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

As an example i'd like to remove all lines containing "NODE1" giving
me:

NODE USER TIMESTAP VERSION LOCALREG
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;

Any help would be greatly appreciated....

Tom



Jun 23 '06 #5
Hello all

Thanks all! This was my first post and so many people are helping!
Thank you very much!
I've once made a class that replaces certain characters (code below),
but as most of you suggested it also needs a temp file to be created
and then copies the file to the original location.
I just got the idea using XML instead of bare text files since
manipulations seem to be much easier using the SelectSingleNode,
InnerXml and ReplaceChild method of XmlElement. I found a pretty good
article here (might help someone else too)
http://www.codeproject.com/soap/myXPath.asp

Code to replace strings in a textfile here (I used this for a
commandline search and replace tool downloadable at
http://www.tomtown.net/?com=tech&sco...browse&cid=002 ):

// ================================================== ===============

public static void ReplaceString(string textFileName, string
searchStr, string replaceStr, int backup)
{
string tempFileName = Path.GetTempFileName();

StreamReader sr = null;
sr = new StreamReader(textFileName,
Encoding.GetEncoding("windows-1252"));
StreamWriter sw = null;
sw = new StreamWriter(tempFileName, false,

Encoding.GetEncoding("windows-1252"));
string line;

System.Text.StringBuilder newline = new
System.Text.StringBuilder();

while ((line = sr.ReadLine()) != null)
{
string correctString = line.Replace(searchStr,
replaceStr);
sw.WriteLine(correctString);
}

sr.Close();
sw.Close();

if (backup == 1)
{
if (File.Exists(textFileName + "_bak"))
File.Delete(textFileName + "_bak");
File.Move(textFileName, textFileName + "_bak");
}

File.Delete(textFileName);
File.Move(tempFileName, textFileName);
}

// ================================================== ===============

Thanx again for all your efforts!!!

Tom

Jun 24 '06 #6

"tomtown.net" <to*******@gmail.com> wrote in message
news:11**********************@m73g2000cwd.googlegr oups.com...
Hello all

Thanks all! This was my first post and so many people are helping!
Thank you very much!
I've once made a class that replaces certain characters (code below),
but as most of you suggested it also needs a temp file to be created
and then copies the file to the original location.
If all write operations are append-only, and the resulting text is always no
longer than the original, you can do the operation in-place.

If you have multiple processes writing the file, presumably each locks the
file for write to prevent corruption, and then closes the file immediately
afterwards? This is important, because when you reach the end of the file
you'll do almost exactly the same thing, except you'll call SetEndOfFile
instead of appending data.
I just got the idea using XML instead of bare text files since
manipulations seem to be much easier using the SelectSingleNode,
InnerXml and ReplaceChild method of XmlElement. I found a pretty good
article here (might help someone else too)
http://www.codeproject.com/soap/myXPath.asp

Code to replace strings in a textfile here (I used this for a
commandline search and replace tool downloadable at
http://www.tomtown.net/?com=tech&sco...browse&cid=002 ):

// ================================================== ===============

public static void ReplaceString(string textFileName, string
searchStr, string replaceStr, int backup)
{
string tempFileName = Path.GetTempFileName();

StreamReader sr = null;
sr = new StreamReader(textFileName,
Encoding.GetEncoding("windows-1252"));
StreamWriter sw = null;
sw = new StreamWriter(tempFileName, false,

Encoding.GetEncoding("windows-1252"));
string line;

System.Text.StringBuilder newline = new
System.Text.StringBuilder();

while ((line = sr.ReadLine()) != null)
{
string correctString = line.Replace(searchStr,
replaceStr);
sw.WriteLine(correctString);
}

sr.Close();
sw.Close();

if (backup == 1)
{
if (File.Exists(textFileName + "_bak"))
File.Delete(textFileName + "_bak");
File.Move(textFileName, textFileName + "_bak");
}

File.Delete(textFileName);
File.Move(tempFileName, textFileName);
}

// ================================================== ===============

Thanx again for all your efforts!!!

Tom

Jun 26 '06 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Bob A | last post by:
I do a weekly import of data from two text files, using two schema files. I have automated the import process that gets the data from the two files using a macro, and everything works great. ...
4
by: Paul | last post by:
Hello Everybody, Being new to VBA for Access, I have a question about inputing a text file into access. The text file has 23 lines per file and needs to go into 1 record in a table. The 1st...
13
by: Dan V. | last post by:
How do I create a one line text file with these control codes? e.g.: 144 = 0x90 and 147 = 0x93? I am trying to create a one line text file with these characters all one one row with no spaces. ...
2
by: Doug Oliver | last post by:
I've got an intermittent problem when trying to write to a text file (used as a log). ASPNET has modify on folder, and at times logs to file, then I get a null reference exception, followed on...
8
by: MLH | last post by:
Am trying to import 20,000+ lines of text in a file FTP'd from a UNIX platform to windows via FTP session in a DOS box. About 2000 records have multiple lines in them separated by CRLF's. ...
2
by: Pipex News Group | last post by:
I Have a text file that I want to delete using the On Click event of a forms button, the text file is along the following path: C:\1.txt I have managed to create the text file from Access...
1
by: Child of His | last post by:
I have been through every trick I know, or has been suggested. I have a one to two million line fixed field database in text format. I want to bring it into Access 97. When I use the external...
2
by: cdun2 | last post by:
Hello, I have some code that reads each line of a text file, and if a line is found where the length of the string in the line is 384, it writes the line to a text file. The other step that I...
1
by: Tim Kelley | last post by:
I am using a streamreader to read a one line text file. The line looks like this: "99999","04/30/2007","XXXXXXXX","XXXXX, XX","XX","" When the line is read in I get this: i ...
13
by: JJ | last post by:
I have a need to input a large tab delimited text file, which I will parse to check it has the expected columns, before allowing the user to submit it to the database. The user may paste the file...
1
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
0
by: veera ravala | last post by:
ServiceNow is a powerful cloud-based platform that offers a wide range of services to help organizations manage their workflows, operations, and IT services more efficiently. At its core, ServiceNow...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
by: jimatqsi | last post by:
The boss wants the word "CONFIDENTIAL" overlaying certain reports. He wants it large, slanted across the page, on every page, very light gray, outlined letters, not block letters. I thought Word Art...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.