Hello
I'm trying to get a single line removed from a text file using a search
pattern (pretty simple: if line contains "NODE1") -> remove line). To
achieve this I's like to operate with only the original file and no
temp file since the file is 1. on a network resource 2. pretty big 3.
accessed by plenty of clients. Copying the file or reading everything
into an array and then writing it back using the StreamWriter would
lock the file for an unacceptable period of time.
using a database is not an option :o(
The file is ment for logging a progam execution and contains the
following:
NODE USER TIMESTAP VERSION LOCALREG
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3;
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
As an example i'd like to remove all lines containing "NODE1" giving
me:
NODE USER TIMESTAP VERSION LOCALREG
NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
Any help would be greatly appreciated....
Tom 6 41964
Tom,
Doing it in-file is not possible. You can't shorten a file (not by any
means I know).
Rather, what you have to do is resort to creating a new file, and then
overwriting the original file with that.
So, that being said, you would read through your file, and as you find
the records you want to keep, you would write them to the new file.
Then, when you have the new, shorter file, delete the old file, and
rename the new file.
Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m
"tomtown.ne t" <to*******@gmai l.com> wrote in message
news:11******** **************@ m73g2000cwd.goo glegroups.com.. . Hello
I'm trying to get a single line removed from a text file using a search pattern (pretty simple: if line contains "NODE1") -> remove line). To achieve this I's like to operate with only the original file and no temp file since the file is 1. on a network resource 2. pretty big 3. accessed by plenty of clients. Copying the file or reading everything into an array and then writing it back using the StreamWriter would lock the file for an unacceptable period of time. using a database is not an option :o(
The file is ment for logging a progam execution and contains the following:
NODE USER TIMESTAP VERSION LOCALREG NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
As an example i'd like to remove all lines containing "NODE1" giving me:
NODE USER TIMESTAP VERSION LOCALREG NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
Any help would be greatly appreciated....
Tom
Hello tomtown.net,
Well, first off, you don't *remove* anything from a file... any file. You
can *add* (append) to a file or or delete ALL the contents of a file and
re-write it. If you can modify the consuming application (provided people
aren't opening the file directly in notepad or some crazy shit like that)
then you may want to consider an index file. Using an index you could easily
mark records as deleted or to be ignored.
Unfortunately you can't treat the file with an ODBC connection/command because
the ODBC text file driver does not support the DELETE command.
-Boo Hello
I'm trying to get a single line removed from a text file using a search pattern (pretty simple: if line contains "NODE1") -> remove line). To achieve this I's like to operate with only the original file and no temp file since the file is 1. on a network resource 2. pretty big 3. accessed by plenty of clients. Copying the file or reading everything into an array and then writing it back using the StreamWriter would lock the file for an unacceptable period of time. using a database is not an option :o( The file is ment for logging a progam execution and contains the following:
NODE USER TIMESTAP VERSION LOCALREG NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3; As an example i'd like to remove all lines containing "NODE1" giving me:
NODE USER TIMESTAP VERSION LOCALREG NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3; Any help would be greatly appreciated....
Tom
"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard .caspershouse.c om> wrote in
message news:uk******** ******@TK2MSFTN GP03.phx.gbl... Tom,
Doing it in-file is not possible. You can't shorten a file (not by any means I know).
SetEndOfFile Rather, what you have to do is resort to creating a new file, and then overwriting the original file with that.
So, that being said, you would read through your file, and as you find the records you want to keep, you would write them to the new file.
Correct. But, you can do that in-place, as far as the file is concerned.
Think of the way memmove works.
How about the following algorithm? Note that ptrin and ptrout can be
pointers, indexes, or whatever your language prefers.
char[] buffer1 = allocate 2k;
char[] buffer2 = allocate 1k;
read from file at [buffer1 + 0, buffer1 + 2k)
ptrin = buffer1
ptrout = buffer2
flag removeit = false
until (reached end of input file) do
if ptrin reached buffer1 + 1k then
move [buffer1 + 1k ... buffer1 + 2k) to buffer1 + 0
ptrin -= 1k
read from file at [buffer1 + 1k, buffer + 2k)
endif
if ptrout reached buffer2 + 1k then
write to file [buffer2 + 0, buffer2 + 1k)
ptrout = buffer2
endif
if removeit then
if value at ptrin is newline then
removeit = false
endif
else
if value at ptrin is "NODE1" then
removeit = true
else
move value at ptrin to ptrout
advance ptrout
endif
endif
advance ptrin
loop
write to file [buffer2, ptrout)
set end of file Then, when you have the new, shorter file, delete the old file, and rename the new file.
Hope this helps.
-- - Nicholas Paldino [.NET/C# MVP] - mv*@spam.guard. caspershouse.co m
"tomtown.ne t" <to*******@gmai l.com> wrote in message news:11******** **************@ m73g2000cwd.goo glegroups.com.. . Hello
I'm trying to get a single line removed from a text file using a search pattern (pretty simple: if line contains "NODE1") -> remove line). To achieve this I's like to operate with only the original file and no temp file since the file is 1. on a network resource 2. pretty big 3. accessed by plenty of clients. Copying the file or reading everything into an array and then writing it back using the StreamWriter would lock the file for an unacceptable period of time. using a database is not an option :o(
The file is ment for logging a progam execution and contains the following:
NODE USER TIMESTAP VERSION LOCALREG NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
As an example i'd like to remove all lines containing "NODE1" giving me:
NODE USER TIMESTAP VERSION LOCALREG NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
Any help would be greatly appreciated....
Tom
"Ben Voigt" <rb*@nospam.nos pam> wrote in message
news:uR******** ********@TK2MSF TNGP04.phx.gbl. .. "Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard .caspershouse.c om> wrote in message news:uk******** ******@TK2MSFTN GP03.phx.gbl... Tom,
Doing it in-file is not possible. You can't shorten a file (not by any means I know). SetEndOfFile
Rather, what you have to do is resort to creating a new file, and then overwriting the original file with that.
So, that being said, you would read through your file, and as you find the records you want to keep, you would write them to the new file.
Correct. But, you can do that in-place, as far as the file is concerned.
Think of the way memmove works.
How about the following algorithm? Note that ptrin and ptrout can be pointers, indexes, or whatever your language prefers.
More notes:
* Of course you must keep the read and write offsets in the file separate.
* You will need to lock the file while checking the file length, but you can
release the lock immediately.
* Run this on the file server if at all possible, to avoid the roundtrips
and minimize the length of time locked.
* For the last block (partial block), keep the file locked, so that you can
truncate the file before some client extends it more.
* You can pause and resume anytime, just by saving your pointers, because
the clients are writing to a different area of the file. This means if you
know the start the file is already processed, you can skip to the new data.
* This algorithm will actually only remove from the word "NODE1" to the end
of the line. If the keyword does not appear at the start of the line, some
of the line will remain. To remove the entire line, you can save the
pointer/index each time you hit a newline, and move back to writing at that
location. Note that this affects your in-memory pointer and potentially
also your file write pointer. char[] buffer1 = allocate 2k; char[] buffer2 = allocate 1k; read from file at [buffer1 + 0, buffer1 + 2k)
ptrin = buffer1 ptrout = buffer2
flag removeit = false until (reached end of input file) do if ptrin reached buffer1 + 1k then move [buffer1 + 1k ... buffer1 + 2k) to buffer1 + 0 ptrin -= 1k read from file at [buffer1 + 1k, buffer + 2k) endif if ptrout reached buffer2 + 1k then write to file [buffer2 + 0, buffer2 + 1k) ptrout = buffer2 endif if removeit then if value at ptrin is newline then removeit = false endif else if value at ptrin is "NODE1" then removeit = true else move value at ptrin to ptrout advance ptrout endif endif
advance ptrin loop write to file [buffer2, ptrout) set end of file
Then, when you have the new, shorter file, delete the old file, and rename the new file.
Hope this helps.
-- - Nicholas Paldino [.NET/C# MVP] - mv*@spam.guard. caspershouse.co m
"tomtown.ne t" <to*******@gmai l.com> wrote in message news:11******** **************@ m73g2000cwd.goo glegroups.com.. . Hello
I'm trying to get a single line removed from a text file using a search pattern (pretty simple: if line contains "NODE1") -> remove line). To achieve this I's like to operate with only the original file and no temp file since the file is 1. on a network resource 2. pretty big 3. accessed by plenty of clients. Copying the file or reading everything into an array and then writing it back using the StreamWriter would lock the file for an unacceptable period of time. using a database is not an option :o(
The file is ment for logging a progam execution and contains the following:
NODE USER TIMESTAP VERSION LOCALREG NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE1 USERNAME1 23.06.2006 16:28:28 1.1 LL=3; NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
As an example i'd like to remove all lines containing "NODE1" giving me:
NODE USER TIMESTAP VERSION LOCALREG NODE2 USERNAME2 23.06.2006 16:29:48 1.1 LL=3;
Any help would be greatly appreciated....
Tom
Hello all
Thanks all! This was my first post and so many people are helping!
Thank you very much!
I've once made a class that replaces certain characters (code below),
but as most of you suggested it also needs a temp file to be created
and then copies the file to the original location.
I just got the idea using XML instead of bare text files since
manipulations seem to be much easier using the SelectSingleNod e,
InnerXml and ReplaceChild method of XmlElement. I found a pretty good
article here (might help someone else too) http://www.codeproject.com/soap/myXPath.asp
Code to replace strings in a textfile here (I used this for a
commandline search and replace tool downloadable at http://www.tomtown.net/?com=tech&sco...browse&cid=002 ):
// =============== =============== =============== =============== =====
public static void ReplaceString(s tring textFileName, string
searchStr, string replaceStr, int backup)
{
string tempFileName = Path.GetTempFil eName();
StreamReader sr = null;
sr = new StreamReader(te xtFileName,
Encoding.GetEnc oding("windows-1252"));
StreamWriter sw = null;
sw = new StreamWriter(te mpFileName, false,
Encoding.GetEnc oding("windows-1252"));
string line;
System.Text.Str ingBuilder newline = new
System.Text.Str ingBuilder();
while ((line = sr.ReadLine()) != null)
{
string correctString = line.Replace(se archStr,
replaceStr);
sw.WriteLine(co rrectString);
}
sr.Close();
sw.Close();
if (backup == 1)
{
if (File.Exists(te xtFileName + "_bak"))
File.Delete(tex tFileName + "_bak");
File.Move(textF ileName, textFileName + "_bak");
}
File.Delete(tex tFileName);
File.Move(tempF ileName, textFileName);
}
// =============== =============== =============== =============== =====
Thanx again for all your efforts!!!
Tom
"tomtown.ne t" <to*******@gmai l.com> wrote in message
news:11******** **************@ m73g2000cwd.goo glegroups.com.. . Hello all
Thanks all! This was my first post and so many people are helping! Thank you very much! I've once made a class that replaces certain characters (code below), but as most of you suggested it also needs a temp file to be created and then copies the file to the original location.
If all write operations are append-only, and the resulting text is always no
longer than the original, you can do the operation in-place.
If you have multiple processes writing the file, presumably each locks the
file for write to prevent corruption, and then closes the file immediately
afterwards? This is important, because when you reach the end of the file
you'll do almost exactly the same thing, except you'll call SetEndOfFile
instead of appending data.
I just got the idea using XML instead of bare text files since manipulations seem to be much easier using the SelectSingleNod e, InnerXml and ReplaceChild method of XmlElement. I found a pretty good article here (might help someone else too) http://www.codeproject.com/soap/myXPath.asp
Code to replace strings in a textfile here (I used this for a commandline search and replace tool downloadable at http://www.tomtown.net/?com=tech&sco...browse&cid=002 ):
// =============== =============== =============== =============== =====
public static void ReplaceString(s tring textFileName, string searchStr, string replaceStr, int backup) { string tempFileName = Path.GetTempFil eName();
StreamReader sr = null; sr = new StreamReader(te xtFileName, Encoding.GetEnc oding("windows-1252")); StreamWriter sw = null; sw = new StreamWriter(te mpFileName, false,
Encoding.GetEnc oding("windows-1252")); string line;
System.Text.Str ingBuilder newline = new System.Text.Str ingBuilder();
while ((line = sr.ReadLine()) != null) { string correctString = line.Replace(se archStr, replaceStr); sw.WriteLine(co rrectString); }
sr.Close(); sw.Close();
if (backup == 1) { if (File.Exists(te xtFileName + "_bak")) File.Delete(tex tFileName + "_bak"); File.Move(textF ileName, textFileName + "_bak"); }
File.Delete(tex tFileName); File.Move(tempF ileName, textFileName); }
// =============== =============== =============== =============== =====
Thanx again for all your efforts!!!
Tom This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Bob A |
last post by:
I do a weekly import of data from two text files, using two schema
files. I have automated the import process that gets the data from the
two files using a macro, and everything works great.
Every week the files are manually replaced by two new files of the
same name.
I would like to put a failsafe to prevent the files from being
imported more than once. Is there an easy way to accomplish this?
Perhaps automatically deleting the file...
|
by: Paul |
last post by:
Hello Everybody,
Being new to VBA for Access, I have a question about inputing a text
file into access. The text file has 23 lines per file and needs to go
into 1 record in a table. The 1st four lines are garbage and don't need
to be inputted. THe rest of the lines only need what is after an equal
sign (=) inputted. Can somebody please help me?
|
by: Dan V. |
last post by:
How do I create a one line text file with these control codes? e.g.: 144 =
0x90 and 147 = 0x93?
I am trying to create a one line text file with these characters all one one
row with no spaces.
1. 144 = 0x90
2. 147 = 0x93
3. STX = (^B = 2 = 0x2)
4. NUL = (^@ = 0 = 0x0)
|
by: Doug Oliver |
last post by:
I've got an intermittent problem when trying to write to a text file (used as
a log). ASPNET has modify on folder, and at times logs to file, then I get a
null reference exception, followed on re-attempt with a "being used by
another process" exception. I then have to recycle the w3wp.exe process,
IIS, and sometimes re-deploy the application on the file system. Seems for
some reason the filesystem cannot see the file... Moreove, if I...
|
by: MLH |
last post by:
Am trying to import 20,000+ lines of text in a file
FTP'd from a UNIX platform to windows via FTP
session in a DOS box. About 2000 records have
multiple lines in them separated by CRLF's.
That's not the idea. Each line was supposed to
be a separate record. I don't know why the 2000
records having MULTIPLE lines of text are so
stubborn. I've run UNIX2DOS utility against the
raw text file - no help. I know that UNIX and MAC
| |
by: Pipex News Group |
last post by:
I Have a text file that I want to delete using the On Click event of a forms
button, the text file is along the following path:
C:\1.txt
I have managed to create the text file from Access using DoCmd.OutputTo, I
can change the text files attributes using SetAttr, but I don't know the way
to Delete the text file?
Any help would be appreciated.
|
by: Child of His |
last post by:
I have been through every trick I know, or has been suggested.
I have a one to two million line fixed field database in text format.
I want to bring it into Access 97.
When I use the external data import, the list of files to import does
not include text files of any type, only other database formats.
On a 102,000 line text file, I was able to split it with Word 97,
import the split files into Excel 97 one at a time, and then save...
|
by: cdun2 |
last post by:
Hello,
I have some code that reads each line of a text file, and if a line is
found where the length of the string in the line is 384, it writes
the line to a text file.
The other step that I need to take is to delete the line from the
source file. The code is as follows;
-*****************************************
Public Sub Main()
|
by: Tim Kelley |
last post by:
I am using a streamreader to read a one line text file. The line looks like
this:
"99999","04/30/2007","XXXXXXXX","XXXXX, XX","XX",""
When the line is read in I get this: i
"\"99999\",\"04/30/2007\",\"XXXXXXXXXX\",\"XXXXXX, XX\",\"XX\",\"\""
I don't get where all the \ are coming from. Is there a different method I
|
by: JJ |
last post by:
I have a need to input a large tab delimited text file, which I will parse
to check it has the expected columns, before allowing the user to submit it
to the database. The user may paste the file into a textbox, or upload it
(haven't decided yet).
The problem I have is that the text file consists of around 3000 lines, and
I want to display the formatted columns to the user before submitting it to
the database (perhaps allowing them to...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |