By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
435,628 Members | 1,175 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 435,628 IT Pros & Developers. It's quick & easy.

replacing a specific part of a specific line of text inside a file

blazedaces
100+
P: 284
Alright guys, so the title explains exactly my goal. The truth is I'm going to be reading in a lot of data from an xml file. The file is too large and there's too much data to store in arraylists without running out of memory, so I'm reading and as I'm reading I'm going to write to a file.

This is the thing though, I already can do this and have it done, but I want to modify the program so you can choose what data you want to take out. To do this I would set up the text file to show something like this:

Expand|Select|Wrap|Line Numbers
  1. one:1,2,3,4,5,
  2. two:1,2,3,4,5,
  3. three:1,2,3,4,5,
  4. four:1,2,3,4,5,
  5. five:1,2,3,4,5,
  6.  
Where one, two, etc. are data names and the numbers are data separated by commas as you see.

Here's my problem. I will not run into all the data at one time. Every second is another piece of information, so when I get to that element in the xml I want to open the file, locate the correct data (what line), then replace that line, so it could look like this afterwards:

Expand|Select|Wrap|Line Numbers
  1. one:1,2,3,4,5,
  2. two:1,2,3,4,5,
  3. three:1,2,3,4,5,
  4. four:1,2,3,4,5,6,
  5. five:1,2,3,4,5,
  6.  
Now, I wrote something using RandomAccessFile in java, here's all my code (it's a test program):

Expand|Select|Wrap|Line Numbers
  1. import java.util.*;
  2. import java.io.*;
  3.  
  4. public class testwriteMoreToFile {
  5.     private RandomAccessFile raf;
  6.     private static String fileToBe;
  7.  
  8.     public void writeMoreToFile(String tagName, String newData) {
  9.         try {
  10.             try { 
  11.                 raf = new RandomAccessFile( new File(fileToBe), "rw");
  12.             } catch (FileNotFoundException e) {
  13.                 raf = new RandomAccessFile(fileToBe, "rw");
  14.             } 
  15.  
  16.             StringBuffer contents = new StringBuffer();
  17.             String line = null;
  18.             long prevFilePointer = 0;
  19.  
  20.             while ((line = raf.readLine()) != null) {
  21.                 if (line.substring(0,tagName.length()).equals(tagName)) {
  22.                     contents.append(line).append(newData).append(",").append(System.getProperty("line.separator"));
  23.                     raf.seek(prevFilePointer);
  24.                     raf.writeChars(contents.toString());
  25.                     break;
  26.                 }
  27.                 prevFilePointer = raf.getFilePointer();
  28.             }
  29.         } catch (IOException e2) { 
  30.             e2.printStackTrace();
  31.         } finally {
  32.             try {
  33.                 raf.close();
  34.             } catch (IOException e2) { 
  35.                 e2.printStackTrace();
  36.             }
  37.         }
  38.     }
  39.  
  40.     public static void main(String args[]) {
  41.         String[] tagNames = new String[]{ "one","two","three","four","five" };
  42.         String line = new String("1,2,3,4,5,");
  43.         String[] lines = new String[5];
  44.         fileToBe = "Z:\\test.txt";
  45.         PrintWriter out = null;
  46.         for(int i = 0; i < lines.length; i++) {
  47.             lines[i] = tagNames[i]+":"+line;
  48.         }
  49.         try {
  50.             out = new PrintWriter(fileToBe);
  51.             for (int i = 0; i < lines.length; i++) {
  52.                 out.println(lines[i]);
  53.             }
  54.         } catch (IOException e2) { 
  55.             e2.printStackTrace();
  56.         } finally {
  57.             out.close();
  58.         }
  59.  
  60.         testwriteMoreToFile test = new testwriteMoreToFile();
  61.  
  62.         test.writeMoreToFile(tagNames[3], "6");
  63.     }
  64. }
  65.  
I predicted this problem before I tried it, but thought maybe it could work so I'll try it anyway.

The output looks like this in the text file:

Expand|Select|Wrap|Line Numbers
  1. one:1,2,3,4,5,
  2. two:1,2,3,4,5,
  3. three:1,2,3,4,5,
  4.  f o u r : 1 , 2 , 3 , 4 , 5 , 6 , 
  5.  
  6.  
  7.  
After I add the line (and it doesn't even seem to add correctly, why the spaces?) bytes after that probably don't read correctly..

Any ideas guys, just ideas, what do you think? Any better concepts?

I've got this one that I read somewhere:

Two files:

Copy first line of file 1 to file 2
repeat until you get to line you want to change
change the line, put it in file 2
continue till end copying to file 2

Problem is if I do this sooooo many times because of all the data I"ll need to do it with will this take a very long time? Even so, what, 300 copies of a file? I should delete the old one right, I believe something was mentioned about a kill method... Can I delete the old one and then change the new file name to be the old file name?

Thanks for all the help guys...

-blazed
Jul 6 '07 #1
Share this Question
Share on Google+
10 Replies


P: 7
First, there are many different XML-parsers.
- Those which load the complete file into memory (bad for big files)
- Those which walk the XML-file from top to bottom (Visitor-Pattern i think)
maybe SAX-parser, if i'm not mistaken

Maybe using another XML-parser solves your problem in the first place.

Second i would do something like this:
Expand|Select|Wrap|Line Numbers
  1. public class FileTest {
  2.  
  3.     public static void main(String[] args) throws Exception {
  4.         BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(new File("input.txt"))));
  5.         try {
  6.             BufferedWriter writer = new BufferedWriter(new FileWriter(new File("output.txt")));
  7.             try {
  8.                 // for performance compile once and use repeatedly (instead of line.matches("four.*"))
  9.                 Pattern pattern = Pattern.compile("(four.*)"); // brackets only needed if using regex replace below ...
  10.                 String line;
  11.                 while ((line = reader.readLine()) != null) {
  12.                     Matcher matcher = pattern.matcher(line);
  13.                     if (matcher.matches()) {
  14.                         line = line + "6,"; // your data
  15.                         // line = matcher.replaceAll("$16,"); // or with regex-replace ... "$1" is the first "()" in the pattern above
  16.                     }
  17.                     writer.append(line);
  18.                     writer.newLine();
  19.                     // writer.flush(); // to see everything that is written immediately in the file [it's buffered otherwise]
  20.                 }
  21.             } finally {
  22.                 reader.close();
  23.             }
  24.         } finally {
  25.             reader.close();
  26.         }
  27.     }
  28.  
  29. }
Greetings Finomosec;
Jul 6 '07 #2

Expert 10K+
P: 11,448
I'd say use a (relational) database using one table with two columns: the primary
key ("one", "two" "three" etc.) and a second column that simply contains the
text ("1, 2, 3, 4, 5" etc.).

It'll be easy to update/delete/insert data then and when you have to read data in
you could do the processing on that second column. Using plain text files is an
unmanageable burden (as you already sketched).

kind regards,

Jos
Jul 6 '07 #3

blazedaces
100+
P: 284
I'd say use a (relational) database using one table with two columns: the primary
key ("one", "two" "three" etc.) and a second column that simply contains the
text ("1, 2, 3, 4, 5" etc.).

It'll be easy to update/delete/insert data then and when you have to read data in
you could do the processing on that second column. Using plain text files is an
unmanageable burden (as you already sketched).

kind regards,

Jos
Looks like I'll be looking into databases then (relational). Thank you...

-blaze
Jul 6 '07 #4

blazedaces
100+
P: 284
First, there are many different XML-parsers.
- Those which load the complete file into memory (bad for big files)
- Those which walk the XML-file from top to bottom (Visitor-Pattern i think)
maybe SAX-parser, if i'm not mistaken

Greetings Finomosec;
Sorry that I was unclear dude. I have the xml reading data part down, I'm using SAX. Thing is I'm reading 200mb+ files, sometimes they can be gigabytes in size, too big to use something like DOM. Thanks though...

As for your suggestion for the code I think it might be easier to take Jos' suggestion and lookup databases, then simply print it to a file or input into something like excel (have no idea if this is possible).

Just wanted to respond to your post as well... Thanks for your input,

-blazed
Jul 6 '07 #5

10K+
P: 13,264
Speaking of databases MySQL 5.0 + has improved the MySQL database a lot. If you want something quick and free you can consider using that one.
Jul 7 '07 #6

Expert 10K+
P: 11,448
Speaking of databases MySQL 5.0 + has improved the MySQL database a lot. If you want something quick and free you can consider using that one.
Personally I use Caché; it's an object oriented database that got rid of that
nasty OR mapping (Object - Relational). Relational database tables are not
well suited for storing objects. Persisting objects using this database is a
breeze; it's just like serializing Java POJOs but you can search, manipulate
them just like they are in memory all the time. No need for all those clumsy
DAO, DTO etc. patterns anymore. (Caché is free, google for it).

kind regards,

Jos

ps. I'm just a happy user, no commercial interest in Caché at all.
Jul 7 '07 #7

10K+
P: 13,264
Personally I use Caché; it's an object oriented database that got rid of that
nasty OR mapping (Object - Relational). Relational database tables are not
well suited for storing objects. Persisting objects using this database is a
breeze; it's just like serializing Java POJOs but you can search, manipulate
them just like they are in memory all the time. No need for all those clumsy
DAO, DTO etc. patterns anymore. (Caché is free, google for it).

kind regards,

Jos

ps. I'm just a happy user, no commercial interest in Caché at all.
My knowlegde of object oriented databases is rather limited(still in the learning phase for them I'd say).
Caché looks good. (First link I opened was a clothes shop).
Have you worked with JavaDB before?
Jul 7 '07 #8

Expert 10K+
P: 11,448
My knowlegde of object oriented databases is rather limited(still in the learning phase for them I'd say).
Caché looks good. (First link I opened was a clothes shop).
Have you worked with JavaDB before?
I think I've played with it a bit once, but I'm not sure about it. Caché is made by
Intersystems. Those folks made "MUMPS" years ago. For reasons that are
beyond me MUMPS is still extremely popular in the medical equipment industry.
You hardly find any, say, Oracle or DB2 or whatever overthere.

kind regards,

Jos
Jul 7 '07 #9

10K+
P: 13,264
I think I've played with it a bit once, but I'm not sure about it. Caché is made by
Intersystems. Those folks made "MUMPS" years ago. For reasons that are
beyond me MUMPS is still extremely popular in the medical equipment industry.
You hardly find any, say, Oracle or DB2 or whatever overthere.

kind regards,

Jos
They are calling it the world's fastest database. I've requested a free cd ( I do not want to download 242mb of it on our network today)
I hope they'll send the CD.
Jul 7 '07 #10

Expert 10K+
P: 11,448
They are calling it the world's fastest database. I've requested a free cd ( I do not want to download 242mb of it on our network today)
I hope they'll send the CD.
I didn't even request a free CD but they still sent me one: within a few weeks,
packed with a lot of propaganda paperwork ;-) I like that database, I still haven't
used it for any commercial application yet, just playtime. I like their 'jalapeno'
technique (JAva LAnguage PErsistency NO mapping); lousy acronym though.

kind regards,

Jos
Jul 7 '07 #11

Post your reply

Sign in to post your reply or Sign up for a free account.