C++ / Problems using "seekg()" method

[Rubenintx]

Need some help with this code / Using UNIX.

The goals is to read a big file (2,000,000) lines, using fork() to create children to help in the process.

I am able to get the program (C++) running and this is what it does:
1) creates the number of children the user indicates
2) can read the number lines in the TXT file
3) tells how many lines each child will read

This is how I understand what the program will do and I want to make sure thatI am right or wrong:

As an example, lets assume that we have: 2 children & 10 TXT lines, each child will read five lines.

When the loop creates a child, that child will go to the TXT file will read its own set of lines (in this case 5) and will copy them to the pipe. After this, the loop will continue and each new child will do the same, with its own set of lines.
Is this correct?...

If this is correct... then I need help in defining the propper way to have the 2nd child created, continue reading the next line, meaning:
Child 1 reads lines: 0,1,2,3,4
Child 2 reads lines: 5,6,7,8,9 <<< how to make the second child go to the proper line to start?

I know that the seekg method goes to a specific line, still I am having problems adding this functionallity to the code.

Please advise...

Thank you...

[RRick]

Seekg will take you to a specific byte location, which can be in the middle of a line. Your idea will work easily if all of the lines are exactly the same size. Then you simply seek to lineNo * lineSize.

If the lines are of a different size, you will have to search for a line after you do the initial seek. You will also have to make sure that no lines fall through the cracks between the different processes.


There is another way to process the lines without this type of complication. You have one master process that reads in the lines from the file and then sends each line out to other processes. You can communicate over stdin/stdout for this type of communication. This will work if the IO between processes is not too large. Also, the master process can determine how many processes are needed.

This type of processing can also be done with threads. In fact, I would suggest using threads in this case. This type of reader thread and multiple process threads is very common for threaded programs. If interested, take a look at the linux pthreads doc and online tutorials.