boost:threads tweaks?

Hi All,

I am converting some C code into C++ code. The objective is to improve throughput.

I have some code written in C which serially parses through a list of files[~10000], opens each one of them, processes the data and closes the file. All the files are processed one by one.

The obvious performance bottleneck that i could think of is the wasted cpu cycles for file i/o.

*My solution* was to spawn multiple threads to do the file i/o. For around ~10000 files i spawned around 10 threads. All the file names are in one stl container [deque used as queue]. The threads process the files from the container in a round-robin fashion. So whenever a thread finishes processing a particular file it gets another one from the container untill the container is empty

I used the boost::thread_group to manage the 10 threads.

However i notice that after i changed the code by introducing boost::threads and an stl container the performance drops and the code runs slower than the version without multiple threads.

some details:
size of each file: approx 2kb
boost version : 1.32
compiler: msvc 7.1
cpu: intel 4 core, 3.20 ghz

I have made a concious effort to keep the critical sections to the minimum and to make the code as efficient as possible [by me]

Is there some way in which the boost::thread_group can be tweaked by passing config params etc? I could not find anything relevant in the boost documentation for 1.32.

Thanks!