Larry I Smith wrote:
simon wrote:On my pc (an old Gateway PII 450MHZ with 384MB of RAM):
simon.cpp runs in 2.20 seconds and uses 5624KB of memory.
Thanks for that, I get 1.24 sec and 6mb.
I just need to check what the difference is with my code.
simon2.cpp runs in 2.22 seconds and uses 6272KB of memory.
Your mileage may vary. I'm running SuSE Linux v9.3 and
using the GCC "g++" compiler v3.3.5.
Regards,
Larry
Here are the 3 programs:
<snip code>
Regards,
Larry
Thanks for that, this is great.
I wonder if my Trim(...) function was not part of the problem.
After profiling I noticed that delete [], (or even free(..) ) takes
around 50% of the whole time.
Maybe I should get rid of the dynamic allocation all together.
Simon
What does your profiler say about simon2.cpp?
Actually 1.24 seconds is pretty good for 100000 records.
As far as the memory usage goes, did you read the 2
articles on malloc that I posted earlier? Whether you
use new/delete or std::string (which does its own new/delete
behind the scenes) doesn't make much difference in performance
or memory usage, but std::string allows you much more
flexibility when manipulating the strings after you've
filled your vector (i.e. later in the program).
Due to the many (200000) tiny memory allocations, your memory
usage would be about:
2.5 * (sizeof(sFiledD ata) * 100000)
when both strings (sSomeString1 & sSomeString2) are small.
16 bytes minimum (plus the pointer kept in sFileData) will
be allocated for each of those strings. So, using pointers
in sFileData, the actual memory used for one sFiledData
is at least 48 bytes.
Regards,
Larry
What am I missing here? There must be some part of the problem that I
missed. The following reads in 100000 data paris, one per line, in far
less than a second.
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <iomanip>
#include <iterator>
using namespace std;
template<typena me Key_T, typename Value_T>
struct Data {
Key_T _key;
Value_T _value;
istream& fromStream(istr eam& in) {
return in >> _key >> _value;
}
ostream& toStream(ostrea m& out) const {
ios::fmtflags oldFlags(out.se tf(ios::left));
out << setw(25) << _key << _value;
out.setf(oldFla gs);
}
};
template<typena me Key_T, typename Value_T>
istream& operator>>(istr eam& in, Data<Key_T, Value_T>& data) {
return data.fromStream (in);
}
template<typena me Key_T, typename Value_T>
ostream& operator<<(ostr eam& out, const Data<Key_T,Valu e_T>& data) {
return data.toStream(o ut);
}
int main(int argc, char* argv[]) {
if(!(argc > 1)) {
cerr << "records filename: " << endl;
return -1;
}
ifstream ifs(argv[1]);
if(!ifs.is_open ()) {
cerr << "Failed to open file: " << argv[1] << endl;
return -1;
}
typedef Data<string, string> D_T;
typedef vector<D_T> DV_T;
DV_T dv;
copy(istream_it erator<D_T>(ifs ),istream_itera tor<D_T>(),back _inserter(dv));
// copy(dv.begin() , dv.end(),ostrea m_iterator<D_T> (cout,"\n"));
}
--
If our hypothesis is about anything and not about some one or more
particular things, then our deductions constitute mathematics. Thus
mathematics may be defined as the subject in which we never know what we
are talking about, nor whether what we are saying is true.-Bertrand Russell
