473,287 Members | 1,574 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

convert std::string to (byte*, DWORD)

I have the contents of an image of type std::string. How can I make a
CxImage object with this type.

The parameters to CxImage is:

CxImage(byte* data, DWORD size)

Thx in advance
Aug 20 '05 #1
16 16353
Khuong Dinh Pham wrote:
I have the contents of an image of type std::string. How can I make a
CxImage object with this type.

The parameters to CxImage is:

CxImage(byte* data, DWORD size)


std::string stuff;

CxImage( convert_to_byte( stuff.data() ), stuff.size() )

convert_to_byte could be as simple as:

const_cast<byte *>( stuff.data() )

good luck
Aug 20 '05 #2
Khuong Dinh Pham wrote:
I have the contents of an image of type std::string. How can I make a
CxImage object with this type.

The parameters to CxImage is:

CxImage(byte* data, DWORD size)

Thx in advance

std::string.c_str() returns a 'const char *' (a pointer
to a string that can not be modified).

std::string.length() returns the data length
of the content.

Since CxImage() takes a 'byte *' rather than a
'const byte *', I would expect that it modifies
the memory pointed to by its first arg ('data').

If CxImage() does NOT modify the memory pointed to by its
first arg ('data'), and your image is in a std::string named
'myStr', then you might try something like this:

CxImage(myStr.c_str(), myStr.length());

You may have to add some casts to convert the first arg
to 'byte *' and the 2nd arg to 'DWORD'.

If CxImage() DOES modify the memory pointed to by its first
arg ('data'), then you could copy the string to a char buf:

try {
char * buf = new char[myStr.length()];
memcpy(buf, myStr.c_str(), myStr.length());
CxImage(buf, myStr.length());
delete[] buf;
}
catch (std::bad_alloc) {
std::cerr << "memory allocation failed\n";
}

Larry
Aug 20 '05 #3
Larry I Smith wrote:
Khuong Dinh Pham wrote:
I have the contents of an image of type std::string. How can I make a
CxImage object with this type.

The parameters to CxImage is:

CxImage(byte* data, DWORD size)

Thx in advance


std::string.c_str() returns a 'const char *' (a pointer
to a string that can not be modified).

std::string.length() returns the data length
of the content.

Since CxImage() takes a 'byte *' rather than a
'const byte *', I would expect that it modifies
the memory pointed to by its first arg ('data').

If CxImage() does NOT modify the memory pointed to by its
first arg ('data'), and your image is in a std::string named
'myStr', then you might try something like this:

CxImage(myStr.c_str(), myStr.length());

You may have to add some casts to convert the first arg
to 'byte *' and the 2nd arg to 'DWORD'.

If CxImage() DOES modify the memory pointed to by its first
arg ('data'), then you could copy the string to a char buf:

try {
char * buf = new char[myStr.length()];
memcpy(buf, myStr.c_str(), myStr.length());
CxImage(buf, myStr.length());
delete[] buf;
}
catch (std::bad_alloc) {
std::cerr << "memory allocation failed\n";
}

Larry


I'd prefer:

std::vector<char> buf(myStr.begin(), myStr.end());
CxImage(&buf[0],buf.size());

No try/catch block necessary.
Aug 20 '05 #4
red floyd wrote:
Larry I Smith wrote:
Khuong Dinh Pham wrote:
I have the contents of an image of type std::string. How can I make a
CxImage object with this type.

The parameters to CxImage is:

CxImage(byte* data, DWORD size)

Thx in advance


std::string.c_str() returns a 'const char *' (a pointer
to a string that can not be modified).

std::string.length() returns the data length
of the content.

Since CxImage() takes a 'byte *' rather than a
'const byte *', I would expect that it modifies
the memory pointed to by its first arg ('data').

If CxImage() does NOT modify the memory pointed to by its
first arg ('data'), and your image is in a std::string named
'myStr', then you might try something like this:

CxImage(myStr.c_str(), myStr.length());

You may have to add some casts to convert the first arg
to 'byte *' and the 2nd arg to 'DWORD'.

If CxImage() DOES modify the memory pointed to by its first
arg ('data'), then you could copy the string to a char buf:

try {
char * buf = new char[myStr.length()];
memcpy(buf, myStr.c_str(), myStr.length());
CxImage(buf, myStr.length());
delete[] buf;
}
catch (std::bad_alloc) {
std::cerr << "memory allocation failed\n";
}

Larry


I'd prefer:

std::vector<char> buf(myStr.begin(), myStr.end());
CxImage(&buf[0],buf.size());

No try/catch block necessary.


I believe that the new/memcpy approach would be much faster
(a single alloc and copy versus possible multiple reallocs
and copies as the vector expands itself), and provides a means
to handle errors (try/catch) when the image is large (as many are).
I proposed that approach on the assumption that CxImage() is a 'C'
function rather than a C++ function (based on its use of 'byte'
and DWORD).

Both approaches should work.

Larry
Aug 21 '05 #5
First, to the OP: be sure to check whether CxImage _copies_ the specified
data or not, and if not, whether it takes over deallocation responsibility,
in which case you'll have to allocate the data correspondingly.

* Larry I Smith:
* red floyd:
* Larry I Smith:

If CxImage() DOES modify the memory pointed to by its first
arg ('data'), then you could copy the string to a char buf:

try {
char * buf = new char[myStr.length()];
memcpy(buf, myStr.c_str(), myStr.length());
CxImage(buf, myStr.length());
delete[] buf;
}
catch (std::bad_alloc) {
std::cerr << "memory allocation failed\n";
}
I'd prefer:

std::vector<char> buf(myStr.begin(), myStr.end());
CxImage(&buf[0],buf.size());

No try/catch block necessary.


I believe that the new/memcpy approach would be much faster


Nope.

Premature "optimization" is the root of all evil.

In this case the "optimization" makes for more complex and brittle code,
with no speedwise advantage (and no other advantage whatsoever).

(a single alloc and copy versus possible multiple reallocs
and copies as the vector expands itself),
It doesn't, in any acceptable quality implementation.

And if that should turn out to be a problem, simply declare the vector with
the required initial capacity.

and provides a means
to handle errors (try/catch) when the image is large (as many are).
Note that your code doesn't do that correctly: it doesn't deallocate, it
doesn't propagate the exception or abort, and it has side-effects (it should
also catch that exception by reference, but that's not very important).

All that can be fixed, but is the usual way things go when you do premature
optimization and choose low-level abstractions instead of higher level ones.

Don't.

I proposed that approach on the assumption that CxImage() is a 'C'
function rather than a C++ function (based on its use of 'byte'
and DWORD).


First, that wouldn't matter. Second, the OP said otherwise. Third, your
own code says otherwise. ;-)

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Aug 21 '05 #6
Alf P. Steinbach wrote:
First, to the OP: be sure to check whether CxImage _copies_ the specified
data or not, and if not, whether it takes over deallocation responsibility,
in which case you'll have to allocate the data correspondingly.

* Larry I Smith:
* red floyd:
* Larry I Smith:
If CxImage() DOES modify the memory pointed to by its first
arg ('data'), then you could copy the string to a char buf:

try {
char * buf = new char[myStr.length()];
memcpy(buf, myStr.c_str(), myStr.length());
CxImage(buf, myStr.length());
delete[] buf;
}
catch (std::bad_alloc) {
std::cerr << "memory allocation failed\n";
}
I'd prefer:

std::vector<char> buf(myStr.begin(), myStr.end());
CxImage(&buf[0],buf.size());

No try/catch block necessary. I believe that the new/memcpy approach would be much faster


Nope.

Premature "optimization" is the root of all evil.


Perhaps, but this particular one (new/memcpy vs vector) has proven
to be much faster in our corporate apps that have to compile/run
on many different platform/OS/compiler combinations, so I'm used to
using it automatically. In future responses here I'll try to
stick to generics.

In this case the "optimization" makes for more complex and brittle code,
with no speedwise advantage (and no other advantage whatsoever).

(a single alloc and copy versus possible multiple reallocs
and copies as the vector expands itself),
It doesn't, in any acceptable quality implementation.

And if that should turn out to be a problem, simply declare the vector with
the required initial capacity.

and provides a means
to handle errors (try/catch) when the image is large (as many are).


Note that your code doesn't do that correctly: it doesn't deallocate, it
doesn't propagate the exception or abort, and it has side-effects (it should
also catch that exception by reference, but that's not very important).


Sorry, it wasn't meant to be complete, merely an example snip.
It is based on the example code for 'f()' in section '6.2.6.2 Memory
Exhaustion' (page 129) from Stroustrup's "C++ Programming Language
Third Edition". I'll try to post more complete snips in the future.
All that can be fixed, but is the usual way things go when you do premature
optimization and choose low-level abstractions instead of higher level ones.

Don't.

I proposed that approach on the assumption that CxImage() is a 'C'
function rather than a C++ function (based on its use of 'byte'
and DWORD).


First, that wouldn't matter. Second, the OP said otherwise. Third, your
own code says otherwise. ;-)


I did not interpret the OP's use of the phrase "CxImage object" to
imply that CxImage was an object in the C++ sense; the
CxImage(byte *, DWORD) signature looked to me like an MS Windows
function call (a 3rd party lib API perhaps) - but then again,
I'm not that familiar with MS Windows... (:

Regards,
Larry
Aug 21 '05 #7
* Larry I Smith:

Perhaps, but this particular one (new/memcpy vs vector) has proven
to be much faster in our corporate apps that have to compile/run
on many different platform/OS/compiler combinations, so I'm used to
using it automatically.


Uhm ... extraordinary claims require extraordinary proofs... ;-)

Do you have some (preferentially small) example code the readers of this
thread could discuss & time?

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Aug 21 '05 #8
Alf P. Steinbach wrote:
* Larry I Smith:
Perhaps, but this particular one (new/memcpy vs vector) has proven
to be much faster in our corporate apps that have to compile/run
on many different platform/OS/compiler combinations, so I'm used to
using it automatically.


Uhm ... extraordinary claims require extraordinary proofs... ;-)

Do you have some (preferentially small) example code the readers of this
thread could discuss & time?


By force of habit I used an approach specified by my company's
Design Standards doc. Now you want me to prove that our
Corporate Engineering Council is correct in their design decisions.
As they say, "I just work here". The design standards to which we
must conform were written by folks far smarter than I to ensure
platform portability (win98/2k/xp; various versions of HP/UX, SunOS,
Solaris, Linux, etc, etc, etc). Those design standards forbid,
or restrict, our use of many common C++ features because they may
have problems (portability or performance) on one or more of the
supported platforms. Those who work here have no control over
these design standards, so further discussion on it is pointless.

As I stated in my earlier post, I'll make an effort in the
future to not impose those design limitations on code
snips I post here.

As far as the original OP's question (how to get the data from
a std::string into a seperate byte array), I put together a test
program (see below) which compares several approaches. Results
of running the program on WinXP and Linux are in the comments
at the top of the program. I know the code could be greatly
improved (better error handling, etc), but I've already spent
way too much time on this issue.

On Windows all but one of the vector approaches are 5 to 15
times slower than the new/memcpy approach. (I begin to see
why the Design Doc mandates the new/memcpy approach...)

On Linux the vector approaches are comparable to the new/memcpy
approach. (: (:

Excluding the new/memcpy approach, it seems that the following
approach using std::string.c_str() is the most portable
(i.e. matches the new/memcpy approach); although it does depend
on strict left-to-right argument evaluation - which I'm not
allowed to use, but others may be.

// 'str' is a std::string containing the input data.
const char * data;
vector<char> vec(data = str.c_str(), data + str.length());

Regards,
Larry

// vtest.cpp - test vector creation vs new/memcpy.
// 1) tests with std::string iterators as the input data
// 2) tests with std::string.c_str() as the input data
// 3) tests with a char buffer as the input data source.
// to compile:
// Windows: cl /EHsc vtest.cpp
// Linux: g++ -o vtest vtest.cpp

/*
* Sample output using MSVC v7 on WinXP,
* on a P4 2GHZ with 512MB RAM:
* Input test data size bytes is: 10485760
* Test creating a vector from std::string iterators:
* 0.046 seconds - (1x) new,memcpy()
* 0.655 seconds - (14x) vector(first,last)
* 0.686 seconds - (15x) vector(sz),copy()
* 0.608 seconds - (13x) vector(),reserve(sz),copy()
* Test creating a vector from std::string.c_str():
* 0.047 seconds - (1x) new,memcpy()
* 0.046 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()
* Test creating a vector from a char buffer:
* 0.047 seconds - (1x) new,memcpy()
* 0.047 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()
*
* Sample output using g++ v3.3.5 on SuSE Linux 9.3,
* on a P2 450MHZ with 384MB RAM:
* Input test data size bytes is: 10485760
* Test creating a vector from std::string iterators:
* 0.15 seconds - (1x) new,memcpy()
* 0.17 seconds - (1x) vector(first,last)
* 0.17 seconds - (1x) vector(sz),copy()
* 0.15 seconds - (1x) vector(),reserve(sz),copy()
* Test creating a vector from std::string.c_str():
* 0.15 seconds - (1x) new,memcpy()
* 0.15 seconds - (1x) vector(first,last)
* 0.17 seconds - (1x) vector(sz),copy()
* 0.16 seconds - (1x) vector(),reserve(sz),copy()
* Test creating a vector from a char buffer:
* 0.1 seconds - (1x) new,memcpy()
* 0.15 seconds - (1x) vector(first,last)
* 0.21 seconds - (2x) vector(sz),copy()
* 0.15 seconds - (1x) vector(),reserve(sz),copy()
*/

#include <iostream>
#include <cstdlib>
#include <ctime>
#include <string>
#include <vector>
#include <algorithm>

using namespace std;

// print the elapsed time in seconds
double elapsed(const string& txt, clock_t start, clock_t end,
double orig)
{
double et = static_cast<double>(end - start) / CLOCKS_PER_SEC;

cout << " "
<< et
<< " seconds - ";

if (orig > 0.0)
{
cout << " ("
<< static_cast<int>(et/orig + 0.5) << "x)";
}
else
{
cout << " (1x)";
}

cout << " " << txt << endl;

return et;
}

template <class InIt>
void testvec(string::size_type sz, const char * data,
InIt first, InIt last)
{
clock_t start, end;
double orig = 0.0;

// ------------------------------------
// time using new/memcpy to create a char buffer
// copy of the test data
{
char *buf = 0;

start = clock();

// allocate the memory for buf[].
// for this simple program, we just terminate if
// any exception is thrown.
try
{
buf = new char[sz];
}
catch(...)
{
cout << "buf = new[] failed" << endl;

return;
}

memcpy(buf, data, sz);

end = clock();

orig = elapsed("new,memcpy()", start, end, orig);

delete[] buf;
}

// ------------------------------------
// time creating a vector from the test data using
// the vector's constructor
{
start = clock();

vector<char> vec1(first, last);

end = clock();

elapsed("vector(first,last)" ,start, end, orig);
}

// ------------------------------------
// time creating a vector from the test data using a
// pre-sized vector and copy()
{
start = clock();

vector<char> vec2(sz);

copy(first, last, vec2.begin());

end = clock();

elapsed("vector(sz),copy()", start, end, orig);
}

// ------------------------------------
// time creating a vector from the test data using
// vector.reserve() and copy()
{
start = clock();

vector<char> vec3;

vec3.reserve(sz);

copy(first, last, vec3.begin());

end = clock();

elapsed("vector(),reserve(sz),copy()", start, end, orig);
}

return;
}

int main()
{
// we'll use 10MB as the test data size
string::size_type sz = (1024 * 1024 * 10);

cout << "Input test data size bytes is: " << sz << endl;

// test vector using string iterators to access the input data
{
cout << "Test creating a vector from"
<< " std::string iterators:" <<endl;

// make a test string holding 'sz' blanks
string str(sz, ' ');

testvec(sz, str.c_str(), str.begin(), str.end());
}

// test vector using string.c_str() as input data
{
cout << "Test creating a vector from"
<< " std::string.c_str():" <<endl;

const char * data;

// make a test string holding 'sz' blanks
string str(sz, ' ');

// WARNING: this assumes that left-to-right
// argument evaluation is guaranteed
// by the compiler.
testvec(sz, data = str.c_str(), data, data + sz);
}

// test vector using a char buffer as input data
{
cout << "Test creating a vector from a"
<< " char buffer:" <<endl;

try
{
// make a char buffer holding 'sz' blanks
char * data = new char[sz];

memset(data, ' ', sz);

testvec(sz, data, data, data + sz);

delete[] data;
}
catch(...)
{
cout << "data = new[] failed" << endl;
}
}

return 0;
}
Aug 23 '05 #9
* Larry I Smith:

As far as the original OP's question (how to get the data from
a std::string into a seperate byte array), I put together a test
program (see below) which compares several approaches. Results
of running the program on WinXP and Linux are in the comments
at the top of the program. I know the code could be greatly
improved (better error handling, etc), but I've already spent
way too much time on this issue.

On Windows all but one of the vector approaches are 5 to 15
times slower than the new/memcpy approach. (I begin to see
why the Design Doc mandates the new/memcpy approach...)
That's not what I get, except with a debug build, which should not be used
for timing.

/*
* Sample output using MSVC v7 on WinXP,
* on a P4 2GHZ with 512MB RAM:
* Input test data size bytes is: 10485760
* Test creating a vector from std::string iterators:
* 0.046 seconds - (1x) new,memcpy()
* 0.655 seconds - (14x) vector(first,last)
* 0.686 seconds - (15x) vector(sz),copy()
* 0.608 seconds - (13x) vector(),reserve(sz),copy()
* Test creating a vector from std::string.c_str():
* 0.047 seconds - (1x) new,memcpy()
* 0.046 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()
* Test creating a vector from a char buffer:
* 0.047 seconds - (1x) new,memcpy()
* 0.047 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()


For a slightly modified program (more test cases), MSVC 7.1 on a 1.8 GHz PC
with 256 MiB -- "should" be slower than yours but is way faster (did you
obtain the numbers above with a debug build?):

Input test data size bytes is: 10485760
Test creating a vector from std::string iterators:
0.031 seconds - (1x) new,memcpy() // This line doesn't use iterators.
0.141 seconds - (5x) vector(first,last) <--
0.11 seconds - (4x) vector(sz),copy()
0.235 seconds - (8x) vector(sz),assign()
0.047 seconds - (2x) vector(sz),memcopy()
0.11 seconds - (4x) vector(),reserve(sz),copy()
Test creating a vector from std::string.c_str():
0.031 seconds - (1x) new,memcpy()
0.032 seconds - (1x) vector(first,last) <--
0.126 seconds - (4x) vector(sz),copy()
0.141 seconds - (5x) vector(sz),assign()
0.062 seconds - (2x) vector(sz),memcopy()
0.094 seconds - (3x) vector(),reserve(sz),copy()
Test creating a vector from a char buffer:
0.031 seconds - (1x) new,memcpy()
0.047 seconds - (2x) vector(first,last) <--
0.11 seconds - (4x) vector(sz),copy()
0.125 seconds - (4x) vector(sz),assign()
0.063 seconds - (2x) vector(sz),memcopy()
0.094 seconds - (3x) vector(),reserve(sz),copy()

Of course the times vary somewhat. In a few cases the vector is faster:

Input test data size bytes is: 10485760
Test creating a vector from std::string iterators:
0.047 seconds - (1x) new,memcpy() // This line doesn't use iterators.
0.157 seconds - (3x) vector(first,last) <--
0.11 seconds - (2x) vector(sz),copy()
0.25 seconds - (5x) vector(sz),assign()
0.047 seconds - (1x) vector(sz),memcopy()
0.094 seconds - (2x) vector(),reserve(sz),copy()
Test creating a vector from std::string.c_str():
0.047 seconds - (1x) new,memcpy()
0.031 seconds - (1x) vector(first,last) <--
0.126 seconds - (3x) vector(sz),copy()
0.141 seconds - (3x) vector(sz),assign()
0.062 seconds - (1x) vector(sz),memcopy()
0.094 seconds - (2x) vector(),reserve(sz),copy()
Test creating a vector from a char buffer:
0.047 seconds - (1x) new,memcpy()
0.032 seconds - (1x) vector(first,last) <--
0.125 seconds - (3x) vector(sz),copy()
0.157 seconds - (3x) vector(sz),assign()
0.047 seconds - (1x) vector(sz),memcopy()
0.094 seconds - (2x) vector(),reserve(sz),copy()

With g++ 3.4.2 on the same machine I get essentially the same results,
except one run where new+memcpy fell down to 0.016 secs (not reproducable):

Input test data size bytes is: 10485760
Test creating a vector from std::string iterators:
0.031 seconds - (1x) new,memcpy() // This line doesn't use iterators.
0.032 seconds - (1x) vector(first,last) <--
0.062 seconds - (2x) vector(sz),copy()
0.063 seconds - (2x) vector(sz),assign()
0.047 seconds - (2x) vector(sz),memcopy()
0.031 seconds - (1x) vector(),reserve(sz),copy()
Test creating a vector from std::string.c_str():
0.031 seconds - (1x) new,memcpy()
0.032 seconds - (1x) vector(first,last) <--
0.047 seconds - (2x) vector(sz),copy()
0.047 seconds - (2x) vector(sz),assign()
0.031 seconds - (1x) vector(sz),memcopy()
0.031 seconds - (1x) vector(),reserve(sz),copy()
Test creating a vector from a char buffer:
0.032 seconds - (1x) new,memcpy()
0.047 seconds - (1x) vector(first,last) <--
0.047 seconds - (1x) vector(sz),copy()
0.062 seconds - (2x) vector(sz),assign()
0.047 seconds - (1x) vector(sz),memcopy()
0.047 seconds - (1x) vector(),reserve(sz),copy()

It seems that for those two compilers, MSVC 7.1 and g++ 3.4.2,

void f( std::string const& s )
{
char const* const data = s.c_str();
std::vector v( data, data + s.length() );
//...
}

is nearly always as fast as new+memcpy, sometimes faster, and _much_ safer.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Aug 23 '05 #10
In message <iVwOe.10620$g47.1857@trnddc07>, Larry I Smith
<la***********@verizon.net> writes
Alf P. Steinbach wrote:
* Larry I Smith:
Perhaps, but this particular one (new/memcpy vs vector) has proven
to be much faster in our corporate apps that have to compile/run
on many different platform/OS/compiler combinations, so I'm used to
using it automatically.
Uhm ... extraordinary claims require extraordinary proofs... ;-)

Do you have some (preferentially small) example code the readers of this
thread could discuss & time?


By force of habit I used an approach specified by my company's
Design Standards doc. Now you want me to prove that our
Corporate Engineering Council is correct in their design decisions.


[big snip]

vector<char> vec3;

vec3.reserve(sz);

copy(first, last, vec3.begin());


I hope they didn't mandate *that*! :-(

Incidentally, I don't think this program proves anything at all. By
running the tests consecutively, you're making the later ones dependent
on how the earlier ones mangled free store, so they're not independent.

--
Richard Herring
Aug 23 '05 #11
Alf P. Steinbach wrote:
* Larry I Smith:
As far as the original OP's question (how to get the data from
a std::string into a seperate byte array), I put together a test
program (see below) which compares several approaches. Results
of running the program on WinXP and Linux are in the comments
at the top of the program. I know the code could be greatly
improved (better error handling, etc), but I've already spent
way too much time on this issue.

On Windows all but one of the vector approaches are 5 to 15
times slower than the new/memcpy approach. (I begin to see
why the Design Doc mandates the new/memcpy approach...)
That's not what I get, except with a debug build, which should not be used
for timing.


As stated in the code comments, here are the
compile comands I used on Windows and Linux:

// to compile:
// Windows: cl /EHsc vtest.cpp
// Linux: g++ -o vtest vtest.cpp

No "debug build" was involved.

/*
* Sample output using MSVC v7 on WinXP,
* on a P4 2GHZ with 512MB RAM:
* Input test data size bytes is: 10485760
* Test creating a vector from std::string iterators:
* 0.046 seconds - (1x) new,memcpy()
* 0.655 seconds - (14x) vector(first,last)
* 0.686 seconds - (15x) vector(sz),copy()
* 0.608 seconds - (13x) vector(),reserve(sz),copy()
* Test creating a vector from std::string.c_str():
* 0.047 seconds - (1x) new,memcpy()
* 0.046 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()
* Test creating a vector from a char buffer:
* 0.047 seconds - (1x) new,memcpy()
* 0.047 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()


For a slightly modified program (more test cases), MSVC 7.1 on a 1.8 GHz PC
with 256 MiB -- "should" be slower than yours but is way faster (did you
obtain the numbers above with a debug build?):

No "debug build" was used. See my comment above.
The Windows pc is a small Dell desktop; the model escapes
me at the moment (it's at work and I'm at home). It's specs
mentioned in my earlier post were copied from the "Properties"
pop-up of its "My Computer" desktop icon.

Input test data size bytes is: 10485760
Test creating a vector from std::string iterators:
0.031 seconds - (1x) new,memcpy() // This line doesn't use iterators.
0.141 seconds - (5x) vector(first,last) <--
0.11 seconds - (4x) vector(sz),copy()
0.235 seconds - (8x) vector(sz),assign()
0.047 seconds - (2x) vector(sz),memcopy()
0.11 seconds - (4x) vector(),reserve(sz),copy()
Test creating a vector from std::string.c_str():
0.031 seconds - (1x) new,memcpy()
0.032 seconds - (1x) vector(first,last) <--
0.126 seconds - (4x) vector(sz),copy()
0.141 seconds - (5x) vector(sz),assign()
0.062 seconds - (2x) vector(sz),memcopy()
0.094 seconds - (3x) vector(),reserve(sz),copy()
Test creating a vector from a char buffer:
0.031 seconds - (1x) new,memcpy()
0.047 seconds - (2x) vector(first,last) <--
0.11 seconds - (4x) vector(sz),copy()
0.125 seconds - (4x) vector(sz),assign()
0.063 seconds - (2x) vector(sz),memcopy()
0.094 seconds - (3x) vector(),reserve(sz),copy()

Of course the times vary somewhat. In a few cases the vector is faster:

Input test data size bytes is: 10485760
Test creating a vector from std::string iterators:
0.047 seconds - (1x) new,memcpy() // This line doesn't use iterators.
0.157 seconds - (3x) vector(first,last) <--
0.11 seconds - (2x) vector(sz),copy()
0.25 seconds - (5x) vector(sz),assign()
0.047 seconds - (1x) vector(sz),memcopy()
0.094 seconds - (2x) vector(),reserve(sz),copy()
Test creating a vector from std::string.c_str():
0.047 seconds - (1x) new,memcpy()
0.031 seconds - (1x) vector(first,last) <--
0.126 seconds - (3x) vector(sz),copy()
0.141 seconds - (3x) vector(sz),assign()
0.062 seconds - (1x) vector(sz),memcopy()
0.094 seconds - (2x) vector(),reserve(sz),copy()
Test creating a vector from a char buffer:
0.047 seconds - (1x) new,memcpy()
0.032 seconds - (1x) vector(first,last) <--
0.125 seconds - (3x) vector(sz),copy()
0.157 seconds - (3x) vector(sz),assign()
0.047 seconds - (1x) vector(sz),memcopy()
0.094 seconds - (2x) vector(),reserve(sz),copy()

With g++ 3.4.2 on the same machine I get essentially the same results,
except one run where new+memcpy fell down to 0.016 secs (not reproducable):

Input test data size bytes is: 10485760
Test creating a vector from std::string iterators:
0.031 seconds - (1x) new,memcpy() // This line doesn't use iterators.
0.032 seconds - (1x) vector(first,last) <--
0.062 seconds - (2x) vector(sz),copy()
0.063 seconds - (2x) vector(sz),assign()
0.047 seconds - (2x) vector(sz),memcopy()
0.031 seconds - (1x) vector(),reserve(sz),copy()
Test creating a vector from std::string.c_str():
0.031 seconds - (1x) new,memcpy()
0.032 seconds - (1x) vector(first,last) <--
0.047 seconds - (2x) vector(sz),copy()
0.047 seconds - (2x) vector(sz),assign()
0.031 seconds - (1x) vector(sz),memcopy()
0.031 seconds - (1x) vector(),reserve(sz),copy()
Test creating a vector from a char buffer:
0.032 seconds - (1x) new,memcpy()
0.047 seconds - (1x) vector(first,last) <--
0.047 seconds - (1x) vector(sz),copy()
0.062 seconds - (2x) vector(sz),assign()
0.047 seconds - (1x) vector(sz),memcopy()
0.047 seconds - (1x) vector(),reserve(sz),copy()

It seems that for those two compilers, MSVC 7.1 and g++ 3.4.2,

void f( std::string const& s )
{
char const* const data = s.c_str();
std::vector v( data, data + s.length() );
//...
}

is nearly always as fast as new+memcpy, sometimes faster, and _much_ safer.

Aug 23 '05 #12
Richard Herring wrote:
In message <iVwOe.10620$g47.1857@trnddc07>, Larry I Smith
<la***********@verizon.net> writes
Alf P. Steinbach wrote:
* Larry I Smith:
Perhaps, but this particular one (new/memcpy vs vector) has proven
to be much faster in our corporate apps that have to compile/run
on many different platform/OS/compiler combinations, so I'm used to
using it automatically.

Uhm ... extraordinary claims require extraordinary proofs... ;-)

Do you have some (preferentially small) example code the readers of this
thread could discuss & time?

By force of habit I used an approach specified by my company's
Design Standards doc. Now you want me to prove that our
Corporate Engineering Council is correct in their design decisions.


[big snip]

vector<char> vec3;

vec3.reserve(sz);

copy(first, last, vec3.begin());


I hope they didn't mandate *that*! :-(

No, they mandate the new/memcpy approach (always).


Incidentally, I don't think this program proves anything at all. By
running the tests consecutively, you're making the later ones dependent
on how the earlier ones mangled free store, so they're not independent.


Ok. When I have time to write ten seperate programs, I'll look
into it.

Regards,
Larry
Aug 23 '05 #13
In message <R9OOe.10891$_f.2592@trnddc03>, Larry I Smith
<la***********@verizon.net> writes
Richard Herring wrote:
In message <iVwOe.10620$g47.1857@trnddc07>, Larry I Smith
<la***********@verizon.net> writes
Alf P. Steinbach wrote:
* Larry I Smith:
> Perhaps, but this particular one (new/memcpy vs vector) has proven
> to be much faster in our corporate apps that have to compile/run
> on many different platform/OS/compiler combinations, so I'm used to
> using it automatically.

Uhm ... extraordinary claims require extraordinary proofs... ;-)

Do you have some (preferentially small) example code the readers of this
thread could discuss & time?
By force of habit I used an approach specified by my company's
Design Standards doc. Now you want me to prove that our
Corporate Engineering Council is correct in their design decisions.
[big snip]

vector<char> vec3;

vec3.reserve(sz);

copy(first, last, vec3.begin());


I hope they didn't mandate *that*! :-(

No, they mandate the new/memcpy approach


I fear you may have missed the point. In case anyone else did, the above
is UB. reserve(sz) or not, vec3.size() is 0.
(always).
And what do they mandate for dynamic arrays of rule-of-3 objects?

Incidentally, I don't think this program proves anything at all. By
running the tests consecutively, you're making the later ones dependent
on how the earlier ones mangled free store, so they're not independent.


Ok. When I have time to write ten seperate programs, I'll look
into it.

Regards,
Larry


--
Richard Herring
Aug 24 '05 #14
Richard Herring wrote:
In message <R9OOe.10891$_f.2592@trnddc03>, Larry I Smith
<la***********@verizon.net> writes
Richard Herring wrote:
In message <iVwOe.10620$g47.1857@trnddc07>, Larry I Smith
<la***********@verizon.net> writes
Alf P. Steinbach wrote:
> * Larry I Smith:
>> Perhaps, but this particular one (new/memcpy vs vector) has proven
>> to be much faster in our corporate apps that have to compile/run
>> on many different platform/OS/compiler combinations, so I'm used to
>> using it automatically.
>
> Uhm ... extraordinary claims require extraordinary proofs... ;-)
>
> Do you have some (preferentially small) example code the readers of
> this
> thread could discuss & time?
>

By force of habit I used an approach specified by my company's
Design Standards doc. Now you want me to prove that our
Corporate Engineering Council is correct in their design decisions.

[big snip]
vector<char> vec3;

vec3.reserve(sz);

copy(first, last, vec3.begin());

I hope they didn't mandate *that*! :-(

No, they mandate the new/memcpy approach


I fear you may have missed the point. In case anyone else did, the above
is UB. reserve(sz) or not, vec3.size() is 0.

Ahhh yes, that's what happens when one codes after midnight...

You are correct, the above code should be this:

vector<char> vec3;

vec3.assign(first, last);

(always).


And what do they mandate for dynamic arrays of rule-of-3 objects?


The new/memcpy rule applies only to native types (char, int, double,
etc).

Regards,
Larry
Aug 24 '05 #15
Richard Herring wrote:

[snip]

Incidentally, I don't think this program proves anything at all. By
running the tests consecutively, you're making the later ones dependent
on how the earlier ones mangled free store, so they're not independent.


I split it into multiple programs and tested on
Linux and Windows.

The times remained within 1% to 2% of those reported
by the original all-in-one program. A few times
went up (from 0.5% to 1.5%), a few went down (from
0.5% to 2%), and a few remained the same.

So, the original all-in-one program is just fine
for testing purposes.

Regards,
Larry

Aug 30 '05 #16
Larry I Smith wrote:
Alf P. Steinbach wrote:
* Larry I Smith:
Perhaps, but this particular one (new/memcpy vs vector) has proven
to be much faster in our corporate apps that have to compile/run
on many different platform/OS/compiler combinations, so I'm used to
using it automatically. Uhm ... extraordinary claims require extraordinary proofs... ;-)

Do you have some (preferentially small) example code the readers of this
thread could discuss & time?


By force of habit I used an approach specified by my company's
Design Standards doc. Now you want me to prove that our
Corporate Engineering Council is correct in their design decisions.
As they say, "I just work here". The design standards to which we
must conform were written by folks far smarter than I to ensure
platform portability (win98/2k/xp; various versions of HP/UX, SunOS,
Solaris, Linux, etc, etc, etc). Those design standards forbid,
or restrict, our use of many common C++ features because they may
have problems (portability or performance) on one or more of the
supported platforms. Those who work here have no control over
these design standards, so further discussion on it is pointless.

As I stated in my earlier post, I'll make an effort in the
future to not impose those design limitations on code
snips I post here.

As far as the original OP's question (how to get the data from
a std::string into a seperate byte array), I put together a test
program (see below) which compares several approaches. Results
of running the program on WinXP and Linux are in the comments
at the top of the program. I know the code could be greatly
improved (better error handling, etc), but I've already spent
way too much time on this issue.

On Windows all but one of the vector approaches are 5 to 15
times slower than the new/memcpy approach. (I begin to see
why the Design Doc mandates the new/memcpy approach...)

On Linux the vector approaches are comparable to the new/memcpy
approach. (: (:

Excluding the new/memcpy approach, it seems that the following
approach using std::string.c_str() is the most portable
(i.e. matches the new/memcpy approach); although it does depend
on strict left-to-right argument evaluation - which I'm not
allowed to use, but others may be.

// 'str' is a std::string containing the input data.
const char * data;
vector<char> vec(data = str.c_str(), data + str.length());

Regards,
Larry

// vtest.cpp - test vector creation vs new/memcpy.
// 1) tests with std::string iterators as the input data
// 2) tests with std::string.c_str() as the input data
// 3) tests with a char buffer as the input data source.
// to compile:
// Windows: cl /EHsc vtest.cpp
// Linux: g++ -o vtest vtest.cpp

/*
* Sample output using MSVC v7 on WinXP,
* on a P4 2GHZ with 512MB RAM:

Correction to the above two lines:

Sample output using MS Visual Studio .NET 2003 on Win-XP (SP2),
on a Dell Optiplex GX240 (P4 2GHZ) with 512MB of RAM:

* Input test data size bytes is: 10485760
* Test creating a vector from std::string iterators:
* 0.046 seconds - (1x) new,memcpy()
* 0.655 seconds - (14x) vector(first,last)
* 0.686 seconds - (15x) vector(sz),copy()
* 0.608 seconds - (13x) vector(),reserve(sz),copy()
* Test creating a vector from std::string.c_str():
* 0.047 seconds - (1x) new,memcpy()
* 0.046 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()
* Test creating a vector from a char buffer:
* 0.047 seconds - (1x) new,memcpy()
* 0.047 seconds - (1x) vector(first,last)
* 0.343 seconds - (7x) vector(sz),copy()
* 0.249 seconds - (5x) vector(),reserve(sz),copy()
*
* Sample output using g++ v3.3.5 on SuSE Linux 9.3,
* on a P2 450MHZ with 384MB RAM:

Correction to the above line:

on a Gateway E-4200 (P2 450MHZ) with 384MB of RAM:

* Input test data size bytes is: 10485760
* Test creating a vector from std::string iterators:
* 0.15 seconds - (1x) new,memcpy()
* 0.17 seconds - (1x) vector(first,last)
* 0.17 seconds - (1x) vector(sz),copy()
* 0.15 seconds - (1x) vector(),reserve(sz),copy()
* Test creating a vector from std::string.c_str():
* 0.15 seconds - (1x) new,memcpy()
* 0.15 seconds - (1x) vector(first,last)
* 0.17 seconds - (1x) vector(sz),copy()
* 0.16 seconds - (1x) vector(),reserve(sz),copy()
* Test creating a vector from a char buffer:
* 0.1 seconds - (1x) new,memcpy()
* 0.15 seconds - (1x) vector(first,last)
* 0.21 seconds - (2x) vector(sz),copy()
* 0.15 seconds - (1x) vector(),reserve(sz),copy()
*/

#include <iostream>
#include <cstdlib>
#include <ctime>
#include <string>
#include <vector>
#include <algorithm>

using namespace std;

// print the elapsed time in seconds
double elapsed(const string& txt, clock_t start, clock_t end,
double orig)
{
double et = static_cast<double>(end - start) / CLOCKS_PER_SEC;

cout << " "
<< et
<< " seconds - ";

if (orig > 0.0)
{
cout << " ("
<< static_cast<int>(et/orig + 0.5) << "x)";
}
else
{
cout << " (1x)";
}

cout << " " << txt << endl;

return et;
}

template <class InIt>
void testvec(string::size_type sz, const char * data,
InIt first, InIt last)
{
clock_t start, end;
double orig = 0.0;

// ------------------------------------
// time using new/memcpy to create a char buffer
// copy of the test data
{
char *buf = 0;

start = clock();

// allocate the memory for buf[].
// for this simple program, we just terminate if
// any exception is thrown.
try
{
buf = new char[sz];
}
catch(...)
{
cout << "buf = new[] failed" << endl;

return;
}

memcpy(buf, data, sz);

end = clock();

orig = elapsed("new,memcpy()", start, end, orig);

delete[] buf;
}

// ------------------------------------
// time creating a vector from the test data using
// the vector's constructor
{
start = clock();

vector<char> vec1(first, last);

end = clock();

elapsed("vector(first,last)" ,start, end, orig);
}

// ------------------------------------
// time creating a vector from the test data using a
// pre-sized vector and copy()
{
start = clock();

vector<char> vec2(sz);

copy(first, last, vec2.begin());

end = clock();

elapsed("vector(sz),copy()", start, end, orig);
}

// ------------------------------------
// time creating a vector from the test data using
// vector.reserve() and copy()
{
start = clock();

vector<char> vec3;

vec3.reserve(sz);

//copy(first, last, vec3.begin());

As Richard Herring pointed out, the above copy() is incorrect.
It should be:

vec3.assign(first, last);


end = clock();

//elapsed("vector(),reserve(sz),copy()", start, end, orig);
elapsed("vector(),reserve(sz),assign()", start, end, orig);
}

return;
}

int main()
{
// we'll use 10MB as the test data size
string::size_type sz = (1024 * 1024 * 10);

cout << "Input test data size bytes is: " << sz << endl;

// test vector using string iterators to access the input data
{
cout << "Test creating a vector from"
<< " std::string iterators:" <<endl;

// make a test string holding 'sz' blanks
string str(sz, ' ');

testvec(sz, str.c_str(), str.begin(), str.end());
}

// test vector using string.c_str() as input data
{
cout << "Test creating a vector from"
<< " std::string.c_str():" <<endl;

const char * data;

// make a test string holding 'sz' blanks
string str(sz, ' ');

// WARNING: this assumes that left-to-right
// argument evaluation is guaranteed
// by the compiler.
testvec(sz, data = str.c_str(), data, data + sz);
}

// test vector using a char buffer as input data
{
cout << "Test creating a vector from a"
<< " char buffer:" <<endl;

try
{
// make a char buffer holding 'sz' blanks
char * data = new char[sz];

memset(data, ' ', sz);

testvec(sz, data, data, data + sz);

delete[] data;
}
catch(...)
{
cout << "data = new[] failed" << endl;
}
}

return 0;
}

Aug 30 '05 #17

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Nathan | last post by:
Is there a way to convert a string to a CipherMessage? I am calling a function that decrypts a CipherMessage and returns the value. The only problem is when I want to use an encrypted value stored...
5
by: Karthik | last post by:
Hello, How can I convert a BSTR data type to std::string??? Thanks a much! Karthik
32
by: Wolfgang Draxinger | last post by:
I understand that it is perfectly possible to store UTF-8 strings in a std::string, however doing so can cause some implicaions. E.g. you can't count the amount of characters by length() | size()....
8
by: ppcdev | last post by:
Here's what I try : LPCTSTR tst = (LPCTSTR) (LPCWSTR) Marshal::StringToHGlobalUni(str); c:\MyNetPrj\Prj0001\stunt.cpp(244): error C2440: 'type cast' : cannot convert from 'System::IntPtr' to...
10
by: sposes | last post by:
Im very much a newbie but perhaps somehone can help me. Ive been searching for a way to convert a std::string to a unsigned char* The situation is I have a function that wants a unsigned char*...
11
by: Sudzzz | last post by:
Hi, I'm trying to convert a string something like this "{201,23,240,56,23,45,34,23}" into an array in C++ Please help. Thanks, Sudzzz
4
by: barnum | last post by:
Hi, I have a std::string which I know is UTF-8 encoded. How can I make a System::String^ from it? I tried UTF8Encoding class, but it wants a Byte array, and I don't know how to get that from a...
25
by: Bala2508 | last post by:
Hi, I have a C++ application that extensively uses std::string and std::ostringstream in somewhat similar manner as below std::string msgHeader; msgHeader = "<"; msgHeader += a; msgHeader...
12
by: Peter | last post by:
Trying to convert string to byte array. the following code returns byte array of {107, 62, 194, 139, 64} how can I convert this string to a byte array of {107, 62, 139, 65} ...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.