memory pool?

[Uli Kunkel]

I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that returns
1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Anyway thanks in advance for any suggestions.

[Pascal J. Bourguignon]

Uli Kunkel <ge*******@yahoo.comwrites:

I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that
returns 1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Not on comp.lang.c++. For CString, you should ask on comp.lang.c.

Anyway thanks in advance for any suggestions.

typedef unsigned char byte;
const int KByte=1024;
const int MByte=1024*KByte;
const int few=6;
std::vector<byteimage(few*MByte,0);

Then you can receive copy small buffers into it:
std::vector<byte>::iterator pos=image.begin();
std::vector<bytebuffer(1*KByte);

while(receive(&buffer)){
pos=std::copy(buffer.begin(),buffer.end(),pos);
}
--
__Pascal Bourguignon__

[Juan Antonio Zaratiegui Vallecillo]

Uli Kunkel escribió:

I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that returns
1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Anyway thanks in advance for any suggestions.

Use a vector, taking advantage from its reserve() function.

#include <vector>

(...)

std::vector<unsigned charmyPhoto;
myPhoto.reserve(1024*1024);

for (size_t part=0; part !=1024;++part) {
<read image data into &myPhoto(part*1024) >
}
(...)
<you now have your photo in myPhoto.size() sequential bytes at
&myPhoto[0]
Just customize such piece for your exact needs, substsituting all magic
constants with nice real constants.
This code skeleton takes as granted that char size is one byte, you may
need to correct the code it it is otherwise

best regards,

Zara

[Hendrik Schober]

Juan Antonio Zaratiegui Vallecillo wrote:

Uli Kunkel escribió:

>I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that returns
1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Anyway thanks in advance for any suggestions.

Use a vector, taking advantage from its reserve() function.

#include <vector>

(...)

std::vector<unsigned charmyPhoto;
myPhoto.reserve(1024*1024);

for (size_t part=0; part !=1024;++part) {
<read image data into &myPhoto(part*1024) >

This should read "append data to 'myPhoto'". Remember, reserved
memory is just that: reserved memory. Specifically, it's not
objects you could overwrite.

[...]
Zara

Schobi

[Uli Kunkel]

Juan Antonio Zaratiegui Vallecillo wrote:

Uli Kunkel escribió:

>I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that
returns 1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Anyway thanks in advance for any suggestions.

Use a vector, taking advantage from its reserve() function.

#include <vector>

(...)

std::vector<unsigned charmyPhoto;
myPhoto.reserve(1024*1024);

for (size_t part=0; part !=1024;++part) {
<read image data into &myPhoto(part*1024) >
}
(...)
<you now have your photo in myPhoto.size() sequential bytes at
&myPhoto[0]
Just customize such piece for your exact needs, substsituting all magic
constants with nice real constants.
This code skeleton takes as granted that char size is one byte, you may
need to correct the code it it is otherwise

best regards,

Zara

Thanks for the answers.
I'm wondering if I could use reserve to allocate a few MB and then if
the size is not enough that the vector grows automatically?
I read that vector resizing is very costly so I presume that reserving
just 1KB is not enough.

[peter koch]

On 10 Nov., 12:49, Uli Kunkel <[e-mail]:

Juan Antonio Zaratiegui Vallecillo wrote:

Uli Kunkel escribió:

I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that
returns 1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Anyway thanks in advance for any suggestions.

Use a vector, taking advantage from its reserve() function.

#include <vector>

(...)

* * std::vector<unsigned charmyPhoto;
* * myPhoto.reserve(1024*1024);

* * for (size_t part=0; part !=1024;++part) {
* * * *<read image data into &myPhoto(part*1024) >
* * }
(...)
* * <you now have your photo in myPhoto.size() sequential bytes at
&myPhoto[0]

Just customize such piece for your exact needs, substsituting all magic
constants with nice real constants.
This code skeleton takes as granted that char size is one byte, you may
need to correct the code it it is otherwise

best regards,

Zara

Thanks for the answers.
I'm wondering if I could use reserve to allocate a few MB and then if
the size is not enough that the vector grows automatically?
I read that vector resizing is very costly so I presume that reserving
just 1KB is not enough.

std::vector resizes autmatically. As an alternative, you could also
use std::deque which memorywise is more optimal with regard to
reallocation but overall is slightly slower (does more allocations)
than a std::vector.

/Peter

[Hendrik Schober]

Uli Kunkel wrote:

[...]
I'm wondering if I could use reserve to allocate a few MB and then if
the size is not enough that the vector grows automatically?
I read that vector resizing is very costly so I presume that reserving
just 1KB is not enough.

There's a difference between a vector's size (member functions
'size()'/'resize()') and its capacity ('capacity()'/'reserve()').
The former is related to the number of actual objects in the
vector, the latter to the number of objects the vector is able
to store without having to reallocate.
When you tell a vector to reserve memory for an amount of
elements, it allocates the memory needed (or more), but does
not create the objects. When you tell it to resize to a
specific amount of objects, it will actually contain this
amount of constructed objects (while its capacity might be
higher). The capacity concept allows to write code adding
objects to a vector in the most obvious way (keep appending)
and, if performance or memory fragmentation etc. are issues,
reserve before-hand to reduce or eliminate reallocations. It's
only there for performance-reasons and (besides performance)
has no influence on your algorithm.

So when you know how many objects you will need before-hand,
reserve that amount and then keep appending incoming objects.
If not, strategies depend on your specific problem.

Schobi

[Uli Kunkel]

Pascal J. Bourguignon wrote:

Uli Kunkel <ge*******@yahoo.comwrites:

>I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that
returns 1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Not on comp.lang.c++. For CString, you should ask on comp.lang.c.

>Anyway thanks in advance for any suggestions.

typedef unsigned char byte;
const int KByte=1024;
const int MByte=1024*KByte;
const int few=6;
std::vector<byteimage(few*MByte,0);

Then you can receive copy small buffers into it:
std::vector<byte>::iterator pos=image.begin();
std::vector<bytebuffer(1*KByte);

while(receive(&buffer)){
pos=std::copy(buffer.begin(),buffer.end(),pos);
}

I wrote some code and I have a problem that I can't open the picture.
Here is the simplified code:
----------------------------
//This part of the code pushes back every character to myPhoto vector

for(unsigned long i=0; i < Length; i++)
{
myPhoto.push_back(srcData[i]);
}
if(complete) //Write myPhoto vector to a file
{
ofstream outf("C:\\test_0.jpg");
for ( pos=myPhoto.begin()
; pos!=myPhoto.end()
; ++pos)
{
outf << *pos;
}
outf.close();
}

//This part of the code append every chunk to the file
//It's for testing only
ofstream myFile ("C:\\test_1.jpg", ios::out | ios::binary | ios::app);
myFile.write ((char *)pProgress->pbData, pProgress->lLength);
myFile.close();
-----------------------------
What happens is that the first photo has 8 or 9 KB more than the second
one and it can't be opened.
The second picture is ok but I don't want to do it this way.
Also I opened them and they seem the samo at the begining but the first
one is larger..

I tried 2 different ways to writing them but with the same effect.

Any suggestions?

[Rolf Magnus]

Uli Kunkel wrote:

I wrote some code and I have a problem that I can't open the picture.
Here is the simplified code:
----------------------------
//This part of the code pushes back every character to myPhoto vector

for(unsigned long i=0; i < Length; i++)
{
myPhoto.push_back(srcData[i]);
}
if(complete) //Write myPhoto vector to a file
{
ofstream outf("C:\\test_0.jpg");
for ( pos=myPhoto.begin()
; pos!=myPhoto.end()
; ++pos)
{
outf << *pos;
}
outf.close();
}

//This part of the code append every chunk to the file
//It's for testing only
ofstream myFile ("C:\\test_1.jpg", ios::out | ios::binary | ios::app);
myFile.write ((char *)pProgress->pbData, pProgress->lLength);
myFile.close();
-----------------------------
What happens is that the first photo has 8 or 9 KB more than the second
one and it can't be opened.

The first file is opened in text mode, while the second is opened in binary
mode. Depending on the platform you're using, that means that the first file
might get altered in some way while being written.

[Uli Kunkel]

Rolf Magnus wrote:

Uli Kunkel wrote:

>I wrote some code and I have a problem that I can't open the picture.
Here is the simplified code:
----------------------------
//This part of the code pushes back every character to myPhoto vector

for(unsigned long i=0; i < Length; i++)
{
myPhoto.push_back(srcData[i]);
}
if(complete) //Write myPhoto vector to a file
{
ofstream outf("C:\\test_0.jpg");
for ( pos=myPhoto.begin()
; pos!=myPhoto.end()
; ++pos)
{
outf << *pos;
}
outf.close();
}

//This part of the code append every chunk to the file
//It's for testing only
ofstream myFile ("C:\\test_1.jpg", ios::out | ios::binary | ios::app);
myFile.write ((char *)pProgress->pbData, pProgress->lLength);
myFile.close();
-----------------------------
What happens is that the first photo has 8 or 9 KB more than the second
one and it can't be opened.

The first file is opened in text mode, while the second is opened in binary
mode. Depending on the platform you're using, that means that the first file
might get altered in some way while being written.

That was the problem.
I didn't pay much attention on that part..
Thanks for the help.

[Juan Antonio Zaratiegui Vallecillo]

Juan Antonio Zaratiegui Vallecillo escribió:

Uli Kunkel escribió:

>I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that
returns 1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Anyway thanks in advance for any suggestions.

Use a vector, taking advantage from its reserve() function.

#include <vector>

(...)

std::vector<unsigned charmyPhoto;
myPhoto.reserve(1024*1024);

for (size_t part=0; part !=1024;++part) {
<read image data into &myPhoto(part*1024) >

ThankÅ› to all for the comments.

I really meant to say
<append imaged data into myPhoto.end()>
But it was too early in the morning

}
(...)
<you now have your photo in myPhoto.size() sequential bytes at
&myPhoto[0]

(...)

And I keep on favouring the use of reserve, as it will be surely faster
than letting the container resize as needed, because we already know the
final size.

Best regards,

Zara

PS: Any error in this new message,, it is because it is too late in the
afternoon ;-)

[Uli Kunkel]

Juan Antonio Zaratiegui Vallecillo wrote:

Juan Antonio Zaratiegui Vallecillo escribió:

>Uli Kunkel escribió:

>>I have a situation where I'm getting a picture from a camera in small
chunks.
So the photo is a few MBytes and I have a callback function that
returns 1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one variable?
I heard something about using memory pools for this but I only need
something simple for this occasion.
Could I use CString?

Anyway thanks in advance for any suggestions.

Use a vector, taking advantage from its reserve() function.

#include <vector>

(...)

std::vector<unsigned charmyPhoto;
myPhoto.reserve(1024*1024);

for (size_t part=0; part !=1024;++part) {
<read image data into &myPhoto(part*1024) >

ThankÅ› to all for the comments.

I really meant to say
<append imaged data into myPhoto.end()>
But it was too early in the morning

> }
(...)
<you now have your photo in myPhoto.size() sequential bytes at
&myPhoto[0]

(...)

And I keep on favouring the use of reserve, as it will be surely faster
than letting the container resize as needed, because we already know the
final size.

Best regards,

Zara

PS: Any error in this new message,, it is because it is too late in the
afternoon ;-)

In the end I did something like this:
First I reserved 2 MB. Then I check the capacyty and reserve a 1KB more
if needed:

if(myPhoto.size() + pProgress->lLength < myPhoto.capacity())
myPhoto.reserve(1024*1024);

Thank you all for your help.

[Default User]

Pascal J. Bourguignon wrote:

Uli Kunkel <ge*******@yahoo.comwrites:

I have a situation where I'm getting a picture from a camera in
small chunks.
So the photo is a few MBytes and I have a callback function that
returns 1KB until it finishes transfering the picture.
Any suggestions or examples how should I put a picture in one
variable? I heard something about using memory pools for this but
I only need something simple for this occasion.
Could I use CString?

Not on comp.lang.c++. For CString, you should ask on comp.lang.c.

No, he should not. CString is not a C construct, but rather an MFC
class.


Brian

[James Kanze]

On Nov 10, 12:49*pm, Uli Kunkel <[e-mail]:

Juan Antonio Zaratiegui Vallecillo wrote:
I'm wondering if I could use reserve to allocate a few MB and
then if the size is not enough that the vector grows
automatically? I read that vector resizing is very costly so
I presume that reserving just 1KB is not enough.

What you read was wrong. Resizing isn't that costly for
primitive object types (like unsigned char). There are other
reasons you might want to avoid it, but they probably don't
apply here.

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, [phone]

[Uli Kunkel]

James Kanze wrote:

On Nov 10, 12:49 pm, Uli Kunkel <[e-mail]:

>Juan Antonio Zaratiegui Vallecillo wrote:
I'm wondering if I could use reserve to allocate a few MB and
then if the size is not enough that the vector grows
automatically? I read that vector resizing is very costly so
I presume that reserving just 1KB is not enough.

What you read was wrong. Resizing isn't that costly for
primitive object types (like unsigned char). There are other
reasons you might want to avoid it, but they probably don't
apply here.

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, [phone]

I tried 2 things:

1. reserve 3MB and then push_back every unsigned char (or byte) to the
vector.
2. Reserve 1KB on every callback

U didn't measure the time precisely, but it is approximatly the same.
It takes very long to complete, around 10 seconds.
This is to much.I don't know what the problem is.

[peter koch]

On 11 Nov., 11:21, Uli Kunkel <[e-mail]:

James Kanze wrote:

On Nov 10, 12:49 pm, Uli Kunkel <[e-mail]:

Juan Antonio Zaratiegui Vallecillo wrote:
I'm wondering if I could use reserve to allocate a few MB and
then if the size is not enough that the vector grows
automatically? *I read that vector resizing is very costly so
I presume that reserving just 1KB is not enough.

What you read was wrong. *Resizing isn't that costly for
primitive object types (like unsigned char). *There are other
reasons you might want to avoid it, but they probably don't
apply here.

--
James Kanze (GABI Software) * * * * * * email:[e-mail]
Conseils en informatique orientée objet/
* * * * * * * * * *Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, [phone]

I tried 2 things:

1. reserve 3MB and then push_back every unsigned char (or byte) to the
vector.
2. Reserve 1KB on every callback

That is the wrong approach. You either reserve once at the beginning
or you don't reserve at all. Reserving in small chunks will cause your
algorithm to run much slower than without any reserves at all.

>
U didn't measure the time precisely, but it is approximatly the same.
It takes very long to complete, around 10 seconds.
This is to much.I don't know what the problem is.

That is far to much - something else must be wrong. You can isolate
the vector code by simply appending "random data" to the vector
without reading it from the source. If that is not a snap, you should
post the codde here.

/Peter

[Uli Kunkel]

peter koch wrote:

On 11 Nov., 11:21, Uli Kunkel <[e-mail]:

>James Kanze wrote:

>>On Nov 10, 12:49 pm, Uli Kunkel <[e-mail]:
Juan Antonio Zaratiegui Vallecillo wrote:
I'm wondering if I could use reserve to allocate a few MB and
then if the size is not enough that the vector grows
automatically? I read that vector resizing is very costly so
I presume that reserving just 1KB is not enough.
What you read was wrong. Resizing isn't that costly for
primitive object types (like unsigned char). There are other
reasons you might want to avoid it, but they probably don't
apply here.
--
James Kanze (GABI Software) email:[e-mail]
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, [phone]

I tried 2 things:

1. reserve 3MB and then push_back every unsigned char (or byte) to the
vector.
2. Reserve 1KB on every callback

That is the wrong approach. You either reserve once at the beginning
or you don't reserve at all. Reserving in small chunks will cause your
algorithm to run much slower than without any reserves at all.

>U didn't measure the time precisely, but it is approximatly the same.
It takes very long to complete, around 10 seconds.
This is to much.I don't know what the problem is.

That is far to much - something else must be wrong. You can isolate
the vector code by simply appending "random data" to the vector
without reading it from the source. If that is not a snap, you should
post the codde here.

/Peter

I agree that it is better to allocate a couple of MB than to allocate
1KB so many times.
People on the group said that the latter isn't such a big deal.
Anyway the code is simple:
-----------------------------
//first I allocate 3MB in some initialization
myPhoto.reserve(3*1024*1024);

//then I assign one char at a time
for(unsigned long i=0; i < pProgress->lLength; i++)
{
myPhoto.push_back(pProgress->pbData[i]);
}
-------------------------------

What do you think?
Maybe I should use insert, instead of putting one char at the time.
I tried this but pbData is unsigned char* type and could't do it...

[Uli Kunkel]

Uli Kunkel wrote:

peter koch wrote:

>On 11 Nov., 11:21, Uli Kunkel <[e-mail]:

>>James Kanze wrote:
On Nov 10, 12:49 pm, Uli Kunkel <[e-mail]:
Juan Antonio Zaratiegui Vallecillo wrote:
I'm wondering if I could use reserve to allocate a few MB and
then if the size is not enough that the vector grows
automatically? I read that vector resizing is very costly so
I presume that reserving just 1KB is not enough.
What you read was wrong. Resizing isn't that costly for
primitive object types (like unsigned char). There are other
reasons you might want to avoid it, but they probably don't
apply here.
--
James Kanze (GABI Software) email:[e-mail]
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, [phone]
I tried 2 things:

1. reserve 3MB and then push_back every unsigned char (or byte) to the
vector.
2. Reserve 1KB on every callback

That is the wrong approach. You either reserve once at the beginning
or you don't reserve at all. Reserving in small chunks will cause your
algorithm to run much slower than without any reserves at all.

>>U didn't measure the time precisely, but it is approximatly the same.
It takes very long to complete, around 10 seconds.
This is to much.I don't know what the problem is.

That is far to much - something else must be wrong. You can isolate
the vector code by simply appending "random data" to the vector
without reading it from the source. If that is not a snap, you should
post the codde here.

/Peter

I agree that it is better to allocate a couple of MB than to allocate
1KB so many times.
People on the group said that the latter isn't such a big deal.
Anyway the code is simple:
-----------------------------
//first I allocate 3MB in some initialization
myPhoto.reserve(3*1024*1024);

//then I assign one char at a time
for(unsigned long i=0; i < pProgress->lLength; i++)
{
myPhoto.push_back(pProgress->pbData[i]);
}
-------------------------------

What do you think?
Maybe I should use insert, instead of putting one char at the time.
I tried this but pbData is unsigned char* type and could't do it...

I had one big oversight.
In the callback function I can define the length of each chunk.I thought
it is integer type.As I recall that is only 36635.
I check again and saw it is actually unsigned long.That was very stupid
of me.
Now I pass 1MB and it takes 2,3 seconds which is much better.

If I'm still doing something wrong I would appreciate any help.

[peter koch]

On 11 Nov., 12:49, Uli Kunkel <[e-mail]:

Uli Kunkel wrote:

peter koch wrote:

On 11 Nov., 11:21, Uli Kunkel <[e-mail]:
James Kanze wrote:
On Nov 10, 12:49 pm, Uli Kunkel <[e-mail]:
Juan Antonio Zaratiegui Vallecillo wrote:
I'm wondering if I could use reserve to allocate a few MB and
then if the size is not enough that the vector grows
automatically? *I read that vector resizing is very costly so
I presume that reserving just 1KB is not enough.
What you read was wrong. *Resizing isn't that costly for
primitive object types (like unsigned char). *There are other
reasons you might want to avoid it, but they probably don't
apply here.
--
James Kanze (GABI Software) * * * * * * email:[e-mail]
Conseils en informatique orientée objet/
* * * * * * * * * *Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, [phone]
I tried 2 things:

>1. reserve 3MB and then push_back every unsigned char (or byte) to the
vector.
2. Reserve 1KB on every callback

That is the wrong approach. You either reserve once at the beginning
or you don't reserve at all. Reserving in small chunks will cause your
algorithm to run much slower than without any reserves at all.
U didn't measure the time precisely, but it is approximatly the same.
It takes very long to complete, around 10 seconds.
This is to much.I don't know what the problem is.

That is far to much - something else must be wrong. You can isolate
the vector code by simply appending "random data" to the vector
without reading it from the source. If that is not a snap, you should
post the codde here.

/Peter

I agree that it is better to allocate a couple of MB than to allocate
1KB so many times.
People on the group said that the latter isn't such a big deal.
Anyway the code is simple:
-----------------------------
//first I allocate 3MB in some initialization
myPhoto.reserve(3*1024*1024);

//then I assign one char at a time
for(unsigned long i=0; i < pProgress->lLength; i++)
{
* * myPhoto.push_back(pProgress->pbData[i]);

No need to push_back each item individually. Better would be to use
std::vector.insert or std::copy with a back_inserter. Look these up.

}
-------------------------------

What do you think?
Maybe I should use insert, instead of putting one char at the time.
I tried this but pbData is unsigned char* type and could't do it...

I had one big oversight.
In the callback function I can define the length of each chunk.I thought
it is integer type.As I recall that is only 36635.

It is implementation defined, but i doubt max(int) is 36635 - it might
be 32767.

I check again and saw it is actually unsigned long.That was very stupid
of me.
Now I pass 1MB and it takes 2,3 seconds which is much better.

If I'm still doing something wrong I would appreciate any help.

See above.

/Peter

[Uli Kunkel]

peter koch wrote:

On 11 Nov., 12:49, Uli Kunkel <[e-mail]:

>Uli Kunkel wrote:

>>peter koch wrote:
On 11 Nov., 11:21, Uli Kunkel <[e-mail]:
James Kanze wrote:
>On Nov 10, 12:49 pm, Uli Kunkel <[e-mail]:
>>Juan Antonio Zaratiegui Vallecillo wrote:
>>I'm wondering if I could use reserve to allocate a few MB and
>>then if the size is not enough that the vector grows
>>automatically? I read that vector resizing is very costly so
>>I presume that reserving just 1KB is not enough.
>What you read was wrong. Resizing isn't that costly for
>primitive object types (like unsigned char). There are other
>reasons you might want to avoid it, but they probably don't
>apply here.
>--
>James Kanze (GABI Software) email:[e-mail]
>Conseils en informatique orientée objet/
> Beratung in objektorientierter Datenverarbeitung
>9 place Sémard, 78210 St.-Cyr-l'École, France, [phone]
I tried 2 things:
1. reserve 3MB and then push_back every unsigned char (or byte) to the
vector.
2. Reserve 1KB on every callback
That is the wrong approach. You either reserve once at the beginning
or you don't reserve at all. Reserving in small chunks will cause your
algorithm to run much slower than without any reserves at all.
U didn't measure the time precisely, but it is approximatly the same.
It takes very long to complete, around 10 seconds.
This is to much.I don't know what the problem is.
That is far to much - something else must be wrong. You can isolate
the vector code by simply appending "random data" to the vector
without reading it from the source. If that is not a snap, you should
post the codde here.
/Peter
I agree that it is better to allocate a couple of MB than to allocate
1KB so many times.
People on the group said that the latter isn't such a big deal.
Anyway the code is simple:
-----------------------------
//first I allocate 3MB in some initialization
myPhoto.reserve(3*1024*1024);
//then I assign one char at a time
for(unsigned long i=0; i < pProgress->lLength; i++)
{
myPhoto.push_back(pProgress->pbData[i]);

No need to push_back each item individually. Better would be to use
std::vector.insert or std::copy with a back_inserter. Look these up.

>>}
-------------------------------
What do you think?
Maybe I should use insert, instead of putting one char at the time.
I tried this but pbData is unsigned char* type and could't do it...

I had one big oversight.
In the callback function I can define the length of each chunk.I thought
it is integer type.As I recall that is only 36635.

It is implementation defined, but i doubt max(int) is 36635 - it might
be 32767.

>I check again and saw it is actually unsigned long.That was very stupid
of me.
Now I pass 1MB and it takes 2,3 seconds which is much better.

If I'm still doing something wrong I would appreciate any help.

See above.

/Peter

I have a silly problem.
Insert takes const byte&. And variable pProgress->pbData that I pass is
unsigned char*.
Anyway it inserts an empty value, something is not right..

[Richard Herring]

In message <gf**********@news.metronet.hr>, Uli Kunkel
<ge*******@yahoo.comwrites

>peter koch wrote:

>On 11 Nov., 12:49, Uli Kunkel <[e-mail]:

>>Uli Kunkel wrote:

[...]

>>>I agree that it is better to allocate a couple of MB than to allocate
1KB so many times.
People on the group said that the latter isn't such a big deal.
Anyway the code is simple:
-----------------------------
//first I allocate 3MB in some initialization
myPhoto.reserve(3*1024*1024);
//then I assign one char at a time
for(unsigned long i=0; i < pProgress->lLength; i++)
{
myPhoto.push_back(pProgress->pbData[i]);

> No need to push_back each item individually. Better would be to use
std::vector.insert or std::copy with a back_inserter. Look these up.

>>>}
-------------------------------
What do you think?
Maybe I should use insert, instead of putting one char at the time.
I tried this but pbData is unsigned char* type and could't do it...

[...]

>>If I'm still doing something wrong I would appreciate any help.

See above.

I have a silly problem.
Insert takes const byte&. And variable pProgress->pbData that I pass is
unsigned char*.

Use the two-iterator form of insert:

myPhoto.insert(myPhoto.end(), pProgress->pBData, pProgress->pBData+pProgress->lLength);

Pointers are iterators.

>Anyway it inserts an empty value, something is not right..

--
Richard Herring