I have a StringBuilder that has a string with 12,000,000 characters.
When I do a ToString(), I expect to have ~25,000,000 bytes worth of
memory, yet, I end up with ~43,000,000 bytes. That's almost double
the size. The string returned from ToString() is actually of size
StringBuilder.Capacity, NOT StringBuilder.Length. It may have a
end-of-string character at StringBuilder.Length, but its actual memory
size is StringBuilder.Capacity.
Does this sound right?
15  2764 
DV, I have a StringBuilder that has a string with 12,000,000 characters.
When I do a ToString(), I expect to have ~25,000,000 bytes worth of
memory, yet, I end up with ~43,000,000 bytes. That's almost double
the size. The string returned from ToString() is actually of size
StringBuilder.Capacity, NOT StringBuilder.Length. It may have a
end-of-string character at StringBuilder.Length, but its actual memory
size is StringBuilder.Capacity.
Capacity of a StringBuilder is how many characters can be stored in the
StringBuilder at a given time. Length is how many characters are
represented in the string that the StringBuilder represents at a given time.
The maximum value of length and capacity is 2,147,483,647.
Unicode characters can consume 2, 3 or 4 bytes. Therefore, depending on
which characters are being stored in the StringBuilder, it would be possible
to have 12,000,000 characters in the StringBuilder that translate to about
43,000,000 bytes. You will have to iterate the characters in the string
counting the bytes to see if the number of bytes can be explained by the
storage of unicode characters.
Regards,
Randy
"DV" <da*****@gmail.com> wrote in message
news:11**********************@j33g2000cwa.googlegr oups.com...
|I have a StringBuilder that has a string with 12,000,000 characters.
| When I do a ToString(), I expect to have ~25,000,000 bytes worth of
| memory, yet, I end up with ~43,000,000 bytes. That's almost double
| the size. The string returned from ToString() is actually of size
| StringBuilder.Capacity, NOT StringBuilder.Length. It may have a
| end-of-string character at StringBuilder.Length, but its actual memory
| size is StringBuilder.Capacity.
|
| Does this sound right?
|
No it doesn't, I really don't know where you got this value from but it's
not the size of the SB. The Capacity and Length property values are the
number of characters the SB can hold and the actual number it holds. Note
the "number of characters", not the number of bytes.
If you create a stringbuilder like this:
string s = new String('\u0306', 12000000);
StringBuilder sb = new StringBuilder(s);
you will end with a SB buffer (a String) with a Capacity of 16777216 and a
Length of 12000000.
The length of the char[] (the string backing store) in bytes is 33554432.
Note that the string backing store is a char[], so the buffer size does not
depend encoding used as the other replier suggests, a char is fixed 2 bytes
in .NET.
Willy.
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:uB**************@tk2msftngp13.phx.gbl...
|
| "DV" <da*****@gmail.com> wrote in message
| news:11**********************@j33g2000cwa.googlegr oups.com...
||I have a StringBuilder that has a string with 12,000,000 characters.
|| When I do a ToString(), I expect to have ~25,000,000 bytes worth of
|| memory, yet, I end up with ~43,000,000 bytes. That's almost double
|| the size. The string returned from ToString() is actually of size
|| StringBuilder.Capacity, NOT StringBuilder.Length. It may have a
|| end-of-string character at StringBuilder.Length, but its actual memory
|| size is StringBuilder.Capacity.
||
|| Does this sound right?
||
|
| No it doesn't, I really don't know where you got this value from but it's
| not the size of the SB. The Capacity and Length property values are the
| number of characters the SB can hold and the actual number it holds. Note
| the "number of characters", not the number of bytes.
| If you create a stringbuilder like this:
|
| string s = new String('\u0306', 12000000);
| StringBuilder sb = new StringBuilder(s);
|
| you will end with a SB buffer (a String) with a Capacity of 16777216 and a
| Length of 12000000.
| The length of the char[] (the string backing store) in bytes is 33554432.
| Note that the string backing store is a char[], so the buffer size does
not
| depend encoding used as the other replier suggests, a char is fixed 2
bytes
| in .NET.
|
| Willy.
To add to my previous reply, if you create a SB like this:
StringBuilder sb = new StringBuilder(s, 12000000);
Your buffer will be 24000000 bytes (SB Capactity=Length=12000000). The
reason for this is that now you create a SB with a predefined length, while
in the previous sample, the SB starts with a Capactity of 16 and expands by
doubling it's capacity each time it gets filled completely.
Willy.
DV <da*****@gmail.com> wrote: I have a StringBuilder that has a string with 12,000,000 characters.
When I do a ToString(), I expect to have ~25,000,000 bytes worth of
memory, yet, I end up with ~43,000,000 bytes. That's almost double
the size. The string returned from ToString() is actually of size
StringBuilder.Capacity, NOT StringBuilder.Length. It may have a
end-of-string character at StringBuilder.Length, but its actual memory
size is StringBuilder.Capacity.
Does this sound right?
Its size in terms of memory consumption is indeed represented by the
capacity.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Randy A. Ynchausti <ra*************@msn.com> wrote:
<snip> Unicode characters can consume 2, 3 or 4 bytes.
That's true in some encodings, but .NET internally uses UTF-16, which
is exactly 2 bytes per character. Surrogate pairs are treated as two
characters when it comes to things like String.Length.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Willy Denoyette [MVP] <wi*************@telenet.be> wrote: No it doesn't, I really don't know where you got this value from but it's
not the size of the SB. The Capacity and Length property values are the
number of characters the SB can hold and the actual number it holds. Note
the "number of characters", not the number of bytes.
If you create a stringbuilder like this:
string s = new String('\u0306', 12000000);
StringBuilder sb = new StringBuilder(s);
you will end with a SB buffer (a String) with a Capacity of 16777216 and a
Length of 12000000.
Yes - but if you then call ToString() on that StringBuilder, you'll
find that the string only has a Length of 12000000 but takes up
2*16777216+overhead bytes. In other words, StringBuilder doesn't trim
the string to its length before returning it.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
That's why I said....
The length of the char[] (the string backing store) in bytes is 33554432.
Willy.
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om...
| Willy Denoyette [MVP] <wi*************@telenet.be> wrote:
| > No it doesn't, I really don't know where you got this value from but
it's
| > not the size of the SB. The Capacity and Length property values are the
| > number of characters the SB can hold and the actual number it holds.
Note
| > the "number of characters", not the number of bytes.
| > If you create a stringbuilder like this:
| >
| > string s = new String('\u0306', 12000000);
| > StringBuilder sb = new StringBuilder(s);
| >
| > you will end with a SB buffer (a String) with a Capacity of 16777216 and
a
| > Length of 12000000.
|
| Yes - but if you then call ToString() on that StringBuilder, you'll
| find that the string only has a Length of 12000000 but takes up
| 2*16777216+overhead bytes. In other words, StringBuilder doesn't trim
| the string to its length before returning it.
|
| --
| Jon Skeet - <sk***@pobox.com>
| http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
| If replying to the group, please do not mail me too
Hit send too fast.
The String reference returned from SB.ToString() is just the same reference
to the existing String object, trimming would involve the creation of a new
string, this would increase the memory footprint and would take a
performance hit.
Willy.
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:OP**************@TK2MSFTNGP14.phx.gbl...
| That's why I said....
| The length of the char[] (the string backing store) in bytes is 33554432.
|
|
| Willy.
|
|
|
| "Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
| news:MP************************@msnews.microsoft.c om...
|| Willy Denoyette [MVP] <wi*************@telenet.be> wrote:
|| > No it doesn't, I really don't know where you got this value from but
| it's
|| > not the size of the SB. The Capacity and Length property values are the
|| > number of characters the SB can hold and the actual number it holds.
| Note
|| > the "number of characters", not the number of bytes.
|| > If you create a stringbuilder like this:
|| >
|| > string s = new String('\u0306', 12000000);
|| > StringBuilder sb = new StringBuilder(s);
|| >
|| > you will end with a SB buffer (a String) with a Capacity of 16777216
and
| a
|| > Length of 12000000.
||
|| Yes - but if you then call ToString() on that StringBuilder, you'll
|| find that the string only has a Length of 12000000 but takes up
|| 2*16777216+overhead bytes. In other words, StringBuilder doesn't trim
|| the string to its length before returning it.
||
|| --
|| Jon Skeet - <sk***@pobox.com>
|| http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
|| If replying to the group, please do not mail me too
|
|
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:OX**************@TK2MSFTNGP11.phx.gbl... Hit send too fast.
The String reference returned from SB.ToString() is just the same
reference
to the existing String object, trimming would involve the creation of a
new
string, this would increase the memory footprint and would take a
performance hit.
Willy.
That's what I thought at first and then I thought about subsequent changes
to the builder - It must keep a reference to the string returned just in
case - subsequent mods must cause the copying of the string into a new
buffer but since this is uncommon in real code it's worth the optimisation.
"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:OP**************@TK2MSFTNGP14.phx.gbl...
| That's why I said....
| The length of the char[] (the string backing store) in bytes is
33554432.
|
|
| Willy.
|
|
|
| "Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
| news:MP************************@msnews.microsoft.c om...
|| Willy Denoyette [MVP] <wi*************@telenet.be> wrote:
|| > No it doesn't, I really don't know where you got this value from but
| it's
|| > not the size of the SB. The Capacity and Length property values are
the
|| > number of characters the SB can hold and the actual number it holds.
| Note
|| > the "number of characters", not the number of bytes.
|| > If you create a stringbuilder like this:
|| >
|| > string s = new String('\u0306', 12000000);
|| > StringBuilder sb = new StringBuilder(s);
|| >
|| > you will end with a SB buffer (a String) with a Capacity of 16777216
and
| a
|| > Length of 12000000.
||
|| Yes - but if you then call ToString() on that StringBuilder, you'll
|| find that the string only has a Length of 12000000 but takes up
|| 2*16777216+overhead bytes. In other words, StringBuilder doesn't trim
|| the string to its length before returning it.
||
|| --
|| Jon Skeet - <sk***@pobox.com>
|| http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
|| If replying to the group, please do not mail me too
|
|
Thanks all... I think i got it...
Interestingly enough..
If I do...
StringBuilder sb = new StringBuilder(10000000);
for (int i=0; i<100; i++)
sb.Append("Hello World");
string a1 = sb.ToString();
and..
StringBuilder sb = new StringBuilder();
for (int i=0; i<100; i++)
sb.Append("Hello World");
string a2 = sb.ToString();
in this case, a1 actually takes LESS physical memory then a2. a1 gets
trimmed while a2 returns the internal string.
i guess the lesson is to put a capacity whenever possible. but that
also has drawbacks..
Willy Denoyette [MVP] <wi*************@telenet.be> wrote: Hit send too fast.
The String reference returned from SB.ToString() is just the same reference
to the existing String object, trimming would involve the creation of a new
string, this would increase the memory footprint and would take a
performance hit.
Exactly - so the OP's supposition that "The string returned from
ToString() is actually of size StringBuilder.Capacity, NOT
StringBuilder.Length" sounds right to me.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om...
| Willy Denoyette [MVP] <wi*************@telenet.be> wrote:
| > Hit send too fast.
| > The String reference returned from SB.ToString() is just the same
reference
| > to the existing String object, trimming would involve the creation of a
new
| > string, this would increase the memory footprint and would take a
| > performance hit.
|
| Exactly - so the OP's supposition that "The string returned from
| ToString() is actually of size StringBuilder.Capacity, NOT
| StringBuilder.Length" sounds right to me.
|
Not really, what delimits a string is it's Length, the fact that the string
buffer (char[]) is larger is due to the way SB's handle the underlying
'String', they can add char's to the buffer without the need to create a new
String object (until the buffer is filled completely), which is the reason
why SBs exist. What is returned by SB.ToString() is a String reference and
what counts is the Length property value, what's stored beyond is not part
of the String, there isn't a managed way to get at the size of the buffer,
nor can you get at the contents of the store beyond the last char position +
1 (the 0x0000 terminator) in the buffer.
But this was not really my point, the OP assumes that the capacity is 43MB
for a string of 12000000 which is definitely not true.
Willy.
Willy Denoyette [MVP] wrote: | Exactly - so the OP's supposition that "The string returned from
| ToString() is actually of size StringBuilder.Capacity, NOT
| StringBuilder.Length" sounds right to me.
Not really, what delimits a string is it's Length, the fact that the string
buffer (char[]) is larger is due to the way SB's handle the underlying
'String', they can add char's to the buffer without the need to create a new
String object (until the buffer is filled completely), which is the reason
why SBs exist. What is returned by SB.ToString() is a String reference and
what counts is the Length property value, what's stored beyond is not part
of the String, there isn't a managed way to get at the size of the buffer,
nor can you get at the contents of the store beyond the last char position +
1 (the 0x0000 terminator) in the buffer.
No - but given the rest of the OP's question, I thought it reasonable
to assume that the "size" he's talking about is the size in memory, not
the "logical" length of the string. Given that reading, isn't the OP's
supposition valid? The "buffer size" (in characters) of the returned
string is the same as (or maybe one or two characters more or less -
not sure) the capacity of the StringBuilder it was returned from.
Even though you can't get at the rest of the buffer, I'd say it still
"counts" in terms of being valuable information. For instance, if you
were creating a lot of long-lived strings from StringBuilders, it may
be worth creating a copy of the string which effectively trims the
extra buffer, allowing the oversized string to then be garbage
collected. (I've done this many times in Java, particularly when
reading lines from a file. The initial buffer is 80 characters, and if
you're reading a dictionary a line at a time and the average line
length is only 5 or 6 characters, the wastage can be very, very
significant.)
But this was not really my point, the OP assumes that the capacity is 43MB
for a string of 12000000 which is definitely not true.
That much is certainly true - *if* by "string of 12000000" you mean a
"string of internal buffer size 12000000 characters".
Jon
"DV" <da*****@gmail.com> wrote in message
news:11**********************@u72g2000cwu.googlegr oups.com...
| Thanks all... I think i got it...
| Interestingly enough..
|
| If I do...
| StringBuilder sb = new StringBuilder(10000000);
| for (int i=0; i<100; i++)
| sb.Append("Hello World");
| string a1 = sb.ToString();
|
| and..
| StringBuilder sb = new StringBuilder();
| for (int i=0; i<100; i++)
| sb.Append("Hello World");
| string a2 = sb.ToString();
|
| in this case, a1 actually takes LESS physical memory then a2. a1 gets
| trimmed while a2 returns the internal string.
| i guess the lesson is to put a capacity whenever possible. but that
| also has drawbacks..
|
Right, what you see here is the result of an optimization.
Let me try to explain what's happening ....
First you need to know how a SB and a String looks like on the managed heap,
this is how a StringBuilder object looks:
<standard object header> // to IntPtr sized values, not relevant here
IntPtr m_currentThread;
int m_maxCapacity;
string m_StringValue;
while a String looks like:
<object header>
int m_arrayLength;
int m_stringLength;
char m_firstChar;
In the first case, you create a SB with capacity 10000000, that means that
the size of the underlying String object is larger than 85Kb, so, the String
will end on the Large Object Heap (LOH).
Then you start filling the string buffer, the result at the end of the loop
looks like:
Your StrinBuilder sb on the Gen0 heap
m_currentThread = xxxx // not important here
m_maxCapacity = 2147483647 // 2GB
m_StringValue = 03271000 // reference - points to a string on the
LOH (value as a sample)
m_arrayLength = 10000001 // Buffer space (in no. of char)
m_stringLength = 1100 // actual string Length
m_firstChar = 'H' // First char in buffer (start of buffer
.... // following chars
.... = 'd' // last char of string (buffer position 1100)
.... = 0x0000 // last char in buffer (buffer position 1101)
Now when you execute ... sb.ToString();
the CLR rightfully decides that this String object doesn't belong to the LOH
(is < 85Kb), so he creates a new String on the Gen0 heap and returns it's
reference in a1, the new string object looks now like:
m_arrayLength = 1101 // Buffer space (in no. of char)
m_stringLength = 1100 // actual string Length
m_firstChar = 'H' // First char in buffer (start of buffer
.... // following chars
.... = 'd' // last char of string (buffer position 1100)
.... = 0x0000
Notice the new m_arrayLength of 1100 ...
Note that the m_arrayLength = 10000001 has never been committed, only
reserved that is why you don't see this allocated in physical memory.
What's happening in the second case is:
A SB is created on the Gen0 heap and looks like:
m_currentThread = xxxx // not important here
m_maxCapacity = 2147483647 // 2GB
m_StringValue = 01274e34 // reference - points to a string on the Gen0
heap(value as a sample)
and the underlying String object at the end of the loop:
m_arrayLength = 2049 // Buffer space (in no. of char)
m_stringLength = 1100 // actual string Length
m_firstChar = 'H' // First char in buffer (start of buffer
.... // following chars
.... = 'd' // last char of string (buffer position 1100)
.... = 0x0000
Notice the m_arrayLength ...
But before you get this final string, a number of temporary strings need to
be build. Remember that the SB starts with an m_arrayLength = 17 (16 + 1 for
the 0x0000 string termination char).
That means that after the loop you have effectively created 8 intermediate
string objects (16, 32, 64, ...2048).
That means that you have wasted some memory, but also some CPU cycles, more,
you also put some additional stress on the GC which will have to clean-up
the intermediate objects.
Conclusion: you should try to pre-allocate SB's whenever possible. Note that
this is especially important for server applications and for client
(WinForms) applications that need to run in Terminal Server environments.
Hope this clears things up a bit :-)
Willy.
Jon, you should read his original question, that starts with ...
I have a StringBuilder that has a string with 12,000,000 characters.
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:11**********************@j33g2000cwa.googlegr oups.com...
| Willy Denoyette [MVP] wrote:
| > | Exactly - so the OP's supposition that "The string returned from
| > | ToString() is actually of size StringBuilder.Capacity, NOT
| > | StringBuilder.Length" sounds right to me.
| >
| > Not really, what delimits a string is it's Length, the fact that the
string
| > buffer (char[]) is larger is due to the way SB's handle the underlying
| > 'String', they can add char's to the buffer without the need to create a
new
| > String object (until the buffer is filled completely), which is the
reason
| > why SBs exist. What is returned by SB.ToString() is a String reference
and
| > what counts is the Length property value, what's stored beyond is not
part
| > of the String, there isn't a managed way to get at the size of the
buffer,
| > nor can you get at the contents of the store beyond the last char
position +
| > 1 (the 0x0000 terminator) in the buffer.
|
| No - but given the rest of the OP's question, I thought it reasonable
| to assume that the "size" he's talking about is the size in memory, not
| the "logical" length of the string. Given that reading, isn't the OP's
| supposition valid? The "buffer size" (in characters) of the returned
| string is the same as (or maybe one or two characters more or less -
| not sure) the capacity of the StringBuilder it was returned from.
|
| Even though you can't get at the rest of the buffer, I'd say it still
| "counts" in terms of being valuable information. For instance, if you
| were creating a lot of long-lived strings from StringBuilders, it may
| be worth creating a copy of the string which effectively trims the
| extra buffer, allowing the oversized string to then be garbage
| collected. (I've done this many times in Java, particularly when
| reading lines from a file. The initial buffer is 80 characters, and if
| you're reading a dictionary a line at a time and the average line
| length is only 5 or 6 characters, the wastage can be very, very
| significant.)
|
| > But this was not really my point, the OP assumes that the capacity is
43MB
| > for a string of 12000000 which is definitely not true.
|
| That much is certainly true - *if* by "string of 12000000" you mean a
| "string of internal buffer size 12000000 characters".
Jon,
The OP asked....
I have a StringBuilder that has a string with 12,000,000 characters.
And I replied with .. you will end with a SB buffer (a String) with a
Capacity of 16777216 and a
Length of 12000000.
You see, the capacity of a SB holding a string with Length 12000000 is
always 16777216 or 33554432 bytes unless you pre-allocate the SB.
The SB does not consume 43MB in this case. Actually the OP doesn't know the
size of the SB at all, he is simply watching the memory counters in Taskman
I'm sure, these don't reveal the size of whatever object in memory. And
that's my point. Sorry if I'm confusing you ;-)
Willy.
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Kevin C |
last post by:
Quick Question:
StringBuilder is obviously more efficient dealing with string concatenations
than the old '+=' method... however, in dealing with relatively large string
concatenations (ie,...
|
by: |
last post by:
I know how to use a StringBuilder, which supposedly does
not create a new copy of it each time you modify it
contents by adding or removing text.
But, I wonder how does it do that internally ?
I...
|
by: KH |
last post by:
API question...
Why doesn't StringBuilder have IndexOf and other similar methods like
String? I can't think of a good reason off the top of my head.
Easy to write helper functions to do the...
|
by: genc_ymeri |
last post by:
Hi over there,
Propably this subject is discussed over and over several times. I did google
it too but I was a little bit surprised what I read on internet when it
comes 'when to use what'.
Most...
|
by: Richard Lewis Haggard |
last post by:
I thought that the whole point of StringBuilder was that it was supposed to
be a faster way of building strings than string. However, I just put
together a simple little application to do a...
|
by: Hardy Wang |
last post by:
Hi all,
I know it is better to handle large string with a StringBuilder, but how
does StringBuilder class improve the performance in the background?
Thanks!
--
WWW:...
|
by: raylopez99 |
last post by:
StringBuilder better and faster than string for adding many strings.
Look at the below. It's amazing how much faster StringBuilder is than
string.
The last loop below is telling: for adding...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
| |
