Destroying STL Strings

[RP]

I have a requirement to sanitise a particular std::string, myString, during
destruction. To achieve this we use the replace function as follows:

myString.replace(0, 10, "UUUUUUUUUU");

According to Josuttis in his "The C++ Standard Library: A Tutorial and
Reference", this should replace, at most, 10 characters of *this, starting
with the index 0, with all 'U's.

However, I have been informed that this is unsatisfactory because
std::string in C++ does not behave like a simple char* string in C. When the
value of the string changes, the old string data is not overwritten; instead
the memory is simply deallocated and a new chunk of approrpiate length is
allocated on the heap. The std::string.replace function is no exception and
will leave the string unsanitised.

Can anyone shed any light on this as there appears to be a contradiction.

TIA

RP.

[Howard]

"RP" <[e-mail]> wrote in message
news:zC*******************@fe2.news.blueyonder.co. uk...

I have a requirement to sanitise a particular std::string, myString, during
destruction. To achieve this we use the replace function as follows:

myString.replace(0, 10, "UUUUUUUUUU");

According to Josuttis in his "The C++ Standard Library: A Tutorial and
Reference", this should replace, at most, 10 characters of *this, starting
with the index 0, with all 'U's.

However, I have been informed that this is unsatisfactory because
std::string in C++ does not behave like a simple char* string in C. When
the value of the string changes, the old string data is not overwritten;
instead the memory is simply deallocated and a new chunk of approrpiate
length is allocated on the heap. The std::string.replace function is no
exception and will leave the string unsanitised.

Can anyone shed any light on this as there appears to be a contradiction.

I don't see anywhere that says replace has to be implemented in-place, and
can't therefore re-allocate. But (if you have to) why not just loop over
the string, using myString[i] = 'U';?

-Howard

[Daniel T.]

In article <zC*******************@fe2.news.blueyonder.co.uk >,
"RP" <[e-mail]> wrote:

I have a requirement to sanitise a particular std::string, myString, during
destruction. To achieve this we use the replace function as follows:

myString.replace(0, 10, "UUUUUUUUUU");

According to Josuttis in his "The C++ Standard Library: A Tutorial and
Reference", this should replace, at most, 10 characters of *this, starting
with the index 0, with all 'U's.

However, I have been informed that this is unsatisfactory because
std::string in C++ does not behave like a simple char* string in C. When the
value of the string changes, the old string data is not overwritten; instead
the memory is simply deallocated and a new chunk of approrpiate length is
allocated on the heap. The std::string.replace function is no exception and
will leave the string unsanitised.

Can anyone shed any light on this as there appears to be a contradiction.

AFAIK, there are no guarantees either way. I think to satisfy such a
requirement, you will have to roll your own. That's not surprising BTW,
of all the standard classes, string seems to be the one that is least
satisfactory in real projects.

--
Magic depends on tradition and belief. It does not welcome observation,
nor does it profit by experiment. On the other hand, science is based
on experience; it is open to correction by observation and experiment.

[Phlip]

RP wrote:

myString.replace(0, 10, "UUUUUUUUUU");

Sutter and ... Andrei cover this in their /C++ Coding Standards/ book. To
grab control of the controlled character array within a string, use &s[0].
So this will work:

memset(&s[0], 'U', s.size());

Note that size() returns the number of elements, not bytes. And note that
'\xff' will stamp out all bits.

However, the string might reallocate in memory at any time the implementors
feel like, including just before the [] call.

Now if you really want to sanitize your string, put your computer inside an
RF cage so I can't use remote masers to peek at your bus and sniff every
packet out. (That's how the Dept of Homeland Security keeps tabs on
Democrats and other terrorists.)

The contents of your string will exist in several places in memory, and in
the CPU cache, and the odds are good that some of these places will still
contain the contents, as garbage, after you sanitize the last version of
the controlled array inside that string.

If you have a protected mode operating system, then your memory is as secure
from normal programs as your OS's binary image is secure. Other programs
cannot get to your memory because the CPU's hardware protects it.

But if the virtual memory system swaps your string to disk, then a scanning
tunneling electron microscope might find it, in only a few months of labor.

If your boss is telling your clients your program is "secure", and you have
not yet studied the full envelop of security, and if your team thinks that
writing on strings will secure them, then something fishy is going on.

--
Phlip
http://www.greencheese.org/ZeekLand <-- NOT a blog!!!

[Kai-Uwe Bux]

Phlip wrote:

RP wrote:

myString.replace(0, 10, "UUUUUUUUUU");

Sutter and ... Andrei cover this in their /C++ Coding Standards/ book. To
grab control of the controlled character array within a string, use &s[0].
So this will work:

memset(&s[0], 'U', s.size());

I think, the standard makes no guarantee that std::string objects are stored
contiguously. The specifications for std::string, however, suggest a
contiguous implementation. Thus, the above line is likely to work on most
implementations, but it is not blessed by the standard.

[snip]
Best

Kai-Uwe Bux

[rossum]

RP wrote:

However, I have been informed that this is unsatisfactory because
std::string in C++ does not behave like a simple char* string in C. When the
value of the string changes, the old string data is not overwritten; instead
the memory is simply deallocated and a new chunk of approrpiate length is
allocated on the heap. The std::string.replace function is no exception and
will leave the string unsanitised.

Can anyone shed any light on this as there appears to be a contradiction.

TIA

RP.

You have described the behaviour of strings in C#, they are immutable
and all changes are done by deallocating and reallocating. The same
may also apply to C++/CLI aka Managed C++ under Visual Studio. Since
standard C++ strings are mutable the same should not apply to
std::string though compiler writers can do strange things on occasions.

HTH

rossum

[Roy Smith]

In article <78*****************@newssvr33.news.prodigy.com> ,
Phlip <ph*******@gmail.com> wrote:

However, the string might reallocate in memory at any time the implementors
feel like, including just before the [] call.

I must admit I've never done it myself, but this sounds like a place where
you might want to write your own allocator. Have the deallocator
over-write the data.

[Daniel T.]

In article <78*****************@newssvr33.news.prodigy.com> ,
Phlip <ph*******@gmail.com> wrote:

RP wrote:

myString.replace(0, 10, "UUUUUUUUUU");
Sutter and ... Andrei cover this in their /C++ Coding Standards/ book. To
grab control of the controlled character array within a string, use &s[0].
So this will work:

memset(&s[0], 'U', s.size());

I don't think that is guaranteed to work at any level. (a) there is no
guarantee that the memory in the string is contiguous. It may be
implemented like a deque for example or even a series of shared memory
blocks in which case something like the above might change other strings.

However, the string might reallocate in memory at any time the implementors
feel like, including just before the [] call.

Now if you really want to sanitize your string, put your computer inside an
RF cage so I can't use remote masers to peek at your bus and sniff every
packet out. (That's how the Dept of Homeland Security keeps tabs on
Democrats and other terrorists.)

The contents of your string will exist in several places in memory, and in
the CPU cache, and the odds are good that some of these places will still
contain the contents, as garbage, after you sanitize the last version of
the controlled array inside that string.

If you have a protected mode operating system, then your memory is as secure
from normal programs as your OS's binary image is secure. Other programs
cannot get to your memory because the CPU's hardware protects it.

But if the virtual memory system swaps your string to disk, then a scanning
tunneling electron microscope might find it, in only a few months of labor.

If your boss is telling your clients your program is "secure", and you have
not yet studied the full envelop of security, and if your team thinks that
writing on strings will secure them, then something fishy is going on.

I can't help but wonder what the test would look like that ensures that
the string was actually sanitized... If the OP can come up with a test,
then maybe we can come up with a way to pass it. Until then...

--
Magic depends on tradition and belief. It does not welcome observation,
nor does it profit by experiment. On the other hand, science is based
on experience; it is open to correction by observation and experiment.

[Kai-Uwe Bux]

Roy Smith wrote:

In article <78*****************@newssvr33.news.prodigy.com> ,
Phlip <ph*******@gmail.com> wrote:

However, the string might reallocate in memory at any time the
implementors feel like, including just before the [] call.

I must admit I've never done it myself, but this sounds like a place where
you might want to write your own allocator. Have the deallocator
over-write the data.

Maybe you are onto something. I was playing with this idea:

#include <memory>
#include <cstring>

template < typename T, typename Alloc = std::allocator<T> >
struct destructive_allocator : public Alloc {

void deallocate ( typename Alloc::pointer ptr,
typename Alloc::size_type length ) {
unsigned char * uc_ptr = reinterpret_cast< unsigned char * >( ptr );
std::memset( uc_ptr, 0, sizeof(T)*length );
Alloc::deallocate( ptr, length );
}

template < typename S >
struct rebind {
typedef destructive_allocator< S,
typename Alloc::template rebind<S>::other > other;
};

}; // class<T> allocator_base
#include <string>

typedef std::basic_string< char, std::char_traits< char >,
destructive_allocator< char > > my_string;
#include <iostream>

int main ( void ) {
my_string str ( "hello world!" );
std::cout << str << '\n';
}
However, there are catches:

a) I think, the deallocator has undefined behavior -- that might not be a
problem since the "expected" undefined behavior could be exactly what the
OP needs.

b) I also think, the compiler is entitled to eliminate the call to
std::memset under the as-if rule: modification of memory that is about to
be reclaimed is very likely not observable.
Best

Kai-Uwe Bux

[Pete Becker]

Kai-Uwe Bux wrote:

I think, the standard makes no guarantee that std::string objects are stored
contiguously.

As of yesterday's Library Working Group vote, it does. Subject to
approval by the full Standards Committee tomorrow.

--

Pete Becker
Roundhouse Consulting, Ltd.

[Kai-Uwe Bux]

Pete Becker wrote:

Kai-Uwe Bux wrote:

I think, the standard makes no guarantee that std::string objects are
stored contiguously.

As of yesterday's Library Working Group vote, it does. Subject to
approval by the full Standards Committee tomorrow.

Cool. That's a valuable piece of information.

BTW: are those decisions of the LWG / Standards Committee published
somewhere (if so, a pointer would be highly appreciated) or do mortals like
me have to wait until C++0X comes out?
Thanks

Kai-Uwe Bux

[Pete Becker]

Kai-Uwe Bux wrote:

BTW: are those decisions of the LWG / Standards Committee published
somewhere (if so, a pointer would be highly appreciated) or do mortals like
me have to wait until C++0X comes out?

The day-by-day decisons aren't publicly available, but if you check the
LWG issues lists you can see which ones have been marked DR, which means
we've agreed on a fix, or WP, which means the fix is in the Working
Draft (formerly Working Paper, hence WP). Don't take the difference too
seriously: often things that are marked DR don't get changed to WP when
they've gone into the draft. After each meeting there's a new Working
Draft, and they're available in the papers section on the WG21 web site.
That's www.open-std.org/jtc1/sc22/wg21.

--

Pete Becker
Roundhouse Consulting, Ltd.

[Alf P. Steinbach]

* Pete Becker:

Kai-Uwe Bux wrote:

BTW: are those decisions of the LWG / Standards Committee published
somewhere (if so, a pointer would be highly appreciated) or do mortals
like
me have to wait until C++0X comes out?

The day-by-day decisons aren't publicly available, but if you check the
LWG issues lists you can see which ones have been marked DR, which means
we've agreed on a fix, or WP, which means the fix is in the Working
Draft (formerly Working Paper, hence WP). Don't take the difference too
seriously: often things that are marked DR don't get changed to WP when
they've gone into the draft. After each meeting there's a new Working
Draft, and they're available in the papers section on the WG21 web site.
That's www.open-std.org/jtc1/sc22/wg21.

Presumably this issue, about contiguous strings, is the issue numbered
530, "Must elements of a string be contiguous?", in <url:
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html>.

That's not marked as being decided in any way.

Am I looking in the wrong place?

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

[Pete Becker]

Alf P. Steinbach wrote:

Presumably this issue, about contiguous strings, is the issue numbered
530, "Must elements of a string be contiguous?", in <url:
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html>.

That's not marked as being decided in any way.

Am I looking in the wrong place?

Yes, in the four-dimensional space-time continuum. The active issues
list dated Feb. 24, 2006 does not contain changes made after it was written.

--

Pete Becker
Roundhouse Consulting, Ltd.