Managed String Library

[rCs]

The SEI has published CMU/SEI-2006-TR-006 "Specifications for Managed
Strings" and released a "proof-of-concept" implementation of the
managed string library.

The specification, source code for the library, and other resources
related to managed strings are available for download from the CERT web
site at:

http://www.cert.org/secure-coding/managedstring.html

The following is a brief summary of the managed string library:

The managed string library was developed in response to the need for a
string library that can improve the quality and security of newly
developed C-language programs while eliminating obstacles to widespread
adoption and possible standardization. As the name implies, the managed
string library is based on a dynamic approach; memory is allocated and
reallocated as required. This approach eliminates the possibility of
unbounded copies, null-termination errors, and truncation by ensuring
that there is always adequate space available for the resulting string
(including the terminating null character). The one exception is if
memory is exhausted; that is treated as an error condition. In this
way, the managed string library accomplishes the goal of indicating
either success or failure. The managed string library also protects
against improper data sanitization by (optionally) ensuring that all
characters in a string belong to a predefined set of "safe" characters.

rCs

--
Robert C. Seacord
Senior Vulnerability Analyst
CERT/CC

[jacob navia]

rCs wrote:

The SEI has published CMU/SEI-2006-TR-006 "Specifications for Managed
Strings" and released a "proof-of-concept" implementation of the
managed string library.

The specification, source code for the library, and other resources
related to managed strings are available for download from the CERT web
site at:

http://www.cert.org/secure-coding/managedstring.html

The following is a brief summary of the managed string library:

The managed string library was developed in response to the need for a
string library that can improve the quality and security of newly
developed C-language programs while eliminating obstacles to widespread
adoption and possible standardization. As the name implies, the managed
string library is based on a dynamic approach; memory is allocated and
reallocated as required. This approach eliminates the possibility of
unbounded copies, null-termination errors, and truncation by ensuring
that there is always adequate space available for the resulting string
(including the terminating null character). The one exception is if
memory is exhausted; that is treated as an error condition. In this
way, the managed string library accomplishes the goal of indicating
either success or failure. The managed string library also protects
against improper data sanitization by (optionally) ensuring that all
characters in a string belong to a predefined set of "safe" characters.

rCs

This is a much better approach than TR 24731. No longer is the
programmer required to keep track of the length of the strings,
an essential point if we are going to make things more sure.

Up to minor details this is the same principle of lcc-win32's
implementation of a string library.

It would be nice if the people in the comitee looked at this.

jacob

[Robert Seacord]

Jacob,

Comments below.

This is a much better approach than TR 24731. No longer is the
programmer required to keep track of the length of the strings,
an essential point if we are going to make things more sure.

Up to minor details this is the same principle of lcc-win32's
implementation of a string library.

It would be nice if the people in the comitee looked at this.

I am on the WG14 committee and I have looked into it. 8^)

I don't believe this is a better approach than TR 24731--I think it
solves a different problem.

TR 24731 is a very good approach for remediation of existing code
because it offers close replacements to existing functions.

I think the managed strings library is a better approach for new
development--including the development of new modules or subsystems--for
the reasons you gave.

The WG14 committee reviewed my proposal at both the Mt. Tremblant and
Berlin meetings. The feedback I received was that it was a reasonable
technical solution but that there wasn't enough market demand yet to
publish this as a ISO/IEC TR.

rCs

[jacob navia]

Robert Seacord a écrit :

The WG14 committee reviewed my proposal at both the Mt. Tremblant and
Berlin meetings. The feedback I received was that it was a reasonable
technical solution but that there wasn't enough market demand yet to
publish this as a ISO/IEC TR.

Interesting. The evolution of C is now "market driven". What counts,
if I understand you correctly, is "market demand".

This comes as a surprise to me since companies like Microsoft feel
that it is worth to spend time and money researching this string
problem, and spending money in proposing alternatives.

Of course I suppose that the standards comitee has a better researched
investigation of the market than Microsoft.

This assertion that "there isn't enough market demand" then,
is not just based on the "feelings of the comitee members"
but in hard won market research that was done previously to that
decision.

It would be interesting to see it.

How does the C comitee evaluate the "market demands"?

jacob

[Andrew Poelstra]

On 2006-06-15, jacob navia <ja***@jacob.remcomp.fr> wrote:

Robert Seacord a écrit :

The WG14 committee reviewed my proposal at both the Mt. Tremblant and
Berlin meetings. The feedback I received was that it was a reasonable
technical solution but that there wasn't enough market demand yet to
publish this as a ISO/IEC TR.

Interesting. The evolution of C is now "market driven". What counts,
if I understand you correctly, is "market demand".

Judging by C99's reception, yes.
This comes as a surprise to me since companies like Microsoft feel
that it is worth to spend time and money researching this string
problem, and spending money in proposing alternatives.
MS also felt it was worth it to spend time and money on IE5, 6, and 7, as well
as Visual Studio itself, .NET, C#...

Think "crushing real standards".
Of course I suppose that the standards comitee has a better researched
investigation of the market than Microsoft.
Considering MS's biased research on everything (most notably server
performance), yes.
This assertion that "there isn't enough market demand" then,
is not just based on the "feelings of the comitee members"
but in hard won market research that was done previously to that
decision.

It would be interesting to see it.

How does the C comitee evaluate the "market demands"?

Perhaps by counting the number of Standard purchases? I hear that C89 is
*still* more popular than C99.

--
Andrew Poelstra < http://www.wpsoftware.net/blog >
To email me, use "apoelstra" at the above address.
I know that area of town like the back of my head.

[Malcolm]

"jacob navia" <ja***@jacob.remcomp.fr> wrote in message

Interesting. The evolution of C is now "market driven". What counts,
if I understand you correctly, is "market demand".

This comes as a surprise to me since companies like Microsoft feel
that it is worth to spend time and money researching this string
problem, and spending money in proposing alternatives.

Of course I suppose that the standards comitee has a better researched
investigation of the market than Microsoft.

Software companies want to sell versions of their particular compiler,
whilst programmers want tools that will help them do their jobs.

From a programmer's point of view, a standard is usually a good thing. It
means that they know how a piece of software will work, that the same
software can be compiled by different compilers, that they don't have to
constantly learn new languages.

The compiler vendor's attitude is a bit more ambivalent. If everyone merely
releases an ANSI standard C compiler, the cheapest one will win. Marketing
people like to be able to differentiate their product. They want
interoperability in so far as it helps people move from rival products to
their own, but not if it helps people go in the reverse direction, except
that they cannot ignore user demands entirely.

Things like a "better" string library which break compatibility with
virtually every program written in standard C are a dream for a compiler
vendor. They are good for the customer only if the benefits of the library
outweigh all the costs. Sometimes this may be the case, for instance if you
only write software for Microsoft operating systems, or if efficiency in
string handling is paramount. Often, however, the vendor is the one who is
winning.
--
Buy my book 12 Common Atheist Arguments (refuted)
$1.25 download or $7.20 paper, available www.lulu.com/bgy1mm

[jacob navia]

Malcolm a écrit :

"jacob navia" <ja***@jacob.remcomp.fr> wrote in message

Interesting. The evolution of C is now "market driven". What counts,
if I understand you correctly, is "market demand".

This comes as a surprise to me since companies like Microsoft feel
that it is worth to spend time and money researching this string
problem, and spending money in proposing alternatives.

Of course I suppose that the standards comitee has a better researched
investigation of the market than Microsoft.

Software companies want to sell versions of their particular compiler,
whilst programmers want tools that will help them do their jobs.

From a programmer's point of view, a standard is usually a good thing. It
means that they know how a piece of software will work, that the same
software can be compiled by different compilers, that they don't have to
constantly learn new languages.

The compiler vendor's attitude is a bit more ambivalent. If everyone merely
releases an ANSI standard C compiler, the cheapest one will win. Marketing
people like to be able to differentiate their product. They want
interoperability in so far as it helps people move from rival products to
their own, but not if it helps people go in the reverse direction, except
that they cannot ignore user demands entirely.

Things like a "better" string library which break compatibility with
virtually every program written in standard C are a dream for a compiler
vendor. They are good for the customer only if the benefits of the library
outweigh all the costs. Sometimes this may be the case, for instance if you
only write software for Microsoft operating systems, or if efficiency in
string handling is paramount. Often, however, the vendor is the one who is
winning.

You misunderstood the problem Malcolm.

This is about a STANDARD string library precisely, not tied to a single
vendor.

Microsoft proposed a STANDARD library, i.e. proposed to agree with all
other compiler vendors in a common set of string functions.

Besides Microsoft, the CERT also worked in this direction. What I
am complaining is that the comitee refuses to publish a standard
string library that goes beyond the awful string functions that we
have in C.

With NO reason, just vague "market is not there" claims, that are
NOT backed up with an investigation of the market!

[Malcolm]

"jacob navia" <ja***@jacob.remcomp.fr> wrote in message
news:44*********************@news.orange.fr...

Malcolm a écrit :

"jacob navia" <ja***@jacob.remcomp.fr> wrote in message

Interesting. The evolution of C is now "market driven". What counts,
if I understand you correctly, is "market demand".

This comes as a surprise to me since companies like Microsoft feel
that it is worth to spend time and money researching this string
problem, and spending money in proposing alternatives.

Of course I suppose that the standards comitee has a better researched
investigation of the market than Microsoft.

Software companies want to sell versions of their particular compiler,
whilst programmers want tools that will help them do their jobs.

From a programmer's point of view, a standard is usually a good thing. It
means that they know how a piece of software will work, that the same
software can be compiled by different compilers, that they don't have to
constantly learn new languages.

The compiler vendor's attitude is a bit more ambivalent. If everyone
merely releases an ANSI standard C compiler, the cheapest one will win.
Marketing people like to be able to differentiate their product. They
want interoperability in so far as it helps people move from rival
products to their own, but not if it helps people go in the reverse
direction, except that they cannot ignore user demands entirely.

Things like a "better" string library which break compatibility with
virtually every program written in standard C are a dream for a compiler
vendor. They are good for the customer only if the benefits of the
library outweigh all the costs. Sometimes this may be the case, for
instance if you only write software for Microsoft operating systems, or
if efficiency in string handling is paramount. Often, however, the vendor
is the one who is winning.

You misunderstood the problem Malcolm.

This is about a STANDARD string library precisely, not tied to a single
vendor.

Microsoft proposed a STANDARD library, i.e. proposed to agree with all
other compiler vendors in a common set of string functions.

Besides Microsoft, the CERT also worked in this direction. What I
am complaining is that the comitee refuses to publish a standard
string library that goes beyond the awful string functions that we
have in C.

With NO reason, just vague "market is not there" claims, that are
NOT backed up with an investigation of the market!

Really?
I've obviously misjudged them.
I'm a bit sour because I was investigating the possibility of a new PC to
run their new OS on in Beta. Microsoft Publisher retails at about 170
pounds. Everything else I can get for a reasonable price through the
university. Publisher I just need to do a few bits and pieces, but because
of some licensing agreement the university won't sell it to me. This is MS
all over - they charge absurd prices which virtually no one pays, and
everyone goes through irritating little hoops to make things work.

I simply assumed that the string library would be another conspiracy against
the community.
--
Buy my book 12 Common Atheist Arguments (refuted)
$1.25 download or $7.20 paper, available www.lulu.com/bgy1mm