How do I converted a null (0) terminated string to a Python string?

[Michael]

Hi All,

I've received (via UDP) a null terminated string and need to convert it
into a Python string. Can anyone tell me how this is done? If it helps,
I know the number of characters in the string.

Thanks,
M. McDonnell

[Steve Holden]

Michael wrote:

Hi All,

I've received (via UDP) a null terminated string and need to convert it
into a Python string. Can anyone tell me how this is done? If it helps,
I know the number of characters in the string.

Thanks,
M. McDonnell

Have you received this string in Python or in C? If the former, then
just throw away the last character of the string you've received and
you're done!

s = s[:-1]

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[John Machin]

Michael wrote:

Hi All,

I've received (via UDP) a null terminated string and need to convert it
into a Python string. Can anyone tell me how this is done? If it helps,
I know the number of characters in the string.

I think you mean NUL, not null.

What have you received it into, if it's not a Python string?

You probably need/want this:

if strg[-1] == "\0":
strg = strg[:-1]
alternatively:
strg = strg.rstrip("\0") # requires Python 2.2.2 or later

It's possible you may be talking about a fixed length string which
contains useful_stuff + "\0" + padding -- in that case you need

strg = strg.split("\0")[0] # grab upto (but not including) the first
NUL (if any)

If you're not sure what you've got, print repr(the_input_string)

HTH,
John

[Michael]

Thank you very much for your responses. To answer some of the
questions... Yes, I am in Python receiving a C language 0 terminated
string that was sent to my Python program in a UDP packet (which is how
I know the count). Are your responses still correct given this
clarification?

Thanks much,
MDM

John Machin wrote:

Michael wrote:

Hi All,

I've received (via UDP) a null terminated string and need to convert it
into a Python string. Can anyone tell me how this is done? If it helps,
I know the number of characters in the string.

I think you mean NUL, not null.

What have you received it into, if it's not a Python string?

You probably need/want this:

if strg[-1] == "\0":
strg = strg[:-1]
alternatively:
strg = strg.rstrip("\0") # requires Python 2.2.2 or later

It's possible you may be talking about a fixed length string which
contains useful_stuff + "\0" + padding -- in that case you need

strg = strg.split("\0")[0] # grab upto (but not including) the first
NUL (if any)

If you're not sure what you've got, print repr(the_input_string)

HTH,
John

[John Machin]

Michael top-posted [corrected]:

John Machin wrote:

Michael wrote:

Hi All,
>
I've received (via UDP) a null terminated string and need to convert it
into a Python string. Can anyone tell me how this is done? If it helps,
I know the number of characters in the string.
>

I think you mean NUL, not null.

What have you received it into, if it's not a Python string?

You probably need/want this:

if strg[-1] == "\0":
strg = strg[:-1]
alternatively:
strg = strg.rstrip("\0") # requires Python 2.2.2 or later

It's possible you may be talking about a fixed length string which
contains useful_stuff + "\0" + padding -- in that case you need

strg = strg.split("\0")[0] # grab upto (but not including) the first
NUL (if any)

If you're not sure what you've got, print repr(the_input_string)

HTH,
John

Thank you very much for your responses. To answer some of the
questions... Yes, I am in Python receiving a C language 0 terminated
string that was sent to my Python program in a UDP packet (which is how
I know the count). Are your responses still correct given this
clarification?

My responses are correct. Your "clarification" indicates to me that you
are going by what you are told, not by inspection of (several instances
of) the packet contents, using repr(). It's up to you whether you want
to be skeptical about the packet contents or not. I certainly wouldn't
be throwing the last byte away without checking that it was in fact a
NUL.

Cheers,
John

[Michael]

John,

Thanks for your reply. Just wondering... how are Python strings
formatted? Evidently they're not 0 terminated.

Thanks again,
MDM

John Machin wrote:

Michael top-posted [corrected]:

John Machin wrote:

Michael wrote:
Hi All,

I've received (via UDP) a null terminated string and need to convert it
into a Python string. Can anyone tell me how this is done? If it helps,
I know the number of characters in the string.

>
I think you mean NUL, not null.
>
What have you received it into, if it's not a Python string?
>
You probably need/want this:
>
if strg[-1] == "\0":
strg = strg[:-1]
alternatively:
strg = strg.rstrip("\0") # requires Python 2.2.2 or later
>
It's possible you may be talking about a fixed length string which
contains useful_stuff + "\0" + padding -- in that case you need
>
strg = strg.split("\0")[0] # grab upto (but not including) the first
NUL (if any)
>
If you're not sure what you've got, print repr(the_input_string)
>
HTH,
John

Thank you very much for your responses. To answer some of the
questions... Yes, I am in Python receiving a C language 0 terminated
string that was sent to my Python program in a UDP packet (which is how
I know the count). Are your responses still correct given this
clarification?

My responses are correct. Your "clarification" indicates to me that you
are going by what you are told, not by inspection of (several instances
of) the packet contents, using repr(). It's up to you whether you want
to be skeptical about the packet contents or not. I certainly wouldn't
be throwing the last byte away without checking that it was in fact a
NUL.

Cheers,
John

[John Machin]

Michael top-posted [again]:

>
John Machin wrote:

Michael top-posted [corrected]:

John Machin wrote:
Michael wrote:
Hi All,
>
I've received (via UDP) a null terminated string and need to convert it
into a Python string. Can anyone tell me how this is done? If it helps,
I know the number of characters in the string.
>

I think you mean NUL, not null.

What have you received it into, if it's not a Python string?

You probably need/want this:

if strg[-1] == "\0":
strg = strg[:-1]
alternatively:
strg = strg.rstrip("\0") # requires Python 2.2.2 or later

It's possible you may be talking about a fixed length string which
contains useful_stuff + "\0" + padding -- in that case you need

strg = strg.split("\0")[0] # grab upto (but not including) the first
NUL (if any)

If you're not sure what you've got, print repr(the_input_string)

HTH,
John
Thank you very much for your responses. To answer some of the
questions... Yes, I am in Python receiving a C language 0 terminated
string that was sent to my Python program in a UDP packet (which is how
I know the count). Are your responses still correct given this
clarification?

My responses are correct. Your "clarification" indicates to me that you
are going by what you are told, not by inspection of (several instances
of) the packet contents, using repr(). It's up to you whether you want
to be skeptical about the packet contents or not. I certainly wouldn't
be throwing the last byte away without checking that it was in fact a
NUL.

Cheers,
John

John,

Thanks for your reply. Just wondering... how are Python strings
formatted? Evidently they're not 0 terminated.

A Python string is an object. The details of the internal storage may
vary between implementations. CPython has 8-bit str objects and 16-bit
or 32-bit Unicode objects. In IronPython, (str is Unicode) is true, and
they are 16 bits. In any case the object knows its own length without
having to scan for a terminator. Thus, a string can contain NULs.

Having said all that, the CPython str implementation does have an
additional byte at the end; this is set to zero and is not counted in
the length. However you never see that and don't really need to know
unless you are writing an extension module in C -- it's handy to know
that you don't have to append a NUL if you want to call a C library
function.

Cheers,
John

[Fredrik Lundh]

Michael wrote:

Thanks for your reply. Just wondering... how are Python strings
formatted? Evidently they're not 0 terminated.

have you tried *printing* the thing you got via UDP?

to get a programmer-friendly representation of an arbitrary object, use

print repr(obj)

(where obj is your string, in this case).

</F>

[John Machin]

Fredrik Lundh wrote:

Michael wrote:

Thanks for your reply. Just wondering... how are Python strings
formatted? Evidently they're not 0 terminated.

have you tried *printing* the thing you got via UDP?

to get a programmer-friendly representation of an arbitrary object, use

print repr(obj)

(where obj is your string, in this case).

Probably not; there was no indication after the two messages where I
mentioned repr :-)

[Michael]

John,

Since I'm new to Python, I'm having trouble understanding what this
means (see below). Would appreciate any help.

if strg[-1] == "\0":
strg = strg[:-1]

Thanks,
MDM

John Machin wrote:

Fredrik Lundh wrote:

Michael wrote:

Thanks for your reply. Just wondering... how are Python strings
formatted? Evidently they're not 0 terminated.

have you tried *printing* the thing you got via UDP?

to get a programmer-friendly representation of an arbitrary object, use

print repr(obj)

(where obj is your string, in this case).

Probably not; there was no indication after the two messages where I
mentioned repr :-)

[John Machin]

Michael wrote:

John,

Since I'm new to Python, I'm having trouble understanding what this
means (see below). Would appreciate any help.

if strg[-1] == "\0":

If the last (i.e index -1) byte in the string equals ASCII NUL:

strg = strg[:-1]

then take a slice of the string from the start up to but not including
the last byte and assign that to "strg"

In other words, if the last byte of strg is NUL, throw it away.

The truly paranoid would code that as
if strg and strg[-1] etc etc
so that it wouldn't blow up if strg is empty -- strange things can
happen when you are reading other people's data :-)

Perhaps you should work through the tutorial; all the above concepts
are treated in this section:
http://docs.python.org/tut/node5.htm...00000000000000

HTH,
John

[Marc 'BlackJack' Rintsch]

In <11**********************@k70g2000cwa.googlegroups .com>, John Machin
wrote:

In other words, if the last byte of strg is NUL, throw it away.

The truly paranoid would code that as
if strg and strg[-1] etc etc
so that it wouldn't blow up if strg is empty -- strange things can
happen when you are reading other people's data :-)

I would spell it:

if strg.endswith('\0'):
strg = strg[:-1]

Ciao,
Marc 'BlackJack' Rintsch

[Richard Brodie]

"Marc 'BlackJack' Rintsch" <bj****@gmx.netwrote in message
news:pa****************************@gmx.net...

I would spell it:

if strg.endswith('\0'):
strg = strg[:-1]

I would just go with: strg = strg.rstrip('\0')

[Michael]

I guess, I still don't see how this will work. I'm receiving a C
zero-terminated string in my Python program as a 1K byte block (UDP
datagram). If the string sent was "abc", then what I receive in Python
is <a><b><c><0><garbage><garbage>...<last_garbage_byt e>. How is Python
going to know where in this 1K byte block the end of the string is? It
seems that what I need to do is tell Python that the string ends at
zero-relative index 3. What am I missing here?

Marc 'BlackJack' Rintsch wrote:

In <11**********************@k70g2000cwa.googlegroups .com>, John Machin
wrote:

In other words, if the last byte of strg is NUL, throw it away.

The truly paranoid would code that as
if strg and strg[-1] etc etc
so that it wouldn't blow up if strg is empty -- strange things can
happen when you are reading other people's data :-)

I would spell it:

if strg.endswith('\0'):
strg = strg[:-1]

Ciao,
Marc 'BlackJack' Rintsch

[Robert Kern]

Michael wrote:

I guess, I still don't see how this will work. I'm receiving a C
zero-terminated string in my Python program as a 1K byte block (UDP
datagram). If the string sent was "abc", then what I receive in Python
is <a><b><c><0><garbage><garbage>...<last_garbage_byt e>. How is Python
going to know where in this 1K byte block the end of the string is? It
seems that what I need to do is tell Python that the string ends at
zero-relative index 3. What am I missing here?

Nothing. This is what I would do:
In [34]: s
Out[34]: 'abc\x00garbage'

In [35]: s.split('\x00', 1)[0]
Out[35]: 'abc'

In [36]: s.split?
Type: builtin_function_or_method
Base Class: <type 'builtin_function_or_method'>
String Form: <built-in method split of str object at 0x6ada2c8>
Namespace: Interactive
Docstring:
S.split([sep [,maxsplit]]) -list of strings

Return a list of the words in the string S, using sep as the
delimiter string. If maxsplit is given, at most maxsplit
splits are done. If sep is not specified or is None, any
whitespace string is a separator.
Using the maxsplit argument saves split from having to do unnecessary work
splitting the garbage portion if there are nulls there, too.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Robert Kern]

Robert Kern wrote:

Michael wrote:

>I guess, I still don't see how this will work. I'm receiving a C
zero-terminated string in my Python program as a 1K byte block (UDP
datagram). If the string sent was "abc", then what I receive in Python
is <a><b><c><0><garbage><garbage>...<last_garbage_byt e>. How is Python
going to know where in this 1K byte block the end of the string is? It
seems that what I need to do is tell Python that the string ends at
zero-relative index 3. What am I missing here?

Nothing. This is what I would do:
In [34]: s
Out[34]: 'abc\x00garbage'

In [35]: s.split('\x00', 1)[0]
Out[35]: 'abc'

And I see that this is the advice that John Machin already gave you. Shame on me
for not reading the thread before replying.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Fredrik Lundh]

Marc 'BlackJack' Rintsch wrote:

I would spell it:

if strg.endswith('\0'):
strg = strg[:-1]

unless you're in a hurry; startswith and endswith are horribly
inefficient compared to ordinary indexing/slicing.

(as I've pointed out elsewhere, even "s[:len(t)] == t" is usually faster
than "s.startswith(t)" for short prefixes, where "short" is a lot longer
than you may think).

</F>

[Gabriel Genellina]

At Thursday 14/9/2006 11:45, Michael wrote:

>Since I'm new to Python, I'm having trouble understanding what this
means (see below). Would appreciate any help.

if strg[-1] == "\0":
strg = strg[:-1]

The Python Tutorial will answer your questions, it is enlightning and
easy to follow.

Gabriel Genellina
Softlab SRL

__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

[John Machin]

Michael wrote:

I guess, I still don't see how this will work. I'm receiving a C
zero-terminated string in my Python program as a 1K byte block (UDP
datagram). If the string sent was "abc", then what I receive in Python
is <a><b><c><0><garbage><garbage>...<last_garbage_byt e>. How is Python
going to know where in this 1K byte block the end of the string is? It
seems that what I need to do is tell Python that the string ends at
zero-relative index 3. What am I missing here?

You are missing this, contained in the third message in this thread
i.e. my first reply to you:
"""
It's possible you may be talking about a fixed length string which
contains useful_stuff + "\0" + padding -- in that case you need

strg = strg.split("\0")[0] # grab upto (but not including) the first
NUL (if any)
"""

Please note that Robert Kern's solution is better than the above (as he
says, it won't waste time splitting up the garbage, the mind-boggling
relative size of which I hadn't catered for).

You are also missing the frequent (3 at last count) exhortations to:
print repr(your_string)
which is much more precise than verbal descriptions.

Cheers,
John

[Michael]

Robert,

Thanks to you and everyone else for the help. The "s.split('\x00',
1)[0] " solved the problem.

Thanks again,
MDM

Robert Kern wrote:

Michael wrote:

I guess, I still don't see how this will work. I'm receiving a C
zero-terminated string in my Python program as a 1K byte block (UDP
datagram). If the string sent was "abc", then what I receive in Python
is <a><b><c><0><garbage><garbage>...<last_garbage_byt e>. How is Python
going to know where in this 1K byte block the end of the string is? It
seems that what I need to do is tell Python that the string ends at
zero-relative index 3. What am I missing here?

Nothing. This is what I would do:
In [34]: s
Out[34]: 'abc\x00garbage'

In [35]: s.split('\x00', 1)[0]
Out[35]: 'abc'

In [36]: s.split?
Type: builtin_function_or_method
Base Class: <type 'builtin_function_or_method'>
String Form: <built-in method split of str object at 0x6ada2c8>
Namespace: Interactive
Docstring:
S.split([sep [,maxsplit]]) -list of strings

Return a list of the words in the string S, using sep as the
delimiter string. If maxsplit is given, at most maxsplit
splits are done. If sep is not specified or is None, any
whitespace string is a separator.
Using the maxsplit argument saves split from having to do unnecessary work
splitting the garbage portion if there are nulls there, too.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[George Sakkis]

Michael wrote:

Robert,

Thanks to you and everyone else for the help. The "s.split('\x00',
1)[0] " solved the problem.

And a probably faster version: s[:s.index('\x00')]

George

[Robert Kern]

George Sakkis wrote:

Michael wrote:

>Robert,

Thanks to you and everyone else for the help. The "s.split('\x00',
1)[0] " solved the problem.

And a probably faster version: s[:s.index('\x00')]

Yup. About twice as fast for at least one dataset:
In [182]: import timeit

In [183]: t1 = timeit.Timer("s.split('\\x00', 1)[0]", "s='abc\\x00'*256")

In [184]: t2 = timeit.Timer("s[:s.index('\\x00')]", "s='abc\\x00'*256")

In [192]: t1.repeat(3, 100000)
Out[192]: [0.68063879013061523, 0.67146611213684082, 0.66347002983093262]

In [193]: t2.repeat(3, 100000)
Out[193]: [0.35819387435913086, 0.35968899726867676, 0.37595295906066895]

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco