471,896 Members | 1,868 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,896 software developers and data experts.

unicode memory usage

We have an application which involves storing a lot of strings in RAM. It
would be most convenient to use Unicode strings, but I am wary of doubling
memory usage. My fear is based on the idea that unicode strings may take two
bytes per character in order to accomodate non-ascii characters.

But I don't know whether that's actually how Python strings work internally.

So, my question: Do unicode strings in Python take substantially more memory
than classic python strings or not, assuming the strings are generally 99%
ASCII characters (but not 100%)?
--Gary

--
Putting http://wecanstopspam.org in your email helps it pass through
overzealous spam filters.

Gary Robinson
CEO
Transpose, LLC
gr*******@transpose.com
207-942-3463
http://www.transpose.com
http://radio.weblogs.com/0101454

Jul 18 '05 #1
1 2019
Gary Robinson wrote:
But I don't know whether that's actually how Python strings work internally.
Python Unicode objects use normally 2 bytes per character, unless Python
is built in UCS-4 mode, in which case they use 4 bytes per character.
So, my question: Do unicode strings in Python take substantially more memory
than classic python strings or not, assuming the strings are generally 99%
ASCII characters (but not 100%)?


Yes; you can expect that 99% of the storage for characters are null
bytes, then. Whether this is substantial depends on the total amount of
storage that you need for string objects, compared to the storage needed
for other things, or the storage available.

Regards,
Martin

Jul 18 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

23 posts views Thread by Hallvard B Furuseth | last post: by
2 posts views Thread by tomvr | last post: by
20 posts views Thread by Philip Carnstam | last post: by
5 posts views Thread by Tamir Khason | last post: by
13 posts views Thread by Tomás | last post: by
1 post views Thread by Jean-Paul Calderone | last post: by
reply views Thread by zermasroor | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.