473,248 Members | 1,165 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,248 software developers and data experts.

unicode memory usage

We have an application which involves storing a lot of strings in RAM. It
would be most convenient to use Unicode strings, but I am wary of doubling
memory usage. My fear is based on the idea that unicode strings may take two
bytes per character in order to accomodate non-ascii characters.

But I don't know whether that's actually how Python strings work internally.

So, my question: Do unicode strings in Python take substantially more memory
than classic python strings or not, assuming the strings are generally 99%
ASCII characters (but not 100%)?
--Gary

--
Putting http://wecanstopspam.org in your email helps it pass through
overzealous spam filters.

Gary Robinson
CEO
Transpose, LLC
gr*******@transpose.com
207-942-3463
http://www.transpose.com
http://radio.weblogs.com/0101454

Jul 18 '05 #1
1 2082
Gary Robinson wrote:
But I don't know whether that's actually how Python strings work internally.
Python Unicode objects use normally 2 bytes per character, unless Python
is built in UCS-4 mode, in which case they use 4 bytes per character.
So, my question: Do unicode strings in Python take substantially more memory
than classic python strings or not, assuming the strings are generally 99%
ASCII characters (but not 100%)?


Yes; you can expect that 99% of the storage for characters are null
bytes, then. Whether this is substantial depends on the total amount of
storage that you need for string objects, compared to the storage needed
for other things, or the storage available.

Regards,
Martin

Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

23
by: Hallvard B Furuseth | last post by:
Has someone got a Python routine or module which converts Unicode strings to lowercase (or uppercase)? What I actually need to do is to compare a number of strings in a case-insensitive manner,...
5
by: Justice | last post by:
Currently I'm doing some experimenting with the XMLHTTP object in Javascript. Now, the XMLHttp object is asynchronous (at least in this case), and the following code causes a significant memory...
2
by: tomvr | last post by:
Hello I have noticed some 'weird' memory usage in a vb.net windows app The situation is as follows I have an app (heavy on images) with 2 forms (actually there are more forms and on starting...
6
by: Tom | last post by:
We have a VERY simple .NET C# Form Application, that has about a 23MB Memory Footprint. It starts a window runs a process and does a regular expression. I have done a GC.Collect to make sure that,...
3
by: Ian Taite | last post by:
Hello, I'm exploring why one of my C# .NET apps has "high" memory usage, and whether I can reduce the memory usage. I have an app that wakes up and processes text files into a database...
20
by: Philip Carnstam | last post by:
How come .Net applications use so much memory? Every application I compile uses at least 10 MB of memory, even the ones consisting of only a form and nothing else. If I minimize them though the...
5
by: Tamir Khason | last post by:
I have a program will really big embedded text resources. Because of internationalization I have to save the embedded text in UTF-8, but it more then triple bigger then the original file. The last...
13
by: Tomás | last post by:
Let's start off with: class Nation { public: virtual const char* GetName() const = 0; } class Norway : public Nation { public: virtual const char* GetName() const
1
by: Jean-Paul Calderone | last post by:
On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yzghan@gmail.com wrote: The test doesn't demonstrate any leaks. It does demonstrate that memory usage can remain at or near peak memory usage even after...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.