473,898 Members | 3,135 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to convert arbitrary objects directly to base64 without initial string conversion?

I've got a case where I want to convert binary blocks of data (various
ctypes objects) to base64 strings.

The conversion calls in the base64 module expect strings as input, so
right now I'm converting the binary blocks to strings first, then
converting the resulting string to base64. This seems highly
inefficient and I'd like to just go straight from binary to a base64
string.

Here is the conversion we're using from object to string...

import ctypes
def ObjAsString(obj ):
sz = ctypes.sizeof(o bj)
charArray = ctypes.c_char * sz
buf = charArray.from_ address(ctypes. addressof(obj))
return buf.raw[:sz]

The returned string can then be sent to base64 for conversion (although
we're actually using xmlrpc.Binary), but there is obviously some waste
in here.

import base64
b64 = base64.b64encod e(ObjAsString(f oo))

Is there a canned/pre-existing way to convert a block of memory to a
base64 string more efficiently? I'd like to avoid writing my own
base64 conversion routine if possible. Anyone have any good ideas?
Even a mroe efficient/less clunky way of conevrting an arbitrary object
to a string would be appreciated.

Thanks,
Russ

Jul 13 '06 #1
4 5559
Russell Warren schrieb:
I've got a case where I want to convert binary blocks of data (various
ctypes objects) to base64 strings.

The conversion calls in the base64 module expect strings as input, so
right now I'm converting the binary blocks to strings first, then
converting the resulting string to base64. This seems highly
inefficient and I'd like to just go straight from binary to a base64
string.
[...]
Is there a canned/pre-existing way to convert a block of memory to a
base64 string more efficiently? I'd like to avoid writing my own
base64 conversion routine if possible. Anyone have any good ideas?
Even a mroe efficient/less clunky way of conevrting an arbitrary object
to a string would be appreciated.
Many functions that operate on strings also accept buffer objects as parameters,
this seems also be the case for the base64.encodest ring function. ctypes objects
support the buffer interface.

So, base64.b64encod e(buffer(ctypes _instance)) should work efficiently.

Thomas

Jul 13 '06 #2
Many functions that operate on strings also accept buffer objects as parameters,
this seems also be the case for the base64.encodest ring function. ctypes objects
support the buffer interface.

So, base64.b64encod e(buffer(ctypes _instance)) should work efficiently.
Thanks! I have never used (or even heard of) the buffer objects. I'll
check it out.

Russ

Jul 13 '06 #3
After some digging around it appears there is not a tonne of
documentation on buffer objects, although they are clearly core and
ancient... been sifting through some hits circa 1999, long before my
python introduction.

What I can find says that buffer is deprecated (Python in a Nutshell),
or non-essential/for-older-versions (Python documentation).

At least it no longer seems terribly weird to me that I never noticed
this built-in before... I got this from the python docs in reference to
buffer and others:

"Python programmers, trainers, students and bookwriters should feel
free to bypass these functions without concerns about missing something
important".

Is buffer safe to use? Is there an alternative?
ctypes objects support the buffer interface
How can you tell what objects support the buffer interface? Is
anything visible at the python level, or do you need to dig into the C
source?

Regarding documentation, I assume the C PyBufferObject is the
underlying thing for the python-level buffer? If so, is the best place
for docs on this ancient object to glean what I can from this link:
http://www.python.org/doc/1.5.2p2/ap...erObjects.html ?

Any help is appreciated... I'd like to understand what I can about this
object if I'm to use it... I'm wary of nasty surprises.

Russ

Jul 13 '06 #4
Russell Warren wrote:
How can you tell what objects support the buffer interface? Is
anything visible at the python level, or do you need to dig into the C
source?
At the C level, there is a function for testing:

int PyObject_CheckR eadBuffer(PyObj ect* o)

http://docs.python.org/dev/api/abstract-buffer.html

According to

http://docs.python.org/dev/lib/non-e...-in-funcs.html

a Python call to buffer(obj) will either return a buffer object or (tested)
raise a TypeError:
>>buffer("123 ")
<read-only buffer for 0x2b597d52b9c0, size -1, offset 0 at 0x2b597d52e340>
>>buffer(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: buffer object expected

Stefan
Jul 13 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
3752
by: praba kar | last post by:
Dear All, I have doubt regarding date string to time conversion function. In Python I cannot find flexible date string conversion function like php strtotime. I try to use following type function 1) parsedate function failed if a date string like below format "16-MAY-2005 01:26" 2)parsedate_tz function also failed if a date string
3
3371
by: RC | last post by:
Just wondering if there are good reasons (performance, ?) to use one of these string conversion alternatives over the other: 1. Convert.ToString() 2. someObject.ToString() Example: Convert.ToString(DataRow1) vs. DataRow1.ToString()
2
31498
by: Diablo | last post by:
Hi, I am converting a byte array to string using System.Text.Encoding.UTF8.GetString() or Unicode.GetString(). Then the resulting text is converted back to binary, using the reverse function: System.Text.Encoding.UTF8.GetBytes() or Unicode.GetBytes. But the result does not match the original byte array! Is the conversion loosing some characters? Why? Thanks,
1
21166
by: James dean | last post by:
The following code is the current method i have for converting an image to Base64.Is this the most efficient way to convert an image to base64 or are there others. The Image class's Save method is slow here and i am wondering if i can do it some other way to make it faster.....Any ideas public string ImageToBase64String(Image imageData ImageFormat format) { string base64; MemoryStream memory = new MemoryStream(); imageData.Save(memory,...
18
34165
by: Ger | last post by:
I have not been able to find a simple, straight forward Unicode to ASCII string conversion function in VB.Net. Is that because such a function does not exists or do I overlook it? I found Encoding.Convert, but that needs byte arrays. Thanks, /Ger
7
2938
by: LuB | last post by:
I have to convert a wide (16bit) char to a standard (8bit) char. Please nevermind that I am using WideCharToMultiByte ... but my question is as follows: Does this approach make any sense? My main thought is I hate allocating a new char* buf ... and then simply assigning it again to the std:;string out parameter. I'd prefer to REPLACE the underlying char* in the std::string or somehow explicitly reallocate the std::string's internal...
28
1622
by: walterbyrd | last post by:
Python seems to have a log of ways to do collections of arbitrary objects: lists, tuples, dictionaries. But what if I want a collection of non-arbitrary objects? A list of records, or something like that?
2
4982
by: Saeed Amrollahi | last post by:
Dear All Hi I usually use the following function for built-in types to std::string conversion: // convert a type (typically built-in type) to std::string template<class T> inline std::string ToStdStr(const T& t) { std::istringstream s;
9
3736
by: jl_post | last post by:
Hi, A few months back I remember reading through C++ newsgroups trying to find a way to quickly convert a number to a C++ std::string. I often see code like: // Create a string that holds a number: int num = 7; // for example char tmp; sprintf(tmp, "%d", num);
0
9993
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
1
10954
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10487
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
8036
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7191
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5882
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4708
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4297
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3308
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.