I know XML is more (processor) costly than cPickle, but how bad is it?
The idea is I want to store data that can be described as XML into my
database as cPickle objects. Except my web framework has no support for
BLOB datatype yet, and I might have to go with XML.
Ideas are appreciated,
Thanks,
Mike
2  1754 
[Mike] I know XML is more (processor) costly than cPickle, but how bad is it?
Are you sure you know that?
I'd guess that XML serialisation with cElementTree is both cpu and
memory competitive with cpickle, if not superior. Although I'm too lazy
to fire up the timeit module right now :-)
Also, how quickly the relevant parsers work depends on the input, i.e.
your data structures. Only you can take measurements with your data
structures ....
The idea is I want to store data that can be described as XML
can != should
into my
database as cPickle objects. Except my web framework has no support for
BLOB datatype yet, and I might have to go with XML.
Or you could encode the binary pickle in a text-safe encoding such as
base64, and store the result in a text column. Although that will
obviously increase your processing time, both going in and out of the
database.
Ideas are appreciated,
I'd write a few simple prototypes and take some empirical measurements.
HTH,
--
alan kennedy
------------------------------------------------------
email alan: http://xhaus.com/contact/alan
> I'd guess that XML serialisation with cElementTree is both cpu and memory competitive with cpickle, if not superior. Although I'm too lazy
to fire up the timeit module right now :-)
That maybe true, but I bet Marshal is the fastest. ...right?
Also, how quickly the relevant parsers work depends on the input, i.e.
your data structures. Only you can take measurements with your data
structures ....
True.
The idea is I want to store data that can be described as XML
can != should
I certainly 'can', I don't think I should.
into my
database as cPickle objects. Except my web framework has no support for
BLOB datatype yet, and I might have to go with XML.
Or you could encode the binary pickle in a text-safe encoding such as
base64, and store the result in a text column. Although that will
obviously increase your processing time, both going in and out of the
database.
base64... (used to convert arbitrary binary data to plain text), sounds
fantastic. Except, I don't think I need it when marshaling anymore
since marshaling gives you clear text anyways. (right?)
Ideas are appreciated,
I'd write a few simple prototypes and take some empirical measurements.
I am doing it now. Thanks,
Mike This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Drochom |
last post by:
Hello,
I have a huge problem with loading very simple structure into memory
it is a list of tuples, it has 6MB and consists of 100000 elements
>import cPickle
>plik = open("mealy","r")
>mealy = cPickle.load(plik)
>plik.close()
|
by: Guenter Walser |
last post by:
Hello,
When using the codeline:
pickle.dump(self, file, pickle.HIGHEST_PROTOCOL)
my program runs perfectly fine:
--------------------------- Testprotocol
-----------------------------
|
by: Richard Kessler |
last post by:
I am attempting a GUI using BOA Constructor. I have some simple code to
pickle an object, but for some reason when I use cPickle it hangs the
system, but pickle works just fine. I do not have a clue why cPickle will
not work.
The code is
myObj = MyObject()
myObj.SetSomeProperties
f = open("my file name",'w')
|
by: sh |
last post by:
Hi guys,
Well, I have a (maybe dumb) question.
I want to write my own little blog using Python (as a fairly small but doable
project for myself to learn more deaply Python in a web context).
I don't want so far to use a database as a backend, I'd prefer use XML which is
enough for a small amount of data the blog would have to deal with.
|
by: Alex Polite |
last post by:
I need to put recursive data structures on disc and found out that cPickle
doesn't like recursion.
What are my options?
alex
--
Alex Polite
http://polite.se
| | |
by: A.B., Khalid |
last post by:
I wonder if someone can explain what is wrong here. I am pickling a
list of dictionaries (see code attached) and unpickling it back using
the HIGHEST_PROTOCOL of pickle and cPickle. I am getting an error
message and trace backs if the list exceeds eight items. Whether I use
pickle or cPickle does not matter, i.e., the eight number causes a
problem in both modules, although the trace backs are of course
dissimilar.
This pickling and...
|
by: Marcus Lowland |
last post by:
Hello, I'm fairly new to python and have read about and wanted to begin
experimenting with cpickle. As I understand, this should be a native
module in the python library. I have python 2.3 and now just installed
2.4, but am not able to import or find cpickle.py in any directory of
the install, or in the previous version (pickle.py is present and
imports correctly). Is there a seperate module package that must be
downloaded and installed......
|
by: Mingus Tsai |
last post by:
Hello- please help with unpickling problem:
I am using Python version 2.3.4 with IDLE version 1.0.3 on a Windows
XPhome system.
My problem is with using cPickle to deserialize my pickled arrays of
datetime.datetime instances. The following is the code I have written:
import cPickle, datetime
import Numeric
|
by: Carl J. Van Arsdall |
last post by:
Hey everyone, cPickle is raising an ImportError that I just don't quite
understand. Before I paste the code, let me explain the application.
Basically the part of the application that failed is a function that
loads a list of objects from a file using cPickle. This list is a queue
of requests. I've done some research and it looks like cPickle tries to
load some modules as some kind of test. From what I can tell the module
that cPickle...
|
by: Victor Kryukov |
last post by:
Hello list,
The following behavior is completely unexpected. Is it a bug or a by-
design feature?
Regards,
Victor.
-----------------
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| | |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
| | |
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
