I need to store pickled objects in postgresql. I reading through the pickle
docs it says to always open a file in binary mode because you can't be sure
if the pickled data is binary or text. So I have 2 question. Can I set the
pickle to be text -- and then store it in a 'text' type field in my sql
table, or should what sql type should I save the pickle as?
--
David Bear
-- let me buy your intellectual property, I want to own your thoughts -- 6 8138
En Tue, 10 Jul 2007 20:32:01 -0300, David Bear <da********@asu.edu>
escribió:
I need to store pickled objects in postgresql. I reading through the
pickle
docs it says to always open a file in binary mode because you can't be
sure
if the pickled data is binary or text. So I have 2 question. Can I set
the
pickle to be text -- and then store it in a 'text' type field in my sql
table, or should what sql type should I save the pickle as?
I'd use a binary datatype (raw, blob, binary, whatever postgres calls it).
Text columns might be converted or reencoded in some way, binary data
should never be modified in any way.
--
Gabriel Genellina
David Bear <da********@asu.eduwrote:
I need to store pickled objects in postgresql. I reading through the pickle
docs it says to always open a file in binary mode because you can't be sure
if the pickled data is binary or text. So I have 2 question. Can I set the
pickle to be text -- and then store it in a 'text' type field in my sql
table, or should what sql type should I save the pickle as?
You could always encode it into text form, eg
>>from cPickle import dumps, loads a = range(10) a
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>b = dumps(a).encode("zip").encode("base64").strip() b
'eJzTyCkw5PI04Er0NARiIyA2BmITIDYFYjMgNgdiCyC25ErUA wD5DQqD'
>>loads(b.decode("base64").decode("zip"))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick
On 7/11/07, Nick Craig-Wood <ni**@craig-wood.comwrote:
David Bear <da********@asu.eduwrote:
I need to store pickled objects in postgresql. I reading through the pickle
docs it says to always open a file in binary mode because you can't be sure
if the pickled data is binary or text. So I have 2 question. Can I set the
pickle to be text -- and then store it in a 'text' type field in my sql
table, or should what sql type should I save the pickle as?
You could always encode it into text form, eg
>>from cPickle import dumps, loads
>>a = range(10)
>>a
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>b = dumps(a).encode("zip").encode("base64").strip()
>>b
'eJzTyCkw5PI04Er0NARiIyA2BmITIDYFYjMgNgdiCyC25ErUA wD5DQqD'
>>loads(b.decode("base64").decode("zip"))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
Protocol 0 (the default) is a text protocol, it's safe to store in a
text field or write to a text file. base64 encoding will work on
protocol 1 and 2 (which are binary protocols and not text-safe), but
doing that removes the main benefit of the higher protocols, which is
smaller pickle data.
On Wed, 11 Jul 2007 17:14:43 -0500, Chris Mellon wrote:
[pickle]
Protocol 0 (the default) is a text protocol, it's safe to store in a
text field or write to a text file.
It's not really a text protocol it's more a binary protocol that uses just
the ASCII range of byte values. You have to write and read the "text"
files in binary mode or they break if taken across platform boundaries
because of the different line endings in Linux and Windows for instance.
Ciao,
Marc 'BlackJack' Rintsch
On 12 Jul 2007 06:00:59 GMT, Marc 'BlackJack' Rintsch <bj****@gmx.netwrote:
On Wed, 11 Jul 2007 17:14:43 -0500, Chris Mellon wrote:
[pickle]
Protocol 0 (the default) is a text protocol, it's safe to store in a
text field or write to a text file.
It's not really a text protocol it's more a binary protocol that uses just
the ASCII range of byte values. You have to write and read the "text"
files in binary mode or they break if taken across platform boundaries
because of the different line endings in Linux and Windows for instance.
It's fine as long as you use universal line endings mode.
En Thu, 12 Jul 2007 14:18:11 -0300, Chris Mellon <ar*****@gmail.com>
escribió:
On 12 Jul 2007 06:00:59 GMT, Marc 'BlackJack' Rintsch <bj****@gmx.net>
wrote:
>On Wed, 11 Jul 2007 17:14:43 -0500, Chris Mellon wrote:
[pickle]
Protocol 0 (the default) is a text protocol, it's safe to store in a
text field or write to a text file.
It's not really a text protocol it's more a binary protocol that uses just the ASCII range of byte values. You have to write and read the "text" files in binary mode or they break if taken across platform boundaries because of the different line endings in Linux and Windows for instance.
It's fine as long as you use universal line endings mode.
Neither. Won't work for Unicode objects then. See bug#1724366
<http://python.org/sf/1724366>
--
Gabriel Genellina This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Erik Max Francis |
last post by:
Is there any prohibition against mixing different protocols within the
same pickle? I don't see anything about this in the Python Library
Reference and, after all, the pickle.dump function takes a...
|
by: tuvok |
last post by:
How can objects of different types be stored in a collection?
For example:
struct S1 { /*...*/ };
struct S2 { /*...*/ };
struct Sn { /*...*/ };
template<typename T> class X
{
public:
|
by: Ann Huxtable |
last post by:
Hi,
I am writing a nested table structure to mimic data returned from a
database (actually, a heirarchical recordset).
I am representing the cells where the actual data is stored, by a union:...
|
by: Jacek Dziedzic |
last post by:
Hello!
Suppose I have a class that contains only public members
of builtin types and a default constructor. Because of the
default constructor it is no longer an aggregate and therefore
no...
|
by: jkugler |
last post by:
Hello,
I am trying to store huge amounts of data from xml files and put them
into a MySQL database. The xml files all are in this format:
<?xml version="1.0" encoding="UTF-8"?>...
|
by: tshad |
last post by:
I am trying to find the best procedure for storing keys used for encryption.
This would also be a question for the connection string to the database. At
the moment, this is kept in the web.info...
|
by: icemani |
last post by:
Is there any advantages that storing data in hexadecimal octal formate in data base?
By Manikandan.
|
by: ashokkuncha |
last post by:
hi all,
i am new bee and hope some one help me out.
in my web application i am using access as data base, one table of the dabase has candidates details as records ( emp no, name , skill set...
|
by: Josip |
last post by:
I'm trying to limit a value stored by object (either int or float):
class Limited(object):
def __init__(self, value, min, max):
self.min, self.max = min, max
self.n = value
def...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: Faith0G |
last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
|
by: ryjfgjl |
last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
| |