Readable Persistance?

[Chris S.]

I'm trying to write programs that make use of a lot of dynamic
structures. To aid in debugging and data extraction, I'd like to persist
these objects in a form that's readable, but one that also preserves
handles (ie for lambda expressions, dictionaries, lists, etc). Using the
pickle module, I can more-or-less accomplish the latter, but not the
former. YAML can accomplish the former to some degree, but still can't
represent handles. I understand the inherent difficulties in
representing complex objects in a human readable format, but is there
any way to do this with Python? Any help is appreciated.

[Diez B. Roggisch]

Chris S. wrote:

I'm trying to write programs that make use of a lot of dynamic
structures. To aid in debugging and data extraction, I'd like to persist
these objects in a form that's readable, but one that also preserves
handles (ie for lambda expressions, dictionaries, lists, etc). Using the
pickle module, I can more-or-less accomplish the latter, but not the
former. YAML can accomplish the former to some degree, but still can't
represent handles. I understand the inherent difficulties in
representing complex objects in a human readable format, but is there
any way to do this with Python? Any help is appreciated.

Never actually tried it, but isn't there a xml-variant for pickling? Maybe
that helps you.

Regards,

Diez

[Simon Dahlbacka]

Diez B. Roggisch wrote:

Chris S. wrote:

I'm trying to write programs that make use of a lot of dynamic
structures. To aid in debugging and data extraction, I'd like to persist
these objects in a form that's readable, but one that also preserves
handles (ie for lambda expressions, dictionaries, lists, etc). Using the
pickle module, I can more-or-less accomplish the latter, but not the
former. YAML can accomplish the former to some degree, but still can't
represent handles. I understand the inherent difficulties in
representing complex objects in a human readable format, but is there
any way to do this with Python? Any help is appreciated.

Never actually tried it, but isn't there a xml-variant for pickling? Maybe
that helps you.

Regards,

Diez

http://gnosis.cx/download/Gnosis_Utils-current.tar.gz

according to my tests it works nicely as a pickle replacement that is at
least a bit more readable than plain old pickle. Not sure if I'd want to
edit it by hand though..

/Simon

[David Bolen]

"Chris S." <ch*****@NOSPAMudel.edu> writes:

I'm trying to write programs that make use of a lot of dynamic
structures. To aid in debugging and data extraction, I'd like to
persist these objects in a form that's readable, but one that also
preserves handles (ie for lambda expressions, dictionaries, lists,
etc).

Can you clarify what you might mean by "debugging and data extraction"?

The reason I ask is that if you're primarily concerned with the
ability to access the data easily outside of the program that
generated it, then have you considered just interatively loading your
picking file and performing whatever manipulations you might want?

That is, if pickle satisfies your need for the persistence of the data
structures themselves, perhaps the readability of the raw pickle file
need not be a problem if you consider just using Python to load that
file in when you need to examine it. You could always interactively
(or with small scripts if you do repetitive operations) then dump
selected portions of the data to more parseable flat text files for
use with other tools if necessary, while not losing any of the
richness of the original data structure.

I know that I've done this numerous times in the past (both with
pickle files as well as with ZODB databases) as a way to examine
stored program state in an interactive manner outside of the original
application.

-- David

[Chris S.]

David Bolen wrote:

"Chris S." <ch*****@NOSPAMudel.edu> writes:

I'm trying to write programs that make use of a lot of dynamic
structures. To aid in debugging and data extraction, I'd like to
persist these objects in a form that's readable, but one that also
preserves handles (ie for lambda expressions, dictionaries, lists,
etc).

Can you clarify what you might mean by "debugging and data extraction"?

The reason I ask is that if you're primarily concerned with the
ability to access the data easily outside of the program that
generated it, then have you considered just interatively loading your
picking file and performing whatever manipulations you might want?

That is, if pickle satisfies your need for the persistence of the data
structures themselves, perhaps the readability of the raw pickle file
need not be a problem if you consider just using Python to load that
file in when you need to examine it. You could always interactively
(or with small scripts if you do repetitive operations) then dump
selected portions of the data to more parseable flat text files for
use with other tools if necessary, while not losing any of the
richness of the original data structure.

I know that I've done this numerous times in the past (both with
pickle files as well as with ZODB databases) as a way to examine
stored program state in an interactive manner outside of the original
application.

-- David

The reason why I'd prefer the data stored in a readable format is so I
wouldn't have to use an interpreter to view the data (well, aside from a
text editor). Pickle doesn't even fully cut it for me since it's unable
to save lambda expressions and user defined function handles.