Convert string to command..

[Abandoned]

I want to convert a string to command..
For example i have a string:
a="['1']"
I want to do this list..
How can i do ?

[Abandoned]

On Oct 18, 6:35 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

Abandoned wrote:

On Oct 18, 6:14 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

Abandoned wrote:
Thanks you all answer..
But "eval" is very slow at very big dictionary {2:3,4:5,6:19.. ..}
(100.000 elements)
Is there any easy alternative ?

How big? How slow? For me, a 10000-element list takes 0.04 seconds to be
parsed. Which I find fast.

Diez

173.000 dict elements and it tooks 2.2 seconds this very big time for
my project

Where does the data come from?

Diez

Data come from database..
I want to cache to speed up my system and i save the dictionary to
database for speed up but eval is very slow for do this.
Not: 2.2 second only eval operation.

[Marc 'BlackJack' Rintsch]

On Thu, 18 Oct 2007 08:41:30 -0700, Abandoned wrote:

import cPickle as pickle
a="{2:3,4:6,2:7 }"
s=pickle.dumps( a, -1)
g=pickle.loads( s);
print g
'{2:3,4:6,2:7}'

Thank you very much for your answer but result is a string ??

In Python terms yes, strings in Python can contain any byte value. If you
want to put this into a database you need a BLOB column or encode it as
base64 or something similar more ASCII safe.

Ciao,
Marc 'BlackJack' Rintsch

[Diez B. Roggisch]

Abandoned wrote:

On Oct 18, 6:35 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

>Abandoned wrote:

On Oct 18, 6:14 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:
Abandoned wrote:
Thanks you all answer..
But "eval" is very slow at very big dictionary {2:3,4:5,6:19.. ..}
(100.000 elements)
Is there any easy alternative ?

>How big? How slow? For me, a 10000-element list takes 0.04 seconds to
be parsed. Which I find fast.

>Diez

173.000 dict elements and it tooks 2.2 seconds this very big time for
my project

Where does the data come from?

Diez

Data come from database..
I want to cache to speed up my system and i save the dictionary to
database for speed up but eval is very slow for do this.
Not: 2.2 second only eval operation.

Does the dictionary change often?

And you should store a pickle to the database then. Besides, making a
database-query of that size (after all, we're talking a few megs here) will
take a while as well - so are you SURE the 2.2 seconds are a problem? Or is
it just that you think they are?

Diez

[Abandoned]

On Oct 18, 6:57 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

Abandoned wrote:

On Oct 18, 6:35 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

Abandoned wrote:
On Oct 18, 6:14 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:
Abandoned wrote:
Thanks you all answer..
But "eval" is very slow at very big dictionary {2:3,4:5,6:19.. ..}
(100.000 elements)
Is there any easy alternative ?

How big? How slow? For me, a 10000-element list takes 0.04 seconds to
be parsed. Which I find fast.

Diez

173.000 dict elements and it tooks 2.2 seconds this very big time for
my project

Where does the data come from?

Diez

Data come from database..
I want to cache to speed up my system and i save the dictionary to
database for speed up but eval is very slow for do this.
Not: 2.2 second only eval operation.

Does the dictionary change often?

And you should store a pickle to the database then. Besides, making a
database-query of that size (after all, we're talking a few megs here) will
take a while as well - so are you SURE the 2.2 seconds are a problem? Or is
it just that you think they are?

Diez- Hide quoted text -

- Show quoted text -

I'm very confused :(
I try to explain main problem...
I have a table like this:
id-1 | id-2 | value
23 24 34
56 68 66
56 98 32455
55 62 655
56 28 123
..... ( 3 millions elements)

I select where id=56 and 100.000 rows are selecting but this took 2
second. (very big for my project)
I try cache to speed up this select operation..
And create a cache table:
id-1 | all
56 {68:66, 98:32455, 62:655}

When i select where id 56 i select 1 row and its took 0.09 second but
i must convert text to dictionary..

Have you got any idea what can i do this conver operation ?
or
Have you got any idea what can i do cache for this table ?

[Hrvoje Niksic]

Abandoned <be*****@gmail. comwrites:

import cPickle as pickle
a="{2:3,4:6,2:7 }"
s=pickle.dumps( a, -1)
g=pickle.loads( s);
print g
'{2:3,4:6,2:7}'

Thank you very much for your answer but result is a string ??

Because you gave it a string. If you give it a dict, you'll get a
dict:

>>import cPickle as pickle
a = {1:2, 3:4}
s = pickle.dumps(a, -1)
g = pickle.loads(s)
g

{1: 2, 3: 4}

If your existing database already has data in the "{...}" format, then
eval it only the first time. Then you'll get the dict which you can
cache thruogh the use of dumps/loads.

[Abandoned]

On Oct 18, 7:02 pm, Hrvoje Niksic <hnik...@xemacs .orgwrote:

Abandoned <best...@gmail. comwrites:

import cPickle as pickle
a="{2:3,4:6,2:7 }"
s=pickle.dumps( a, -1)
g=pickle.loads( s);
print g
'{2:3,4:6,2:7}'

Thank you very much for your answer but result is a string ??

Because you gave it a string. If you give it a dict, you'll get a
dict:

>import cPickle as pickle
a = {1:2, 3:4}
s = pickle.dumps(a, -1)
g = pickle.loads(s)
g

{1: 2, 3: 4}

If your existing database already has data in the "{...}" format, then
eval it only the first time. Then you'll get the dict which you can
cache thruogh the use of dumps/loads.

Sorry i can't understand :(
Yes my database already has data in the "{..}" format and i select
this and i want to use it for dictionary..
in your example:
first data is a string
finally data is already string

I want to command like eval. (eval is not good because it is slow for
my project)

[Hrvoje Niksic]

Abandoned <be*****@gmail. comwrites:

I select where id=56 and 100.000 rows are selecting but this took 2
second. (very big for my project)
I try cache to speed up this select operation..
And create a cache table:
id-1 | all
56 {68:66, 98:32455, 62:655}

If you use Python to create this cache table, then simply don't dump
it as a dictionary, but as a pickle:

id-1 | all
56 <some weird string produced by cPickle.dumps>

When you load it, convert the string to dict with cPickle.loads
instead of with eval.

[Matimus]

On Oct 18, 9:09 am, Abandoned <best...@gmail. comwrote:

On Oct 18, 6:57 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

Abandoned wrote:

On Oct 18, 6:35 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:
>Abandoned wrote:
On Oct 18, 6:14 pm, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:
>Abandoned wrote:
Thanks you all answer..
But "eval" is very slow at very big dictionary {2:3,4:5,6:19.. ..}
(100.000 elements)
Is there any easy alternative ?

>How big? How slow? For me, a 10000-element list takes 0.04 seconds to
>be parsed. Which I find fast.

>Diez

173.000 dict elements and it tooks 2.2 seconds this very big time for
my project

>Where does the data come from?

>Diez

Data come from database..
I want to cache to speed up my system and i save the dictionary to
database for speed up but eval is very slow for do this.
Not: 2.2 second only eval operation.

Does the dictionary change often?

And you should store a pickle to the database then. Besides, making a
database-query of that size (after all, we're talking a few megs here) will
take a while as well - so are you SURE the 2.2 seconds are a problem? Or is
it just that you think they are?

Diez- Hide quoted text -

- Show quoted text -

I'm very confused :(
I try to explain main problem...
I have a table like this:
id-1 | id-2 | value
23 24 34
56 68 66
56 98 32455
55 62 655
56 28 123
.... ( 3 millions elements)

I select where id=56 and 100.000 rows are selecting but this took 2
second. (very big for my project)
I try cache to speed up this select operation..
And create a cache table:
id-1 | all
56 {68:66, 98:32455, 62:655}

When i select where id 56 i select 1 row and its took 0.09 second but
i must convert text to dictionary..

Have you got any idea what can i do this conver operation ?
or
Have you got any idea what can i do cache for this table ?

I think several people have given you the correct answer, but for some
reason you aren't getting it. Instead of saving the string
representation of a dictionary to the database, pickle the dictionary
(not the string representation, but the actual dictionary) and save
the pickled object as a BLOB to the database. using pickle to re-
create the dictionary should be much faster than evaluating a string
representation of it.

Matt

[Hrvoje Niksic]

Abandoned <be*****@gmail. comwrites:

Sorry i can't understand :(
Yes my database already has data in the "{..}" format and i select
this and i want to use it for dictionary..

But, do you use Python to create that data? If so, simply convert it
to pickle binary format instead of "{...}". As explained in my other
post:

id-1 | all
56 {68:66, 98:32455, 62:655}

If you use Python to create this cache table, then simply don't dump
it as a dictionary, but as a pickle:

id-1 | all
56 <some weird string produced by cPickle.dumps>

When you load it, convert the string to dict with cPickle.loads
instead of with eval.

[Abandoned]

On Oct 18, 7:40 pm, Hrvoje Niksic <hnik...@xemacs .orgwrote:

Abandoned <best...@gmail. comwrites:

Sorry i can't understand :(
Yes my database already has data in the "{..}" format and i select
this and i want to use it for dictionary..

But, do you use Python to create that data? If so, simply convert it
to pickle binary format instead of "{...}". As explained in my other
post:

id-1 | all
56 {68:66, 98:32455, 62:655}

If you use Python to create this cache table, then simply don't dump
it as a dictionary, but as a pickle:

id-1 | all
56 <some weird string produced by cPickle.dumps>

When you load it, convert the string to dict with cPickle.loads
instead of with eval.

Yes i understand and this very very good ;)
But i have a problem..
a=eval(a)
a=pickle.dumps( a, -1)
cursor.execute( "INSERT INTO cache2 VALUES ('%s')" % (a))
conn.commit()

and give me error:

psycopg2.Progra mmingError: invalid byte sequence for encoding "UTF8":
0x80
HINT: This error can also happen if the byte sequence does not match
the encoding expected by the server, which is controlled by
"client_encodin g".