By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,645 Members | 1,065 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,645 IT Pros & Developers. It's quick & easy.

SimpleJson is slow .... is there any C Compiled version ?

P: n/a
Hello All,

I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.

Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.

Thank you,
Sanket
Jul 25 '08 #1
Share this Question
Share on Google+
5 Replies


P: n/a
sanket wrote:
Hello All,

I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.

Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.

Thank you,
Sanket
simplejson is not the only JSON library out there. For example, there's
python-cjson, which is written entirely in C:

<http://pypi.python.org/pypi/python-cjson>

There's also an enhanced version of it:

<http://python.cx.hu/python-cjson/>

I think simplejson has some small, optional C bits that will improve
performance if you compile them.

Also, be aware that I think simplejson is being integrated into the
stdlib in Python 2.6.

Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or something.
--
Jul 26 '08 #2

P: n/a
On Jul 25, 5:52*pm, Matt Nordhoff <mnordh...@mattnordhoff.comwrote:
sanket wrote:
Hello All,
I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.
Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.
Thank you,
Sanket

simplejson is not the only JSON library out there. For example, there's
python-cjson, which is written entirely in C:

<http://pypi.python.org/pypi/python-cjson>

There's also an enhanced version of it:

<http://python.cx.hu/python-cjson/>

I think simplejson has some small, optional C bits that will improve
performance if you compile them.

Also, be aware that I think simplejson is being integrated into the
stdlib in Python 2.6.

Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or something.
--
Thanks Matt.
I used cjson and it appears to be 5-6 times faster.

Thank you,
Sanket
Jul 26 '08 #3

P: n/a
On Jul 25, 5:52*pm, Matt Nordhoff <mnordh...@mattnordhoff.comwrote:
Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or something.
--
They also have different interface. simplejson uses load/loads/dump/
dumps, whereas cjson using encode/decode (its pretty simple to
monkeypatch cjson with some lambda's so it has the same interface,
though).

Yeah, its frustrating that there are so many json encoders for
python. Its a bit disappointing simplejson is going into the stdlib;
i hope they rewrite it in C?

http://blog.hill-street.net/?p=7
Has a good comparison of encoders.
Jul 26 '08 #4

P: n/a
On Sat, 26 Jul 2008 00:49:20 -0700, Richard Levasseur wrote:
On Jul 25, 5:52*pm, Matt Nordhoff <mnordh...@mattnordhoff.comwrote:
>Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or
something. --

They also have different interface. simplejson uses load/loads/dump/
dumps, whereas cjson using encode/decode (its pretty simple to
monkeypatch cjson with some lambda's so it has the same interface,
though).

Yeah, its frustrating that there are so many json encoders for python.
Its a bit disappointing simplejson is going into the stdlib; i hope they
rewrite it in C?
C's great for performance, but if you want something featureful and
reliable, you're probably better off in something else - including but
not limited to python, optionally with psyco.

I wish the pypy people would package up their stuff into a form that you
can just ./configure && make && make install and then #! to - I believe
it'd grow/mature faster than it is if they did (and they may have, but
last time I looked it didn't appear so). But once they do, that might be
a convenient way of getting better performance.

Jul 26 '08 #5

P: n/a
sanket wrote:
Hello All,

I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.

Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.
simplejson does have a C module. It is compiled automatically when
installed. If you're installing on Windows, then it probably isn't getting
compiled.

j

Jul 28 '08 #6

This discussion thread is closed

Replies have been disabled for this discussion.