Re: Where to locate existing standard encodings in python

[Philip Semanchuk]

On Nov 9, 2008, at 7:00 PM, News123 wrote:

Hi,

I was googling quite some time before finding the answer to my
question:
'what are the names for the encodings supported by python?'

I found the answer at http://python.active-venture.com/lib/
node127.html
Now my question:

Can I find the same info in the standard python doc or query python
with
a certain command to print out all existing codings?

Look under the heading "Standard Encodings":
http://docs.python.org/library/codecs.html

Note that both the page you found (which appears to be a copy of the
Python documentation) and the reference I provide say, "Neither the
list of aliases nor the list of languages is meant to be exhaustive".

I guess one reason for this is that different Python implementations
could choose to offer codecs for additional encodings.

[Philip Semanchuk]

On Nov 11, 2008, at 9:10 AM, News123 wrote:

Hi Philip,

Your answer touches exaclty one point, which I was slightly afraid of:
- The list is not exhaustive
- python versions might have implemented different codecs.

This is why I wondered whether there's any way of querying python
for a
list of codecs it supports.

Try this:
Python 2.5.1 (r251:54863, Nov 17 2007, 21:19:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

>>import encodings.aliases

encodings.aliases.aliases

"aliases" in the encodings.aliases module is a dict mapping alias
names (the dict keys) to encodings (the dict values). Thus, this will
give you the list of supported encodings:

>>set(encodings.aliases.aliases.values())

The encodings module isn't in the documentation (?!?); I found it when
looking through the Python source code. For that reason I can't say
more about how it works. You may want to experiment to see if
encodings added via codecs.register() show up in the
encodings.aliases.aliases dict.
Have fun
Philip

>
Philip Semanchuk wrote:

>>
On Nov 9, 2008, at 7:00 PM, News123 wrote:

>>Hi,

I was googling quite some time before finding the answer to my
question:
'what are the names for the encodings supported by python?'

I found the answer at http://python.active-venture.com/lib/node127.html
Now my question:

Can I find the same info in the standard python doc or query
python with
a certain command to print out all existing codings?

Look under the heading "Standard Encodings":
http://docs.python.org/library/codecs.html

Note that both the page you found (which appears to be a copy of the
Python documentation) and the reference I provide say, "Neither the
list
of aliases nor the list of languages is meant to be exhaustive".

I guess one reason for this is that different Python implementations
could choose to offer codecs for additional encodings.

--
http://mail.python.org/mailman/listinfo/python-list

[Philip Semanchuk]

On Nov 11, 2008, at 1:08 PM, News123 wrote:

Hi Philip,

Thanks for your answer:
The fact, that a module 'encodings' exists was new to me.

We both learned something new today. =)

encodings.aliases.aliases has however one problem.
It helps to locate all encoding aliases, but it won't find entries for
which no aliases exist:

Ooops, I hadn't thought about that.

What gives me a list of quite some encodings on my host is the shell
command
ls /usr/lib/python2.5/encodings | sed -n 's/\.py$//p' | sort
(soma false hits, bit this is fine for me purposes)

I don't know if really all encodings are represented with a .py file
and
if all encodigns have to be in this directory, but it's a start.
Using shell commands is not that pythonic:

I could try to rewrite this in python by
1.) determine from which directory encodings was imported and
then using the glob module to list all .py files located there.

Yes, I'd thought about this but I agree with you that it seems
unpythonic and fragile. Unfortunately I can't think of anything better
at this point.

Good luck
Philip

>
Philip Semanchuk wrote:

>>
On Nov 11, 2008, at 9:10 AM, News123 wrote:

>>Hi Philip,

Your answer touches exaclty one point, which I was slightly afraid
of:
- The list is not exhaustive
- python versions might have implemented different codecs.

This is why I wondered whether there's any way of querying python
for a
list of codecs it supports.

Try this:
Python 2.5.1 (r251:54863, Nov 17 2007, 21:19:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more
information.

>>>>import encodings.aliases
>
encodings.aliases.aliases

"aliases" in the encodings.aliases module is a dict mapping alias
names
(the dict keys) to encodings (the dict values). Thus, this will
give you
the list of supported encodings:

>>>>set(encodings.aliases.aliases.values())

The encodings module isn't in the documentation (?!?); I found it
when
looking through the Python source code. For that reason I can't say
more
about how it works. You may want to experiment to see if encodings
added
via codecs.register() show up in the encodings.aliases.aliases dict.
Have fun
Philip

>>>
Philip Semanchuk wrote:

On Nov 9, 2008, at 7:00 PM, News123 wrote:

Hi,
>
I was googling quite some time before finding the answer to my
question:
'what are the names for the encodings supported by python?'
>
I found the answer at http://python.active-venture.com/lib/node127.html
>
>
Now my question:
>
Can I find the same info in the standard python doc or query
python
with
a certain command to print out all existing codings?
Look under the heading "Standard Encodings":
http://docs.python.org/library/codecs.html

Note that both the page you found (which appears to be a copy of
the
Python documentation) and the reference I provide say, "Neither
the list
of aliases nor the list of languages is meant to be exhaustive".

I guess one reason for this is that different Python
implementations
could choose to offer codecs for additional encodings.
--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

[rurpy]

On Nov 11, 11:19 am, Philip Semanchuk <[e-mail]:

On Nov 11, 2008, at 1:08 PM, News123 wrote:

Hi Philip,

Thanks for your answer:
The fact, that a module 'encodings' exists was new to me.

We both learned something new today. =)

encodings.aliases.aliases has however one problem.
It helps to locate all encoding aliases, but it won't find entries for
which no aliases exist:

Ooops, I hadn't thought about that.

What gives me a list of quite some encodings on my host is the shell
command
ls /usr/lib/python2.5/encodings | sed -n 's/\.py$//p' | sort
(soma false hits, bit this is fine for me purposes)

I don't know if really all encodings are represented with a .py file
and
if all encodigns have to be in this directory, but it's a start.

Using shell commands is not that pythonic:

I could try to rewrite this in python by
1.) determine from which directory encodings was imported and
then using the glob module to list all .py files located there.

Yes, I'd thought about this but I agree with you that it seems
unpythonic and fragile. Unfortunately I can't think of anything better
at this point.

Good luck
Philip

....snip...

If it's of any help, in a post on 2007-07-22 by Peter Otten,
(though I can't get a url for it at the moment) he took the
same approach. From a saved copy of that post:

import encodings
import os
import glob

def encodings_from_modulenames():
ef = os.path.dirname(encodings.__file__)
for fn in glob.glob(os.path.join(ef, "*.py")):
fn = os.path.basename(fn)
yield os.path.splitext(fn)[0]