473,605 Members | 2,455 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Program works great, except under less, cron or execl (Unicode?)

Sam
I have a program which works great when run from the command line.

But when I run it combined with something else such as:
- piping it through less
- cron
- execl (i.e. calling it from another python program)

it gives me a unicode error

File "../myparser.py", line 261, in set_attributes
print "self.atd['Name'] is: ", self.atd['Name']
UnicodeEncodeEr ror: 'ascii' codec can't encode character u'\xeb' in
position 7: ordinal not in range(128)

I'd post the whole program here, except it involves weird Unicode
strings.

I could probably change the program to get it working under less/cron/
etc.

But I'd rather understand exactly what the issue is. Why does it work
fine when run directly from the command line, but not otherwise?
Sep 18 '08 #1
4 2417
Sam wrote:
I have a program which works great when run from the command line.

But when I run it combined with something else such as:
- piping it through less
- cron
- execl (i.e. calling it from another python program)

it gives me a unicode error

File "../myparser.py", line 261, in set_attributes
print "self.atd['Name'] is: ", self.atd['Name']
UnicodeEncodeEr ror: 'ascii' codec can't encode character u'\xeb' in
position 7: ordinal not in range(128)

I'd post the whole program here, except it involves weird Unicode
strings.

I could probably change the program to get it working under less/cron/
etc.

But I'd rather understand exactly what the issue is. Why does it work
fine when run directly from the command line, but not otherwise?
Most probably because when to running directly inside a terminal, it gets
it's stdin/stdout as pipes - and python can't attempt to guess the proper
encoding on that, as it does on a terminal.

And thus, when you print unicode to the pipe, it can't decide which encoding
to use.

To circumvene this, try & wrap stdout into a codecs-module wrapper with a
proper encoding applied (e.g. utf-8).

You might make that conditionally based on the sys.stdout.enco ding-variable
being set or not, albeit I'm not 100% sure to what it actually gets set
when used in a subprocess. But this should give you the idea where to look.

Diez
Sep 18 '08 #2
Most probably because when to running directly inside a terminal, it gets

That was of course meant to be "not running directly inside a terminal".
it's stdin/stdout as pipes - and python can't attempt to guess the proper
encoding on that, as it does on a terminal.
Diez
Sep 18 '08 #3
Sam
Diez for the win... :)

sys.stdout.enco ding does indeed have the proper value when called from
the command line of UTF-8.

But when piped into anything or called from anywhere it's None.

Just for completeness, here's my test program:
#!/usr/bin/env python
import sys
print sys.stdout.enco ding

And here are the results:
$ ./encoding.py
UTF-8
$ ./encoding.py | cat
None

Really, really annoying!

So how can I set sys.stdout.enco ding so it's UTF-8 when piped through
cat (or anything else).

I tried assigning to it, but no dice.

On Sep 18, 2:12*am, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:
Sam wrote:
I have a program which works great when run from the command line.
But when I run it combined with something else such as:
- piping it through less
- cron
- execl (i.e. calling it from another python program)
it gives me a unicode error
*File "../myparser.py", line 261, in set_attributes
* * print "self.atd['Name'] is: ", self.atd['Name']
UnicodeEncodeEr ror: 'ascii' codec can't encode character u'\xeb' in
position 7: ordinal not in range(128)
I'd post the whole program here, except it involves weird Unicode
strings.
I could probably change the program to get it working under less/cron/
etc.
But I'd rather understand exactly what the issue is. *Why does it work
fine when run directly from the command line, but not otherwise?

Most probably because when to running directly inside a terminal, it gets
it's stdin/stdout as pipes - and python can't attempt to guess the proper
encoding on that, as it does on a terminal.

And thus, when you print unicode to the pipe, it can't decide which encoding
to use.

To circumvene this, try & wrap stdout into a codecs-module wrapper with a
proper encoding applied (e.g. utf-8).

You might make that conditionally based on the sys.stdout.enco ding-variable
being set or not, albeit I'm not 100% sure to what it actually gets set
when used in a subprocess. But this should give you the idea where to look.

Diez
Sep 19 '08 #4
In message
<c8************ *************** *******@v16g200 0prc.googlegrou ps.com>, Sam
wrote:
So how can I set sys.stdout.enco ding so it's UTF-8 when piped through
cat (or anything else).

I tried assigning to it, but no dice.
You could try wrapping it in a file object that does explicit encoding
translation, using codecs.EncodedF ile
<http://docs.python.org/lib/module-codecs.html>.
Sep 24 '08 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
2205
by: Danny | last post by:
VB6 windows XP home I run a small app that has no forms in it, only a few BAS modules. It's function is to open a text file and load a few pieces of data into memory and then periodically update that same text file. When the data in the text file reaches certain parameters it is to close the modem connection. This program works perfectly while in design mode in the VB6
1
2188
by: Arthur T. Murray | last post by:
"Bill Modlin" <modlin1@metrocast.net> wrote on Fri, 26 Sep 2003: <snip> >>> If I wanted to explain how one of my programs worked >>> I would not give someone a print out of the code. >>> First I would explain it in higher level (folk psychology?) >>> terms and then explain how I embodied these ideas in actual code. >> I'm not sure why - especially if the person you're explaining it to >> is a programmer and knows the language. BM:
2
2499
by: David | last post by:
On every web browser except Safari, this website works great. (Well, by "every" I mean Mozilla, Netscape, and Internet Explorer, for Mac and Windows). The site is: http://www.ruleofthirds.com (the problem does not exist on the first page... just all the other pages, so you'll have to click on something to see the problem) The problem lies with images swapping on rollover. I use a small
10
2003
by: free2cric | last post by:
Hi, FOllowing is a program which is written by someone else. Its output is a wonderful poem. I dont understand how it works really. can anyone tell. thanks cric #include <stdio.h> main(int t,char _,char *a) {
1
1830
by: Morrison | last post by:
My HelloWorld program works on localhost but not on server. I have 2 hello world applications, one in VS 2003 and other in VS 2005. Both programs is working on my computer but if I uppload it on server it doesn't work. Message is "Server Error in '/' Application.". My web hosting provider say that they have included NET Framework 2.0 (i don't know is that means that 1.1 is included automaticly). I uploaded all directory with...
5
2110
by: dabhand | last post by:
Hi This page http://www.dabhand.co.nz/ayupdev/gallery-riders.html works great in IE but not in Firefox... any help would be appreciated. It refers to an external javascript file which I have included the code to below: The file name is imgriders.js - as referred to in source of the html file above. if (document.images) { image1 = new Image; image2 = new Image; image3 = new Image;
2
1008
by: raids51 | last post by:
I made a link crawler in vb.net and when i debug it, it all works fine. but when i try and launch the .exe file then it looks like its working, but it seems to ignore all the links and pretend their not there. Any help would be great
4
3563
by: ArizonaJohn | last post by:
Hello, The code below works great. The user enters a name into an HTML form, the code looks up a table with that name, and then that table is displayed. I am trying to use pagination with it, and the pagination almost works. The first page of the pagination works fine, but when I click on one of the links for one of the next pages, the page is blank. I have seen people mention this problem, and they have been told that a variable is...
0
7931
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
1
8067
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8281
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
6740
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
3911
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
3956
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2437
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1530
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1270
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.