Here is a minimal example that produces the strange results:
#### begin code ####
#!/usr/bin/env python
from Tkinter import *
import tkFileDialog
import tkMessageBox
def openFile():
r = tkFileDialog.askopenfilename(filetypes=(("some","* .ini"), ("all","*")))
print 'openFile():', type(r), r
def askYesNo():
r = tkMessageBox._show('yes/no', "Do or don't?", 'question', 'yesno')
print 'askYesNo():', type(r), r
root = Tk()
top = Menu(root)
root.config(menu=top)
mFile = Menu(top)
mFile.add_command(label='Open...', command=openFile)
mFile.add_command(label='YesNo', command=askYesNo)
mFile.add_command(label='Quit', command=root.quit)
top.add_cascade(label='File', menu=mFile, underline=0)
root.mainloop()
#### end code ####
In the code above, I unpacked the original function
tkMessageBox.askyesno(), which looks like this:
def askyesno(title=None, message=None, **options):
"Ask a question; return true if the answer is yes"
s = _show(title, message, QUESTION, YESNO, **options)
return s == YES
A session on a machine that goes fine:
askYesNo(): <type 'str'> yes
askYesNo(): <type 'str'> no
openFile(): <type 'str'> # selected 'cancel'
openFile(): <type 'str'> /home/peter/leven/L04/pyL04/Project.ini
askYesNo(): <type 'str'> yes
askYesNo(): <type 'str'> no
And here on another machine were things don't go as they should:
askYesNo(): <type 'str'> yes
askYesNo(): <type 'str'> no
openFile(): <type 'unicode'> # selected 'cancel'
openFile(): <type 'str'> /users3/kleiweg/pyL04/Project.ini
askYesNo(): <type 'bool'> True # causes problems, because True != 'yes'
askYesNo(): <type 'str'> no
Both are Linux machines running Python 2.3.4
The machine with problems has tcl/tk 8.4
The other machine has tcl/tk 8.3
Is this a bug I should report? Is it a bug in Python, or in tk?
--
Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
info:
http://www.let.rug.nl/~kleiweg/ls.html 
