472,344 Members | 1,348 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,344 software developers and data experts.

Re: Safe eval of insecure strings containing Python data structures?

On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <wa****@delsci.comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>>More generally, you should never execute (via eval, exec, or whatever)
*any* instruction from an untrusted path; especially not arbitrary
data from an input stream.
I second this.
Wow, for the record, I completely disagree with this point of view:
Today's web apps wouldn't exist without safe forms of untrusted eval/exec
(Javascript anyone?). Such dogma is appropriate when dealing with the
CPython VM, but not as a general principle.
It's far better to use Data Structures
rather than Programming Constructs
to represent and transmit your data.
"Rocket fuel may be dangerous, but you ain't shooting the moon without it!"
Do we trust fuel from untrusted sources ?

cheers
James

--
--
-- "Problems are solved by method"
Oct 9 '08 #1
3 2508
On Oct 9, 4:32 am, "James Mills" <prolo...@shortcircuit.net.auwrote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci.comwrote:
JSON rocks! Thanks everyone.

Yes it does :)
Ben wrote:
>More generally, you should never execute (via eval, exec, or whatever)
*any* instruction from an untrusted path; especially not arbitrary
data from an input stream.

rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.

there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).

there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.

this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.

it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.

*as defined by the application developer* [not by the developers of
python itself]

if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".

instead, there should be a mechanism in place which allows them to
choose which foot they want to lose with the loaded gun they're
pointing.

l.
Oct 13 '08 #2
On Oct 13, 8:36 am, lkcl <luke.leigh...@googlemail.comwrote:
On Oct 9, 4:32 am, "James Mills" <prolo...@shortcircuit.net.auwrote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci.comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>>More generally, you should never execute (via eval, exec, or whatever)
>>*any* instruction from an untrusted path; especially not arbitrary
>>data from an input stream.

rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.

there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).

there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.

this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.

it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.

*as defined by the application developer* [not by the developers of
python itself]

if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".
That's why eval and exec still exist (and will probably be around for
a long time, if not forever). If you define your own external to
python security contexts, what did the deprecated rexec buy you that
eval/exec don't ? In any case, rexec is a single pure python module;
nothing stops you from copying it over to your project, hacking it and
keep using it at your own risk.

George
Oct 13 '08 #3
On Oct 13, 6:12 pm, George Sakkis <george.sak...@gmail.comwrote:
On Oct 13, 8:36 am,lkcl<luke.leigh...@googlemail.comwrote:
On Oct 9, 4:32 am, "James Mills" <prolo...@shortcircuit.net.auwrote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci.comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>More generally, you should never execute (via eval, exec, or whatever)
>*any* instruction from an untrusted path; especially not arbitrary
>data from an input stream.
rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.
there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).
there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.
this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.
it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.
*as defined by the application developer* [not by the developers of
python itself]
if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".

That's why eval and exec still exist (and will probably be around for
a long time, if not forever). If you define your own external to
python security contexts, what did the deprecated rexec buy you that
eval/exec don't ?
* being able to store python modules in a mysql database!
* being able to add context to selecting which python module
and which python function should be retrieved from the db (*1)
* being able to "vet" function names, allowing only those which
are supported routines (out of the database) and banning
all of the "standard" modules.

(*1) the context in which rexec.py was being used was for a data
centre "scanner" tool. a really damn good one, too :) some five
years later, we got things like nessus and the other scanner tools
being able to do "ping escalation", automated installs, ssh login
checks etc. etc. but this tool was written in early 2001 (!)

what we had was a 3-way-join on database tables:
* asset, comprising an id, name, OS name and IP address
* scripts, comprising an id, script content and the "module" name
* the scripts-to-os-mapper table, comprising an id, "module" name and
OS name

the 3way join was between asset.os-name and scripts-to-os-mapper.os-
name; scripts-to-os-mapper.module-name and scripts.module-name.

the implications were that we could write per-OS modules (each with
identical function names, function parameters and purpose, of course).

then, if the customer decided that they wanted NT 4.0 instead of
Redhat 5, we simply changed the OS type in the assets table, called up
the "installOS" script, and it would be up to the 3-way-join to select
the appropriate script for the job. no other work on our part was
needed (yes we had an automated way to network-install NT 4 and
Windows 2000).

the example i remember best was "loginssh" - using the standard Popen
python library - which of course was slightly different on a per-OS
basis, because for NT there were CRLF issues to deal with, and also we
had installed a commercial version of sshd which behaved differently.
regarding exec / eval: yes, i _have_ used that in a similar sort of
way, in another project. catching Name exceptions when executing a
piece of code, i would then retrieve the value for the variable which
came up from the exception by a SQL database call (which, perhaps
unsurprisingly, had been put there from a web interface).

by substituting the retrieved value into a dictionary to be used as
"locals" in the exec / eval call, i was able to repeatedly perform
this trick until the exec / eval succeeded, or the patience of the
user ran out.

it was awfully inefficient - O(N^2) - but, given that the code being
executed wasn't particularly large (200 lines, max?) it wasn't that
important.

however, these were _purely_ mathematical evaluations - returning
numbers or booleans. there wasn't anything radically complex - not
even _function_ calls.

so, the trick of doing overloading of "import" and "from x import y"
wasn't needed.

In any case, rexec is a single pure python module;
nothing stops you from copying it over to your project, hacking it and
keep using it at your own risk.
i knoww. it just annoyed and disappointed me that the issue wasn't
resolved at the right level.

Oct 14 '08 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

17
by: Gordon Airport | last post by:
Has anyone suggested introducing a mutable string type (yes, of course) and distinguishing them from standard strings by the quote type - single or...
42
by: Irmen de Jong | last post by:
Pickle and marshal are not safe. They can do harmful things if fed maliciously constructed data. That is a pity, because marshal is fast. I need a...
3
by: Edg Bamyasi | last post by:
This Is A Late Cross Post from comp.lang.python. It seems the mistery is deeper then i expected. What is the running time of conactination on...
6
by: bwooster47 | last post by:
I've to use ConfigParser. It returns values that are exactly in the config file, so get string variables like: int1 with quotes and characers:...
95
by: hstagni | last post by:
Where can I find a library to created text-based windows applications? Im looking for a library that can make windows and buttons inside console.....
1
by: =?ISO-8859-1?Q?Tor_Erik_S=F8nvisen?= | last post by:
Hi, A while ago I asked a question on the list about a simple eval function, capable of eval'ing simple python constructs (tuples, dicts, lists,...
7
by: bvdp | last post by:
I'm finding my quest for a safe eval() quite frustrating :) Any comments on this: Just forget about getting python to do this and, instead, grab...
16
by: Fett | last post by:
I am creating a program that requires some data that must be kept up to date. What I plan is to put this data up on a web-site then have the...
3
by: Warren DeLano | last post by:
I would like to parse arbitrary insecure text string containing nested Python data structures in eval-compatible form: # For example, given a...
0
by: Chris Rebert | last post by:
On Wed, Oct 8, 2008 at 5:34 PM, Warren DeLano <warren@delsci.comwrote: Assuming the data structures are sufficiently basic, i.e. no class...
0
better678
by: better678 | last post by:
Question: Discuss your understanding of the Java platform. Is the statement "Java is interpreted" correct? Answer: Java is an object-oriented...
0
by: teenabhardwaj | last post by:
How would one discover a valid source for learning news, comfort, and help for engineering designs? Covering through piles of books takes a lot of...
0
by: Naresh1 | last post by:
What is WebLogic Admin Training? WebLogic Admin Training is a specialized program designed to equip individuals with the skills and knowledge...
0
jalbright99669
by: jalbright99669 | last post by:
Am having a bit of a time with URL Rewrite. I need to incorporate http to https redirect with a reverse proxy. I have the URL Rewrite rules made...
0
by: antdb | last post by:
Ⅰ. Advantage of AntDB: hyper-convergence + streaming processing engine In the overall architecture, a new "hyper-convergence" concept was...
0
by: Matthew3360 | last post by:
Hi there. I have been struggling to find out how to use a variable as my location in my header redirect function. Here is my code. ...
0
by: AndyPSV | last post by:
HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computerHOW CAN I CREATE AN AI with an .executable...
0
by: Arjunsri | last post by:
I have a Redshift database that I need to use as an import data source. I have configured the DSN connection using the server, port, database, and...
0
hi
by: WisdomUfot | last post by:
It's an interesting question you've got about how Gmail hides the HTTP referrer when a link in an email is clicked. While I don't have the specific...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.