473,396 Members | 1,936 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

Re: Safe eval of insecure strings containing Python data structures?

On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <wa****@delsci.comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>>More generally, you should never execute (via eval, exec, or whatever)
*any* instruction from an untrusted path; especially not arbitrary
data from an input stream.
I second this.
Wow, for the record, I completely disagree with this point of view:
Today's web apps wouldn't exist without safe forms of untrusted eval/exec
(Javascript anyone?). Such dogma is appropriate when dealing with the
CPython VM, but not as a general principle.
It's far better to use Data Structures
rather than Programming Constructs
to represent and transmit your data.
"Rocket fuel may be dangerous, but you ain't shooting the moon without it!"
Do we trust fuel from untrusted sources ?

cheers
James

--
--
-- "Problems are solved by method"
Oct 9 '08 #1
3 2641
On Oct 9, 4:32 am, "James Mills" <prolo...@shortcircuit.net.auwrote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci.comwrote:
JSON rocks! Thanks everyone.

Yes it does :)
Ben wrote:
>More generally, you should never execute (via eval, exec, or whatever)
*any* instruction from an untrusted path; especially not arbitrary
data from an input stream.

rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.

there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).

there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.

this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.

it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.

*as defined by the application developer* [not by the developers of
python itself]

if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".

instead, there should be a mechanism in place which allows them to
choose which foot they want to lose with the loaded gun they're
pointing.

l.
Oct 13 '08 #2
On Oct 13, 8:36 am, lkcl <luke.leigh...@googlemail.comwrote:
On Oct 9, 4:32 am, "James Mills" <prolo...@shortcircuit.net.auwrote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci.comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>>More generally, you should never execute (via eval, exec, or whatever)
>>*any* instruction from an untrusted path; especially not arbitrary
>>data from an input stream.

rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.

there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).

there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.

this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.

it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.

*as defined by the application developer* [not by the developers of
python itself]

if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".
That's why eval and exec still exist (and will probably be around for
a long time, if not forever). If you define your own external to
python security contexts, what did the deprecated rexec buy you that
eval/exec don't ? In any case, rexec is a single pure python module;
nothing stops you from copying it over to your project, hacking it and
keep using it at your own risk.

George
Oct 13 '08 #3
On Oct 13, 6:12 pm, George Sakkis <george.sak...@gmail.comwrote:
On Oct 13, 8:36 am,lkcl<luke.leigh...@googlemail.comwrote:
On Oct 9, 4:32 am, "James Mills" <prolo...@shortcircuit.net.auwrote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci.comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>More generally, you should never execute (via eval, exec, or whatever)
>*any* instruction from an untrusted path; especially not arbitrary
>data from an input stream.
rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.
there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).
there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.
this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.
it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.
*as defined by the application developer* [not by the developers of
python itself]
if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".

That's why eval and exec still exist (and will probably be around for
a long time, if not forever). If you define your own external to
python security contexts, what did the deprecated rexec buy you that
eval/exec don't ?
* being able to store python modules in a mysql database!
* being able to add context to selecting which python module
and which python function should be retrieved from the db (*1)
* being able to "vet" function names, allowing only those which
are supported routines (out of the database) and banning
all of the "standard" modules.

(*1) the context in which rexec.py was being used was for a data
centre "scanner" tool. a really damn good one, too :) some five
years later, we got things like nessus and the other scanner tools
being able to do "ping escalation", automated installs, ssh login
checks etc. etc. but this tool was written in early 2001 (!)

what we had was a 3-way-join on database tables:
* asset, comprising an id, name, OS name and IP address
* scripts, comprising an id, script content and the "module" name
* the scripts-to-os-mapper table, comprising an id, "module" name and
OS name

the 3way join was between asset.os-name and scripts-to-os-mapper.os-
name; scripts-to-os-mapper.module-name and scripts.module-name.

the implications were that we could write per-OS modules (each with
identical function names, function parameters and purpose, of course).

then, if the customer decided that they wanted NT 4.0 instead of
Redhat 5, we simply changed the OS type in the assets table, called up
the "installOS" script, and it would be up to the 3-way-join to select
the appropriate script for the job. no other work on our part was
needed (yes we had an automated way to network-install NT 4 and
Windows 2000).

the example i remember best was "loginssh" - using the standard Popen
python library - which of course was slightly different on a per-OS
basis, because for NT there were CRLF issues to deal with, and also we
had installed a commercial version of sshd which behaved differently.
regarding exec / eval: yes, i _have_ used that in a similar sort of
way, in another project. catching Name exceptions when executing a
piece of code, i would then retrieve the value for the variable which
came up from the exception by a SQL database call (which, perhaps
unsurprisingly, had been put there from a web interface).

by substituting the retrieved value into a dictionary to be used as
"locals" in the exec / eval call, i was able to repeatedly perform
this trick until the exec / eval succeeded, or the patience of the
user ran out.

it was awfully inefficient - O(N^2) - but, given that the code being
executed wasn't particularly large (200 lines, max?) it wasn't that
important.

however, these were _purely_ mathematical evaluations - returning
numbers or booleans. there wasn't anything radically complex - not
even _function_ calls.

so, the trick of doing overloading of "import" and "from x import y"
wasn't needed.

In any case, rexec is a single pure python module;
nothing stops you from copying it over to your project, hacking it and
keep using it at your own risk.
i knoww. it just annoyed and disappointed me that the issue wasn't
resolved at the right level.

Oct 14 '08 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

17
by: Gordon Airport | last post by:
Has anyone suggested introducing a mutable string type (yes, of course) and distinguishing them from standard strings by the quote type - single or double? As far as I know ' and " are currently...
42
by: Irmen de Jong | last post by:
Pickle and marshal are not safe. They can do harmful things if fed maliciously constructed data. That is a pity, because marshal is fast. I need a fast and safe (secure) marshaler. Is xdrlib the...
3
by: Edg Bamyasi | last post by:
This Is A Late Cross Post from comp.lang.python. It seems the mistery is deeper then i expected. What is the running time of conactination on character strings. i.e. >> joe="123" >>...
6
by: bwooster47 | last post by:
I've to use ConfigParser. It returns values that are exactly in the config file, so get string variables like: int1 with quotes and characers: "42" this is easy to convert to int: realint =...
95
by: hstagni | last post by:
Where can I find a library to created text-based windows applications? Im looking for a library that can make windows and buttons inside console.. Many old apps were make like this, i guess ...
1
by: =?ISO-8859-1?Q?Tor_Erik_S=F8nvisen?= | last post by:
Hi, A while ago I asked a question on the list about a simple eval function, capable of eval'ing simple python constructs (tuples, dicts, lists, strings, numbers etc) in a secure manner:...
7
by: bvdp | last post by:
I'm finding my quest for a safe eval() quite frustrating :) Any comments on this: Just forget about getting python to do this and, instead, grab my set of values (from a user supplied text file)...
16
by: Fett | last post by:
I am creating a program that requires some data that must be kept up to date. What I plan is to put this data up on a web-site then have the program periodically pull the data off the web-site. ...
3
by: Warren DeLano | last post by:
I would like to parse arbitrary insecure text string containing nested Python data structures in eval-compatible form: # For example, given a "config.txt" such as: { 'my_atom' : 1.20,...
0
by: Chris Rebert | last post by:
On Wed, Oct 8, 2008 at 5:34 PM, Warren DeLano <warren@delsci.comwrote: Assuming the data structures are sufficiently basic, i.e. no class instanciations, you can just use the json (AKA...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.