473,729 Members | 2,344 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Re: Safe eval of insecure strings containing Python data structures?

On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <wa****@delsci. comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>>More generally, you should never execute (via eval, exec, or whatever)
*any* instruction from an untrusted path; especially not arbitrary
data from an input stream.
I second this.
Wow, for the record, I completely disagree with this point of view:
Today's web apps wouldn't exist without safe forms of untrusted eval/exec
(Javascript anyone?). Such dogma is appropriate when dealing with the
CPython VM, but not as a general principle.
It's far better to use Data Structures
rather than Programming Constructs
to represent and transmit your data.
"Rocket fuel may be dangerous, but you ain't shooting the moon without it!"
Do we trust fuel from untrusted sources ?

cheers
James

--
--
-- "Problems are solved by method"
Oct 9 '08 #1
3 2685
On Oct 9, 4:32 am, "James Mills" <prolo...@short circuit.net.auw rote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci. comwrote:
JSON rocks! Thanks everyone.

Yes it does :)
Ben wrote:
>More generally, you should never execute (via eval, exec, or whatever)
*any* instruction from an untrusted path; especially not arbitrary
data from an input stream.

rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.

there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).

there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.

this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.

it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.

*as defined by the application developer* [not by the developers of
python itself]

if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".

instead, there should be a mechanism in place which allows them to
choose which foot they want to lose with the loaded gun they're
pointing.

l.
Oct 13 '08 #2
On Oct 13, 8:36 am, lkcl <luke.leigh...@ googlemail.comw rote:
On Oct 9, 4:32 am, "James Mills" <prolo...@short circuit.net.auw rote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci. comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>>More generally, you should never execute (via eval, exec, or whatever)
>>*any* instruction from an untrusted path; especially not arbitrary
>>data from an input stream.

rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.

there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).

there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.

this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.

it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.

*as defined by the application developer* [not by the developers of
python itself]

if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".
That's why eval and exec still exist (and will probably be around for
a long time, if not forever). If you define your own external to
python security contexts, what did the deprecated rexec buy you that
eval/exec don't ? In any case, rexec is a single pure python module;
nothing stops you from copying it over to your project, hacking it and
keep using it at your own risk.

George
Oct 13 '08 #3
On Oct 13, 6:12 pm, George Sakkis <george.sak...@ gmail.comwrote:
On Oct 13, 8:36 am,lkcl<luke.le igh...@googlema il.comwrote:
On Oct 9, 4:32 am, "James Mills" <prolo...@short circuit.net.auw rote:
On Thu, Oct 9, 2008 at 2:26 PM, Warren DeLano <war...@delsci. comwrote:
JSON rocks! Thanks everyone.
Yes it does :)
Ben wrote:
>More generally, you should never execute (via eval, exec, or whatever)
>*any* instruction from an untrusted path; especially not arbitrary
>data from an input stream.
rubbish. this is why a project i was involved with, to do execution
of code from a database instead of a filesystem had to be abandoned,
back in 2001.
there are perfectly good systems for associating security context
with "arbitrary data" (as the security models of SE/Linux, based on
Flask, and the security model of windows nt, based on VAX/VMS
security, show).
there was a flawed design decision in python 2.2 or python 2.3 which
resulted in an "escape route" - i believe it centered around either
__class__ or __new__ - in the c code, which the developers had not
considered, and would not correct.
this decision resulted in the abandonment of the rexec.py module in
python: you can see for yourself because it raises a runtime exception
when you try to use it, issuing a warning.
it's _perfectly_ possible to define security contexts and boundaries,
and to allow access to functions and modules on a per-security-context
basis.
*as defined by the application developer* [not by the developers of
python itself]
if an individual developer wants to allow "arbitrary code execution
from any data stream", it most certainly is _not_ anyone's place to
dictate to them that they "cannot do this".

That's why eval and exec still exist (and will probably be around for
a long time, if not forever). If you define your own external to
python security contexts, what did the deprecated rexec buy you that
eval/exec don't ?
* being able to store python modules in a mysql database!
* being able to add context to selecting which python module
and which python function should be retrieved from the db (*1)
* being able to "vet" function names, allowing only those which
are supported routines (out of the database) and banning
all of the "standard" modules.

(*1) the context in which rexec.py was being used was for a data
centre "scanner" tool. a really damn good one, too :) some five
years later, we got things like nessus and the other scanner tools
being able to do "ping escalation", automated installs, ssh login
checks etc. etc. but this tool was written in early 2001 (!)

what we had was a 3-way-join on database tables:
* asset, comprising an id, name, OS name and IP address
* scripts, comprising an id, script content and the "module" name
* the scripts-to-os-mapper table, comprising an id, "module" name and
OS name

the 3way join was between asset.os-name and scripts-to-os-mapper.os-
name; scripts-to-os-mapper.module-name and scripts.module-name.

the implications were that we could write per-OS modules (each with
identical function names, function parameters and purpose, of course).

then, if the customer decided that they wanted NT 4.0 instead of
Redhat 5, we simply changed the OS type in the assets table, called up
the "installOS" script, and it would be up to the 3-way-join to select
the appropriate script for the job. no other work on our part was
needed (yes we had an automated way to network-install NT 4 and
Windows 2000).

the example i remember best was "loginssh" - using the standard Popen
python library - which of course was slightly different on a per-OS
basis, because for NT there were CRLF issues to deal with, and also we
had installed a commercial version of sshd which behaved differently.
regarding exec / eval: yes, i _have_ used that in a similar sort of
way, in another project. catching Name exceptions when executing a
piece of code, i would then retrieve the value for the variable which
came up from the exception by a SQL database call (which, perhaps
unsurprisingly, had been put there from a web interface).

by substituting the retrieved value into a dictionary to be used as
"locals" in the exec / eval call, i was able to repeatedly perform
this trick until the exec / eval succeeded, or the patience of the
user ran out.

it was awfully inefficient - O(N^2) - but, given that the code being
executed wasn't particularly large (200 lines, max?) it wasn't that
important.

however, these were _purely_ mathematical evaluations - returning
numbers or booleans. there wasn't anything radically complex - not
even _function_ calls.

so, the trick of doing overloading of "import" and "from x import y"
wasn't needed.

In any case, rexec is a single pure python module;
nothing stops you from copying it over to your project, hacking it and
keep using it at your own risk.
i knoww. it just annoyed and disappointed me that the issue wasn't
resolved at the right level.

Oct 14 '08 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

17
7396
by: Gordon Airport | last post by:
Has anyone suggested introducing a mutable string type (yes, of course) and distinguishing them from standard strings by the quote type - single or double? As far as I know ' and " are currently interchangeable in all circumstances (as long as they're paired) so there's no overloading to muddy the language. Of course there could be some interesting problems with current code that doesn't make a distinction, but it would be dead easy to fix...
42
2594
by: Irmen de Jong | last post by:
Pickle and marshal are not safe. They can do harmful things if fed maliciously constructed data. That is a pity, because marshal is fast. I need a fast and safe (secure) marshaler. Is xdrlib the only option? I would expect that it is fast and safe because it (the xdr spec) has been around for so long. Or are there better options (perhaps 3rd party libraries)?
3
2310
by: Edg Bamyasi | last post by:
This Is A Late Cross Post from comp.lang.python. It seems the mistery is deeper then i expected. What is the running time of conactination on character strings. i.e. >> joe="123" >> joe+="99999999999999999"
6
2309
by: bwooster47 | last post by:
I've to use ConfigParser. It returns values that are exactly in the config file, so get string variables like: int1 with quotes and characers: "42" this is easy to convert to int: realint = int(int1) I've read the tutorial, and the FAQ, and not sure if I missed it, but other than calling eval (which everyone says is unsafe), I don't know
95
5389
by: hstagni | last post by:
Where can I find a library to created text-based windows applications? Im looking for a library that can make windows and buttons inside console.. Many old apps were make like this, i guess ____________________________________ | | | ------------------ | | | BUTTON | | | ...
1
2215
by: =?ISO-8859-1?Q?Tor_Erik_S=F8nvisen?= | last post by:
Hi, A while ago I asked a question on the list about a simple eval function, capable of eval'ing simple python constructs (tuples, dicts, lists, strings, numbers etc) in a secure manner: http://groups.google.com/group/comp.lang.python/browse_thread/thread/58a01273441d445f/ also pointed to a simple eval function by Fredrik Lundh: http://effbot.org/zone/simple-iterator-parser.htm. His solution, using module tokenize, was short and...
7
242
by: bvdp | last post by:
I'm finding my quest for a safe eval() quite frustrating :) Any comments on this: Just forget about getting python to do this and, instead, grab my set of values (from a user supplied text file) and call an external program like 'bc' to do the dirty work. I think that this would avoid someone from embedding os.system("rm ...") in what I thought would be a math expression and having it maybe do damage? Perhaps I'm getting too paranoid in...
16
3930
by: Fett | last post by:
I am creating a program that requires some data that must be kept up to date. What I plan is to put this data up on a web-site then have the program periodically pull the data off the web-site. My problem is that when I pull the data (currently stored as a dictionary on the site) off the site, it is a string, I can use eval() to make that string into a dictionary, and everything is great. However, this means that I am using eval() on...
3
1920
by: Warren DeLano | last post by:
I would like to parse arbitrary insecure text string containing nested Python data structures in eval-compatible form: # For example, given a "config.txt" such as: { 'my_atom' : 1.20, 'my_dict' : { 2:50 , 'hi':'mom'}, 'my_list' : , 'foo', 0 ] }
0
255
by: Chris Rebert | last post by:
On Wed, Oct 8, 2008 at 5:34 PM, Warren DeLano <warren@delsci.comwrote: Assuming the data structures are sufficiently basic, i.e. no class instanciations, you can just use the json (AKA simplejson) library to deserialize the data in the string. Python and JSON conveniently happen to share the same syntax for literals (except for booleans IIRC). Also, if this is your program's config file, you might consider changing it to INI-format and...
0
8917
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9426
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
9200
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9142
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8148
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
4525
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4795
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
2680
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2163
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.