473,326 Members | 1,972 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Location of bytecode files (pyc)

Rim
Hi,

How can I control the location of the bytecode files?

The reason I ask is I want to place the bytecode in a directory
different than source code for backup reasons. Our backup system is
directory based and I only want to have source code backed up, not
object files or pyc files.

Thanks,
-Rim
Jul 18 '05 #1
8 3084
ri*******@yahoo.com (Rim) writes:
How can I control the location of the bytecode files?


You currently can't. See PEP 304, though, at

http://www.python.org/peps/pep-0304.html

Comments on the PEP are encouraged: If you won't comment now on
whether the proposed change would solve your problem, it might be that
you find something useless got implemented later.

Regards,
Martin
Jul 18 '05 #2

"Rim" <ri*******@yahoo.com> wrote in message
news:6f**************************@posting.google.c om...
Hi,

How can I control the location of the bytecode files?

The reason I ask is I want to place the bytecode in a directory
different than source code for backup reasons. Our backup system is
directory based and I only want to have source code backed up, not
object files or pyc files.
If you're on Windows say:

del /S *.pyc

to remove all .pyc's from cwd and below.

Thanks,
-Rim

Jul 18 '05 #3
On 03 Sep 2003 07:12:07 +0200, ma****@v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) wrote:
ri*******@yahoo.com (Rim) writes:
How can I control the location of the bytecode files?


You currently can't. See PEP 304, though, at

http://www.python.org/peps/pep-0304.html

Comments on the PEP are encouraged: If you won't comment now on
whether the proposed change would solve your problem, it might be that
you find something useless got implemented later.

Personally, I am a minimalist when it comes to environment variables.
IMO that name space has the same problems as any global namespace,
and since a single default set of user's environment variables tends to
presented to most programs s/he runs from a command window, the name space
usage tends towards a hodgepodge and/or wrapping apps in setup scripts (which
can work fine, but I still don't like it as a standard way to go).

IMO the os.environ name space should be treated like a root directory name space,
and not have application data per se in it (with reluctant exceptions where it is used wholesale
as in CGI). Rather, IMO, and only if necessary in the first place, it should be used to specify
location or search path info for config data, not *be* config data.

And a user-set environment variable should not be able to cause a bypass of root/admin-defined
config info where the distinction is necessary.

(The PYTHONBYTECODEBASE variable does refer to a directory, but now that makes two variables,
counting PYTHONPATH, and where will it end?)

Provision for admin/root level config data separate from user preference and session state type
config data should be made as necessary and desirable, but secondary/user config data search
should be controllable by the primary/root/admin config data (which e.g. could say to ignore
or use user-controlled/attempted-control environment variables etc.).

This would seem to imply a standard place to look for root/admin level config data, not directed
by env variable. E.g., a read-only file in the same directory as the python interpreter executable,
with, say, .conf or .cfg appended to the name. *That* file can then specify how/whether to look
for user config stuff etc., or specify password access to some features, etc. etc., if we wind up
doing restricted exec stuff.

A user config file overriding any other *user* config info could be specified by command line option,
e.g., -cfg myConfigFile.conf, and whether this or other command line options were allowed could be
(and should be able to be when control is necessary) specified in the root/admin config file.

.... just my .02USD

Regards,
Bengt Richter
Jul 18 '05 #4
Rim
> If you're on Windows say:

del /S *.pyc

to remove all .pyc's from cwd and below.


Yes, and in linux 'find . -name "*.pyc" -exec rm -rf {} \;', but that
is not the point. I did not ask about how to remove pyc files, I asked
about placing them far from source code, in another directory, so they
would not be seen by the backup software.

The backup software runs continuously day, night, weekends, all the
time. Sweeping the pyc's under the carpet each time the backup is
about to be saving my files is impossible. That is why we store all
gcc object files and executables we produce on non-backed up
filesystems by prefixing target filenames in makefiles with a variable
like $OBJECT_DIR and $EXEC_DIR.

We save hundreds of Gigs of disk space this way.

PEP 304 appears to give me what I need.

As Bengt suggest, adding yet another env var to control the operation
of an application polutes the name space for all applications, but
each app only needs to look at the variables it is concerned with.
PYTHONBYTECODEBASE is a cleaner approach that has less overhead than a
configuration file. Also, env variables are directly usable in
makefiles, which makes them very attractive and easy to use for
managing where compiler output goes.

Say you work on multiple projects simultaneously with different
locations for the byte code. The configuration files approach becomes
an administrative burden with conditionals based on the project name
in your config file.

The typical way we handle project specific configurations is through
environment variables that get set in the startup files. Depending on
a PROJECT environment variable, the PYTHONBYTECODEBASE can easily be
set to different directories without creating additional .cfg files.

Regards,
-Rim
Jul 18 '05 #5
Rim wrote:
If you're on Windows say:

del /S *.pyc

to remove all .pyc's from cwd and below.


Yes, and in linux 'find . -name "*.pyc" -exec rm -rf {} \;', but that
is not the point. I did not ask about how to remove pyc files, I asked
about placing them far from source code, in another directory, so they
would not be seen by the backup software.

The backup software runs continuously day, night, weekends, all the
time. Sweeping the pyc's under the carpet each time the backup is
about to be saving my files is impossible. That is why we store all
gcc object files and executables we produce on non-backed up
filesystems by prefixing target filenames in makefiles with a variable
like $OBJECT_DIR and $EXEC_DIR.


You have a fancy backup system like this, involving hundreds of Gigs,
and it can't apply a simple filter to avoid .pyc and .obj files?!

-Peter
Jul 18 '05 #6

Bengt,

Thanks for your feedback...

MvL> http://www.python.org/peps/pep-0304.html

MvL> Comments on the PEP are encouraged: ...

Bengt> Personally, I am a minimalist when it comes to environment
Bengt> variables. IMO that name space has the same problems as any
Bengt> global namespace, and since a single default set of user's
Bengt> environment variables tends to presented to most programs s/he
Bengt> runs from a command window, the name space usage tends towards a
Bengt> hodgepodge and/or wrapping apps in setup scripts (which can work
Bengt> fine, but I still don't like it as a standard way to go).

You can set sys.bytecodebase explicitly, though you might have trouble
setting it early enough to push all your .pyc files where you want. Note
that the patch associated with PEP 304 is written in C (import.c and
sysmodule.c are affected).

Bengt> Provision for admin/root level config data separate from user
Bengt> preference and session state type config data should be made as
Bengt> necessary and desirable, but secondary/user config data search
Bengt> should be controllable by the primary/root/admin config data
Bengt> (which e.g. could say to ignore or use
Bengt> user-controlled/attempted-control environment variables etc.).

Do you want Python to locate and load a config file at startup? By the time
a config file parser is loaded and sys.bytecodebase set, lots of .pycs may
well have already been generated in the wrong place. This shouldn't happen
with the patch I created.

Bengt> This would seem to imply a standard place to look for root/admin
Bengt> level config data, not directed by env variable. E.g., a
Bengt> read-only file in the same directory as the python interpreter
Bengt> executable, with, say, .conf or .cfg appended to the name. *That*
Bengt> file can then specify how/whether to look for user config stuff
Bengt> etc., or specify password access to some features, etc. etc., if
Bengt> we wind up doing restricted exec stuff.

Like I said, lots of other stuff will likely have been imported by the time
you realize your .pyc files belong "over there" instead of where they're
being written, or were you volunteering to write a C version of ConfigParser
which is statically linked into the interpreter? ;-)

Skip

Jul 18 '05 #7

Rim> As Bengt suggest, adding yet another env var to control the
Rim> operation of an application polutes the name space for all
Rim> applications, but each app only needs to look at the variables it
Rim> is concerned with. PYTHONBYTECODEBASE is a cleaner approach that
Rim> has less overhead than a configuration file. Also, env variables
Rim> are directly usable in makefiles, which makes them very attractive
Rim> and easy to use for managing where compiler output goes.

You can also set sys.bytecodebase in your application code instead of
relying on PYTHONBYTECODEBASE. That may not be helpful when doing a big pyc
compile, as when running compileall.py. In that case, you can simply set
PYTHONBYTECODE base for that specific command. Still, in your environment,
it sounds like you'd want to set it in the /etc/profile (or similar). It
could still be set to a user-specific directory, just one that's on a
filesystem which is not backed up.

The intention was that the most common usage would be for a person to
execute something like

PYTHONBYTECODEBASE=/tmp/$USER/python ; export PYTHONBYTECODEBASE

in their .profile (or equivalent for non-sh-style shells or platforms).
Clearly, other usage is possible however.

Skip

Jul 18 '05 #8
Rim
> The intention was that the most common usage would be for a person to
execute something like

PYTHONBYTECODEBASE=/tmp/$USER/python ; export PYTHONBYTECODEBASE


This is exactly how I intent to use it.

Regards,
-Rim
Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

46
by: Jon Perez | last post by:
Can one run a 1.5 .pyc file with the 2.x version interpreters and vice versa? How about running a 2.x .pyc using a 2.y interpreter?
5
by: LutherRevisited | last post by:
This may be a dumb question, but are there any practical advantages of compiling a python application to *.pyo, or *.pyc? I haven't noticed any difference in the performance of text *.py or a...
1
by: Bo Jacobsen | last post by:
I have a number of files compiled to bytecode using py_compile.compile(). The .pyc files can be invoked by python directly ($python file.pyc), but "loading" them by execfile just throws an...
33
by: Maurice LING | last post by:
Hi, I've been using Python for about 2 years now, for my honours project and now my postgrad project. I must say that I am loving it more and more now. From my knowledge, Python bytecodes are...
0
by: sxanth | last post by:
>What puzzles me, though, are bytecodes 17, 39 and 42 - surely these aren't >reachable? Does the compiler just throw in a default 'return None' >epilogue, with routes there from every code path,...
4
by: kwatch | last post by:
Hi, It is possible to get bytecode from code object. Reversely, is it possible to create code object from bytecode? ex. ## python code (not a module) pycode = '''\ print "<ul>\n" for item...
2
by: warhero | last post by:
First question, I can't seem to get any python bytecode to be produced. I've tried different techniques from chapter 30.8 to chapter 31 of the python guide.. I was under the assumption that after...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.