.pyc files??

[Ishwar Rattan]

Say a filea.py contains a number of functions defined in it (using
def afn(): etc.)

in pyhton, filea is imported to use afn (filea.afn() etc.), it cerates
a filea.pyc file, what is the purpose of this file?

-ishwar

[Jeremy Jones]

Ishwar Rattan wrote:

Say a filea.py contains a number of functions defined in it (using
def afn(): etc.)

in pyhton, filea is imported to use afn (filea.afn() etc.), it cerates
a filea.pyc file, what is the purpose of this file?

-ishwar

This file contains compiled bytecode that the Python interpreter uses.
If, in your example, filea.pyc exists and its timestamp is newer than
filea.py, then the interpreter uses filea.pyc without even attempting to
recompile it (which saves the time spent compiling in subsequent runs).
If filea.pyc doesn't exist, or its timestamp is older than filea.py, the
Python interpreter compiles filea.py to bytecode and writes it to
filea.pyc (if it has permissions to do so).

[Peter Hansen]

Jeremy Jones wrote:

This file contains compiled bytecode that the Python interpreter uses.
If, in your example, filea.pyc exists and its timestamp is newer than
filea.py, then the interpreter uses filea.pyc without even attempting to
recompile it (which saves the time spent compiling in subsequent runs).
If filea.pyc doesn't exist, or its timestamp is older than filea.py, the
Python interpreter compiles filea.py to bytecode and writes it to
filea.pyc (if it has permissions to do so).

Minor nit-picking: I'm fairly sure that the interpreter just compares
the timestamp for equality, not for "earlier than". I could be
wrong, but I think it's a case of "if the .pyc embedded timestamp
doesn't match the timestamp of the corresponding .py file exactly,
then recompile the .pyc file".

And a quick test at the command line appears to confirm this...

-Peter

[Jeff Shannon]

Peter Hansen wrote:

Jeremy Jones wrote:

This file contains compiled bytecode that the Python interpreter
uses. If, in your example, filea.pyc exists and its timestamp is
newer than filea.py, then the interpreter uses filea.pyc without even
attempting to recompile it (which saves the time spent compiling in
subsequent runs). If filea.pyc doesn't exist, or its timestamp is
older than filea.py, the Python interpreter compiles filea.py to
bytecode and writes it to filea.pyc (if it has permissions to do so).

Minor nit-picking: I'm fairly sure that the interpreter just compares
the timestamp for equality, not for "earlier than". I could be
wrong, but I think it's a case of "if the .pyc embedded timestamp
doesn't match the timestamp of the corresponding .py file exactly,
then recompile the .pyc file".

And a quick test at the command line appears to confirm this...

Really? All my .pyc files are timestamped later than the .py file that
they were generated from, but they don't seem to be recreated when I
import the module...

[ach]$ ls -l *.py*
-rwxr-xr-x 1 pick pick 1709 Aug 3 18:36 http_multipart.py
-rw-r--r-- 1 pick pick 2437 Aug 4 00:12 http_multipart.pyc
[ach]$ python
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import http_multipart

[ach]$ ls -l *.py*
-rwxr-xr-x 1 pick pick 1709 Aug 3 18:36 http_multipart.py
-rw-r--r-- 1 pick pick 2437 Aug 4 00:12 http_multipart.pyc
[ach]$

Jeff Shannon
Technician/Programmer
Credit International

[Andrew Dalke]

[on when .pyc files are regenerated from .py files]

The .pyc file contains the modification timestamp
in the file itself. It's this embedded timestamp
which is used to check if the .py needs to be rebuilt,
and not a comparison of the respective timestamps
for the file. (Surprised me, I also thought it was the
latter.)

The code is in dist/src/Python/import.c. See
check_compiled_module, which checks the first long
as the magic number then the second long for the
timestamp to use to compare with the mtime of the
..py file. Here's a way to test it:

# Create a simple Python file

% rm spam.py*
% cat > spam.py
1
% python
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import spam
% ls -l spam.py*
-rw-r--r-- 1 dalke staff 2 2 Sep 15:23 spam.py
-rw-r--r-- 1 dalke staff 96 2 Sep 15:23 spam.pyc
%
# Let's gets some information about the .pyc

% sum spam.pyc
16387 1 spam.pyc
% od -c spam.pyc
0000000 c 362 \r \n ? 217 7 A c \0 \0 \0 \0 \0 \0 \0
0000020 \0 001 \0 \0 \0 @ \0 \0 \0 s \b \0 \0 \0 d \0
0000040 \0 001 d 001 \0 S ( 002 \0 \0 \0 i 001 \0 \0 \0
0000060 N ( \0 \0 \0 \0 ( \0 \0 \0 \0 ( \0 \0 \0 \0
0000100 ( \0 \0 \0 \0 t 007 \0 \0 \0 s p a m . p
0000120 y t 001 \0 \0 \0 ? 001 \0 \0 \0 s \0 \0 \0 \0
0000140
%

# modify the mtime for the .py file

% touch spam.py
% python
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information. import spam
% ls -l spam.py*
-rw-r--r-- 1 dalke staff 2 2 Sep 15:23 spam.py
-rw-r--r-- 1 dalke staff 96 2 Sep 15:23 spam.pyc
%

# any changes? Looks like it.

% sum spam.pyc
21410 1 spam.pyc
% od -c spam.pyc
0000000 c 362 \r \n \ 217 7 A c \0 \0 \0 \0 \0 \0 \0
0000020 \0 001 \0 \0 \0 @ \0 \0 \0 s \b \0 \0 \0 d \0
0000040 \0 001 d 001 \0 S ( 002 \0 \0 \0 i 001 \0 \0 \0
0000060 N ( \0 \0 \0 \0 ( \0 \0 \0 \0 ( \0 \0 \0 \0
0000100 ( \0 \0 \0 \0 t 007 \0 \0 \0 s p a m . p
0000120 y t 001 \0 \0 \0 ? 001 \0 \0 \0 s \0 \0 \0 \0
0000140
%

# the difference was the '?' turned to a '\' in the 5th byte.
# That probably indicates a difference of 29 seconds.
# Now modify the .pyc's mtime

% touch 01010101 spam.pyc
% ls -l spam.py*
-rw-r--r-- 1 dalke staff 2 2 Sep 15:23 spam.py
-rw-r--r-- 1 dalke staff 96 1 Jan 2004 spam.pyc
% python
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information. import spam
%

# Notice, no change to the .pyc file. Its mtime is ignored
# and the .pyc is not rebuilt.

% ls -l spam.py*
-rw-r--r-- 1 dalke staff 2 2 Sep 15:23 spam.py
-rw-r--r-- 1 dalke staff 96 1 Jan 2004 spam.pyc
% sum spam.pyc
21410 1 spam.pyc
%

# Another way to see it is with the -v flag to Python.
# Again, the mtime for the .pyc is << that of the .py file.
# See that the .pyc is used without a recompile.

% python -v
[... many lines removed ...]
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import readline # dynamically loaded from
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/lib-dynload/readline.so import spam # spam.pyc matches spam.py
import spam # precompiled from spam.pyc [... many lines removed ...]
%

# Now tweak the mtime of the .py file and see that
# the .pyc is regenerated.

% touch spam.py
% python -v
[... many lines removed ...]
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import readline # dynamically loaded from
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/lib-dynload/readline.so import spam # spam.pyc has bad mtime
import spam # from spam.py
# wrote spam.pyc

[... many lines removed ...]
Andrew
da***@dalkescientific.com