I am aware, that it is maybe the wrong group to ask this question, but
as I would like to know the history of past file operations from within
a Python script I see a chance, that someone in this group was into it
already and is so kind to share here his experience.
I have put already much efforts into this subject googling around, but
up to now in vain. Best option I encountered yet is usage of
the Greyware 'System Change Log' service which monitors disks for
changes ( http://www.greyware.com/software/sys...log/index.asp),
but in own tests it turned out, that the created log file does not cover
all file events as e.g. it is not possible to detect when a file is
moved to a new directory (creation of a new file is logged, but deletion
is not, not mentioning I would expect a file 'move' event).
The own Windows logging service rejected to start on my XP SP2 system
for unknown to me reasons - I don't know how to get it to work (yes, I
have used the administrator account).
I can't believe, that using NTFS file system in Microsoft Windows 2000
or XP it is not possible to track file events as:
- updating/modifying of an existing file/directory
- deleting an existing file/directory
- creating a new file/directory
- _moving_ an existing file/directory (should _NOT_ be covered by the
event duo of deleting an existing and creating a new file/directory)
Any hints towards enlightenment?
Claudio Grondi 9 2217
you want a directory watching daemon. it isn't hard at all to build
from scratch.
first, determine which directories should be watched.
then, os.walk each directory, building a mapping from filename to mtime
[modified time; os.path.getmtime].
next is your main event loop. this while loop consists of os.walk-ing
each directory again, comparing the current mtime to the corresponding
entry in the mapping. if they differ, or if a filename isn't in the
mapping, something happened, at which point you can logick out whether
a file was moved, deleted, changed, or created.
so many folks have looked for this that i'll just write a generic one
and put it in the cheeseshop. look for "dirmon" in about a week.
Claudio Grondi wrote:
I am aware, that it is maybe the wrong group to ask this question, but
as I would like to know the history of past file operations from within
a Python script I see a chance, that someone in this group was into it
already and is so kind to share here his experience.
I have put already much efforts into this subject googling around, but
up to now in vain. Best option I encountered yet is usage of
the Greyware 'System Change Log' service which monitors disks for
changes (http://www.greyware.com/software/sys...log/index.asp),
but in own tests it turned out, that the created log file does not cover
all file events as e.g. it is not possible to detect when a file is
moved to a new directory (creation of a new file is logged, but deletion
is not, not mentioning I would expect a file 'move' event).
The own Windows logging service rejected to start on my XP SP2 system
for unknown to me reasons - I don't know how to get it to work (yes, I
have used the administrator account).
I can't believe, that using NTFS file system in Microsoft Windows 2000
or XP it is not possible to track file events as:
- updating/modifying of an existing file/directory
- deleting an existing file/directory
- creating a new file/directory
- _moving_ an existing file/directory (should _NOT_ be covered by the
event duo of deleting an existing and creating a new file/directory)
Any hints towards enlightenment?
Claudio Grondi
Claudio Grondi wrote:
I am aware, that it is maybe the wrong group to ask this question, but
as I would like to know the history of past file operations from within
a Python script I see a chance, that someone in this group was into it
already and is so kind to share here his experience.
I can't believe, that using NTFS file system in Microsoft Windows 2000
or XP it is not possible to track file events as:
- updating/modifying of an existing file/directory
- deleting an existing file/directory
- creating a new file/directory
- _moving_ an existing file/directory (should _NOT_ be covered by the
event duo of deleting an existing and creating a new file/directory)
Any hints towards enlightenment?
Claudio Grondi
On the offchance that you haven't seen it, you might
look at this: http://timgolden.me.uk/python/win32_...rectorychanges
but since it doesn't fulfil your criterion of *not*
representing renames by a delete and an add, it may
well not be suitable. Apart from that, I think it does
what you want.
TJG
"faulkner" <fa*********@comcast.netwrote in message
news:11**********************@b28g2000cwb.googlegr oups.com...
you want a directory watching daemon. it isn't hard at all to build
from scratch.
first, determine which directories should be watched.
then, os.walk each directory, building a mapping from filename to mtime
[modified time; os.path.getmtime].
next is your main event loop. this while loop consists of os.walk-ing
each directory again, comparing the current mtime to the corresponding
entry in the mapping. if they differ, or if a filename isn't in the
mapping, something happened, at which point you can logick out whether
a file was moved, deleted, changed, or created.
so many folks have looked for this that i'll just write a generic one
and put it in the cheeseshop. look for "dirmon" in about a week.
While I am a fan of "brute force"
"faulkner" <fa*********@comcast.netwrote in message
news:11**********************@b28g2000cwb.googlegr oups.com...
you want a directory watching daemon. it isn't hard at all to build
from scratch.
first, determine which directories should be watched.
then, os.walk each directory, building a mapping from filename to mtime
[modified time; os.path.getmtime].
next is your main event loop. this while loop consists of os.walk-ing
each directory again, comparing the current mtime to the corresponding
entry in the mapping. if they differ, or if a filename isn't in the
mapping, something happened, at which point you can logick out whether
a file was moved, deleted, changed, or created.
so many folks have looked for this that i'll just write a generic one
and put it in the cheeseshop. look for "dirmon" in about a week.
Ahem... (sorry for premature usenet-post-ication...)
While I am a big fan of "brute force", there are OS services (at least on
Windows) for doing just this function, with asynchronous callbacks when
files are created, deleted, etc.
Here is a link that does a much better comparison of several options than I
could (including your brute force version): http://tgolden.sc.sabren.com/python/...r_changes.html
Good luck!
-- Paul
faulkner wrote:
you want a directory watching daemon. it isn't hard at all to build
from scratch.
first, determine which directories should be watched.
then, os.walk each directory, building a mapping from filename to mtime
[modified time; os.path.getmtime].
next is your main event loop. this while loop consists of os.walk-ing
each directory again, comparing the current mtime to the corresponding
entry in the mapping. if they differ, or if a filename isn't in the
mapping, something happened, at which point you can logick out whether
a file was moved, deleted, changed, or created.
so many folks have looked for this that i'll just write a generic one
and put it in the cheeseshop. look for "dirmon" in about a week.
Yes, I _know_ about it and exactly this knowledge is the reason I am
looking for tracking single file system related _events_ as I expect a
professional operating system like Windows to provide such service. If
there is none, this will be sure a severe reason to go for Linux if it
has such one instead of going for a SVN server or special file systems
if there are any.
Has someone experience with SVN handling million(s) of files?
The problem is, that brute force applied to large amount of
files/directories is not a convenient way to backup/synchronize the few
new/changed/deleted/moved files/directories multiple times a day as the
brute force approach just makes the hard drive(s) unnecessary wasting
much energy and getting hot.
Claudio Grondi
>
Claudio Grondi wrote:
>>I am aware, that it is maybe the wrong group to ask this question, but as I would like to know the history of past file operations from within a Python script I see a chance, that someone in this group was into it already and is so kind to share here his experience.
I have put already much efforts into this subject googling around, but up to now in vain. Best option I encountered yet is usage of the Greyware 'System Change Log' service which monitors disks for changes (http://www.greyware.com/software/sys...log/index.asp), but in own tests it turned out, that the created log file does not cover all file events as e.g. it is not possible to detect when a file is moved to a new directory (creation of a new file is logged, but deletion is not, not mentioning I would expect a file 'move' event). The own Windows logging service rejected to start on my XP SP2 system for unknown to me reasons - I don't know how to get it to work (yes, I have used the administrator account).
I can't believe, that using NTFS file system in Microsoft Windows 2000 or XP it is not possible to track file events as:
- updating/modifying of an existing file/directory - deleting an existing file/directory - creating a new file/directory - _moving_ an existing file/directory (should _NOT_ be covered by the event duo of deleting an existing and creating a new file/directory)
Any hints towards enlightenment?
Claudio Grondi
Tim Golden wrote:
Claudio Grondi wrote:
>I am aware, that it is maybe the wrong group to ask this question, but as I would like to know the history of past file operations from within a Python script I see a chance, that someone in this group was into it already and is so kind to share here his experience.
I can't believe, that using NTFS file system in Microsoft Windows 2000 or XP it is not possible to track file events as:
- updating/modifying of an existing file/directory - deleting an existing file/directory - creating a new file/directory - _moving_ an existing file/directory (should _NOT_ be covered by the event duo of deleting an existing and creating a new file/directory)
Any hints towards enlightenment?
Claudio Grondi
On the offchance that you haven't seen it, you might
look at this:
http://timgolden.me.uk/python/win32_...rectorychanges
but since it doesn't fulfil your criterion of *not*
representing renames by a delete and an add, it may
well not be suitable. Apart from that, I think it does
what you want.
TJG
It seems, that it will be necessary to use some logic based on the
sequence of events to exactly detect rename and move changes done to
files/directories, but in principle it is the best approach I know about
yet.
Thank you!
By the way:
Is there something similar/same available for Linux?
Claudio Grondi
Tim Golden wrote:
Claudio Grondi wrote:
>I am aware, that it is maybe the wrong group to ask this question, but as I would like to know the history of past file operations from within a Python script I see a chance, that someone in this group was into it already and is so kind to share here his experience.
I can't believe, that using NTFS file system in Microsoft Windows 2000 or XP it is not possible to track file events as:
- updating/modifying of an existing file/directory - deleting an existing file/directory - creating a new file/directory - _moving_ an existing file/directory (should _NOT_ be covered by the event duo of deleting an existing and creating a new file/directory)
Any hints towards enlightenment?
Claudio Grondi
On the offchance that you haven't seen it, you might
look at this:
http://timgolden.me.uk/python/win32_...rectorychanges
but since it doesn't fulfil your criterion of *not*
representing renames by a delete and an add, it may
well not be suitable. Apart from that, I think it does
what you want.
TJG
Here a small update to the code at http://timgolden.me.uk/python/win32_...rectorychanges
:
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something"
5 : "Renamed to something",
}
The correction above is according to entries:
#define FILE_ACTION_ADDED 0x00000001
#define FILE_ACTION_REMOVED 0x00000002
#define FILE_ACTION_MODIFIED 0x00000003
#define FILE_ACTION_RENAMED_OLD_NAME 0x00000004
#define FILE_ACTION_RENAMED_NEW_NAME 0x00000005
in ..\PlatformSDK\Include\WinNT.h
Claudio Grondi
Claudio Grondi wrote:
Here a small update to the code at http://timgolden.me.uk/python/win32_...rectorychanges
:
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something"
5 : "Renamed to something",
}
The correction above is according to entries:
#define FILE_ACTION_ADDED 0x00000001
#define FILE_ACTION_REMOVED 0x00000002
#define FILE_ACTION_MODIFIED 0x00000003
#define FILE_ACTION_RENAMED_OLD_NAME 0x00000004
#define FILE_ACTION_RENAMED_NEW_NAME 0x00000005
in ..\PlatformSDK\Include\WinNT.h
Claudio Grondi
Thanks. I've updated the site.
TJG This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: j vickroy |
last post by:
Hello,
I'm trying to understand the behavior of the Python 2.3 logging module (MS
Windows 2k) with regard to RotatingFileHandler. The following script
illustrates a puzzling problem. What is...
|
by: j vickroy |
last post by:
My system:
MSW XP professional
Python 2.3.3
logging package: 0.4.9.2
My problem:
The log_test3.py script, provided with the logging package distribution,
generates an unexpected message:
No...
|
by: Neil Benn |
last post by:
Hello,
I'm running a test and having issues with logging, if I call
logging.shutdown() and then want to start the logging going again then I
get a problem as if I call shutdown, I can't get the...
|
by: qwweeeit |
last post by:
Hi all,
I wonder if it is possible to change (temporarily) a built-in function
for logging purposes.
Let me explain:
I want to log all the 'open' operations, recording the file to be
opened, the...
|
by: Oliver Eichler |
last post by:
Hi,
I experience several exceptions from python's logging system when using the
rollover feature on Windows.
Traceback (most recent call last):
File "c:\Python24\lib\logging\handlers.py",...
|
by: timb |
last post by:
Hi,
does any have any sample code of the above?
i have tried the example from the help but am unable to get my windows
service to create a new logfile and start logging to it.
I am able to...
|
by: Einar Høst |
last post by:
Hi,
I'm getting into the Trace-functionality in .NET, using it to provide some
much-needed logging across dlls in the project we're working on. However,
being a newbie, I'm wondering if some...
|
by: flupke |
last post by:
Hi,
i'm getting errors with the log module concerning RotatingFileHandler.
I'm using Python 2.4.3 on Windows XP SP2. This used to work in previous
python versions but since i upgraded to 2.4.3...
|
by: Chris Curvey |
last post by:
Hi all,
I just upgraded to 2.4.3 (from 2.4.1) on Windows. Now each time I run
my unit tests, they always throw this error at the end of the test run:
Error in atexit._run_exitfuncs:...
|
by: ryjfgjl |
last post by:
ExcelToDatabase: batch import excel into database automatically...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: Vimpel783 |
last post by:
Hello!
Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
|
by: ArrayDB |
last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
|
by: PapaRatzi |
last post by:
Hello,
I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: Defcon1945 |
last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
|
by: af34tf |
last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
| |