473,497 Members | 2,158 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Rewriting a bash script in python

I wrote a trivial backup script that does the following:

If tonight is the first day of the month:

save last month's archive
do a full backup of the system

else:

save last night's differential backup
back up all changes relative to the current month's full backup

endif

I wrote this in bash and although it's worked w/o a glitch for the last
couple of months .. I'm not comfortable with .. "the look & feel of it".

I am thinking of rewriting it in python using OOP tactics/strategy.

Please advise.

Thanks!
CJ

Nov 5 '08 #1
15 1977
Chris Jones <cj******@gmail.comwrites:
I am thinking of rewriting it in python using OOP tactics/strategy.

Please advise.
I advise you not to have the object-oriented programming hammer as
your only tool, because it's easy to then treat every problem as
though it were a nail.

What you describe seems trivially a procedural sequence of branching
steps using simple types from the standard library. Use functions to
encapsulate the conceptually-related steps, and call those functions
in a simple branch as you demonstrated. I see no need to apply
object-oriented techniques for this problem.

import datetime

def is_full_backup_required(when):
""" Determine whether a full backup is required for datetime ‘when’ """
result = False
if determine_whether_full_backup_required_for_datetim e(when):
result = True
return result

def most_recent_incremental_backup(when):
""" Return the most recent incremental backup before datetime ‘when’ """
backup = most_recent_backup(when)
return backup

def save_previous_full_archive(when):
""" Save the full archive previous to datetime ‘when’ """

def save_previous_incremental_archive(when):
""" Save the incremental archive previous to datetime ‘when’ """

def perform_full_backup(system):
""" Do a full backup of the system """

def perform_incremental_backup(system):
""" Do an incremental backup of the system """

def get_this_system():
""" Return the current system for backups """
system = determine_what_this_system_is()
return system

system = get_this_system()
when = datetime.datetime.now()
if is_full_backup_required(when):
save_previous_month_archive(when)
perform_full_backup(system)
else:
save_previous_incremental_archive(when)
perform_incremental_backup(system)
I don't see any need for creating new classes for any of this, unless
a “system” is more complicated than just a string for the system
name, or a “backup” is more complicated than a tarball file.

Instead, the problem seems one best solved simply by abstracting the
steps involved at each point of the *procedure*, as I've done in my
example. Each one of those functions could be very simple or could be
complex enough to spawn a whole host of helper functions, possibly
even separate modules. That seems a more useful approach than trying
to force-fit object-orientation to this procedural problem.

--
\ “Some mornings, it's just not worth chewing through the leather |
`\ straps.” —Emo Philips |
_o__) |
Ben Finney
Nov 5 '08 #2
In message <ma**************************************@python.o rg>, Chris
Jones wrote:
I wrote this in bash and although it's worked w/o a glitch for the last
couple of months .. I'm not comfortable with .. "the look & feel of it".
Engineering rule #1: if it works, don't fix it.
Nov 5 '08 #3
On Wed, 05 Nov 2008 17:42:43 +1300, Lawrence D'Oliveiro <ld*@geek-central.gen.new_zealandwrote:
In message <ma**************************************@python.o rg>, Chris
Jones wrote:
>I wrote this in bash and although it's worked w/o a glitch for the last
couple of months .. I'm not comfortable with .. "the look & feel of it".

Engineering rule #1: if it works, don't fix it.
Especially if it handles your backups ;-)

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.se R'lyeh wgah'nagl fhtagn!
Nov 5 '08 #4
On Tue, 04 Nov 2008 22:11:37 -0500, Chris Jones <cj******@gmail.comwrote:
I wrote a trivial backup script that does the following:

If tonight is the first day of the month:

save last month's archive
do a full backup of the system

else:

save last night's differential backup
back up all changes relative to the current month's full backup

endif

I wrote this in bash and although it's worked w/o a glitch for the last
couple of months .. I'm not comfortable with .. "the look & feel of it".
This sounds like a job for cron(8) -- it's designed for exactly these
things: doing things based on calendar events. I assume you already
run your backups from cron rather than staying up every night to do
your stuff, so what about changing this in your crontab

00 1 * * * full_or_incremental_backup

to this?

00 1 1 * * backup --full
00 1 2-31 * * backup --incremental

(Or maybe two different scripts suit you better.)

That way you needn't implement your own logic for deciding when to do
a full backup. (You still need a way to find the latest full backup,
though.)

/Jorgen

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.se R'lyeh wgah'nagl fhtagn!
Nov 5 '08 #5
On Wed, 05 Nov 2008 17:12:08 +0000, Jorgen Grahn wrote:
On Wed, 05 Nov 2008 17:42:43 +1300, Lawrence D'Oliveiro
>Engineering rule #1: if it works, don't fix it.

Especially if it handles your backups ;-)
Well, if it handles your backups it doesn't work. It just pretends until
you really *need* the backed up data. ;-)

Ciao,
Marc 'BlackJack' Rintsch
Nov 5 '08 #6
On Tue, Nov 04, 2008 at 11:11:17PM EST, Ben Finney wrote:
Chris Jones <cj******@gmail.comwrites:
I am thinking of rewriting it in python using OOP tactics/strategy.

Please advise.

I advise you not to have the object-oriented programming hammer as
your only tool, because it's easy to then treat every problem as
though it were a nail.
Naturally! But then.. knowing when to use a given programming technique
is part of the problem. I have a spouse who just ruined a perfectly good
chef's knife that cost me about $80.00 because she didn't bother looking
for a screwdriver .. and thank goodness I didn't have to take her the ER
to boot.
What you describe seems trivially a procedural sequence of branching
steps using simple types from the standard library.
This is for a standalone laptop.

I went to the trouble of designing, coding, and cron'ing my B/U script
for two reasons:

1. I wanted the laptop backed up regularly and -- see other posts in
this thread.. "restorable".

2. The more I worked on this project, the more interesting it became ..
along "what happens if" lines .. and less trivial.

Now that's over and done with.

But then I started thinking .. what if for instance I had to scale my
effort from my single system to a large "data center" with hundred of
hosts .. with different backup policies .. what if I had to take into
account time slots where the backups ran .. had to consider bandwidth &
performance issues .. media costs & their rotation and such like.

In a vague way I had this feeling that if added machines one at a time,
my current script would not scale all that well.

I hope I'm making some sense but this is why, rather than take your word
for it, which I tend to intuitively agree with anyway .. I would much
prefer trying to oop this myself and try to figure why in this instance
using object-orientation would probably only add much overhead and
complexity without any real benefits.

IOW--we only learn by our own mistakes.
Use functions to encapsulate the conceptually-related steps, and call
those functions in a simple branch as you demonstrated. I see no need
to apply object-oriented techniques for this problem.
import datetime

def is_full_backup_required(when):
""" Determine whether a full backup is required for datetime 'when' """
result = False
if determine_whether_full_backup_required_for_datetim e(when):
result = True
return result

def most_recent_incremental_backup(when):
""" Return the most recent incremental backup before datetime 'when' """
backup = most_recent_backup(when)
return backup

def save_previous_full_archive(when):
""" Save the full archive previous to datetime 'when' """

def save_previous_incremental_archive(when):
""" Save the incremental archive previous to datetime 'when' """

def perform_full_backup(system):
""" Do a full backup of the system """

def perform_incremental_backup(system):
""" Do an incremental backup of the system """

def get_this_system():
""" Return the current system for backups """
system = determine_what_this_system_is()
return system

system = get_this_system()
when = datetime.datetime.now()
if is_full_backup_required(when):
save_previous_month_archive(when)
perform_full_backup(system)
else:
save_previous_incremental_archive(when)
perform_incremental_backup(system)
Thank you very much for providing this example skeleton.
I don't see any need for creating new classes for any of this, unless
a "system" is more complicated than just a string for the system
name, or a "backup" is more complicated than a tarball file.
Well .. relative to my far-fetched example of scaling above .. I thought
that maybe a "target system" -- one to backup, that is .. would be an
obvious class .. that one could find some criteria or other that would
categorize systems into subgroups .. with related backup policies, for
instance .. and each actual system would end up being an instance of
these latter "sub-classes".

Likewise a "backup" .. full or incremental .. with a system name and a
calendar date as natural attributes ..?

No harm in trying?

:-)
Instead, the problem seems one best solved simply by abstracting the
steps involved at each point of the *procedure*, as I've done in my
example.
To loosely quote someone's post on this thread .. I have AFAIK already
"solved" my problem .. So why bother? .. As indicated above this is a
case of looking at the problem from a different angle and hopefully
learn something.
Each one of those functions could be very simple or could be
complex enough to spawn a whole host of helper functions, possibly
even separate modules. That seems a more useful approach than trying
to force-fit object-orientation to this procedural problem.
Begs the question .. how do I tell what is an object-oriented vs. a
procedural problem?

Also, what classes of problems would be naturals for OOP?

Thanks!
CJ


Nov 6 '08 #7
2008/11/6 Chris Jones <cj******@gmail.com>:
Begs the question .. how do I tell what is an object-oriented vs. a
procedural problem?
Practice, largely, so you're doing the right thing (provided you don't
trust your /real/ backup data to a tutorial program). If you find that
the program is at its neatest when it's just a couple of methods in
one class, it was a procedural problem.

Actually, from your description of the problem -- that the interest is
in all the "what if" cases -- suggests that it's a specification
problem rather than a design or implementation problem, so rather than
recoding it it might be worth looking at various approaches to
requirements capture and specification.

--
Tim Rowe
Nov 6 '08 #8
In message <6n************@mid.uni-berlin.de>, Marc 'BlackJack' Rintsch
wrote:
Well, if it handles your backups it doesn't work. It just pretends until
you really *need* the backed up data. ;-)
That's why a backup system needs to be absolutely simple as possible. Avoid
complicated formats, a straightforward mirrored filesystem (e.g. updated
via rsync) is the easiest to verify that you can read your precious data.
Nov 6 '08 #9
In message <ma**************************************@python.o rg>, Chris
Jones wrote:
But then I started thinking .. what if for instance I had to scale my
effort from my single system to a large "data center" with hundred of
hosts .. with different backup policies .. what if I had to take into
account time slots where the backups ran .. had to consider bandwidth &
performance issues .. media costs & their rotation and such like.
<http://www.google.co.nz/search?q=second+system+effect>
Nov 6 '08 #10
On Wed, Nov 05, 2008 at 08:12:40PM EST, Tim Rowe wrote:
2008/11/6 Chris Jones <cj******@gmail.com>:
Begs the question .. how do I tell what is an object-oriented vs. a
procedural problem?

Practice, largely, so you're doing the right thing (provided you don't
trust your /real/ backup data to a tutorial program). If you find that
the program is at its neatest when it's just a couple of methods in
one class, it was a procedural problem.
but you do have to go through the motions .. to "find" it ..
Actually, from your description of the problem -- that the interest is
in all the "what if" cases -- suggests that it's a specification
problem rather than a design or implementation problem, so rather than
recoding it it might be worth looking at various approaches to
requirements capture and specification.
I'm one of them that need to get their hands dirty a bith .. lack the
power of abstraction. Need to do some building before I can specify.
Nov 6 '08 #11
In message <ma**************************************@python.o rg>, Chris
Jones wrote:
I'm one of them that need to get their hands dirty a bith .. lack the
power of abstraction. Need to do some building before I can specify.
Nothing wrong with rapid prototyping. :)
Nov 6 '08 #12
On Wed, Nov 05, 2008 at 09:21:38PM EST, Lawrence D'Oliveiro wrote:
In message <6n************@mid.uni-berlin.de>, Marc 'BlackJack' Rintsch
wrote:
Well, if it handles your backups it doesn't work. It just pretends until
you really *need* the backed up data. ;-)

That's why a backup system needs to be absolutely simple as possible. Avoid
complicated formats, a straightforward mirrored filesystem (e.g. updated
via rsync) is the easiest to verify that you can read your precious data.
--
http://mail.python.org/mailman/listinfo/python-list
_Any_ system and backup systems in particular.

There are tradeoffs though .. the above is beyond my budget ..

Couldn't agree more with what you're saying, though.

Less is better!

Nov 6 '08 #13
2008/11/6 Lawrence D'Oliveiro <ld*@geek-central.gen.new_zealand>:
Nothing wrong with rapid prototyping. :)
That's one requirements capture methodology, certainly. Actually,
there's quite a lot wrong with rapid prototyping, but there's quite a
lot wrong with all other requirements capture methodologies too, so
rapid prototyping is up there with the rest of them.
--
Tim Rowe
Nov 6 '08 #14
In message <ma**************************************@python.o rg>, Tim Rowe
wrote:
Actually, there's quite a lot wrong with rapid prototyping, but there's
quite a lot wrong with all other requirements capture methodologies too,
so rapid prototyping is up there with the rest of them.
Sounds like what Churchill said about democracy: that it's the worst system,
apart from all the others. :)
Nov 6 '08 #15
On Wed, Nov 05, 2008 at 09:23:02PM EST, Lawrence D'Oliveiro wrote:
In message <ma**************************************@python.o rg>, Chris
Jones wrote:
But then I started thinking .. what if for instance I had to scale my
effort from my single system to a large "data center" with hundred of
hosts .. with different backup policies .. what if I had to take into
account time slots where the backups ran .. had to consider bandwidth &
performance issues .. media costs & their rotation and such like.

<http://www.google.co.nz/search?q=second+system+effect>
Thanks..

I am not building anything .. planning to implement ..

I am a novice getting my feet wet.

Nov 8 '08 #16

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
3259
by: Tom Purl | last post by:
I just wrote a Python script that is going to be called from bash script. If the Python script fails, I want the bash script to also stop running. Unfortunately, I can't seem to get that to work. ...
11
2976
by: Magnus Jonneryd | last post by:
Hi, I'm planning on writing a program that interactively is fed input via a shell (bash). I also want to be able to write a shell script that executes various commands related to my program. In...
16
3917
by: John Salerno | last post by:
Hi all. I just installed Ubuntu and I'm learning how to use the bash shell. Aside from the normal commands you can use, I was wondering if it's possible to use Python from the terminal instead of...
6
3928
by: Ishpeck | last post by:
I'm using Python to automate testing software for my company. I wanted the computers in my testing lab to automatically fetch the latest version of the python scripts from a CVS repository and...
3
3843
by: Gros Bedo | last post by:
Hello :-) I have a question about Python and Linux shell. I have a python program which is permanently resident in the end-user system. I'm currently producing a RPM package, and it works nicely....
1
2970
by: Gros Bedo | last post by:
Yes I've seen that each python script calls its own instance of Python. Buthow to know which is the good one in bash ? Is there a command that gets the parameters of process, so I could use grep to...
0
1990
by: norseman | last post by:
Gros Bedo wrote: ============================== Yes. man ps explains try ps -AFL | grep then kill -9 found (check it more than twice) 1) If your script is known to hang use what...
1
3245
by: Fredrik Lundh | last post by:
John Lawrence wrote: doesn't exactly work for Python scripts, though: $ cat env.py #!/usr/bin/env python import os os.environ = "hello"
1
1736
by: James Mills | last post by:
On Wed, Oct 22, 2008 at 10:32 AM, gita ziabari <gitaziabari@gmail.comwrote: The only possibility is to use the subprocess module and this is a more general problem, one involving the invocation...
0
6993
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
7162
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
7197
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
1
6881
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
7375
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
5456
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development projectplanning, coding, testing,...
0
4584
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
1411
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
0
287
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.