Is there an obvious way to do this in python?

[H J van Rooyen]

Hi,

I want to write a small system that is transaction based.

I want to split the GUI front end data entry away from the file handling and
record keeping.

Now it seems almost trivially easy using the sockets module to communicate
between machines on the same LAN, so that I want to do the record keeping on one
machine.

I want to keep the "server" machine as simple as possible - just doing record
keeping on a stimulus response basis - I would prefer it to do one thing at a
time to completion because this style of operation, though limited in
performance, keeps a lot of hassles out of life - a transaction has either
completed, or it has not - recovery scenarios are relatively easy...

Up to this point, I don't have a problem - my toy system can create a dummy
transaction, and I can echo it from the "server" machine, with more than one
"user" machine running - so I think it is feasible to have several tens of "data
entry terminal" systems running, served by one not very strong machine.

Now what I would really like to do is to differentiate between the 'User"
machines, so that some can do a full range of transactions, and others a limited
range.

And I would like to make this flexible, so that it becomes easy to introduce new
transactions, without having to run around updating the code in all the user
machines, with the concomitant version number hassles.

And I would like to do the whole thing in python - so my question is this - is
it possible to do the equivalent of dynamic linking? - i.e. if I keep a list of
what a user is allowed to do - can I somehow send him just the bits he needs to
do the job, without having to change the static code on his machine? - it seems
to me that the eval() thingy could possibly do this for me, by sending it data
that makes it do import statements followed by calls to whatever... - will this
work, or is there a better way?

Or has all this been done already? - and no I don't want a web server and php
and browsers and Java and html or xml... - I want to write something that works
simply and reliably - its just short message accounting type data...

- Hendrik

[Bruno Desthuilliers]

H J van Rooyen wrote:

"Bruno Desthuilliers" <on***@xiludom. gro>wrote:
| H J van Rooyen wrote:

(snip)

| so if Pyro is for 'moving the scripts around' - Then that is what I must
look at
| very hard...
|
| It's not for "moving the scripts around", it's for remote objects - kind
| of like Java's RMI, but, well, much more pythonic !-). Now the point is
| that Python being very powerful when it comes to dynamism and
| introspection, it should be possible to have a common client that
| basically just knows how to connect to the application server (using
| pyro). Once connected, the client asks the server for a set of objects
| (forms, menus etc) and the corresponding data. These objects then use
| the same mechanism to interact with the server. It's basically similar
| to the interaction between a browser and a web app - in that the client
| is potentially able to run any application sent by the server -, but
| with much more specialized client and app server and another protocol -
| and no other language than Python.
|

This is getting more and more interesting - its not the simple minded mechanism
I had in mind, but it will achieve the same thing, and it exists already

WARNING : Actually, the only thing that "exists already" is Pyro (and of
course Python...) - all the rest is just me thinking out loud, and I by
no mean garantee that what I describe above is viable or sensible or
even reasonably implementable.

and I
can imagine it to be a very powerful mechanism, if I understand you correctly -
I am going to have to cry "time out!" to go and do some reading...

Indeed. And please don't hold it against me if it ends up being some
kind of Frankenstein's monster !-)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'on***@xiludom. gro'.split('@')])"

[H J van Rooyen]

From: "Bryan Olson" <fa*********@no where.org>
| H J van Rooyen wrote:
| [...]
| This is broadly what I had in mind, yes - but sort of down to
| a transaction
| level - this user does invoicing, this one enters cheques,
| this one does credit
| notes, and their supervisor can do all three, and in a
| different department its
| different because the jobs are different, but the invoicing
| GUI module is the
| same for wherever its used...
|
| Pretty much any business has to deal with those things. You are
| right about the need for transactions and authorization, and
| that you don't want to have to manually update all the clients
| whenever your modify the applications. Everyone has those
| problems, and consequently there's a large body of experience
| and conventional best practice. The highly redundant advice
| you've been getting is recommending that you to do those things
| that way most successful modern systems do them.
|
| No question what you describe calls for a DBMS. Further, this
| sounds like a job for a "three tier" architecture, what Dennis
| L. Bieber explained as [db-server] <-[app-server] <-[client].
| In the old days a three-tier system called for a large
| data-processing staff; today one guy can set up a simple one in
| an afternoon.
|
| In a three-tier system, either the database, the server
| application can enforce authorization policy. In order for the
| database to enforce authorization, the application creates one
| database connection for each client user, and logs in as the
| user. That works well for most accounting-type systems, where
| all the users are known in advance. In systems such as web
| shopping carts, the application typically has its own user ID,
| and creates all database connections as itself. The app, not the
| database, is then responsible for keeping users from reading and
| altering each other's data.
|
| Trying to enforce authorization at the client is generally a
| mistake. The user has too much control over his local machine.
| He is not limited to running just the code you provide.
|
|
| Don't get too hung up on choice of DBMS, but stick with portable
| standard SQL. I bet you'll find your potential customers want it
| to run against the database they already have. The particular
| tables your system needs will become part of their larger
| schema.
|
|
| --
| --Bryan

Thank you - this is a sane summary of the general business case, and I
particularly like the bit about keeping the SQL portable - its something I did
not even consider...

The conventional best practice advice is swamping my original query of how to do
something in Python...

Just a note:

If conventional best practice is followed all the time, without question - how
is it ever made better?
but that is probably a question for a separate thread - please ignore it
here... - it has nothing to do with the Python language.

- Hendrik

[H J van Rooyen]

"Nick Vatamaniuc" <va******@gmail .comwrote:

| Hendrik,
|
| ---snip---
| Now part of the reason I would like to go the transaction type route
| instead of the "per user" route is robustness and maintainability , and
| the ability it would give me to introduce new transaction types easily
| - as I see it if say an invoice's GUI code is stable I would never have
| to touch it again even if I combine it with anything else, as it would
| have been designed from the start to combine with others of it's own
| ilk, under a kind of communications controller that is standard...
| ---snip---
|
| It could be possible to separate your GUI into various modules. So you
| could have for example:
| BillingGUI.py
| InvoicingGUI.py
| QueryGUI.py
| and so on.
| Then your central control application (Application.py ) would get the
| set of module names allowed for that client to run during login. So
| after the user logs in, the next thing would be to "SELECT ..." all
| the GUI modules from the database to be downloaded. The client won't
| know which ones are there, only the server.
|
| So the set of possible GUIs will be sent over to the client (preferable
| in a zipped form). They are unzipped in some temporary folder (here you
| can do some caching to only download if the set changed).
|
| Then it would be imporant for you to create some nameing rule or some
| interface so the Application.py will know what GUI modules look like.
| For example you could have the pattern [Capitalizedname]GUI.py be a
| GUI module. The Application.py will then inspect the folder, and create
| a button and label for each possible GUI module and present that to the
| user. When the user clicks on the Button, the Application.py window
| gets hidden and the [Modulename]GUI.py module is executed. When done,
| the Application.py will be shown again and the user can either continue
| with another module they are allowed to use or quit.
|
| How does that sound?

This is the sort of thing I am looking for, thanks - its a bit rough as it
depends on magic gui names, but hey - I think it would be easy to implement,
even not using magic names - the server could tell the core app the names of the
files explicitly during the logon type dialog... *grin* that will move the magic
out of the client and on to the server...
| Also, you mentioned that you won't have more than a couple of simple
| fill-in forms and with drop-down options and so on. HTML then might be
| the way to go. If that's all there will ever be just try to do HTML
| (see Cherrypy, Turbogears and others....). If your GUI will be more
| complicated in the future, just stick with what you know (Tkinter for
| example).
|
| Good luck,
| Nick Vatamaniuc
|

Thank you for the support - now I would like to call "Time out" - I have a lot
of reading to do...

- Hendrik

[H J van Rooyen]

"Dennis Lee Bieber" <wl*****@ix.net com.comwrote:
| On Thu, 3 Aug 2006 16:50:15 +0200, "H J van Rooyen"
| <ma**@microcorp .co.zadeclaimed the following in comp.lang.pytho n:
| Now part of the reason I would like to go the transaction type route instead
of
| the "per user" route is robustness and maintainability , and the ability it
| would give me to introduce new transaction types easily - as I see it if say
an
| invoice's GUI code is stable I would never have to touch it again even if I
| combine it with anything else, as it would have been designed from the start
to
| combine with others of it's own ilk, under a kind of communications
controller
| that is standard...
| >
| Uhm... You've just described the "raison d'etre" of Object Oriented
| design/programming.
Oh No! - curses! It has slipped in under the radar! - I have always thought that
the reason for going the OO route is to obfuscate the code so that mere mortals
could never understand it...
|
| Each of your "transactio ns" (tasks/jobs/functions) would be a single
| "class" (this does not mean they may not incorporate other classes as
| part of them). Ideally, each class should be independent of the others
| except for a defined API. The "invoice" would be one "class" (actually,
| you may have a class for "invoice data" -- this being the information
| passing between the client and the server; and a class for "invoice
| presentation" -- the GUI. Plugging in another task should only involve
| the main client (new menu entry) with maybe an "import new_task" so
| picking the menu entry instantiates new_task.
| --

Yes got it, thanks- Hendrik

[H J van Rooyen]

"Dennis Lee Bieber" <wl*****@ix.net com.comwrote:

| On Thu, 3 Aug 2006 16:50:15 +0200, "H J van Rooyen"
| <ma**@microcorp .co.zadeclaimed the following in comp.lang.pytho n:
|
| >
| This is broadly what I had in mind, yes - but sort of down to a transaction
| level - this user does invoicing, this one enters cheques, this one does
credit
| notes, and their supervisor can do all three, and in a different department
its
| different because the jobs are different, but the invoicing GUI module is
the
| same for wherever its used...
| >
| Confusing use of "transaction".. . Many are probably looking at
| "transactio n" in terms of DBMS -- eg: a sequence of SQL operations
| (inserts/updates/deletes) that ALL must be successful (and committed)
| or, if any operation fails, ALL operations will be undone (rolled back)
| as if none had been performed.
|
| What you describe is something I'd refer to as different operator
| "tasks".

Sorry - I have spent too long in an environment where a transaction is something
you built, shipped, and got an answer for...

|
| For this control, and presuming you do NOT want to use a web
| interface, I'd still build all capability (ie, all menu and form
| definitions) into the client that is installed on all user stations.
| THEN I'd use the application server (since I still feel you want to
| control access to the database to one central point, not have to create
| a user account in the DBMS for each employee using the clients) to
| handle a login from the client -- the login would be used to access a
| privilege table from the DBMS. This privilege table is essential a long
| record of booleans -- one for each menu/form/task in the client. The
| privilege record is sent to the client; the client uses the record to
| enable and disable the relevant menu/form/task so that the user can only
| activate the functions valid for them.
|
*grin* this is the easy - cop out route - there is of course nothing
intrinsically wrong with it - I was just trying to find out how to do the harder
bits in Python - namely to dynamically present different things...
| Otherwise you have a situation where, say a lowly data-entry clerk
| has been using the application, maybe makes a mistake that requires
| supervisor override, and the supervisor then has to wait for "their"
| modules to be downloaded and started to correct the mistake. (Note: one
| menu entry that should be on the "all-in-one" client is an ability to
| change login without having to shut down the client -- basically the
| client does a log-off of the application server, then a fresh log-in
| with new parameters and gets a new privilege record with which to
| reconfigure all the menu/form/task GUI).
|

This is a disadvantage that I had not thought about - I was not yet this far
down the track

| If you go the web application route, each "login" would use a cookie
| to control session, so the application server can determine what
| functions to present to the user. You might even be able to use
| something like Plone to build the application server; it already has
| capability to present different views based upon login.

Know squat about Plone - another thing to add to my list of reading *sigh*
|
| My thinking is simpler than this, because you are already thinking in "data
base
| speak" - now dont get me wrong - I realise that I will have to use a
database -
| but for me to start thinking in terms of views and stuff is kind of
premature
| when I dont even have a clear picture in my head of the kind of data the
said
| management system should keep, as I am not sure of what can, and cannot be
done.
| >
| We're talking computer software -- enough money and time and you
| could have the sun, moon, and stars (granted, a decade ago I /was/
| involved with swapping out one satellite ephemeris package for another,
| so it was literally sun, moon, and stars <G>)
|

sounds like it was fun...

| You need to determine
|
| 1) architecture partitioning (DBMS <-thick client; DBMS <->
| application server <-thick client, DBMS <-application/web server <->
| thin client) [Thick client: Python program that handles GUI and talks to
| application server or DBMS; Thin client: web browser forms with some
| Javascript/AJAX].

I am kind of egregious - despite all the sound advice I am getting - I am still
thinking (from the back end) of:

dbms <app server <thick client (python)

|
| 2) Database schema (done without care of the final DBMS); take into
| account things you may not think of as the main data of the application
| -- things like the above privilege control (in the case of the
| DBMS<->thick client, it means using the grant tables and managing client
| accounts per user, in the application server model you only have one
| DBMS user that is the application server, but still have client user
| accounts to manage as application data). If the DBMS has stored
| procedures, you may need some of those... Heck, in the DBMS<->thick
| client model, you could let any client access ANY form/operation/task...
| What you'd do is run stored procedures for any DBMS transaction -- the
| first thing that would happen is that the user privileges are checked
| and the stored procedure either proceeds or rejects with "no privilege
| for operation" (and this could be either programmed in as part of the
| stored procedure, or be part of the DBMS internal privilege control).
| --

Yes this is the tricky part - it is so easy to overlook something, or to use an
awkward definition that comes back later to byte you in the butt (yes I know
bite is not spellt like that)
And it is not easy to do all this so that you can maintain it - when you get
back to it six months (or years) later, you are no longer the same person... the
"Who wrote this crap? " syndrome - and the answer - "Oh it was I"

;-)

So one has to follow some kind of discipline that you feel comfortable with...

Thanks for the input - I want to call "time out" now so that I can go and do the
reading I have been pointed to...

[H J van Rooyen]

"Dennis Lee Bieber" <wl*****@ix.net com.comwrote:

8<------------------------------

| There may be something in-between. IFF this is to be used strictly
| on an internal LAN with uniform architecture (all Linux or all WinXP)
| for the client machines. You'd have to set up something so a reboot

no such luck - reality will probably be Linux for server, and a horrible mix of
windoze machines on the client side - from 95 through 98 and 2000 to XP... will
have to get SAMBA running at least - and it could be tricky with some of the
older hardware/software around - but that is another fight, that I would have to
solve anyway.

| remounts correctly but... (In WinXP terms) Create a read-only "share" on
| a file server. The file server will contain the Python modules that make
| up the client. The client start-up still uses a login to obtain a
| privilege map, which controls the menu/form access. However, rather than
| having to push the modules to each client, you delay the module import
| until the form it controls is invoked. The start-up module would have to

This is more the kind of thing I had in mind - but I was not thinking in terms
of having the redirecting done by the OS and network file sharing - stupid I
suppose...

| add the "share" to the pythonpath (hence you want a uniform system
| configuration so each machine mounts the share on the same name).
|

I will have to think of a solution to this *shudders* - config files, maybe...
| It's still an all-in-one client, but you don't have to install
| anything on the user machines (except the "share" and a shortcut to the
| start-up module).
|
| For Linux, this would be an NFS mount.

*nods* Thanks Dennis - I am busy drinking out of the Pyro fire hose at the
moment - and the stuff I have seen so far looks bloody awesome - you do some
little bit of magic setup - and hey - you have a name server that you can query
and then you can remotely execute a method on a remote object just as if its
here in your machine, and you get the returns just like you would if the object
were local to the machine your Python script is running in...

And they have made a sort of mirror of the name server thingy so that you can
fall back to a non broken one and resync when things go wrong go wrong go
wrong...

As a low level johnny this sort of functionality impresses the hell out of me -
when I think of the time I have spent struggling to get a few small tightly
coupled machines to work together in a primitive way - my mind boggles at this -
you could create awesome capability by having multiple copies of objects hanging
around in different machines - and simply keep track of how much they are loaded
by monitoring their input queues - and make more instances as you need them as
loading gets higher... The only bottleneck is the LAN's capacity to move the
data around - but in most WAN type applications, that is not where the
bottleneck is...
and its not inconceivable to add an additional "Python Execution Gigabit
Backbone" to a cluster of machines, dedicated to this remote calling task...

Its almost too good for my simple little local job, but I am still reading...

I have to remember to thank Bruno for the pointer...

- Hendrik

[Bruno Desthuilliers]

H J van Rooyen a écrit :

"Dennis Lee Bieber" <wl*****@ix.net com.comwrote:

(snip)

| If you go the web application route, each "login" would use a cookie
| to control session, so the application server can determine what
| functions to present to the user. You might even be able to use
| something like Plone to build the application server; it already has
| capability to present different views based upon login.

Know squat about Plone - another thing to add to my list of reading *sigh*

Well, actually, I would not bother too much reading about Plone here -
Plone is (fairly complex and somewhat slow) CMS built on top of Zope,
which is itself a web application server that's not easy to get started
with and is (IMHO) definitively much more suited to CMS than to
accounting apps.

[H J van Rooyen]

"Dennis Lee Bieber" <wl*****@ix.net com.comwrote:
| On Sat, 5 Aug 2006 11:20:59 +0200, "H J van Rooyen"
| <ma**@microcorp .co.zadeclaimed the following in comp.lang.pytho n:
|
| >
| no such luck - reality will probably be Linux for server, and a horrible mix
of
| windoze machines on the client side - from 95 through 98 and 2000 to XP...
will
| have to get SAMBA running at least - and it could be tricky with some of the
| older hardware/software around - but that is another fight, that I would
have to
| solve anyway.
| >
| Well, other than the security differences -- which may not apply
| when the clients are mounting a share, only when the define a shareable
| partition -- I think the Windows side may be similar all the way
| through.
|
| This is more the kind of thing I had in mind - but I was not thinking in
terms
| of having the redirecting done by the OS and network file sharing - stupid I
| suppose...
| >
|
| Not really -- if one first is thinking in terms of "internet", which
| means unsecured global operations, instead of an internal-only, behind
| firewall, system.

*grin* you are being too kind - I was at no stage thinking internet...

|
| | add the "share" to the pythonpath (hence you want a uniform system
| | configuration so each machine mounts the share on the same name).
| |
| >
| I will have to think of a solution to this *shudders* - config files,
maybe...
| >
| I think Windows can be set to "reconnect on start-up", but it may be
| better just to add a BAT file to the client machines' system start-up
| directory containing the command line that mounts such a share.
|
|

This is a good idea - thanks

- Hendrik

[H J van Rooyen]

"Bruno Desthuilliers" <bd************ *****@free.quel quepart.fr>
H J van Rooyen a écrit :

"Dennis Lee Bieber" <wl*****@ix.net com.comwrote:

(snip)

| If you go the web application route, each "login" would use a cookie
| to control session, so the application server can determine what
| functions to present to the user. You might even be able to use
| something like Plone to build the application server; it already has
| capability to present different views based upon login.

Know squat about Plone - another thing to add to my list of reading *sigh*

Well, actually, I would not bother too much reading about Plone here -
Plone is (fairly complex and somewhat slow) CMS built on top of Zope,
which is itself a web application server that's not easy to get started
with and is (IMHO) definitively much more suited to CMS than to
accounting apps.

*deletes the reference to Plone from the little text file*

- Thanks