473,287 Members | 1,570 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Creating a multi-tier client/server application

Hello everyone. I've searched through the archives here, and it seems
that questions similar to this one have come up in the past, but I was
hoping that I could pick your Pythonic brains a bit.

Here's a broad overview of what I need to do: cross-platform, client-
side GUI apps that interact with a server backed by a database. I'd
also like the possibility of having a web interface for small portions
of the app. It will be a fairly complex system for managing personnel
(and training, performance reviews, etc.) and payroll, and I will
probably have about a year or more to write it.

None of this is written yet--actually, it's not even to the point of
specs yet, I'm just trying to learn the best way to do this *before* I
get to the specs. I want to plan this application out as thoroughly
as possible before writing any code, and I want to do it The Right
Way(TM), hence I have (hopefully) convinced my manager (and clients)
that Python is the way to go. I have a couple years of experience in
Python, but it's mostly been web apps (which I am thoroughly sick of
making, thanks to PHP) and simple CLI programs. Not sure if any of
that information helps, but I'm tired, so please bear with me.

The application definitely needs at least three tiers: client, server,
and DB. I want as little logic in the client as possible (if not
none), as there will be sensitive data in the DB. I also can't have
the clients connect to the DB directly, as it will need to only accept
connections on the local host. The client will probably be installed
on 50-100 machines, and will be used by several hundred people (yes,
they're sharing machines--the users will be computer lab consultants/
server operators/helpdesk employees, etc. and their supervisors), so
authentication (using LDAP) and authorization (using groups I will
define) will be very important. Also important will be automatic
updates of the client, which I figure I'll do by comparing version
numbers, then downloading an archive (egg, maybe?) of the latest
(compiled? byte-?) code.

I will probably make the GUI in wxPython, as that seems to be most
flexible and nicely cross-platform choice--but I am open to
suggestions. Not too worried about that one, though.

For the database, I'll use Postgres, because I love Postgres.
Probably not going to budge on that one unless anyone has any good
reasons to. I'd also like to use SQLAlchemy (because it is so nice),
but this is not a requirement (I can write SQL, but... SQLAlchemy is
so nice.) I'd like to use an ORM mainly because it would facilitate
code reuse for the (very simple) web interface, and other offshoot
tools down the line.

So, my real question (sorry for the all the exposition) is this: how
to do the networking? I'd really like to have the client send a
simple request to the server along the lines of "fetch this (or these)
object(s)", or "update this value in this object", then let the server
decide if the user actually is allowed to do so, and then make the
change in the database.

My first thought was XML-RPC, because I've used it before, and it is
so nice and clean--but then I can't send objects, which is really
rather crucial.

Then I looked at Pyro, which seems awesome, but I'm confused as to how
(if possible) I would use it with a DB. Would I have to figure out on
the client what SQL query to send to get the right objects? That would
kind of kill the separation of logic. And would it work at all with
SQLAlchemy (or any ORM for that matter)?

I looked at SOAP. It made me feel unclean. I looked at some CORBA,
at which point my eyes glazed over (possibly not CORBA's fault, I will
look at it again later).

I checked out Twisted, but there are so vastly many parts to it, that
I'm not sure where to even start.

I'm not ruling out writing my own simple protocol, but it seems like
I'd be reinventing a wheel that's already been reinvented countless
times.

So, all that said--any advice on how to do the networking? And on the
auto-update? For that matter, any advice on anything I've mentioned
(or haven't?) Please feel free to ask me to clarify anything, or tell
me if I'm being an idiot about anything.

Thanks,
Jeff

Aug 29 '07 #1
11 4841
Jeff <je*****@gmail.comwrites:
Here's a broad overview of what I need to do: cross-platform, client-
side GUI apps that interact with a server backed by a database. I'd
also like the possibility of having a web interface for small portions
of the app. ... any advice on anything I've mentioned
(or haven't?)
Why not do the entire app as a web app, completely bagging the client
side and just using a browser? It will help your deployment problems
a lot, and using https will help with security since it looks like the
app will be transmitting sensitive data. It also will simplify your
networking questions a lot since you're just writing a server.

See the big debate between Chris Mellon and me a few weeks ago about
web apps vs client gui apps. Yes you can get a level of
responsiveness from a client app that can't easily be done with a web
app. However, relatively few client gui business apps that I've
personally used really seem better than the equivalent things done as
web apps in the obvious way.

I would certainly prototype the app as a web app, and then think about
writing a client gui only if it was clear that usability would really
benefit from it. Even still, the server would still look like a web
server, translating xml requests into database actions and responding
with xml.
I want to plan this application out as thoroughly
as possible before writing any code,
There is a school of thought (sometimes associated with "extreme
programming") that this is more planning than you should really do.
Think of a cross-country automobile trip. Figure out the general
route you want to take, then get in the car and go, making low-level
decisions as you get to them, rather than trying to plan every gas and
rest and restaurant stop before you leave.
Aug 29 '07 #2
Paul Rubin a écrit :
Jeff <je*****@gmail.comwrites:
>Here's a broad overview of what I need to do: cross-platform, client-
side GUI apps that interact with a server backed by a database. I'd
also like the possibility of having a web interface for small portions
of the app. ... any advice on anything I've mentioned
(or haven't?)

Why not do the entire app as a web app, completely bagging the client
side and just using a browser? It will help your deployment problems
a lot, and using https will help with security since it looks like the
app will be transmitting sensitive data. It also will simplify your
networking questions a lot since you're just writing a server.
(snip)
>
I would certainly prototype the app as a web app, and then think about
writing a client gui only if it was clear that usability would really
benefit from it. Even still, the server would still look like a web
server, translating xml requests into database actions and responding
with xml.
I'd personnaly favor json, which is much more lightweight, perfect for
structured data, and widely supported.

Else, and however the client is implemented (rich client or web -
another possible solution being something based on the Mozilla platform,
but I have no working experience with it so I can't tell if it's a good
idea...), I'd second the suggestion to have a closer look at the http
protocol. There's no shortage of support for writing http-based server
applications in Python (given the context, I'd strongly suggest Pylons).
>I want to plan this application out as thoroughly
as possible before writing any code,

There is a school of thought (sometimes associated with "extreme
programming") that this is more planning than you should really do.
Think of a cross-country automobile trip. Figure out the general
route you want to take, then get in the car and go, making low-level
decisions as you get to them, rather than trying to plan every gas and
rest and restaurant stop before you leave.
Anyway, unless you're a genius with decades of working experience, you
can bet you'll make lots of mistakes in your "planning", mistakes that
you'll discover when implementing a probably over-engineered design. I'm
of course *not* advertising a cowboy-coding approach here, and
preliminary work is certainly mandatory for anything non-trivial, but
take care about what you really have to worry about here. AFAICT, you
seem to be on the right track so far (thinking about the possible
architectures given some already known requirements), but AMHE,
BigDesignUpFront just doesn't work, so don't overplan.

My 2 cents...
Aug 29 '07 #3
Perhaps Spring Python can help you out (http://springpython.python-
hosting.com). It reuses technologies like Pyro for remoting, offers
database templates, has a plugable security component, an AOP solution
should the need arise, an IoC container, and has a couple of web-app
demos using CherryPy. Of course, you can use whatever web stack you
want. I just picked CherryPy to demo things.

Each component of Spring Python is optional, and you can use it
through the IoC container, or programatically. This might help you
leverage development of an enterprise app.

BTW, I have plans to implement a wxPython front end sample app. I just
haven't done it yet.

Good luck!

Aug 29 '07 #4
Jeff a écrit :
>You could explore something like a custom-made GUI client app
communicating thru the http protocol with a web-server app. http is just
a protocol, and it doesn't necessarily imply using html and a browser...
IIRC, some GUI toolkits uses XML description files for the UI.

That, or something similar, may be what I do. It would mean, however,
developing my own method for transferring objects across the network,
Why transfering "objects" ? You only need to transfer data.

(snip)
Aug 29 '07 #5
Jeff <je*****@gmail.comwrites:
reasons, not the least of which is that I've been working almost
entirely on web apps for the past few years, and I am getting mighty
sick of it. A lot of that is due to the language (PHP, which I have
since grown to hate) I had to use. I've worked on a site for my self
in Python (using Pylons, actually--which is excellent) which was
vastly easier and more fun. But I'd really like to try something
different.
To contribute a data point against your original question - I've a similar
(structurally, not functionality) system I completed recently.

Without trying to get too mired in the thick client v. web application
debate, there were a handful of points that decided me in favor of the
thick client:

* Needed to automate QuickTime viewer for video previews and extraction
of selected frames to serve as thumbnails on web approval page.
* Needed to control transfers to server of multiple very large files
(hundreds of MBs to GBs at a shot)

But assuming a thick client, in terms of your original question of
components to use, here's what I've got. My primary networking component
is Twisted.

The pieces are:

Client (OS X Cocoa application):

* PyObjC based. Twisted for networking, Twisted's PB for the primary
management channel, with an independent direct network connections for
bulk file transfers. (I needed to go Mac native for clean integration of
QuickTime UI elements including frame extraction to thumbnails)

Server:

* Twisted for networking. PB and raw connections for clients, web server
through twisted.web. Genshi for web templating, with Mochikit (might
move to JQuery) for client-side JS/AJAX. Twisted for email transmission
(email construction using normal Python email package). Small UI
front-end module (Cocoa/PyObjC).

The client accesses server-based objects through Twisted PB, which for some
of the server objects also control session change lifetime (transactions).
So at least in my case, having a stateful connection from the client worked
out well, particularly since I needed to coordinate both database changes
as well as filesystem changes through independent file uploads, each of
which can fail independently.

Right now a single server application contains all support for client
connections as well as the web application, but I could fracture that (so
the web server was independent for example) if needed.

For the client, I package it using py2app, and put into an normal Mac
installer, and distribute as a dmg. If it were a Windows client, I'd
probably wrap with py2exe, then Inno Setup. The server's web server has a
restricted URL that provides access to the DMG. The client has a Help menu
item taking users to that URL. Clients are versioned and accepted/rejected
by the server during initial connection - from the server side I can
"retire" old client versions, at which point users get a message at signon
with a button to take them to the download page for the latest DMG.

So right now upgrades are executed manually by the user, and I can support
older clients during any transition period. I may provide built-in support
for automatically pulling down the new image and executing its installer,
but haven't found it a hardship yet. I probably won't bother trying to
automate smaller levels of updates.

-- David
Aug 29 '07 #6
We totally agree with your software engineering goals. Relying on
wxPython and minimizing web reliance brings sanity to the enterprise.
We too love PostgreSQL and avoid XML whether cleaned by SOAP at all
costs. We have found the object-relationship managers to be bloated
and unruly. What are you building and how many hours per month are
you willing to spend supporting it?

Aug 30 '07 #7
We totally agree with your software engineering goals. Relying on
wxPython and minimizing web reliance brings sanity to the enterprise.
We too love PostgreSQL and avoid XML whether cleaned by SOAP at all
costs. We have found the object-relationship managers to be bloated
and unruly. What are you building and how many hours per month are
you willing to spend supporting it?

Aug 30 '07 #8
Jeff <je*****@gmail.comwrites:
David: Sounds like a pretty interesting app. Thanks for the in-depth
description. I went and checked out Twisted PB, and it seems
awesome. I may very well go with that. How was writing code with
it? I may also end up using py2app, but I'm also going to have to
support Windows, (p2exe, then), and possibly Linux. Well, maybe not
Linux, but I'll probably be doing most of the development in Linux, so
I guess that counts.
I find PB very easy, but it's important to first become familiar with
Twisted (in particular Deferred's), which can have a steep, but worth
it IMO, learning curve. PB is a thin, transparent system, so it
doesn't try to hide the fact that you are working remotely. Being
thin, there also isn't very much to have to learn.

For packaging, you don't have to use a single system if you are
multi-platform. Your codebase can be common, and just have separate
setup files using py2app on OS X and py2exe on Windows. A makefile or
equivalent can handle final distribution packaging (e.g,. hdiutil for
dmg on OS X, Inno Setup, NSIS, etc... on Windows). You'll spend some
platform-specific time getting the initial stuff setup, but then new
builds should be easy.

For Linux, depending on the level of your users you can either just
directly ship something like eggs (generated through a setup) or look
into pyInstaller, which was the old Installer package that also
supports single-exe generation for Linux. pyInstaller also does
Windows, so if you have to support them both you could try using
pyInstaller rather than both it and py2exe.

But if you're just developing in Linux, final packaging probably isn't
very important.

-- David
Aug 30 '07 #9
va**********@gmail.com a écrit :
(snip)
We have found the object-relationship managers
Actually, ORM stands for "object-relational mapper"
to be bloated
and unruly.
Which ones ?
Aug 30 '07 #10
Jeff <je*****@gmail.comwrites:
Granted. But what I will be writing really will take a lot of extra
work to get even close to the level of usability needed on the web vs.
a desktop app. And I'll try not to write a crappy GUI ;-)
OK. In the discussion with Chris, one factor that came up is how much
time users will spend in front of the app and the amount of data
they'll physically enter. If it's a lot, that weighs in favor of a
desktop app. If it's not much, then maybe some gain in UI
responsiveness isn't worth the downside of a client installation.
Sorting certainly doesn't have to be done on the server side--in fact,
in most cases I can think of where it would be useful for this app, it
wouldn't have to be--in which case it's more responsive. Certainly
exporting, importing and printing can all be done through the web--
I've done this plenty of times. But there is a huge amount of
flexibility (and, in the case of printing, guaranteed style/quality/
layout) to be gained on the desktop.
This I don't understand at all. How is the least bit of flexibility
or style or quality guarantees gained by a desktop app? If you want
fancy formatting, just use Reportlab to generate a pdf on the server
and send it to the browser.
All that said, I am most likely going to go with a desktop
application. In reality, I will have to do whatever my client wants.
Thank you *very* much for all of your input--one of the first things I
have to do is make them a list of the pros and cons of web vs. desktop
apps--and this will help a lot.
Yes, that sounds like a good idea.
vanrpeter-whatever: Thanks for the support :-) What did you use for
networking? What ORMs did you try (or did you skip them altogether?)
Have you ever written a serious GUI app before? I've never done any
really fancy ones, but even the relatively simple ones I've done have
turned out to be far more time consuming than I expected. Have you
done much concurrent programming, either with threads or something
like twisted? You're going to have to do that to keep your gui
responsive, so you might put that on your list of things to study (in
addition to networking and databases).
Aug 31 '07 #11
Greetings,

I somehow missed some of this thread, but I believe you left a note
saying that you were not able to do Extreme Programming.

However, based on the description of the size of the project and the
size of the development team (is there any more than you?) I would
recommend you consider the following agile techniques:

1. Incremental development
2. Automated testing
3. Continuous integration

I think 1 and 3 are important in giving you a continuously working
system, along with a working system you can use for feedback during
the development process. Automated testing would support 1 and 3 and
keep you from having to do a lot of manual testing. Wikipedia
describes most of those terms.

Also think about having acceptance tests that someone besides you can
verify match the specifications.

Lastly do use the fact that you are (presumably) close to some of
your customers to use face time for additional communication usually
needed beyond specs.

I hope this helps. It sounds like a useful project.

Kathy Van Stone
kv*@acm.org
Sep 4 '07 #12

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Keiron Waites | last post by:
<script language="JavaScript" type="text/javascript"> <!-- var array1 = new Array(); var array1i = new Array(); array1 = array1i; alert(array1); // --> </script>
10
by: *ProteanThread* | last post by:
since a majority of people are still on dial up or at most DSL (though there *IS* broadband) does it help or hurt one's webpage adding multi media ? does it necessarily distract from the theme of...
2
by: Dino | last post by:
I need to create a public multi-dim array that can hold a mixture of string and int values. I will need to search this array for a value and return a value from a subscript. This array will...
1
by: code | last post by:
Hi Grp http://www.books-download.com/?Book=1493-PHP+Hacks+%3a+Tips+%26+Tools+For+Creating+Dynamic+Websites+(Hacks) Description Programmers love its flexibility and speed; designers love its...
5
by: John | last post by:
I have 2 tables, one with dates and information about those dates, and one with people information. I want to create a report listing each date and the people who attended on that date (who have...
4
by: gdarian216 | last post by:
I am doing a multi file program and I got it to work correctly in a single file but when I split it up it isn't working properly. I keep getting errors when i compile. Can anyone help me figure out...
4
by: crescent_au | last post by:
Hi all, I'm doing some research online on creating php-based multi-level marketing (MLM) system. It seems like a complicated system to build as I need to create one from scratch. I'd like to...
5
by: Nayan | last post by:
Hi, If I make a call to function which is in external library, and it goes into wait sate.. disabling my app to proceed further, how can I break this state elegantly? So far, I had to kill my...
1
by: Nathan Sokalski | last post by:
I have a Button control whose text property is "Move To Archives". Because this makes a relatively wide Button when the text is all one line, I want to put a line break between the words "Move To"...
3
by: Nathan Sokalski | last post by:
I am looking to create a multi-column list that is scrollable, kind of like a ListBox control, only I need a way to align the columns of text that I will have (I would use spaces, but because the...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.