473,713 Members | 2,511 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

code packaging

I've been through this kind of thing a few times in the past and
received excellent advice here on clpy about how to deal with specific
technical aspects (e.g. how to use setuptools, InnoSetup, etc).

I'm now wondering where this type of thing is best addressed in more
general terms. What I usually see happening, in projects I've worked
on and in others, is that developers get the code working on their own
computers, using source control tools and (if we're lucky) tests
developed in parallel with the code, but without much attention to
final packaging until the very end when the code is about to be
shipped.

Is this appropriate? Inappropriate? Do your projects start using
serious packaging and distribution tools very early in development,
before the code is anywhere near usable? Should they?

I'm involved in something that I wish was that way, mainly because I
really don't have any way to put a complete installation on my own
computers; I have to develop my subsystem more or less independently
of the overall app, then toss stuff over the wall to the more central
application guys for integration, which gets slow sometimes because
we're distributed all over. This is a fairly complex server app that
depends on a lot of external packages and programs running across
multiple machines, so I could see the packaging problem being messy.
It would cause a significant interruption in development to try to
package things so that the non-central developers could install and
hack it without a lot of assistance. But I think it would be
worthwhile given how much easier it would become to test changes. We
do plan to bundle up and release the code sometime (maybe as a Debian
package) but for now that's deferred until the code is more complete
and stable.

I'm wondering how other projects go about this.
Jul 20 '07 #1
5 1658
Hello Paul,
I'm now wondering where this type of thing is best addressed in more
general terms. What I usually see happening, in projects I've worked
on and in others, is that developers get the code working on their own
computers, using source control tools and (if we're lucky) tests
developed in parallel with the code, but without much attention to
final packaging until the very end when the code is about to be
shipped.
[snipped]

IMO you should invest the time a build automated packaging and some
smoke tests for the product.
After this is done, start using some "continuous automation" tools
(like http://cruisecontrol.sourceforge.net/, http://buildbot.net/trac
and others).

HTH,
--
Miki <mi*********@gm ail.com>
http://pythonwise.blogspot.com

Jul 20 '07 #2
Paul Rubin <http://ph****@NOSPAM.i nvalidwrites:
Is this appropriate? Inappropriate? Do your projects start using
serious packaging and distribution tools very early in development,
before the code is anywhere near usable? Should they?
Yes. Test-driven development has taught me that putting off important
parts of the development process until the end is rarely a good
idea. So, I try to make a point of writing the first features inside a
complete unit test environment, and inside a complete
distribution/build environment. The tests are run continuously while
developing the code, and the distribution/build process is tested
manually.

I'm looking into the "build-bot" approach used to ensure that the
build process is also run automatically and continually during
development, just like the unit tests <URL:http://buildbot.net/>; but
so far I'm running the build manually.

--
\ "Read not to contradict and confute, nor to believe and take |
`\ for granted ... but to weigh and consider." -- Francis Bacon |
_o__) |
Ben Finney
Jul 20 '07 #3
Paul Rubin <http://ph****@NOSPAM.i nvalidwrites:
[...]
I'm now wondering where this type of thing is best addressed in more
general terms. What I usually see happening, in projects I've worked
on and in others, is that developers get the code working on their own
computers, using source control tools and (if we're lucky) tests
developed in parallel with the code, but without much attention to
final packaging until the very end when the code is about to be
shipped.

Is this appropriate? Inappropriate? Do your projects start using
serious packaging and distribution tools very early in development,
before the code is anywhere near usable? Should they?

I'm involved in something that I wish was that way, mainly because I
really don't have any way to put a complete installation on my own
computers; I have to develop my subsystem more or less independently
of the overall app, then toss stuff over the wall to the more central
application guys for integration, which gets slow sometimes because
we're distributed all over. This is a fairly complex server app that
depends on a lot of external packages and programs running across
multiple machines, so I could see the packaging problem being messy.
It would cause a significant interruption in development to try to
package things so that the non-central developers could install and
hack it without a lot of assistance. But I think it would be
worthwhile given how much easier it would become to test changes. We
do plan to bundle up and release the code sometime (maybe as a Debian
package) but for now that's deferred until the code is more complete
and stable.

I'm wondering how other projects go about this.
You can go further: in addition to developing a Debian package (or a
few of them) for your own project, build your OS image using
debootstrap (you'll probably want your own local repository(ies) , for
reproducibility ). Then you do your development in a virtual machine
or chroot environment. That way you develop in the same (or very
similar) environment to the one in which your code will eventually be
deployed (one difference being the use of an version control checkout
rather a .deb install for the code you're actually working on, unless
troubleshooting problems that are somehow tied up with that particular
aspect of deployment). I've seen this working in a real-world
project.

Of course, you can set up a server that does a complete build-and-test
every night, and a separate continuous integration server that just
does an "svn up" (rather than a full rebuild) every time somebody
checks in, so that test failures can be detected quickly and the
guilty party suitably humiliated :-)
John
Jul 21 '07 #4
On 7/22/07, Ryan Ginstrom <wrote:
Hi Alex:

Do you develop for Windows? Are you looking to automate a build
process?

The standard library's build module is distutils:
http://docs.python.org/lib/module-distutils.html

As I mentioned in my post, I use a variety of third-party modules
(epydoc, py2exe, Inno Setup, AutoIt), which I glue together with my
own brew of python scripts. Since I develop mostly for Windows (except
for Web stuff, which I use Linux for), my build process is tailored to
that platform.

Regards,
Ryan Ginstrom
Thanks for following up on this. I am mostly used with a world where
platform dependent builds are not very used (Java).

My current attempt to learn Python is by creating a Python equivalent of
the advanced Java testing framework TestNG (http://testng.org) -- but I
will give more details about this when I'll be starting to have
something more solid :-).

However, due to my background where builds are required and there are
some de facto build standards (Ant, Maven, etc.) I am starting to think
I will be needing something similar while working on the tool I've
mentioned. At this point I think that what I am interested in are how
distros are build in Python world (no platform specific distros, but
rather generic ones) and ways to automate the process of creating the
distros (and probably running the framework internal tests etc.)

My research lead me to distutils and/or setuptools, respectively SCon
and/or buildutils. So I am wondering if I am looking in the right
direction or do I need to do some more research :-).

tia,
../alex
--
..w( the_mindstorm )p.

Jul 22 '07 #5
jj*@pobox.com (John J. Lee) writes:
You can go further: in addition to developing a Debian package (or a
few of them) for your own project, build your OS image using
debootstrap (you'll probably want your own local repository(ies) , for
reproducibility ). Then you do your development in a virtual machine
or chroot environment. That way you develop in the same (or very
similar) environment to the one in which your code will eventually be
deployed (one difference being the use of an version control checkout
rather a .deb install for the code you're actually working on, unless
troubleshooting problems that are somehow tied up with that particular
aspect of deployment). I've seen this working in a real-world
project.
Thanks for these suggestions. Buildbot (per Ben Finney) looks
interesting as does the above, and Ryan's notes about experience with
Windows installers is also appreciated. Whether lot of build
automation would have been worth it at the beginning of the thing I'm
working on or not, we probably need to move towards it now that more
of the system is running.
Jul 24 '07 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
3131
by: Barry Edmund Wright | last post by:
Problem when using Xp Packaging Wizard and the database being able to find the Application Icon after database is deployed. Under Menu/Tools/Startup form I have: Application Icon: c:\vms\handshak.ico Unfortunately, when I use the Xp Packaging Wizard the default folder it uses to install the database vms.mdb is c:\program files\vms. Therefore when the database is started it can't find the handshak.ico in the directory c:\vms because it...
1
1887
by: Coy Howe | last post by:
This one seems bizarre! We have a database consisting of a main table and 12 - 15 "sub" tables, which are connected via cascading relationships. The database performs many complex calculations and displays the results in both forms and reports. In the process of developing the database we "entered" 12 sample clients. Before packaging the database we remove the sample clients by deleting them from the main table, which cascades the...
3
2534
by: Paul Aspinall | last post by:
Hi I want to package my C# winforms app, to be deployed with MSDE, as easily as possible for the end user. I want to create an MSI or Installshield (prefer MSI), to setup my C# app, together with MSDE (if not already installed), and the MSDE application DB. How can I detect if MSDE is already installed?? Has anyone does this?? How??
3
1664
by: VJ | last post by:
I have a MSI that installs some Add-ins for office applications. This package was generated by the Add-in project Wizard. How do I include this project as part of the package I build for my application. I am using VS.NET packaging Thanks VJ
2
1707
by: Jozef | last post by:
It's possible I might have found the problem of my previous post, but I'm really not sure. My XP Packaging wizard is adding COMCAT.DLL version 4.71. My development machine uses version 5.0 and the target machine uses version 5.0 as well. I'm not sure where the Packaging Wizard is getting this DLL, but does anyone know how I specify which version to use? Thanks!
0
1151
by: Demetri | last post by:
I have a performance question with regards to packaging assemblies and referencing them. First lets say you have created the following assemblies: Company.Framework.dll Company.Framework.IO.dll Company.Framework.Xml.dll Company.Framework.Data.dll Company.Framework.Web.dll
0
5201
by: blkwebman | last post by:
I'm trying to create a "standalone" install package (without any of the dialog boxes that a standard setup package would have). I think I understand how to do it in VB6 (using PDCmdLn.exe); I have already researched this subject in the MSDN for VB6. Here is the problem: I'm trying to execute a batch file containing this command and I'm running into a couple of errors. Here is the contents of the batch file: ...
0
1265
by: Janelle.Dunlap | last post by:
I have created a database that I need to deploy to users without Access installed on their computers. I bought Microsoft Office Developer's Edition so that I could package the database and use Access Run-Time to take care of this problem. When I use the Packaging Wizard to create a setup package, this packaged database doesn't work right. The database itself is linked to an excel spreadsheet which I have listed as a Dependency file. ...
15
2956
by: John Nagle | last post by:
I've been installing Python and its supporting packages on a dedicated server with Fedora Core 6 for about a day now. This is a standard dedicated rackmount server in a colocation facility, controlled via Plesk control panel, and turned over to me with Fedora Core 6 in an empty state. This is the standard way you get a server in a colo today. Bringing Python up in this completely clean environment is a huge hassle, and it doesn't...
0
8794
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9166
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9066
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
7939
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6620
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5942
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4459
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
3154
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2101
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.