Using python for a CAD program

[63q2o4i02]

Hi, I'm interested in using python to start writing a CAD program for
electrical design. I just got done reading Steven Rubin's book, I've
used "real" EDA tools, and I have an MSEE, so I know what I *want* at
the end of this; I just have never taken on a programming task of this
magnitude. I've seen that some are using python as a utility language
for existing CAD environments, and I've also found some guy who's
writing a 2d drafting tool with python, which is cool. I've been
playing with python recently and so far have a SPICE parser
half-written (it'll do production rules, but no syntax trees yet...)
I'm wondering if the python experts can provide some opinion on
python's efficiency/utility in the following areas, and perhaps where
some things will need to be in C++ and/or pyrex:

1. Databases. I don't mean sql type database, but design databases,
which contain all the components for any given design, including
schematics, layout, simulation parameters, etc. I'm not concerned
about python's ability to do fancy data structures, but I'm wondering
how it'll go in terms of efficiency when things get really big. If the
alternative is home-brewed C++ linked-lists, attributes, arrays, binary
trees, memory management, etc., it looks like python's ability to do
all this neatly without all the bugs waiting to happen if I did this
myself are well worth the efficiency loss that may result from its
uber-generality.

2. GUI. Yes, I know you can do guis with qt, gtk, tkinter, etc. I'm
talking of fancy guis that do alpha blending, animations, nice
shading/gradients, etc. in a quick, smooth, and slick way, such that
moving a scroll bar or jiggling the mouse yields fast game-like
response time, and which gives this program the feeling that you're
actually in the 21st century... ie this is an *interactive*
environment, and I f***king hate crass-hatching, so real colors is a
must. Can this be portable between linux and windows? Is it possible
to "do the whole thing" in opengl, even the 2d stuff? I guess I dont
know enough about guis here. My fear is that my app will turn into
x-hell if I try to do more than 8-bit colors in linux, with flashing
background windows and complaints from the server about bit planes.
Maybe I need to play with linux a bit more from the development side
before commenting on this, since I get the feeling I'm working on old
information here...

3. Computational stuff. I imagine the really heavy computing needs to
be done in c++, especially if I'm stupid enough to write my own
simulator, which my brain seems to want to do (I'm imagining a small
homer simpson brain with a mind of its own telling me what to do...).
But what about other things like rules and constraints about layout,
routing, compaction, undo/redo, etc? These aren't really
computationally expensive, I don't think, but do munge about quite a
bit with the database. Is it likely I'll write something faster in
C++? (answer: no).

4. Programmability. I imagine this is a slam-dunk, relatively
speaking, to have a python interpreter as part-and-parcel of the design
system. But how do I customize the command window to do custom things
like adding special shortcuts (eg '?' brings up help, or help on
commands a la Matlab), or making an API into the CAD program? I'm not
sure conceptually how to merge an embedded python interpreter into the
rest of the CAD program in an intelligent way, so some stuff is
exposed, but not all, and which interacts dynamically with the
graphical portion so that, for instance, context-sensitive help is
available in the console window, or so you can access information about
the currently selected object, or so you can do stuff from the command
line that is based on your current graphical context.

5. Threads and parallelism. One of the things about this is that I'd
like there to be some real-time simulator running so when you change
the values of parts, the output changes, and I'd like to be able to
take advantage of mulitple processors, or dual-core, or whatever. I've
seen fake multi-threading in python (read about it, but I haven't done
it), but that doesn't really use any extra cycles from a separate
processor.

So why am I thinking about this? I've used a few "real" CAD systems,
and from using them, it's obvious the modus is to kludge shit together
over 20 years and screw the user experience. So basically I'd like to
take over the world with my nifty new python-based cad system because
it's so beautiful and fun to use, and so flexible that my time to
market for any given new idea is 5x shorter than the existing tools.

Any comments on the above from people who've actually written stuff
would be greatly appreciated! :)

thanks
ms

[BartlebyScrivener]

Art Haas posts from time to time regarding a program called PythonCad
that he maintains:

http://tinyurl.com/o36t8

Also, here is a search of this forum on "Cad":

http://tinyurl.com/nuobe

[Harry George]

63*******@sneakemail.com writes:

Hi, I'm interested in using python to start writing a CAD program for
electrical design. I just got done reading Steven Rubin's book, I've
used "real" EDA tools, and I have an MSEE, so I know what I *want* at
the end of this; I just have never taken on a programming task of this
magnitude. I've seen that some are using python as a utility language
for existing CAD environments, and I've also found some guy who's
writing a 2d drafting tool with python, which is cool. I've been
playing with python recently and so far have a SPICE parser
half-written (it'll do production rules, but no syntax trees yet...)
I'm wondering if the python experts can provide some opinion on
python's efficiency/utility in the following areas, and perhaps where
some things will need to be in C++ and/or pyrex:

1. Databases. I don't mean sql type database, but design databases,
which contain all the components for any given design, including
schematics, layout, simulation parameters, etc. I'm not concerned
about python's ability to do fancy data structures, but I'm wondering
how it'll go in terms of efficiency when things get really big. If the
alternative is home-brewed C++ linked-lists, attributes, arrays, binary
trees, memory management, etc., it looks like python's ability to do
all this neatly without all the bugs waiting to happen if I did this
myself are well worth the efficiency loss that may result from its
uber-generality.

2. GUI. Yes, I know you can do guis with qt, gtk, tkinter, etc. I'm
talking of fancy guis that do alpha blending, animations, nice
shading/gradients, etc. in a quick, smooth, and slick way, such that
moving a scroll bar or jiggling the mouse yields fast game-like
response time, and which gives this program the feeling that you're
actually in the 21st century... ie this is an *interactive*
environment, and I f***king hate crass-hatching, so real colors is a
must. Can this be portable between linux and windows? Is it possible
to "do the whole thing" in opengl, even the 2d stuff? I guess I dont
know enough about guis here. My fear is that my app will turn into
x-hell if I try to do more than 8-bit colors in linux, with flashing
background windows and complaints from the server about bit planes.
Maybe I need to play with linux a bit more from the development side
before commenting on this, since I get the feeling I'm working on old
information here...

3. Computational stuff. I imagine the really heavy computing needs to
be done in c++, especially if I'm stupid enough to write my own
simulator, which my brain seems to want to do (I'm imagining a small
homer simpson brain with a mind of its own telling me what to do...).
But what about other things like rules and constraints about layout,
routing, compaction, undo/redo, etc? These aren't really
computationally expensive, I don't think, but do munge about quite a
bit with the database. Is it likely I'll write something faster in
C++? (answer: no).

4. Programmability. I imagine this is a slam-dunk, relatively
speaking, to have a python interpreter as part-and-parcel of the design
system. But how do I customize the command window to do custom things
like adding special shortcuts (eg '?' brings up help, or help on
commands a la Matlab), or making an API into the CAD program? I'm not
sure conceptually how to merge an embedded python interpreter into the
rest of the CAD program in an intelligent way, so some stuff is
exposed, but not all, and which interacts dynamically with the
graphical portion so that, for instance, context-sensitive help is
available in the console window, or so you can access information about
the currently selected object, or so you can do stuff from the command
line that is based on your current graphical context.

5. Threads and parallelism. One of the things about this is that I'd
like there to be some real-time simulator running so when you change
the values of parts, the output changes, and I'd like to be able to
take advantage of mulitple processors, or dual-core, or whatever. I've
seen fake multi-threading in python (read about it, but I haven't done
it), but that doesn't really use any extra cycles from a separate
processor.

So why am I thinking about this? I've used a few "real" CAD systems,
and from using them, it's obvious the modus is to kludge shit together
over 20 years and screw the user experience. So basically I'd like to
take over the world with my nifty new python-based cad system because
it's so beautiful and fun to use, and so flexible that my time to
market for any given new idea is 5x shorter than the existing tools.

Any comments on the above from people who've actually written stuff
would be greatly appreciated! :)

thanks
ms

As a BSEE, MSCS, and developer using Lisp and Prolog for
Knowledge-Based Engineering (KBE) of structures and systems, I'm
interested in where you are going. Nothing wrong with reinventing a
few wheels, but I always like to start with a literature search. It
wasn't clear if you had done so. At a minimum, take a look at:

1. OSS EE suites:
http://www.opencollector.org/
http://www.geda.seul.org/

I don't see rewriting ngspice or Icarus Verilog -- people of people
doing that. Putting a good DBMS behind gscheme on the other hand
makes sense. Also, I write Python scripts around geda and there has
been discussion of embedding Python.
2. Databases:

Locally, we have struggled with various DBMSs. It seems OO is
mandatory, so I look to Postgresql with inheritance, or (smaller
scale) ZODB. I really wouldn't want to grow one from B+ trees.

http://www.postgresql.org/
http://www.zope.org/Products/StandaloneZODB

3. 2D CAD and game engines:
http://directory.fsf.org/PythonCAD.html
http://pygame.seul.org/news.html

4. 3D CAD

Locally we write Python code against several commercial CAD packages
(I agree with your assessment). For an OSS effort, the only game in
town seems to be:
http://www.opencascade.org/
http://free-cad.sourceforge.net/

5. Knowledge Based Engineering (KBE) inference engines:

Python already gives you lazy evaluation and memoizing, but a
prolog-based backward chaining engine helps too. We wrote
CAD-engine-calls-python-and-prolog and
python-and-prolog-calls-CAD-engine scripts.

http://sourceforge.net/projects/pyprolog/
http://christophe.delord.free.fr/en/pylog/
http://arts.anu.edu.au/linguistics/P...pyfrompro.html
http://aspn.activestate.com/ASPN/Coo.../Recipe/303057

6. Simulation

http://simpy.sourceforge.net/

Also need FEM for electromagnetic analysis, and CFD for heat
dissapation.
--
Harry George
PLM Engineering Architecture

[skip]

Someone in hiding wrote:

Hi, I'm interested in using python to start writing a CAD program for
electrical design.

Google for PythonCAD.

Skip

[63q2o4i02]

Cool. thanks for the links. I've already looked around quite a bit,
and am very hesitant to just write more shit on top of other shit. The
idea behind this is it's completely mine. So yes, I have a tendency to
want to reinvent a few wheels, but I think it'll give me greater
satisfaction. The problem with geda, etc., is that it's taking the
unix approach -- a buncha little command line tools that somehow make a
"system", using text files as communication medium. I hate that,
largely because it allows the infectious spread of little files all
over your system, encourages people to write scripts from outside the
system that probably won't work for you, and exposes the user
unnecessarily to the implementation of where you keep files,
directories, blabla. I'm more of the windows approach, where you have
one integrated environment, and any text you type is from within the
application itself, and most application-related data is hidden from
the user unless he *really* wants to get at it. (disclosure: I've
never actually installed geda, but I tried icarus once and couldn't get
it to compile -- another bane of open source stuff I can't stand. I'm
not a CS person, so when I download something, I just want it to work,
and I don't get off trying to *make* it work...). Another reason for
doing this on my own is that I'd like a general-purpose CAD/design
framework, of which electrical/IC design is only one part. Also, I
think geda simulations are spice-based, which is batch, which is sooo
your-father's buick, which I hate as much as text files... aaand
another thing is I'm very big on user-experience. I'd like my program
to *look* slick, like it belongs in a movie or something. I think that
means starting from scratch, since I've not seen any CAD program take
any artistic/human/psychological approach to its design.

[Paddy]

Unfortunately, Cadence got their first with their DFII environment for
Schematic based design and their Lisp based language SKILL
(http://www-mtl.mit.edu/users/xiaolin...nguserTOC.html)

I used their environment several years ago and can only say that it was
all you have stated above, and probably more. The environment seems to
be totally accessable by SKILL, from the Schematic database to dash
patterns for drawing lines. Their stuff is optimised for working with
todays Electronics designs.

I too have played what-if games about this and thought wouldn't it be
great to tie Python for the language together with SVG for the graphics
and ??? for the database but it remains a dream for me as it is a BIG
job.

Before you set out on this path, you might waant to see if you can be
shown around Cadence Virtuoso and SKILL?

- Pad.

[skip]

Someone anonymous wrote:

Cool. thanks for the links. I've already looked around quite a bit,
and am very hesitant to just write more shit on top of other shit.

That seems like a gratuitously unkind way to refer to tools you haven't
tried. Have you checked out PythonCAD? <http://www.pythoncad.org/>. I
don't use it, but the guy's been trickling out new releases periodically for
the past four years or so (30 so far). Might be worth a peek.

Skip

[Hannu Krosing]

Ühel kenal päeval, E, 2006-05-15 kell 23:49, kirjutas
63*******@sneakemail.com:

2. GUI. Yes, I know you can do guis with qt, gtk, tkinter, etc. I'm
talking of fancy guis that do alpha blending, animations, nice
shading/gradients, etc. in a quick, smooth, and slick way, such that
moving a scroll bar or jiggling the mouse yields fast game-like
response time, and which gives this program the feeling that you're
actually in the 21st century... ie this is an *interactive*
environment, and I f***king hate crass-hatching, so real colors is a
must. Can this be portable between linux and windows? Is it possible
to "do the whole thing" in opengl, even the 2d stuff? I guess I dont
know enough about guis here. My fear is that my app will turn into
x-hell if I try to do more than 8-bit colors in linux, with flashing
background windows and complaints from the server about bit planes.
Maybe I need to play with linux a bit more from the development side
before commenting on this, since I get the feeling I'm working on old
information here...

for a nice fast corel-draw clone written in python see http://www.skencil.org/

-------------
Hannu

[63q2o4i02]

Yes, I figured I should be pretty expert at what's out there first
before redoing something and making in inferior to the existing
solution. I took a quick peek at cadence courses, and they're out of
my personal price range. I have a new job coming up which should lead
into IC design after some time, and I should probably ask them to send
me to the course.

[Grant Edwards]

On 2006-05-16, 63*******@sneakemail.com <63*******@sneakemail.com> wrote:

Yes, I figured I should be pretty expert at what's out there
first before redoing something and making in inferior to the
existing solution.

Eagle from Cadsoft.de is a pretty decent (and free for
educational/hobby use) integrated schematic capture and board
layout package (including a quite usable auto-router). It
doesn't have integrated simulation, but after messing about
with both gEDA and Eagle for an hour or two the you get a good
feel for the difference between an integrated tool-set and
something cobbled together from disparate utilities.

--
Grant Edwards grante Yow! Of course, you
at UNDERSTAND about the PLAIDS
visi.com in the SPIN CYCLE --

[baalbek]

> 1. Databases. I don't mean sql type database, but design databases,

If you really want to revolutionize the CAD business, PLEASE don't base
your CAD system on a file based system (ala Autocad).

CAD systems available today (Autocad, Archicad, Architectural Desktop,
etc) have one huge flaw: they don't store data to a SQL database, but to
binary files.

This fact has created so much problems (user problems as well as
technical problems) in projects that involve several people.

I know, I have 15 years of experience supporting teams using CAD, as
well as CAD programming (mostly C++).

The fact that Autodesk still to this day has not produced an Autocad
version for a SQL database, shows how little innovation there is in
large companies.

I don't know where to start to list the numerous problems with a CAD
system based on raw binary files, so I won't bother.

Regards, and Good Luck,
Baalbek

[Ben Finney]

baalbek <rc*@bgoark.no> writes:

If you really want to revolutionize the CAD business, PLEASE don't
base your CAD system on a file based system (ala Autocad).

CAD systems available today (Autocad, Archicad, Architectural
Desktop, etc) have one huge flaw: they don't store data to a SQL
database, but to binary files.

Using Python 2.5, one could store (and distribute to others) the data
in a binary file that *is* an SQL database:

<URL:http://docs.python.org/dev/lib/module-sqlite3.html>

--
\ "I was sleeping the other night, alone, thanks to the |
`\ exterminator." -- Emo Philips |
_o__) |
Ben Finney

[Frithiof Andreas Jensen]

<63*******@sneakemail.com> wrote in message
news:11**********************@j73g2000cwa.googlegr oups.com...

Cool. thanks for the links. I've already looked around quite a bit,
and am very hesitant to just write more shit on top of other shit.
All software suck. If you think that yours will not then:

a) you are a narcissist
b) you are the only user
c) all of the above
The
idea behind this is it's completely mine. So yes, I have a tendency to want to reinvent a few wheels, but I think it'll give me greater
satisfaction.
*I* think you bite off way too much and will waste two years and never
get anything usable done.
The problem with geda, etc., is that it's taking the
unix approach -- a buncha little command line tools that somehow make a "system", using text files as communication medium.
That is because this approach actually works, it is proven in serious
work, it can be tested without writing a test application of about the
same complexity as the real job and it deals neatly with the users
needs for automation without forcing the application to contain thick,
bloated object management code for the users extensions.
I hate that,
largely because it allows the infectious spread of little files all
over your system, encourages people to write scripts from outside the system that probably won't work for you, and exposes the user
unnecessarily to the implementation of where you keep files,
directories, blabla.
Wrong on many levels:

The idiot users will get into trouble no matter what you do; while the
user that is smart enough to write an extension will need to deal with
the implementation anyway.

You, the developer, has to decide on wheather it is smarter to write
your very own object manager, API's and the user/developer
documentation thereoff or leave it to the operating system by simply
using what is already there.
I'm more of the windows approach, where you have
one integrated environment, and any text you type is from within the
application itself, and most application-related data is hidden from
the user unless he *really* wants to get at it.
I.M.O. The reason for the "integrated" windows approach is simply
because that OS lacked/lacks efficient tasks, interproces
communication and it does not support scripting.

So each developer will end up rolling his own monolith and because all
the code monkeys see this, it becomes "the way".
(disclosure: I've
never actually installed geda, but I tried icarus once and couldn't get it to compile -- another bane of open source stuff I can't stand.
The odds are great that the problem is to be found in front of the
screen rather than the code.
I'm
not a CS person, so when I download something, I just want it to work, and I don't get off trying to *make* it work...). Another reason for doing this on my own is that I'd like a general-purpose CAD/design
framework, of which electrical/IC design is only one part. Also, I
think geda simulations are spice-based, which is batch,
And this matters ... in what way exactly??
which is sooo
your-father's buick, which I hate as much as text files... aaand
another thing is I'm very big on user-experience.
"Experience" over "Functionality". I see - well, it worked for Bill.

The "knowledge workers" forced to use the pretty etch-a-schetch tools
are less than happy and will seek to minimise the pain by avoiding
most of the alledged functionality. Last time I felt "the pain of the
little used feature" was setting headers and footers in Excel - No,
it does NOT work the same as in Word ("work" is perhaps a strong
word).
I'd like my program
to *look* slick, like it belongs in a movie or something. I think that means starting from scratch, since I've not seen any CAD program take any artistic/human/psychological approach to its design.

That *is* true - the problem with CAD programs are that they need the
*exact* details to be entered at design time so one cannot easily
schetch in them and fix the design errors later.

But it does not follow that there is a need for rewriting the
rendering engine or the design rules checker - you could actually
achieve something by writing a tool that allow an existing CAD system
to support the way designers actually work so that they use the tool
up front instead of as it is now where CAD is the digitiser for a
paper-based design.

[Brendan]

On 17-May-06, at 6:33 AM, comp.lang.python group wrote:

I'd like my program
to *look* slick, like it belongs in a movie or something. I think that
means starting from scratch, since I've not seen any CAD program
take
any artistic/human/psychological approach to its design.

That *is* true - the problem with CAD programs are that they need the
*exact* details to be entered at design time so one cannot easily
schetch in them and fix the design errors later.

Not universally true, and here's a niche for you Andreas:

Most 2D cad requires that you enter ALL design information as you go.
You can't, for instance, draw a line at 30degrees to another line and
then change the second without having to redraw the first. "Sketchers"
from 3D cad programs (like Solidworks, Inventor, Solid Edge,
Pro/Engineer) are more like geometry solvers - by putting in a
dimension, you say "line a is always 30 degrees to line b". Now
when you change the angle of line b, line a changes too! In this way,
you can sketch out the SHAPE of your drawing, and worry about the
DIMENSIONS later. You can't imagine how useful this is. Now that I've
switched to Solidworks, my
drafting speed has doubled.

I haven't seen anyone make a 2D cad package with this behaviour. I'm
sure there's a market for one if you go that route.

-Brendan

[63q2o4i02]

Hi,

Actually, I've wondered about using a custom-rolled binary or
text-based database, vs. something that uses the OpenEDA standard, vs a
'real' database like sql. I guess my impression is that something like
SQL is not designed for something as multi-... uh, dimensional? as an
EE design. Perhaps it would be good for just storing the data, but can
it also store relationships or other meta-things besides the raw
literalness of a design? I'd imagine it can store arbitrary binary
data, but then what's the advantage over a custom binary file? If the
application provides a nice API for geting into its custom binary file,
then what's the problem?

ms

[63q2o4i02]

Thanks for this. I'm enjoying this discussion and I'm learning a lot
about people's views and how they differ from mine.

However, I'm still wondering about my original post.

Can the experts please comment on python's usage for the following:

1. Databases. Assuming I roll my own, does python have any
performance issues for this sort of thing?
2. GUI. Can Python command the various gui libraries fast enough to
qualify as game-quality, with transparency, anti-aliasing, animations,
etc?

3. Computational stuff like simulations and rules-checking. I'm just
going to assume this needs to be in C++ or pyrex or SciPy or something.

4. Programmability. How do I embed an interpreter which has some
custom CLI commands and knowledge of the graphical conext and which has
exposure to the API used in the main application, such that to the user
all the necessary things are exposed by default, and so that it doesn't
feel like it's just a disconnected console window that he/she needs to
set up manually each time?

5. Threads and parallelism. Should I even bother? I've read that
it's possibly more tricky with python than with normal dev tools. I've
never done it, but I'd like to at least think about it up front so
if/when this goes MT it's not a complete rewrite from the ground up.

thanks
ms

[skip]

1. Databases. Assuming I roll my own, does python have any
performance issues for this sort of thing?
Most databases are written in a compiled language (gadfly being an
exception). Even so, in my apps that use databases (mostly SQL-based), the
database is the bottleneck, not Python.
5. Threads and parallelism. Should I even bother? I've read that
it's possibly more tricky with python than with normal dev tools.

Threads help if you're going to be I/O-bound. If you're going to be
CPU-bound I wouldn't bother unless it provides some major structural
advantage to your code. If you think you need more than a couple CPUs, you
probably want a multi-process model anyway so you can leverage multiple
computers on a LAN (think Beowulf-style multiprocessing).

Skip

[David Cuthbert]

baalbek wrote:

CAD systems available today (Autocad, Archicad, Architectural Desktop,
etc) have one huge flaw: they don't store data to a SQL database, but to
binary files.

There's a reason for this. Most CAD data is not in a form (First Normal
Form) suitable for a relational database. In addition, the sheer number
of objects involved in each operation brings performance to its knees.
Finally, most RDBMS provide (and are optimized for) transactional
semantics -- for a CAD system, this is wasted overhead.

People dabbled in this years back but quickly discovered just how
different they are.

If you have access to it, you should take a peek at the source for the
OpenAccess database. Although this is for chips, I suspect some of the
same design principles would go into designing a CAD database.

[David Cuthbert]

Paddy wrote:

Unfortunately, Cadence got their first with their DFII environment for
Schematic based design and their Lisp based language SKILL

Well, SKILL (a Franz Lisp derivative) is very old and has some peculiar
design quirks. Interfacing with anything not written by Cadence or not
written in SKILL is painful, at best.

Interestingly, Python is being used as an extension language for the
OpenAccess database. LSI Logic had a Python wrapper for 2.0 (it looked
like they used SWIG). I had written one for 2.1, but never was able to
release it before my company was acquired by Cadence (and that project
quashed).

There is some interest... just not quite critical mass.

[Terry Hancock]

David Cuthbert wrote:

baalbek wrote:

CAD systems available today (Autocad, Archicad, Architectural Desktop,
etc) have one huge flaw: they don't store data to a SQL database, but to
binary files.

There's a reason for this. Most CAD data is not in a form (First Normal
Form) suitable for a relational database. In addition, the sheer number
of objects involved in each operation brings performance to its knees.
Finally, most RDBMS provide (and are optimized for) transactional
semantics -- for a CAD system, this is wasted overhead.

Object database systems (ODBMS) like ZODB are probably a
much better fit for CAD data than RDBMS. The hierarchical
structure in an object database is a natural fit to the way
CAD drawings are created and manipulated, and most CAD
systems internally represent drawings as some kind of object
tree.

Of course, ZODB *can* serialize its data to an SQL backend,
among several other possibilities. One of the more intriguing
possibilities would be using a CVS or Subversion based back-end
to provide more complete version control help.

I disagree that transactions are bad for CAD -- they may have
a different semantic role and the needed granularity may be
different, but the need to roll data back to an earlier revision
is just as present in drawings as it is for code or financial
transactions.

Cheers,
Terry

--
Terry Hancock (ha*****@AnansiSpaceworks.com)
Anansi Spaceworks http://www.AnansiSpaceworks.com

[David Cuthbert]

Terry Hancock wrote:

I disagree that transactions are bad for CAD -- they may have
a different semantic role and the needed granularity may be
different, but the need to roll data back to an earlier revision
is just as present in drawings as it is for code or financial
transactions.

Sure, but that's called source control. So you do need transactions,
but for the entire database state, not on database operations. (This
was certainly true at Cadence -- there was a thriving third-party market
for handling source control on CDBA databases, and I never encountered
any code which operated on the database in a transactional manner.
OpenAccess did have some basic support for transactions, but I never
heard of anyone using them.)

Some kind of transactionality is needed for undo/redo, but this is
usually done in a different (some might say "more efficient", others
might say "hackier") method than how transactions are implemented for
RDBMS (that I've dealt with, anyway). I suspect this can be attributed
to two major factors: concurrent access is never an issue in such
systems, and the number of database objects representing a given state
is far larger than in, say, e-commerce.

[Mike C. Fletcher]

63*******@sneakemail.com wrote:

Thanks for this. I'm enjoying this discussion and I'm learning a lot
about people's views and how they differ from mine.

However, I'm still wondering about my original post.

Can the experts please comment on python's usage for the following:

1. Databases. Assuming I roll my own, does python have any
performance issues for this sort of thing?
CAD software has generally proven to be somewhat less than amenable to
relational database models. Relational databases are good for storing
and querying large numbers of same-structure things. CAD models tend to
be large numbers of different-structure things. An object database is
likely to give you less of an impedance mismatch. With either approach,
expect your database operations to be slow compared to running in
memory. Storing the model in memory while working (if you can fit it)
is going to be a lot snappier than keeping things out in a database.

There are OO databases that have decent caching (e.g. ZODB) that can
give you some of the benefits of storing in the database without needing
to have everything loaded up at once. But keep in mind that most OO
databases are *not* optimised for this type of use-case, you may need to
rather heavily customise a standard package if you decide to use one.
You'll have to handle the partition of the data such that you can run
your simulations with partial data-sets if you want to take advantage of
that (possible with architecture, interior design, game design,
etceteras, not sure if it's possible with electrical engineering). 2. GUI. Can Python command the various gui libraries fast enough to
qualify as game-quality, with transparency, anti-aliasing, animations,
etc?
While it might be possible to do this fast enough with Python, you will
almost certainly want to use a retained-mode engine for the graphics
display. There are quite a few retained-mode engines for Python, most
are built on OpenGL. Some of those engines are written in Python, more
in C or C++. Basically what the retained mode engine does for you is it
allows you to simply describe what's supposed to be there and let
something else deal with the problem of how to display it. Retained
mode engines tend to be written by fairly experienced 3D programmers who
are somewhat obsessed by performance issues. You can readily setup the
node-graph in (slower) Python code and then have the engine handle the
heavy per-frame rendering.

The retained mode engine will (should) take care of things such as
caching intermediate rendering data (such as simple data-pointers for
rendering entire swaths of your drawings with a single GL call). They
will only update those structures when you actually change something
that affects the rendering. Their formality of operation allows them to
make quite a few optimisations even though some are quite general
platforms (the more general, the fewer optimisations they can easily
make). (For simple drawings it's easy for you to make your own, better,
optimisations, but with more complex scenes such optimisations become
more and more cumbersome without introducing formal structures). 3. Computational stuff like simulations and rules-checking. I'm just
going to assume this needs to be in C++ or pyrex or SciPy or something.
Numpy will often be sufficient for many scientific simulations, but it's
all about what algorithms you need to implement. 4. Programmability. How do I embed an interpreter which has some
custom CLI commands and knowledge of the graphical conext and which has
exposure to the API used in the main application, such that to the user
all the necessary things are exposed by default, and so that it doesn't
feel like it's just a disconnected console window that he/she needs to
set up manually each time?
Most Python GUI libraries have a console widget into which you can pass
context. Doesn't particularly require a lot of work, just follow along
in the docs for the GUI widget. You can see how it's done with wxPython
and OpenGLContext here:

http://pyopengl.cvs.sourceforge.net/...py?view=markup 5. Threads and parallelism. Should I even bother? I've read that
it's possibly more tricky with python than with normal dev tools. I've
never done it, but I'd like to at least think about it up front so
if/when this goes MT it's not a complete rewrite from the ground up.

Threads aren't any more complex with Python than other tools, if
anything they're easier (Python has a good basic API for threading).
The problem is that they are far less *useful* than in C or the like,
because the global interpreter lock means they don't let you take
advantage of multiple processors in the box in most cases. That said,
threading is always tricky, particularly during debugging. If you can
avoid it for most operations and only put things in threads that are
largely disconnected from other processes you'll likely be happier. The
big win would be if you are doing all of your simulation in C with an
event-based update such that the simulation generates changes as a
queued list of updates to the model. You could then have the simulation
yield the interpreter lock and seldom block the rest of the process
(save when it touches a Python object (i.e. the updates coming from or
going to the queue)).

Have fun,
Mike

--
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

[baalbek]

David Cuthbert wrote:

Some kind of transactionality is needed for undo/redo, but this is
usually done in a different (some might say "more efficient", others
might say "hackier") method than how transactions are implemented for
RDBMS (that I've dealt with, anyway). I suspect this can be attributed
to two major factors: concurrent access is never an issue in such
systems, and the number of database objects representing a given state
is far larger than in, say, e-commerce.

No, concurrent access is highly relevant; for example, on a team of
about 50 architects working on design and production drawings for a new
hospital, each floor was one 'drawing' (dwg file), and thus stored on
disk as a separate entity from the other floors.

Now, only one architect could work on one floor at any time! And as info
from the project goes (sums and statistics of data, for example areas),
the only reliable (an efficient) way to gather this was to hack an
in-house application that collected the desired info from the various
binary dwgs stored on disk, and saved this data this to a RDBMS, for
further report processing.

And as far as administering the dwgs (see to it that the mass of
drawings follow one dimensioning convention, similar fonts etc) the CAD
systems today are a pure joke: roughly 10-30% of the time the architects
spend on CAD is pure "hacking" each drawing to conform to standard,
problems with layers being out of sync with the rest, internal blocks
not being uptodate, etc etc etc.

And just to work on one small area of a huge floor, the architect has to
load the whole freaking floor...

This is in 2006, not 1986!

All of these problems (the technical ones, as well as problems of
concurrency) would be neatly solved if the data had been stored in a
RDBMS, instead being spread around on thousands of files on disk.

The Autocad DWG format is already in a database like structure (tables
being joined by keys etc), so the technical difficulty is not the
show-stopper here; I suspect the management at Autodesk will simply not
touch the milk-cow that Autocad has become, and does not understand the
needs of the average CAD user anyway...

I'm myself really fed up with the CAD situation of today, having battled
for 10 years the various problems that comes with todays stone-age CAD
systems. Jeez, 2006, and the state of the art CAD, Autocad, is still
basically a primitive, one-user binary application. What a pathetic joke!

Baalbek

[David Cuthbert]

baalbek wrote:

No, concurrent access is highly relevant; for example, on a team of
about 50 architects working on design and production drawings for a new
hospital, each floor was one 'drawing' (dwg file), and thus stored on
disk as a separate entity from the other floors.

Now, only one architect could work on one floor at any time! And as info
from the project goes (sums and statistics of data, for example areas),
the only reliable (an efficient) way to gather this was to hack an
in-house application that collected the desired info from the various
binary dwgs stored on disk, and saved this data this to a RDBMS, for
further report processing.
You have two orthogonal thoughts going here.

It makes sense to collect statistics from a given drawing into an RDBMS
in such a way that, whenever a design is checked in, the statistics are
synched with the RDBMS. Further, such a system could (and should) make
sure that no previous checkin has been clobbered. This is a classic
source control system. I'd be surprised if there aren't plenty of
add-ons which do this. At Cadence, we provided hooks for such systems
to automatically collect necessary information.

This does not mean the design itself should be stored as an RDBMS. As
I've stated previously, CAD data (both electrical and, it appears,
mechanical) does not lend itself to RDBMS relationship modeling.

What you want is an easy way to manage the information, not dictate the
storage format of the information.
And just to work on one small area of a huge floor, the architect has to
load the whole freaking floor...

I don't have any architectural experience, so I can't say whether this
makes sense or not. However, I think of this as being akin to
complaining, "Just to edit a single function, I have to load the entire
source file/check out the entire module/rebuild and rerun all the tests!"

Not that this is necessarily an invalid idea. However, my experience
with Cadence's tools makes me believe the existing behavior might just
be the lesser of two evils. CDBA has a strict library/cell/view
hierarchy; the checkout granularity here is (usually) at the view level
(that is, when you lock a design, you're locking the view). Two
designers could edit two different cells or views in the same library.
This leads to all kinds of data inconsistency problems -- cell borders
aren't lining up in a layout, for example, or the cached data isn't
valid, etc. It's a nightmare.

The newer OpenAccess system has an option to manage this at the library
level. Unfortunately, this is rarely enabled due to backwards
compatibility.

[baalbek]

David Cuthbert wrote:

This does not mean the design itself should be stored as an RDBMS. As
I've stated previously, CAD data (both electrical and, it appears,
mechanical) does not lend itself to RDBMS relationship modeling.

I simply do not agree with this.

A CAD program (like Autocad) is nothing
but an advanced database editor: the program loads data from a binary
file and creates its object (in memory) from the tables that it reads
from the file on the disk.

The point is that this could as well have been stored on tables in a
RDBMS; the process of loading the object once the data has been fetched
(either from a binary file, or a RDBMS) the process is similar.

The advantage of having the data stored in a RDBMS is many, amongst them
the ability to check-out just the data one needs, better administration
of the drawings (consistent plotting, dimensioning styles, fonts, etc)
and a much better tool for gathering statistics of the project (sums of
areas, rooms, doors, etc etc).

What happens in the CAD program (once loaded in memory) is simply
irrelevant to how the data are stored.

Regards,
Baalbek

[David Cuthbert]

baalbek wrote:

David Cuthbert wrote:

This does not mean the design itself should be stored as an RDBMS. As
I've stated previously, CAD data (both electrical and, it appears,
mechanical) does not lend itself to RDBMS relationship modeling.

I simply do not agree with this.

A CAD program (like Autocad) is nothing
but an advanced database editor: the program loads data from a binary
file and creates its object (in memory) from the tables that it reads
from the file on the disk.

Well, then, good luck with this. Let us know when you've rediscovered
that simply structuring data in a table does not make an RDBMS.

[Paddy]

baalbek wrote:

David Cuthbert wrote:

This does not mean the design itself should be stored as an RDBMS. As
I've stated previously, CAD data (both electrical and, it appears,
mechanical) does not lend itself to RDBMS relationship modeling.

I simply do not agree with this.

A CAD program (like Autocad) is nothing
but an advanced database editor: the program loads data from a binary
file and creates its object (in memory) from the tables that it reads

Hi Baalbek,
Athouh they are database editors, they are not relational database
editors. Relational database engines fit some type of problems but
others have found that RDBMS don't fit CAD data. They are just too slow
and add complexity in mapping the 'natural CAD data formats' to the
RDBMS table format.

- Pad.

[baalbek]

Paddy wrote:

Hi Baalbek,
Athouh they are database editors, they are not relational database
editors. Relational database engines fit some type of problems but
others have found that RDBMS don't fit CAD data. They are just too slow
and add complexity in mapping the 'natural CAD data formats' to the
RDBMS table format.

Well, considering that most CAD programs (at least those I'm familiar
with) structures its data in a very similar fashion as a RDBMS, AND
given the advanced object/relational mapping tools we have today, how
hard could it be?

The CAD system does not even have to be a straight client-server, but a
three-tiered system where the application server do batch
update/fetching of the data for efficiency.
Regards,
Baalbek

[baalbek]

David Cuthbert wrote:

baalbek wrote:

David Cuthbert wrote:

This does not mean the design itself should be stored as an RDBMS.
As I've stated previously, CAD data (both electrical and, it appears,
mechanical) does not lend itself to RDBMS relationship modeling.

I simply do not agree with this.

A CAD program (like Autocad) is nothing
but an advanced database editor: the program loads data from a binary
file and creates its object (in memory) from the tables that it reads
from the file on the disk.

Well, then, good luck with this. Let us know when you've rediscovered
that simply structuring data in a table does not make an RDBMS.

Remember, a CAD binary file (at least most of them) stores its data in
tables that are already in a (first?) normal form, similar to that of a
RDBMS.

The CAD program parses the data, and the CAD objects are created by
database joins between the different tables (one table for layers, one
table for colors, one table for geometrical data of the CAD entities,
etc etc).

I never said it would be easy, but remember, the O/R mapping of today is
far ahead of what one had only 10 years ago.

Of course, I would not have the CAD client talk directly to the RDBMS,
with constantly updating the database, but through a O/R layer
(Hibernate is a good one for this) that the CAD client connects to
(either through CORBA, ICE or similar), and have the application server
do batch update/fetching of the data.

I think that those that shy away from a RDBMS CAD solution exaggerates
the complexity of the CAD structure, and underestimates the technologies
we have available today.

Regards,
Baalbek

[Paddy]

I guess the 'advanced O/R mapping tools' make it easier to map the data
to an RDBMS, but their is still the performance issue. Since this has
degenerated into a an issue of performance then I suggest the original
poster create a clear interface between his data and its persistance
method. This should make it easier to attach another, if his first
choice is not up to it.

- Cheers, Pad.