SCons build tool speed

[ted]

How does the speed of the Scons build tool compare with Ant? Right now with
out Ant builds take around an hour. Hoping to speed that up.

TIA,
Ted

[Christopher De Vries]

On Sat, Feb 12, 2005 at 07:16:02PM +0000, ted wrote:

How does the speed of the Scons build tool compare with Ant?

I would recommend asking this question on us***@scons.tig ris.org , but my
impressions is that most of the time is probably spent in the compiler. If you
are working on a java project you could try switching from javac to jikes and
that might improve your time, though it has been a while since I used jikes.

Chris

[Peter Hansen]

ted wrote:

How does the speed of the Scons build tool compare with Ant? Right now with
out Ant builds take around an hour. Hoping to speed that up.

Don't tools like Scons, Ant, and for that matter "make" just
execute other programs? So that 99% of the time is consumed
external to the Scons, Ant, or make process itself? Why
do you think the speed of the build tool is of any significance
compared to the time for things like compilers and linkers
to execute?

-Peter

[Mike Meyer]

Peter Hansen <pe***@engcorp. com> writes:

ted wrote:

How does the speed of the Scons build tool compare with Ant? Right
now with out Ant builds take around an hour. Hoping to speed that up.

Don't tools like Scons, Ant, and for that matter "make" just
execute other programs? So that 99% of the time is consumed
external to the Scons, Ant, or make process itself? Why
do you think the speed of the build tool is of any significance
compared to the time for things like compilers and linkers
to execute?

Actually, these tools do a number of things oter than run other
programs. For instance, they parse input files to construct a build
tree, walk the build tree figuring out what needs to be rebuilt, and
only *then* do they get around to executing those other
programs. Assuming that the work prior to running those other programs
is "neglible" can lead to some truly atrocious build-time behavior. In
fact, one major advantage that ant/scons/jam/etc. have over Make is
that they avoid the defacto standard "recursive make" build
system. See <URL:
http://www.pcug.org.au/~millerp/rmch...cons-harm.html > for
details on that.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Neil Hodgson]

ted:

How does the speed of the Scons build tool compare with Ant?
Right now with out Ant builds take around an hour. Hoping to
speed that up.

Scons emphasises accuracy over speed and is normally a little slower than
other build tools although still fast enough for most purposes. One cause of
slowness is that it reads the source files, traces include files and
checksums them all rather than relying on file times. There are some things
you can do to speed it up:
http://www.scons.org/cgi-bin/wiki/GoFastButton

Neil

[PA]

On Feb 13, 2005, at 12:20, Neil Hodgson wrote:

http://www.scons.org/cgi-bin/wiki/GoFastButton

Out of curiosity, why does scons uses MD5 by default? Is that not, er,
somewhat heavy handed for most practical purpose?

Would not some sort of lightweight CRC be "good enough" to start with?

http://samba.anu.edu.au/rsync/tech_report/node3.html

Cheers

--
PA, Onnay Equitursay
http://alt.textdrive.com/

[knight]

Hi Neil, ted, et al.--

How does the speed of the Scons build tool compare with Ant?
Right now with out Ant builds take around an hour. Hoping to
speed that up.

Scons emphasises accuracy over speed and is normally a little
slower than other build tools although still fast enough for most
purposes. One cause of slowness is that it reads the source files,
traces include files and checksums them all rather than relying
on file times. There are some things you can do to speed it up:
http://www.scons.org/cgi-bin/wiki/GoFastButton

In anticipation of the next release of SCons, we've been doing
a *lot* of work on profiling the performance and eliminating
bottlenecks.

It turns out that scanning the source files and performing the MD5
calculation is not the dominant factor that we've all assumed it is.
There were some other inefficiencies in some of our algorithms that
were much more significant, including some unnecessary disk scans,
recomputing the same dependencies for every target in a list of
targets generated by one command, and repeated just-in-time creation
of some internal objects that could be created just once and cached.

That said, SCons will never be as fast as Make, because it *is*
doing more for you under the covers. Although the next version
won't necessarily speed up every configuration (the bottlenecks
are extremely configuration dependent), it should be a significant
improvement for many configurations out there.

--SK

[knight]

Hi Peter--

How does the speed of the Scons build tool compare with
Ant? Right now with out Ant builds take around an hour. Hoping
to speed that up.

Don't tools like Scons, Ant, and for that matter "make" just
execute other programs? So that 99% of the time is consumed
external to the Scons, Ant, or make process itself? Why
do you think the speed of the build tool is of any significance
compared to the time for things like compilers and linkers
to execute?

You're right that build times are dominated by the external commands
when rebuilds occur. If nothing needs to be rebuilt, though, the
wall-clock time is obviously dominated by how long it takes for
the build tool to decide that. A build tool that doesn't decide
whether or not targets are up-to-date *and* dispatch the commnds
to rebuild them reasonably quickly and efficiently doesn't do a
good job of serving its users.

--SK

[Peter Hansen]

kn****@baldmt.c om wrote:

You're right that build times are dominated by the external commands
when rebuilds occur. If nothing needs to be rebuilt, though, the
wall-clock time is obviously dominated by how long it takes for
the build tool to decide that. A build tool that doesn't decide
whether or not targets are up-to-date *and* dispatch the commnds
to rebuild them reasonably quickly and efficiently doesn't do a
good job of serving its users.

That answer, combined with Mike's response pointing out
that tools more sophisticated than basic "make" actually
can delve into the source and identify the dependencies,
removes my confusion. Thanks. :)

-Peter