The SETT Programming Contest: The fastest set<T> implementation

[SETT Programming Contest]

The SETT Programming Contest: The fastest set<Timplementa tion

Write the fastest set<Timplementa tion using only standard C++/C.
Ideally it should have the same interface like std::set.
At least the following methods must be implemented:
insert(), find(), begin(), end(), erase(), size(), operator<(),
and at least the forward iterator.

Here, speed and correctness are the 2 most important factors.
Functionally it should behave similar to std::set,
have practically no limitation on the number of items,
and be significantly faster, either always or under specified
conditions (for example for random items etc.).

The class should be placed in an own namespace.
Supply also a benchmark routine which compares your
implementation with that of std::set, and briefly document
and comment these results.

Hint: Since items in a set are always in an ordered sequence
(as determined by operator<()) you need to use some tree
routines (AVL, red-black, splay, etc.) or use your own tree
method in the heart of your implementation.

See also this paper:
Ben Pfaff, "Performanc e Analysis of BST in System Software", Stanford University
http://www.stanford.edu/~blp/papers/libavl.pdf

Publish your finished project somewhere on the Web
(for example at your own homepage or at http://sourceforge.net/ )
and announce it in comp.lang.c++ .

There is no deadline on this contest. The fastest method
will be the winner of the SETT Programming Contest,
until someone comes up with a faster method.

Good Luck!

[Bartc]

"SETT Programming Contest" <se**@sett.prog ramming.contest .org.invalidwro te
in message news:g2******** **@aioe.org...

The SETT Programming Contest: The fastest set<Timplementa tion

Write the fastest set<Timplementa tion using only standard C++/C.
Ideally it should have the same interface like std::set.
At least the following methods must be implemented:
insert(), find(), begin(), end(), erase(), size(), operator<(),
and at least the forward iterator.

I'm not going to enter this (since, until today, I'd never heard of set<T>
or BST).

But I'm interested in clearing up what the task is.

A set<Tis just an arbitrary collection of values (each unique) of type T?
Like a Pascal set? (OK a Pascal set is limited to scalars and has a pre-set
range).

And T can be anything at all, provided operator < is meaningful for it?

In this case it all sounds pretty straightforward , so why limit the contest
to C++ only? Is it to eliminate languages that already have very fast
implementations built-in?

--
Bartc

[Richard Harter]

On Tue, 3 Jun 2008 21:57:52 +0100, "Stephen Howe"
<sjhoweATdialDO TpipexDOTcomwro te:

>As others have observed, the conditions stop C being used.

The basic issue for most set implmentations is that insert/delete is slowest
as memory allocation/deallocation occurs per-node.
If you double this, the perforamance of set doubles.
So an allocator geared towards allocation of nodes the same size is where
time should be spent.

Point well taken; however that part is fairly trivial (for
sufficiently difficult of versions of trivial, of course.) The
idea is to create a linked list of blocks of storage that are
used as the equivalent of automatic storage, along with a free
list of nodes. Insert snatches a node from the free list, delete
puts one back on the free list. If the free list is empty,
insert nibbles at the current block. The ime cost is a small
fixed cost per operation, negligible compared to the operation
time cost.

The space is returned by freeing the chained list of blocks one
at a time when the set is deleted.

Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[Richard Harter]

On Tue, 3 Jun 2008 16:36:06 +0200, "SETT Programming Contest"
<se**@sett.prog ramming.contest .org.invalidwro te:

>The SETT Programming Contest: The fastest set<Timplementa tion

Write the fastest set<Timplementa tion using only standard C++/C.
Ideally it should have the same interface like std::set.
At least the following methods must be implemented:
insert(), find(), begin(), end(), erase(), size(), operator<(),
and at least the forward iterator.

What about union, intersection, and set difference?
Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[Stephen Howe]

Point well taken; however that part is fairly trivial (for

sufficiently difficult of versions of trivial, of course.) The
idea is to create a linked list of blocks of storage that are
used as the equivalent of automatic storage, along with a free
list of nodes. Insert snatches a node from the free list, delete
puts one back on the free list. If the free list is empty,
insert nibbles at the current block. The ime cost is a small
fixed cost per operation, negligible compared to the operation
time cost.

The space is returned by freeing the chained list of blocks one
at a time when the set is deleted.

I wouldn't do that. The scheme above works fine providing you are recycling
nodes.
If you intend to insert nodes and the only point of deletion is when the set
is deleted, you have not gained anything.

I would looking to use a pool scheme, where large blocks of raw memory are
carved off, and sliced up per node.
Anything to reduce the cost of node allocation.

Stephen Howe

[Ben Pfaff]

"SETT Programming Contest" <se**@sett.prog ramming.contest .org.invalid>
writes:

Write the fastest set<Timplementa tion using only standard C++/C.
Ideally it should have the same interface like std::set.
At least the following methods must be implemented:
insert(), find(), begin(), end(), erase(), size(), operator<(),
and at least the forward iterator.

Wouldn't it be easier to compare the implementations if you
required them to meet all the requirements placed on std::set by
the C++ standard?

See also this paper:
Ben Pfaff, "Performanc e Analysis of BST in System Software", Stanford University
http://www.stanford.edu/~blp/papers/libavl.pdf

How amusing.
--
Ben Pfaff
http://benpfaff.org

[Stephen Howe]

I would looking to use a pool scheme, where large blocks of raw memory are

carved off, and sliced up per node.
Anything to reduce the cost of node allocation.

Perhaps run both schemes or similar. A reap springs to mind (heap inside a
region)

Stephen Howe

[Richard Harter]

On Wed, 4 Jun 2008 17:31:39 +0100, "Stephen Howe"
<sjhoweATdialDO TpipexDOTcomwro te:

>Point well taken; however that part is fairly trivial (for
sufficiently difficult of versions of trivial, of course.) The
idea is to create a linked list of blocks of storage that are
used as the equivalent of automatic storage, along with a free
list of nodes. Insert snatches a node from the free list, delete
puts one back on the free list. If the free list is empty,
insert nibbles at the current block. The ime cost is a small
fixed cost per operation, negligible compared to the operation
time cost.

The space is returned by freeing the chained list of blocks one
at a time when the set is deleted.

I wouldn't do that. The scheme above works fine providing you are recycling
nodes.
If you intend to insert nodes and the only point of deletion is when the set
is deleted, you have not gained anything.

I would looking to use a pool scheme, where large blocks of raw memory are
carved off, and sliced up per node.
Anything to reduce the cost of node allocation.

What we have here is a failure to communicate. I read you as
saying, "Don't do that. It won't work. Instead do what you're
doing." :-)

No doubt the fault is mine for not spelling out more details.
The aforementioned blocks are big blocks, aka pools, large enough
to hold upwards of 100 nodes. (I sort of took it for granted
that saying "used as the equivalent of automatic storage" was
sufficient.) Blocks are in a structure that looks something like
this:

struct stgpool {
stgpool * link;
size_t pos;
char * space;
};

We have a routine to create a pool, one to get storage from the
pool, and one to delete the pool. The argument for the routine
to get storage from a pool is the amount of storage required.
The pool allocator adds the requested size to the position
(making due allowance for alignment) and returns the old
position. In case it is not clear, "space" is the big block of
space and "pos" is the index into "space". When a block is
created we malloc space for the struct, space for a block, link
in the new stgpool, and set pos = 0.

The freelist is a refinement, letting us recycle nodes if there
are inserts and deletes. This rather simple pool allocator is a
high water allocator, i.e., it keeps all of the storage it
mallocs until the pool delete routine is called.

Incidentally, you do need to chain the pools when you don't know
in advance how much space you are actually going to need. Also,
there is no need to realloc.

Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[Richard Harter]

On Wed, 4 Jun 2008 19:34:09 +0100, "Stephen Howe"
<sjhoweATdialDO TpipexDOTcomwro te:

>I would looking to use a pool scheme, where large blocks of raw memory are
carved off, and sliced up per node.
Anything to reduce the cost of node allocation.

Perhaps run both schemes or similar. A reap springs to mind (heap inside a
region)

Actually, what I proposed was a combination of a pool allocator
and a freelist. Apparently that wasn't clear. I don't see that
a reap buys you anything.
Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[Mike]

In article <g2**********@a ioe.org>, se**@sett.progr amming.contest. org.invalid says...

The SETT Programming Contest: The fastest set<Timplementa tion

Write the fastest set<Timplementa tion using only standard C++/C.
Ideally it should have the same interface like std::set.
At least the following methods must be implemented:
insert(), find(), begin(), end(), erase(), size(), operator<(),
and at least the forward iterator.

Here, speed and correctness are the 2 most important factors.

Does this mean that a totally incorect but exceedingly fast implementation can still get a 50% pass mark?

[shablool]

On Jun 4, 7:56 pm, Ben Pfaff <[e-mail] rd.eduwrote:

"SETT Programming Contest" <[e-mail] ramming.contest .org.invalid>
writes:

Write the fastest set<Timplementa tion using only standard C++/C.
Ideally it should have the same interface like std::set.
At least the following methods must be implemented:
insert(), find(), begin(), end(), erase(), size(), operator<(),
and at least the forward iterator.

Wouldn't it be easier to compare the implementations if you
required them to meet all the requirements placed on std::set by
the C++ standard?

See also this paper:
Ben Pfaff, "Performanc e Analysis of BST in System Software", Stanford University
http://www.stanford.edu/~blp/papers/libavl.pdf

How amusing.
--
Ben Pfaffhttp://benpfaff.org

As stated in previous post, this requirement alone assumes certain
memory model, which may affect the actual performance. Nevertheless, I
would like propose the following code fragment as possible candidate
for the actual test (after-all, we should start somewhere):
template <typename SetT>
void sett_contest(Se tT& s, int n)
{
typedef typename SetT::value_typ e value_type;

int i;
std::vector<val ue_type v(n);

for (i = 0; i < n; ++i)
{
v[i] = value_type(i);
}

std::random_shu ffle(v.begin(), v.end());
s.insert(v.begi n(), v.end());

std::random_shu ffle(v.begin(), v.end());
for (i = 0; i < n; ++i)
{
s.erase(s.find( v[i]));
}
}

ShacharS.