473,390 Members | 1,301 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,390 software developers and data experts.

Is there a capacity limit in hashtables?

When I insert a long value as key and the object to a hashtable. it fails at
about 600,000 data points.

ht.add(long, long)

We are developing a real time application that is required to keep about 2.5
million objects in a hashtable and the object should be readly available to
the app very fast at any time.
Is it better to come up with an algorithm to have a hashtable of
hashtables(holding 200,000 objects each) ?

if it has limit, any other suggestions ?

Thanks
Jul 19 '05 #1
3 9787
Cengiz Dincoglu <ce*************@hotmail.com> wrote:
When I insert a long value as key and the object to a hashtable. it fails at
about 600,000 data points.
How much memory do you have? Bear in mind that each long you add (each
of key and value) is going to take up 16 bytes when boxed, and there'll
also be the reference to each of them (4 bytes each). The hashtable
also remembers the hashcode, which is another 4 bytes. The internal
capacity is also always larger than the capacity, but I still can't
understand why it would be failing after 600000 iterations... indeed,
it works fine on my box:

using System;
using System.Collections;

public class Test
{
static void Main()
{
Hashtable t = new Hashtable();
for (long i=0; i < 3000000000; i++)
{
if (i%100000==0)
{
Console.WriteLine
("i={0}, memory={1}", i, GC.GetTotalMemory(true));
}
t[i]=i;
}
}
}

Which version of .NET are you using? In an old version of 1.0, there
was a problem with the large object heap not getting garbage collected
- might that be what's wrong?
ht.add(long, long)

We are developing a real time application that is required to keep about 2.5
million objects in a hashtable and the object should be readly available to
the app very fast at any time.
Is it better to come up with an algorithm to have a hashtable of
hashtables(holding 200,000 objects each) ?

if it has limit, any other suggestions ?


Well, you'd save an awful lot of memory by writing your own custom
collection which *only* stored longs as the keys and values, if that's
really what you'll be doing.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet/
If replying to the group, please do not mail me too
Jul 19 '05 #2
Have you considered other data structures? For example, you could use store
your objects in an array (either a strongly-typed array or an ArrayList),
sort the array, and then do a binary search to look up the right items. If
you want to use the BinarySearch method of the array or ArrayList, you could
implement IComparable using the object's key value (which wouldn't be
hard.) Otherwise, you could implement your own searching algorithm.
However, this solution wouldn't be optimal if the list frequently changes,
since you'd have to re-sort the array each time the list changes.

You could also try implementing a sorted binary tree, which would also
enable both fast key lookups (using a binary search) and fast insertions and
deletions of objects. There isn't one built into the .Net framework,
though, so you'd have to build your own.

--Robert Jacobson
"Cengiz Dincoglu" <ce*************@hotmail.com> wrote in message
news:li********************@twister.nyc.rr.com...
When I insert a long value as key and the object to a hashtable. it fails at about 600,000 data points.

ht.add(long, long)

We are developing a real time application that is required to keep about 2.5 million objects in a hashtable and the object should be readly available to the app very fast at any time.
Is it better to come up with an algorithm to have a hashtable of
hashtables(holding 200,000 objects each) ?

if it has limit, any other suggestions ?

Thanks

Jul 19 '05 #3
Jon Skeet wrote:
|| Willy Denoyette [MVP] <wi*************@skynet.be> wrote:
||| Jon Skeet wrote:
||| .....
||||| Hashtable t = new Hashtable();
||||| for (long i=0; i < 3000000000; i++)
||||| {
||| .....
|||
||| Jon,
|||
||| Are you sure this runs on your system?
||
|| Oops - not to completion, no :)
||
|| However, it *does* get past the 2.5 million stage fairly easily.
||
|| --
|| Jon Skeet - <sk***@pobox.com>
|| http://www.pobox.com/~skeet/
|| If replying to the group, please do not mail me too

Otherwise you would have to tell us what HW and OS you did run this on ;-)

Willy.
Jul 19 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Oliver Spiesshofer | last post by:
Hi, when uploadig files through a form larger than 600 kb on my server, I am getting a " does not allow request data with POST requests, or the amount of data provided in the request exceeds the...
1
by: Claranews | last post by:
What is the maximum capacity of a collection? I have a routine that adds listviewitems to a collection, and stores them so I can pick and choose which I add/delete in future. There are 6000...
2
by: AAVF IT | last post by:
Hi I am developing an application in MS Access 2000 that is to bolt on to an existing Access database, that links via ODBC to our UNIX-based factory management system. I need some help in this...
1
by: Curtis | last post by:
Does anyone know the proper method to save information to embedded hashtables. I am trying to save parent/child information to hashtables but I am not getting the correct results I have made a...
3
by: Cengiz Dincoglu | last post by:
When I insert a long value as key and the object to a hashtable. it fails at about 600,000 data points. ht.add(long, long) We are developing a real time application that is required to keep...
9
by: freduchi | last post by:
Hi, I need sugestions. I need to implement the next and I donīt know how to do it: I am developing a website in which users are signed up and share information(Messages, Photos...). When a...
2
by: PAzevedo | last post by:
I have this Hashtable of Hashtables, and I'm accessing this object from multiple threads, now the Hashtable object is thread safe for reading, but not for writing, so I lock the object every time I...
2
by: Tem | last post by:
Is there a way to set a hard limit on the capacity of a List<>?
3
by: maheswaran | last post by:
Hi want to know, Total size/capacity of mysql, (If have then how to increase) Total size/capacity of Database(every), ...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.