When I insert a long value as key and the object to a hashtable. it fails at
about 600,000 data points.
ht.add(long, long)
We are developing a real time application that is required to keep about 2.5
million objects in a hashtable and the object should be readly available to
the app very fast at any time.
Is it better to come up with an algorithm to have a hashtable of
hashtables(hold ing 200,000 objects each) ?
if it has limit, any other suggestions ?
Thanks 3 9816
Cengiz Dincoglu <ce************ *@hotmail.com> wrote: When I insert a long value as key and the object to a hashtable. it fails at about 600,000 data points.
How much memory do you have? Bear in mind that each long you add (each
of key and value) is going to take up 16 bytes when boxed, and there'll
also be the reference to each of them (4 bytes each). The hashtable
also remembers the hashcode, which is another 4 bytes. The internal
capacity is also always larger than the capacity, but I still can't
understand why it would be failing after 600000 iterations... indeed,
it works fine on my box:
using System;
using System.Collecti ons;
public class Test
{
static void Main()
{
Hashtable t = new Hashtable();
for (long i=0; i < 3000000000; i++)
{
if (i%100000==0)
{
Console.WriteLi ne
("i={0}, memory={1}", i, GC.GetTotalMemo ry(true));
}
t[i]=i;
}
}
}
Which version of .NET are you using? In an old version of 1.0, there
was a problem with the large object heap not getting garbage collected
- might that be what's wrong?
ht.add(long, long)
We are developing a real time application that is required to keep about 2.5 million objects in a hashtable and the object should be readly available to the app very fast at any time. Is it better to come up with an algorithm to have a hashtable of hashtables(hold ing 200,000 objects each) ?
if it has limit, any other suggestions ?
Well, you'd save an awful lot of memory by writing your own custom
collection which *only* stored longs as the keys and values, if that's
really what you'll be doing.
--
Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet/
If replying to the group, please do not mail me too
Have you considered other data structures? For example, you could use store
your objects in an array (either a strongly-typed array or an ArrayList),
sort the array, and then do a binary search to look up the right items. If
you want to use the BinarySearch method of the array or ArrayList, you could
implement IComparable using the object's key value (which wouldn't be
hard.) Otherwise, you could implement your own searching algorithm.
However, this solution wouldn't be optimal if the list frequently changes,
since you'd have to re-sort the array each time the list changes.
You could also try implementing a sorted binary tree, which would also
enable both fast key lookups (using a binary search) and fast insertions and
deletions of objects. There isn't one built into the .Net framework,
though, so you'd have to build your own.
--Robert Jacobson
"Cengiz Dincoglu" <ce************ *@hotmail.com> wrote in message
news:li******** ************@tw ister.nyc.rr.co m... When I insert a long value as key and the object to a hashtable. it fails
at about 600,000 data points.
ht.add(long, long)
We are developing a real time application that is required to keep about
2.5 million objects in a hashtable and the object should be readly available
to the app very fast at any time. Is it better to come up with an algorithm to have a hashtable of hashtables(hold ing 200,000 objects each) ?
if it has limit, any other suggestions ?
Thanks
Jon Skeet wrote:
|| Willy Denoyette [MVP] <wi************ *@skynet.be> wrote:
||| Jon Skeet wrote:
||| .....
||||| Hashtable t = new Hashtable();
||||| for (long i=0; i < 3000000000; i++)
||||| {
||| .....
|||
||| Jon,
|||
||| Are you sure this runs on your system?
||
|| Oops - not to completion, no :)
||
|| However, it *does* get past the 2.5 million stage fairly easily.
||
|| --
|| Jon Skeet - <sk***@pobox.co m>
|| http://www.pobox.com/~skeet/
|| If replying to the group, please do not mail me too
Otherwise you would have to tell us what HW and OS you did run this on ;-)
Willy. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Oliver Spiesshofer |
last post by:
Hi,
when uploadig files through a form larger than 600 kb on my server,
I am getting a
" does not allow request data with POST requests, or the amount
of data provided in the request exceeds the capacity limit."...
but the post_max_size and the upload_max_filesize are set higher than
that...
I have this problem in phpMyAdmin and Squirrelmail and other scripts... I
|
by: Claranews |
last post by:
What is the maximum capacity of a collection?
I have a routine that adds listviewitems to a collection, and stores them so
I can pick and choose which I add/delete in future. There are 6000 items in
this collection...but when it reaches 584, I get the "This collection has
reached its maximum capacity" exception.
How would I work around this?
Thanks
|
by: AAVF IT |
last post by:
Hi
I am developing an application in MS Access 2000 that is to bolt on to an
existing Access database, that links via ODBC to our UNIX-based factory
management system. I need some help in this area as am not a VB programmer,
and I try (where possible) to keep everything to simple queries etc.
We have orders that have to be scheduled for manufacture on a specific date.
However, the area in which they are manufactured has a capacity...
|
by: Curtis |
last post by:
Does anyone know the proper method to save information to embedded
hashtables.
I am trying to save parent/child information to hashtables but I am not
getting the correct results I have made a very simplified example of
problem with the following code:
Dim cust As New Hashtable
Dim invoice As New Hashtable
Dim invoiceLines As New Hashtable
|
by: Cengiz Dincoglu |
last post by:
When I insert a long value as key and the object to a hashtable. it fails at
about 600,000 data points.
ht.add(long, long)
We are developing a real time application that is required to keep about 2.5
million objects in a hashtable and the object should be readly available to
the app very fast at any time.
Is it better to come up with an algorithm to have a hashtable of
hashtables(holding 200,000 objects each) ?
| |
by: freduchi |
last post by:
Hi, I need sugestions.
I need to implement the next and I don´t know how to do it:
I am developing a website in which users are signed up and share
information(Messages, Photos...).
When a user is signed up, his data is stored in a database and he is
asigned with a folder.
What I need, is to limit the folder capacity of each user to a given
|
by: PAzevedo |
last post by:
I have this Hashtable of Hashtables, and I'm accessing this object from
multiple threads, now the Hashtable object is thread safe for reading,
but not for writing, so I lock the object every time I need to write to
it, but now it occurred to me that maybe I could just lock one of the
Hashtables inside without locking the entire object, but then I thought
maybe some thread could instruct the outside Hashtable to remove an
inside Hashtable...
|
by: Tem |
last post by:
Is there a way to set a hard limit on the capacity of a List<>?
|
by: maheswaran |
last post by:
Hi want to know,
Total size/capacity of mysql,
(If have then how to increase)
Total size/capacity of Database(every),
(If have then how to increase)
Total size/capacity of Table(every).....
(If have then how to increase)
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |