By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,061 Members | 1,539 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,061 IT Pros & Developers. It's quick & easy.

Thread and memory consumption

P: n/a
Hi,

Does anybody is experiencing a lot of RAM consumption when using many
threads ?

If yes, how can we reduce that level of used memory?

Thanks tou!
Marty
Nov 21 '05 #1
Share this Question
Share on Google+
10 Replies


P: n/a
Use less threads? Reduce the amount of allocations each thread is doing?

It all depends on what resources your threads are using. Presumably they
aren't just idling ;)
"Marty" <xm******@hotmail.com> wrote in message
news:txgqe.41479$wr.27088@clgrps12...
Hi,

Does anybody is experiencing a lot of RAM consumption when using many
threads ?

If yes, how can we reduce that level of used memory?

Thanks tou!
Marty

Nov 21 '05 #2

P: n/a
Hi Robin,

I run a thread in an object (named myQueue) that raise an event for
every item within a queue. So it is an object that empty a queue of
pending messages.

That object, myQueue, is used in a server that accept up to 50
concurrent client connection, each connection having its own myQueue.
Each client sends messages to the server and message are queued in the
client's myQueue.

The thread will sleep for 10 milliseconds when it finish an iteration of
its queue.

This is very low on ressources (it use only a queue in each thread) and
I was wondering if using such threads would be the reason for a high
memory usage. Does microsoft document the consumption of a single
thread, just idling? How could I quantify it without looking in the task
manager?

Thanks,
Marty

Robin Tucker wrote:
Use less threads? Reduce the amount of allocations each thread is doing?

It all depends on what resources your threads are using. Presumably they
aren't just idling ;)
"Marty" <xm******@hotmail.com> wrote in message
news:txgqe.41479$wr.27088@clgrps12...
Hi,

Does anybody is experiencing a lot of RAM consumption when using many
threads ?

If yes, how can we reduce that level of used memory?

Thanks tou!
Marty



Nov 21 '05 #3

P: n/a
Why do you queue the messages? How did you setup your server? Does each
client have its own socket?
"Marty" <xm******@hotmail.com> wrote in message
news:G1hqe.41490$wr.27266@clgrps12...
Hi Robin,

I run a thread in an object (named myQueue) that raise an event for
every item within a queue. So it is an object that empty a queue of
pending messages.

That object, myQueue, is used in a server that accept up to 50
concurrent client connection, each connection having its own myQueue.
Each client sends messages to the server and message are queued in the
client's myQueue.

The thread will sleep for 10 milliseconds when it finish an iteration of
its queue.

This is very low on ressources (it use only a queue in each thread) and
I was wondering if using such threads would be the reason for a high
memory usage. Does microsoft document the consumption of a single
thread, just idling? How could I quantify it without looking in the task
manager?

Thanks,
Marty

Robin Tucker wrote:
Use less threads? Reduce the amount of allocations each thread is doing?

It all depends on what resources your threads are using. Presumably they
aren't just idling ;)
"Marty" <xm******@hotmail.com> wrote in message
news:txgqe.41479$wr.27088@clgrps12...
Hi,

Does anybody is experiencing a lot of RAM consumption when using many
threads ?

If yes, how can we reduce that level of used memory?

Thanks tou!
Marty


Nov 21 '05 #4

P: n/a
ok, the server has a socket listener that listen for potential clients.
When a client connect to the server, the server assign a socket for
that new client and continue to listen for other clients.

When the server connect the client, it create also a myQueue that I
explained previously. The client begin to send messages to the server,
when messages are received on the server side, they are queued in
myQueue, then unqueued to be processed. The queue is used to have a
smoother execution, so I don't have to allocate a big buffer for the
socket incoming data.

So each clients has its own queue in the server.

Any idea if that number of threads could increase memory usage?


Some Guy wrote:
Why do you queue the messages? How did you setup your server? Does each
client have its own socket?
"Marty" <xm******@hotmail.com> wrote in message
news:G1hqe.41490$wr.27266@clgrps12...
Hi Robin,

I run a thread in an object (named myQueue) that raise an event for
every item within a queue. So it is an object that empty a queue of
pending messages.

That object, myQueue, is used in a server that accept up to 50
concurrent client connection, each connection having its own myQueue.
Each client sends messages to the server and message are queued in the
client's myQueue.

The thread will sleep for 10 milliseconds when it finish an iteration of
its queue.

This is very low on ressources (it use only a queue in each thread) and
I was wondering if using such threads would be the reason for a high
memory usage. Does microsoft document the consumption of a single
thread, just idling? How could I quantify it without looking in the task
manager?

Thanks,
Marty

Robin Tucker wrote:

Use less threads? Reduce the amount of allocations each thread is doing?

It all depends on what resources your threads are using. Presumably they
aren't just idling ;)
"Marty" <xm******@hotmail.com> wrote in message
news:txgqe.41479$wr.27088@clgrps12...
Hi,

Does anybody is experiencing a lot of RAM consumption when using many
threads ?

If yes, how can we reduce that level of used memory?

Thanks tou!
Marty


Nov 21 '05 #5

P: n/a
50 threads is way too much. You should have one thread that processes the
incoming data from all the clients and not a thread for each one. When the
data arrives you should place it in a class wide Queue or and ArrayList.

"Marty" <xm******@hotmail.com> wrote in message
news:y5Zre.64509$tt5.23456@edtnps90...
ok, the server has a socket listener that listen for potential clients.
When a client connect to the server, the server assign a socket for that
new client and continue to listen for other clients.

When the server connect the client, it create also a myQueue that I
explained previously. The client begin to send messages to the server,
when messages are received on the server side, they are queued in myQueue,
then unqueued to be processed. The queue is used to have a smoother
execution, so I don't have to allocate a big buffer for the socket
incoming data.

So each clients has its own queue in the server.

Any idea if that number of threads could increase memory usage?


Some Guy wrote:
Why do you queue the messages? How did you setup your server? Does each
client have its own socket?
"Marty" <xm******@hotmail.com> wrote in message
news:G1hqe.41490$wr.27266@clgrps12...
Hi Robin,

I run a thread in an object (named myQueue) that raise an event for
every item within a queue. So it is an object that empty a queue of
pending messages.

That object, myQueue, is used in a server that accept up to 50
concurrent client connection, each connection having its own myQueue.
Each client sends messages to the server and message are queued in the
client's myQueue.

The thread will sleep for 10 milliseconds when it finish an iteration of
its queue.

This is very low on ressources (it use only a queue in each thread) and
I was wondering if using such threads would be the reason for a high
memory usage. Does microsoft document the consumption of a single
thread, just idling? How could I quantify it without looking in the task
manager?

Thanks,
Marty

Robin Tucker wrote:
Use less threads? Reduce the amount of allocations each thread is
doing?

It all depends on what resources your threads are using. Presumably
they aren't just idling ;)
"Marty" <xm******@hotmail.com> wrote in message
news:txgqe.41479$wr.27088@clgrps12...
>Hi,
>
>Does anybody is experiencing a lot of RAM consumption when using many
>threads ?
>
>If yes, how can we reduce that level of used memory?
>
>Thanks tou!
>Marty


Nov 21 '05 #6

P: n/a
Why 50 thread would be too much? They do not perform a lot of work,
mostly idling, and ponctually performing clients requests.

I know that in process automation, systems can handles more than 1000s
of threads (in C++) and system are very reliables.

How can I quantify the ratio threads/memory and CPU to have a good and
fast execution?

Do you have any web references?

Thanks very much!
Marty
Some Guy wrote:
50 threads is way too much. You should have one thread that processes the
incoming data from all the clients and not a thread for each one. When the
data arrives you should place it in a class wide Queue or and ArrayList.

"Marty" <xm******@hotmail.com> wrote in message
news:y5Zre.64509$tt5.23456@edtnps90...
ok, the server has a socket listener that listen for potential clients.
When a client connect to the server, the server assign a socket for that
new client and continue to listen for other clients.

When the server connect the client, it create also a myQueue that I
explained previously. The client begin to send messages to the server,
when messages are received on the server side, they are queued in myQueue,
then unqueued to be processed. The queue is used to have a smoother
execution, so I don't have to allocate a big buffer for the socket
incoming data.

So each clients has its own queue in the server.

Any idea if that number of threads could increase memory usage?


Some Guy wrote:

Why do you queue the messages? How did you setup your server? Does each
client have its own socket?
"Marty" <xm******@hotmail.com> wrote in message
news:G1hqe.41490$wr.27266@clgrps12...
Hi Robin,

I run a thread in an object (named myQueue) that raise an event for
every item within a queue. So it is an object that empty a queue of
pending messages.

That object, myQueue, is used in a server that accept up to 50
concurrent client connection, each connection having its own myQueue.
Each client sends messages to the server and message are queued in the
client's myQueue.

The thread will sleep for 10 milliseconds when it finish an iteration of
its queue.

This is very low on ressources (it use only a queue in each thread) and
I was wondering if using such threads would be the reason for a high
memory usage. Does microsoft document the consumption of a single
thread, just idling? How could I quantify it without looking in the task
manager?

Thanks,
Marty

Robin Tucker wrote:

>Use less threads? Reduce the amount of allocations each thread is
>doing?
>
>It all depends on what resources your threads are using. Presumably
>they aren't just idling ;)
>
>
>"Marty" <xm******@hotmail.com> wrote in message
>news:txgqe.41479$wr.27088@clgrps12...
>
>
>
>>Hi,
>>
>>Does anybody is experiencing a lot of RAM consumption when using many
>>threads ?
>>
>>If yes, how can we reduce that level of used memory?
>>
>>Thanks tou!
>>Marty
>
>
>

Nov 21 '05 #7

P: n/a
Start Here:

http://msdn.microsoft.com/msdnmag/is...ETProfilerAPI/

"Marty" <xm******@hotmail.com> wrote in message
news:R_dse.64546$on1.28273@clgrps13...
Why 50 thread would be too much? They do not perform a lot of work,
mostly idling, and ponctually performing clients requests.

I know that in process automation, systems can handles more than 1000s of
threads (in C++) and system are very reliables.

How can I quantify the ratio threads/memory and CPU to have a good and
fast execution?

Do you have any web references?

Thanks very much!
Marty
Some Guy wrote:
50 threads is way too much. You should have one thread that processes the
incoming data from all the clients and not a thread for each one. When
the data arrives you should place it in a class wide Queue or and
ArrayList.

"Marty" <xm******@hotmail.com> wrote in message
news:y5Zre.64509$tt5.23456@edtnps90...
ok, the server has a socket listener that listen for potential clients.
When a client connect to the server, the server assign a socket for that
new client and continue to listen for other clients.

When the server connect the client, it create also a myQueue that I
explained previously. The client begin to send messages to the server,
when messages are received on the server side, they are queued in
myQueue, then unqueued to be processed. The queue is used to have a
smoother execution, so I don't have to allocate a big buffer for the
socket incoming data.

So each clients has its own queue in the server.

Any idea if that number of threads could increase memory usage?


Some Guy wrote:
Why do you queue the messages? How did you setup your server? Does each
client have its own socket?
"Marty" <xm******@hotmail.com> wrote in message
news:G1hqe.41490$wr.27266@clgrps12...
>Hi Robin,
>
>I run a thread in an object (named myQueue) that raise an event for
>every item within a queue. So it is an object that empty a queue of
>pending messages.
>
>That object, myQueue, is used in a server that accept up to 50
>concurrent client connection, each connection having its own myQueue.
>Each client sends messages to the server and message are queued in the
>client's myQueue.
>
>The thread will sleep for 10 milliseconds when it finish an iteration
>of
>its queue.
>
>This is very low on ressources (it use only a queue in each thread) and
>I was wondering if using such threads would be the reason for a high
>memory usage. Does microsoft document the consumption of a single
>thread, just idling? How could I quantify it without looking in the
>task
>manager?
>
>Thanks,
>Marty
>
>Robin Tucker wrote:
>
>
>
>>Use less threads? Reduce the amount of allocations each thread is
>>doing?
>>
>>It all depends on what resources your threads are using. Presumably
>>they aren't just idling ;)
>>
>>
>>"Marty" <xm******@hotmail.com> wrote in message
>>news:txgqe.41479$wr.27088@clgrps12...
>>
>>
>>
>>>Hi,
>>>
>>>Does anybody is experiencing a lot of RAM consumption when using many
>>>threads ?
>>>
>>>If yes, how can we reduce that level of used memory?
>>>
>>>Thanks tou!
>>>Marty
>>
>>
>>

Nov 21 '05 #8

P: n/a
Thank you, I know more about the GC.
Marty

Some Guy wrote:
Start Here:

http://msdn.microsoft.com/msdnmag/is...ETProfilerAPI/

"Marty" <xm******@hotmail.com> wrote in message
news:R_dse.64546$on1.28273@clgrps13...
Why 50 thread would be too much? They do not perform a lot of work,
mostly idling, and ponctually performing clients requests.

I know that in process automation, systems can handles more than 1000s of
threads (in C++) and system are very reliables.

How can I quantify the ratio threads/memory and CPU to have a good and
fast execution?

Do you have any web references?

Thanks very much!
Marty
Some Guy wrote:
50 threads is way too much. You should have one thread that processes the
incoming data from all the clients and not a thread for each one. When
the data arrives you should place it in a class wide Queue or and
ArrayList.

"Marty" <xm******@hotmail.com> wrote in message
news:y5Zre.64509$tt5.23456@edtnps90...
ok, the server has a socket listener that listen for potential clients.
When a client connect to the server, the server assign a socket for that
new client and continue to listen for other clients.

When the server connect the client, it create also a myQueue that I
explained previously. The client begin to send messages to the server,
when messages are received on the server side, they are queued in
myQueue, then unqueued to be processed. The queue is used to have a
smoother execution, so I don't have to allocate a big buffer for the
socket incoming data.

So each clients has its own queue in the server.

Any idea if that number of threads could increase memory usage?


Some Guy wrote:

>Why do you queue the messages? How did you setup your server? Does each
>client have its own socket?
>
>
>"Marty" <xm******@hotmail.com> wrote in message
>news:G1hqe.41490$wr.27266@clgrps12...
>
>
>
>>Hi Robin,
>>
>>I run a thread in an object (named myQueue) that raise an event for
>>every item within a queue. So it is an object that empty a queue of
>>pending messages.
>>
>>That object, myQueue, is used in a server that accept up to 50
>>concurrent client connection, each connection having its own myQueue.
>>Each client sends messages to the server and message are queued in the
>>client's myQueue.
>>
>>The thread will sleep for 10 milliseconds when it finish an iteration
>>of
>>its queue.
>>
>>This is very low on ressources (it use only a queue in each thread) and
>>I was wondering if using such threads would be the reason for a high
>>memory usage. Does microsoft document the consumption of a single
>>thread, just idling? How could I quantify it without looking in the
>>task
>>manager?
>>
>>Thanks,
>>Marty
>>
>>Robin Tucker wrote:
>>
>>
>>
>>
>>>Use less threads? Reduce the amount of allocations each thread is
>>>doing?
>>>
>>>It all depends on what resources your threads are using. Presumably
>>>they aren't just idling ;)
>>>
>>>
>>>"Marty" <xm******@hotmail.com> wrote in message
>>>news:txgqe.41479$wr.27088@clgrps12...
>>>
>>>
>>>
>>>
>>>>Hi,
>>>>
>>>>Does anybody is experiencing a lot of RAM consumption when using many
>>>>threads ?
>>>>
>>>>If yes, how can we reduce that level of used memory?
>>>>
>>>>Thanks tou!
>>>>Marty
>>>
>>>
>>>


Nov 21 '05 #9

P: n/a
Marty,

It's not necessarily bad, but it's not as scalable as other
architectures that use a fixed size thread pool. See the following
link for information about alternative design patterns that are
generally better than the thread-per-connection approach.

<http://www.cs.wustl.edu/~schmidt/patterns-ace.html>

Brian

Marty wrote:
Why 50 thread would be too much? They do not perform a lot of work,
mostly idling, and ponctually performing clients requests.

I know that in process automation, systems can handles more than 1000s
of threads (in C++) and system are very reliables.

How can I quantify the ratio threads/memory and CPU to have a good and
fast execution?

Do you have any web references?

Thanks very much!
Marty


Nov 21 '05 #10

P: n/a
Thank you Brian,
That sound very interesting, I'll read it!
Marty

Brian Gideon wrote:
Marty,

It's not necessarily bad, but it's not as scalable as other
architectures that use a fixed size thread pool. See the following
link for information about alternative design patterns that are
generally better than the thread-per-connection approach.

<http://www.cs.wustl.edu/~schmidt/patterns-ace.html>

Brian

Marty wrote:
Why 50 thread would be too much? They do not perform a lot of work,
mostly idling, and ponctually performing clients requests.

I know that in process automation, systems can handles more than 1000s
of threads (in C++) and system are very reliables.

How can I quantify the ratio threads/memory and CPU to have a good and
fast execution?

Do you have any web references?

Thanks very much!
Marty


Nov 21 '05 #11

This discussion thread is closed

Replies have been disabled for this discussion.