473,225 Members | 1,187 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,225 software developers and data experts.

Performance issues with multi-dimensional arrays

Hi,

consider the attached code.

Serializing the multi-dimensional array takes about 36s
vs. 0.36s for the single-dimensional array.

Initializing the multi-dimensional array takes about 4s
vs. 0.3s for the single-dimensional array.
(I know initializing is not necessary in this simple example,
but in my application it was necessary to frequently
re-initialize an array)

Are there any workarounds other than using single-dimensional arrays,
store the array bounds in additional fields (in the actual code the
arrays are not always zero based) and do the index calculations in code?

TIA,
Henrik

byte[, ,] multiDimArray = new byte[1000, 1000, 64];
byte[] singleDimArray = new byte[64000000];

DateTime start = DateTime.Now;
using (Stream stream = File.Open(@"d:\test.tst", FileMode.OpenOrCreate))
{
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, multiDimArray);
}
Console.WriteLine("Serialize multi dim " + (DateTime.Now -
start).TotalSeconds);

start = DateTime.Now;
using (Stream stream = File.Open(@"d:\test.tst", FileMode.OpenOrCreate))
{
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, singleDimArray);
}
Console.WriteLine("Serialize single dim " + (DateTime.Now -
start).TotalSeconds);

start = DateTime.Now;
for (int i = multiDimArray.GetLowerBound(0); i <=
multiDimArray.GetUpperBound(0); ++i)
for (int j = multiDimArray.GetLowerBound(1); j <=
multiDimArray.GetUpperBound(1); ++j)
for (int k = multiDimArray.GetLowerBound(2); k <=
multiDimArray.GetUpperBound(2); ++k)
multiDimArray[i, j, k] = 0;
Console.WriteLine("Init multi dim " + (DateTime.Now - start).TotalSeconds);

start = DateTime.Now;
for (int i = 0; i < singleDimArray.Length; ++i)
singleDimArray[i] = 0;
Console.WriteLine("Init single dim " + (DateTime.Now - start).TotalSeconds);

Nov 12 '08 #1
4 7299
Lee
Henrik,

I'm not sure about the serialization itself but I noticed that you are
always doing the muti array first. Remember that spinning up objects
take time. When I ran your code, I got similar numbers, but when I
switched the call around to run the single array serialization first I
got these numbers:

Serialize single dim 3.1087183
Serialize multi dim 18.9803655

True, the multi is still higher, but not 10x higher.

Also, in your code to initialize the arrays, you make repeated calls
to GetLowerBound and GetUpperBound. These take time, lots of time. I
changed your code to store them off as temporary variables first:

for (int i = a; i <= x; ++i)
for (int j = b; j <= y; ++j)
for (int k = c; k <= z; ++k)
multiDimArray[i, j, k] = 0;

Console.WriteLine("Init multi dim " + (DateTime.Now -
start).TotalSeconds);

And I got this for the times:

Init multi dim 0.5155161
Init single dim 0.8904369

Now the multi is faster.

Hope any of this helps,

L. Lee Saunders
http://oldschooldotnet.blogspot.com
Nov 12 '08 #2
Thank you, Lee.

You are right, storing the array bounds in temporaries
does speed things up in the multi dim case.
I didn't think of this, because for single dim it actually
hurts performance.
By storing the bounds in variables, I could improve the performance
of my application.

There is still the issue with (de)serialization.
And the difference is more like 100x, not 10x.
My little test program may not be optimal, but I don't see
much difference when serializing the single dim array first.

Anyway, I worked around this by changing this huge array to
one-dimensional and this dramatically reduced the time for
opening and saving documents in the application.

Thanks again.

Henrik

"Lee" wrote:
Henrik,

I'm not sure about the serialization itself but I noticed that you are
always doing the muti array first. Remember that spinning up objects
take time. When I ran your code, I got similar numbers, but when I
switched the call around to run the single array serialization first I
got these numbers:

Serialize single dim 3.1087183
Serialize multi dim 18.9803655

True, the multi is still higher, but not 10x higher.

Also, in your code to initialize the arrays, you make repeated calls
to GetLowerBound and GetUpperBound. These take time, lots of time. I
changed your code to store them off as temporary variables first:

for (int i = a; i <= x; ++i)
for (int j = b; j <= y; ++j)
for (int k = c; k <= z; ++k)
multiDimArray[i, j, k] = 0;

Console.WriteLine("Init multi dim " + (DateTime.Now -
start).TotalSeconds);

And I got this for the times:

Init multi dim 0.5155161
Init single dim 0.8904369

Now the multi is faster.

Hope any of this helps,

L. Lee Saunders
http://oldschooldotnet.blogspot.com
Nov 13 '08 #3
Henrik Schmid wrote:
Hi,

consider the attached code.

Serializing the multi-dimensional array takes about 36s
vs. 0.36s for the single-dimensional array.

Initializing the multi-dimensional array takes about 4s
vs. 0.3s for the single-dimensional array.
(I know initializing is not necessary in this simple example,
but in my application it was necessary to frequently
re-initialize an array)

Are there any workarounds other than using single-dimensional arrays,
store the array bounds in additional fields (in the actual code the
arrays are not always zero based) and do the index calculations in code?

TIA,
Henrik

byte[, ,] multiDimArray = new byte[1000, 1000, 64];
You can also use byte[][][], which is called a 'jagged' array. A jagged
array is much faster. it requires a bit of different code to work with
them, but that won't be rocket science. See:
http://dotnetperls.com/Content/Jagged-Array.aspx

FB
--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------
Nov 14 '08 #4
Hi,

thanks for the reply.

Actually, the first thing I tried was a "semi-jagged" array: byte[,][]
which was even slower.
No I tried byte[][][], which is a bit faster (factor 4) than multi dim,
but still slower than single dim (factor 20).

Given that in my real application the first two dimensions are not zero-based,
I would still have to do some index calculation, so I can as well use a
single dim array and have the full performance.

Maybe some future framework or compiler version can apply similar
optimizations
to multi dim arrays.

Thanks anyway.

Henrik

"Frans Bouma [C# MVP]" wrote:
byte[, ,] multiDimArray = new byte[1000, 1000, 64];

You can also use byte[][][], which is called a 'jagged' array. A jagged
array is much faster. it requires a bit of different code to work with
them, but that won't be rocket science. See:
http://dotnetperls.com/Content/Jagged-Array.aspx

FB
Nov 14 '08 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: sandy | last post by:
Hi All, I am a newbie to MySQL and Python. At the first place, I would like to know what are the general performance issues (if any) of using MySQL with Python. By performance, I wanted to...
3
by: Amit Dedhia | last post by:
Hi I am developing a Dot net application (involving image processing) on a uni processor. It works well on my machine. I then take all my code on a multi processor, build and run the application...
3
by: adsheehan | last post by:
Hi all, Wondering if a GIL lock/unlock causes a re-schedule/contect swap when embedding Python in a multi-threaded C/C++ app on Unix ? If so, do I have any control or influence on this...
9
by: bluedolphin | last post by:
Hello All: I have been brought onboard to help on a project that had some performance problems last year. I have taken some steps to address the issues in question, but a huge question mark...
24
by: Bob Alston | last post by:
Most of my Access database implementations have been fairly small in terms of data volume and number of concurrent users. So far I haven't had performance issues to worry about. <knock on wood> ...
115
by: Mark Shelor | last post by:
I've encountered a troublesome inconsistency in the C-language Perl extension I've written for CPAN (Digest::SHA). The problem involves the use of a static array within a performance-critical...
2
by: 1944USA | last post by:
I am re-architecting a C# application written as a multithreaded Windows Service and trying to squeeze every bit of performance out of it. 1) Does the thread that an object is instantiated on...
3
by: yonil | last post by:
Over the years of using C++ I've begun noticing that freestore management functions (malloc/free) become performance bottlenecks in complex object-oriented libraries. This is usually because these...
4
by: skotapal | last post by:
Hello I manage a web based VB .net application. This application has 3 components: 1. Webapp (this calls the executibles) 2. database 3. business logic is contained in individual exe...
2
by: jwalsh604 | last post by:
I have an Access db application that is intended to present the user with the next available record to make an outbound phone call on. Originally, I had set it up as a "split" database but, due to...
1
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: mar23 | last post by:
Here's the situation. I have a form called frmDiceInventory with subform called subfrmDice. The subform's control source is linked to a query called qryDiceInventory. I've been trying to pick up the...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
by: jimatqsi | last post by:
The boss wants the word "CONFIDENTIAL" overlaying certain reports. He wants it large, slanted across the page, on every page, very light gray, outlined letters, not block letters. I thought Word Art...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.