473,396 Members | 1,916 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

ISA Server DestinationSet Performance

Dear,

I've made a little tool that synchronizes my ISA Server block list with a
database. However the performance of this is very low.

I'm retreiving 41140 rows from the database, storing them in a hash table
and clearing the input tables within 1 or 2 seconds.
Getting 41140 rows from the ISA server destination set takes me an awsome
173 seconds!

Doing some processing (lookup and deletes on the hashtable) makes this
retreiving good for 379 seconds.
173 seconds for retreiving the entries from the ISA server.
206 seconds for getting the entry elements and removing the entries from the
hash table.

The objects that I'm working with and probably gives the problem is
FPCDestinationSet (and FPCDestination). These are COM objects.
I use a enumerator to work through the FPCDestinationSet (there is no other
way to do so).

My initial conclusion (though premature) would be that the COM interface is
very slow. Loading the same list in the ISA Managment console takes below 2
seconds!

What can I do to improve the performance of this application?
Should I use unsafe code for the COM parts?
A secondary problem is using over 100 MB of RAM for the application (I got
it down already from 160 MB). 50k* 200 Bytes = 10MB (50k rows with <80 byte
strings (take 120 byte for overhead). How do I get this down to a
respectible level?

In .NET the FPCDestinationSet has the function SerializeToBuffer returning a
signed byte array, but this function is not documented anywhere.

- Joris
Nov 16 '05 #1
2 1861
Are you sure the list is completely filled (try scrolling to the bottom) in
2 seconds when running from the management console, could it be possible
that the list is filled with data taken asynchronously?
Could you try the same using a simple script (vbscript or jscript) and
check the time to load the whole 50K elements and check memory consumption?
As for the time spent to get and remove the elements from the hashtable, it
would help if you could post some code.

Willy.

"Joris Dobbelsteen" <jo***************@mail.com> wrote in message
news:41***********************@news.wanadoo.nl...
Dear,

I've made a little tool that synchronizes my ISA Server block list with a
database. However the performance of this is very low.

I'm retreiving 41140 rows from the database, storing them in a hash table
and clearing the input tables within 1 or 2 seconds.
Getting 41140 rows from the ISA server destination set takes me an awsome
173 seconds!

Doing some processing (lookup and deletes on the hashtable) makes this
retreiving good for 379 seconds.
173 seconds for retreiving the entries from the ISA server.
206 seconds for getting the entry elements and removing the entries from
the
hash table.

The objects that I'm working with and probably gives the problem is
FPCDestinationSet (and FPCDestination). These are COM objects.
I use a enumerator to work through the FPCDestinationSet (there is no
other
way to do so).

My initial conclusion (though premature) would be that the COM interface
is
very slow. Loading the same list in the ISA Managment console takes below
2
seconds!

What can I do to improve the performance of this application?
Should I use unsafe code for the COM parts?
A secondary problem is using over 100 MB of RAM for the application (I got
it down already from 160 MB). 50k* 200 Bytes = 10MB (50k rows with <80
byte
strings (take 120 byte for overhead). How do I get this down to a
respectible level?

In .NET the FPCDestinationSet has the function SerializeToBuffer returning
a
signed byte array, but this function is not documented anywhere.

- Joris

Nov 16 '05 #2
"Willy Denoyette [MVP]" <wi*************@pandora.be> wrote in message
news:uN**************@TK2MSFTNGP10.phx.gbl...
Are you sure the list is completely filled (try scrolling to the bottom) in 2 seconds when running from the management console, could it be possible
that the list is filled with data taken asynchronously?
I wasn't intending to count 40000 rows :(
But a light look made it look like its was OK. The start was good and so was
the end. Also the list didn't scroll too fast, making it all appear it had
40.000 rows listed.
(Immediately after it started).

The FPCDestinationSet has a function called:
SerializeToBuffer (which results in a array of signed bytes).
This function works is very fast, but its completely undocumented.
Could you try the same using a simple script (vbscript or jscript) and
check the time to load the whole 50K elements and check memory consumption?

I did it in VB6 (code at the end of the post)
Startup 19 MB.
Ending 50 MB
Delta is 30 MB for the destination array (I staticly allocated 99999
elements, 41140 are filled).

This only takes 2 seconds (oposite to 2-3 minutes!).
I would conclude that .NET-COM interop is extremely slow.

Are the any recommendations on how to increase its performance?
As for the time spent to get and remove the elements from the hashtable, it would help if you could post some code.
The hash table isn't the problem.
The problem is getting the data from the enumerated objects (they are COM
too). And the above conclusion was that .NET - COM support is very crappy.
The hashtable works at nearly instant speeds for even 40000 elements.
The source is a DataTable that is put into it, removed and put into it
again.
Code below (all is done within a second of CPU time):
Sorry for the crappy layout (I need to drop Outlook Express some day).

*** START ***
#region "Table To Hashtable"
protected static Hashtable TableToStrings(DataTable dt)
{
Hashtable ht = new Hashtable(dt.Rows.Count, (float)0.1);

System.Diagnostics.Debug.WriteLine("* " + DateTime.Now.ToString("hh:mm:ss")
+ " filling");
foreach (DataRow dr in dt.Rows)
{
string key = ((string)dr[0]).Replace('%', '*');
if (dr[1] != null && !(dr[1] is DBNull))
key += "$" + ((string)dr[1]).Replace('%', '*');
ht.Add(key, null);
}

System.Diagnostics.Debug.WriteLine("* " + DateTime.Now.ToString("hh:mm:ss")
+ " clearing");
foreach (DataRow dr in dt.Rows)
{
string key = ((string)dr[0]).Replace('%', '*');
if (dr[1] != null && !(dr[1] is DBNull))
key += "$" + ((string)dr[1]).Replace('%', '*');
if (ht.Contains(key))
ht.Remove(key);
}

System.Diagnostics.Debug.WriteLine("* " + DateTime.Now.ToString("hh:mm:ss")
+ " " + ht.Count + " remainders, clearing rest");
ht.Clear();

System.Diagnostics.Debug.WriteLine("* " + DateTime.Now.ToString("hh:mm:ss")
+ " filling again");
foreach (DataRow dr in dt.Rows)
{
string key = ((string)dr[0]).Replace('%', '*');
if (dr[1] != null && !(dr[1] is DBNull))
key += "$" + ((string)dr[1]).Replace('%', '*');
ht.Add(key, null);
}
return ht;
}
#endregion
*** END ***

Willy.

[snip]

*** NOTES ***
Add "Microsoft Internet Securty and Acceleration Server Adminstrative
Objects" as a reference
*** START VB6 CODE FOR READING ELEMENTS ***
Private Sub Form_Load()
Dim thefpc As FPCLib.fpc
Set thefpc = CreateObject("FPC.Root")

Dim fpce As FPCLib.FPCEnterprise
Set fpce = thefpc.Enterprise

Dim polelem As FPCLib.FPCPolicyElements
Set polelem = fpce.PolicyElements

Dim destSet As FPCLib.FPCDestinationSet
Set destSet = polelem.DestinationSets.Item("Explicit Content")

Debug.Print Format(Now, "hh:mm:ss") & " navigation complete..."

Dim da() As String
ReDim da(0 To 99999) As String

Debug.Print Format(Now, "hh:mm:ss") & " allocation complete..."

Dim dest As FPCLib.FPCDestination
Dim x As Long
x = 0
For Each dest In destSet
Select Case dest.Type
Case FPCLib.FpcDestinationAddressType.fpcDestinationTyp eDomain:
da(x) = dest.DomainName
Case
FPCLib.FpcDestinationAddressType.fpcDestinationTyp eSingleIP:
da(x) = dest.IP_From
Case FPCLib.FpcDestinationAddressType.fpcDestinationTyp eIPRange:
da(x) = dest.IP_From & "^" & dest.IP_To
End Select

If Not IsNull(dest.Path) Then
da(x) = da(x) & "$" & dest.Path
End If

x = x + 1
Next

Debug.Print Format(Now, "hh:mm:ss") & " finished with " & x & "
elements..."
MsgBox "Done..."
End Sub
*** END ***
Nov 16 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Lorax | last post by:
I'm on the IS team of a medium-sized non-profit with international reach. We're trying to make some decisions regarding our Web server and database server as we expand our web site to have more...
3
by: Varkey | last post by:
Dear friends, I am new to .NET based app development and have a pretty elementary query, I suppose... I have caught up with the basics of .NET pretty well, thanks to some Microsoft VB/ASP...
2
by: Marc Melancon | last post by:
Will the next release of SQL Server 2000 64bit sp provide performance counter? MarcM
26
by: David W. Fenton | last post by:
A client is panicking about their large Access application, which has been running smoothly with 100s of thousands of records for quite some time. They have a big project in the next year that will...
6
by: Daniel Walzenbach | last post by:
Hi, I have a web application which sometimes throws an “out of memory” exception. To get an idea what happens I traced some values using performance monitor and got the following values (for...
3
by: datapro01 | last post by:
I am a DB2 DBA that has been asked to become familiar enough with SQL Server in order to become actively involved in its installation, implementation, and to review database backup/recovery...
29
by: Jan | last post by:
Hi: I have an Access database that's been running (in one form or another) for a couple of different clients for a few years. Now a new client has requested that it be implemented with a SQL...
4
by: dorpnospam | last post by:
We have an old but very critical application that was written in VB 6 against Access 95 dbs. We need to ditch this decrepit old unstable db platform but we are trying to determine the best...
2
by: kodart | last post by:
Introduction Performance is the main concern to most server application developers. Thats why many of them anticipate using .NET platform to develop high performance server application regardless...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development projectplanning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.