By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
464,448 Members | 1,055 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 464,448 IT Pros & Developers. It's quick & easy.

Help on sppeding up a 15,000 line batch process

P: n/a
VM
In my Windows app, I'm running a batch process that's composed of a FOR loop
that'll run 15,000 times (datatable row count), copy cthe data of each
row -3 fields- to a struct, and send the strct to an external method (using
DLLImport). The problem is that the processing speed is very inconsistent-
it may run very FAST during certain points but then it runs VERY slow at
other moments (especially when it's getting to the end of the table: after
12,000 records). I have an idea of the speed because I have a counter being
displayed in the form (with an Application.DoEvents()). I've eliminated the
DoEvents but it's still inconsistent.

This is a new version of an application that was previously made in MS
C/C++. The only differences between my C# app and the previous version
(besides the language it was written in) is that I had to recreate the
struct in C# (which the external method receives as parameter and modifies)
and that I use the DLLImport for the method. Would this be a reason why it's
so slow? Also, the memory usage (in Task Manager) is at 100% when running.

Thanks,
VM
Nov 16 '05 #1
Share this Question
Share on Google+
17 Replies

P: n/a
Can I suggest you hook up a profiler to see where the problem lies...

Chances are it's a Interop problem but difficult to tell without the source
code etc...

There are some very good profilers out there and most have a 30 day trial
:-)

cheers,

g

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@tk2msftngp13.phx.gbl...
In my Windows app, I'm running a batch process that's composed of a FOR loop that'll run 15,000 times (datatable row count), copy cthe data of each
row -3 fields- to a struct, and send the strct to an external method (using DLLImport). The problem is that the processing speed is very inconsistent- it may run very FAST during certain points but then it runs VERY slow at
other moments (especially when it's getting to the end of the table: after
12,000 records). I have an idea of the speed because I have a counter being displayed in the form (with an Application.DoEvents()). I've eliminated the DoEvents but it's still inconsistent.

This is a new version of an application that was previously made in MS
C/C++. The only differences between my C# app and the previous version
(besides the language it was written in) is that I had to recreate the
struct in C# (which the external method receives as parameter and modifies) and that I use the DLLImport for the method. Would this be a reason why it's so slow? Also, the memory usage (in Task Manager) is at 100% when running.

Thanks,
VM

Nov 16 '05 #2

P: n/a
VM
Thanks for your reply. Would interops naturally make the system slower or
would it be something with my code? In other words, can the speed be
improved or is it a dot net limitation?

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#x*************@tk2msftngp13.phx.gbl...
Can I suggest you hook up a profiler to see where the problem lies...

Chances are it's a Interop problem but difficult to tell without the source code etc...

There are some very good profilers out there and most have a 30 day trial
:-)

cheers,

g

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@tk2msftngp13.phx.gbl...
In my Windows app, I'm running a batch process that's composed of a FOR

loop
that'll run 15,000 times (datatable row count), copy cthe data of each
row -3 fields- to a struct, and send the strct to an external method

(using
DLLImport). The problem is that the processing speed is very

inconsistent-
it may run very FAST during certain points but then it runs VERY slow at
other moments (especially when it's getting to the end of the table: after 12,000 records). I have an idea of the speed because I have a counter

being
displayed in the form (with an Application.DoEvents()). I've eliminated

the
DoEvents but it's still inconsistent.

This is a new version of an application that was previously made in MS
C/C++. The only differences between my C# app and the previous version
(besides the language it was written in) is that I had to recreate the
struct in C# (which the external method receives as parameter and

modifies)
and that I use the DLLImport for the method. Would this be a reason why

it's
so slow? Also, the memory usage (in Task Manager) is at 100% when running.
Thanks,
VM


Nov 16 '05 #3

P: n/a
I guess that depends on the Interops in question!

If you think it through then some Interop operations could certainly be
tuned - for example - if you've got a 100MB of string data and you're
passing it through Interop then the following would be true:

1. .Net stores it's string as Unicode
2. Calling an ANSI Interop function would require converting those
strings to ANSI to and from the Interop call
3. That's got to be slower than calling a UNICODE Interop function

I think the example above shows how Interop could be slower than the raw C++
but there are so many other factors involved. In my experience of calls via
Interop I haven't noticed any particular slow down so the profiler would
certainly identify which bit of code is causing the problem.

It could also be a GC issue - why don't you post the basic code and it may
be obvious (or probably won't be!)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:e6**************@TK2MSFTNGP09.phx.gbl...
Thanks for your reply. Would interops naturally make the system slower or
would it be something with my code? In other words, can the speed be
improved or is it a dot net limitation?

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#x*************@tk2msftngp13.phx.gbl...
Can I suggest you hook up a profiler to see where the problem lies...

Chances are it's a Interop problem but difficult to tell without the

source
code etc...

There are some very good profilers out there and most have a 30 day trial
:-)

cheers,

g

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@tk2msftngp13.phx.gbl...
In my Windows app, I'm running a batch process that's composed of a FOR
loop
that'll run 15,000 times (datatable row count), copy cthe data of each
row -3 fields- to a struct, and send the strct to an external method

(using
DLLImport). The problem is that the processing speed is very

inconsistent-
it may run very FAST during certain points but then it runs VERY slow
at other moments (especially when it's getting to the end of the table:

after 12,000 records). I have an idea of the speed because I have a counter

being
displayed in the form (with an Application.DoEvents()). I've eliminated the
DoEvents but it's still inconsistent.

This is a new version of an application that was previously made in MS
C/C++. The only differences between my C# app and the previous
version (besides the language it was written in) is that I had to recreate the
struct in C# (which the external method receives as parameter and

modifies)
and that I use the DLLImport for the method. Would this be a reason
why it's
so slow? Also, the memory usage (in Task Manager) is at 100% when

running.
Thanks,
VM



Nov 16 '05 #4

P: n/a
VM
The code's really short...and really easy. And this particular run takes
over 30 minutes and the speed is very inconsistent. The only long code is
the zip4Parm struct (I didn't post it) that contains over 60 fields and
they're all modified during the z4adrinq call. That's why I assumed it could
be the DLL call and the amount of time it takes to fill the struct:

[DllImport("ZIP4_W32.DLL")]
public static extern int z4adrinq([In,
Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
private void btn_run_Click(object sender, System.EventArgs e)
{
......
for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
{
Application.DoEvents();
sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
ZIP4_PARM zip4Parm = new ZIP4_PARM();
zip4Parm.iprurb = sFBU;
zip4Parm.iadl1 = sDEL;
zip4Parm.ictyi = sCTY;
int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
fields are modified
statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString() + "
rows ";
iRecCount++;
}
}

Thanks again,
VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#R**************@TK2MSFTNGP12.phx.gbl...
I guess that depends on the Interops in question!

If you think it through then some Interop operations could certainly be
tuned - for example - if you've got a 100MB of string data and you're
passing it through Interop then the following would be true:

1. .Net stores it's string as Unicode
2. Calling an ANSI Interop function would require converting those
strings to ANSI to and from the Interop call
3. That's got to be slower than calling a UNICODE Interop function

I think the example above shows how Interop could be slower than the raw C++ but there are so many other factors involved. In my experience of calls via Interop I haven't noticed any particular slow down so the profiler would
certainly identify which bit of code is causing the problem.

It could also be a GC issue - why don't you post the basic code and it may
be obvious (or probably won't be!)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:e6**************@TK2MSFTNGP09.phx.gbl...
Thanks for your reply. Would interops naturally make the system slower or
would it be something with my code? In other words, can the speed be
improved or is it a dot net limitation?

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#x*************@tk2msftngp13.phx.gbl...
Can I suggest you hook up a profiler to see where the problem lies...

Chances are it's a Interop problem but difficult to tell without the source
code etc...

There are some very good profilers out there and most have a 30 day

trial :-)

cheers,

g

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@tk2msftngp13.phx.gbl...
> In my Windows app, I'm running a batch process that's composed of a FOR loop
> that'll run 15,000 times (datatable row count), copy cthe data of each > row -3 fields- to a struct, and send the strct to an external method
(using
> DLLImport). The problem is that the processing speed is very
inconsistent-
> it may run very FAST during certain points but then it runs VERY slow at
> other moments (especially when it's getting to the end of the table:

after
> 12,000 records). I have an idea of the speed because I have a
counter being
> displayed in the form (with an Application.DoEvents()). I've

eliminated the
> DoEvents but it's still inconsistent.
>
> This is a new version of an application that was previously made in MS > C/C++. The only differences between my C# app and the previous version > (besides the language it was written in) is that I had to recreate the > struct in C# (which the external method receives as parameter and
modifies)
> and that I use the DLLImport for the method. Would this be a reason why it's
> so slow? Also, the memory usage (in Task Manager) is at 100% when

running.
>
> Thanks,
> VM
>
>



Nov 16 '05 #5

P: n/a
OK - a couple of good practices things to point out first

I know the compiler may pick these up but better with than without.

Accessing a row by index is probably not the fastest way so get yourself a
reference to the row and use that rather than indexing into the table each
time

e.g.
DataRow row = Datatable_AuditList.Rows[i];

Next one up - if you know that the row.Item[] is a string then cast is
rather than using ToString() - ToString has greater overhead so you end up
with.

sFBU = (string)row["FBU"];

And back to the first one - why not use a foreach(...) loop rather than by
index? or is there a specific requirement to go through like that?

All of the above are more than likely negligible in terms of performance -
rem out the interop call and see how long it takes - I'm guessing fractions
of a second.

Which leads us to believe that it must be the marshalling that is causing
the problem.

Could you post the zip4Parm structure as that's probably where we make the
changes - as you're passing in a LPStruct then I'm assuming that the
DllImport statement doesn't need to specify Unicode or ANSI.

cheers,

g

PS Spent the weekend doing Interop so lets get to the bottom of this one - I
got beat by .Net as the code I was up against required __cdecl callbacks via
delegates which aren't supported without an IL tweak using ILDASM... :-(
"VM" <vo******@yahoo.com> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
The code's really short...and really easy. And this particular run takes
over 30 minutes and the speed is very inconsistent. The only long code is
the zip4Parm struct (I didn't post it) that contains over 60 fields and
they're all modified during the z4adrinq call. That's why I assumed it could be the DLL call and the amount of time it takes to fill the struct:

[DllImport("ZIP4_W32.DLL")]
public static extern int z4adrinq([In,
Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
private void btn_run_Click(object sender, System.EventArgs e)
{
.....
for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
{
Application.DoEvents();
sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
ZIP4_PARM zip4Parm = new ZIP4_PARM();
zip4Parm.iprurb = sFBU;
zip4Parm.iadl1 = sDEL;
zip4Parm.ictyi = sCTY;
int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
fields are modified
statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString() + " rows ";
iRecCount++;
}
}

Thanks again,
VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#R**************@TK2MSFTNGP12.phx.gbl...
I guess that depends on the Interops in question!

If you think it through then some Interop operations could certainly be
tuned - for example - if you've got a 100MB of string data and you're
passing it through Interop then the following would be true:

1. .Net stores it's string as Unicode
2. Calling an ANSI Interop function would require converting those
strings to ANSI to and from the Interop call
3. That's got to be slower than calling a UNICODE Interop function

I think the example above shows how Interop could be slower than the raw C++
but there are so many other factors involved. In my experience of calls

via
Interop I haven't noticed any particular slow down so the profiler would
certainly identify which bit of code is causing the problem.

It could also be a GC issue - why don't you post the basic code and it may
be obvious (or probably won't be!)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:e6**************@TK2MSFTNGP09.phx.gbl...
Thanks for your reply. Would interops naturally make the system slower or would it be something with my code? In other words, can the speed be
improved or is it a dot net limitation?

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#x*************@tk2msftngp13.phx.gbl...
> Can I suggest you hook up a profiler to see where the problem lies... >
> Chances are it's a Interop problem but difficult to tell without the
source
> code etc...
>
> There are some very good profilers out there and most have a 30 day

trial
> :-)
>
> cheers,
>
> g
>
> "VM" <vo******@yahoo.com> wrote in message
> news:%2****************@tk2msftngp13.phx.gbl...
> > In my Windows app, I'm running a batch process that's composed of a FOR
> loop
> > that'll run 15,000 times (datatable row count), copy cthe data of each > > row -3 fields- to a struct, and send the strct to an external
method > (using
> > DLLImport). The problem is that the processing speed is very
> inconsistent-
> > it may run very FAST during certain points but then it runs VERY slow
at
> > other moments (especially when it's getting to the end of the table: after
> > 12,000 records). I have an idea of the speed because I have a

counter > being
> > displayed in the form (with an Application.DoEvents()). I've

eliminated
> the
> > DoEvents but it's still inconsistent.
> >
> > This is a new version of an application that was previously made
in MS > > C/C++. The only differences between my C# app and the previous

version
> > (besides the language it was written in) is that I had to recreate the > > struct in C# (which the external method receives as parameter and
> modifies)
> > and that I use the DLLImport for the method. Would this be a

reason why
> it's
> > so slow? Also, the memory usage (in Task Manager) is at 100% when
running.
> >
> > Thanks,
> > VM
> >
> >
>
>



Nov 16 '05 #6

P: n/a
VM
Thanks for all your help. I just fixed all those details you told me about.
About the struct, here's a section of it. I'm not going to post the whole
thing because the rest is the same as this. In the C version of the
application, the struct was in a header file so they didn't have to do any
marshalling. Also, during the run I noticed that the loop may be in a
particular record for longer (maybe a second or more). Basically, the
extern method finds information based on the data I copied to the ZIP4_PARM
struct and writes it to the struct in other empty fields when it finds is.
It's similar to a database :

namespace ZipMaster
{
....
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public class ZIP4_PARM
{
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
public string rsvd0;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl1;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl2;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string ictyi;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
public string istai;
public string county;
public short respn;
public char retcc;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
public string adrkey;
public char auto_zone_ind;
public struct footer
{
public char a;
public char b;
public char c;
public char d;
public char e;
}
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
public string rsvd3;
}
}

Thanks again,
VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:eF**************@TK2MSFTNGP11.phx.gbl...
OK - a couple of good practices things to point out first

I know the compiler may pick these up but better with than without.

Accessing a row by index is probably not the fastest way so get yourself a
reference to the row and use that rather than indexing into the table each
time

e.g.
DataRow row = Datatable_AuditList.Rows[i];

Next one up - if you know that the row.Item[] is a string then cast is
rather than using ToString() - ToString has greater overhead so you end up
with.

sFBU = (string)row["FBU"];

And back to the first one - why not use a foreach(...) loop rather than by
index? or is there a specific requirement to go through like that?

All of the above are more than likely negligible in terms of performance -
rem out the interop call and see how long it takes - I'm guessing fractions of a second.

Which leads us to believe that it must be the marshalling that is causing
the problem.

Could you post the zip4Parm structure as that's probably where we make the
changes - as you're passing in a LPStruct then I'm assuming that the
DllImport statement doesn't need to specify Unicode or ANSI.

cheers,

g

PS Spent the weekend doing Interop so lets get to the bottom of this one - I got beat by .Net as the code I was up against required __cdecl callbacks via delegates which aren't supported without an IL tweak using ILDASM... :-(
"VM" <vo******@yahoo.com> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
The code's really short...and really easy. And this particular run takes
over 30 minutes and the speed is very inconsistent. The only long code is
the zip4Parm struct (I didn't post it) that contains over 60 fields and
they're all modified during the z4adrinq call. That's why I assumed it could
be the DLL call and the amount of time it takes to fill the struct:

[DllImport("ZIP4_W32.DLL")]
public static extern int z4adrinq([In,
Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
private void btn_run_Click(object sender, System.EventArgs e)
{
.....
for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
{
Application.DoEvents();
sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
ZIP4_PARM zip4Parm = new ZIP4_PARM();
zip4Parm.iprurb = sFBU;
zip4Parm.iadl1 = sDEL;
zip4Parm.ictyi = sCTY;
int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
fields are modified
statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString() + "
rows ";
iRecCount++;
}
}

Thanks again,
VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#R**************@TK2MSFTNGP12.phx.gbl...
I guess that depends on the Interops in question!

If you think it through then some Interop operations could certainly be tuned - for example - if you've got a 100MB of string data and you're
passing it through Interop then the following would be true:

1. .Net stores it's string as Unicode
2. Calling an ANSI Interop function would require converting those
strings to ANSI to and from the Interop call
3. That's got to be slower than calling a UNICODE Interop function

I think the example above shows how Interop could be slower than the raw C++
but there are so many other factors involved. In my experience of
calls via
Interop I haven't noticed any particular slow down so the profiler wou
ld certainly identify which bit of code is causing the problem.

It could also be a GC issue - why don't you post the basic code and it
may be obvious (or probably won't be!)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:e6**************@TK2MSFTNGP09.phx.gbl...
> Thanks for your reply. Would interops naturally make the system slower or
> would it be something with my code? In other words, can the speed be
> improved or is it a dot net limitation?
>
> VM
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:#x*************@tk2msftngp13.phx.gbl...
> > Can I suggest you hook up a profiler to see where the problem lies... > >
> > Chances are it's a Interop problem but difficult to tell without
the > source
> > code etc...
> >
> > There are some very good profilers out there and most have a 30 day trial
> > :-)
> >
> > cheers,
> >
> > g
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:%2****************@tk2msftngp13.phx.gbl...
> > > In my Windows app, I'm running a batch process that's composed of a FOR
> > loop
> > > that'll run 15,000 times (datatable row count), copy cthe data
of
each
> > > row -3 fields- to a struct, and send the strct to an external method > > (using
> > > DLLImport). The problem is that the processing speed is very
> > inconsistent-
> > > it may run very FAST during certain points but then it runs VERY

slow
at
> > > other moments (especially when it's getting to the end of the table: > after
> > > 12,000 records). I have an idea of the speed because I have a

counter
> > being
> > > displayed in the form (with an Application.DoEvents()). I've
eliminated
> > the
> > > DoEvents but it's still inconsistent.
> > >
> > > This is a new version of an application that was previously made in
MS
> > > C/C++. The only differences between my C# app and the previous
version
> > > (besides the language it was written in) is that I had to

recreate the
> > > struct in C# (which the external method receives as parameter

and > > modifies)
> > > and that I use the DLLImport for the method. Would this be a

reason why
> > it's
> > > so slow? Also, the memory usage (in Task Manager) is at 100% when > running.
> > >
> > > Thanks,
> > > VM
> > >
> > >
> >
> >
>
>



Nov 16 '05 #7

P: n/a
Did setting ANSI vs Unicode make any differences?

g
"VM" <vo******@yahoo.com> wrote in message
news:OS*************@TK2MSFTNGP10.phx.gbl...
Thanks for all your help. I just fixed all those details you told me about. About the struct, here's a section of it. I'm not going to post the whole
thing because the rest is the same as this. In the C version of the
application, the struct was in a header file so they didn't have to do any
marshalling. Also, during the run I noticed that the loop may be in a
particular record for longer (maybe a second or more). Basically, the
extern method finds information based on the data I copied to the ZIP4_PARM struct and writes it to the struct in other empty fields when it finds is.
It's similar to a database :

namespace ZipMaster
{
...
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public class ZIP4_PARM
{
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
public string rsvd0;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl1;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl2;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string ictyi;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
public string istai;
public string county;
public short respn;
public char retcc;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
public string adrkey;
public char auto_zone_ind;
public struct footer
{
public char a;
public char b;
public char c;
public char d;
public char e;
}
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
public string rsvd3;
}
}

Thanks again,
VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:eF**************@TK2MSFTNGP11.phx.gbl...
OK - a couple of good practices things to point out first

I know the compiler may pick these up but better with than without.

Accessing a row by index is probably not the fastest way so get yourself a
reference to the row and use that rather than indexing into the table each time

e.g.
DataRow row = Datatable_AuditList.Rows[i];

Next one up - if you know that the row.Item[] is a string then cast is
rather than using ToString() - ToString has greater overhead so you end up with.

sFBU = (string)row["FBU"];

And back to the first one - why not use a foreach(...) loop rather than by index? or is there a specific requirement to go through like that?

All of the above are more than likely negligible in terms of performance - rem out the interop call and see how long it takes - I'm guessing fractions
of a second.

Which leads us to believe that it must be the marshalling that is causing the problem.

Could you post the zip4Parm structure as that's probably where we make the changes - as you're passing in a LPStruct then I'm assuming that the
DllImport statement doesn't need to specify Unicode or ANSI.

cheers,

g

PS Spent the weekend doing Interop so lets get to the bottom of this one - I
got beat by .Net as the code I was up against required __cdecl callbacks via
delegates which aren't supported without an IL tweak using ILDASM... :-(
"VM" <vo******@yahoo.com> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
The code's really short...and really easy. And this particular run takes over 30 minutes and the speed is very inconsistent. The only long code is the zip4Parm struct (I didn't post it) that contains over 60 fields
and they're all modified during the z4adrinq call. That's why I assumed it could
be the DLL call and the amount of time it takes to fill the struct:

[DllImport("ZIP4_W32.DLL")]
public static extern int z4adrinq([In,
Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
private void btn_run_Click(object sender, System.EventArgs e)
{
.....
for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000 {
Application.DoEvents();
sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
ZIP4_PARM zip4Parm = new ZIP4_PARM();
zip4Parm.iprurb = sFBU;
zip4Parm.iadl1 = sDEL;
zip4Parm.ictyi = sCTY;
int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
fields are modified
statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString() +
"
rows ";
iRecCount++;
}
}

Thanks again,
VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#R**************@TK2MSFTNGP12.phx.gbl...
> I guess that depends on the Interops in question!
>
> If you think it through then some Interop operations could certainly
be > tuned - for example - if you've got a 100MB of string data and
you're > passing it through Interop then the following would be true:
>
> 1. .Net stores it's string as Unicode
> 2. Calling an ANSI Interop function would require converting those > strings to ANSI to and from the Interop call
> 3. That's got to be slower than calling a UNICODE Interop function >
> I think the example above shows how Interop could be slower than the

raw C++
> but there are so many other factors involved. In my experience of calls via
> Interop I haven't noticed any particular slow down so the profiler wou ld
> certainly identify which bit of code is causing the problem.
>
> It could also be a GC issue - why don't you post the basic code and
it may
> be obvious (or probably won't be!)
>
> cheers,
>
> g
>
>
> "VM" <vo******@yahoo.com> wrote in message
> news:e6**************@TK2MSFTNGP09.phx.gbl...
> > Thanks for your reply. Would interops naturally make the system slower or
> > would it be something with my code? In other words, can the speed
be > > improved or is it a dot net limitation?
> >
> > VM
> >
> > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > news:#x*************@tk2msftngp13.phx.gbl...
> > > Can I suggest you hook up a profiler to see where the problem

lies...
> > >
> > > Chances are it's a Interop problem but difficult to tell without

the > > source
> > > code etc...
> > >
> > > There are some very good profilers out there and most have a 30 day > trial
> > > :-)
> > >
> > > cheers,
> > >
> > > g
> > >
> > > "VM" <vo******@yahoo.com> wrote in message
> > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > In my Windows app, I'm running a batch process that's composed of
a
> FOR
> > > loop
> > > > that'll run 15,000 times (datatable row count), copy cthe data

of each
> > > > row -3 fields- to a struct, and send the strct to an external

method
> > > (using
> > > > DLLImport). The problem is that the processing speed is very
> > > inconsistent-
> > > > it may run very FAST during certain points but then it runs VERY slow
> at
> > > > other moments (especially when it's getting to the end of the

table:
> > after
> > > > 12,000 records). I have an idea of the speed because I have a
counter
> > > being
> > > > displayed in the form (with an Application.DoEvents()). I've
> eliminated
> > > the
> > > > DoEvents but it's still inconsistent.
> > > >
> > > > This is a new version of an application that was previously made in
MS
> > > > C/C++. The only differences between my C# app and the
previous > version
> > > > (besides the language it was written in) is that I had to

recreate the
> > > > struct in C# (which the external method receives as parameter and > > > modifies)
> > > > and that I use the DLLImport for the method. Would this be a

reason
> why
> > > it's
> > > > so slow? Also, the memory usage (in Task Manager) is at 100% when > > running.
> > > >
> > > > Thanks,
> > > > VM
> > > >
> > > >
> > >
> > >
> >
> >
>
>



Nov 16 '05 #8

P: n/a
In addition to keep a reference to the row (instead of indexing each time),
you shouldn't do a string based column lookup each time either (["DEL"]).
The datatable can give you the column (or you can just use the index). Then
you can pass the column or index -- that should be faster too.

Also, you don't need to update the rowcount *every* time, do you? I'd
suggest doing this work on another thread, and then every second or so, have
your UI thread grab the completed rowcount and display it.

I think what you need to do is figure out where the slowdown is (obviously
:)). See if you can narrow it down to a particular set of rows. You
mentioned that near the end of the table, it goes slow. What happens if you
run it only on the last 5000 rows -- does it still go slow? A profiler
definately should tell you where the memory is going. If the memory grows,
and then hits a cieling, and things slow down, perhaps memory isn't being
freed somewhere (in unmanaged code perhaps)?

You *could* try a GC.Collect(2) each 5000 rows and see if that has any
effect (it'd only matter if the GC was getting messed up for some reason).

-mike
MVP

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
The code's really short...and really easy. And this particular run takes
over 30 minutes and the speed is very inconsistent. The only long code is
the zip4Parm struct (I didn't post it) that contains over 60 fields and
they're all modified during the z4adrinq call. That's why I assumed it
could
be the DLL call and the amount of time it takes to fill the struct:

[DllImport("ZIP4_W32.DLL")]
public static extern int z4adrinq([In,
Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
private void btn_run_Click(object sender, System.EventArgs e)
{
.....
for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
{
Application.DoEvents();
sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
ZIP4_PARM zip4Parm = new ZIP4_PARM();
zip4Parm.iprurb = sFBU;
zip4Parm.iadl1 = sDEL;
zip4Parm.ictyi = sCTY;
int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
fields are modified
statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString() +
"
rows ";
iRecCount++;
}
}

Thanks again,
VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#R**************@TK2MSFTNGP12.phx.gbl...
I guess that depends on the Interops in question!

If you think it through then some Interop operations could certainly be
tuned - for example - if you've got a 100MB of string data and you're
passing it through Interop then the following would be true:

1. .Net stores it's string as Unicode
2. Calling an ANSI Interop function would require converting those
strings to ANSI to and from the Interop call
3. That's got to be slower than calling a UNICODE Interop function

I think the example above shows how Interop could be slower than the raw

C++
but there are so many other factors involved. In my experience of calls

via
Interop I haven't noticed any particular slow down so the profiler would
certainly identify which bit of code is causing the problem.

It could also be a GC issue - why don't you post the basic code and it
may
be obvious (or probably won't be!)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:e6**************@TK2MSFTNGP09.phx.gbl...
> Thanks for your reply. Would interops naturally make the system slower or > would it be something with my code? In other words, can the speed be
> improved or is it a dot net limitation?
>
> VM
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:#x*************@tk2msftngp13.phx.gbl...
> > Can I suggest you hook up a profiler to see where the problem lies...
> >
> > Chances are it's a Interop problem but difficult to tell without the
> source
> > code etc...
> >
> > There are some very good profilers out there and most have a 30 day

trial
> > :-)
> >
> > cheers,
> >
> > g
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:%2****************@tk2msftngp13.phx.gbl...
> > > In my Windows app, I'm running a batch process that's composed of a

FOR
> > loop
> > > that'll run 15,000 times (datatable row count), copy cthe data of each > > > row -3 fields- to a struct, and send the strct to an external
> > > method
> > (using
> > > DLLImport). The problem is that the processing speed is very
> > inconsistent-
> > > it may run very FAST during certain points but then it runs VERY slow
at
> > > other moments (especially when it's getting to the end of the
> > > table:
> after
> > > 12,000 records). I have an idea of the speed because I have a

counter > > being
> > > displayed in the form (with an Application.DoEvents()). I've

eliminated
> > the
> > > DoEvents but it's still inconsistent.
> > >
> > > This is a new version of an application that was previously made in MS > > > C/C++. The only differences between my C# app and the previous

version
> > > (besides the language it was written in) is that I had to recreate the > > > struct in C# (which the external method receives as parameter and
> > modifies)
> > > and that I use the DLLImport for the method. Would this be a reason

why
> > it's
> > > so slow? Also, the memory usage (in Task Manager) is at 100% when
> running.
> > >
> > > Thanks,
> > > VM
> > >
> > >
> >
> >
>
>




Nov 16 '05 #9

P: n/a
VM
For 1,000 records, the difference was 66 sec. vs. 130 secs. which is pretty
good. I haven't tried it with the 15,000 records yet. But the old system did
the 15,000 records in just under 4 mins.

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:en**************@TK2MSFTNGP12.phx.gbl...
Did setting ANSI vs Unicode make any differences?

g
"VM" <vo******@yahoo.com> wrote in message
news:OS*************@TK2MSFTNGP10.phx.gbl...
Thanks for all your help. I just fixed all those details you told me about.
About the struct, here's a section of it. I'm not going to post the whole
thing because the rest is the same as this. In the C version of the
application, the struct was in a header file so they didn't have to do any marshalling. Also, during the run I noticed that the loop may be in a
particular record for longer (maybe a second or more). Basically, the
extern method finds information based on the data I copied to the

ZIP4_PARM
struct and writes it to the struct in other empty fields when it finds is. It's similar to a database :

namespace ZipMaster
{
...
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public class ZIP4_PARM
{
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
public string rsvd0;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl1;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl2;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string ictyi;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
public string istai;
public string county;
public short respn;
public char retcc;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
public string adrkey;
public char auto_zone_ind;
public struct footer
{
public char a;
public char b;
public char c;
public char d;
public char e;
}
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
public string rsvd3;
}
}

Thanks again,
VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:eF**************@TK2MSFTNGP11.phx.gbl...
OK - a couple of good practices things to point out first

I know the compiler may pick these up but better with than without.

Accessing a row by index is probably not the fastest way so get yourself a reference to the row and use that rather than indexing into the table each time

e.g.
DataRow row = Datatable_AuditList.Rows[i];

Next one up - if you know that the row.Item[] is a string then cast is
rather than using ToString() - ToString has greater overhead so you
end
up with.

sFBU = (string)row["FBU"];

And back to the first one - why not use a foreach(...) loop rather
than
by index? or is there a specific requirement to go through like that?

All of the above are more than likely negligible in terms of performance - rem out the interop call and see how long it takes - I'm guessing fractions
of a second.

Which leads us to believe that it must be the marshalling that is causing the problem.

Could you post the zip4Parm structure as that's probably where we make the changes - as you're passing in a LPStruct then I'm assuming that the
DllImport statement doesn't need to specify Unicode or ANSI.

cheers,

g

PS Spent the weekend doing Interop so lets get to the bottom of this one -
I
got beat by .Net as the code I was up against required __cdecl
callbacks via
delegates which aren't supported without an IL tweak using ILDASM...
:-(

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
> The code's really short...and really easy. And this particular run

takes > over 30 minutes and the speed is very inconsistent. The only long code
is
> the zip4Parm struct (I didn't post it) that contains over 60 fields

and > they're all modified during the z4adrinq call. That's why I assumed it could
> be the DLL call and the amount of time it takes to fill the struct:
>
> [DllImport("ZIP4_W32.DLL")]
> public static extern int z4adrinq([In,
> Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
> private void btn_run_Click(object sender, System.EventArgs e)
> {
> .....
> for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000 > {
> Application.DoEvents();
> sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
> sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
> sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
> sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
> ZIP4_PARM zip4Parm = new ZIP4_PARM();
> zip4Parm.iprurb = sFBU;
> zip4Parm.iadl1 = sDEL;
> zip4Parm.ictyi = sCTY;
> int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+ > fields are modified
> statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString()
+
"
> rows ";
> iRecCount++;
> }
> }
>
> Thanks again,
> VM
>
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:#R**************@TK2MSFTNGP12.phx.gbl...
> > I guess that depends on the Interops in question!
> >
> > If you think it through then some Interop operations could certainly be
> > tuned - for example - if you've got a 100MB of string data and you're > > passing it through Interop then the following would be true:
> >
> > 1. .Net stores it's string as Unicode
> > 2. Calling an ANSI Interop function would require converting those > > strings to ANSI to and from the Interop call
> > 3. That's got to be slower than calling a UNICODE Interop function > >
> > I think the example above shows how Interop could be slower than
the raw
> C++
> > but there are so many other factors involved. In my experience of

calls
> via
> > Interop I haven't noticed any particular slow down so the profiler wou
ld
> > certainly identify which bit of code is causing the problem.
> >
> > It could also be a GC issue - why don't you post the basic code

and it may
> > be obvious (or probably won't be!)
> >
> > cheers,
> >
> > g
> >
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:e6**************@TK2MSFTNGP09.phx.gbl...
> > > Thanks for your reply. Would interops naturally make the system slower
> or
> > > would it be something with my code? In other words, can the
speed be > > > improved or is it a dot net limitation?
> > >
> > > VM
> > >
> > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > news:#x*************@tk2msftngp13.phx.gbl...
> > > > Can I suggest you hook up a profiler to see where the problem
lies...
> > > >
> > > > Chances are it's a Interop problem but difficult to tell
without the
> > > source
> > > > code etc...
> > > >
> > > > There are some very good profilers out there and most have a
30 day
> > trial
> > > > :-)
> > > >
> > > > cheers,
> > > >
> > > > g
> > > >
> > > > "VM" <vo******@yahoo.com> wrote in message
> > > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > > In my Windows app, I'm running a batch process that's
composed of
a
> > FOR
> > > > loop
> > > > > that'll run 15,000 times (datatable row count), copy cthe
data of
> each
> > > > > row -3 fields- to a struct, and send the strct to an
external method
> > > > (using
> > > > > DLLImport). The problem is that the processing speed is very > > > > inconsistent-
> > > > > it may run very FAST during certain points but then it runs

VERY > slow
> > at
> > > > > other moments (especially when it's getting to the end of the table:
> > > after
> > > > > 12,000 records). I have an idea of the speed because I have a > counter
> > > > being
> > > > > displayed in the form (with an Application.DoEvents()). I've
> > eliminated
> > > > the
> > > > > DoEvents but it's still inconsistent.
> > > > >
> > > > > This is a new version of an application that was previously made in
> MS
> > > > > C/C++. The only differences between my C# app and the previous > > version
> > > > > (besides the language it was written in) is that I had to

recreate
> the
> > > > > struct in C# (which the external method receives as

parameter and
> > > > modifies)
> > > > > and that I use the DLLImport for the method. Would this be a
reason
> > why
> > > > it's
> > > > > so slow? Also, the memory usage (in Task Manager) is at 100%

when
> > > running.
> > > > >
> > > > > Thanks,
> > > > > VM
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>



Nov 16 '05 #10

P: n/a
VM
I ran it with 15,000 records and the speed difference was incredible. It
processed them in 2 mins. The only problem I seem to ba having is that the
the fields that are supposed to have data have strange characters in them.
Is it possible that this change (ansi to unicode) affects the way the
external method writes to the struct?

Thanks again,
VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:en**************@TK2MSFTNGP12.phx.gbl...
Did setting ANSI vs Unicode make any differences?

g
"VM" <vo******@yahoo.com> wrote in message
news:OS*************@TK2MSFTNGP10.phx.gbl...
Thanks for all your help. I just fixed all those details you told me about.
About the struct, here's a section of it. I'm not going to post the whole
thing because the rest is the same as this. In the C version of the
application, the struct was in a header file so they didn't have to do any marshalling. Also, during the run I noticed that the loop may be in a
particular record for longer (maybe a second or more). Basically, the
extern method finds information based on the data I copied to the

ZIP4_PARM
struct and writes it to the struct in other empty fields when it finds is. It's similar to a database :

namespace ZipMaster
{
...
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public class ZIP4_PARM
{
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
public string rsvd0;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl1;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl2;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string ictyi;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
public string istai;
public string county;
public short respn;
public char retcc;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
public string adrkey;
public char auto_zone_ind;
public struct footer
{
public char a;
public char b;
public char c;
public char d;
public char e;
}
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
public string rsvd3;
}
}

Thanks again,
VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:eF**************@TK2MSFTNGP11.phx.gbl...
OK - a couple of good practices things to point out first

I know the compiler may pick these up but better with than without.

Accessing a row by index is probably not the fastest way so get yourself a reference to the row and use that rather than indexing into the table each time

e.g.
DataRow row = Datatable_AuditList.Rows[i];

Next one up - if you know that the row.Item[] is a string then cast is
rather than using ToString() - ToString has greater overhead so you
end
up with.

sFBU = (string)row["FBU"];

And back to the first one - why not use a foreach(...) loop rather
than
by index? or is there a specific requirement to go through like that?

All of the above are more than likely negligible in terms of performance - rem out the interop call and see how long it takes - I'm guessing fractions
of a second.

Which leads us to believe that it must be the marshalling that is causing the problem.

Could you post the zip4Parm structure as that's probably where we make the changes - as you're passing in a LPStruct then I'm assuming that the
DllImport statement doesn't need to specify Unicode or ANSI.

cheers,

g

PS Spent the weekend doing Interop so lets get to the bottom of this one -
I
got beat by .Net as the code I was up against required __cdecl
callbacks via
delegates which aren't supported without an IL tweak using ILDASM...
:-(

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
> The code's really short...and really easy. And this particular run

takes > over 30 minutes and the speed is very inconsistent. The only long code
is
> the zip4Parm struct (I didn't post it) that contains over 60 fields

and > they're all modified during the z4adrinq call. That's why I assumed it could
> be the DLL call and the amount of time it takes to fill the struct:
>
> [DllImport("ZIP4_W32.DLL")]
> public static extern int z4adrinq([In,
> Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
> private void btn_run_Click(object sender, System.EventArgs e)
> {
> .....
> for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000 > {
> Application.DoEvents();
> sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
> sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
> sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
> sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
> ZIP4_PARM zip4Parm = new ZIP4_PARM();
> zip4Parm.iprurb = sFBU;
> zip4Parm.iadl1 = sDEL;
> zip4Parm.ictyi = sCTY;
> int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+ > fields are modified
> statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString()
+
"
> rows ";
> iRecCount++;
> }
> }
>
> Thanks again,
> VM
>
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:#R**************@TK2MSFTNGP12.phx.gbl...
> > I guess that depends on the Interops in question!
> >
> > If you think it through then some Interop operations could certainly be
> > tuned - for example - if you've got a 100MB of string data and you're > > passing it through Interop then the following would be true:
> >
> > 1. .Net stores it's string as Unicode
> > 2. Calling an ANSI Interop function would require converting those > > strings to ANSI to and from the Interop call
> > 3. That's got to be slower than calling a UNICODE Interop function > >
> > I think the example above shows how Interop could be slower than
the raw
> C++
> > but there are so many other factors involved. In my experience of

calls
> via
> > Interop I haven't noticed any particular slow down so the profiler wou
ld
> > certainly identify which bit of code is causing the problem.
> >
> > It could also be a GC issue - why don't you post the basic code

and it may
> > be obvious (or probably won't be!)
> >
> > cheers,
> >
> > g
> >
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:e6**************@TK2MSFTNGP09.phx.gbl...
> > > Thanks for your reply. Would interops naturally make the system slower
> or
> > > would it be something with my code? In other words, can the
speed be > > > improved or is it a dot net limitation?
> > >
> > > VM
> > >
> > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > news:#x*************@tk2msftngp13.phx.gbl...
> > > > Can I suggest you hook up a profiler to see where the problem
lies...
> > > >
> > > > Chances are it's a Interop problem but difficult to tell
without the
> > > source
> > > > code etc...
> > > >
> > > > There are some very good profilers out there and most have a
30 day
> > trial
> > > > :-)
> > > >
> > > > cheers,
> > > >
> > > > g
> > > >
> > > > "VM" <vo******@yahoo.com> wrote in message
> > > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > > In my Windows app, I'm running a batch process that's
composed of
a
> > FOR
> > > > loop
> > > > > that'll run 15,000 times (datatable row count), copy cthe
data of
> each
> > > > > row -3 fields- to a struct, and send the strct to an
external method
> > > > (using
> > > > > DLLImport). The problem is that the processing speed is very > > > > inconsistent-
> > > > > it may run very FAST during certain points but then it runs

VERY > slow
> > at
> > > > > other moments (especially when it's getting to the end of the table:
> > > after
> > > > > 12,000 records). I have an idea of the speed because I have a > counter
> > > > being
> > > > > displayed in the form (with an Application.DoEvents()). I've
> > eliminated
> > > > the
> > > > > DoEvents but it's still inconsistent.
> > > > >
> > > > > This is a new version of an application that was previously made in
> MS
> > > > > C/C++. The only differences between my C# app and the previous > > version
> > > > > (besides the language it was written in) is that I had to

recreate
> the
> > > > > struct in C# (which the external method receives as

parameter and
> > > > modifies)
> > > > > and that I use the DLLImport for the method. Would this be a
reason
> > why
> > > > it's
> > > > > so slow? Also, the memory usage (in Task Manager) is at 100%

when
> > > running.
> > > > >
> > > > > Thanks,
> > > > > VM
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>



Nov 16 '05 #11

P: n/a

I would do something like this. This is not compiled and tested.

Call the DoWork function via worker thread or spawn separate thread.

....

delegate void GUIUpdater(int recCount);

private void UpdateGUI(int recCount)
{
statusBar.Panels[0].Text = "Row Count = " + recCount.ToString() + " rows";
}

....

void DoWork()
{
DataColumn colFBU = Datatable_AuditList.Columns["FBU"];
DataColumn colDEL = Datatable_AuditList.Columns["DEL"];
DataColumn colCTY = Datatable_AuditList.Columns["CTY"];
int iError = 0;
int iCount = 0;
foreach(DataRow row in Datatable_AuditList.Rows)
{
ZIP4_PARM zip4Parm = new ZIP4_PARM();
zip4Parm.iprurb = row[colFBU] as string;
zip4Parm.iadl1 = row[colDEL] as string;
zip4Parm.ictyi = row[colCTY] as string;
iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
++iCount;
if (iCount % 200 == 0)
this.Invoke(new GUIUpdater(this.UpdateGUI), new object[] { iCount});
}
}

This removes the most obvious bottlenecks on the C# part of the loop.

I have found that DoEvents and updating gui can be slow at times.

Moving it to a separate thread and using the forms Invoke method with
a delegate that updates the GUI is usually the fastest way to update the
gui and the rest overhead should be in the interop part ;)

You also could wrap the z4adrinq function in your own managed dll and
only ship the parameters needed if the marshalling copying of the 60+
output variables in the zip4Parm isn't in use. From the snippet it looks
like z4adrinq has a side effect that your after.

Well some tips that could be worth something ;)

Regards
/Michel
VM wrote:
The code's really short...and really easy. And this particular run takes
over 30 minutes and the speed is very inconsistent. The only long code is
the zip4Parm struct (I didn't post it) that contains over 60 fields and
they're all modified during the z4adrinq call. That's why I assumed it could
be the DLL call and the amount of time it takes to fill the struct:

[DllImport("ZIP4_W32.DLL")]
public static extern int z4adrinq([In,
Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
private void btn_run_Click(object sender, System.EventArgs e)
{
.....
for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
{
Application.DoEvents();
sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
ZIP4_PARM zip4Parm = new ZIP4_PARM();
zip4Parm.iprurb = sFBU;
zip4Parm.iadl1 = sDEL;
zip4Parm.ictyi = sCTY;
int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
fields are modified
statusBar.Panels[0].Text = "Row Count = " + iRecCount.ToString() + "
rows ";
iRecCount++;
}
}

Thanks again,
VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#R**************@TK2MSFTNGP12.phx.gbl...
I guess that depends on the Interops in question!

If you think it through then some Interop operations could certainly be
tuned - for example - if you've got a 100MB of string data and you're
passing it through Interop then the following would be true:

1. .Net stores it's string as Unicode
2. Calling an ANSI Interop function would require converting those
strings to ANSI to and from the Interop call
3. That's got to be slower than calling a UNICODE Interop function

I think the example above shows how Interop could be slower than the raw


C++
but there are so many other factors involved. In my experience of calls


via
Interop I haven't noticed any particular slow down so the profiler would
certainly identify which bit of code is causing the problem.

It could also be a GC issue - why don't you post the basic code and it may
be obvious (or probably won't be!)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:e6**************@TK2MSFTNGP09.phx.gbl...
Thanks for your reply. Would interops naturally make the system slower
or
would it be something with my code? In other words, can the speed be
improved or is it a dot net limitation?

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:#x*************@tk2msftngp13.phx.gbl...

Can I suggest you hook up a profiler to see where the problem lies...

Chances are it's a Interop problem but difficult to tell without the

source

code etc...

There are some very good profilers out there and most have a 30 day


trial
:-)

cheers,

g

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@tk2msftngp13.phx.gbl.. .

>In my Windows app, I'm running a batch process that's composed of a


FOR
loop

>that'll run 15,000 times (datatable row count), copy cthe data of
each
row -3 fields- to a struct, and send the strct to an external method

(using

>DLLImport). The problem is that the processing speed is very

inconsistent-

>it may run very FAST during certain points but then it runs VERY
slow
at
>other moments (especially when it's getting to the end of the table:

after

>12,000 records). I have an idea of the speed because I have a


counter
being

>displayed in the form (with an Application.DoEvents()). I've


eliminated
the

>DoEvents but it's still inconsistent.
>
>This is a new version of an application that was previously made in
MS
C/C++. The only differences between my C# app and the previous


version
>(besides the language it was written in) is that I had to recreate
the
struct in C# (which the external method receives as parameter and

modifies)

>and that I use the DLLImport for the method. Would this be a reason


why
it's

>so slow? Also, the memory usage (in Task Manager) is at 100% when

running.

>Thanks,
>VM
>
>


Nov 16 '05 #12

P: n/a
Sorry - you need to find out which of ANSI vs Unicode the destination
function is written in and use the appropriate one.

(Do you know the difference? If not then ANSI is a single byte ber character
whereas Unicode takes up two bytes per character.)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:eS**************@TK2MSFTNGP12.phx.gbl...
For 1,000 records, the difference was 66 sec. vs. 130 secs. which is pretty good. I haven't tried it with the 15,000 records yet. But the old system did the 15,000 records in just under 4 mins.

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:en**************@TK2MSFTNGP12.phx.gbl...
Did setting ANSI vs Unicode make any differences?

g
"VM" <vo******@yahoo.com> wrote in message
news:OS*************@TK2MSFTNGP10.phx.gbl...
Thanks for all your help. I just fixed all those details you told me about.
About the struct, here's a section of it. I'm not going to post the whole thing because the rest is the same as this. In the C version of the
application, the struct was in a header file so they didn't have to do any marshalling. Also, during the run I noticed that the loop may be in a
particular record for longer (maybe a second or more). Basically, the
extern method finds information based on the data I copied to the

ZIP4_PARM
struct and writes it to the struct in other empty fields when it finds is. It's similar to a database :

namespace ZipMaster
{
...
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public class ZIP4_PARM
{
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
public string rsvd0;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl1;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string iadl2;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
public string ictyi;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
public string istai;
public string county;
public short respn;
public char retcc;
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
public string adrkey;
public char auto_zone_ind;
public struct footer
{
public char a;
public char b;
public char c;
public char d;
public char e;
}
[MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
public string rsvd3;
}
}

Thanks again,
VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:eF**************@TK2MSFTNGP11.phx.gbl...
> OK - a couple of good practices things to point out first
>
> I know the compiler may pick these up but better with than without.
>
> Accessing a row by index is probably not the fastest way so get yourself
a
> reference to the row and use that rather than indexing into the table
each
> time
>
> e.g.
> DataRow row = Datatable_AuditList.Rows[i];
>
> Next one up - if you know that the row.Item[] is a string then cast
is > rather than using ToString() - ToString has greater overhead so you

end
up
> with.
>
> sFBU = (string)row["FBU"];
>
> And back to the first one - why not use a foreach(...) loop rather

than
by
> index? or is there a specific requirement to go through like that?
>
> All of the above are more than likely negligible in terms of

performance -
> rem out the interop call and see how long it takes - I'm guessing
fractions
> of a second.
>
> Which leads us to believe that it must be the marshalling that is

causing
> the problem.
>
> Could you post the zip4Parm structure as that's probably where we make the
> changes - as you're passing in a LPStruct then I'm assuming that the
> DllImport statement doesn't need to specify Unicode or ANSI.
>
> cheers,
>
> g
>
> PS Spent the weekend doing Interop so lets get to the bottom of this

one -
I
> got beat by .Net as the code I was up against required __cdecl callbacks via
> delegates which aren't supported without an IL tweak using ILDASM... :-( >
>
> "VM" <vo******@yahoo.com> wrote in message
> news:%2****************@TK2MSFTNGP12.phx.gbl...
> > The code's really short...and really easy. And this particular run

takes
> > over 30 minutes and the speed is very inconsistent. The only long

code
is
> > the zip4Parm struct (I didn't post it) that contains over 60
fields and
> > they're all modified during the z4adrinq call. That's why I
assumed it > could
> > be the DLL call and the amount of time it takes to fill the
struct: > >
> > [DllImport("ZIP4_W32.DLL")]
> > public static extern int z4adrinq([In,
> > Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
> > private void btn_run_Click(object sender, System.EventArgs e)
> > {
> > .....
> > for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
> > {
> > Application.DoEvents();
> > sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
> > sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
> > sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
> > sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
> > ZIP4_PARM zip4Parm = new ZIP4_PARM();
> > zip4Parm.iprurb = sFBU;
> > zip4Parm.iadl1 = sDEL;
> > zip4Parm.ictyi = sCTY;
> > int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's

60+ > > fields are modified
> > statusBar.Panels[0].Text = "Row Count = " +

iRecCount.ToString()
+
> "
> > rows ";
> > iRecCount++;
> > }
> > }
> >
> > Thanks again,
> > VM
> >
> >
> > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > news:#R**************@TK2MSFTNGP12.phx.gbl...
> > > I guess that depends on the Interops in question!
> > >
> > > If you think it through then some Interop operations could certainly be
> > > tuned - for example - if you've got a 100MB of string data and

you're
> > > passing it through Interop then the following would be true:
> > >
> > > 1. .Net stores it's string as Unicode
> > > 2. Calling an ANSI Interop function would require converting

those
> > > strings to ANSI to and from the Interop call
> > > 3. That's got to be slower than calling a UNICODE Interop

function
> > >
> > > I think the example above shows how Interop could be slower than the raw
> > C++
> > > but there are so many other factors involved. In my experience of calls
> > via
> > > Interop I haven't noticed any particular slow down so the profiler wou
ld
> > > certainly identify which bit of code is causing the problem.
> > >
> > > It could also be a GC issue - why don't you post the basic code and
it
> may
> > > be obvious (or probably won't be!)
> > >
> > > cheers,
> > >
> > > g
> > >
> > >
> > > "VM" <vo******@yahoo.com> wrote in message
> > > news:e6**************@TK2MSFTNGP09.phx.gbl...
> > > > Thanks for your reply. Would interops naturally make the
system slower
> > or
> > > > would it be something with my code? In other words, can the

speed
be
> > > > improved or is it a dot net limitation?
> > > >
> > > > VM
> > > >
> > > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > > news:#x*************@tk2msftngp13.phx.gbl...
> > > > > Can I suggest you hook up a profiler to see where the problem > lies...
> > > > >
> > > > > Chances are it's a Interop problem but difficult to tell

without the
> > > > source
> > > > > code etc...
> > > > >
> > > > > There are some very good profilers out there and most have a 30 day
> > > trial
> > > > > :-)
> > > > >
> > > > > cheers,
> > > > >
> > > > > g
> > > > >
> > > > > "VM" <vo******@yahoo.com> wrote in message
> > > > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > > > In my Windows app, I'm running a batch process that's composed of
> a
> > > FOR
> > > > > loop
> > > > > > that'll run 15,000 times (datatable row count), copy cthe data of
> > each
> > > > > > row -3 fields- to a struct, and send the strct to an external > method
> > > > > (using
> > > > > > DLLImport). The problem is that the processing speed is very > > > > > inconsistent-
> > > > > > it may run very FAST during certain points but then it runs VERY
> > slow
> > > at
> > > > > > other moments (especially when it's getting to the end of the > table:
> > > > after
> > > > > > 12,000 records). I have an idea of the speed because I
have a > > counter
> > > > > being
> > > > > > displayed in the form (with an Application.DoEvents()).
I've > > > eliminated
> > > > > the
> > > > > > DoEvents but it's still inconsistent.
> > > > > >
> > > > > > This is a new version of an application that was previously
made
> in
> > MS
> > > > > > C/C++. The only differences between my C# app and the

previous
> > > version
> > > > > > (besides the language it was written in) is that I had to
recreate
> > the
> > > > > > struct in C# (which the external method receives as

parameter and
> > > > > modifies)
> > > > > > and that I use the DLLImport for the method. Would this be

a > reason
> > > why
> > > > > it's
> > > > > > so slow? Also, the memory usage (in Task Manager) is at 100% when
> > > > running.
> > > > > >
> > > > > > Thanks,
> > > > > > VM
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>



Nov 16 '05 #13

P: n/a
VM
Just to be sure, if I use Unicode and the struct doesn't have legible data
(after being modified), then I must use ANSI? Isn't there some way to cast
the data so I can keep on using Unicode?

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:OA**************@tk2msftngp13.phx.gbl...
Sorry - you need to find out which of ANSI vs Unicode the destination
function is written in and use the appropriate one.

(Do you know the difference? If not then ANSI is a single byte ber character whereas Unicode takes up two bytes per character.)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:eS**************@TK2MSFTNGP12.phx.gbl...
For 1,000 records, the difference was 66 sec. vs. 130 secs. which is pretty
good. I haven't tried it with the 15,000 records yet. But the old system

did
the 15,000 records in just under 4 mins.

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:en**************@TK2MSFTNGP12.phx.gbl...
Did setting ANSI vs Unicode make any differences?

g
"VM" <vo******@yahoo.com> wrote in message
news:OS*************@TK2MSFTNGP10.phx.gbl...
> Thanks for all your help. I just fixed all those details you told me
about.
> About the struct, here's a section of it. I'm not going to post the

whole
> thing because the rest is the same as this. In the C version of the
> application, the struct was in a header file so they didn't have to do
any
> marshalling. Also, during the run I noticed that the loop may be in
a > particular record for longer (maybe a second or more). Basically, the > extern method finds information based on the data I copied to the
ZIP4_PARM
> struct and writes it to the struct in other empty fields when it finds
is.
> It's similar to a database :
>
> namespace ZipMaster
> {
> ...
> [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
> public class ZIP4_PARM
> {
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
> public string rsvd0;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> public string iadl1;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> public string iadl2;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> public string ictyi;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
> public string istai;
> public string county;
> public short respn;
> public char retcc;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
> public string adrkey;
> public char auto_zone_ind;
> public struct footer
> {
> public char a;
> public char b;
> public char c;
> public char d;
> public char e;
> }
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
> public string rsvd3;
> }
> }
>
> Thanks again,
> VM
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:eF**************@TK2MSFTNGP11.phx.gbl...
> > OK - a couple of good practices things to point out first
> >
> > I know the compiler may pick these up but better with than
without. > >
> > Accessing a row by index is probably not the fastest way so get

yourself
a
> > reference to the row and use that rather than indexing into the

table each
> > time
> >
> > e.g.
> > DataRow row = Datatable_AuditList.Rows[i];
> >
> > Next one up - if you know that the row.Item[] is a string then cast is
> > rather than using ToString() - ToString has greater overhead so
you end
up
> > with.
> >
> > sFBU = (string)row["FBU"];
> >
> > And back to the first one - why not use a foreach(...) loop rather than
by
> > index? or is there a specific requirement to go through like that?
> >
> > All of the above are more than likely negligible in terms of
performance -
> > rem out the interop call and see how long it takes - I'm guessing
> fractions
> > of a second.
> >
> > Which leads us to believe that it must be the marshalling that is
causing
> > the problem.
> >
> > Could you post the zip4Parm structure as that's probably where we make the
> > changes - as you're passing in a LPStruct then I'm assuming that
the > > DllImport statement doesn't need to specify Unicode or ANSI.
> >
> > cheers,
> >
> > g
> >
> > PS Spent the weekend doing Interop so lets get to the bottom of this one -
> I
> > got beat by .Net as the code I was up against required __cdecl

callbacks
> via
> > delegates which aren't supported without an IL tweak using ILDASM... :-(
> >
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:%2****************@TK2MSFTNGP12.phx.gbl...
> > > The code's really short...and really easy. And this particular
run takes
> > > over 30 minutes and the speed is very inconsistent. The only long code
> is
> > > the zip4Parm struct (I didn't post it) that contains over 60

fields and
> > > they're all modified during the z4adrinq call. That's why I assumed
it
> > could
> > > be the DLL call and the amount of time it takes to fill the

struct: > > >
> > > [DllImport("ZIP4_W32.DLL")]
> > > public static extern int z4adrinq([In,
> > > Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
> > > private void btn_run_Click(object sender, System.EventArgs e)
> > > {
> > > .....
> > > for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
> > > {
> > > Application.DoEvents();
> > > sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
> > > sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
> > > sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
> > > sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
> > > ZIP4_PARM zip4Parm = new ZIP4_PARM();
> > > zip4Parm.iprurb = sFBU;
> > > zip4Parm.iadl1 = sDEL;
> > > zip4Parm.ictyi = sCTY;
> > > int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
> > > fields are modified
> > > statusBar.Panels[0].Text = "Row Count = " +
iRecCount.ToString()
> +
> > "
> > > rows ";
> > > iRecCount++;
> > > }
> > > }
> > >
> > > Thanks again,
> > > VM
> > >
> > >
> > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > news:#R**************@TK2MSFTNGP12.phx.gbl...
> > > > I guess that depends on the Interops in question!
> > > >
> > > > If you think it through then some Interop operations could

certainly
> be
> > > > tuned - for example - if you've got a 100MB of string data and
you're
> > > > passing it through Interop then the following would be true:
> > > >
> > > > 1. .Net stores it's string as Unicode
> > > > 2. Calling an ANSI Interop function would require
converting those
> > > > strings to ANSI to and from the Interop call
> > > > 3. That's got to be slower than calling a UNICODE Interop
function
> > > >
> > > > I think the example above shows how Interop could be slower than
the
> raw
> > > C++
> > > > but there are so many other factors involved. In my experience of > calls
> > > via
> > > > Interop I haven't noticed any particular slow down so the profiler wou
> ld
> > > > certainly identify which bit of code is causing the problem.
> > > >
> > > > It could also be a GC issue - why don't you post the basic
code and
it
> > may
> > > > be obvious (or probably won't be!)
> > > >
> > > > cheers,
> > > >
> > > > g
> > > >
> > > >
> > > > "VM" <vo******@yahoo.com> wrote in message
> > > > news:e6**************@TK2MSFTNGP09.phx.gbl...
> > > > > Thanks for your reply. Would interops naturally make the system > slower
> > > or
> > > > > would it be something with my code? In other words, can the

speed
be
> > > > > improved or is it a dot net limitation?
> > > > >
> > > > > VM
> > > > >
> > > > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > > > news:#x*************@tk2msftngp13.phx.gbl...
> > > > > > Can I suggest you hook up a profiler to see where the problem > > lies...
> > > > > >
> > > > > > Chances are it's a Interop problem but difficult to tell

without
> the
> > > > > source
> > > > > > code etc...
> > > > > >
> > > > > > There are some very good profilers out there and most have
a 30
> day
> > > > trial
> > > > > > :-)
> > > > > >
> > > > > > cheers,
> > > > > >
> > > > > > g
> > > > > >
> > > > > > "VM" <vo******@yahoo.com> wrote in message
> > > > > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > > > > In my Windows app, I'm running a batch process that's

composed
> of
> > a
> > > > FOR
> > > > > > loop
> > > > > > > that'll run 15,000 times (datatable row count), copy
cthe data
> of
> > > each
> > > > > > > row -3 fields- to a struct, and send the strct to an

external
> > method
> > > > > > (using
> > > > > > > DLLImport). The problem is that the processing speed is

very
> > > > > > inconsistent-
> > > > > > > it may run very FAST during certain points but then it runs VERY
> > > slow
> > > > at
> > > > > > > other moments (especially when it's getting to the end
of the
> > table:
> > > > > after
> > > > > > > 12,000 records). I have an idea of the speed because I

have
a
> > > counter
> > > > > > being
> > > > > > > displayed in the form (with an Application.DoEvents()).

I've > > > > eliminated
> > > > > > the
> > > > > > > DoEvents but it's still inconsistent.
> > > > > > >
> > > > > > > This is a new version of an application that was previously made
> > in
> > > MS
> > > > > > > C/C++. The only differences between my C# app and the
previous
> > > > version
> > > > > > > (besides the language it was written in) is that I had
to > recreate
> > > the
> > > > > > > struct in C# (which the external method receives as

parameter
> and
> > > > > > modifies)
> > > > > > > and that I use the DLLImport for the method. Would this

be a > > reason
> > > > why
> > > > > > it's
> > > > > > > so slow? Also, the memory usage (in Task Manager) is at 100% > when
> > > > > running.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > VM
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>



Nov 16 '05 #14

P: n/a
VM
If I use Unicode and the struct doesn't have legible data (after being
modified), then I must use ANSI? Isn't there some way to cast the data so I
can keep on using Unicode? Since the DLL is proprietary, I have no way of
finding out what they used to write it.

VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:OA**************@tk2msftngp13.phx.gbl...
Sorry - you need to find out which of ANSI vs Unicode the destination
function is written in and use the appropriate one.

(Do you know the difference? If not then ANSI is a single byte ber character whereas Unicode takes up two bytes per character.)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:eS**************@TK2MSFTNGP12.phx.gbl...
For 1,000 records, the difference was 66 sec. vs. 130 secs. which is pretty
good. I haven't tried it with the 15,000 records yet. But the old system

did
the 15,000 records in just under 4 mins.

VM

"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:en**************@TK2MSFTNGP12.phx.gbl...
Did setting ANSI vs Unicode make any differences?

g
"VM" <vo******@yahoo.com> wrote in message
news:OS*************@TK2MSFTNGP10.phx.gbl...
> Thanks for all your help. I just fixed all those details you told me
about.
> About the struct, here's a section of it. I'm not going to post the

whole
> thing because the rest is the same as this. In the C version of the
> application, the struct was in a header file so they didn't have to do
any
> marshalling. Also, during the run I noticed that the loop may be in
a > particular record for longer (maybe a second or more). Basically, the > extern method finds information based on the data I copied to the
ZIP4_PARM
> struct and writes it to the struct in other empty fields when it finds
is.
> It's similar to a database :
>
> namespace ZipMaster
> {
> ...
> [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
> public class ZIP4_PARM
> {
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
> public string rsvd0;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> public string iadl1;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> public string iadl2;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> public string ictyi;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
> public string istai;
> public string county;
> public short respn;
> public char retcc;
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
> public string adrkey;
> public char auto_zone_ind;
> public struct footer
> {
> public char a;
> public char b;
> public char c;
> public char d;
> public char e;
> }
> [MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
> public string rsvd3;
> }
> }
>
> Thanks again,
> VM
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:eF**************@TK2MSFTNGP11.phx.gbl...
> > OK - a couple of good practices things to point out first
> >
> > I know the compiler may pick these up but better with than
without. > >
> > Accessing a row by index is probably not the fastest way so get

yourself
a
> > reference to the row and use that rather than indexing into the

table each
> > time
> >
> > e.g.
> > DataRow row = Datatable_AuditList.Rows[i];
> >
> > Next one up - if you know that the row.Item[] is a string then cast is
> > rather than using ToString() - ToString has greater overhead so
you end
up
> > with.
> >
> > sFBU = (string)row["FBU"];
> >
> > And back to the first one - why not use a foreach(...) loop rather than
by
> > index? or is there a specific requirement to go through like that?
> >
> > All of the above are more than likely negligible in terms of
performance -
> > rem out the interop call and see how long it takes - I'm guessing
> fractions
> > of a second.
> >
> > Which leads us to believe that it must be the marshalling that is
causing
> > the problem.
> >
> > Could you post the zip4Parm structure as that's probably where we make the
> > changes - as you're passing in a LPStruct then I'm assuming that
the > > DllImport statement doesn't need to specify Unicode or ANSI.
> >
> > cheers,
> >
> > g
> >
> > PS Spent the weekend doing Interop so lets get to the bottom of this one -
> I
> > got beat by .Net as the code I was up against required __cdecl

callbacks
> via
> > delegates which aren't supported without an IL tweak using ILDASM... :-(
> >
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:%2****************@TK2MSFTNGP12.phx.gbl...
> > > The code's really short...and really easy. And this particular
run takes
> > > over 30 minutes and the speed is very inconsistent. The only long code
> is
> > > the zip4Parm struct (I didn't post it) that contains over 60

fields and
> > > they're all modified during the z4adrinq call. That's why I assumed
it
> > could
> > > be the DLL call and the amount of time it takes to fill the

struct: > > >
> > > [DllImport("ZIP4_W32.DLL")]
> > > public static extern int z4adrinq([In,
> > > Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
> > > private void btn_run_Click(object sender, System.EventArgs e)
> > > {
> > > .....
> > > for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = 15,000
> > > {
> > > Application.DoEvents();
> > > sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
> > > sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
> > > sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
> > > sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
> > > ZIP4_PARM zip4Parm = new ZIP4_PARM();
> > > zip4Parm.iprurb = sFBU;
> > > zip4Parm.iadl1 = sDEL;
> > > zip4Parm.ictyi = sCTY;
> > > int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's 60+
> > > fields are modified
> > > statusBar.Panels[0].Text = "Row Count = " +
iRecCount.ToString()
> +
> > "
> > > rows ";
> > > iRecCount++;
> > > }
> > > }
> > >
> > > Thanks again,
> > > VM
> > >
> > >
> > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > news:#R**************@TK2MSFTNGP12.phx.gbl...
> > > > I guess that depends on the Interops in question!
> > > >
> > > > If you think it through then some Interop operations could

certainly
> be
> > > > tuned - for example - if you've got a 100MB of string data and
you're
> > > > passing it through Interop then the following would be true:
> > > >
> > > > 1. .Net stores it's string as Unicode
> > > > 2. Calling an ANSI Interop function would require
converting those
> > > > strings to ANSI to and from the Interop call
> > > > 3. That's got to be slower than calling a UNICODE Interop
function
> > > >
> > > > I think the example above shows how Interop could be slower than
the
> raw
> > > C++
> > > > but there are so many other factors involved. In my experience of > calls
> > > via
> > > > Interop I haven't noticed any particular slow down so the profiler wou
> ld
> > > > certainly identify which bit of code is causing the problem.
> > > >
> > > > It could also be a GC issue - why don't you post the basic
code and
it
> > may
> > > > be obvious (or probably won't be!)
> > > >
> > > > cheers,
> > > >
> > > > g
> > > >
> > > >
> > > > "VM" <vo******@yahoo.com> wrote in message
> > > > news:e6**************@TK2MSFTNGP09.phx.gbl...
> > > > > Thanks for your reply. Would interops naturally make the system > slower
> > > or
> > > > > would it be something with my code? In other words, can the

speed
be
> > > > > improved or is it a dot net limitation?
> > > > >
> > > > > VM
> > > > >
> > > > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > > > news:#x*************@tk2msftngp13.phx.gbl...
> > > > > > Can I suggest you hook up a profiler to see where the problem > > lies...
> > > > > >
> > > > > > Chances are it's a Interop problem but difficult to tell

without
> the
> > > > > source
> > > > > > code etc...
> > > > > >
> > > > > > There are some very good profilers out there and most have
a 30
> day
> > > > trial
> > > > > > :-)
> > > > > >
> > > > > > cheers,
> > > > > >
> > > > > > g
> > > > > >
> > > > > > "VM" <vo******@yahoo.com> wrote in message
> > > > > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > > > > In my Windows app, I'm running a batch process that's

composed
> of
> > a
> > > > FOR
> > > > > > loop
> > > > > > > that'll run 15,000 times (datatable row count), copy
cthe data
> of
> > > each
> > > > > > > row -3 fields- to a struct, and send the strct to an

external
> > method
> > > > > > (using
> > > > > > > DLLImport). The problem is that the processing speed is

very
> > > > > > inconsistent-
> > > > > > > it may run very FAST during certain points but then it runs VERY
> > > slow
> > > > at
> > > > > > > other moments (especially when it's getting to the end
of the
> > table:
> > > > > after
> > > > > > > 12,000 records). I have an idea of the speed because I

have
a
> > > counter
> > > > > > being
> > > > > > > displayed in the form (with an Application.DoEvents()).

I've > > > > eliminated
> > > > > > the
> > > > > > > DoEvents but it's still inconsistent.
> > > > > > >
> > > > > > > This is a new version of an application that was previously made
> > in
> > > MS
> > > > > > > C/C++. The only differences between my C# app and the
previous
> > > > version
> > > > > > > (besides the language it was written in) is that I had
to > recreate
> > > the
> > > > > > > struct in C# (which the external method receives as

parameter
> and
> > > > > > modifies)
> > > > > > > and that I use the DLLImport for the method. Would this

be a > > reason
> > > > why
> > > > > > it's
> > > > > > > so slow? Also, the memory usage (in Task Manager) is at 100% > when
> > > > > running.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > VM
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>



Nov 16 '05 #15

P: n/a
If the unmanaged code is expecting ANSI, and you pass Unicode, it's just
going to mess up :(.
-mike
MVP

"VM" <vo******@yahoo.com> wrote in message
news:OC**************@tk2msftngp13.phx.gbl...
If I use Unicode and the struct doesn't have legible data (after being
modified), then I must use ANSI? Isn't there some way to cast the data so
I
can keep on using Unicode? Since the DLL is proprietary, I have no way of
finding out what they used to write it.

VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:OA**************@tk2msftngp13.phx.gbl...
Sorry - you need to find out which of ANSI vs Unicode the destination
function is written in and use the appropriate one.

(Do you know the difference? If not then ANSI is a single byte ber

character
whereas Unicode takes up two bytes per character.)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:eS**************@TK2MSFTNGP12.phx.gbl...
> For 1,000 records, the difference was 66 sec. vs. 130 secs. which is

pretty
> good. I haven't tried it with the 15,000 records yet. But the old
> system

did
> the 15,000 records in just under 4 mins.
>
> VM
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:en**************@TK2MSFTNGP12.phx.gbl...
> > Did setting ANSI vs Unicode make any differences?
> >
> > g
> >
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:OS*************@TK2MSFTNGP10.phx.gbl...
> > > Thanks for all your help. I just fixed all those details you told
> > > me
> > about.
> > > About the struct, here's a section of it. I'm not going to post the
> whole
> > > thing because the rest is the same as this. In the C version of the
> > > application, the struct was in a header file so they didn't have to do > any
> > > marshalling. Also, during the run I noticed that the loop may be in a > > > particular record for longer (maybe a second or more). Basically, the > > > extern method finds information based on the data I copied to the
> > ZIP4_PARM
> > > struct and writes it to the struct in other empty fields when it finds > is.
> > > It's similar to a database :
> > >
> > > namespace ZipMaster
> > > {
> > > ...
> > > [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
> > > public class ZIP4_PARM
> > > {
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
> > > public string rsvd0;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> > > public string iadl1;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> > > public string iadl2;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> > > public string ictyi;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
> > > public string istai;
> > > public string county;
> > > public short respn;
> > > public char retcc;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
> > > public string adrkey;
> > > public char auto_zone_ind;
> > > public struct footer
> > > {
> > > public char a;
> > > public char b;
> > > public char c;
> > > public char d;
> > > public char e;
> > > }
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
> > > public string rsvd3;
> > > }
> > > }
> > >
> > > Thanks again,
> > > VM
> > >
> > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > news:eF**************@TK2MSFTNGP11.phx.gbl...
> > > > OK - a couple of good practices things to point out first
> > > >
> > > > I know the compiler may pick these up but better with than without. > > > >
> > > > Accessing a row by index is probably not the fastest way so get
> yourself
> > a
> > > > reference to the row and use that rather than indexing into the

table
> > each
> > > > time
> > > >
> > > > e.g.
> > > > DataRow row = Datatable_AuditList.Rows[i];
> > > >
> > > > Next one up - if you know that the row.Item[] is a string then cast
is
> > > > rather than using ToString() - ToString has greater overhead so

you > end
> > up
> > > > with.
> > > >
> > > > sFBU = (string)row["FBU"];
> > > >
> > > > And back to the first one - why not use a foreach(...) loop
> > > > rather
> than
> > by
> > > > index? or is there a specific requirement to go through like
> > > > that?
> > > >
> > > > All of the above are more than likely negligible in terms of
> > performance -
> > > > rem out the interop call and see how long it takes - I'm guessing
> > > fractions
> > > > of a second.
> > > >
> > > > Which leads us to believe that it must be the marshalling that is
> > causing
> > > > the problem.
> > > >
> > > > Could you post the zip4Parm structure as that's probably where we

make
> > the
> > > > changes - as you're passing in a LPStruct then I'm assuming that the > > > > DllImport statement doesn't need to specify Unicode or ANSI.
> > > >
> > > > cheers,
> > > >
> > > > g
> > > >
> > > > PS Spent the weekend doing Interop so lets get to the bottom of this > > one -
> > > I
> > > > got beat by .Net as the code I was up against required __cdecl
> callbacks
> > > via
> > > > delegates which aren't supported without an IL tweak using ILDASM... > :-(
> > > >
> > > >
> > > > "VM" <vo******@yahoo.com> wrote in message
> > > > news:%2****************@TK2MSFTNGP12.phx.gbl...
> > > > > The code's really short...and really easy. And this particular run > > takes
> > > > > over 30 minutes and the speed is very inconsistent. The only long > > code
> > > is
> > > > > the zip4Parm struct (I didn't post it) that contains over 60

fields
> > and
> > > > > they're all modified during the z4adrinq call. That's why I

assumed
> it
> > > > could
> > > > > be the DLL call and the amount of time it takes to fill the

struct:
> > > > >
> > > > > [DllImport("ZIP4_W32.DLL")]
> > > > > public static extern int z4adrinq([In,
> > > > > Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
> > > > > private void btn_run_Click(object sender, System.EventArgs e)
> > > > > {
> > > > > .....
> > > > > for (int i=0;i<Datatable_AuditList.Rows.Count;i++) //count = > > 15,000
> > > > > {
> > > > > Application.DoEvents();
> > > > > sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
> > > > > sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
> > > > > sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
> > > > > sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
> > > > > ZIP4_PARM zip4Parm = new ZIP4_PARM();
> > > > > zip4Parm.iprurb = sFBU;
> > > > > zip4Parm.iadl1 = sDEL;
> > > > > zip4Parm.ictyi = sCTY;
> > > > > int iError = z4adrinq(zip4Parm); //dll call - zip4Parm's > 60+
> > > > > fields are modified
> > > > > statusBar.Panels[0].Text = "Row Count = " +
> > iRecCount.ToString()
> > > +
> > > > "
> > > > > rows ";
> > > > > iRecCount++;
> > > > > }
> > > > > }
> > > > >
> > > > > Thanks again,
> > > > > VM
> > > > >
> > > > >
> > > > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > > > news:#R**************@TK2MSFTNGP12.phx.gbl...
> > > > > > I guess that depends on the Interops in question!
> > > > > >
> > > > > > If you think it through then some Interop operations could
> certainly
> > > be
> > > > > > tuned - for example - if you've got a 100MB of string data
> > > > > > and
> > you're
> > > > > > passing it through Interop then the following would be true:
> > > > > >
> > > > > > 1. .Net stores it's string as Unicode
> > > > > > 2. Calling an ANSI Interop function would require converting > > those
> > > > > > strings to ANSI to and from the Interop call
> > > > > > 3. That's got to be slower than calling a UNICODE Interop
> > function
> > > > > >
> > > > > > I think the example above shows how Interop could be slower than > the
> > > raw
> > > > > C++
> > > > > > but there are so many other factors involved. In my
> > > > > > experience

of
> > > calls
> > > > > via
> > > > > > Interop I haven't noticed any particular slow down so the

profiler
> > wou
> > > ld
> > > > > > certainly identify which bit of code is causing the problem.
> > > > > >
> > > > > > It could also be a GC issue - why don't you post the basic code > and
> > it
> > > > may
> > > > > > be obvious (or probably won't be!)
> > > > > >
> > > > > > cheers,
> > > > > >
> > > > > > g
> > > > > >
> > > > > >
> > > > > > "VM" <vo******@yahoo.com> wrote in message
> > > > > > news:e6**************@TK2MSFTNGP09.phx.gbl...
> > > > > > > Thanks for your reply. Would interops naturally make the

system
> > > slower
> > > > > or
> > > > > > > would it be something with my code? In other words, can the
> speed
> > be
> > > > > > > improved or is it a dot net limitation?
> > > > > > >
> > > > > > > VM
> > > > > > >
> > > > > > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > > > > > news:#x*************@tk2msftngp13.phx.gbl...
> > > > > > > > Can I suggest you hook up a profiler to see where the

problem
> > > > lies...
> > > > > > > >
> > > > > > > > Chances are it's a Interop problem but difficult to tell
> without
> > > the
> > > > > > > source
> > > > > > > > code etc...
> > > > > > > >
> > > > > > > > There are some very good profilers out there and most
> > > > > > > > have a > 30
> > > day
> > > > > > trial
> > > > > > > > :-)
> > > > > > > >
> > > > > > > > cheers,
> > > > > > > >
> > > > > > > > g
> > > > > > > >
> > > > > > > > "VM" <vo******@yahoo.com> wrote in message
> > > > > > > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > > > > > > In my Windows app, I'm running a batch process that's
> composed
> > > of
> > > > a
> > > > > > FOR
> > > > > > > > loop
> > > > > > > > > that'll run 15,000 times (datatable row count), copy cthe > data
> > > of
> > > > > each
> > > > > > > > > row -3 fields- to a struct, and send the strct to an
> external
> > > > method
> > > > > > > > (using
> > > > > > > > > DLLImport). The problem is that the processing speed
> > > > > > > > > is
> very
> > > > > > > > inconsistent-
> > > > > > > > > it may run very FAST during certain points but then it

runs
> > VERY
> > > > > slow
> > > > > > at
> > > > > > > > > other moments (especially when it's getting to the end of > the
> > > > table:
> > > > > > > after
> > > > > > > > > 12,000 records). I have an idea of the speed because I

have
> a
> > > > > counter
> > > > > > > > being
> > > > > > > > > displayed in the form (with an Application.DoEvents()).

I've
> > > > > > eliminated
> > > > > > > > the
> > > > > > > > > DoEvents but it's still inconsistent.
> > > > > > > > >
> > > > > > > > > This is a new version of an application that was

previously
> > made
> > > > in
> > > > > MS
> > > > > > > > > C/C++. The only differences between my C# app and the
> > previous
> > > > > > version
> > > > > > > > > (besides the language it was written in) is that I had to > > > recreate
> > > > > the
> > > > > > > > > struct in C# (which the external method receives as
> parameter
> > > and
> > > > > > > > modifies)
> > > > > > > > > and that I use the DLLImport for the method. Would this

be
a
> > > > reason
> > > > > > why
> > > > > > > > it's
> > > > > > > > > so slow? Also, the memory usage (in Task Manager) is at

100%
> > > when
> > > > > > > running.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > VM
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>




Nov 16 '05 #16

P: n/a
VM
So assuming it just accepts ANSI, would it be possible that calling this DLL
function (made in C) from within a C application is so much faster than
converting the C-struct to C# and marshalling the data for the interop?
The previous version (made in C) did the whole process in less than 4 mins.
while the C# version takes over 30 mins.

Vaughn

"Michael Giagnocavo [MVP]" <mg*******@Atrevido.net> wrote in message
news:uC**************@TK2MSFTNGP10.phx.gbl...
If the unmanaged code is expecting ANSI, and you pass Unicode, it's just
going to mess up :(.
-mike
MVP

"VM" <vo******@yahoo.com> wrote in message
news:OC**************@tk2msftngp13.phx.gbl...
If I use Unicode and the struct doesn't have legible data (after being
modified), then I must use ANSI? Isn't there some way to cast the data so I
can keep on using Unicode? Since the DLL is proprietary, I have no way of finding out what they used to write it.

VM
"Gary Hunt" <be*******@codequest.co.uk> wrote in message
news:OA**************@tk2msftngp13.phx.gbl...
Sorry - you need to find out which of ANSI vs Unicode the destination
function is written in and use the appropriate one.

(Do you know the difference? If not then ANSI is a single byte ber

character
whereas Unicode takes up two bytes per character.)

cheers,

g
"VM" <vo******@yahoo.com> wrote in message
news:eS**************@TK2MSFTNGP12.phx.gbl...
> For 1,000 records, the difference was 66 sec. vs. 130 secs. which is
pretty
> good. I haven't tried it with the 15,000 records yet. But the old
> system
did
> the 15,000 records in just under 4 mins.
>
> VM
>
> "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> news:en**************@TK2MSFTNGP12.phx.gbl...
> > Did setting ANSI vs Unicode make any differences?
> >
> > g
> >
> >
> > "VM" <vo******@yahoo.com> wrote in message
> > news:OS*************@TK2MSFTNGP10.phx.gbl...
> > > Thanks for all your help. I just fixed all those details you told
> > > me
> > about.
> > > About the struct, here's a section of it. I'm not going to post the > whole
> > > thing because the rest is the same as this. In the C version of the > > > application, the struct was in a header file so they didn't have to
do
> any
> > > marshalling. Also, during the run I noticed that the loop may be
in a
> > > particular record for longer (maybe a second or more).
Basically, the
> > > extern method finds information based on the data I copied to the
> > ZIP4_PARM
> > > struct and writes it to the struct in other empty fields when it

finds
> is.
> > > It's similar to a database :
> > >
> > > namespace ZipMaster
> > > {
> > > ...
> > > [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
> > > public class ZIP4_PARM
> > > {
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=4 )]
> > > public string rsvd0;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> > > public string iadl1;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> > > public string iadl2;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=51 )]
> > > public string ictyi;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=3 )]
> > > public string istai;
> > > public string county;
> > > public short respn;
> > > public char retcc;
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=12 )]
> > > public string adrkey;
> > > public char auto_zone_ind;
> > > public struct footer
> > > {
> > > public char a;
> > > public char b;
> > > public char c;
> > > public char d;
> > > public char e;
> > > }
> > > [MarshalAs( UnmanagedType.ByValTStr, SizeConst=6)]
> > > public string rsvd3;
> > > }
> > > }
> > >
> > > Thanks again,
> > > VM
> > >
> > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > news:eF**************@TK2MSFTNGP11.phx.gbl...
> > > > OK - a couple of good practices things to point out first
> > > >
> > > > I know the compiler may pick these up but better with than

without.
> > > >
> > > > Accessing a row by index is probably not the fastest way so get
> yourself
> > a
> > > > reference to the row and use that rather than indexing into the
table
> > each
> > > > time
> > > >
> > > > e.g.
> > > > DataRow row = Datatable_AuditList.Rows[i];
> > > >
> > > > Next one up - if you know that the row.Item[] is a string then

cast
is
> > > > rather than using ToString() - ToString has greater overhead so

you
> end
> > up
> > > > with.
> > > >
> > > > sFBU = (string)row["FBU"];
> > > >
> > > > And back to the first one - why not use a foreach(...) loop
> > > > rather
> than
> > by
> > > > index? or is there a specific requirement to go through like
> > > > that?
> > > >
> > > > All of the above are more than likely negligible in terms of
> > performance -
> > > > rem out the interop call and see how long it takes - I'm
guessing > > > fractions
> > > > of a second.
> > > >
> > > > Which leads us to believe that it must be the marshalling that is > > causing
> > > > the problem.
> > > >
> > > > Could you post the zip4Parm structure as that's probably where we make
> > the
> > > > changes - as you're passing in a LPStruct then I'm assuming that the
> > > > DllImport statement doesn't need to specify Unicode or ANSI.
> > > >
> > > > cheers,
> > > >
> > > > g
> > > >
> > > > PS Spent the weekend doing Interop so lets get to the bottom of

this
> > one -
> > > I
> > > > got beat by .Net as the code I was up against required __cdecl
> callbacks
> > > via
> > > > delegates which aren't supported without an IL tweak using

ILDASM...
> :-(
> > > >
> > > >
> > > > "VM" <vo******@yahoo.com> wrote in message
> > > > news:%2****************@TK2MSFTNGP12.phx.gbl...
> > > > > The code's really short...and really easy. And this
particular run
> > takes
> > > > > over 30 minutes and the speed is very inconsistent. The only

long
> > code
> > > is
> > > > > the zip4Parm struct (I didn't post it) that contains over 60
fields
> > and
> > > > > they're all modified during the z4adrinq call. That's why I
assumed
> it
> > > > could
> > > > > be the DLL call and the amount of time it takes to fill the
struct:
> > > > >
> > > > > [DllImport("ZIP4_W32.DLL")]
> > > > > public static extern int z4adrinq([In,
> > > > > Out][MarshalAs(UnmanagedType.LPStruct)] ZIP4_PARM zip4_parm);
> > > > > private void btn_run_Click(object sender, System.EventArgs e)
> > > > > {
> > > > > .....
> > > > > for (int i=0;i<Datatable_AuditList.Rows.Count;i++)
//count =
> > 15,000
> > > > > {
> > > > > Application.DoEvents();
> > > > > sFBU = Datatable_AuditList.Rows[i]["FBU"].ToString();
> > > > > sDEL = Datatable_AuditList.Rows[i]["DEL"].ToString();
> > > > > sCTY = Datatable_AuditList.Rows[i]["CTY"].ToString();
> > > > > sZ4 = Datatable_AuditList.Rows[i]["Z4"].ToString();
> > > > > ZIP4_PARM zip4Parm = new ZIP4_PARM();
> > > > > zip4Parm.iprurb = sFBU;
> > > > > zip4Parm.iadl1 = sDEL;
> > > > > zip4Parm.ictyi = sCTY;
> > > > > int iError = z4adrinq(zip4Parm); //dll call -

zip4Parm's
> 60+
> > > > > fields are modified
> > > > > statusBar.Panels[0].Text = "Row Count = " +
> > iRecCount.ToString()
> > > +
> > > > "
> > > > > rows ";
> > > > > iRecCount++;
> > > > > }
> > > > > }
> > > > >
> > > > > Thanks again,
> > > > > VM
> > > > >
> > > > >
> > > > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > > > news:#R**************@TK2MSFTNGP12.phx.gbl...
> > > > > > I guess that depends on the Interops in question!
> > > > > >
> > > > > > If you think it through then some Interop operations could
> certainly
> > > be
> > > > > > tuned - for example - if you've got a 100MB of string data
> > > > > > and
> > you're
> > > > > > passing it through Interop then the following would be
true: > > > > > >
> > > > > > 1. .Net stores it's string as Unicode
> > > > > > 2. Calling an ANSI Interop function would require

converting
> > those
> > > > > > strings to ANSI to and from the Interop call
> > > > > > 3. That's got to be slower than calling a UNICODE Interop > > function
> > > > > >
> > > > > > I think the example above shows how Interop could be slower

than
> the
> > > raw
> > > > > C++
> > > > > > but there are so many other factors involved. In my
> > > > > > experience
of
> > > calls
> > > > > via
> > > > > > Interop I haven't noticed any particular slow down so the
profiler
> > wou
> > > ld
> > > > > > certainly identify which bit of code is causing the problem. > > > > > >
> > > > > > It could also be a GC issue - why don't you post the basic

code
> and
> > it
> > > > may
> > > > > > be obvious (or probably won't be!)
> > > > > >
> > > > > > cheers,
> > > > > >
> > > > > > g
> > > > > >
> > > > > >
> > > > > > "VM" <vo******@yahoo.com> wrote in message
> > > > > > news:e6**************@TK2MSFTNGP09.phx.gbl...
> > > > > > > Thanks for your reply. Would interops naturally make the
system
> > > slower
> > > > > or
> > > > > > > would it be something with my code? In other words, can the > speed
> > be
> > > > > > > improved or is it a dot net limitation?
> > > > > > >
> > > > > > > VM
> > > > > > >
> > > > > > > "Gary Hunt" <be*******@codequest.co.uk> wrote in message
> > > > > > > news:#x*************@tk2msftngp13.phx.gbl...
> > > > > > > > Can I suggest you hook up a profiler to see where the
problem
> > > > lies...
> > > > > > > >
> > > > > > > > Chances are it's a Interop problem but difficult to tell > without
> > > the
> > > > > > > source
> > > > > > > > code etc...
> > > > > > > >
> > > > > > > > There are some very good profilers out there and most
> > > > > > > > have

a
> 30
> > > day
> > > > > > trial
> > > > > > > > :-)
> > > > > > > >
> > > > > > > > cheers,
> > > > > > > >
> > > > > > > > g
> > > > > > > >
> > > > > > > > "VM" <vo******@yahoo.com> wrote in message
> > > > > > > > news:%2****************@tk2msftngp13.phx.gbl...
> > > > > > > > > In my Windows app, I'm running a batch process that's
> composed
> > > of
> > > > a
> > > > > > FOR
> > > > > > > > loop
> > > > > > > > > that'll run 15,000 times (datatable row count), copy

cthe
> data
> > > of
> > > > > each
> > > > > > > > > row -3 fields- to a struct, and send the strct to an
> external
> > > > method
> > > > > > > > (using
> > > > > > > > > DLLImport). The problem is that the processing speed
> > > > > > > > > is
> very
> > > > > > > > inconsistent-
> > > > > > > > > it may run very FAST during certain points but then it runs
> > VERY
> > > > > slow
> > > > > > at
> > > > > > > > > other moments (especially when it's getting to the end of
> the
> > > > table:
> > > > > > > after
> > > > > > > > > 12,000 records). I have an idea of the speed because
I have
> a
> > > > > counter
> > > > > > > > being
> > > > > > > > > displayed in the form (with an Application.DoEvents()). I've
> > > > > > eliminated
> > > > > > > > the
> > > > > > > > > DoEvents but it's still inconsistent.
> > > > > > > > >
> > > > > > > > > This is a new version of an application that was
previously
> > made
> > > > in
> > > > > MS
> > > > > > > > > C/C++. The only differences between my C# app and the > > previous
> > > > > > version
> > > > > > > > > (besides the language it was written in) is that I had to
> > > recreate
> > > > > the
> > > > > > > > > struct in C# (which the external method receives as
> parameter
> > > and
> > > > > > > > modifies)
> > > > > > > > > and that I use the DLLImport for the method. Would
this be
a
> > > > reason
> > > > > > why
> > > > > > > > it's
> > > > > > > > > so slow? Also, the memory usage (in Task Manager) is

at 100%
> > > when
> > > > > > > running.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > VM
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>


Nov 16 '05 #17

P: n/a
Are you only using ASCII (7-bits)? If so, perhaps you can do your own
marshalling and speed it up (assuming that the marshaller is converting
unicode to some different codepage). The reason being that going to ascii is
simply taking the first 7bits, while true encoding conversion is more
involved.

Perhaps you should summarize the problem now that you've narrowed it down
(right?) and post to ms.public.dotnet.framework.performance.

-Michael
MVP

"VM" <vo******@yahoo.com> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
So assuming it just accepts ANSI, would it be possible that calling this
DLL
function (made in C) from within a C application is so much faster than
converting the C-struct to C# and marshalling the data for the interop?
The previous version (made in C) did the whole process in less than 4
mins.
while the C# version takes over 30 mins.

Vaughn

"Michael Giagnocavo [MVP]" <mg*******@Atrevido.net> wrote in message
news:uC**************@TK2MSFTNGP10.phx.gbl...
If the unmanaged code is expecting ANSI, and you pass Unicode, it's just
going to mess up :(.
-mike
MVP

"VM" <vo******@yahoo.com> wrote in message
news:OC**************@tk2msftngp13.phx.gbl...
> If I use Unicode and the struct doesn't have legible data (after being
> modified), then I must use ANSI? Isn't there some way to cast the data

so


Nov 16 '05 #18

This discussion thread is closed

Replies have been disabled for this discussion.