CodeDom and GenerateInMemor y memory leak

[=?Utf-8?B?QU1lcmNlcg==?=]

I may have painted myself into a corner with GenerateInMemor y=true.

My app need a custom user step. Users want to code (sort of - they are not
programmers) some refinements to a search procedure. They can concoct a
moderately complex boolean expression, and that suffices. I will take their
statements, add top and bottom text, and now I can do a CodeDom compile.

I generate an assembly via CodeDom with GenerateInMemor y. I make a
delegate, and I can call my function, and all of this works fine. It runs at
good speed (no boxing of arguments), and that is what I need since I will be
looping over an array of data to be searched.

The problem is that I have a memory leak because of not unloading the
assembly. So, what I need is a new appdomain, and at this point I am stuck.
I don't know how to call CompileAssembly FromSource() with
GenerateInMemor y=True and put the generated assembly in a new appdomain. I
have no dll to load into the new appdomain.

Does anyone have any bright ideas? Working VS2005 VB source code (with the
leak) is included below. Every call to TestDynamicCode () adds two handles as
reported by the task manager. The call to CompileAssembly FromSource() is
what produces the leak. What I think I want to do is make an appdomain, tell
codedom to put my compiler output assembly in the appdomain, do my
calculations, and then unload the appdomain. I don't see how to do it.

-------------------------

Public Module DynamicCode

Public Function SourceCode() As String
' user provides the commented lines, my code provides the rest
' the user lines coded herein are just for testing
Dim a() As String = { _
"Imports System.Math", _
"Public Class DynamicClass", _
" Public Shared Function DynamicFunction ( _", _
" Byval s1 As String, _", _
" Byval s2 As String, _", _
" Byval i1 As Integer, _", _
" Byval i2 As Integer, _", _
" Byval d1 As Double, _", _
" Byval d2 As Double, _", _
" Byval d3 As Double, _", _
" Byval d4 As Double) _", _
" As Boolean", _
" Dim Qualified, b1, b2, b3, b4, b5, b6, b7, b8, b9 As Boolean", _
" Dim d As Double = Sqrt(d1)", _
" b1 = (d1>d2) ' user provides this and perhaps many others", _
" Qualified = b1 ' user provides this, the final result", _
" Return Qualified", _
" End Function", _
"End Class"}
Return Join(a, vbLf)
End Function

Public Delegate Function DynamicDelegate ( _
ByVal s1 As String, _
ByVal s2 As String, _
ByVal i1 As Integer, _
ByVal i2 As Integer, _
ByVal d1 As Double, _
ByVal d2 As Double, _
ByVal d3 As Double, _
ByVal d4 As Double) _
As Boolean ' this delegate must match the signature in SourceCode() above

Public Sub TestDynamicCode () ' success is no exceptions and no assert
failures

' make params, the compiler parameters
Dim params As New CodeDom.Compile r.CompilerParam eters
params.Generate InMemory = True ' Assembly is created in memory
params.TreatWar ningsAsErrors = False
params.WarningL evel = 4
params.Compiler Options = "/optionexplicit /optionstrict /nowarn
/optimize-"
'Dim refs() As String = {"System.dll ", "Microsoft.Visu alBasic.dll"}
'params.Referen cedAssemblies.A ddRange(refs)

' make results by compiling the source code
' CompileAssembly FromSource leaks 2 handles and about 1k memory
' i think it is because of not unloading the assembly
' however, you can't unload an assembly, you can only unload an appdomain
' everything works ok except for this leak
Dim provider As New Microsoft.Visua lBasic.VBCodePr ovider
Dim results As CodeDom.Compile r.CompilerResul ts =
provider.Compil eAssemblyFromSo urce(params, SourceCode)
provider.Dispos e()

' report compilation errors
Dim errors As String = ""
For Each err As CodeDom.Compile r.CompilerError In results.Errors
Const f As String = "Line {0}, Col {1}: Error {2} - {3}"
errors &= [String].Format(f, err.Line, err.Column, err.ErrorNumber ,
err.ErrorText) & vbLf
Next err
Debug.Assert(er rors = "", errors)

' make DynamicFunction Delegate so we can execute strongly typed and
without boxing
Dim DynamicClass As Type =
results.Compile dAssembly.GetTy pe("DynamicClas s")
Dim flags As Reflection.Bind ingFlags = Reflection.Bind ingFlags.Public Or
Reflection.Bind ingFlags.Static
Dim DynamicFunction As Reflection.Meth odInfo =
DynamicClass.Ge tMethod("Dynami cFunction", flags)
Dim DynamicFunction Delegate As DynamicDelegate =
CType([Delegate].CreateDelegate (GetType(Dynami cDelegate), Nothing,
DynamicFunction ), DynamicDelegate )
Debug.Assert(No t DynamicFunction Delegate Is Nothing)

' prove that it works, raise the loop limit to measure execution speed
For i As Integer = 1 To 100
Dim a As Double = Rnd(1)
Dim b As Double = Rnd(1)
Dim c As Double = 0
Dim bResult As Boolean = DynamicFunction Delegate("", "", 1, 2, a, b,
c, c)
Debug.Assert(bR esult = (a b))
Next i

End Sub

End Module

[Tom Shelton]

On Oct 17, 8:01 am, AMercer <AMer...@discus sions.microsoft .comwrote:

I may have painted myself into a corner with GenerateInMemor y=true.

My app need a custom user step. Users want to code (sort of - they are not
programmers) some refinements to a search procedure. They can concoct a
moderately complex boolean expression, and that suffices. I will take their
statements, add top and bottom text, and now I can do a CodeDom compile.

I generate an assembly via CodeDom with GenerateInMemor y. I make a
delegate, and I can call my function, and all of this works fine. It runs at
good speed (no boxing of arguments), and that is what I need since I will be
looping over an array of data to be searched.

The problem is that I have a memory leak because of not unloading the
assembly. So, what I need is a new appdomain, and at this point I am stuck.
I don't know how to call CompileAssembly FromSource() with
GenerateInMemor y=True and put the generated assembly in a new appdomain. I
have no dll to load into the new appdomain.

Does anyone have any bright ideas? Working VS2005 VB source code (with the
leak) is included below. Every call to TestDynamicCode () adds two handles as
reported by the task manager. The call to CompileAssembly FromSource() is
what produces the leak. What I think I want to do is make an appdomain, tell
codedom to put my compiler output assembly in the appdomain, do my
calculations, and then unload the appdomain. I don't see how to do it.

-------------------------

Public Module DynamicCode

Public Function SourceCode() As String
' user provides the commented lines, my code provides the rest
' the user lines coded herein are just for testing
Dim a() As String = { _
"Imports System.Math", _
"Public Class DynamicClass", _
" Public Shared Function DynamicFunction ( _", _
" Byval s1 As String, _", _
" Byval s2 As String, _", _
" Byval i1 As Integer, _", _
" Byval i2 As Integer, _", _
" Byval d1 As Double, _", _
" Byval d2 As Double, _", _
" Byval d3 As Double, _", _
" Byval d4 As Double) _", _
" As Boolean", _
" Dim Qualified, b1, b2, b3, b4, b5, b6, b7, b8, b9 As Boolean", _
" Dim d As Double = Sqrt(d1)", _
" b1 = (d1>d2) ' user provides this and perhaps many others", _
" Qualified = b1 ' user provides this, the final result", _
" Return Qualified", _
" End Function", _
"End Class"}
Return Join(a, vbLf)
End Function

Public Delegate Function DynamicDelegate ( _
ByVal s1 As String, _
ByVal s2 As String, _
ByVal i1 As Integer, _
ByVal i2 As Integer, _
ByVal d1 As Double, _
ByVal d2 As Double, _
ByVal d3 As Double, _
ByVal d4 As Double) _
As Boolean ' this delegate must match the signature in SourceCode() above

Public Sub TestDynamicCode () ' success is no exceptions and no assert
failures

' make params, the compiler parameters
Dim params As New CodeDom.Compile r.CompilerParam eters
params.Generate InMemory = True ' Assembly is created in memory
params.TreatWar ningsAsErrors = False
params.WarningL evel = 4
params.Compiler Options = "/optionexplicit /optionstrict /nowarn
/optimize-"
'Dim refs() As String = {"System.dll ", "Microsoft.Visu alBasic.dll"}
'params.Referen cedAssemblies.A ddRange(refs)

' make results by compiling the source code
' CompileAssembly FromSource leaks 2 handles and about 1k memory
' i think it is because of not unloading the assembly
' however, you can't unload an assembly, you can only unload an appdomain
' everything works ok except for this leak
Dim provider As New Microsoft.Visua lBasic.VBCodePr ovider
Dim results As CodeDom.Compile r.CompilerResul ts =
provider.Compil eAssemblyFromSo urce(params, SourceCode)
provider.Dispos e()

' report compilation errors
Dim errors As String = ""
For Each err As CodeDom.Compile r.CompilerError In results.Errors
Const f As String = "Line {0}, Col {1}: Error {2} - {3}"
errors &= [String].Format(f, err.Line, err.Column, err.ErrorNumber ,
err.ErrorText) & vbLf
Next err
Debug.Assert(er rors = "", errors)

' make DynamicFunction Delegate so we can execute strongly typed and
without boxing
Dim DynamicClass As Type =
results.Compile dAssembly.GetTy pe("DynamicClas s")
Dim flags As Reflection.Bind ingFlags = Reflection.Bind ingFlags.Public Or
Reflection.Bind ingFlags.Static
Dim DynamicFunction As Reflection.Meth odInfo =
DynamicClass.Ge tMethod("Dynami cFunction", flags)
Dim DynamicFunction Delegate As DynamicDelegate =
CType([Delegate].CreateDelegate (GetType(Dynami cDelegate), Nothing,
DynamicFunction ), DynamicDelegate )
Debug.Assert(No t DynamicFunction Delegate Is Nothing)

' prove that it works, raise the loop limit to measure execution speed
For i As Integer = 1 To 100
Dim a As Double = Rnd(1)
Dim b As Double = Rnd(1)
Dim c As Double = 0
Dim bResult As Boolean = DynamicFunction Delegate("", "", 1, 2, a, b,
c, c)
Debug.Assert(bR esult = (a b))
Next i

End Sub

End Module

I don't have the time at the moment to work up a complete example, but
here are some general thoughts on what you are going to have to do to
accomplish your goal.

Since you have no way of specifying the appdomain for the in memory
assembly, then you're going to have to actually compile the assembly
in the target domain. While this may sound straight forward, it
isn't... The reason is that you're going to have to set up a method
to communicate with this domain with out actually referencing ANY
types that are in the dynamic assembly. Why? While, as soon as you
do, then it will also be loaded into your main appdomain, and you
won't be able to remove it from memory...

The trick is to create a class in a separate dll that inherits from
MarshalByRefObj ect that acts as the actual class loader. Then you can
do something like this in your code:

Dim compileDomain As AppDomain = AppDomain.Creat eDomain
("CompileDomain ")
Dim loader As ClassLoader = DirectCast
(compileDomain. CreateInstanceA ndUnwrap("Loade rNamespace",
"LoaderNamespac e.ClassLoader")

loader.DoYourWo rk("paramerter s here")
AppDomain.Unloa d(compileDomain )

That's pretty rough, but the idea is that you only reference the class
loader type. Then, you create an instance of that in the new domain
(which is what the CreateInstanceA ndUnwrap method does for you). At
this point all your calls ot the loader are then marshaled into the
new appdomain via remoting - which shouldn't cause you much of a
performance issue IF you can engineer your Loader class in such a way
that you don't have to make to many calls to it. In other words, keep
the interface chunky, not chatty.

--
Tom Shelton