AppDomain won't recycle on web.config change since 1.1 to 2.0 swit

[=?Utf-8?B?U3RlZmFuIEJhcmxvdw==?=]

This has been working perfectly for months. Since we switched from ASP.NET
1.1 to 2.0, we have constant and sporadic issues with updating our
applications.

Touching the web.config works about 40% of the time to cause an AppDomain to
reload and flush all assemblies. Sometimes it works perfectly, but other
times some Assemblies are not reloaded into the AppDomain, even after
multiple web.config file touches. Sometimes multiple versions of the same
assembly are loaded, even though the web.config clearly defines the correct
version to load.

Sometimes, we have to force the entire app pool to recycle before the
assemblies load correctly. No amount of web.config touching will fix it.

Most of the time some assemblies will load, and randomly other assemblies
will not load.

We are receiving 2 primary error messages:

System.IO.FileL oadException: The located assembly's manifest definition does
not match the assembly reference. (Exception from HRESULT: 0x80131040)

The type 'Core.CMS.Page' is ambiguous: it could come from assembly
'C:\WINDOWS\ass embly\GAC_MSIL\ Core\4.2.100.6_ _326259bf81ba92 e7\AmeriCommerc e.dll'
or from assembly
'C:\WINDOWS\ass embly\GAC_MSIL\ Core\4.2.100.8_ _326259bf81ba92 e7\AmeriCommerc e.dll'. Please specify the assembly explicitly in the type name.

My environment is as follows:

Multiple web servers access a DFS share for the home directory for a site.

Automated update system adds new versioned assemblies to the GAC on the web
servers.

Web.Config file is updated with all new assembly versions.

--
Stefan Barlow
MCSD.Net

[Walter Wang [MSFT]]

Hi Stefan,

Based on my understanding, this issue only occurs after you've migrated
your web application to 2.0?

I cannot seem to find some related or known issue for such symptom in our
internal support database. As a general troubleshooting method, you could
turn on the Fusion log to see why the assembly loading is failing. Please
follow the steps in this blog to learn how to generate the fusion log and
post here for further investigation:

#Suzanne Cook's .NET CLR Notes : Debugging Assembly Loading Failures
http://blogs.msdn.com/suzcook/archiv.../29/57120.aspx
Sincerely,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications. If you are using Outlook Express, please make sure you clear the
check box "Tools/Options/Read: Get 300 headers at a time" to see your reply
promptly.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[=?Utf-8?B?U3RlZmFuIEJhcmxvdw==?=]

I am absolutely positive it started with the switch to 2.0.

We will look into the Fusion logs, thanks. I'll post my findings.

--
Stefan Barlow
MCSD.Net
"Walter Wang [MSFT]" wrote:

Hi Stefan,

Based on my understanding, this issue only occurs after you've migrated
your web application to 2.0?

I cannot seem to find some related or known issue for such symptom in our
internal support database. As a general troubleshooting method, you could
turn on the Fusion log to see why the assembly loading is failing. Please
follow the steps in this blog to learn how to generate the fusion log and
post here for further investigation:

#Suzanne Cook's .NET CLR Notes : Debugging Assembly Loading Failures
http://blogs.msdn.com/suzcook/archiv.../29/57120.aspx
Sincerely,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications. If you are using Outlook Express, please make sure you clear the
check box "Tools/Options/Read: Get 300 headers at a time" to see your reply
promptly.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[=?Utf-8?B?U3RlZmFuIEJhcmxvdw==?=]

The code continues the old version, but after touching the web.config it also
loads the new code version into the app domain. When I call a page with this
code on it:

System.Web.Http Runtime.UnloadA ppDomain();

And re-load the page, all works as expected. There is definaltely an issue
with how it is watching the web.config. Possibly due to using the DFS share.

--
Stefan Barlow
MCSD.Net
"Stefan Barlow" wrote:

I am absolutely positive it started with the switch to 2.0.

We will look into the Fusion logs, thanks. I'll post my findings.

--
Stefan Barlow
MCSD.Net
"Walter Wang [MSFT]" wrote:

Hi Stefan,

Based on my understanding, this issue only occurs after you've migrated
your web application to 2.0?

I cannot seem to find some related or known issue for such symptom in our
internal support database. As a general troubleshooting method, you could
turn on the Fusion log to see why the assembly loading is failing. Please
follow the steps in this blog to learn how to generate the fusion log and
post here for further investigation:

#Suzanne Cook's .NET CLR Notes : Debugging Assembly Loading Failures
http://blogs.msdn.com/suzcook/archiv.../29/57120.aspx
Sincerely,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications. If you are using Outlook Express, please make sure you clear the
check box "Tools/Options/Read: Get 300 headers at a time" to see your reply
promptly.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[Walter Wang [MSFT]]

Hi Stefan,

I can not seem to find any known issue related to DFS. Is it possible for
you to try your web site on a normal folder?
Regards,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[=?Utf-8?B?U3RlZmFuIEJhcmxvdw==?=]

Not with our current configuration.

What I've found is touching the web.config doesn't necessarily cause an app
domain recycle. I added a method to Application_Sta rt to do something to a
file in the root when the app starts back up. It does not fire when I touch
the web.config. When I add and remove a folder in the root, it does fire.
This has temporarily fixed the ambiguous assembly issue, but the whole
concept of an app domain recycle on deleting non-config files is a horrible
solution in our case and I wish we could disable that feature and find
another way. People modify their own sites all day long and we don't want an
app domain reload to be triggered by uploading/removing images

However, this error has not been resolved by an app domain recycle:
The located assembly's manifest definition does not match the assembly
reference. (Exception from HRESULT: 0x80131040)

This appeared on sites today that have not been updated in a few days. It
happened to all sites in the app pool regarding a third party assembly that
never changes. I had to recycle the entire app pool to resolve this issue.
This is a biiiig problem.

--
Stefan Barlow
MCSD.Net
"Walter Wang [MSFT]" wrote:

Hi Stefan,

I can not seem to find any known issue related to DFS. Is it possible for
you to try your web site on a normal folder?
Regards,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[=?Utf-8?B?U3RlZmFuIEJhcmxvdw==?=]

Here is the Fusion log output from the Manifest issue:

The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified.

Assembly manager loaded from:
C:\WINDOWS\Micr osoft.NET\Frame work\v2.0.50727 \mscorwks.dll
Running under executable c:\windows\syst em32\inetsrv\w3 wp.exe
--- A detailed error log follows.

=== Pre-bind state information ===
LOG: User = Unknown
LOG: DisplayName = RadEditor, Version=5.6.1.0 , Culture=neutral ,
PublicKeyToken= 852c9eb6525c1b5 3
(Fully-specified)
LOG: Appbase = file://xxxx/WebData/WebSites/xxxx/xxxx/www/
LOG: Initial PrivatePath = \\xxxx\WebData\ WebSites\xxxx\x xxx\www\bin
LOG: Dynamic Base = C:\WINDOWS\Micr osoft.NET\Frame work\v2.0.50727 \Temporary
ASP.NET Files\root\5d8a c2d8
LOG: Cache Base = C:\WINDOWS\Micr osoft.NET\Frame work\v2.0.50727 \Temporary
ASP.NET Files\root\5d8a c2d8
LOG: AppName = db79f8c9
Calling assembly : (Unknown).

===
LOG: Start binding of native image RadEditor, Version=5.6.1.0 ,
Culture=neutral , PublicKeyToken= 852c9eb6525c1b5 3.
LOG: IL assembly loaded from
C:\WINDOWS\asse mbly\GAC\RadEdi tor\5.6.1.0__85 2c9eb6525c1b53\ RadEditor.dll.
WRN: No matching native image found.
LOG: Bind to native image assembly did not succeed. Use IL image.


The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified.

Assembly manager loaded from:
C:\WINDOWS\Micr osoft.NET\Frame work\v2.0.50727 \mscorwks.dll
Running under executable c:\windows\syst em32\inetsrv\w3 wp.exe
--- A detailed error log follows.

=== Pre-bind state information ===
LOG: User = Unknown
LOG: DisplayName = RadEditor
(Partial)
LOG: Appbase = file://xxxx/WebData/WebSites/xxxx/xxxx/www/

LOG: Initial PrivatePath = \\xxxx\WebData\ WebSites\xxxx\x xxx\www\bin
LOG: Dynamic Base = C:\WINDOWS\Micr osoft.NET\Frame work\v2.0.50727 \Temporary
ASP.NET Files\root\2b34 cbc5
LOG: Cache Base = C:\WINDOWS\Micr osoft.NET\Frame work\v2.0.50727 \Temporary
ASP.NET Files\root\2b34 cbc5
LOG: AppName = 3fc9d41d
Calling assembly : (Unknown).

===
LOG: This bind starts in default load context.
LOG: Using application configuration file:
\\xxxx\WebData\ WebSites\xxxx\x xxx\www\web.con fig

LOG: Using host configuration file:
\\?\C:\WINDOWS\ Microsoft.Net\F ramework\v2.0.5 0727\aspnet.con fig
LOG: Using machine configuration file from
C:\WINDOWS\Micr osoft.NET\Frame work\v2.0.50727 \config\machine .config.
LOG: Policy not being applied to reference at this time (private, custom,
partial, or location-based assembly bind).
LOG: The same bind was seen before, and was failed with hr = 0x80070002.
ERR: Unrecoverable error occurred during pre-download check (hr = 0x80070002).
--
Stefan Barlow
MCSD.Net
"Walter Wang [MSFT]" wrote:

Hi Stefan,

I can not seem to find any known issue related to DFS. Is it possible for
you to try your web site on a normal folder?
Regards,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[Walter Wang [MSFT]]

Hi Stefan,

Let me summarize the issue so far:

1) You have multiple web servers using a single DFS share (I believe you
already properly configured in IIS to use this DFS share since you need a
specific account to visit that DFS share)

2) Some assemblies used by your web application are installed in GAC; each
web server has auto-update system to update them in GAC and touch
web.config to cause the web application restart

3) This is previously working fine in ASP.NET 1.1; now you've upgraded to
ASP.NET 2.0 and you found touching web.config is not causing the web
application restart and even it did, some assembly will not load correctly
and only restaring the AppPool could solve the problem.

4) You also found that in ASP.NET 2.0, creating a folder in the web site's
root folder will cause the AppDomain restart and you're wondering if
uploading image files will cause it restart too.

Let me know if I've misunderstood or missed anything.

Well, previously in ASP.NET 1.1, touching web.config, global.asax and /bin
will cause AppDomain restart. Now since ASP.NET 2.0 has some special folder
such as App_Code, App_Data, it now will restart the AppDomain whenever a
folder is created/deleted under the root folder. This is by design in 2.0.

#If broken it is, fix it you should : ASP.NET Case Study: Lost session
variables and appdomain recycles
http://blogs.msdn.com/tess/archive/2...dy-lost-sessio
n-variables-and-appdomain-recycles.aspx

I'm not sure why touching web.config didn't cause the AppDomain restart in
your case. As you described, adding a folder will cause it restart,
therefore the FileSystemWatch er (internally used by ASP.NET to detect
changes) should be working correctly. I'm afraid this is an environment
specific scenario that needs live debugging or dump analysis to further
troubleshoot the root cause, which is unfortunately not possible in
newsgroup.

From the fusion log you posted, the problematic assembly is located at your
/bin folder, right? Remeber that updating a file in /bin will cause the
AppDomain restart.

This issue might be related to the 1.1 to 2.0 migration. Have you tried to
create a brand new 2.0 web application following your deployment scenario
and update an assembly used?

At last, I highly recommend you to contact our Customer Support and Service
to fully troubleshooting this issue. With live debugging and dump analysis,
I believe the issue will be more easier and sooner to solve.

Regards,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[=?Utf-8?B?U3RlZmFuIEJhcmxvdw==?=]

1) Correct, the DFS has been in use for these web sites for months.
2) All assemblies are in the GAC. We change the version numbers referenced
in the GAC on the web.config and save it.
3) Correct.
4) Correct. Does creating files / folders under a child folder cause this
as well?

All assemblies are in the GAC, we do not have a /bin folder.

Thanks,
--
Stefan Barlow
MCSD.Net
"Walter Wang [MSFT]" wrote:

Hi Stefan,

Let me summarize the issue so far:

1) You have multiple web servers using a single DFS share (I believe you
already properly configured in IIS to use this DFS share since you need a
specific account to visit that DFS share)

2) Some assemblies used by your web application are installed in GAC; each
web server has auto-update system to update them in GAC and touch
web.config to cause the web application restart

3) This is previously working fine in ASP.NET 1.1; now you've upgraded to
ASP.NET 2.0 and you found touching web.config is not causing the web
application restart and even it did, some assembly will not load correctly
and only restaring the AppPool could solve the problem.

4) You also found that in ASP.NET 2.0, creating a folder in the web site's
root folder will cause the AppDomain restart and you're wondering if
uploading image files will cause it restart too.

Let me know if I've misunderstood or missed anything.

Well, previously in ASP.NET 1.1, touching web.config, global.asax and /bin
will cause AppDomain restart. Now since ASP.NET 2.0 has some special folder
such as App_Code, App_Data, it now will restart the AppDomain whenever a
folder is created/deleted under the root folder. This is by design in 2.0.

#If broken it is, fix it you should : ASP.NET Case Study: Lost session
variables and appdomain recycles
http://blogs.msdn.com/tess/archive/2...dy-lost-sessio
n-variables-and-appdomain-recycles.aspx

I'm not sure why touching web.config didn't cause the AppDomain restart in
your case. As you described, adding a folder will cause it restart,
therefore the FileSystemWatch er (internally used by ASP.NET to detect
changes) should be working correctly. I'm afraid this is an environment
specific scenario that needs live debugging or dump analysis to further
troubleshoot the root cause, which is unfortunately not possible in
newsgroup.

From the fusion log you posted, the problematic assembly is located at your
/bin folder, right? Remeber that updating a file in /bin will cause the
AppDomain restart.

This issue might be related to the 1.1 to 2.0 migration. Have you tried to
create a brand new 2.0 web application following your deployment scenario
and update an assembly used?

At last, I highly recommend you to contact our Customer Support and Service
to fully troubleshooting this issue. With live debugging and dump analysis,
I believe the issue will be more easier and sooner to solve.

Regards,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[Walter Wang [MSFT]]

Hi Stefan,

Sorry for delayed reply.

Following document lists actions that will cause the application restart
(search for section "Applicatio n Restarts"):

#ASP.NET Application Life Cycle Overview
http://msdn2.microsoft.com/en-us/library/ms178473.aspx

However, in your particular case, modifying the Web.Config will not cause
the application restart; which means there must be something wrong. Since
it's not reproducible on my side, I really recommend you to contact
Microsoft Customer Support and Service for further troubleshooting .

To answer you question in 4): if the parent folder is a special one such as
App_Code, creating files/folders will surely make the application restart.

Regards,
Walter Wang (wa****@online. microsoft.com, remove 'online.')
Microsoft Online Community Support

=============== =============== =============== =====
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.