Our team has just fixed a really nasty problem that appears to be
caused by an obscure bug in the sort method of Arraylist and
daisy-chained webservice stubs.
We've fixed the bug so I'll not investigate any further but there was
nothing about it when googling the newsgroups so I am posting a note in
case it helps somebody else out.
The problem was in a system with a web front end calling through to a
chain of 3 webservices i.e.
WebUI->WebServiceA->WebServiceB->WebServiceC
A Business object that exposed an arraylist of other bus. objects was
exposed in WebServiceC with stubs referenced by WebService A, B and
WebUI in a daisy-chain.
This arraylist was being custom sorted in WebServiceC using
Arraylist.Sort() and an IComparer interface implemented in the objects
making up the list. This code worked fine in production in Dec 2004
but, with no code changes to this part of the code we have had problems
with the August 2005 release.
The arraylist was being returned to the UI with overwritten entries,
missing entries and not sorted as expected. Only occasionally, not
faithfully reproducible and not reproducible in the development
environment, only system & UAT.
Tracing showed that the problem was not in code at all as we dumped the
arraylist just before the return call from B and just after the return
to A. It was happening in the serialisation/deserialisation of the
business object result between WebserviceB and WebserviceA. Obviously
we didn't think that the problem was in something we were doing in
WebserviceC but that appears to be the case from the fix.
In the end we found that the problem went away only if we used sort on
a new instance of an arraylist in WebServiceC (after an item by item
copy) and did an item by item copy back into the target ArrayList.
For lack of a reasonable answer it appears that the arraylist was
trying to "remember" its sort and serialisation was trying to do
something clever between Webservice B & A but because the IComparer
interface was not implemented in the stub between B & A things went
awol.
Sounds like pure fantasy but the "fix" did nothing special besides
sorting a separate arraylist instance and copying the items back in. We
didn't solve the problem until we started grasping at straws so if you
are having a similar weird problem this post might help.