IE 5.0 SP2: Redirect form to another frame in modal dialog

Using window.showModalDialog, I open a frameset comprising two frames.
The top frame contains a form, the target of the form is the bottom
frame. If the user has made an error filling out the form, then the
bottom frame will show the errors. If there is no error, the modal
dialog will be closed.

In IE 6, the form target works as intended, and the error messages
show up in the bottom frame.

In IE 5.0, using window.open instead of window.showModalDialog, it
also works.

But in IE 5.0, using window.showModalDialog, the error messages show
up in the top frame, replacing the form.

Is there a way to make IE 5.0 behave like IE 6?

If not, how do I distinguish between the browsers so that I can use
the right behavior for each one? (A workaround would be that the
search form redisplays itself on error, with error messages appended.
But I can't use that approach in IE 6 because IE 6 does not allow the
form to redraw itself. IE6 opens a new window in that case.)

Kai