Jet 4 Replication Service Pack

See:

Updated version of the Microsoft Jet 4.0 Service Pack 8
replication
files is available in the Download Center
http://support.microsoft.com/?scid=kb;en-us;321076

This includes the Jet 4 synchronizer.

This allows anyone to do indirect replication, even without
purchasing the Office Developer tools. You don't get Replication
Manager, but ReplMan has never been all that great a UI to managing
replication, anyway.

With mstran40.exe and JRO and/or the TSI Synchronizer, you should be
able to do any Jet indirect synchronization (or Internet synchs, as
mstrai40.exe is also in the update).

This is an important change in MS policy, distributing the
synchronizer outside the developer tools. I have discovered that
it's actually been a around a while, since before Jet 4 SP6, as can
be seen from this article about an error that an earlier fix of
these files addressed:

http://support.microsoft.com/kb/320803/

And, just looking at the first URL, I see it is dated over a year
ago, but I just learned about this in Woody's Office Watch, which
reports it as an update related to the new Office 2003 SP2.

Not sure what's going on here, but it makes it possible for anyone
to do indirect replication without needing to have the Office
developer tools.

--
David W. Fenton http://www.bway.net/~dfenton
dfenton at bway dot net http://www.bway.net/~dfassoc

re: Jet 4 Replication Service Pack

David,

What is indirect replication and where can I get more information about it?

Thanks!

Steve


"David W. Fenton" <dXXXfenton@bway.net.invalid> wrote in message
news:Xns96E4DB08F1394dfentonbwaynetinvali@216.196. 97.142...[color=blue]
> See:
>
> Updated version of the Microsoft Jet 4.0 Service Pack 8
> replication
> files is available in the Download Center
> http://support.microsoft.com/?scid=kb;en-us;321076
>
> This includes the Jet 4 synchronizer.
>
> This allows anyone to do indirect replication, even without
> purchasing the Office Developer tools. You don't get Replication
> Manager, but ReplMan has never been all that great a UI to managing
> replication, anyway.
>
> With mstran40.exe and JRO and/or the TSI Synchronizer, you should be
> able to do any Jet indirect synchronization (or Internet synchs, as
> mstrai40.exe is also in the update).
>
> This is an important change in MS policy, distributing the
> synchronizer outside the developer tools. I have discovered that
> it's actually been a around a while, since before Jet 4 SP6, as can
> be seen from this article about an error that an earlier fix of
> these files addressed:
>
> http://support.microsoft.com/kb/320803/
>
> And, just looking at the first URL, I see it is dated over a year
> ago, but I just learned about this in Woody's Office Watch, which
> reports it as an update related to the new Office 2003 SP2.
>
> Not sure what's going on here, but it makes it possible for anyone
> to do indirect replication without needing to have the Office
> developer tools.
>
> --
> David W. Fenton http://www.bway.net/~dfenton
> dfenton at bway dot net http://www.bway.net/~dfassoc[/color]

re: Jet 4 Replication Service Pack

"PC Datasheet" <nospam@nospam.spam> wrote in
news:eal0f.8072$QE1.5965@newsread2.news.atl.earthl ink.net:
[color=blue]
> What is indirect replication and where can I get more information
> about it?[/color]

In answering, I'll assume you already know what Jet replication is
and what it does.

Jet replication via the Access menus is intended for use over a LAN,
because when it synchronizes two replicas, it opens both of them.

If you are remote from the replica you are synchronizing with, you
are likely to have much-reduced bandwidth, which makes pulling the
entire remote replica across the wire not only a performance
bottleneck, but also rather dangerous, because of the possibility of
a droppeed connection, which could corrupt the remote database.

That kind of replication is called "direct replication."

To address the problems of direct replication, Microsoft created two
alternative synchronization methods, indirect and Internet
replication. Both work by using a process running on either end of
the synchronization that opens the replica locally, and all
communication is between the two synchronizers. Indirect replication
uses MSTRAN>>.EXE and Internet replication use MSTRAIXX.EXE (where
XX is the Jet version, 40 for Jet 4, 35 for 3.5).

Indirect replication uses a dropbox on each end that is accessible
to the remote synchronizer. If you've got ReplicaA and ReplicaB,
ReplicaA has DropboxA and ReplicaB has DropBoxB. SynchronizerA
communicates with SynchronizerB and determines what generations need
to be sent (based on each synchronizer reading its local replica's
replication tables). SymchronizerA will then drop message files and
small MDBs into DropBoxB that include all the changes in ReplicaA
that ReplicaB does not have. SynchronizerB then applies the data in
DropBoxB to ReplicaB, then places files in DropBoxA for
SynchronizerA to apply to ReplicaA. This is a reiterative process,
because certain changes have to be successful before certain others
can happen (e.g., a parent record must be successfully inserted
before a child record attached to it can be inserted), and so you
can have a cycle of several round-trips between the two
synchronizers before the replicas reach a state where they have
identical data.

The advantage of indirect replication over direct is that the only
data that goes across the wire is the changes (actually, the data
necessary to apply the changes, which is going to be greater than
the content of the changes themselves, but substantially less than
if you pulled the whole remote replica across the wire). A direct
replication over a slow connection can take hours, assuming it
succeeds. An indirect synch of the same data usually takes less than
10 minutes, and usually substantially less (when replicas are
synching regularly, e.g., daily, it often takes 2 or 3 minutes, but
it depends on the number and type of changes being synched).

Internet replication does basically the same thing, but uses FTP to
communicate, rather than the SMB networking that indirect
replication is based on. The problem from my point of view with
Internet replication is that it has hardwired dependencies on IIS --
you must use the IIS FTP service for it. You can't use some other
FTP service. And it requires that IIS (to support the FTP service)
be running on all PCs that do the synchronizing. That means that if
you want to set up laptops to synch with the mothership via Internet
replication, those laptops have to have IIS installed and running.

To me, that's an unacceptable requirement because:

1. it's an additional administrative requirement that is hard to
manage remotely.

2. IIS is the most unsafe widely-used HTTP server out there. Yes, MS
has addressed a lot of the problems that were there that led to the
worms that propagated so widely a couple of years ago, but I'm not
convinced. Apache is a much safer and more secure HTTP server that
is not intimately hardwired into the OS, but you can't use it, or
any alternative FTP server.

3. because of the dependencies outside of Access and outside of file
system permissions, you now have a dependency on a monolithic
component that really oughtn't be running on workstations at all.
That dependency makes Internet replication particularly fragile,
since not only can file access configuration issues cause problems
(as they also can with indirect replication), but you also add in
the problems of properly configuring the IIS/FTP services, which,
from the history of problems we've seen reported in
microsoft.public.access.replication, is not a triviality at all.

Now, the original problem with indirect replication was that it
required an SMB networking connection. That meant the Internet was
out of the question -- you either needed a WAN that was NETBios
end-to-end, or you needed some form of dialup networking (which was
always fine for indirect replication, even at 28.8). This meant
dialin servers and other administrative headaches and expenses in
order to support many remote users.

However, these days with VPNs being widely in use and so easy to set
up from within Windows, the solution for indirect replication is to
run it across the Internet over a VPN. It's not terribly difficult
once the VPN infrastructure is in place (as it already is in a large
number of offices).

--
David W. Fenton http://www.bway.net/~dfenton
dfenton at bway dot net http://www.bway.net/~dfassoc

re: Jet 4 Replication Service Pack

Thanks, David! I really appreciate the indepth discussion.

Steve


"David W. Fenton" <dXXXfenton@bway.net.invalid> wrote in message
news:Xns96E597C49590Adfentonbwaynetinvali@216.196. 97.142...[color=blue]
> "PC Datasheet" <nospam@nospam.spam> wrote in
> news:eal0f.8072$QE1.5965@newsread2.news.atl.earthl ink.net:
>[color=green]
>> What is indirect replication and where can I get more information
>> about it?[/color]
>
> In answering, I'll assume you already know what Jet replication is
> and what it does.
>
> Jet replication via the Access menus is intended for use over a LAN,
> because when it synchronizes two replicas, it opens both of them.
>
> If you are remote from the replica you are synchronizing with, you
> are likely to have much-reduced bandwidth, which makes pulling the
> entire remote replica across the wire not only a performance
> bottleneck, but also rather dangerous, because of the possibility of
> a droppeed connection, which could corrupt the remote database.
>
> That kind of replication is called "direct replication."
>
> To address the problems of direct replication, Microsoft created two
> alternative synchronization methods, indirect and Internet
> replication. Both work by using a process running on either end of
> the synchronization that opens the replica locally, and all
> communication is between the two synchronizers. Indirect replication
> uses MSTRAN>>.EXE and Internet replication use MSTRAIXX.EXE (where
> XX is the Jet version, 40 for Jet 4, 35 for 3.5).
>
> Indirect replication uses a dropbox on each end that is accessible
> to the remote synchronizer. If you've got ReplicaA and ReplicaB,
> ReplicaA has DropboxA and ReplicaB has DropBoxB. SynchronizerA
> communicates with SynchronizerB and determines what generations need
> to be sent (based on each synchronizer reading its local replica's
> replication tables). SymchronizerA will then drop message files and
> small MDBs into DropBoxB that include all the changes in ReplicaA
> that ReplicaB does not have. SynchronizerB then applies the data in
> DropBoxB to ReplicaB, then places files in DropBoxA for
> SynchronizerA to apply to ReplicaA. This is a reiterative process,
> because certain changes have to be successful before certain others
> can happen (e.g., a parent record must be successfully inserted
> before a child record attached to it can be inserted), and so you
> can have a cycle of several round-trips between the two
> synchronizers before the replicas reach a state where they have
> identical data.
>
> The advantage of indirect replication over direct is that the only
> data that goes across the wire is the changes (actually, the data
> necessary to apply the changes, which is going to be greater than
> the content of the changes themselves, but substantially less than
> if you pulled the whole remote replica across the wire). A direct
> replication over a slow connection can take hours, assuming it
> succeeds. An indirect synch of the same data usually takes less than
> 10 minutes, and usually substantially less (when replicas are
> synching regularly, e.g., daily, it often takes 2 or 3 minutes, but
> it depends on the number and type of changes being synched).
>
> Internet replication does basically the same thing, but uses FTP to
> communicate, rather than the SMB networking that indirect
> replication is based on. The problem from my point of view with
> Internet replication is that it has hardwired dependencies on IIS --
> you must use the IIS FTP service for it. You can't use some other
> FTP service. And it requires that IIS (to support the FTP service)
> be running on all PCs that do the synchronizing. That means that if
> you want to set up laptops to synch with the mothership via Internet
> replication, those laptops have to have IIS installed and running.
>
> To me, that's an unacceptable requirement because:
>
> 1. it's an additional administrative requirement that is hard to
> manage remotely.
>
> 2. IIS is the most unsafe widely-used HTTP server out there. Yes, MS
> has addressed a lot of the problems that were there that led to the
> worms that propagated so widely a couple of years ago, but I'm not
> convinced. Apache is a much safer and more secure HTTP server that
> is not intimately hardwired into the OS, but you can't use it, or
> any alternative FTP server.
>
> 3. because of the dependencies outside of Access and outside of file
> system permissions, you now have a dependency on a monolithic
> component that really oughtn't be running on workstations at all.
> That dependency makes Internet replication particularly fragile,
> since not only can file access configuration issues cause problems
> (as they also can with indirect replication), but you also add in
> the problems of properly configuring the IIS/FTP services, which,
> from the history of problems we've seen reported in
> microsoft.public.access.replication, is not a triviality at all.
>
> Now, the original problem with indirect replication was that it
> required an SMB networking connection. That meant the Internet was
> out of the question -- you either needed a WAN that was NETBios
> end-to-end, or you needed some form of dialup networking (which was
> always fine for indirect replication, even at 28.8). This meant
> dialin servers and other administrative headaches and expenses in
> order to support many remote users.
>
> However, these days with VPNs being widely in use and so easy to set
> up from within Windows, the solution for indirect replication is to
> run it across the Internet over a VPN. It's not terribly difficult
> once the VPN infrastructure is in place (as it already is in a large
> number of offices).
>
> --
> David W. Fenton http://www.bway.net/~dfenton
> dfenton at bway dot net http://www.bway.net/~dfassoc[/color]

re: Jet 4 Replication Service Pack

David W. Fenton wrote:[color=blue]
> 2. IIS is the most unsafe widely-used HTTP server out there. Yes, MS
> has addressed a lot of the problems that were there that led to the
> worms that propagated so widely a couple of years ago, but I'm not
> convinced. Apache is a much safer and more secure HTTP server that
> is not intimately hardwired into the OS, but you can't use it, or
> any alternative FTP server.
> ...
> --
> David W. Fenton http://www.bway.net/~dfenton
> dfenton at bway dot net http://www.bway.net/~dfassoc[/color]

Mike Shelton from Microsoft stated that SQL Server 2005 does not need
to use IIS. Perhaps I am taking this out of context. If true, would
that mean that replication using SQL Server 2005 will not be as
problematic as it was with earlier versions?

James A. Fortune

re: Jet 4 Replication Service Pack

jimfortune@compumarc.com wrote in
news:1128458288.899658.35680@z14g2000cwz.googlegro ups.com:
[color=blue]
> David W. Fenton wrote:[color=green]
>> 2. IIS is the most unsafe widely-used HTTP server out there. Yes,
>> MS has addressed a lot of the problems that were there that led
>> to the worms that propagated so widely a couple of years ago, but
>> I'm not convinced. Apache is a much safer and more secure HTTP
>> server that is not intimately hardwired into the OS, but you
>> can't use it, or any alternative FTP server.[/color]
>
> Mike Shelton from Microsoft stated that SQL Server 2005 does not
> need to use IIS. Perhaps I am taking this out of context. If
> true, would that mean that replication using SQL Server 2005 will
> not be as problematic as it was with earlier versions?[/color]

I haven't a clue. I know zilch about SQL Server replication -- I'm
talking about Jet replication, one type of which (Internet
replication) is hardwired to use IIS's FTP services.

SQL Server is wholly unrelated.

--
David W. Fenton http://www.bway.net/~dfenton
dfenton at bway dot net http://www.bway.net/~dfassoc

re: Jet 4 Replication Service Pack

David W. Fenton wrote:[color=blue]
> jimfortune@compumarc.com wrote in
> news:1128458288.899658.35680@z14g2000cwz.googlegro ups.com:
>[color=green]
> > David W. Fenton wrote:[color=darkred]
> >> 2. IIS is the most unsafe widely-used HTTP server out there. Yes,
> >> MS has addressed a lot of the problems that were there that led
> >> to the worms that propagated so widely a couple of years ago, but
> >> I'm not convinced. Apache is a much safer and more secure HTTP
> >> server that is not intimately hardwired into the OS, but you
> >> can't use it, or any alternative FTP server.[/color]
> >
> > Mike Shelton from Microsoft stated that SQL Server 2005 does not
> > need to use IIS. Perhaps I am taking this out of context. If
> > true, would that mean that replication using SQL Server 2005 will
> > not be as problematic as it was with earlier versions?[/color]
>
> I haven't a clue. I know zilch about SQL Server replication -- I'm
> talking about Jet replication, one type of which (Internet
> replication) is hardwired to use IIS's FTP services.
>
> SQL Server is wholly unrelated.
>
> --
> David W. Fenton http://www.bway.net/~dfenton
> dfenton at bway dot net http://www.bway.net/~dfassoc[/color]

I know zilch about SQL Server replication also, but that implies I
don't know if it's wholly unrelated or not. Perhaps someone can
enlighten us. Are they related? Is there a URL that explains the
differences? A friend of mine uses Jet replication a lot. After
seeing some of the nasty problems Jet replication can cause I'm sure
I'll never want to use any kind of Jet replication until better ways
are found to implement it. Perhaps you have encountered fewer
problems. I realize that MS providing the capability of replication in
Access was a difficult problem in general. Do you really recommend any
kind of Jet replication? Do you know of any companies that are using
it without having lots of problems?

Thanks in advance,

James A. Fortune

re: Jet 4 Replication Service Pack

jimfortune@compumarc.com wrote in
news:1128486255.673131.248790@z14g2000cwz.googlegr oups.com:
[color=blue]
> David W. Fenton wrote:[color=green]
>> jimfortune@compumarc.com wrote in
>> news:1128458288.899658.35680@z14g2000cwz.googlegro ups.com:
>>[color=darkred]
>> > David W. Fenton wrote:
>> >> 2. IIS is the most unsafe widely-used HTTP server out there.
>> >> Yes, MS has addressed a lot of the problems that were there
>> >> that led to the worms that propagated so widely a couple of
>> >> years ago, but I'm not convinced. Apache is a much safer and
>> >> more secure HTTP server that is not intimately hardwired into
>> >> the OS, but you can't use it, or any alternative FTP server.
>> >
>> > Mike Shelton from Microsoft stated that SQL Server 2005 does
>> > not need to use IIS. Perhaps I am taking this out of context.
>> > If true, would that mean that replication using SQL Server 2005
>> > will not be as problematic as it was with earlier versions?[/color]
>>
>> I haven't a clue. I know zilch about SQL Server replication --
>> I'm talking about Jet replication, one type of which (Internet
>> replication) is hardwired to use IIS's FTP services.
>>
>> SQL Server is wholly unrelated.[/color]
>
> I know zilch about SQL Server replication also, but that implies I
> don't know if it's wholly unrelated or not. Perhaps someone can
> enlighten us. Are they related? . . .[/color]

Not in any real way. However, my understanding is that some of the
people on the Jet replication team migrated over to SQL Server to
implement replication there. That is, Jet replication came first,
and SQL Server replication was added later. SQL Server replication
started out as one-way replication, the kind provided by MySQL and
PostgreSQL, which is in database servers for the purpose of
providing redundancy and load balancing, not to support editing at
multiple sites. Merge replication was added in the version of SQL
Server after basic replication was implmeneted, and I think it was
in SQL Server 7 (along with the introduction of Jet 4) that
heterogenous replication was implemented, allowing a Jet database to
participate as a synch partner in a SQL Server replication set.

They have certain things in common because both are replication, so,
of necessity, certain concepts and structures will be the same. But
other than that, they are very different. In many respects, Jet
replication is more flexible and powerful than SQL Server
replication, though SQL Server does offer transactional replication
(which Jet never can), which is better, seems to me, than anything
Jet can offer.

But merge replication on any platform is a complex operation, and
not easy or simple to implement.
[color=blue]
> . . . Is there a URL that explains the
> differences? . . .[/color]

I don't know of any. Just googling a bit, I can't find anything
that's relevant as a comparison.
[color=blue]
> . . . A friend of mine uses Jet replication a lot. After
> seeing some of the nasty problems Jet replication can cause I'm
> sure I'll never want to use any kind of Jet replication until
> better ways are found to implement it. . . .[/color]

Microsoft has basically abandoned Jet replication -- there are never
going to be any "better ways" to do it, just as there is unlikely to
ever be a better version of Jet.

It will be interesting to see if MS continues to provide replication
capability in the new Office 12 database format. My understanding of
what I've read about it is that the big change is not in Jet, but in
the Access MDB format, which will now separate out the data tables
from the Access project, with the project no longer stored in a Jet
database at all (but I could be wrong in that reading). If that's
correct, that would mean that you'd no longer be able to replicate
the front end, which is no loss, because, for all practical
purposes, you can't do that now (doing so is unstalbe and eventually
leads to corruption and loss of the whole project in almost all
cases).
[color=blue]
> . . . Perhaps you have
> encountered fewer problems. . . .[/color]

Replication is difficult, but I haven't run into significant
problems with it that are insurmountable. You just have to know how
to do it right, and most people haven't taken the time to learn. Of
course, I can't really blame them, because Microsoft has never
properly documented replication best practices. The FAQs are filled
with misleading marketing speak and claims for replication that
aren't really justifiable.

The other reason replication is hard is because it has a number of
dependencies outside Access. That has never been an issue for me as
I make my living both as an Access developer and as a system
administrator, so I have the NT administration and networking skills
necessary to manage end-to-end replication scenarios. Most people
wanting to implement replication don't have that background, or in
many cases, even understand that they *need* that kind of knowledge.
[color=blue]
> . . . I realize that MS providing the
> capability of replication in Access was a difficult problem in
> general. Do you really recommend any kind of Jet replication? Do
> you know of any companies that are using it without having lots of
> problems?[/color]

My clients using replication don't have lots of problems. Now,
administering a replicated database is more involved than
administering non-replicated databases, but that's a difference
between *no adminstration* (for non-replicated) and *some*
management required.

I don't recommend Jet replication except for one scenario, and
that's the disconnected laptops scenario. For fixed remote offices,
I recommend Terminal Server hosting instead. But in 1998 when my
first trans-Atlantic replication client went live with their
database (with offices in New York and London, and two telecommuting
subcontractors), Terminal Server was way too expensive an option in
comparison to Jet replication. That has now changed.

But for laptops in the field that need to have editable data when
not connected to the Internet, Jet replication works just fine. And
if they are going back to the main office and can synch via a LAN,
it's not even hard to set up and administer.

Replication is complicated, yes.

But it is not dangerous or unstable if properly implemented and
properly cared for. The problem is that most people don't know that
extra management is required for jet replication and because of
that, they make all the mistakes that cause things to go south.

But that's the case with Access in general -- it provides powerful
functionality that any number of novices often mis-use. That doesn't
mean there's something wrong with Access or with the basic
technologies. Indeed, it means that Access is remarkably forgiving
of error in even allowing it to work WRONG.

Jet replication works, and there is nothing comparably powerful in
any desktop database engine that I am aware of. Indeed, it's more
powerful than a lot of server databases. With power comes
complexity, and it's complexity that many people don't realize is
there until they've made mistakes in setting up Jet replication.

That's too bad, but I'd hardly blame that on Microsoft for
implementing Jet replication.

--
David W. Fenton http://www.bway.net/~dfenton
dfenton at bway dot net http://www.bway.net/~dfassoc

re: Jet 4 Replication Service Pack

David,

Do you know of a resource that would explain how to set up the Jet
Synchronizer for indirect synchronization without using ReplMan or TSI
synchronizer? I know how to use code to initiate an indirect synchronization
but how do I set up the drop box locations and let the Jet Synchronizer know
about them?

Thanks!

"David W. Fenton" <dXXXfenton@bway.net.invalid> wrote in message
news:Xns96E4DB08F1394dfentonbwaynetinvali@216.196. 97.142...[color=blue]
> See:
>
> Updated version of the Microsoft Jet 4.0 Service Pack 8
> replication
> files is available in the Download Center
> http://support.microsoft.com/?scid=kb;en-us;321076
>
> This includes the Jet 4 synchronizer.
>
> This allows anyone to do indirect replication, even without
> purchasing the Office Developer tools. You don't get Replication
> Manager, but ReplMan has never been all that great a UI to managing
> replication, anyway.
>
> With mstran40.exe and JRO and/or the TSI Synchronizer, you should be
> able to do any Jet indirect synchronization (or Internet synchs, as
> mstrai40.exe is also in the update).
>
> This is an important change in MS policy, distributing the
> synchronizer outside the developer tools. I have discovered that
> it's actually been a around a while, since before Jet 4 SP6, as can
> be seen from this article about an error that an earlier fix of
> these files addressed:
>
> http://support.microsoft.com/kb/320803/
>
> And, just looking at the first URL, I see it is dated over a year
> ago, but I just learned about this in Woody's Office Watch, which
> reports it as an update related to the new Office 2003 SP2.
>
> Not sure what's going on here, but it makes it possible for anyone
> to do indirect replication without needing to have the Office
> developer tools.
>
> --
> David W. Fenton http://www.bway.net/~dfenton
> dfenton at bway dot net http://www.bway.net/~dfassoc[/color]

re: Jet 4 Replication Service Pack

"Tom Stoddard" <tstoddard@andrewspaper.com> wrote in
news:OnckHYpyFHA.2552@TK2MSFTNGP10.phx.gbl:
[color=blue]
> Do you know of a resource that would explain how to set up the Jet
> Synchronizer for indirect synchronization without using ReplMan or
> TSI synchronizer? I know how to use code to initiate an indirect
> synchronization but how do I set up the drop box locations and let
> the Jet Synchronizer know about them?[/color]

Nope. I've always used ReplMan for it.

It's probanly just registry keys that need to be set, but I don't
know that it's documented anywhere, and that it's the same between
versions of ReplMan.

Have you looked through JRO to see if there are methods for setting
dropbox locations? I can't see how MS could drop ReplMan in the
Visual Studio Office Developer tools if only ReplMan could be used
to set these things.

--
David W. Fenton http://www.bway.net/~dfenton
dfenton at bway dot net http://www.bway.net/~dfassoc