Mirroring tables from Oracle to Postgresql

Hello Miguel,

On Monday 05 April 2004 15:58, Miguel Guzman Miranda wrote:[color=blue]
> My issue is that I need to do some replication from Oracle tables to
> PSQL. One-way only, incremental, and to a remote machine.[/color]

I have to replicate a lot of data from Oracle/MS SQL/Progress to PostgreSQL
I have written an Java library which synchronize the data from a database to
another database and do some usefull things if a record was
inserted/changed/deleted. In our situation, we synchronize once a day.

At the moment I'm rewritting the library to use Metadata.

Regards,
Cees.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

On Mon, Apr 05, 2004 at 03:58:36PM +0200, Miguel Guzman Miranda wrote:[color=blue]
> I've been suggested to do a triggered procedure in Oracle so that
> everytime a file is inserted, deleted or updated, a perl script is run
> which modifies in turn the PSQL DB.[/color]

If you go to gborg and get the latest erserver code, you could
probably use that as a base. It works pretty much as you described.
At the moment, it is designed to use Postgres as the master, but I've
been thinking about it lately, and it strikes me that it could be
modified to use other RDBMS. Since the data is pushed around using
JDBC, I suspect that it could be modified to sync any systems for
which you have a JDBC driver. Some work will be required for that,
though. In particular, it depends on the Postgres system tables, so
you'd have to figure out how to fix that.

Note that it does not do DDL, and it imposes a noticable cost to
transactions on the master database.

A

--
Andrew Sullivan | Join Bytes!

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to Join Bytes!)

The java code is reasonably agnostic, provided the JDBC driver is smart
enough to
deal with any datatype properly as strings, although that intelligence
can be added.
Something else as a slave would be considerable easier, of course. For
something
else to be the master, you would have to find a way to mimic the
trigger code to make
the entry into the _rserve_log_?_ tables.

On Apr 16, 2004, at 8:17 AM, Andrew Sullivan wrote:
[color=blue]
> On Mon, Apr 05, 2004 at 03:58:36PM +0200, Miguel Guzman Miranda wrote:[color=green]
>> I've been suggested to do a triggered procedure in Oracle so that
>> everytime a file is inserted, deleted or updated, a perl script is run
>> which modifies in turn the PSQL DB.[/color]
>
> If you go to gborg and get the latest erserver code, you could
> probably use that as a base. It works pretty much as you described.
> At the moment, it is designed to use Postgres as the master, but I've
> been thinking about it lately, and it strikes me that it could be
> modified to use other RDBMS. Since the data is pushed around using
> JDBC, I suspect that it could be modified to sync any systems for
> which you have a JDBC driver. Some work will be required for that,
> though. In particular, it depends on the Postgres system tables, so
> you'd have to figure out how to fix that.
>
> Note that it does not do DDL, and it imposes a noticable cost to
> transactions on the master database.
>
> A
>
> --
> Andrew Sullivan | Join Bytes!
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to
> Join Bytes!)
>[/color]
--------------------

Andrew Rawnsley
President
The Ravensfield Digital Resource Group, Ltd.
(740) 587-0114
www.ravensfield.com


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Andrew Rawnsley wrote:
[color=blue]
> The java code is reasonably agnostic, provided the JDBC driver is smart
> enough to
> deal with any datatype properly as strings, although that intelligence
> can be added.
> Something else as a slave would be considerable easier, of course. For
> something
> else to be the master, you would have to find a way to mimic the
> trigger code to make
> the entry into the _rserve_log_?_ tables.[/color]

Not only that. I was wondering how exactly a "snapshot" would be defined
in Oracle. I don't know if there are things like minxid, maxxid and xip
in any other DB than PostgreSQL.


Jan
[color=blue]
>
> On Apr 16, 2004, at 8:17 AM, Andrew Sullivan wrote:
>[color=green]
>> On Mon, Apr 05, 2004 at 03:58:36PM +0200, Miguel Guzman Miranda wrote:[color=darkred]
>>> I've been suggested to do a triggered procedure in Oracle so that
>>> everytime a file is inserted, deleted or updated, a perl script is run
>>> which modifies in turn the PSQL DB.[/color]
>>
>> If you go to gborg and get the latest erserver code, you could
>> probably use that as a base. It works pretty much as you described.
>> At the moment, it is designed to use Postgres as the master, but I've
>> been thinking about it lately, and it strikes me that it could be
>> modified to use other RDBMS. Since the data is pushed around using
>> JDBC, I suspect that it could be modified to sync any systems for
>> which you have a JDBC driver. Some work will be required for that,
>> though. In particular, it depends on the Postgres system tables, so
>> you'd have to figure out how to fix that.
>>
>> Note that it does not do DDL, and it imposes a noticable cost to
>> transactions on the master database.
>>
>> A
>>
>> --
>> Andrew Sullivan | Join Bytes!
>>
>> ---------------------------(end of
>> broadcast)---------------------------
>> TIP 2: you can get off all lists at once with the unregister command
>> (send "unregister YourEmailAddressHere" to
>> Join Bytes!)
>>[/color]
> --------------------
>
> Andrew Rawnsley
> President
> The Ravensfield Digital Resource Group, Ltd.
> (740) 587-0114
> www.ravensfield.com
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html[/color]


--
#================================================= =====================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================= = Join Bytes! #


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to Join Bytes! so that your
message can get through to the mailing list cleanly

Andrew Rawnsley wrote:
[color=blue]
> The java code is reasonably agnostic, provided the JDBC driver is smart
> enough to
> deal with any datatype properly as strings, although that intelligence
> can be added.
> Something else as a slave would be considerable easier, of course. For
> something
> else to be the master, you would have to find a way to mimic the
> trigger code to make
> the entry into the _rserve_log_?_ tables.[/color]

Not only that. I was wondering how exactly a "snapshot" would be defined
in Oracle. I don't know if there are things like minxid, maxxid and xip
in any other DB than PostgreSQL.


Jan
[color=blue]
>
> On Apr 16, 2004, at 8:17 AM, Andrew Sullivan wrote:
>[color=green]
>> On Mon, Apr 05, 2004 at 03:58:36PM +0200, Miguel Guzman Miranda wrote:[color=darkred]
>>> I've been suggested to do a triggered procedure in Oracle so that
>>> everytime a file is inserted, deleted or updated, a perl script is run
>>> which modifies in turn the PSQL DB.[/color]
>>
>> If you go to gborg and get the latest erserver code, you could
>> probably use that as a base. It works pretty much as you described.
>> At the moment, it is designed to use Postgres as the master, but I've
>> been thinking about it lately, and it strikes me that it could be
>> modified to use other RDBMS. Since the data is pushed around using
>> JDBC, I suspect that it could be modified to sync any systems for
>> which you have a JDBC driver. Some work will be required for that,
>> though. In particular, it depends on the Postgres system tables, so
>> you'd have to figure out how to fix that.
>>
>> Note that it does not do DDL, and it imposes a noticable cost to
>> transactions on the master database.
>>
>> A
>>
>> --
>> Andrew Sullivan | Join Bytes!
>>
>> ---------------------------(end of
>> broadcast)---------------------------
>> TIP 2: you can get off all lists at once with the unregister command
>> (send "unregister YourEmailAddressHere" to
>> Join Bytes!)
>>[/color]
> --------------------
>
> Andrew Rawnsley
> President
> The Ravensfield Digital Resource Group, Ltd.
> (740) 587-0114
> www.ravensfield.com
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html[/color]


--
#================================================= =====================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================= = Join Bytes! #


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to Join Bytes! so that your
message can get through to the mailing list cleanly