Keeping files synced between multiple computers

[kardon33]

Hello Bytes,

For reference I'm running Linux Debian x86 but the problem is not related to an architecture.

I have a networked system with multiple computers (say 20-50). On each computer there is a directory that needs to be consistent with all other nodes in the system. Each computer has the privileges to make changes to the directory and those changes need to be reflected across the entire system over some amount of time.

I first tried to use timestamps in the directory to compare each computers directory to. However that caused errors because clocks were out of sync.

Does anyone have any methods either conceptual ideas or actual programs that could implement the syncing.

Thanks for any help.

[jkmyoung]

I'm not sure at what level you're saying trying to keep the dbs synchronized.

Normally, there is a some sort of field in the db that states the change number. When anyone makes a change to the db, they lock the db, read the number and increment it, sending update messages to any listeners. Everyone keeps their own local copies of these change numbers.

Now when the listener gets the message, they know they have to update that table, or that row in the table, and update their own local change number.

If a device somehow misses the update message, the next time they want to read the db, they will notice that change number has increased, so they have to update themselves before they make any changes.

Not sure if this is what you're looking for.

[kardon33]

Yes that is close to what im looking for. Thanks

If a device somehow misses the update message, the next time they want to read the db, they will notice that change number has increased, so they have to update themselves before they make any changes.


One question though, if a device misses an update message how do they know that the number has increased when making a change?

[jkmyoung]

It depends on when you check for updates.
It might be every 5 seconds, it might be only when the data is requested for read, or when requested for write.

[kardon33]

@jkmyoung
What I meant was, does the local device compare its local database increment number to the other remote dbs. If it has a lower value it assumes it needs to update its database.

[jkmyoung]

I misunderstood; what I described was a centralized system with a main db.

Haven't done much work with a non-centralized 'cloud' system, due to the inherent complications with collisions. If there is no main db, then you need to define what happens in the following scenarios among others:

1. Trying to update a db, but cannot reach one of the other databases to lock the specific table.
2. Db gets locked, but then the request never comes in to unlock it. Is there a time-out on the lock?