Database programminf question...

[Andrew Meador]

I am working on an application that will store data in SQL Server
2005 Express. The database will reside on our server at the office. I
need to write an application that can work with this data off site
during the day and then synch the changes back to the database at the
end of the day (or next morning).

I have been reading up on ADO.NET and see that you can use a
DataSet and a DataAdaptor to manage change tracking and updating of
the database with the changes. However, what I have read seems to
imply that this is all done in RAM. The laptop that will go out during
the day needs to be able to safeguard the changes to file, so in case
the laptop goes off, needs restarted, or whatever, the data (and
specifically the changes to the data) will not be lost. But, in the
end, I still need to push these changes back to the database when the
network is available again.

We are operating in an area that does not permit cellular based
internet access, so we have to do this based on no network access,
except for in the morning before going out, and later when we return
to the office.

I was wondering about having two DataAdaptors to work with the one
DataSet; one to pull the data from the database into the DataSet, and
another one to push/pull the data from to DataSet to XML files stored
on the laptop (the working set while out of the office). I can see how
to deal with reading and writing to the XML files and how to pull the
data from the database into the DataSet, but if the program closes and
thus the DataAdaptor that handles the link between the database and
the DataSet is killed, how can I get the changes abck to the database?

Another question here is how well the overall idea will work
performance wise. We will have about a dozen tables that need to be
worked with, two of which will have as many as 10,000 records each (at
~1.5KB per record). If these would have to be completely re-written
with every change I'd imagine this would get to be pretty slow. The
laptop is brand new with 2GB RAM, 120GB Hard Drive, and 2.2GHz Intel®
Core® 2 Duo Processor, so maybe it wouldn't be tooo bad, but...

I am looking for ideas here. If you have a better appraoch that has
nothing to do with XML that's fine, just so long as it is not costly,
crazy complex, or something I can't do with VS 2005 Pro and SQL Server
2005 Express (can't afford SQL Server 2005 Standard) ;) You know what
I mean...

Thanks in advance!!!

[Michael C]

"Andrew Meador" <am******@hotmail.comwrote in message
news:11**********************@v3g2000hsg.googlegro ups.com...

Whatever solution you come up with will have to store the data on the HDD of
the laptop somehow. Just pick your method
1) XML
2) SQLExpress
3) MS Access
4) Some other method

Seeing it's free, I'd go with no 2 myself. It's also got replication
abilities built in, although these might be restricted for the free version.
You still get the full speed and flexibility of sqlserver and can execute
all your usual sql statements and use integrity, stored procs etc.

[=?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=]

If you are working with simple data that is being added to datatables in a
dataset, you can simply use the DataSet.WriteXml and ReadXml methods to store
and retrieve the contents of your dataset to an xml file.

-- Peter
Recursion: see Recursion
site: http://www.eggheadcafe.com
unBlog: http://petesbloggerama.blogspot.com
BlogMetaFinder: http://www.blogmetafinder.com

"Andrew Meador" wrote:

I am working on an application that will store data in SQL Server
2005 Express. The database will reside on our server at the office. I
need to write an application that can work with this data off site
during the day and then synch the changes back to the database at the
end of the day (or next morning).

I have been reading up on ADO.NET and see that you can use a
DataSet and a DataAdaptor to manage change tracking and updating of
the database with the changes. However, what I have read seems to
imply that this is all done in RAM. The laptop that will go out during
the day needs to be able to safeguard the changes to file, so in case
the laptop goes off, needs restarted, or whatever, the data (and
specifically the changes to the data) will not be lost. But, in the
end, I still need to push these changes back to the database when the
network is available again.

We are operating in an area that does not permit cellular based
internet access, so we have to do this based on no network access,
except for in the morning before going out, and later when we return
to the office.

I was wondering about having two DataAdaptors to work with the one
DataSet; one to pull the data from the database into the DataSet, and
another one to push/pull the data from to DataSet to XML files stored
on the laptop (the working set while out of the office). I can see how
to deal with reading and writing to the XML files and how to pull the
data from the database into the DataSet, but if the program closes and
thus the DataAdaptor that handles the link between the database and
the DataSet is killed, how can I get the changes abck to the database?

Another question here is how well the overall idea will work
performance wise. We will have about a dozen tables that need to be
worked with, two of which will have as many as 10,000 records each (at
~1.5KB per record). If these would have to be completely re-written
with every change I'd imagine this would get to be pretty slow. The
laptop is brand new with 2GB RAM, 120GB Hard Drive, and 2.2GHz Intel®
Core® 2 Duo Processor, so maybe it wouldn't be tooo bad, but...

I am looking for ideas here. If you have a better appraoch that has
nothing to do with XML that's fine, just so long as it is not costly,
crazy complex, or something I can't do with VS 2005 Pro and SQL Server
2005 Express (can't afford SQL Server 2005 Standard) ;) You know what
I mean...

Thanks in advance!!!

[Andrew Meador]

Does writing the data to the XML files on the laptop retain the
original and changed states of the data so that it can be updated to
the server later? Also, I would have to create a new DataSet instance
and DataAdaptor instance to read the XLM file and push them back to
the database, will this work? All of the status is retained in the XML
files allowing them to be read back in without loosing any database
update potential?

Also, another thought I had, somewhat related to Michael C's post,
was: Is there a way to put an instance of SQL Server 2005 Express on
the laptop and have the DataSets/DataAdaptors be able to pull the data
from the server SQL instance and be able to maintain their state in
the laptop SQL instance in a way that would allow using the DataSet/
DataAdaptor (or another set) to finally update the changes back to the
server SQL instance? I would think that would be the faster more
efficient way (as far as using the data on the laptop), but I'm
inclined to think that the DataSet/DataAdaptors are not able to handle
that as they currently are designed. Am I wrong?

Thanks!

On Oct 30, 8:36 pm, Peter Bromberg [C# MVP]
<[e-mail] :

If you are working with simple data that is being added to datatables in a
dataset, you can simply use the DataSet.WriteXml and ReadXml methods to store
and retrieve the contents of your dataset to an xml file.

-- Peter
Recursion: see Recursion
site: http://www.eggheadcafe.com
unBlog: http://petesbloggerama.blogspot.com
BlogMetaFinder: http://www.blogmetafinder.com

"Andrew Meador" wrote:

I am working on an application that will store data in SQL Server
2005 Express. The database will reside on our server at the office. I
need to write an application that can work with this data off site
during the day and then synch the changes back to the database at the
end of the day (or next morning).

I have been reading up on ADO.NET and see that you can use a
DataSet and a DataAdaptor to manage change tracking and updating of
the database with the changes. However, what I have read seems to
imply that this is all done in RAM. The laptop that will go out during
the day needs to be able to safeguard the changes to file, so in case
the laptop goes off, needs restarted, or whatever, the data (and
specifically the changes to the data) will not be lost. But, in the
end, I still need to push these changes back to the database when the
network is available again.

We are operating in an area that does not permit cellular based
internet access, so we have to do this based on no network access,
except for in the morning before going out, and later when we return
to the office.

I was wondering about having two DataAdaptors to work with the one
DataSet; one to pull the data from the database into the DataSet, and
another one to push/pull the data from to DataSet to XML files stored
on the laptop (the working set while out of the office). I can see how
to deal with reading and writing to the XML files and how to pull the
data from the database into the DataSet, but if the program closes and
thus the DataAdaptor that handles the link between the database and
the DataSet is killed, how can I get the changes abck to the database?

Another question here is how well the overall idea will work
performance wise. We will have about a dozen tables that need to be
worked with, two of which will have as many as 10,000 records each (at
~1.5KB per record). If these would have to be completely re-written
with every change I'd imagine this would get to be pretty slow. The
laptop is brand new with 2GB RAM, 120GB Hard Drive, and 2.2GHz Intel®
Core® 2 Duo Processor, so maybe it wouldn't be tooo bad, but...

I am looking for ideas here. If you have a better appraoch that has
nothing to do with XML that's fine, just so long as it is not costly,
crazy complex, or something I can't do with VS 2005 Pro and SQL Server
2005 Express (can't afford SQL Server 2005 Standard) ;) You know what
I mean...

Thanks in advance!!!- Hide quoted text -

- Show quoted text -

[Michael C]

"Andrew Meador" <am******@hotmail.comwrote in message
news:11*********************@o3g2000hsb.googlegrou ps.com...
Does writing the data to the XML files on the laptop retain the
original and changed states of the data so that it can be updated to
the server later? Also, I would have to create a new DataSet instance
and DataAdaptor instance to read the XLM file and push them back to
the database, will this work? All of the status is retained in the XML
files allowing them to be read back in without loosing any database
update potential?

Also, another thought I had, somewhat related to Michael C's post,
was: Is there a way to put an instance of SQL Server 2005 Express on
the laptop and have the DataSets/DataAdaptors be able to pull the data
from the server SQL instance and be able to maintain their state in
the laptop SQL instance in a way that would allow using the DataSet/
DataAdaptor (or another set) to finally update the changes back to the
server SQL instance? I would think that would be the faster more
efficient way (as far as using the data on the laptop), but I'm
inclined to think that the DataSet/DataAdaptors are not able to handle
that as they currently are designed. Am I wrong?
What about this. Add a column, say UpdateTime, to your database which stores
the date/time records are updated. Use a trigger to keep this column correct
so the time is always taken from the server, not the client. When copying
data to the laptop copy this column also. Then create a second column, say
UpdateTimeRemote, in the data on the laptop only which stores the date/time
that updates are done on the laptop. Then when copying data back to the PC
grab all the records that have a NOT NULL UpdateTimeRemote and copy each
back to the sqlserver database. For each row that you copy back check that
UpdateTime is the same, if it is not then you know the data has been changed
on the PC by another user and you can decide what to do (fail, overwrite or
prompt the user).

As a side note, this isn't really any different to when you have a direct
connection to the database because when your windows app grabs the data from
the database it makes a copy and when it saves it it can check the
UpdateTime column to see if another user has changed the data.

BTW, in answer to your question above, I don't think the XML will keep the
state of your datatable, if you have rows marked as changed then I believe
the changed flag will be cleared when you save.

Michael

[John B]

Andrew Meador wrote:

I am working on an application that will store data in SQL Server
2005 Express. The database will reside on our server at the office. I
need to write an application that can work with this data off site
during the day and then synch the changes back to the database at the
end of the day (or next morning).

I have been reading up on ADO.NET and see that you can use a
DataSet and a DataAdaptor to manage change tracking and updating of
the database with the changes. However, what I have read seems to
imply that this is all done in RAM. The laptop that will go out during
the day needs to be able to safeguard the changes to file, so in case
the laptop goes off, needs restarted, or whatever, the data (and
specifically the changes to the data) will not be lost. But, in the
end, I still need to push these changes back to the database when the
network is available again.

We are operating in an area that does not permit cellular based
internet access, so we have to do this based on no network access,
except for in the morning before going out, and later when we return
to the office.

I was wondering about having two DataAdaptors to work with the one
DataSet; one to pull the data from the database into the DataSet, and
another one to push/pull the data from to DataSet to XML files stored
on the laptop (the working set while out of the office). I can see how
to deal with reading and writing to the XML files and how to pull the
data from the database into the DataSet, but if the program closes and
thus the DataAdaptor that handles the link between the database and
the DataSet is killed, how can I get the changes abck to the database?

Another question here is how well the overall idea will work
performance wise. We will have about a dozen tables that need to be
worked with, two of which will have as many as 10,000 records each (at
~1.5KB per record). If these would have to be completely re-written
with every change I'd imagine this would get to be pretty slow. The
laptop is brand new with 2GB RAM, 120GB Hard Drive, and 2.2GHz Intel®
Core® 2 Duo Processor, so maybe it wouldn't be tooo bad, but...

I am looking for ideas here. If you have a better appraoch that has
nothing to do with XML that's fine, just so long as it is not costly,
crazy complex, or something I can't do with VS 2005 Pro and SQL Server
2005 Express (can't afford SQL Server 2005 Standard) ;) You know what
I mean...

Thanks in advance!!!

I would use an sql express database on the laptop which would (when
online) be synced to/from the master.
I would also have (in the master & client) a timestamp column on each
table that needed synching which would be used to determine the last
change/insert time.
You would need to keep your clock's in sync to preserve latest changes
though.
Check out the Timestamp datatype in MSSQL BOL.

JB

[Marc Gravell]

If you can wait for VS2008, then you might want to look at "Local
Database Cache". I haven't fully investigated it myself yet, other
than it appears as a template in VS2008 beta 2, and *sounds* like it
might do something along this line.

http://techedbloggers.net/News/19002.item

Marc

[Marc Gravell]

I think nntp ate my post... I don't know much about it, but if you can
wait for VS 2008, you might want to look into "Local Database Cache":
http://techedbloggers.net/News/19002.item

I haven't look into it yet (it is on my list...) - but it sounds like
it might be useful...

Marc

[Michael C]

"Marc Gravell" <ma**********@gmail.comwrote in message
news:11**********************@d55g2000hsg.googlegr oups.com...

>I think nntp ate my post...

No, it's there.

Michael

[Andrew Meador]

On Oct 31, 5:30 pm, Marc Gravell <[e-mail]:

I think nntp ate my post... I don't know much about it, but if you can
wait for VS 2008, you might want to look into "Local Database Cache":http://techedbloggers.net/News/19002.item

I haven't look into it yet (it is on my list...) - but it sounds like
it might be useful...

Marc

That looks awsome Marc - thanks for the heads up! I asked "The Elder"
about how this will be working with SQL Server 2005 Express (as well
as other OLE and ODBC datbases as well), as his post only discussed
this in relation to using SQL Server Compact Edition 3.5. I'll post
his response if/when he answers me. But yes, this looks like it will
do this type of thing very well. It even has controls/forms for
letting users decide how to handle update synchronozation issues
(where the data has been changed on the server since the initial
sync). This looks very nice from what his post showed. I may just work
on other parts of this project and hold off on implementing this part
until VS 2008 is released.

Thanks again.

[Andrew Meador]

On Oct 30, 11:10 pm, John B <[e-mail]:

Andrew Meador wrote:

I am working on an application that will store data in SQL Server
2005 Express. The database will reside on our server at the office. I
need to write an application that can work with this data off site
during the day and then synch the changes back to the database at the
end of the day (or next morning).

I have been reading up on ADO.NET and see that you can use a
DataSet and a DataAdaptor to manage change tracking and updating of
the database with the changes. However, what I have read seems to
imply that this is all done in RAM. The laptop that will go out during
the day needs to be able to safeguard the changes to file, so in case
the laptop goes off, needs restarted, or whatever, the data (and
specifically the changes to the data) will not be lost. But, in the
end, I still need to push these changes back to the database when the
network is available again.

We are operating in an area that does not permit cellular based
internet access, so we have to do this based on no network access,
except for in the morning before going out, and later when we return
to the office.

I was wondering about having two DataAdaptors to work with the one
DataSet; one to pull the data from the database into the DataSet, and
another one to push/pull the data from to DataSet to XML files stored
on the laptop (the working set while out of the office). I can see how
to deal with reading and writing to the XML files and how to pull the
data from the database into the DataSet, but if the program closes and
thus the DataAdaptor that handles the link between the database and
the DataSet is killed, how can I get the changes abck to the database?

Another question here is how well the overall idea will work
performance wise. We will have about a dozen tables that need to be
worked with, two of which will have as many as 10,000 records each (at
~1.5KB per record). If these would have to be completely re-written
with every change I'd imagine this would get to be pretty slow. The
laptop is brand new with 2GB RAM, 120GB Hard Drive, and 2.2GHz Intel®
Core® 2 Duo Processor, so maybe it wouldn't be tooo bad, but...

I am looking for ideas here. If you have a better appraoch that has
nothing to do with XML that's fine, just so long as it is not costly,
crazy complex, or something I can't do with VS 2005 Pro and SQL Server
2005 Express (can't afford SQL Server 2005 Standard) ;) You know what
I mean...

Thanks in advance!!!

I would use an sql express database on the laptop which would (when
online) be synced to/from the master.
I would also have (in the master & client) a timestamp column on each
table that needed synching which would be used to determine the last
change/insert time.
You would need to keep your clock's in sync to preserve latest changes
though.
Check out the Timestamp datatype in MSSQL BOL.

JB- Hide quoted text -

- Show quoted text -

Thanks Michael C and John B, these ideas would work. I was hoping
there would already be a pre-built solution to this, but doesn't
appear to be. Marc Gravell posted a link in this thread that I looked
into that looks promising, but it is in VS 2008. Check it out. It
looks to me like it is design for this purpose and works well. But,
have to wait for VS 2008 for that part of the project to be completed.

Anyway, thanks for your help and suggestions - they are pretty good as
it seems this is how VS 2008 is basically handling it too.

Andrew

[Marc Gravell]

Well, you can always freely download beta 2 [the full VS, not the
express editions] and give it a whirl? But you run the risk that
things may change between now and release.

Marc

[Marc Gravell]

on a quick test, only compact edition was offered

[Andrew Meador]

I got a message back from Keith Elder ("The Elder") about the types
of data sources that will work with Synchronization Services in VS
2008 and he said, "Yes, you can use SQL Server 2005 and express. As
far as Oracle it can as well but I don't know if it is drag and drop
out of the box but it can definitely work. Pretty much anything that
Ado.Net can hit it can be used. It is very flexible in how and where
you get your data."

So, I think I will try working with this solution. I am waiting on
a new MSDN subscription to come, as I want to play with VS 2008 to
start working on this, but I want to do it in a VPC so I don't mess
with my main workstation yet.

Anyway, this really looks like a very nice addition to .Net
Thanks all!

[Marc Gravell]

I don't think you need MSDN for this:

http://msdn2.microsoft.com/en-gb/vstudio/aa700831.aspx (bottom of
page)
http://www.microsoft.com/downloads/d...displaylang=en

Watch out for the gotcha about November 1st ;-p

Personally, I found it easier to bite the bullet and go for a straight
install. If it messes up the PC (which it did the first time I tried)
then I rebuild it. No real harm. Obviously it helps if you have a non-
critical PC lying around...

Marc

[Andrew Meador]

On Nov 1, 12:51 pm, Marc Gravell <[e-mail]:

I don't think you need MSDN for this:

http://msdn2.microsoft.com/en-gb/vst...31.aspx(bottom of
page)http://www.microsoft.com/downloads/d...d=3B72271C-E99...

Watch out for the gotcha about November 1st ;-p

Personally, I found it easier to bite the bullet and go for a straight
install. If it messes up the PC (which it did the first time I tried)
then I rebuild it. No real harm. Obviously it helps if you have a non-
critical PC lying around...

Marc

Oh, I agree (about having a non-critical PC lying around), but I
don't have one here at work. The only system I have is my workstations
which has a lot of stuff on it that I REALLY don't want to have to re-
install. GIS Software, database instances, etc... I am almost finished
downloading VS 2008 beta 2 now, but I just don't want to install it
directly, I would rather kill a VPC ;) But right now, I don't have VPC
either, so when my MSDN stuff gets here, I'll have it and can put in
all in place. I'm downloading now so I can play with this some at home
though - I do have a few extra PC's there.

BTW, was also looking more at this. I downloaded the books online
for Synchronization Services for ADO.NET from Microsoft, and from what
I get from it, the client side of the setup has to be SQL Server
Compact Edition 3.5. The server side can be any database that has an
ADO.NET provider (or some other option, but I'm not sure what they're
talking about). Anyway, I reasked Keith Elder about this and am
waiting for his reply and will keep looking further myself. I might
just have to get it install and try it to find out. But, I think the
MS Books Online docs seem clear on this.

Andrew