For ease of configuration and other reasons, I would like for my
single-user GUI app to be able to use postgresql in-process as a library
accessing a database created in the users home directory. I think I
could possibly get what I want by launching a captive copy of postmaster
with appropriate args but it seems conceptually cleaner to not have a
seperate process at all. Has anyone tried to do anything like this?
I've looked at sqlite and it might be workable for my project but I
would prefer the more standard and featureful SQL of postgres. In
particular sqlite lacks date arithmetic and has some funny type issues
(it claims to be typeless, except when it isn't, or something).
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org 59  7088 
* Jeff Bowden <jl*@houseofdistraction.com> [2004-01-13 13:38:02 -0800]: For ease of configuration and other reasons, I would like for my
single-user GUI app to be able to use postgresql in-process as a library
accessing a database created in the users home directory. I think I
could possibly get what I want by launching a captive copy of postmaster
with appropriate args but it seems conceptually cleaner to not have a
seperate process at all. Has anyone tried to do anything like this?
hmm, i've also played around a little bit w/ postgresql standalone.
you could also do nice things like calling postmaster from inetd ;-)
but I dont think you really wanna have the postmaster in the same
process w/ your application, since it relies on forks, signals,
mmap() and some other things which directly affect the process control -
this probably infers with your application. On the application side
you cant really control, what it does, so it sounds quite difficult.
Having the postmaster in a separate process connected by a socket or
pty seems to be the better solution.
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT services
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: co*****@metux.de
cellphone: +49 174 7066481
---------------------------------------------------------------------
Diese Mail wurde mit UUCP versandt. http://www.metux.de/uucp/
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly
I have just about the same sort of needs now and concluded that postgres
just is not suited for embedding into apps like that. I am going with
sqlite and it is working fairly well. We just made user defined functions
in php for sqlite to match the date functions in postgres. Who knows what
other issues we may run into but so far sqlite is working fairly well.
I think that sqlite stores everything as a string but for searching,
sorting, ordering etc it uses the type info.
I to would absolutely love to just include a postgres dll and have postgres
in process and just store everything in a nice little file but from what I
have read hear that would involve major changes that the developers on not
interested in making. So far sqlite has done the job for me.
Also I am considering looking into firebird for an embedded database
solution since it can you pretty much what you are talking about here with
an in proccess db that uses a single file to store it's data. While I don't
want to switch my server stuff to it, it is probably more full featured than
sqlite (I'm sure it has date and arithmatic functions) and is made to work
well in embedded situations.
I would be intersted to see what you end up doing.
Rick
----- Original Message -----
From: "Jeff Bowden" <jl*@houseofdistraction.com>
To: "pgsql-general" <pg***********@postgresql.org>
Sent: Tuesday, January 13, 2004 2:38 PM
Subject: [GENERAL] serverless postgresql For ease of configuration and other reasons, I would like for my
single-user GUI app to be able to use postgresql in-process as a library
accessing a database created in the users home directory. I think I
could possibly get what I want by launching a captive copy of postmaster
with appropriate args but it seems conceptually cleaner to not have a
seperate process at all. Has anyone tried to do anything like this?
I've looked at sqlite and it might be workable for my project but I
would prefer the more standard and featureful SQL of postgres. In
particular sqlite lacks date arithmetic and has some funny type issues
(it claims to be typeless, except when it isn't, or something).
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
"Rick Gigger" <ri**@alpinenetworking.com> writes: I to would absolutely love to just include a postgres dll and have postgres
in process and just store everything in a nice little file but from what I
have read hear that would involve major changes that the developers on not
interested in making.
Not only are the developers uninterested in it, the developers actively
oppose it. We think an embedded database library cannot be reliable
enough to meet our notion of a "database", since it would be subject to
failures anytime the surrounding application has a bug. Keeping the
client code in a separate process is a far more robust design.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
Tom Lane wrote: "Rick Gigger" <ri**@alpinenetworking.com> writes:
I to would absolutely love to just include a postgres dll and have postgres
in process and just store everything in a nice little file but from what I
have read hear that would involve major changes that the developers on not
interested in making.
Not only are the developers uninterested in it, the developers actively
oppose it. We think an embedded database library cannot be reliable
enough to meet our notion of a "database", since it would be subject to
failures anytime the surrounding application has a bug. Keeping the
client code in a separate process is a far more robust design.
That makes sense to me. I wonder if sqlite suffers for this problem
(e.g. app crashing and corrupting the database).
What about the notion of running postmaster on-demand as the user? Is
that something that anyone has experience with? It seems like it would
solve the complex configuration problems without compromising robustness
or requiring any special support other than sufficient command-line
parameters.
Oh yeah, that brings me to another question. I was looking at the
postmaster command-line switches and I couldn't find any that would
allow me to point it at an arbitrary config file but then I had a look
around and it seems that the .conf files are already used by the various
startup scripts. Does postmaster itself ever read the .conf files or is
it controlled strictly by switches?
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives? http://archives.postgresql.org
On Jan 13, 2004, at 13:38, Jeff Bowden wrote: but it seems conceptually cleaner to not have a seperate process at
all.
That depends a lot on what you mean by cleaner. A few small programs
in their own process space dedicated to a specific set of functionality
with well-defined interfaces can make for a much more reliable
application (see postfix).
--
Dustin Sallings
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Jeff Bowden <jl*@houseofdistraction.com> writes: That makes sense to me. I wonder if sqlite suffers for this problem
(e.g. app crashing and corrupting the database).
Likely. I can tell you that Ann Harrison once told me she made a decent
amount of money as a consultant fixing broken Interbase/Firebird
database files. It would be hard to make a living in the same game for
Postgres. Now I don't think that Firebird is any buggier than Postgres.
But it comes in an embedded-library form; I'll bet lunch that most of
those data corruption problems were actually induced by crashes of
surrounding applications.
What about the notion of running postmaster on-demand as the user?
Possibly. You'd have to think carefully about what conditions the
postmaster should be shut down under, and especially what conditions
it should NOT be shut down under --- eg, a kill to the parent client
application shouldn't cause an ungraceful postmaster exit. It could
be tricky to get the signal handling right, especially under shells that
try to deliver signals to all children of a process being signaled.
On the whole I suspect it'd be easier just to leave the postmaster
running in the background...
Oh yeah, that brings me to another question. I was looking at the
postmaster command-line switches and I couldn't find any that would
allow me to point it at an arbitrary config file
The config files all live in $PGDATA and so are determined by the -D
switch. There was some talk of changing this, awhile back, but it
foundered on lack of consensus about exactly what to do instead.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match jl*@houseofdistraction.com (Jeff Bowden) wrote: Tom Lane wrote:Not only are the developers uninterested in it, the developers actively
oppose it. We think an embedded database library cannot be reliable
enough to meet our notion of a "database", since it would be subject to
failures anytime the surrounding application has a bug. Keeping the
client code in a separate process is a far more robust design.
That makes sense to me. I wonder if sqlite suffers for this problem
(e.g. app crashing and corrupting the database).
I'm not sure its use has yet been so widespread that there is a good
feel for this. It is doubtless _possible_; "crash patterns" likely
_don't_ lead to enormous disasters when databases are small,
localized, and the apps probably _don't_ crash all the time.
What about the notion of running postmaster on-demand as the user?
Is that something that anyone has experience with? It seems like it
would solve the complex configuration problems without compromising
robustness or requiring any special support other than sufficient
command-line parameters.
I haven't had call for this being controlled "by the application," but
I can't see it being vastly troublesome. And I _don't_ see it
requiring lots of "command line" parameters; you just need to specify
the directory where the configuration is.
Oh yeah, that brings me to another question. I was looking at the
postmaster command-line switches and I couldn't find any that would
allow me to point it at an arbitrary config file but then I had a
look around and it seems that the .conf files are already used by
the various startup scripts. Does postmaster itself ever read the
.conf files or is it controlled strictly by switches?
All of the .conf files are in one directory, and that directory is
controlled by either the value of environment variable PGDATA or the
"-D" command parameter.
It seems not-overly-valuable to have the .conf files be able to be
specified in random other locations.
Is there some particular reason you have in mind why you would want to
_ignore_ the configuration in $HOME/DBDIR and instead use
configuration in some other random location? I would think that
collecting the config into one directory, as is done right now, is a
_good_ thing.
--
let name="cbbrowne" and tld="ntlug.org" in name ^ "@" ^ tld;; http://www3.sympatico.ca/cbbrowne/lsf.html
"Even in the area of anticompetitive conduct, Microsoft is mainly an
imitator." -- Ralph Nader (1998/11/11)
Tom Lane wrote: Jeff Bowden <jl*@houseofdistraction.com> writes:
What about the notion of running postmaster on-demand as the user?
Possibly. You'd have to think carefully about what conditions the
postmaster should be shut down under, and especially what conditions
it should NOT be shut down under --- eg, a kill to the parent client
application shouldn't cause an ungraceful postmaster exit. It could
be tricky to get the signal handling right, especially under shells that
try to deliver signals to all children of a process being signaled.
On the whole I suspect it'd be easier just to leave the postmaster
running in the background...
Details, details.... :-) Oh yeah, that brings me to another question. I was looking at the
postmaster command-line switches and I couldn't find any that would
allow me to point it at an arbitrary config file
The config files all live in $PGDATA and so are determined by the -D
switch. There was some talk of changing this, awhile back, but it
foundered on lack of consensus about exactly what to do instead.
As long as it can be done.
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives? http://archives.postgresql.org
Quoth jl*@houseofdistraction.com (Jeff Bowden): For ease of configuration and other reasons, I would like for my
single-user GUI app to be able to use postgresql in-process as a
library accessing a database created in the users home directory. I
think I could possibly get what I want by launching a captive copy of
postmaster with appropriate args but it seems conceptually cleaner to
not have a seperate process at all. Has anyone tried to do anything
like this?
One of the properties of Unix is that spawning extra processes to "do
their thing" allows each application to be designed more 'cleanly.'
Curiously enough, the implementors of alternatives to the Sendmail MTA
have typically concluded that it was not only conceptually cleaner to
have a set of separate cooperating processes, but that they would also
reap benefits in terms of speed and improved security.
Thus, I think you'll find that people disagree with you on this. It
seems to me that it is conceptually cleaner to have the "application"
do "application stuff," and not muddle things up by mixing that with
the "database stuff."
--
If this was helpful, <http://svcs.affero.net/rm.php?r=cbbrowne> rate me http://cbbrowne.com/info/sgml.html
Rules of the Evil Overlord #87. "My vats of hazardous chemicals will
be covered when not in use. Also, I will not construct walkways above
them." <http://www.eviloverlord.com/>
Tom Lane <tg*@sss.pgh.pa.us> said: ---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
Re: tip 9 :-). In 7.3.5 is it possible that the planner is finding a type
mismatch when there isn't one, resulting in a seq scan being selected? I've
had a problem with joining tables on an int8 column (primary key column in one
of the tables). Tested left outer and where clause syntax.
TIA,
Jim Wilson
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
"Jim Wilson" <ji**@kelcomaine.com> writes: Re: tip 9 :-). In 7.3.5 is it possible that the planner is finding a type
mismatch when there isn't one, resulting in a seq scan being selected?
When you gave no details, how is anyone to know? For questions like
this, you need to post the table definitions, the exact query, and the
results of EXPLAIN ANALYZE on that query. BTW, pgsql-performance is a
more appropriate list than -general for such questions.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly
> Jeff Bowden <jl*@houseofdistraction.com> writes: That makes sense to me. I wonder if sqlite suffers for this problem
(e.g. app crashing and corrupting the database).
Likely. I can tell you that Ann Harrison once told me she made a decent
amount of money as a consultant fixing broken Interbase/Firebird
database files. It would be hard to make a living in the same game for
Postgres. Now I don't think that Firebird is any buggier than Postgres.
But it comes in an embedded-library form; I'll bet lunch that most of
those data corruption problems were actually induced by crashes of
surrounding applications.
I remember reading about how some the original commerical code from
Interbase came with some bugs that corrupted databases and that those bugs
were eventually fixed in Firebird. Don't know if that's related to this but
it could be.
Anyway since postgres uses WAL files to verify the integrity of the database
couldn't it more or less make the same guarantee's in an embedded version?
As long as the app uses the db libs unmodified and doesn't mess with the
files it creates how does simply making it embedded increase the change of
db errors resulting in database corruption?
In addition there are times when you want the ease of a relational database
but you don't need say the same level of reliability that you would on your
server. In these cases it would be nice to just have something that is easy
to drop into your app and have it just work. That being said I think that
in these cases the developer isn't usually going to care if it runs in
process or not. Maybe a good solution would be to make a postgres library
that could be included in process that would interface with the application.
All it would do is manager spawning the real postgres process. Then you
would have the reliability of a real server but the ease of use of an
embedded database. Couldn't that end up being pretty transparent to the
developer? If it was made to be easy postgres could become very popular as
an "embedded" db solution but still not have it's reputation diluted by
rampant database corruption from the application.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
"Rick Gigger" <ri**@alpinenetworking.com> writes: Anyway since postgres uses WAL files to verify the integrity of the database
couldn't it more or less make the same guarantee's in an embedded version?
As long as the app uses the db libs unmodified and doesn't mess with the
files it creates how does simply making it embedded increase the change of
db errors resulting in database corruption?
If an app's internal data structures get corrupted and it goes nuts,
pretty much anything can happen to its address space.
-Doug
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly
> Anyway since postgres uses WAL files to verify the integrity of the database couldn't it more or less make the same guarantee's in an embedded version?
As long as the app uses the db libs unmodified and doesn't mess with the
files it creates how does simply making it embedded increase the change of
db errors resulting in database corruption?
I have a different idea. I've been thinking about coding it, but haven't
had the time. This could be done with no changes to Postgres itself.
Basically, you would have a library which exported functions such as
pg_instance *pg_start(char *directory);
pg_get_connection(pg_instance *);
pg_stop(pg_instance *);
pg_initdb(char *directory)
pg_start would do the following:
1) check "directory" for an instance of the UNIX socket.
if it is there, make a Postgres connection
if it is not there, start the Postmaster server with "-k directory
-D directory/data" and then make a Postgres connection
2) Create a struct to contain the directory and any other data item we
need to connect to the database
3) Return this structure
pg_get_connection would just be a wrapper for pq_connect()
pg_stop would kill the database.
pg_initdb would simply run initdb
Does anyone see a reason why this wouldn't work?
Jon
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
This is just what I recommended (I think) in my earlier post. This would be
FANTASTIC. Then I could scrap sqlite and any sqlite custom code that I end
up having to write (there is a little). Plus I could have the reliability
of a multi-process postgres "embedded".
Now if they just finish the Win32 port sometime soon...
----- Original Message -----
From: "Jonathan Bartlett" <jo*****@eskimo.com>
To: "Rick Gigger" <ri**@alpinenetworking.com>
Cc: "Jeff Bowden" <jl*@houseofdistraction.com>; "Tom Lane"
<tg*@sss.pgh.pa.us>; <pg***********@postgresql.org>
Sent: Wednesday, January 14, 2004 2:07 PM
Subject: Re: [GENERAL] serverless postgresql Anyway since postgres uses WAL files to verify the integrity of the
database couldn't it more or less make the same guarantee's in an embedded
version? As long as the app uses the db libs unmodified and doesn't mess with the
files it creates how does simply making it embedded increase the change
of db errors resulting in database corruption?
I have a different idea. I've been thinking about coding it, but haven't
had the time. This could be done with no changes to Postgres itself.
Basically, you would have a library which exported functions such as
pg_instance *pg_start(char *directory);
pg_get_connection(pg_instance *);
pg_stop(pg_instance *);
pg_initdb(char *directory)
pg_start would do the following:
1) check "directory" for an instance of the UNIX socket.
if it is there, make a Postgres connection
if it is not there, start the Postmaster server with "-k directory
-D directory/data" and then make a Postgres connection
2) Create a struct to contain the directory and any other data item we
need to connect to the database
3) Return this structure
pg_get_connection would just be a wrapper for pq_connect()
pg_stop would kill the database.
pg_initdb would simply run initdb
Does anyone see a reason why this wouldn't work?
Jon
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
Jeff Bowden wrote: For ease of configuration and other reasons, I would like for my
single-user GUI app to be able to use postgresql in-process as a library
accessing a database created in the users home directory. I think I
could possibly get what I want by launching a captive copy of postmaster
with appropriate args but it seems conceptually cleaner to not have a
seperate process at all. Has anyone tried to do anything like this?
[Sorry for not actually answering this question]
I believe the demands for embedded/"serverless" version of PostgreSQL to
increase significantly once PostgreSQL is natively available on Windows.
So I would expect that official embedded support to follow quite shortly
after win32 port has stabilized. :-)
--
dave
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives? http://archives.postgresql.org
Rick Gigger wrote: I have just about the same sort of needs now and concluded that postgres
just is not suited for embedding into apps like that.
Why not? It's not that the PostgreSQL backend is a mammoth like Oracle.
The Firebird embedded version is pretty much the same as their server,
but without network and client authentication layer.
However, embedded usually demands that the backend be threaded.
Otherwise it will be pretty useless/very inconvenient to use in many
apps. Perhaps this is the major change that's hard to do?
--
dave
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives? http://archives.postgresql.org
Tom Lane wrote: Jeff Bowden <jl*@houseofdistraction.com> writes:
That makes sense to me. I wonder if sqlite suffers for this problem
(e.g. app crashing and corrupting the database).
Likely. I can tell you that Ann Harrison once told me she made a decent
amount of money as a consultant fixing broken Interbase/Firebird
database files. It would be hard to make a living in the same game for
Postgres. Now I don't think that Firebird is any buggier than Postgres.
But it comes in an embedded-library form; I'll bet lunch that most of
those data corruption problems were actually induced by crashes of
surrounding applications.
Do the developers generally oppose the idea of a threaded (but
non-embedded) backend as well? If the backend is thread-safe, then users
can still choose to run multiprocess or multithreaded right?
--
dave
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
> Do the developers generally oppose the idea of a threaded (but non-embedded) backend as well? If the backend is thread-safe, then users
can still choose to run multiprocess or multithreaded right?
I've been under the impression that the developers were opposed to a
threaded server because of the complete lack of consistency in threading
behavior across platforms. However, I don't see how doing it as a
multiprocess server as apposed to a multithreaded server affects embedded
use - as long as it can simply be called from a library, why would a user
or developer care?
Jon
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives? http://archives.postgresql.org tg*@sss.pgh.pa.us says... That makes sense to me. I wonder if sqlite suffers for this problem
(e.g. app crashing and corrupting the database).
Likely. I can tell you that Ann Harrison once told me she made a decent
amount of money as a consultant fixing broken Interbase/Firebird
database files. It would be hard to make a living in the same game for
Postgres. Now I don't think that Firebird is any buggier than Postgres.
But it comes in an embedded-library form; I'll bet lunch that most of
those data corruption problems were actually induced by crashes of
surrounding applications.
The reason IMHO for "server" crashes in IB/FB is quite simply due to the
fact that newbie types who are running IB/FB will tend to run it on the
crappiest PC in the office under Wintendo 95 on the machine used by the
secretary who's a bit of a ditz and keeps switching it off by mistake,
or spilling her coffee on the exposed hard drive.
No forced writes on, no UPS, no RAID, no server room - nada, zip,
diddly. Basically, it boils down to the fact that just because IB/FB
*_lets_* you do something, it's not necessarily a good idea. IB/FB will
run reasonably on the crappiest of hardware, and on the crappiest of
OS's, which is what leads to the problem.
It's unlikely that PostgreSQL faces this problem, since it will only run
under far more robust OS's, and you're not likely to have the office
eejit running the Unix/Linux/BSD box with the db server on it.
Paul...
--
plinehan y_a_h_o_o and d_o_t com
C++ Builder 5 SP1, Interbase 6.0.1.6 IBX 5.04 W2K Pro
Please do not top-post.
"XML avoids the fundamental question of what we should do,
by focusing entirely on how we should do it."
quote from http://www.metatorial.com
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend
> Rick Gigger wrote: I have just about the same sort of needs now and concluded that postgres
just is not suited for embedding into apps like that.
Why not? It's not that the PostgreSQL backend is a mammoth like Oracle.
The Firebird embedded version is pretty much the same as their server,
but without network and client authentication layer.
However, embedded usually demands that the backend be threaded.
Otherwise it will be pretty useless/very inconvenient to use in many
apps. Perhaps this is the major change that's hard to do?
I should have said that postgres isn't suited for running in-proccess with
the rest of your app. Or at least the developers don't think (and I guess I
agree now) that it's a good idea. They could however make a library that
would start up the server in multiprocess mode and run it in the background.
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)
> Tom Lane wrote: Jeff Bowden <jl*@houseofdistraction.com> writes:
That makes sense to me. I wonder if sqlite suffers for this problem
(e.g. app crashing and corrupting the database).
Likely. I can tell you that Ann Harrison once told me she made a decent
amount of money as a consultant fixing broken Interbase/Firebird
database files. It would be hard to make a living in the same game for
Postgres. Now I don't think that Firebird is any buggier than Postgres.
But it comes in an embedded-library form; I'll bet lunch that most of
those data corruption problems were actually induced by crashes of
surrounding applications.
Do the developers generally oppose the idea of a threaded (but
non-embedded) backend as well? If the backend is thread-safe, then users
can still choose to run multiprocess or multithreaded right?
My impression is that they don't want to go multithreaded. There was some
talk of this for the windows port but I belive that Bruce is simply
modifying the multi-proccess code so that it will work on windows.
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)
David Garamond <li***@zara.6.isreserved.com> writes: Do the developers generally oppose the idea of a threaded (but
non-embedded) backend as well? If the backend is thread-safe, then users
can still choose to run multiprocess or multithreaded right?
The backend isn't thread-safe. There have been repeated discussions
about using threading in the backend (see the pgsql-hackers archives),
but so far no one has made a convincing case for it.
BTW, this whole discussion is getting pretty off-topic for -general;
I'd suggest pursuing it on -hackers.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
From: "David Garamond" <li***@zara.6.isreserved.com> I believe the demands for embedded/"serverless" version of PostgreSQL to
increase significantly once PostgreSQL is natively available on Windows.
So I would expect that official embedded support to follow quite shortly
after win32 port has stabilized. :-)
People are always asking for embedded dbms's without really considering the
consequences. For example, if you need to share data, you end up with all
the MS Access sorts of issues. I know because I used to work at Microsoft
in the department that provided support both for Access and the developer
products. I suspect the support nightmares may be part of the reason for
pushing MSDE, and hence *get away* from the embedded dbms model ;-)
For example, do you realize that the marketing info said that Access
supported 256 *concurrent* users to a database? At the same time
troubleshooting corruption was nearly unsupported for a while.
I agree with the approach of a wrapper library which would wrap the
startup/shutdown of a postgresql server so that the programmer doesn't have
to worry about the details, but I would add another idea-- namely that the
library should be able to determine whether the server is running remotely,
and simply pass the connection to libpq. This would also create a
conceptually cleaner framework for configuration of software which may need
to access a local or remote data store.
Best Wishes,
Chris Travers
---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
"Chris Travers" <ch***@travelamericas.com> writes: I agree with the approach of a wrapper library which would wrap the
startup/shutdown of a postgresql server so that the programmer doesn't have
to worry about the details,
The reason that the client programmer doesn't have to worry about
starting/stopping the database is that it's not his responsibility.
I don't think that having the client control this is a good idea at all.
David conveniently ignored the points I made before, but they are
real issues --- if the client is in charge of starting or stopping the
DB, it just adds potential for mucking things up. I can see the bug
reports now: "I decided I'd make the shutdown routine 'kill -9' the
postmaster because I didn't like the multi-second delay for a normal
shutdown. Now my database is corrupt."
Another set of objections to this center around the fact that with this
sort of arrangement, the database files would necessarily belong to the
client user, since there's no way to launch the postmaster as a
different userid. (Unless the client is running as root, which I
sincerely hope he is not.) That means there's no filesystem protection
between the client and the database, which is another recipe for
trouble. Not much point in keeping an address-space firewall between
client and server when the client can scribble on the database anyway.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
> "Chris Travers" <ch***@travelamericas.com> writes: I agree with the approach of a wrapper library which would wrap the
startup/shutdown of a postgresql server so that the programmer doesn't
have to worry about the details,
The reason that the client programmer doesn't have to worry about
starting/stopping the database is that it's not his responsibility.
I don't think that having the client control this is a good idea at all.
David conveniently ignored the points I made before, but they are
real issues --- if the client is in charge of starting or stopping the
DB, it just adds potential for mucking things up. I can see the bug
reports now: "I decided I'd make the shutdown routine 'kill -9' the
postmaster because I didn't like the multi-second delay for a normal
shutdown. Now my database is corrupt."
I recall a discussion a while back where people were complaining that some
os (I think it was mac os x) would just kill all proccesses after something
like 30 seconds on system shutdown if they didn't quit fast enough on their
own. The response was that because of how postgres uses wal files to
prevent db corruption this was not an issue. Why does this not apply in
this situation?
Another set of objections to this center around the fact that with this
sort of arrangement, the database files would necessarily belong to the
client user, since there's no way to launch the postmaster as a
different userid. (Unless the client is running as root, which I
sincerely hope he is not.) That means there's no filesystem protection
between the client and the database, which is another recipe for
trouble. Not much point in keeping an address-space firewall between
client and server when the client can scribble on the database anyway.
By saying "the client can scribble on the database anyway" do you do you
mean the client app actually writing over the db files on disk? It seems
like this would only be a problem with an exceptionally stupid programmer.
How could that happen on accident?
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
> From: "David Garamond" <li***@zara.6.isreserved.com> I believe the demands for embedded/"serverless" version of PostgreSQL to
increase significantly once PostgreSQL is natively available on Windows.
So I would expect that official embedded support to follow quite shortly
after win32 port has stabilized. :-)
People are always asking for embedded dbms's without really considering
the consequences. For example, if you need to share data, you end up with all
the MS Access sorts of issues. I know because I used to work at Microsoft
in the department that provided support both for Access and the developer
products. I suspect the support nightmares may be part of the reason for
pushing MSDE, and hence *get away* from the embedded dbms model ;-)
My argument of having an embedded or pseudo-embedded postgres is this:
I am never, ever going to trust any really, really important data to an
embedded database. If the user has it on his system the moron might delete
the database files in which case it really doesn't matter how your
protecting it. The user can potentially screw everything up. The reason I
need an embedded database is that sometime my clients need to take a copy of
the app off line and look the info, make reports etc. They also may
eventually be filling out small amounts of data that can then be synced with
the live database once they are back on line. If a clients thinkpad lights
on fire while he's on the plane and he loses 6 hours of data entry well
that's not my problem. But I've ALWAYS got most of the data in a nice
consistent postgres database on my server which is backed up constantly to
multiple locations.
So there are cases where reliability requirements just go down and what I
want is something that acts exactly like postgres so that I don't have to
write custom code for 2 databases. Since I end up just running sqlite
anyway I would prefer to just have an embedded (or pseudo-embedded)
postgres. I'm not going to get all up in arms if something bad happens
because I understand the risks of running it embedded. But those risks will
be the same with any embedded app. I just don't care.
Does this apply to the vast majority of embedded users or is it just me?
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
"Rick Gigger" <ri**@alpinenetworking.com> writes: ... just adds potential for mucking things up. I can see the bug
reports now: "I decided I'd make the shutdown routine 'kill -9' the
postmaster because I didn't like the multi-second delay for a normal
shutdown. Now my database is corrupt."
I recall a discussion a while back where people were complaining that some
os (I think it was mac os x) would just kill all proccesses after something
like 30 seconds on system shutdown if they didn't quit fast enough on their
own. The response was that because of how postgres uses wal files to
prevent db corruption this was not an issue. Why does this not apply in
this situation?
Because a system shutdown doesn't leave anything else running. kill -9
on the postmaster leaves orphaned backends still running, and orphaned
shared memory segments still in existence. The latter could prevent you
from starting a fresh postmaster (because two shmem segments could
exceed the kernel's SHMMAX) until you do manual cleanup, which most
people don't know how to do. If you do succeed in starting another
postmaster with a fresh shared memory segment, you will have two
independent sets of backends modifying the database files with no
interlocking, which is a sure recipe for data corruption.
There is an interlock that is supposed to prevent this catastrophic
scenario, but I do not trust it unreservedly (and not at all on
non-Unix-derived platforms, because it depends on some rather arcane
features of the SysV shared memory API, which might not be implemented
fully on other platforms).
This is why the "don't kill -9 the postmaster" TIP is still around, BTW.
It has nothing to do with crash safety.
By saying "the client can scribble on the database anyway" do you do you
mean the client app actually writing over the db files on disk? It seems
like this would only be a problem with an exceptionally stupid programmer.
How could that happen on accident?
Scribbling on memory that doesn't belong to you isn't something one does
intentionally, either --- but it happens. However, my real concern here
is not so much with program bugs as clueless users. We see enough cases
already of people removing lock files or "unnecessary" log files; I fear
it'd be a lot worse if those files were sitting in the user's home
directory.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
Tom Lane wrote: "Chris Travers" <ch***@travelamericas.com> writes:
I agree with the approach of a wrapper library which would wrap the
startup/shutdown of a postgresql server so that the programmer doesn't have
to worry about the details,
The reason that the client programmer doesn't have to worry about
starting/stopping the database is that it's not his responsibility.
I don't think that having the client control this is a good idea at all.
David conveniently ignored the points I made before, but they are
real issues --- if the client is in charge of starting or stopping the
DB, it just adds potential for mucking things up. I can see the bug
reports now: "I decided I'd make the shutdown routine 'kill -9' the
postmaster because I didn't like the multi-second delay for a normal
shutdown. Now my database is corrupt."
Another set of objections to this center around the fact that with this
sort of arrangement, the database files would necessarily belong to the
client user, since there's no way to launch the postmaster as a
different userid. (Unless the client is running as root, which I
sincerely hope he is not.) That means there's no filesystem protection
between the client and the database, which is another recipe for
trouble. Not much point in keeping an address-space firewall between
client and server when the client can scribble on the database anyway.
Still, the main problem I, and I suspect others, would like to solve is
installation/configuration. For my app I don't want the user to have to
understand anything about how keeping data in a shared
system-administered database is different from keeping data in local
files. Everything should "just work". There is no requirement for
concurrent access.
So kill -9 on postmaster can lead to database corruption? What happens
in a power failure?
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
Personally, I would use something like berkelydb for an embedded database
engine. As a general rule, if it needs to be embedded, you don't need sql.
I'm sure you could think of areas where you could use a full blown sql
server, but in practice that generally isn't the case. And like others
have already mentioned, if you are embedding a database in software that a
user has control of, you want it as SIMPLE as possible.
Chris
----- Original Message -----
From: "Tom Lane" <tg*@sss.pgh.pa.us>
To: "Rick Gigger" <ri**@alpinenetworking.com>
Cc: "Chris Travers" <ch***@travelamericas.com>; "David Garamond"
<li***@zara.6.isreserved.com>; "Jeff Bowden" <jl*@houseofdistraction.com>;
"pgsql-general" <pg***********@postgresql.org>
Sent: Friday, January 16, 2004 1:14 PM
Subject: Re: embedded/"serverless" (Re: [GENERAL] serverless postgresql) "Rick Gigger" <ri**@alpinenetworking.com> writes: ... just adds potential for mucking things up. I can see the bug
reports now: "I decided I'd make the shutdown routine 'kill -9' the
postmaster because I didn't like the multi-second delay for a normal
shutdown. Now my database is corrupt." I recall a discussion a while back where people were complaining that
some os (I think it was mac os x) would just kill all proccesses after
something like 30 seconds on system shutdown if they didn't quit fast enough on
their own. The response was that because of how postgres uses wal files to
prevent db corruption this was not an issue. Why does this not apply in
this situation?
Because a system shutdown doesn't leave anything else running. kill -9
on the postmaster leaves orphaned backends still running, and orphaned
shared memory segments still in existence. The latter could prevent you
from starting a fresh postmaster (because two shmem segments could
exceed the kernel's SHMMAX) until you do manual cleanup, which most
people don't know how to do. If you do succeed in starting another
postmaster with a fresh shared memory segment, you will have two
independent sets of backends modifying the database files with no
interlocking, which is a sure recipe for data corruption.
There is an interlock that is supposed to prevent this catastrophic
scenario, but I do not trust it unreservedly (and not at all on
non-Unix-derived platforms, because it depends on some rather arcane
features of the SysV shared memory API, which might not be implemented
fully on other platforms).
This is why the "don't kill -9 the postmaster" TIP is still around, BTW.
It has nothing to do with crash safety.
By saying "the client can scribble on the database anyway" do you do you
mean the client app actually writing over the db files on disk? It
seems like this would only be a problem with an exceptionally stupid
programmer. How could that happen on accident?
Scribbling on memory that doesn't belong to you isn't something one does
intentionally, either --- but it happens. However, my real concern here
is not so much with program bugs as clueless users. We see enough cases
already of people removing lock files or "unnecessary" log files; I fear
it'd be a lot worse if those files were sitting in the user's home
directory.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
> Personally, I would use something like berkelydb for an embedded database engine. As a general rule, if it needs to be embedded, you don't need sql.
I'm sure you could think of areas where you could use a full blown sql
server, but in practice that generally isn't the case. And like others
have already mentioned, if you are embedding a database in software that a
user has control of, you want it as SIMPLE as possible.
But the user isn't the one accessing the database directly. Have you ever
used a program like Delphi to create applications? Using a relational
database, you can create applications using nothing but drag-and-drop and
SQL. No real code. If your application is a simple end-user app (like a
wedding planner, for example), embedded databases are your dream come
true. Linux has no real embedded database. PG would be an excellent
embedded database. I have already proposed some functions that would help
PG as an embedded database, and, if I hve time, I will implement them
(can't right now because I'm finishing a book).
Anyway, I get the feeling that those people who don't like the idea of
having PG as an embedded database have never used one. I think Tom, for
example, has been focusing on enterprise-level applications too much to
think about the personal applications, and Chris, I'm going to guess
you've never built an application w/ an embedded DB.
Anyway, this thread is going nowhere, I'll come back if/when I have some
code. Perhaps with something concrete we can all see better where this is
going.
Jon
Chris
----- Original Message -----
From: "Tom Lane" <tg*@sss.pgh.pa.us>
To: "Rick Gigger" <ri**@alpinenetworking.com>
Cc: "Chris Travers" <ch***@travelamericas.com>; "David Garamond"
<li***@zara.6.isreserved.com>; "Jeff Bowden" <jl*@houseofdistraction.com>;
"pgsql-general" <pg***********@postgresql.org>
Sent: Friday, January 16, 2004 1:14 PM
Subject: Re: embedded/"serverless" (Re: [GENERAL] serverless postgresql)
"Rick Gigger" <ri**@alpinenetworking.com> writes:> ... just adds potential for mucking things up. I can see the bug
> reports now: "I decided I'd make the shutdown routine 'kill -9' the
> postmaster because I didn't like the multi-second delay for a normal
> shutdown. Now my database is corrupt."
I recall a discussion a while back where people were complaining that some os (I think it was mac os x) would just kill all proccesses after something like 30 seconds on system shutdown if they didn't quit fast enough on their own. The response was that because of how postgres uses wal files to
prevent db corruption this was not an issue. Why does this not apply in
this situation?
Because a system shutdown doesn't leave anything else running. kill -9
on the postmaster leaves orphaned backends still running, and orphaned
shared memory segments still in existence. The latter could prevent you
from starting a fresh postmaster (because two shmem segments could
exceed the kernel's SHMMAX) until you do manual cleanup, which most
people don't know how to do. If you do succeed in starting another
postmaster with a fresh shared memory segment, you will have two
independent sets of backends modifying the database files with no
interlocking, which is a sure recipe for data corruption.
There is an interlock that is supposed to prevent this catastrophic
scenario, but I do not trust it unreservedly (and not at all on
non-Unix-derived platforms, because it depends on some rather arcane
features of the SysV shared memory API, which might not be implemented
fully on other platforms).
This is why the "don't kill -9 the postmaster" TIP is still around, BTW.
It has nothing to do with crash safety.
By saying "the client can scribble on the database anyway" do you do you
mean the client app actually writing over the db files on disk? It seems like this would only be a problem with an exceptionally stupid programmer. How could that happen on accident?
Scribbling on memory that doesn't belong to you isn't something one does
intentionally, either --- but it happens. However, my real concern here
is not so much with program bugs as clueless users. We see enough cases
already of people removing lock files or "unnecessary" log files; I fear
it'd be a lot worse if those files were sitting in the user's home
directory.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Jeff Bowden <jl*@houseofdistraction.com> writes: Still, the main problem I, and I suspect others, would like to solve is
installation/configuration. For my app I don't want the user to have to
understand anything about how keeping data in a shared
system-administered database is different from keeping data in local
files. Everything should "just work".
Sure, but most of the existing packagings of PG already try to make this
automatic (at least Lamar's RPMs and Oliver's Debian package do). No
doubt further work could be invested to make it even smoother, but that
doesn't mean we need a client-started database.
So kill -9 on postmaster can lead to database corruption? What happens
in a power failure?
See my last response to Rick Gigger. kill -9 is a lot different from a
power failure.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)
On Fri, Jan 16, 2004 at 02:00:35PM -0800, Jonathan Bartlett wrote: having PG as an embedded database have never used one. I think Tom, for
example, has been focusing on enterprise-level applications too much to
think about the personal applications, and Chris, I'm going to guess
This seems to suggest that Postgres should focus on something other
than enterprise-level applications, and (presumably) on something
else.
Frankly, I am _mighty glad_ that the developers are focussed on
enterprise-level applications. That's what makes Postgres eligible
to back enterprise-level software. Applications that try to do
everything for everyone tend to end up doing nothing very well. I
don't think one needs to look too hard to discover examples of that
phenomenon.
A
--
Andrew Sullivan | aj*@crankycanuck.ca
This work was visionary and imaginative, and goes to show that visionary
and imaginative work need not end up well.
--Dennis Ritchie
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly
Tom Lane wrote: Jeff Bowden <jl*@houseofdistraction.com> writes:
Still, the main problem I, and I suspect others, would like to solve is
installation/configuration. For my app I don't want the user to have to
understand anything about how keeping data in a shared
system-administered database is different from keeping data in local
files. Everything should "just work".
Sure, but most of the existing packagings of PG already try to make this
automatic (at least Lamar's RPMs and Oliver's Debian package do). No
doubt further work could be invested to make it even smoother, but that
doesn't mean we need a client-started database.
So maybe this is a packaging issue. On Debian when I install postgres
it is necessary to do root shit in order to enable non-priveledged users
to create and destroy databases. My understanding has alwasy been that
these operations are restricted because it can allow users to
accidentally or intentionally interefere with each other's use of the
server. If there is some way for this to be set up sanely by default to
allow users to successfully use createdb and dropdb on only their own
data, then I will file a wishlist bug against the Debian packages to add
this. Or is this all a pipe dream?
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend
> This seems to suggest that Postgres should focus on something other than enterprise-level applications, and (presumably) on something
else.
Nope.
Frankly, I am _mighty glad_ that the developers are focussed on
enterprise-level applications.
Me too. But I think we can get it for small-scale stuff with no impact on
the rest. Just an additional function or 6 to make embedded developer's
lives easier.
Jon
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
On Fri, 16 Jan 2004, Jeff Bowden wrote: Tom Lane wrote:
Jeff Bowden <jl*@houseofdistraction.com> writes:
Still, the main problem I, and I suspect others, would like to solve is
installation/configuration. For my app I don't want the user to have to
understand anything about how keeping data in a shared
system-administered database is different from keeping data in local
files. Everything should "just work".
Sure, but most of the existing packagings of PG already try to make this
automatic (at least Lamar's RPMs and Oliver's Debian package do). No
doubt further work could be invested to make it even smoother, but that
doesn't mean we need a client-started database.
So maybe this is a packaging issue. On Debian when I install postgres
it is necessary to do root shit in order to enable non-priveledged users
to create and destroy databases. My understanding has alwasy been that
these operations are restricted because it can allow users to
accidentally or intentionally interefere with each other's use of the
server. If there is some way for this to be set up sanely by default to
allow users to successfully use createdb and dropdb on only their own
data, then I will file a wishlist bug against the Debian packages to add
this. Or is this all a pipe dream?
I don't see what the problem is. PostgreSQL is perfectly able to be installed
and run as any old user. Even if one is distributing via a package system, such
as RPM perhaps, which has problems installing as non-root user if one sees the
package already installed then there's no need to try to install it
again. Even with the package installations there's bugger all stopping any old
user doing an initdb -D myownplace.
The standard postgres user on a system is only special in that it is generally
the user the postmaster is run as. If you have other non-priviledged users and
there's no need for them to share a cluster why try to make them?
--
Nigel Andrews
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Nigel J. Andrews wrote: On Fri, 16 Jan 2004, Jeff Bowden wrote:
So maybe this is a packaging issue. On Debian when I install postgres
it is necessary to do root shit in order to enable non-priveledged users
to create and destroy databases. My understanding has alwasy been that
these operations are restricted because it can allow users to
accidentally or intentionally interefere with each other's use of the
server. If there is some way for this to be set up sanely by default to
allow users to successfully use createdb and dropdb on only their own
data, then I will file a wishlist bug against the Debian packages to add
this. Or is this all a pipe dream?
I don't see what the problem is. PostgreSQL is perfectly able to be installed
and run as any old user. Even if one is distributing via a package system, such
as RPM perhaps, which has problems installing as non-root user if one sees the
package already installed then there's no need to try to install it
again. Even with the package installations there's bugger all stopping any old
user doing an initdb -D myownplace.
The standard postgres user on a system is only special in that it is generally
the user the postmaster is run as. If you have other non-priviledged users and
there's no need for them to share a cluster why try to make them?
I'm sorry, maybe I didn't state my ideas clearly enough. I was
following on to Tom Lane's suggestion that the Debian and RPM packages
make configuration "automatic". I was trying to explain that even after
installation, some administrator configuration is required to make the
server usable for non-privelidged users. Either in the form of creating
and dropping databases or in giving the users rights to do it themselves.
The way I'd like to make it work for my app is to run a seperate
postmaster as the user to avoid involving the machine administrator or
interfering with other users. If a shared install can be made to work
in an equivalent way, then I would rather go with that. I'm not clear
on whether that's possible though. My current understanding is that it
is not and that my original notion is the only way for my app.
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings
I still have to respectfully disagree. Postgresql is IMO just the wrong
software for the job, and given that there are still a number of really
important things that postgresql lacks, it should concentrate on those. I
am not against it however for technical reasons, because those things can
always be overcome. I just wouldn't want postgresql to start branching out
in different directions at this point, it makes no sense if the project
wants to keep focused and one day become comparable side by side to oracle.
IMO that should be it's main goal, and embedded functionality would be a
detour that has more chances of doing harm then good.
Chris
Frankly, I am _mighty glad_ that the developers are focussed on
enterprise-level applications.
Me too. But I think we can get it for small-scale stuff with no impact on
the rest. Just an additional function or 6 to make embedded developer's
lives easier.
Jon
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org
Chris Ochs wrote: I still have to respectfully disagree. Postgresql is IMO just the wrong
software for the job, and given that there are still a number of really
important things that postgresql lacks, it should concentrate on those. I
am not against it however for technical reasons, because those things can
always be overcome. I just wouldn't want postgresql to start branching out
in different directions at this point, it makes no sense if the project
wants to keep focused and one day become comparable side by side to oracle.
IMO that should be it's main goal, and embedded functionality would be a
detour that has more chances of doing harm then good.
I believe the basic mission of Postgres will stay the same [for a long
time]: providing a 1) reliable database; 2) with advanced; 3) and
standard compliant features.
However, venturing into win32 world _will_ generate demands like
embedded and other desktop app-oriented features since I expect that's
what many people will using Postgres for in win32 (currently people are
using IB/FB for this and not MySQL/Postgre; MySQL's embedded version is
GPL/commercial). And we all know it's all a matter of what itches the
developers the most. If enough people are bugging them about something,
they will do it... :-)
--
dave
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)
Hi Rick;
I have been looking for a few years at a similar situation with my CRM
software and soon (once I have the SOAP interfaces done, I will start on a
stand-alone client. I have eventually decided on a very different structure
than you have, though the result is more or less the same.
Rather than using PostgreSQL as an embedded database manager, I have decided
that it makes more sense for my app to "cache objects" in XML documents and
then use those on trips to customers' sites. The XML documents can then be
checked and the objects re-sync'd when the app goes online again. The
cacheing would be automatic and cached objects would expire unless
explicitly saved.
Why not use XML for this sort of problem? Or berkeley DB? Is there any
reason that PostgreSQL would be better? I am not aware of any functional
replication systems which would work in this way, so I am guessing your app
will have to handle all the logic for the syncronization anyway.
Best Wishes,
Chris Travers
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings
Hi Tom and others;
I agree with your reasoning insofar as it pertains to dedicating core
PostgreSQL time into developing such a library. I also agree insofar as it
pertains to the PostgreSQL core distribution.
I do think, however, that given the current demand for a serverless solution
to PostgreSQL (whether or not it is the right tool for the job), I would
suggest that those who are interested and have the necessary skills set up a
gborg project. Perhaps at some point if PostgreSQL distributions become
more commonplace, we will see it used, and this will open up new
oportunities for PostgreSQL. It should be remembered that more options in
critical areas (such as Win32) may mean more community, and hopefully in the
long run, more developers.
Best Wishes,
Chris Travers
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
On Fri, 16 Jan 2004, Jeff Bowden wrote: Nigel J. Andrews wrote:I don't see what the problem is. PostgreSQL is perfectly able to be installed
and run as any old user. Even if one is distributing via a package system, such
as RPM perhaps, which has problems installing as non-root user if one sees the
package already installed then there's no need to try to install it
again. Even with the package installations there's bugger all stopping any old
user doing an initdb -D myownplace.
The standard postgres user on a system is only special in that it is generally
the user the postmaster is run as. If you have other non-priviledged users and
there's no need for them to share a cluster why try to make them?
I'm sorry, maybe I didn't state my ideas clearly enough. I was
following on to Tom Lane's suggestion that the Debian and RPM packages
make configuration "automatic". I was trying to explain that even after
installation, some administrator configuration is required to make the
server usable for non-privelidged users. Either in the form of creating
and dropping databases or in giving the users rights to do it themselves.
The way I'd like to make it work for my app is to run a seperate
postmaster as the user to avoid involving the machine administrator or
interfering with other users. If a shared install can be made to work
in an equivalent way, then I would rather go with that. I'm not clear
on whether that's possible though. My current understanding is that it
is not and that my original notion is the only way for my app.
I think I probably didn't do the explanation clearly enough. This does depend
on what you mean by shared install. I was assuming you just wanted to manage
data independently only using the shared binaries installed from the
package. If that's not the case then obviously this isn't what you want. If
that is the case then you could run the following as any old user, provided the
environment has been set appropiately of course:
initdb -D $HOME/pgdata
sed -e 's/^#port = 5432/port = 12345/' $HOME/pgdata/postgresql.conf > /tmp/p
mv /tmp/p $HOME/pgdata/postgresql.conf
pg_ctl -D $HOME/pgdata start
then access the cluster using:
pgsql -P 12345 template1
That's it. That's access as the db superuser for that cluster. Sure if the
system admin upgrades the shared software then you could have problems.
My apologies if I've got the wrong end of things again, I've got a lot of
unread email and I keep reading things in a very haphazard order.
--
Nigel J. Andrews
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings
On Fri, 2004-01-16 at 16:00, Jonathan Bartlett wrote: Linux has no real embedded database. PG would be an excellent
embedded database.
SQLite works great for these kinds of applications, FWIW.
b.g.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Christopher Browne wrote: jl*@houseofdistraction.com (Jeff Bowden) wrote:
Oh yeah, that brings me to another question. I was looking at the
postmaster command-line switches and I couldn't find any that would
allow me to point it at an arbitrary config file but then I had a
look around and it seems that the .conf files are already used by
the various startup scripts. Does postmaster itself ever read the
.conf files or is it controlled strictly by switches?
All of the .conf files are in one directory, and that directory is
controlled by either the value of environment variable PGDATA or the
"-D" command parameter.
It seems not-overly-valuable to have the .conf files be able to be
specified in random other locations.
Is there some particular reason you have in mind why you would want to
_ignore_ the configuration in $HOME/DBDIR and instead use
configuration in some other random location? I would think that
collecting the config into one directory, as is done right now, is a
_good_ thing.
If the existing user configuration could be reliably determined to be
sane and have sufficient permissions or could be made so, that would be
fine.
I am writing an app for users who are not necessarily knowledgeable
about databases. If they don't have an existing useable postgres
configuration, I would like for my app to be able to create one without
forcing the user to have to be root or learn about postgres
configuration. The data for this app should remain private to the user
who created it and there is no need for concurrent access.
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)
After a long battle with technology, jl*@houseofdistraction.com (Jeff Bowden), an earthling, wrote: Christopher Browne wrote:jl*@houseofdistraction.com (Jeff Bowden) wrote:Oh yeah, that brings me to another question. I was looking at the
postmaster command-line switches and I couldn't find any that would
allow me to point it at an arbitrary config file but then I had a
look around and it seems that the .conf files are already used by
the various startup scripts. Does postmaster itself ever read the
.conf files or is it controlled strictly by switches?
All of the .conf files are in one directory, and that directory is
controlled by either the value of environment variable PGDATA or the
"-D" command parameter.
It seems not-overly-valuable to have the .conf files be able to be
specified in random other locations.
Is there some particular reason you have in mind why you would want to
_ignore_ the configuration in $HOME/DBDIR and instead use
configuration in some other random location? I would think that
collecting the config into one directory, as is done right now, is a
_good_ thing.
If the existing user configuration could be reliably determined to be
sane and have sufficient permissions or could be made so, that would
be fine.
I ask again:
Is there some particular reason why you would expect configuration in
$HOME/DBDIR/postgresql.conf to be unreliable/non-sane in some way that
would be "fixed" by putting the configuration in some other location?
You haven't said anything that even _suggests_ that sticking the
configuration somewhere other than the "PGDATA" directory would have
the slightest effect.
I am writing an app for users who are not necessarily knowledgeable
about databases. If they don't have an existing useable postgres
configuration, I would like for my app to be able to create one
without forcing the user to have to be root or learn about postgres
configuration. The data for this app should remain private to the
user who created it and there is no need for concurrent access.
I cannot say this emphatically enough:
_NOTHING REQUIRES THAT THE USER HAVE ROOT ACCESS._
Watch the process of creating a database without:
a) Forcing the user to be root, or
b) Forcing the user to learn about PostgreSQL configuration.
Here is a shell script that will do this, given a directory and a port
number, sets up a database for the current user.
#!/bin/bash
DIRECTORY=$1
PORT=$2
mkdir -p $DIRECTORY
user=`whoami`
/usr/lib/postgresql/bin/initdb -D $DIRECTORY > /dev/null 2> /dev/null
sed -e "s/#port = 5432/port=$PORT/g" -i $DIRECTORY/postgresql.conf
sed -e "s/max_connections = 100/max_connections = 10/g" -i $DIRECTORY/postgresql.conf
/usr/lib/postgresql/bin/pg_ctl -D $DIRECTORY -l $DIRECTORY/pglogs.log start
echo "# empty out access control list" > $DIRECTORY/pg_hba.conf
echo "local all $user trust" >> $DIRECTORY/pg_hba.conf
/usr/lib/postgresql/bin/pg_ctl -D $DIRECTORY reload
The preferred paths to the PostgreSQL programs may vary; I'm using,
here, the instance installed as part of the Debian Linux distribution.
If you include the PostgreSQL software as part of what you install,
then you'll have no problem controlling where it runs from.
That script actually works...
[cbbrowne@wolfe:cbbrowne] ~/bin/mkpg ~/PostgreSQL/cbbrowneInstallPGNotAsRoot 7981
postmaster successfully started
postmaster successfully signalled
[cbbrowne@wolfe:cbbrowne] psql -p 7981 -l
List of databases
Name | Owner | Encoding
-----------+----------+-----------
template0 | cbbrowne | SQL_ASCII
template1 | cbbrowne | SQL_ASCII
(2 rows)
You could certainly have your application fiddle a bit more with the
configuration in order to degrade the capabilities further.
None of the things the script configures are esoteric or particularly
elaborate; what can you get in 12 lines of code, after all?
I have made no attempt to make it particularly robust; rewriting the
config files would presumably be safer if you used regular expressions
that permitted variations in whitespace and such, on the off chance
that a user might just discover that you were installing PostgreSQL,
and might change some bit of configuration.
I can't see an "embedded library" database being materially simpler to
configure; you'll still need to point to various software and database
paths, configure socket access, and integrate code to "open" and
"close" the database into your application's code.
The configuration satisfies your requirements:
- Does it create the DB without forcing the user to be root?
It sure does.
- Does it force the user to rummage through configuration files?
No, it does not.
- Is the data private to the user?
The access controls are pretty restrictive; another user would have
to go to _some_ effort to get at the data.
As for "there is no need for concurrent access," that isn't really a
constraint.
--
(format nil "~S@~S" "cbbrowne" "ntlug.org") http://www3.sympatico.ca/cbbrowne/spiritual.html
"I've seen estimates that 10% of all IDs in the US are phony. At least
one-fourth of the president's own family has been known to use phony
IDs." -- Bruce Schneier CRYPTO-GRAM, December 15, 2001
> On Fri, Jan 16, 2004 at 02:00:35PM -0800, Jonathan Bartlett wrote: having PG as an embedded database have never used one. I think Tom, for
example, has been focusing on enterprise-level applications too much to
think about the personal applications, and Chris, I'm going to guess
This seems to suggest that Postgres should focus on something other
than enterprise-level applications, and (presumably) on something
else.
Yes but sometimes an enterprise level application may need to be put on a
laptop and taken off-line. Having an embedded database that is compatible
with the one on the server makes this a bit easier to do.
Frankly, I am _mighty glad_ that the developers are focussed on
enterprise-level applications. That's what makes Postgres eligible
to back enterprise-level software. Applications that try to do
everything for everyone tend to end up doing nothing very well. I
don't think one needs to look too hard to discover examples of that
phenomenon.
Well, this very well may be true. It does seem as if many of the features
that add the reliability and performance necessary for an enterprise server
application are what makes it difficult to make a good embedded solution.
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
> Hi Rick;
I have been looking for a few years at a similar situation with my CRM
software and soon (once I have the SOAP interfaces done, I will start on a
stand-alone client. I have eventually decided on a very different
structure than you have, though the result is more or less the same.
Rather than using PostgreSQL as an embedded database manager, I have
decided that it makes more sense for my app to "cache objects" in XML documents
and then use those on trips to customers' sites. The XML documents can then
be checked and the objects re-sync'd when the app goes online again. The
cacheing would be automatic and cached objects would expire unless
explicitly saved.
Why not use XML for this sort of problem? Or berkeley DB? Is there any
reason that PostgreSQL would be better? I am not aware of any functional
replication systems which would work in this way, so I am guessing your
app will have to handle all the logic for the syncronization anyway.
Well since the desktop and server app are using almost exactly the same code
it is nice to not have to recode everything that accesses the database.
While data syncronization code will have to be written either way I want the
existing code to "just work" once taken off line.
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives? http://archives.postgresql.org
On Wed, Jan 21, 2004 at 11:23:40AM -0700, Rick Gigger wrote:
Yes but sometimes an enterprise level application may need to be put on a
laptop and taken off-line. Having an embedded database that is compatible
with the one on the server makes this a bit easier to do.
Why can't you just run a postgres instance for this? What is magic
about "embedded" for this sort of application? Sounds like a clever
wrapper script is all that's necessary for something like that, no?
A
--
Andrew Sullivan | aj*@crankycanuck.ca
My friend, you can put wheels on your mama but that doesn't make her a bus.
--Joel Spolsky
---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
On Wed, 21 Jan 2004, Andrew Sullivan wrote: On Wed, Jan 21, 2004 at 11:23:40AM -0700, Rick Gigger wrote:
Yes but sometimes an enterprise level application may need to be put on a
laptop and taken off-line. Having an embedded database that is compatible
with the one on the server makes this a bit easier to do.
Why can't you just run a postgres instance for this? What is magic
about "embedded" for this sort of application? Sounds like a clever
wrapper script is all that's necessary for something like that, no?
That's what I still don't get. Embedded means something running on [and
probably running] your wrist watch _not_ something running on a full blown
system.
I understand in this thread's context that embedded is used to mean embedded
within an application on an ordinary system, took me a while to realise that
though, but just means I can't see why it is wanted like that. Imagine if there
was a DB around that was used by direct library calls from an application. What
would be one of the first things that would be programmed using it? A server
perhaps?
--
Nigel Andrews
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org) This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: codepirate |
last post by:
Hi All,
I am trying to connct to a serverless/embedded PostgreSQL database from a C# application using VS 2005. First of all, is this possible with PostgreSQL? I know this can be done with some...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
| |
