469,341 Members | 7,180 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,341 developers. It's quick & easy.

State of Beta 2


Anyone out there using beta 2 in production situations? Comments on
stability? I am rolling out a project in the next 4 weeks, and really
don't want to go though an upgrade soon after its released on an
Unsuspecting Client, so I would LIKE to start working with 7.4.

--------------------

Andrew Rawnsley
President
The Ravensfield Digital Resource Group, Ltd.
(740) 587-0114
www.ravensfield.com
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 11 '05
236 8575
>>>>> "TL" == Tom Lane <tg*@sss.pgh.pa.us> writes:

TL> Vivek Khera <kh***@kcilink.com> writes:
Well, for me the create index part of the restore is what takes about
3x the time for the data load. Total about 4 hours. The dump takes 1
hour.


TL> What sort_mem do you use for the restore? Have you tried increasing it?

All tests have these non-default settings:
vacuum_mem = 131702
max_fsm_pages = 1000000
random_page_cost = 2
effective_cache_size = 12760 # `sysctl -n vfs.hibufspace` / BLKSZ
16k pages
30000 shared buffers
The four tests I've run so far are:

checkpoint_segments default
sort_mem 8192
restore time: 15344.57 seconds

checkpoint_segments default
sort_mem 131702
restore time: 15042.00 seconds

checkpoint_segments 50
sort_mem 8192
restore time: 11511.24 seconds

checkpoint_segments 50
sort_mem 131702
restore time: 11287.94 seconds
I have also enabled the extra query/timing logging you requested last
week, and just need to sanitize the table names to prevent any
confidential information from leaking out of the office. I'll send
those along shortly to you directly, as they are pretty large. I
wasn't able to do much work since the 'storm' hit on thursday,
knocking out power to the house until sunday...

Right now I'm running the same above tests with fsync=false to see if
that improves anything. Next test will be Marc's test to disable the
WAL entirely.

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: kh***@kciLink.com Rockville, MD +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly

Nov 11 '05 #201
>>>>> "MGF" == Marc G Fournier <sc*****@postgresql.org> writes:

MGF> I've tried restoring a >5gig database with sort_mem up to 100Meg in size,
MGF> and didn't find that it sped up the index creation enough to make a
MGF> difference ... shaved off a couple of minutes over the whole reload, so
MGF> seconds off of each index ... and that was with the WAL logs also disabled
MGF> :(

Ditto for me. Can you reproduce my results by increasing
checkpoint_buffers to some large value (I use 50)? this shaved
something like 60 minutes off of my restore.

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 11 '05 #202
Vivek Khera <kh***@kcilink.com> writes:
Ditto for me. Can you reproduce my results by increasing
checkpoint_buffers to some large value (I use 50)?


You meant checkpoint_segments, right? It might be interesting to
experiment with wal_buffers, too, though I'm not convinced that will
have a big impact.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org

Nov 11 '05 #203

actually, I didn't get near that kind of benefit ... with wal disabled,
and sort_mem/checkpoint_segments at default, I got:

import start: 22:31:38
end: 23:21:42 (~50min)
buffers: 64
sort_mem: 1024
wal disabled: yes

with checkpoint_segments and sort_mem raised, I shaved about 8min:

import start: 15:56:07
end: 16:38:56 (~42min)
buffers: 640
sort_mem: 102400
checkpoint_segments: 64
wal disabled: yes
fsync disabled: yes

As a side note, a default install with 64 shared memory buffers came in
around 56min ... then again, if looking at percentages, that is about a
25% improvement ... it just doesn't look to be that big looking at the
straight #s :)

On Tue, 23 Sep 2003, Vivek Khera wrote:
>> "MGF" == Marc G Fournier <sc*****@postgresql.org> writes:


MGF> I've tried restoring a >5gig database with sort_mem up to 100Meg in size,
MGF> and didn't find that it sped up the index creation enough to make a
MGF> difference ... shaved off a couple of minutes over the whole reload, so
MGF> seconds off of each index ... and that was with the WAL logs also disabled
MGF> :(

Ditto for me. Can you reproduce my results by increasing
checkpoint_buffers to some large value (I use 50)? this shaved
something like 60 minutes off of my restore.


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org

Nov 11 '05 #204
Ron Johnson wrote:
On Mon, 2003-09-15 at 15:23, Joshua D. Drake wrote:
I'm not going to rehash the arguments I have made before; they are all
archived. Suffice to say you are simply wrong. The number of
complaints over the years shows that there IS a need.

I at no point suggested that there was not a need. I only suggest that
the need may not be as great as some suspect or feel. To be honest -- if
your arguments were the "need" that everyone had... it would have been
implemented some how. It hasn't yet which would suggest that the number
of people that have the "need" at your level is not as great as the
number of people who have different "needs" from PostgreSQL.


But the problem is that as more and more people put larger and larger
datasets, that are mission-critical, into PostgreSQL, the need will
grow larger and larger.

Of course, we understand the "finite resources" issue, and are not
badgering/complaining. Simply, we are trying to make our case that
this is something that should go on the TODO list, and be kept in
the back of developers' minds.


Added to TODO:

* Allow major upgrades without dump/reload, perhaps using
pg_upgrade

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 11 '05 #205
Under circumstances where I have had critical upgrades I have usualy
used a new machine
to build the new upgrade. This allows me to use revitalized equipment
and a "clean" install
for the upgraded server. If something goes sideways, you just switch
back to the old machine.
This is ususaly the quickest and most reliable method of upgrading a server.

Under a few circumstances I have not had a second machine, so I either
put in a new drive
and installed fresh, mounting the original drive to copy the old data to
the new drive before
any modification. Then if the upgrade goes sideways, just switch drives.
This takes longer to
recover.

When I have upgraded under the most stringent ecomonic restraints, I
have backed up the
original data and configuration files before making any changes. This is
the most error
prone method of upgrading a server, and takes the longest time to recover.

Using mirrored drives and splitting the mirror so that you have two
identical data sets can also
be feasible. I did this once successfuly but it requires having a spare
drive or two to rebuild the
mirror without losing the old data.

Andrew Sullivan wrote:
On Thu, Sep 18, 2003 at 06:49:56PM -0300, Marc G. Fournier wrote:

Hadn't thought of it that way ... but, what would prompt someone to
upgrade, then use something like erserver to roll back? All I can think
of is that the upgrade caused alot of problems with the application
itself, but in a case like that, would you have the time to be able to
're-replicate' back to the old version?


The trick is to have your former master set up as slave before you
turn your application back on.

The lack of a rollback strategy in PostgreSQL upgrades is a major
barrier for corporate use. One can only do so much testing, and it's
always possible you've missed something. You need to be able to go
back to some known-working state.

A


--
Guy Fraser
Network Administrator
The Internet Centre
780-450-6787 , 1-888-450-6787

There is a fine line between genius and lunacy, fear not, walk the
line with pride. Not all things will end up as you wanted, but you
will certainly discover things the meek and timid will miss out on.

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 11 '05 #206
>>>>> "TL" == Tom Lane <tg*@sss.pgh.pa.us> writes:

TL> Vivek Khera <kh***@kcilink.com> writes:
Ditto for me. Can you reproduce my results by increasing
checkpoint_buffers to some large value (I use 50)?


TL> You meant checkpoint_segments, right? It might be interesting to
TL> experiment with wal_buffers, too, though I'm not convinced that will
TL> have a big impact.

The difference on restore with fsync=false was 2 seconds. I'm
rebuilding PG with Marc's WAL-disabling patch and will see the change
there.

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: kh***@kciLink.com Rockville, MD +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 12 '05 #207
Tom Lane wrote:
Dunno about MySQL. I'm pretty sure I remember Ann Harrison stating that
FireBird's disk structures haven't changed since the beginning of
Interbase. Which you might take as saying that they were a lot smarter
than we are, but I suspect what it really means is that
FireBird/Interbase hasn't undergone the kind of metamorphosis of purpose
that the Postgres code base has. Keep in mind that it started as an
experimental academic prototype (representing some successful ideas and
some not-so-successful ones), and the current developers have been
laboring to convert it into an industrial-strength production tool ---
keeping the good experimental ideas, but weeding out the bad ones, and
adding production-oriented features that weren't in the original design.
The entire argument that version-to-version stability should be a
critical goal would have been foreign to the original developers of
Postgres.


Thought the fact PostgreSQL came from an academic world are part of it,
the big reason we change on-disk format so often is that we are
improving faster than any other database on the planet.

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #208

With all the discussion and pg_upgrade, I saw no one offer to work on
it.

Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.

---------------------------------------------------------------------------

Lamar Owen wrote:
Marc G. Fournier wrote:
On Mon, 15 Sep 2003, Joshua D. Drake wrote:
I'm not going to rehash the arguments I have made before;I at no point suggested that there was not a need. I only suggest that
the need may not be as great as some suspect or feel. To be honest -- if
your arguments were the "need" that everyone had... it would have been
implemented some how. It hasn't yet which would suggest that the number

Just to add to this ... Bruce *did* start pg_upgrade, but I don't recall
anyone else looking at extending it ... if the *need* was so great,
someone would have step'd up and looked into adding to what was already
there ...


You'ns are going to make a liar out of me yet; I said I wasn't going to
rehash the arguments. But I am going to answer Marc's statement. Need
of the users != developer interest in implementing those. This is the
ugly fact of open source software -- it is developer-driven, not
user-driven. If it were user-driven in this case seamless upgrading
would have already happened. But the sad fact is that the people who
have the necessary knowledge of the codebase in question are so
complacent and comfortable with the current dump/reload cycle that they
really don't seem to care about the upgrade issue. That is quite a
harsh statement to make, yes, and I know that is kind of
uncharacteristic for me. But, Marc, your statement thoroughy ignores
the archived history of this issue on the lists.

While pg_upgrade was a good first step (and I applaud Bruce for working
on it), it was promptly broken because the developers who changed the
on-disk format felt it wasn't important to make it continue working.

Stepping up to the plate on this issue will require an intimate
knowledge of the storage manager subsystem, a thorough knowledge of the
system catalogs, etc. This has been discussed at length; I'll not
repeat it. Just any old developer can't do this -- it needs the
long-term focused attention of Tom, Jan, or Bruce. And that isn't going
to happen. We know Tom's take on it; it's archived. Maybe there's
someone out there with the deep knowledge of the backend to make this
happen who cares enough about it to make it happen, and who has the time
to do it. I care enough to do the work; but I have neither the deep
knowledge necessary nor the time to make it happen. There are many in
my position. But those who could make it happen don't seem to have the
care level to do so.

And that has nothing to do with user need as a whole, since the care
level I mentioned is predicated by the developer interest level. While
I know, Marc, how the whole project got started (I have read the first
posts), and I appreciate that you, Bruce, Thomas, and Vadim started the
original core team because you were and are users of PostgreSQL, I
sincerely believe that in this instance you are out of touch with this
need of many of today's userbase. And I say that with full knowledge of
PostgreSQL Inc.'s support role. If given the choice between upgrading
capability, PITR, and Win32 support, my vote would go to upgrading.
Then migrating to PITR won't be a PITN.

What good are great features if it's a PITN to get upgraded to them?
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org


--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 12 '05 #209
On Saturday 27 September 2003 04:50 pm, Bruce Momjian wrote:
With all the discussion and pg_upgrade, I saw no one offer to work on
it. Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.


Wow, Bruce, are you _that_ behind on your e-mail? Hmmm, a perl conversion
might be interesting.... BTW, thanks for the TODO item.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 12 '05 #210
Bruce Momjian <pg***@candle.pha.pa.us> writes:
With all the discussion and pg_upgrade, I saw no one offer to work on
it.
Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.


The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose. (For example, the
transaction counter, end-of-WAL pointer, etc.) I don't think Perl would
offer anything except creating an entirely new dependency for Postgres.
Also, C code would be easier to keep in sync with the backend code that
accesses the same stuff.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 12 '05 #211
Lamar Owen wrote:
On Saturday 27 September 2003 04:50 pm, Bruce Momjian wrote:
With all the discussion and pg_upgrade, I saw no one offer to work on
it.

Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.


Wow, Bruce, are you _that_ behind on your e-mail? Hmmm, a perl conversion
might be interesting.... BTW, thanks for the TODO item.


This is more the sweep-up of items that need some extra attention. I am
not reading this for the first time.

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly

Nov 12 '05 #212
Tom Lane wrote:
Bruce Momjian <pg***@candle.pha.pa.us> writes:
With all the discussion and pg_upgrade, I saw no one offer to work on
it.
Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.


The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose. (For example, the
transaction counter, end-of-WAL pointer, etc.) I don't think Perl would
offer anything except creating an entirely new dependency for Postgres.
Also, C code would be easier to keep in sync with the backend code that
accesses the same stuff.


True, but doing all that text manipulation is C is going to be very hard
to do and maintain.

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 12 '05 #213
On Sat, 27 Sep 2003, Bruce Momjian wrote:
Tom Lane wrote:
Bruce Momjian <pg***@candle.pha.pa.us> writes:
With all the discussion and pg_upgrade, I saw no one offer to work on
it.
Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.


The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose. (For example, the
transaction counter, end-of-WAL pointer, etc.) I don't think Perl would
offer anything except creating an entirely new dependency for Postgres.
Also, C code would be easier to keep in sync with the backend code that
accesses the same stuff.


True, but doing all that text manipulation is C is going to be very hard
to do and maintain.


What about using embedded perl? I've never done it before but the mention of it
in manpages has flashed past my eyes a couple of times so I know it's possible.

Did the discuss decide on what was required for this. Last I noticed was that
there was a distinction being made between system and user tables but I don't
recall seeing a 'requirements' summary.
Nigel

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 12 '05 #214
Nigel J. Andrews wrote:
The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose. (For example, the
transaction counter, end-of-WAL pointer, etc.) I don't think Perl would
offer anything except creating an entirely new dependency for Postgres.
Also, C code would be easier to keep in sync with the backend code that
accesses the same stuff.


True, but doing all that text manipulation is C is going to be very hard
to do and maintain.


What about using embedded perl? I've never done it before but the mention of it
in manpages has flashed past my eyes a couple of times so I know it's possible.

Did the discuss decide on what was required for this. Last I noticed was that
there was a distinction being made between system and user tables but I don't
recall seeing a 'requirements' summary.


My guess is that we could do it in Perl, and call some C programs as
required.

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly

Nov 12 '05 #215
Bruce Momjian <pg***@candle.pha.pa.us> writes:
Tom Lane wrote:
The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose.
True, but doing all that text manipulation is C is going to be very hard
to do and maintain.


Text manipulation? I don't think that pg_upgrade has to do much text
manipulation. (The shell-script version might do so, but that's only
because it has to cast the problem in terms of program I/O.)

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #216
Tom Lane wrote:
Bruce Momjian <pg***@candle.pha.pa.us> writes:
Tom Lane wrote:
The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose.

True, but doing all that text manipulation is C is going to be very hard
to do and maintain.


Text manipulation? I don't think that pg_upgrade has to do much text
manipulation. (The shell-script version might do so, but that's only
because it has to cast the problem in terms of program I/O.)


Uh, it seems to have to push a lot of data around, filename/relname
mapping, etc. It almost wants a database.

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 12 '05 #217
On Sat, 2003-09-27 at 17:13, Tom Lane wrote:
Bruce Momjian <pg***@candle.pha.pa.us> writes:
Tom Lane wrote:
The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose.

True, but doing all that text manipulation is C is going to be very hard
to do and maintain.


Text manipulation? I don't think that pg_upgrade has to do much text
manipulation. (The shell-script version might do so, but that's only
because it has to cast the problem in terms of program I/O.)


There's always the general point that C has more pitfalls (mainly
from pointers/free()/malloc(), and HLLs do more for you, thus you
have to code less, and, consequently, there are fewer bugs.

--
-----------------------------------------------------------------
Ron Johnson, Jr. ro***********@cox.net
Jefferson, LA USA

484,246 sq mi are needed for 6 billion people to live, 4 persons
per lot, in lots that are 60'x150'.
That is ~ California, Texas and Missouri.
Alternatively, France, Spain and The United Kingdom.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 12 '05 #218
Bruce Momjian wrote:
With all the discussion and pg_upgrade, I saw no one offer to work on
it.

Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.

Actually I offered to put a full time programmer on it for 6 months. If
you review some of my
earlier posts you will see my proposal.

Sincerely,

Joshua Drake





---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Nov 12 '05 #219
Joshua D. Drake wrote:
Bruce Momjian wrote:
With all the discussion and pg_upgrade, I saw no one offer to work on
it.

Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.

Actually I offered to put a full time programmer on it for 6 months. If
you review some of my
earlier posts you will see my proposal.


$$$ -- I wasn't looking to purchase a programmer. :-)

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 12 '05 #220
On Sat, 2003-09-27 at 16:50, Nigel J. Andrews wrote:
On Sat, 27 Sep 2003, Bruce Momjian wrote:
Tom Lane wrote:
Bruce Momjian <pg***@candle.pha.pa.us> writes:
> With all the discussion and pg_upgrade, I saw no one offer to work on
> it.
> Does someone want to convert it to Perl? I think that would be a better
> language than shell script for this purpose, and C is too low-level.

The reason that it needs to be rewritten in C is that it needs access to
internal stuff that the backend doesn't expose. (For example, the
transaction counter, end-of-WAL pointer, etc.) I don't think Perl would
offer anything except creating an entirely new dependency for Postgres.
Also, C code would be easier to keep in sync with the backend code that
accesses the same stuff.

Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
Unixware....
True, but doing all that text manipulation is C is going to be very hard
to do and maintain.


What about using embedded perl? I've never done it before but the mention of it
in manpages has flashed past my eyes a couple of times so I know it's possible.

Did the discuss decide on what was required for this. Last I noticed was that
there was a distinction being made between system and user tables but I don't
recall seeing a 'requirements' summary.


What about Perl w/ C modules? Of course, there's my favorite: Python.
It's got a good facility for writing C modules, and I think it's
better for writing s/w that needs to be constantly updated.

(I swear, it's just circumstance that this particular .signature
came up at this time, but it is apropos.)

--
-----------------------------------------------------------------
Ron Johnson, Jr. ro***********@cox.net
Jefferson, LA USA

YODA: Code! Yes. A programmer's strength flows from code
maintainability. But beware of Perl. Terse syntax... more
than one way to do it...default variables. The dark side of code
maintainability are they. Easily they flow, quick to join you
when code you write. If once you start down the dark path,
forever will it dominate your destiny, consume you it will.
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 12 '05 #221


On Sat, 27 Sep 2003, Ron Johnson wrote:
Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
Unixware....


I know that Solaris now has it included by default ...
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 12 '05 #222
perl ships on UnixWare (5.005, but that will change in UP3).

LER
--On Saturday, September 27, 2003 22:42:02 -0300 "Marc G. Fournier"
<sc*****@postgresql.org> wrote:


On Sat, 27 Sep 2003, Ron Johnson wrote:
Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
Unixware....


I know that Solaris now has it included by default ...
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: le*@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (FreeBSD)

iD8DBQE/dj2ZrRNGhTxJvdYRAgvsAJ9W1/MvHYY0zydWUC4aYZBdoUn3xgCglYJB
FSQz6BZplj1bXOSv7M9TCsg=
=yE+S
-----END PGP SIGNATURE-----

Nov 12 '05 #223

$$$ -- I wasn't looking to purchase a programmer. :-)


Well sometimes it takes money to get things done. Personally I don't see
a big need
for pg_upgrade but there was enough people making noise about it that it
made sense
to make the proposal. Several people did come back and offer to cough up
a little bit
but not enough to get the project done.

My prefernce is to see all that work going into pg_dump, pg_dumpall and
pg_restore.

Sincerely,

Joshua Drake

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 12 '05 #224
On Saturday 27 September 2003 09:45 pm, Joshua D. Drake wrote:
$$$ -- I wasn't looking to purchase a programmer. :-)
Well sometimes it takes money to get things done. Personally I don't see
a big need
for pg_upgrade but there was enough people making noise about it that it
made sense
to make the proposal. Several people did come back and offer to cough up
a little bit
but not enough to get the project done.
I could always forward you my fan mail (context for the following message is
that I was extolling the group of people that help me build the various RPM
sets as an example of how backports of Fedora Core packages could be done to
'Fedora Legacy' stuff (many thanks to those who help me, BTW.)):

===================
Re: I volunteer
From: Chuck Wolber <ch****@quantumlinux.com>
To: fe***************@redhat.com
I as PostgreSQL RPM maintainer for the PostgreSQL Global Development
Group do something similar to this using a loose group of volunteers.


<TROLL>
Ahhh, so you're the one. Perhaps you could write a postgreSQL RPM with
upgrade functionality that actually works?
</TROLL>

-Chuck

--
Quantum Linux Laboratories - ACCELERATING Business with Open Technology
* Education | -=^ Ad Astra Per Aspera ^=-
* Integration | http://www.quantumlinux.com
* Support | ch****@quantumlinux.com
=====================
You know, I don't mind owning up to my own bugs. But this bug ain't mine.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 12 '05 #225


On Sat, 27 Sep 2003, Larry Rosenman wrote:
perl ships on UnixWare (5.005, but that will change in UP3).
In what way? :) It won't ship anymore ... or upgraded?
LER
--On Saturday, September 27, 2003 22:42:02 -0300 "Marc G. Fournier"
<sc*****@postgresql.org> wrote:


On Sat, 27 Sep 2003, Ron Johnson wrote:
Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
Unixware....


I know that Solaris now has it included by default ...
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: le*@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #226


--On Sunday, September 28, 2003 00:14:18 -0300 "Marc G. Fournier"
<sc*****@postgresql.org> wrote:


On Sat, 27 Sep 2003, Larry Rosenman wrote:
perl ships on UnixWare (5.005, but that will change in UP3).


In what way? :) It won't ship anymore ... or upgraded?

upgraded to 5.8.0

(sorry, should have been more clear :-))

LER
--On Saturday, September 27, 2003 22:42:02 -0300 "Marc G. Fournier"
<sc*****@postgresql.org> wrote:
>
>
> On Sat, 27 Sep 2003, Ron Johnson wrote:
>
>> Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
>> Unixware....
>
> I know that Solaris now has it included by default ...
>
>
> ---------------------------(end of
> broadcast)--------------------------- TIP 8: explain analyze is your
> friend
>


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: le*@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: le*@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (FreeBSD)

iD8DBQE/dlKfrRNGhTxJvdYRArzdAJ9CY5jRQDlOl4oLJvJSYvpGBjbr8A CeL8R9
MebY7hhFUfAU1UqZmZTAnHo=
=99HV
-----END PGP SIGNATURE-----

Nov 12 '05 #227
Ron Johnson wrote:
There's always the general point that C has more pitfalls (mainly
from pointers/free()/malloc(), and HLLs do more for you, thus you
have to code less, and, consequently, there are fewer bugs.

Someday, they're going to make a langauge called:

CBC, "C Bounds Checked"

No buffer overflows, all memory allocs and mallocs create a memory
object that self expands or contracts as necessary, or issues an
exception if it tries to go past a limit you put as an argumen to a malloc.

With gigabytes of real memory and 100 gigibytes plus of virtual memory,
the programmer should not handle memory management any more. The
consumers and software users expect programmers to give up their pride
and let go of total control of the memory model, (like they have it now
). The only excetion might be hardware drivers.

Nobody say C#, OK? An Msoft imposed solution that integrates all their
products, mistakes, football stadium sized APIs, and private backdoors
is not the answer.

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #228
On Sat, 2003-09-27 at 22:19, Dennis Gearon wrote:
Ron Johnson wrote:
There's always the general point that C has more pitfalls (mainly
from pointers/free()/malloc(), and HLLs do more for you, thus you
have to code less, and, consequently, there are fewer bugs.
Someday, they're going to make a langauge called:

CBC, "C Bounds Checked"

No buffer overflows, all memory allocs and mallocs create a memory
object that self expands or contracts as necessary, or issues an
exception if it tries to go past a limit you put as an argumen to a malloc.

With gigabytes of real memory and 100 gigibytes plus of virtual memory,
the programmer should not handle memory management any more. The
consumers and software users expect programmers to give up their pride
and let go of total control of the memory model, (like they have it now
). The only excetion might be hardware drivers.


Some would say that that's what Java and C++ are for. I'd do more
Java programming if it didn't have an API the size of Montana, no
make that Alaska and a good chunk of Siberia.

But still, multiple pointers being able to point to the same chunk
of the heap will doom any solution to inefficiency.

IMNSHO, only the kernel and *high-performance* products should be
written in C. Everything else should be written in HLLs. Anything
from COBOL (still a useful language), FORTRAN, modern BASICs, to
pointer-less Pascal, Java, Smalltalk, Lisp, and scripting languages.

Note that I did *not* mention C++.
Nobody say C#, OK? An Msoft imposed solution that integrates all their
products, mistakes, football stadium sized APIs, and private backdoors
is not the answer.


natch!

--
-----------------------------------------------------------------
Ron Johnson, Jr. ro***********@cox.net
Jefferson, LA USA

"they love our milk and honey, but preach about another way of living"
Merle Haggard, "The Fighting Side Of Me"
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org

Nov 12 '05 #229

Ron Johnson <ro***********@cox.net> writes:
Tom Lane wrote:
> The reason that it needs to be rewritten in C is that it needs access to
> internal stuff that the backend doesn't expose. (For example, the
> transaction counter, end-of-WAL pointer, etc.) I don't think Perl would
> offer anything except creating an entirely new dependency for Postgres.
> Also, C code would be easier to keep in sync with the backend code that
> accesses the same stuff.
What about Perl w/ C modules? Of course, there's my favorite: Python.


Fwiw, it's pretty easy to call out to C functions from perl code these days.

bash-2.05b$ perl -e 'use Inline C => "int a(int i,int j) { return i+j;}"; print(a(1,2),"\n")'
3

That said I don't know if this is really such a good approach. I don't see why
you would need much string manipulation at all. The C code can just construct
directly whatever data structures it needs and call directly whatever
functions it needs. Doing string manipulation to construct dynamic sql code
and then hope it gets interpreted and executed the way it's expecting seems a
roundabout way to go about getting things done.

--
greg
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #230
On Sunday 28 September 2003 09:36, Ron Johnson wrote:
On Sat, 2003-09-27 at 22:19, Dennis Gearon wrote:
Ron Johnson wrote:
There's always the general point that C has more pitfalls (mainly
from pointers/free()/malloc(), and HLLs do more for you, thus you
have to code less, and, consequently, there are fewer bugs.


Someday, they're going to make a langauge called:

CBC, "C Bounds Checked"

No buffer overflows, all memory allocs and mallocs create a memory
object that self expands or contracts as necessary, or issues an
exception if it tries to go past a limit you put as an argumen to a
malloc.

With gigabytes of real memory and 100 gigibytes plus of virtual memory,
the programmer should not handle memory management any more. The
consumers and software users expect programmers to give up their pride
and let go of total control of the memory model, (like they have it now
). The only excetion might be hardware drivers.


Some would say that that's what Java and C++ are for. I'd do more
Java programming if it didn't have an API the size of Montana, no
make that Alaska and a good chunk of Siberia.

But still, multiple pointers being able to point to the same chunk
of the heap will doom any solution to inefficiency.

IMNSHO, only the kernel and *high-performance* products should be
written in C. Everything else should be written in HLLs. Anything
from COBOL (still a useful language), FORTRAN, modern BASICs, to
pointer-less Pascal, Java, Smalltalk, Lisp, and scripting languages.

Note that I did *not* mention C++.


Duh. I would say smart pointers in C++ take care of memory errors without
adding inefficiencies and latency of garbage collection. There are plenty of
examples floating on net.

Its not about C's ability to provide built in bounds checking. Its about
programmers follow discipline, abstraction and design. Its just that C makes
those error apparent in very rude and blunt way..:-)

I hate java except for unified APIs it provides. Compensating programmers
mistake with throwing additional resources is not my idea of a good product.
But unfortunately most of the people are concerned about getting a product
out of door than giving it due attention making a robust product( Like the
one I work on.. 10 years old and still going strong..:-))

Business of software development has commoditized itself.. Its just a sad side
effect of it..

Shridhar
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 12 '05 #231
On Sat, Sep 27, 2003 at 10:42:02PM -0300, Marc G. Fournier wrote:
On Sat, 27 Sep 2003, Ron Johnson wrote:
Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
Unixware....


I know that Solaris now has it included by default ...


FWIW, FreeBSD just removed it (in the 5.x versions). Of course you can
still easily install it from ports.
--
Jim C. Nasby, Database Consultant ji*@nasby.net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 12 '05 #232
Jim C. Nasby wrote:
On Sat, Sep 27, 2003 at 10:42:02PM -0300, Marc G. Fournier wrote:
On Sat, 27 Sep 2003, Ron Johnson wrote:
Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
Unixware....


I know that Solaris now has it included by default ...


FWIW, FreeBSD just removed it (in the 5.x versions). Of course you can
still easily install it from ports.


Interesting. Why would they remove it?

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 12 '05 #233


On Sun, 28 Sep 2003, Bruce Momjian wrote:
Jim C. Nasby wrote:
On Sat, Sep 27, 2003 at 10:42:02PM -0300, Marc G. Fournier wrote:
On Sat, 27 Sep 2003, Ron Johnson wrote:

> Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
> Unixware....

I know that Solaris now has it included by default ...


FWIW, FreeBSD just removed it (in the 5.x versions). Of course you can
still easily install it from ports.


Interesting. Why would they remove it?


I can't recall the full justification, but there was alot of work done to
remove any operating dependencies on pelr so that it could be removed ...
there are several versions of perl currently 'stable', and too many ppl
crying out for "can't we have this one default", that it was just easier
to let ppl install which versino they want from ports when they install
the OS ...
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #234
On Sun, Sep 28, 2003 at 12:38:03PM -0400, Bruce Momjian wrote:
Jim C. Nasby wrote:
On Sat, Sep 27, 2003 at 10:42:02PM -0300, Marc G. Fournier wrote:
On Sat, 27 Sep 2003, Ron Johnson wrote:

> Isn't Perl pretty ubiquitous on "Unix" now, though? Except maybe
> Unixware....

I know that Solaris now has it included by default ...


FWIW, FreeBSD just removed it (in the 5.x versions). Of course you can
still easily install it from ports.


Interesting. Why would they remove it?


I believe it was essentially because it was starting to take up a good
chunk of space in the base install and it was beginning to cause
trouble. The parts of the OS that used it depended on version X, while
the user wanted version Y, etc. So they rewrote all the perl code in the
OS to use some other language and pulled it from the base distro.
There's more info to be had in the mailling list archives, either in
freebsd-stable or freebsd-current.

Realistically, many systems will still end up with perl installed, but I
can see where dedicated database servers might well not. And it'd be a
bit of a pain if the PostgreSQL port required perl. But of course this
is just one OS.
--
Jim C. Nasby, Database Consultant ji*@nasby.net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #235
pg***@candle.pha.pa.us (Bruce Momjian) writes:
Jim C. Nasby wrote:
FWIW, FreeBSD just removed it (in the 5.x versions). Of course you can
still easily install it from ports.


Interesting. Why would they remove it?


Because it's a REALLY BIG ball of mud to include as a core dependancy?

Don't get me wrong; I have several Emacs buffers presently open to
Perl programs; it's a _useful_ ball of mud. But big ball of mud it
certainly is, and it seems unremarkable that there would be some
reluctance to be dependent on it.

There's a LOT of stuff going on with Perl (Parrot + Perl6), and for
the FreeBSD folk to be reluctant to "contract" to all that change
seems unsurprising.
--
output = reverse("ofni.smrytrebil" "@" "enworbbc")
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 646 3304 x124 (land)
Nov 12 '05 #236
Christopher Browne <cb******@libertyrms.info> writes:
pg***@candle.pha.pa.us (Bruce Momjian) writes:
Jim C. Nasby wrote:
FWIW, FreeBSD just removed it (in the 5.x versions). Of course you can
still easily install it from ports.


Interesting. Why would they remove it?


Because it's a REALLY BIG ball of mud to include as a core dependancy?


/agree

Also, it's quite probable that people installing/upgrading PG would
have anything from 5.003 onward. Knowing what programming constructs
you can use and still be compatible with older versions requires quite
a bit of scholarship.

I know that if I were upgrading PG on, say, a Red Hat 6.2 box and was
forced to upgrade Perl to 5.8.1 as a dependency I'd be swearing pretty
hard...

-Doug

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 12 '05 #237

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Andrew Rawnsley | last post: by
4 posts views Thread by Chad Crowder | last post: by
8 posts views Thread by Steve Thompson | last post: by
70 posts views Thread by Anson.Stuggart | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by zhoujie | last post: by
reply views Thread by Purva khokhar | last post: by
1 post views Thread by haryvincent176 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.