473,883 Members | 1,527 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

State of Beta 2


Anyone out there using beta 2 in production situations? Comments on
stability? I am rolling out a project in the next 4 weeks, and really
don't want to go though an upgrade soon after its released on an
Unsuspecting Client, so I would LIKE to start working with 7.4.

--------------------

Andrew Rawnsley
President
The Ravensfield Digital Resource Group, Ltd.
(740) 587-0114
www.ravensfield.com
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 11 '05
236 10127
>>>>> "TL" == Tom Lane <tg*@sss.pgh.pa .us> writes:

TL> Vivek Khera <kh***@kcilink. com> writes:
Well, for me the create index part of the restore is what takes about
3x the time for the data load. Total about 4 hours. The dump takes 1
hour.


TL> What sort_mem do you use for the restore? Have you tried increasing it?

All tests have these non-default settings:
vacuum_mem = 131702
max_fsm_pages = 1000000
random_page_cos t = 2
effective_cache _size = 12760 # `sysctl -n vfs.hibufspace` / BLKSZ
16k pages
30000 shared buffers
The four tests I've run so far are:

checkpoint_segm ents default
sort_mem 8192
restore time: 15344.57 seconds

checkpoint_segm ents default
sort_mem 131702
restore time: 15042.00 seconds

checkpoint_segm ents 50
sort_mem 8192
restore time: 11511.24 seconds

checkpoint_segm ents 50
sort_mem 131702
restore time: 11287.94 seconds
I have also enabled the extra query/timing logging you requested last
week, and just need to sanitize the table names to prevent any
confidential information from leaking out of the office. I'll send
those along shortly to you directly, as they are pretty large. I
wasn't able to do much work since the 'storm' hit on thursday,
knocking out power to the house until sunday...

Right now I'm running the same above tests with fsync=false to see if
that improves anything. Next test will be Marc's test to disable the
WAL entirely.

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: kh***@kciLink.c om Rockville, MD +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postg resql.org so that your
message can get through to the mailing list cleanly

Nov 11 '05 #201
>>>>> "MGF" == Marc G Fournier <sc*****@postgr esql.org> writes:

MGF> I've tried restoring a >5gig database with sort_mem up to 100Meg in size,
MGF> and didn't find that it sped up the index creation enough to make a
MGF> difference ... shaved off a couple of minutes over the whole reload, so
MGF> seconds off of each index ... and that was with the WAL logs also disabled
MGF> :(

Ditto for me. Can you reproduce my results by increasing
checkpoint_buff ers to some large value (I use 50)? this shaved
something like 60 minutes off of my restore.

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 11 '05 #202
Vivek Khera <kh***@kcilink. com> writes:
Ditto for me. Can you reproduce my results by increasing
checkpoint_buff ers to some large value (I use 50)?


You meant checkpoint_segm ents, right? It might be interesting to
experiment with wal_buffers, too, though I'm not convinced that will
have a big impact.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 11 '05 #203

actually, I didn't get near that kind of benefit ... with wal disabled,
and sort_mem/checkpoint_segm ents at default, I got:

import start: 22:31:38
end: 23:21:42 (~50min)
buffers: 64
sort_mem: 1024
wal disabled: yes

with checkpoint_segm ents and sort_mem raised, I shaved about 8min:

import start: 15:56:07
end: 16:38:56 (~42min)
buffers: 640
sort_mem: 102400
checkpoint_segm ents: 64
wal disabled: yes
fsync disabled: yes

As a side note, a default install with 64 shared memory buffers came in
around 56min ... then again, if looking at percentages, that is about a
25% improvement ... it just doesn't look to be that big looking at the
straight #s :)

On Tue, 23 Sep 2003, Vivek Khera wrote:
>> "MGF" == Marc G Fournier <sc*****@postgr esql.org> writes:


MGF> I've tried restoring a >5gig database with sort_mem up to 100Meg in size,
MGF> and didn't find that it sped up the index creation enough to make a
MGF> difference ... shaved off a couple of minutes over the whole reload, so
MGF> seconds off of each index ... and that was with the WAL logs also disabled
MGF> :(

Ditto for me. Can you reproduce my results by increasing
checkpoint_buff ers to some large value (I use 50)? this shaved
something like 60 minutes off of my restore.


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 11 '05 #204
Ron Johnson wrote:
On Mon, 2003-09-15 at 15:23, Joshua D. Drake wrote:
I'm not going to rehash the arguments I have made before; they are all
archived. Suffice to say you are simply wrong. The number of
complaints over the years shows that there IS a need.

I at no point suggested that there was not a need. I only suggest that
the need may not be as great as some suspect or feel. To be honest -- if
your arguments were the "need" that everyone had... it would have been
implemented some how. It hasn't yet which would suggest that the number
of people that have the "need" at your level is not as great as the
number of people who have different "needs" from PostgreSQL.


But the problem is that as more and more people put larger and larger
datasets, that are mission-critical, into PostgreSQL, the need will
grow larger and larger.

Of course, we understand the "finite resources" issue, and are not
badgering/complaining. Simply, we are trying to make our case that
this is something that should go on the TODO list, and be kept in
the back of developers' minds.


Added to TODO:

* Allow major upgrades without dump/reload, perhaps using
pg_upgrade

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.ph a.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 11 '05 #205
Under circumstances where I have had critical upgrades I have usualy
used a new machine
to build the new upgrade. This allows me to use revitalized equipment
and a "clean" install
for the upgraded server. If something goes sideways, you just switch
back to the old machine.
This is ususaly the quickest and most reliable method of upgrading a server.

Under a few circumstances I have not had a second machine, so I either
put in a new drive
and installed fresh, mounting the original drive to copy the old data to
the new drive before
any modification. Then if the upgrade goes sideways, just switch drives.
This takes longer to
recover.

When I have upgraded under the most stringent ecomonic restraints, I
have backed up the
original data and configuration files before making any changes. This is
the most error
prone method of upgrading a server, and takes the longest time to recover.

Using mirrored drives and splitting the mirror so that you have two
identical data sets can also
be feasible. I did this once successfuly but it requires having a spare
drive or two to rebuild the
mirror without losing the old data.

Andrew Sullivan wrote:
On Thu, Sep 18, 2003 at 06:49:56PM -0300, Marc G. Fournier wrote:

Hadn't thought of it that way ... but, what would prompt someone to
upgrade, then use something like erserver to roll back? All I can think
of is that the upgrade caused alot of problems with the application
itself, but in a case like that, would you have the time to be able to
're-replicate' back to the old version?


The trick is to have your former master set up as slave before you
turn your application back on.

The lack of a rollback strategy in PostgreSQL upgrades is a major
barrier for corporate use. One can only do so much testing, and it's
always possible you've missed something. You need to be able to go
back to some known-working state.

A


--
Guy Fraser
Network Administrator
The Internet Centre
780-450-6787 , 1-888-450-6787

There is a fine line between genius and lunacy, fear not, walk the
line with pride. Not all things will end up as you wanted, but you
will certainly discover things the meek and timid will miss out on.

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 11 '05 #206
>>>>> "TL" == Tom Lane <tg*@sss.pgh.pa .us> writes:

TL> Vivek Khera <kh***@kcilink. com> writes:
Ditto for me. Can you reproduce my results by increasing
checkpoint_buff ers to some large value (I use 50)?


TL> You meant checkpoint_segm ents, right? It might be interesting to
TL> experiment with wal_buffers, too, though I'm not convinced that will
TL> have a big impact.

The difference on restore with fsync=false was 2 seconds. I'm
rebuilding PG with Marc's WAL-disabling patch and will see the change
there.

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: kh***@kciLink.c om Rockville, MD +1-240-453-8497
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 12 '05 #207
Tom Lane wrote:
Dunno about MySQL. I'm pretty sure I remember Ann Harrison stating that
FireBird's disk structures haven't changed since the beginning of
Interbase. Which you might take as saying that they were a lot smarter
than we are, but I suspect what it really means is that
FireBird/Interbase hasn't undergone the kind of metamorphosis of purpose
that the Postgres code base has. Keep in mind that it started as an
experimental academic prototype (representing some successful ideas and
some not-so-successful ones), and the current developers have been
laboring to convert it into an industrial-strength production tool ---
keeping the good experimental ideas, but weeding out the bad ones, and
adding production-oriented features that weren't in the original design.
The entire argument that version-to-version stability should be a
critical goal would have been foreign to the original developers of
Postgres.


Thought the fact PostgreSQL came from an academic world are part of it,
the big reason we change on-disk format so often is that we are
improving faster than any other database on the planet.

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.ph a.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #208

With all the discussion and pg_upgrade, I saw no one offer to work on
it.

Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.

---------------------------------------------------------------------------

Lamar Owen wrote:
Marc G. Fournier wrote:
On Mon, 15 Sep 2003, Joshua D. Drake wrote:
I'm not going to rehash the arguments I have made before;I at no point suggested that there was not a need. I only suggest that
the need may not be as great as some suspect or feel. To be honest -- if
your arguments were the "need" that everyone had... it would have been
implemented some how. It hasn't yet which would suggest that the number

Just to add to this ... Bruce *did* start pg_upgrade, but I don't recall
anyone else looking at extending it ... if the *need* was so great,
someone would have step'd up and looked into adding to what was already
there ...


You'ns are going to make a liar out of me yet; I said I wasn't going to
rehash the arguments. But I am going to answer Marc's statement. Need
of the users != developer interest in implementing those. This is the
ugly fact of open source software -- it is developer-driven, not
user-driven. If it were user-driven in this case seamless upgrading
would have already happened. But the sad fact is that the people who
have the necessary knowledge of the codebase in question are so
complacent and comfortable with the current dump/reload cycle that they
really don't seem to care about the upgrade issue. That is quite a
harsh statement to make, yes, and I know that is kind of
uncharacteristi c for me. But, Marc, your statement thoroughy ignores
the archived history of this issue on the lists.

While pg_upgrade was a good first step (and I applaud Bruce for working
on it), it was promptly broken because the developers who changed the
on-disk format felt it wasn't important to make it continue working.

Stepping up to the plate on this issue will require an intimate
knowledge of the storage manager subsystem, a thorough knowledge of the
system catalogs, etc. This has been discussed at length; I'll not
repeat it. Just any old developer can't do this -- it needs the
long-term focused attention of Tom, Jan, or Bruce. And that isn't going
to happen. We know Tom's take on it; it's archived. Maybe there's
someone out there with the deep knowledge of the backend to make this
happen who cares enough about it to make it happen, and who has the time
to do it. I care enough to do the work; but I have neither the deep
knowledge necessary nor the time to make it happen. There are many in
my position. But those who could make it happen don't seem to have the
care level to do so.

And that has nothing to do with user need as a whole, since the care
level I mentioned is predicated by the developer interest level. While
I know, Marc, how the whole project got started (I have read the first
posts), and I appreciate that you, Bruce, Thomas, and Vadim started the
original core team because you were and are users of PostgreSQL, I
sincerely believe that in this instance you are out of touch with this
need of many of today's userbase. And I say that with full knowledge of
PostgreSQL Inc.'s support role. If given the choice between upgrading
capability, PITR, and Win32 support, my vote would go to upgrading.
Then migrating to PITR won't be a PITN.

What good are great features if it's a PITN to get upgraded to them?
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org


--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.ph a.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 12 '05 #209
On Saturday 27 September 2003 04:50 pm, Bruce Momjian wrote:
With all the discussion and pg_upgrade, I saw no one offer to work on
it. Does someone want to convert it to Perl? I think that would be a better
language than shell script for this purpose, and C is too low-level.


Wow, Bruce, are you _that_ behind on your e-mail? Hmmm, a perl conversion
might be interesting.... BTW, thanks for the TODO item.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 12 '05 #210

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2393
by: Eric Raymond | last post by:
When installing the rpms (or the tar file) onto Red Hat Enteprise Linux AS Beta 1 (Taroon), we get the follwing error: Installing all prepared tables /usr/bin/mysql_install_db: line 1: 7690 Segmentation fault /usr/sbin/mysqld --bootstrap --skip-grant-tables --basedir=/ --datadir=/var/lib/mysql --skip-innodb --skip-bdb Installation of grant tables failed The log file simply shows a start and a stop of the server.
1
389
by: Andrew Rawnsley | last post by:
Anyone out there using beta 2 in production situations? Comments on stability? I am rolling out a project in the next 4 weeks, and really don't want to go though an upgrade soon after its released on an Unsuspecting Client, so I would LIKE to start working with 7.4. -------------------- Andrew Rawnsley President The Ravensfield Digital Resource Group, Ltd.
4
2349
by: Chad Crowder | last post by:
I've taken a look at this article http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnaspnet/html/asp12282000.asp which someone posted a month or so ago regarding setting up SQL server to handle state data. The article references .Net beta, and the file state.sql in the framwork/1.0x directory, but that file doesn't exist for version 1.1.4322. I'm wondering if there's a component that I need to install, or if I need to...
0
1462
by: Kevin Davidson | last post by:
I am trying to get a clean installation of the .NET framework 1.1 on Windows 2000 Professional. I had previously installed a beta of the 2.0 Framework and uninstalled it. I also uninstalled framework 1.1 and 1.0 and reinstalled 1.1 and service pack 1. When I attempt to start the ASP.NET State Service, I get event log: Event Type: Error Event Source: Service Control Manager Event Category: None
1
3001
by: alundi | last post by:
Greetings: I'd like to post this as a new thread to an article in microsoft.public.dotnet.languages.vb originally made by nevin and replied to by Alan Pretre back in December ("ThreadState == not in the enumeration"). Like nevin, I have a VB.NET multi-threaded application, and quite frequently, after my main application class will call Thread.Suspend() on the thread the ThreadState of the Suspended Thread will be a value of 96. From...
0
1004
by: Shell | last post by:
Hi All, We have an application that uses SQL Server 2000 for State Management data. We previously use to build and deploy this application using ASP.NET Beta 2 version. Now we are using ASP.NET RTM version.With that, we are able to build & deploy the application successfully. But it gives us the error "The Session State information is valid or might be corrupted." while retrieving data from SQL Server. We can see the data stored...
8
2263
by: Steve Thompson | last post by:
Hello all, I was wondering the differnced there were betwee Active State's python and the open source version of python. Would I have to unistall my opend souce python? Additonally, how does Active State's Komodo IDE vs. the eric3 IDE unler SuSE Linux v. 10.i? Addionally, is the eric IDE (version 3) an acceptible IDE or are there more easy and more productive IDE's for perl?
3
8719
by: Nathan Sokalski | last post by:
I am recieving the following error on the second postback of a page I have written: The state information is invalid for this page and might be corrupted Stack Trace: System.Convert.FromBase64String(String s) +0
70
4368
by: Anson.Stuggart | last post by:
I'm designing a debounce filter using Finite State Machine. The FSM behavior is it follows the inital input bit and thinks that's real output until it receives 3 consecutive same bits and it changes output to that 3 consecutive bit until next 3 consecutive bits are received. A reset will set the FSM to output 1s until it receives the correct input and ouput. This is the test sequence with input and correct output. 1 0 0 1 0 1 0 0 0 1...
0
9936
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9791
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10742
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9571
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7970
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7123
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5797
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5990
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4609
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.