An idea for IPU ( We need another acronymn! )

[Dennis Gearon]

IPU, in place updates.
--------------------------

1/ Put a version numbers on each record represented by a byte/word sized
number from a three version number list kept by the table header. This
would then only involve one more byte per record. This version number
list needs to be the VERY first item in the table header. The lowest
ordered byte of the first word of the version number list contains the
size of the words in bytes. The version number list's version number is
the second word in the structure. ( we shouldn't run out of 2^16, 2^32,
2^64, 2^128, or 2^256 versions list versions ).

2/
A/ Put a field in the table version number list showing status of
that version number:
No rows of this number.
Some rows of this number
All rows of this number.

B/ An additional field tags whiuch version number the table header
is at.

C/ The lowest version number is removed when it says it has no rows.
and the table header is at a higher number.

3/ Repeat the same for the indexes.

4/ The tables/indexes features can only be accessed by the version
marked as the table header.

5/ The read/write modules/objects for the backend is a DLL that is
changed dynamically some sort of flagging system. Any NEW backend
process gets the NEW DLL; Cuirrently executing ones keep the one they
started with.

6/ An additinonal two processes are always part of the backend:
A/ A scheduler for updates, which has programmable bandwidth
adjustment paramaters for what percentages of various resources are
given to do updates in place.
B/ The upate process itself. If all tables, indexes, and other
structures are updated, well, it's inactive.

7/ The same structure for the table and row versioning is used for the
database and table versioning, and the schema and database versioning.

8/ Once a new DLL is loaded that contains the necessary information to
read and write the old records, the uudate process starts automagically.

9/ For the listing/treeing structures changes on the
table/database/schema level, the same multiple version numbers are used
and another autonimous process slowly morphs that also. This is probably
more challenging, and might require a copy of both kinds of structures
in existence and the data slowly moving from one to the other. Until the
new structure is COMPLETELY built, the old structure contains pointers
to the locationi in the new structure if the item is there. When both
structures point completely to the new structure's contents, the old
structure's reading code is demoted and the new structure's takes over.

10/ Databases and Schemas do the same thing as 9.
anyway, some thoughts.
---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

[Richard Huxton]

On Friday 19 September 2003 15:03, Dennis Gearon wrote:

IPU, in place updates.
--------------------------

1/ Put a version numbers on each record represented by a byte/word sized
number from a three version number list kept by the table header. This
would then only involve one more byte per record. This version number
list needs to be the VERY first item in the table header. The lowest
ordered byte of the first word of the version number list contains the
size of the words in bytes. The version number list's version number is
the second word in the structure. ( we shouldn't run out of 2^16, 2^32,
2^64, 2^128, or 2^256 versions list versions ).

[snip consideration of the issues involved in migrating the on-disk formats]

Well, you could get away with one bit if you had to. You're only allowed two
versions - the old and the new. Saying that upgrading a half-upgraded
database is disallowed doesn't strike me as unreasonable.

But, the disk format is only the first step. If I'm upgrading in place then I
need to be sure that the behaviour of the system isn't going to change,
because I can't undo (otherwise, I'd just dump/restore).

So - let's say inheritence changes so that primary keys are inherited. You're
going to be in a situation where you need to manage both old and new
behaviour at the same time. In addition, you're going to need some
translation function that can update old schema details to match new schema
constraints.

OK - maybe we say changes like that can't be supported, in which case the
developers need to be careful about what changes are planned/implemented at
what points. Non-trivial stuff.

--
Richard Huxton
Archonet Ltd

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

[Bruce Momjian]

Let's get no-disk-format upgrades working first.

---------------------------------------------------------------------------

Dennis Gearon wrote:

IPU, in place updates.
--------------------------

1/ Put a version numbers on each record represented by a byte/word sized
number from a three version number list kept by the table header. This
would then only involve one more byte per record. This version number
list needs to be the VERY first item in the table header. The lowest
ordered byte of the first word of the version number list contains the
size of the words in bytes. The version number list's version number is
the second word in the structure. ( we shouldn't run out of 2^16, 2^32,
2^64, 2^128, or 2^256 versions list versions ).

2/
A/ Put a field in the table version number list showing status of
that version number:
No rows of this number.
Some rows of this number
All rows of this number.

B/ An additional field tags whiuch version number the table header
is at.

C/ The lowest version number is removed when it says it has no rows.
and the table header is at a higher number.

3/ Repeat the same for the indexes.

4/ The tables/indexes features can only be accessed by the version
marked as the table header.

5/ The read/write modules/objects for the backend is a DLL that is
changed dynamically some sort of flagging system. Any NEW backend
process gets the NEW DLL; Cuirrently executing ones keep the one they
started with.

6/ An additinonal two processes are always part of the backend:
A/ A scheduler for updates, which has programmable bandwidth
adjustment paramaters for what percentages of various resources are
given to do updates in place.
B/ The upate process itself. If all tables, indexes, and other
structures are updated, well, it's inactive.

7/ The same structure for the table and row versioning is used for the
database and table versioning, and the schema and database versioning.

8/ Once a new DLL is loaded that contains the necessary information to
read and write the old records, the uudate process starts automagically.

9/ For the listing/treeing structures changes on the
table/database/schema level, the same multiple version numbers are used
and another autonimous process slowly morphs that also. This is probably
more challenging, and might require a copy of both kinds of structures
in existence and the data slowly moving from one to the other. Until the
new structure is COMPLETELY built, the old structure contains pointers
to the locationi in the new structure if the item is there. When both
structures point completely to the new structure's contents, the old
structure's reading code is demoted and the new structure's takes over.

10/ Databases and Schemas do the same thing as 9.
anyway, some thoughts.
---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

--
Bruce Momjian | http://candle.pha.pa.us
pg***@candle.pha.pa.us | [phone]
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match