Anybody using DB2 9 on Linux x86, please, examine...

[Konstantin Andreev]

I'm almost sure I've found bad bug, but for a while I can't neither confirm nor reject this.

If anybody could make an independent test on it's own system, I'd appreciate it very much.

The possible bug is:

*************** *
*** DB2 refuses DEVICE containers bigger than 4Gb. ***
*************** *

DB2 version affected ( at least ):

| # db2level
| DB21085I Instance "dbi" uses "32" bits and DB2 code release "SQL09012" with level identifier "01030107".
| Informational tokens are "DB2 v9.1.0.2", "s070210", "MI00183", and Fix Pack "2".

The test is rather simple: just try in any database

| # db2 create tablespace X pagesize 4096 managed by database using ( device '/some/device' 1048576 )

The device must be bigger than 4G, and amount of pages indicated to DB2: any number >= 1048576 and <= (device size). The 1048576 because 1048576 * 4096 = 0x100000000, that is 4G.

If you get in response:
| SQL1422N The size of the container is invalid. SQLSTATE=54039

then you are affected too.
--
Konstantin Andreev.

[Serge Rielau]

Which part of:
"...it will be fixed in the next fixpack of DB2 9."
from Haider's post was unsatisfying?
DB2 Development is not in the habit of refusing fixes by blaming the OS.
There will be a fix (period).

Cheers
Serge

--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab

[Konstantin Andreev]

Serge Rielau wrote:

Which part of: "...it will be fixed in the next fixpack of DB2 9." from Haider's post was unsatisfying?

It was not clear that Haider and I discussed the same bug: "...a similar problem was found ...". Haider blamed the broken BLKGETSIZE64 ioctl() call, whereas it's obvious for me, that ioctl is correct. *Nobody* had confirmed the issue I introduced in that thread, so I decided to raise wider testing.

After your confirmation this testing becomes excessive.
Thank you.
--
Konstantin Andreev.

[Jan M. Nelken]

Konstantin Andreev wrote:

I'm almost sure I've found bad bug, but for a while I can't neither
confirm nor reject this.

If anybody could make an independent test on it's own system, I'd
appreciate it very much.

I have confirmed this and posted reply in your previous thread at 18:32
- an *one hour* before you posted this continuation of your saga.

What else do you want?

APAR? the open Apar is LI71236.

Line numbers of line code changed?

If three people are telling you that it will be fixed in DB2 (not Linux)
upcoming Fix Pack 3 - may be you can accept fact that it *will be* fixed
in FP3.

Any further alarming news will have to be assumed to be bad taste trolling.
Jan M. Nelken

[Jan M. Nelken]

Actually - if you *do need* solution before FP3 is available, I can make
special build containing patch which fixes this issue available to you.

From what I can see now - it appears that a solution implemented was to
work around incorrect answers received from ioctl BLKGETSIZE64 - as
Haider indicated.

Solution was to code a work around this issue - this is a fix for which
a special build based on Fixpack 2 can be made available to you - and
which most likely will be coded in FP3.
Jan M. Nelken

[Philip Nelson]

Jan M. Nelken wrote:

>
Actually - if you *do need* solution before FP3 is available, I can make
special build containing patch which fixes this issue available to you.

From what I can see now - it appears that a solution implemented was to
work around incorrect answers received from ioctl BLKGETSIZE64 - as
Haider indicated.

Solution was to code a work around this issue - this is a fix for which
a special build based on Fixpack 2 can be made available to you - and
which most likely will be coded in FP3.
Jan M. Nelken

I guess there are a number of simple workarounds -

1) Use file based containers (just allocated the device as a file system and
then creating the tablespace on it). This works for me -

db2inst1@mahler :~/db2datadb2 "create tablespace sairbig1 managed by
database using (file '/home/db2inst1/db2data/test/part0001' 2000000)"
DB20000I The SQL command completed successfully.

2) Use compression, if not already doing so, to limit the size requirement

3) Use automatic storage (which is virtually equivalent to 1).

4) Use multiple smaller containers

5) Use partitioning (which is almost equivalent to 4).

I guess that the fact it has been so long before this was noticed means one
of a number of things -

a) Virtually nobody is using raw devices any more (and IMHO in most cases
any gain in performance is now so small that it is only justified in a very
small number of cases)

b) Nobody on Linux is creating that amount of data (I know that isn't true)

c) Everybody is using x86_64 Linux for servers rather than x86

Phil

[Konstantin Andreev]

Hello, Jan. You wrote:

>I'm almost sure I've found bad bug, but for a while I can't neither confirm nor reject this.
If anybody could make an independent test on it's own system, I'd appreciate it very much.

I have confirmed this and posted reply in your previous thread at 18:32 - an *one hour* before you posted this continuation of your saga

Maybe, I wouldn't start a new thread if I caught your reply on time.

What else do you want?

I want to hear either of two clear statements:

1) "Yes, DB2 9.1.0.X on Linux x86 is *unable* to use devices containers >=4G, whichever distro you use".
2) "DB2 confirmed to able to use devices >4G on <THATdistro."

If I know a working distro, I just switch to.

From Haider note anyone can conclude: There are distros where the bug appears. So, there are distros where it doesn't.

From my point of view, this is unlikely (my investigation points to the bug in DB2), but I can be wrong. I'm already tryed 3 (three) different distros (gentoo, rhel, debian), all failed. I can't try all of them. In this situation, nothing wrong asking the community to find the working distro.

However, from this thread I can conclude that statement (1) is near the truth.

Actually - if you *do need* solution before FP3 is available, I can make special build containing patch which fixes this issue available to you..

Thank you for suggestion, but I can work around while FP3 is preparing.

From what I can see now - it appears that a solution implemented was to work around incorrect answers received from ioctl BLKGETSIZE64

This is not developer's conference, but... you blame this ioctl() for incorrectness. Even if so, the bug I introduced here is completely *unrelated* to ioctl. The bug appears when ioctl is definitly correct, and completely pertains to the DB2.
--
Konstantin Andreev.

[Konstantin Andreev]

Philip Nelson wrote:

I guess there are a number of simple workarounds -

[... skip ...]

I guess that the fact it has been so long before this was noticed means one of a number of things -

a) Virtually nobody is using raw devices any more (and IMHO in most cases any gain in performance is now so small that it is only justified in a very small number of cases)
b) Nobody on Linux is creating that amount of data (I know that isn't true)
c) Everybody is using x86_64 Linux for servers rather than x86

Strongly agree. I could suppose:

d) it's harder to setup a lot of device containers comparing to setup a lot of files.

Say, standard Peoplesoft installation uses about 100 (one hundred) tablespaces! I've spent a time, planning disks subsetting, and hope to get noticeable performance gain. Too bad, If I am wrong...
--
Konstantin Andreev.

[Jan M. Nelken]

Konstantin Andreev wrote:

Maybe, I wouldn't start a new thread if I caught your reply on time.

Please continue this discussion - if you must - in one thread only.

From my point of view, this is unlikely (my investigation points to the
bug in DB2), but I can be wrong.

We have coded around for ioctl BLKGETSIZE64 incorrect results. This
workaround is available in FP3.

For your convenience I tested ioctl BLKGETSIZE64 results:

--------------------------------------------------------------------
[root@os2rules ~]# fdisk -l

Disk /dev/hda: 40.0 GB, 40027029504 bytes
255 heads, 63 sectors/track, 4866 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 1 8001 a OS/2 Boot Manager
/dev/hda2 2 260 2080386 6 FAT16
/dev/hda3 261 4866 36997695 f W95 Ext'd (LBA)
/dev/hda5 261 1569 10514511 7 HPFS/NTFS
/dev/hda6 1570 1763 1558273+ 82 Linux swap / Solaris
/dev/hda7 1764 4866 24924816 83 Linux
[root@os2rules ~]# /home/db2admin/GetDevSize /dev/hda1
BLKGETSIZE64: Device /dev/hda1: rc=0 - 8193024 sectors, 2000 4K pages
[root@os2rules ~]# /home/db2admin/GetDevSize /dev/hda2
BLKGETSIZE64: Device /dev/hda2: rc=0 - 2130315264 sectors, 520096 4K pages
[root@os2rules ~]# /home/db2admin/GetDevSize /dev/hda3
BLKGETSIZE64: Device /dev/hda3: rc=0 - 1024 sectors, 0 4K pages
[root@os2rules ~]# /home/db2admin/GetDevSize /dev/hda5
BLKGETSIZE64: Device /dev/hda5: rc=0 - 2176924672 sectors, 531475 4K pages
[root@os2rules ~]# /home/db2admin/GetDevSize /dev/hda6
BLKGETSIZE64: Device /dev/hda6: rc=0 - 1595672064 sectors, 389568 4K pages
[root@os2rules ~]# /home/db2admin/GetDevSize /dev/hda7
BLKGETSIZE64: Device /dev/hda7: rc=0 - 4048175104 sectors, 988324 4K pages
[root@os2rules ~]#
--------------------------------------------------------------------

As you can see - number of sectors returned is wrong in all cases.
There are no DB2 functions in test program which I am enclosing here as
well:
-----------------------------------------------------------------------
/* Compile with "g++ -o GetDevSize -D_FILE_OFFSET_B ITS=64 GetDevSize.c" */

#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/statvfs.h>
#include <sys/param.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <errno.h>
#include <getopt.h>

#define BLKGETSIZE64 _IOR(0x12,114,s ize_t)
#define BLKGETSIZE _IO(0x12, 96)

int main (int argc, char *argv[])
{
int fh;
int rc;
size_t sectors = 0;
int pagesize = 4;
char *device = NULL;

if (argc < 2)
{
printf("\nSynta x: GetDevSize <device>\n\n" );
rc = -1;
return rc;
}

device = argv[1];

fh = open(device, O_RDONLY | O_DIRECT | O_LARGEFILE , 0600);
if ( -1 == fh )
{
printf("Error opening device \"%s\": %x\n", device, errno);

1,1 Top
-----------------------------------------------------------------------

Jan M. Nelken

[Konstantin Andreev]

Jan M. Nelken wrote:

> From my point of view, this is unlikely (my investigation points to the bug in DB2), but I can be wrong.

For your convenience I tested ioctl BLKGETSIZE64 results:
As you can see - number of sectors returned is wrong in all cases.

LOL

(fdisk Blocks: 8001) BLKGETSIZE64: /dev/hda1: 8193024 sectors

CORRECT

(fdisk Blocks: 2080386) BLKGETSIZE64: /dev/hda2: 2130315264 sectors

CORRECT

(fdisk Blocks: 36997695) BLKGETSIZE64: /dev/hda3: 1024 sectors

WRONG

(fdisk Blocks: 10514511) BLKGETSIZE64: /dev/hda5: 2176924672 sectors

WRONG

(fdisk Blocks: 1558273+) BLKGETSIZE64: /dev/hda6: 1595672064 sectors

CORRECT

(fdisk Blocks: 24924816) BLKGETSIZE64: /dev/hda7: 4048175104 sectors

WRONG

I see 3 correct results, and 3 wrong results. But it's likely something wrong in your code, not in ioctl(), because you mistakenly count BYTES returned by ioctl(BLKGETSIZ E64) as SECTORS.

I can not review your code, because it's omitted, but you could try this (see below) code instead (really, anyone invited to). I've never seen incorrect results from it.
--
Konstantin Andreev.

--( tests )-----------------------------------
root@life dev# fdisk -l /dev/hda

Disk /dev/hda: 16 heads, 63 sectors, 19386 cylinders
Units = cylinders of 1008 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 1 18966 9558643+ 83 Linux
/dev/hda2 18966 19380 208845 82 Linux swap

root@life dev# ./chkdev /dev/hda{,1,2}
/dev/hda: ior: (0,0) blocks: 19541088 (of 512) bytes: 10005037056 (0x25458c000) 4K pages: 2442636
/dev/hda1: ior: (0,0) blocks: 19117287 (of 512) bytes: 9788050944 (0x24769ce00) 4K pages: 2389660(+ 3584 bytes)
/dev/hda2: ior: (0,0) blocks: 417690 (of 512) bytes: 213857280 (0xcbf3400) 4K pages: 52211(+ 1024 bytes)

avalon db2 # blockdev --getsize /dev/xd{1,2,3}
234441648
234441648
781422768

avalon db2 # ./chkdev /dev/xd{1,2,3}
/dev/xd1: ior: (0,0) blocks: 234441648 (of 512) bytes: 120034123776 (0x1bf2976000) 4K pages: 29305206
/dev/xd2: ior: (0,0) blocks: 234441648 (of 512) bytes: 120034123776 (0x1bf2976000) 4K pages: 29305206
/dev/xd3: ior: (0,0) blocks: 781422768 (of 512) bytes: 400088457216 (0x5d27216000) 4K pages: 97677846

avalon db2 # echo "0 8388616 linear /dev/xd1 48297272" | dmsetup create foo
avalon db2 # ./chkdev /dev/mapper/foo
/dev/mapper/foo: ior: (0,0) blocks: 8388616 (of 512) bytes: 4294971392 (0x100001000) 4K pages: 1048577

root@life dev# uname -a
Linux life 2.6.17.6lf39 #5 PREEMPT Thu Jul 20 15:47:40 MSD 2006 i686 i686 i386 GNU/Linux

avalon db2 # uname -a
Linux avalon 2.6.21.1-av11 #6 SMP PREEMPT Thu May 10 22:12:16 MSD 2007 i686 Intel(R) Pentium(R) 4 CPU 3.20GHz GenuineIntel GNU/Linux

--( source )-----------------------------------
/* Compile with "gcc -Wall chkdev.c -o chkdev" */
#define _GNU_SOURCE

#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdarg.h>
#include <sys/ioctl.h>
#include <stdint.h>

#define BLKGETSIZE _IO(0x12,96)
#define BLKGETSIZE64 0x80041272
#define PAGESIZE 4096

void libcerr( const char *format, ... );
const char *myname = NULL;

int main( int argc, char *argv[] ) {
int io1, io2, fd, j;

uint32_t blocks;
uint64_t bytes;

myname = basename( argv[0] );

for( j = 1; j < argc; ++ j ) {
fd = open( argv[j], O_RDONLY|O_DIRE CT|O_LARGEFILE| O_NONBLOCK );
if( fd < 0 ) {
libcerr( "open(%s)", argv[j] );
continue;
}
io1 = ioctl( fd, BLKGETSIZE, & blocks );
io2 = ioctl( fd, BLKGETSIZE64, & bytes );
if( io1 == -1 )
libcerr( "ioctl(%s,BLKGE TSIZE)", argv[j] );
if( io2 == -1 )
libcerr( "ioctl(%s,BLKGE TSIZE64)", argv[j] );
close( fd );

// ***** print results *****
printf( "%s: ior: (%i,%i)", argv[j], io1, io2 );

if( io1 != -1 ) {
printf( " blocks: %u", blocks );
if( io2 != -1 )
printf( " (of %llu)", bytes / blocks );
}

if( io2 != -1 ) {
printf( " bytes: %llu (0x%llx) 4K pages: %llu",
bytes, bytes, bytes / PAGESIZE );
if( 0 != (bytes %= PAGESIZE) )
printf( "(+ %u bytes)", (unsigned)bytes );
}

putc( '\n', stdout );
} // next device
return 0;
}

void libcerr( const char *format, ... ) {
va_list v;
fwrite( myname, 1, strlen(myname), stderr );
fwrite( ": ", 1, 2, stderr );
va_start( v, format );
vfprintf( stderr, format, v );
va_end( v );
fprintf( stderr, ": [%i] %s.\n", errno, strerror( errno ));
}
--( end )-----------------------------------