By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,190 Members | 765 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,190 IT Pros & Developers. It's quick & easy.

hundred thousands BLOBs in database

P: n/a
Hi

We intend create system to store documents in company.
Size of documents: 100k-2M, hundred thousands or milions files.
We are considering the possibility to store documents in DB2 or in
filesystem.

Can anybody share your experience?

What is performance solution with files in db? Have you any problems with
that?

I am looking for articles, links, hints, advantages and disadvantages for
this solutions.

Thanks in advance
Yaro
Jan 26 '06 #1
Share this Question
Share on Google+
8 Replies


P: n/a
Yaro wrote:
Hi

We intend create system to store documents in company.
Size of documents: 100k-2M, hundred thousands or milions files.
We are considering the possibility to store documents in DB2 or in
filesystem.

Can anybody share your experience?

What is performance solution with files in db? Have you any problems with
that?

I am looking for articles, links, hints, advantages and disadvantages for
this solutions.

Thanks in advance
Yaro


Yaro,

A few initial thoughts based on my (much smaller volume) experiences, if you
are going to use LOB columns -

1) Make sure you specify COMPACT on the column definition
2) Define separate DMS tablespaces for data, index and LOB objects
3) How are you going to back it all up ?
4) Are you intending to log the LOBs ? If so how are you going to handle
the log files, and be careful of individual log file sizes.
5) Perhaps consider disabling file system caching
6) Check that you set maximum file size and disk quotas appropriately (or
make them unlimited if possible)

HTH

Phil
Jan 26 '06 #2

P: n/a
Yaro wrote:
Hi

We intend create system to store documents in company.
Size of documents: 100k-2M, hundred thousands or milions files.
We are considering the possibility to store documents in DB2 or in
filesystem.

Can anybody share your experience?

What is performance solution with files in db? Have you any problems with
that?

I am looking for articles, links, hints, advantages and disadvantages for
this solutions.

Since storing and presumably managing these documents is the core
function of this system I'm wondering whether this discussion should be
more appropriately held around DB2 Content Manager instead of
re-inventing the wheel on the DBMS level.

Cheers
Serge
--
Serge Rielau
DB2 Solutions Development
DB2 UDB for Linux, Unix, Windows
IBM Toronto Lab
Jan 26 '06 #3

P: n/a
Thank you for the reply
A few initial thoughts based on my (much smaller volume) experiences, if
you are going to use LOB columns -
1) Make sure you specify COMPACT on the column definition
2) Define separate DMS tablespaces for data, index and LOB objects
3) How are you going to back it all up ?
4) Are you intending to log the LOBs ? If so how are you going to handle
the log files, and be careful of individual log file sizes.
It is one of my biggest concern.
Can I switch logging LOBs off? Maybe for table or tablespace? I didn't see
option like that.
5) Perhaps consider disabling file system caching
6) Check that you set maximum file size and disk quotas appropriately (or
make them unlimited if possible)
HTH
Phil


Thanks
Yaro


Jan 26 '06 #4

P: n/a
Since storing and presumably managing these documents is the core
function of this system I'm wondering whether this discussion should be
more appropriately held around DB2 Content Manager instead of re-inventing
the wheel on the DBMS level.


We are not familiar with this product.
Thanks for hint

Yaro

Jan 26 '06 #5

P: n/a
Yaro wrote:
Can I switch logging LOBs off? Maybe for table or tablespace? I didn't see
option like that.


Yes, you can. Use the option NOT LOGGED (see the CREATE TABLE statement
documentation) for that.

--
Knut Stolze
DB2 Information Integration Development
IBM Germany
Jan 26 '06 #6

P: n/a
Yaro wrote:
Since storing and presumably managing these documents is the core
function of this system I'm wondering whether this discussion should be
more appropriately held around DB2 Content Manager instead of re-inventing
the wheel on the DBMS level.

We are not familiar with this product.
Thanks for hint

Here is a link:
http://www-306.ibm.com/software/data/cm/
Maybe worthwhile to give IBM a shout, maybe get a briefing....

Cheers
Serge
--
Serge Rielau
DB2 Solutions Development
DB2 UDB for Linux, Unix, Windows
IBM Toronto Lab
Jan 26 '06 #7

P: n/a
Yaro schrieb:
Hi

We intend create system to store documents in company.
Size of documents: 100k-2M, hundred thousands or milions files.
We are considering the possibility to store documents in DB2 or in
filesystem.

Can anybody share your experience?

What is performance solution with files in db? Have you any problems with
that?

I am looking for articles, links, hints, advantages and disadvantages for
this solutions.

Thanks in advance
Yaro


I created a DMS system and I decided for myself, to store the documents
NOT within the database. After 6 years of usage I've about 450000
documents with about 800000 renderings (a document can be available in
different renderings ... formats). These documents need about 80GB
(or compressed 50 GB) for themselves.

The database itselfs (project oriented document management system)
needs about 1.2 GB.

When goig this way, one has to consider how to store the files within
the filesystem. Especially, when a Windows system is the server.

Unix/Linux systems have no performance problems with such large number
of files, but a Windows server can step into big trouble here.

Having the files within the filesystem also means, that you have to
have a transfer path from the storage server to the client, looking
fo a special file.
I do not assume, that you want to have these files in mounted
filesystem, visible for all users !?
One way could be FTP (remember XP-SP2 firewall !) ...

Having the file in filesystems, make them very easily to backup :-)

Marten

Jan 26 '06 #8

P: n/a
Thank you Marten

Your post is very helpful

Yaro
Jan 27 '06 #9

This discussion thread is closed

Replies have been disabled for this discussion.