Article Storage: Files vs. Database

[Jonathan Wood]

I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly indexed
by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and efficiency.

Some advantages of storing them in the database are ease of some operations,
and the option of using SQL Server 2005 text index to implement search.

Can anyone else offer some considerations for choosing between these two
approaches?

Thanks.

Jonathan

[Peter Bromberg [C# MVP]]

We use databases almost exclusively for this kind of "stuff". Its easier to
search, index, etc. and the content can be populated into a "templatize d"
article page. It can also be cached via Partial page caching.
Peter

"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:%2******** ********@TK2MSF TNGP02.phx.gbl. ..

I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly
indexed by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and efficiency.

Some advantages of storing them in the database are ease of some
operations, and the option of using SQL Server 2005 text index to
implement search.

Can anyone else offer some considerations for choosing between these two
approaches?

Thanks.

Jonathan

[Jonathan Wood]

Yeah, these are definitely some of the advantages. Have you made use of SQL
Server 2005's full-text indexing yet? With file-based articles, implementing
search is a pain.

Also, would love to see some samples of the sites you are referring to if
any of them are public.

Jonathan

"Peter Bromberg [C# MVP]" <pb*******@nosp ammin.yahoo.com wrote in message
news:83******** *************** ***********@mic rosoft.com...

We use databases almost exclusively for this kind of "stuff". Its easier
to search, index, etc. and the content can be populated into a
"templatize d" article page. It can also be cached via Partial page
caching.
Peter

"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:%2******** ********@TK2MSF TNGP02.phx.gbl. ..

>I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly
indexed by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and efficiency.

Some advantages of storing them in the database are ease of some
operations, and the option of using SQL Server 2005 text index to
implement search.

Can anyone else offer some considerations for choosing between these two
approaches?

Thanks.

Jonathan

[George Ter-Saakov]

Implementing search functionality is pain.You might do it yourself or get
some third party solution like dtSearch or open source (just google "serarch
engine open source")

But from my experience the SQL server's full-text do not do a good job.

George.
"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:eL******** ********@TK2MSF TNGP05.phx.gbl. ..

Yeah, these are definitely some of the advantages. Have you made use of
SQL Server 2005's full-text indexing yet? With file-based articles,
implementing search is a pain.

Also, would love to see some samples of the sites you are referring to if
any of them are public.

Jonathan

"Peter Bromberg [C# MVP]" <pb*******@nosp ammin.yahoo.com wrote in message
news:83******** *************** ***********@mic rosoft.com...

>We use databases almost exclusively for this kind of "stuff". Its easier
to search, index, etc. and the content can be populated into a
"templatized " article page. It can also be cached via Partial page
caching.
Peter

"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:%2******* *********@TK2MS FTNGP02.phx.gbl ...

>>I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly
indexed by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and efficiency.

Some advantages of storing them in the database are ease of some
operations, and the option of using SQL Server 2005 text index to
implement search.

Can anyone else offer some considerations for choosing between these two
approaches?

Thanks.

Jonathan

[Patrice]

Unless you have really simple needs (is this a static site now and for the
foreseeable future ?) I would use a DB. It's likely you'll have anyway to
store other data thna just the description for each article (price,
availability, mark them as deleted, category, user ratings or whatever
else).

--
Patrice

"Jonathan Wood" <jw***@softcirc uits.coma écrit dans le message de groupe
de discussion : #L************* *@TK2MSFTNGP02. phx.gbl...

I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly
indexed by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and efficiency.

Some advantages of storing them in the database are ease of some
operations, and the option of using SQL Server 2005 text index to
implement search.

Can anyone else offer some considerations for choosing between these two
approaches?

Thanks.

Jonathan

[Jonathan Wood]

Can you elaborate on this? The full-text indexing was created for exactly
this purpose. Why does it not do a good job, or is a pain to use?

Thanks.

Jonathan

"George Ter-Saakov" <gt****@cardone .comwrote in message
news:uF******** ******@TK2MSFTN GP02.phx.gbl...

Implementing search functionality is pain.You might do it yourself or get
some third party solution like dtSearch or open source (just google
"serarch engine open source")

But from my experience the SQL server's full-text do not do a good job.

George.
"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:eL******** ********@TK2MSF TNGP05.phx.gbl. ..

>Yeah, these are definitely some of the advantages. Have you made use of
SQL Server 2005's full-text indexing yet? With file-based articles,
implementing search is a pain.

Also, would love to see some samples of the sites you are referring to if
any of them are public.

Jonathan

"Peter Bromberg [C# MVP]" <pb*******@nosp ammin.yahoo.com wrote in
message news:83******** *************** ***********@mic rosoft.com...

>>We use databases almost exclusively for this kind of "stuff". Its easier
to search, index, etc. and the content can be populated into a
"templatize d" article page. It can also be cached via Partial page
caching.
Peter

"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:%2****** **********@TK2M SFTNGP02.phx.gb l...
I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly
indexed by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and
efficiency .

Some advantages of storing them in the database are ease of some
operations , and the option of using SQL Server 2005 text index to
implement search.

Can anyone else offer some considerations for choosing between these
two approaches?

Thanks.

Jonathan

[Jonathan Wood]

Yes, I agree that a database will be needed for one purpose or another.

But then how good are my options for implementing search functionality?

Thanks.

Jonathan

"Patrice" <http://www.chez.com/scribe/wrote in message
news:10******** *************** ***********@mic rosoft.com...

Unless you have really simple needs (is this a static site now and for the
foreseeable future ?) I would use a DB. It's likely you'll have anyway to
store other data thna just the description for each article (price,
availability, mark them as deleted, category, user ratings or whatever
else).

--
Patrice

"Jonathan Wood" <jw***@softcirc uits.coma écrit dans le message de groupe
de discussion : #L************* *@TK2MSFTNGP02. phx.gbl...

>I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly
indexed by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and efficiency.

Some advantages of storing them in the database are ease of some
operations, and the option of using SQL Server 2005 text index to
implement search.

Can anyone else offer some considerations for choosing between these two
approaches?

Thanks.

Jonathan

[Jonathan Wood]

Any decent article site MUST have good full-text searching.

--
Jonathan Wood
SoftCircuits Programming
http://www.softcircuits.com

"Patrice" <http://www.chez.com/scribe/wrote in message
news:E7******** *************** ***********@mic rosoft.com...

Which kind of search ? LIKE would be my first option for text search.
Depends then on how much rows, how much text columns in which to search
for keywords, their size, if you can filter rows using better criteria
before applying LIKE, if users tends to always use the same keywords
etc...

Only then you can consider full text search (even SQL Express (with
advanced services) has now full text search)...

--
Patrice

"Jonathan Wood" <jw***@softcirc uits.coma écrit dans le message de groupe
de discussion : uf************* *@TK2MSFTNGP06. phx.gbl...

>Yes, I agree that a database will be needed for one purpose or another.

But then how good are my options for implementing search functionality?

Thanks.

Jonathan

"Patrice" <http://www.chez.com/scribe/wrote in message
news:10******* *************** ************@mi crosoft.com...

>>Unless you have really simple needs (is this a static site now and for
the foreseeable future ?) I would use a DB. It's likely you'll have
anyway to store other data thna just the description for each article
(price, availability, mark them as deleted, category, user ratings or
whatever else).

--
Patrice

"Jonathan Wood" <jw***@softcirc uits.coma écrit dans le message de
groupe de discussion : #L************* *@TK2MSFTNGP02. phx.gbl...
I'd like to build a Website that contains many articles. Two basic
approaches are to either store the articles in aspx files, possibly
indexed by the database, or to store the article text in the database.

Some advantages of storing them in files are simplicity, and
efficiency .

Some advantages of storing them in the database are ease of some
operations , and the option of using SQL Server 2005 text index to
implement search.

Can anyone else offer some considerations for choosing between these
two approaches?

Thanks.

Jonathan

[Jonathan Wood]

Of those you listed, plural vs singular would be my biggest worry.

Yeah, I understand people don't know how to spell. But if their search
doesn't turn up anything, then I'm not going to feel too guilty when they
misspelled something. Although I understand it's a bit different if your
site is selling stuff, in which case, the site loses when they don't find
what they're looking for.

Thanks.

--
Jonathan Wood
SoftCircuits Programming
http://www.softcircuits.com
"George Ter-Saakov" <gt****@cardone .comwrote in message
news:eJ******** ******@TK2MSFTN GP05.phx.gbl...

In reality it never works well.
I have a web site http://www.mspiercing.com if you look at it you going to
see search box there.

First problem I had is misspells. People on internet do not know how to
spell things. And SQL's full text does not work with misspells.

Another problem was plural vs singular form.

Another problem was weights. If person is looking for "belly ring" I want
the search to find and rank items with word "belly" first simply cause
"ring" exists in pretty much any item.
George.

"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:OI******** ******@TK2MSFTN GP06.phx.gbl...

>Can you elaborate on this? The full-text indexing was created for exactly
this purpose. Why does it not do a good job, or is a pain to use?

Thanks.

Jonathan

"George Ter-Saakov" <gt****@cardone .comwrote in message
news:uF******* *******@TK2MSFT NGP02.phx.gbl.. .

>>Implementin g search functionality is pain.You might do it yourself or
get some third party solution like dtSearch or open source (just google
"serarch engine open source")

But from my experience the SQL server's full-text do not do a good job.

George.
"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:eL****** **********@TK2M SFTNGP05.phx.gb l...
Yeah, these are definitely some of the advantages. Have you made use of
SQL Server 2005's full-text indexing yet? With file-based articles,
implementi ng search is a pain.

Also, would love to see some samples of the sites you are referring to
if any of them are public.

Jonathan

"Peter Bromberg [C# MVP]" <pb*******@nosp ammin.yahoo.com wrote in
message news:83******** *************** ***********@mic rosoft.com...
We use databases almost exclusively for this kind of "stuff". Its
easier to search, index, etc. and the content can be populated into a
"templatize d" article page. It can also be cached via Partial page
caching.
Peter
>
"Jonathan Wood" <jw***@softcirc uits.comwrote in message
news:%2**** ************@TK 2MSFTNGP02.phx. gbl...
>I'd like to build a Website that contains many articles. Two basic
>approach es are to either store the articles in aspx files, possibly
>indexed by the database, or to store the article text in the
>database .
>>
>Some advantages of storing them in files are simplicity, and
>efficiency .
>>
>Some advantages of storing them in the database are ease of some
>operations , and the option of using SQL Server 2005 text index to
>implemen t search.
>>
>Can anyone else offer some considerations for choosing between these
>two approaches?
>>
>Thanks.
>>
>Jonathan
>>
>

[Jonathan Wood]

George,

You said "I hear all the time how loading a straight file is more
efficient"
And my point was that this is an illusion. That is why many people still
believe it. And you hear it all the time.
File System is designed as a sequential list. If you need to find file
a.txt in it there is no other way but to scan the whole list.

I understand that. My understanding was that transferring the content was
faster from a file that transferring it over a database connection, and not
so much look-up time. My understanding was also that this has been well
testing and documented. But I will certainly admit I haven't ran any tests
myself.

Also MS was planning for long time ago move file system to MS SQL engine.
Not sure if it's still in works or not.

Not sure how great that sounds. Some sort of index to allow a binary lookup
makes sense though.

--
Jonathan Wood
SoftCircuits Programming
http://www.softcircuits.com