Uploaded input text files...need options

[Greg]

I need to create an aspx page that accepts an uploaded text file from a
user and parses the file contents into various, normalized database
tables/columns. The input file is an industry standard, fixed-width
file layout. The file is 'old-school' with the first three columns
representing a segment code that identifies what fields and positions
exist on the line, so it's not just a straight-forward fixed width text
file. I have toyed with a couple of ideas, but I'm a little concerned
about scalability.

I could accept the uploaded file, save as a text file on the server and
parse through it. This seems like it would very expense in terms of
scalability.

I could read the uploaded file directly into an array and parse through
it.

I could store the file directly into the sql database and then call a
stored proc to do the parsing.

If anyone has any other methods to consider or information regarding
the performance impact of any of the above methods, I would greatly
appreciate the feedback.

[bruce barker \(sqlwork.com\)]

it depends on the size of the file. if under < 4mb, then any method will
work. if greater, then you need a new serverside upload control. the asp.net
version read the complete file into memory tying up iis threads, and asp.net
threads, and hurting scaliability.

normally you can get about 20-30 simutinous uploads (but with configuration
you can hit about 250)
you want an module that caches the upload to a file (otherwise you'll force
recycles).

-- bruce (sqlwork.com)
"Greg" <ge******@aol.comwrote in message
news:11**********************@i3g2000cwc.googlegro ups.com...

>I need to create an aspx page that accepts an uploaded text file from a
user and parses the file contents into various, normalized database
tables/columns. The input file is an industry standard, fixed-width
file layout. The file is 'old-school' with the first three columns
representing a segment code that identifies what fields and positions
exist on the line, so it's not just a straight-forward fixed width text
file. I have toyed with a couple of ideas, but I'm a little concerned
about scalability.

I could accept the uploaded file, save as a text file on the server and
parse through it. This seems like it would very expense in terms of
scalability.

I could read the uploaded file directly into an array and parse through
it.

I could store the file directly into the sql database and then call a
stored proc to do the parsing.

If anyone has any other methods to consider or information regarding
the performance impact of any of the above methods, I would greatly
appreciate the feedback.

[Greg]

Thanks for the reply. The files will be small, under 50k for the most
part, so it sounds like whatever is easiest to build/maintain will
work.

bruce barker (sqlwork.com) wrote:

it depends on the size of the file. if under < 4mb, then any method will
work. if greater, then you need a new serverside upload control. the asp.net
version read the complete file into memory tying up iis threads, and asp.net
threads, and hurting scaliability.

normally you can get about 20-30 simutinous uploads (but with configuration
you can hit about 250)
you want an module that caches the upload to a file (otherwise you'll force
recycles).

-- bruce (sqlwork.com)
"Greg" <ge******@aol.comwrote in message
news:11**********************@i3g2000cwc.googlegro ups.com...

I need to create an aspx page that accepts an uploaded text file from a
user and parses the file contents into various, normalized database
tables/columns. The input file is an industry standard, fixed-width
file layout. The file is 'old-school' with the first three columns
representing a segment code that identifies what fields and positions
exist on the line, so it's not just a straight-forward fixed width text
file. I have toyed with a couple of ideas, but I'm a little concerned
about scalability.

I could accept the uploaded file, save as a text file on the server and
parse through it. This seems like it would very expense in terms of
scalability.

I could read the uploaded file directly into an array and parse through
it.

I could store the file directly into the sql database and then call a
stored proc to do the parsing.

If anyone has any other methods to consider or information regarding
the performance impact of any of the above methods, I would greatly
appreciate the feedback.