473,231 Members | 1,460 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,231 software developers and data experts.

Flat files and ASP.NET

Hello

I am developing a system for user tracking. In this I am tracking all the
visitors that came to my website. i developed this using a HTTPModule and
storing the data in a SQL Server database. Everything worked fine till my
website gets good traffic.

When website is getting good traffic, this method is slowing down the entire
application. So I am planning to use flat files instead of SQL Server
database. My doubts are

1 - What is a flat file and how can I connect to it from my ASP.NET
application
2 - If multiple users are browsing website at same time, how can I save
data's to file, because that may be in use.
3 - Is XML files are normal flat files ?

Please help me on this issue.

Sincerly
Navaneeth
Aug 30 '07 #1
9 1861
On 30 , 10:06, Navaneeth.K.N
<Navaneeth....@discussions.microsoft.comwrote:
Hello

I am developing a system for user tracking. In this I am tracking all the
visitors that came to my website. i developed this using a HTTPModule and
storing the data in a SQL Server database. Everything worked fine till my
website gets good traffic.

When website is getting good traffic, this method is slowing down the entire
application. So I am planning to use flat files instead of SQL Server
database. My doubts are

1 - What is a flat file and how can I connect to it from my ASP.NET
application
2 - If multiple users are browsing website at same time, how can I save
data's to file, because that may be in use.
3 - Is XML files are normal flat files ?

Please help me on this issue.

Sincerly
Navaneeth
I have a doubt whether saving data to a file on the hard drive is
faster then saving data in the database. Not to mention the speed of
processing these data.

Regards,
Mykola
http://marss.co.ua

Aug 30 '07 #2
I suggest you stick with a database. Flat files will almost certainly not
give you additional speed or scalability.
Perhaps you could use some sort of a queue to save up log records and only
write to the database every so often.

--
I hope this helps,
Steve C. Orr,
MCSD, MVP, CSM, ASPInsider
http://SteveOrr.net

"Navaneeth.K.N" <Na***********@discussions.microsoft.comwrote in message
news:B9**********************************@microsof t.com...
Hello

I am developing a system for user tracking. In this I am tracking all the
visitors that came to my website. i developed this using a HTTPModule and
storing the data in a SQL Server database. Everything worked fine till my
website gets good traffic.

When website is getting good traffic, this method is slowing down the
entire
application. So I am planning to use flat files instead of SQL Server
database. My doubts are

1 - What is a flat file and how can I connect to it from my ASP.NET
application
2 - If multiple users are browsing website at same time, how can I save
data's to file, because that may be in use.
3 - Is XML files are normal flat files ?

Please help me on this issue.

Sincerly
Navaneeth
Aug 30 '07 #3
Yes, the OP could look into MSMQ, however what levels of traffic are we
talking here? If it isn't extremely heavy I can't understand why SQL would
be causing a bottleneck. I'd look into what is causing the delay before
looking for an alternative solution.

"Steve C. Orr [MCSD, MVP, CSM, ASP Insider]" <St***@Orr.netwrote in
message news:B7**********************************@microsof t.com...
>I suggest you stick with a database. Flat files will almost certainly not
give you additional speed or scalability.
Perhaps you could use some sort of a queue to save up log records and only
write to the database every so often.

--
I hope this helps,
Steve C. Orr,
MCSD, MVP, CSM, ASPInsider
http://SteveOrr.net

"Navaneeth.K.N" <Na***********@discussions.microsoft.comwrote in message
news:B9**********************************@microsof t.com...
>Hello

I am developing a system for user tracking. In this I am tracking all the
visitors that came to my website. i developed this using a HTTPModule and
storing the data in a SQL Server database. Everything worked fine till my
website gets good traffic.

When website is getting good traffic, this method is slowing down the
entire
application. So I am planning to use flat files instead of SQL Server
database. My doubts are

1 - What is a flat file and how can I connect to it from my ASP.NET
application
2 - If multiple users are browsing website at same time, how can I save
data's to file, because that may be in use.
3 - Is XML files are normal flat files ?

Please help me on this issue.

Sincerly
Navaneeth

Aug 30 '07 #4
"Aidy" <ai**@xxnoemailxx.comwrote in message
news:8t******************************@bt.com...
If it isn't extremely heavy I can't understand why SQL would be causing a
bottleneck.
Could be any number of reasons: poor design, no primary key / index,
inefficient use of ADO.NET, not closing connections etc...
--
Mark Rae
ASP.NET MVP
http://www.markrae.net

Aug 30 '07 #5
Could be any number of reasons: poor design, no primary key / index,
inefficient use of ADO.NET, not closing connections etc...
That's what I meant, I don't see that it can be SQL Server's fault but
something to do with the design/coding.

BTW, why would not having an index *harm* insert performance? :p
Aug 30 '07 #6
A flat file likely won't help you - ultimately your database does nothing
more than write data to a file. XML is not he right approach either.

You need to either change your approach to tracking data, using perhaps a
hidden iframe in you pages that impacts a second web server instead of your
primary - or look to scale out for load by adding web servers to create a
farm.

Ultimately, a fix is a design decision as your problem could be in the
module rather than he fact that your talking to a database, you could try to
spawn your write activities off to asynchronous threads and see if it
reduces your problems or simply load your data into the application object
and every 1000 requests or so write it to the DB.

Regards

John Timney (MVP)
http://www.johntimney.com
http://www.johntimney.com/blog
"Navaneeth.K.N" <Na***********@discussions.microsoft.comwrote in message
news:B9**********************************@microsof t.com...
Hello

I am developing a system for user tracking. In this I am tracking all the
visitors that came to my website. i developed this using a HTTPModule and
storing the data in a SQL Server database. Everything worked fine till my
website gets good traffic.

When website is getting good traffic, this method is slowing down the
entire
application. So I am planning to use flat files instead of SQL Server
database. My doubts are

1 - What is a flat file and how can I connect to it from my ASP.NET
application
2 - If multiple users are browsing website at same time, how can I save
data's to file, because that may be in use.
3 - Is XML files are normal flat files ?

Please help me on this issue.

Sincerly
Navaneeth

Aug 30 '07 #7
"Aidy" <ai**@xxnoemailxx.comwrote in message
news:lN*********************@bt.com...
BTW, why would not having an index *harm* insert performance? :p
:-)
--
Mark Rae
ASP.NET MVP
http://www.markrae.net

Aug 30 '07 #8
Hello,

Using Application variables looks like a good solution to me. But what will
be the performance issues where Application variable grows ? And I think I
will loose unsaved data if any server crash happened
Aug 31 '07 #9
Performance clearly depends on how much data you stick in the app object
before you archive it off. You'll need to load test whatever your doing.

You always run the risk of losing data in any application crash before its
written to disk/database. What you need to work out is how many records are
you prepared to lose and make sure your write events minimise the impact.
So if you can only lose 10 records then dont wait until 100 to write to the
database.

Regards

John Timney (MVP)
http://www.johntimney.com
http://www.johntimney.com/blog
"Navaneeth.K.N" <Na*********@discussions.microsoft.comwrote in message
news:C2**********************************@microsof t.com...
Hello,

Using Application variables looks like a good solution to me. But what
will
be the performance issues where Application variable grows ? And I think I
will loose unsaved data if any server crash happened

Aug 31 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: hakhan | last post by:
Hello all, How can I convert unstructured flat files into structured XML? Are there any scientific articles and tools that handles this subject? Bye, hakhan
1
by: Pierre Maricq | last post by:
Hi, I am using Win2000 and Access2000. I need to build build a macro or write a VBA in Access that would screen all files contained in a directory on my C drive (files are structrured DAT...
2
by: eddieb7 | last post by:
Hi, I am new to visual Studio 2005 C++ and am looking for some directions on where best to start. I come from a mainly Delphi background and looking to switch to VS 2005 C++ or C#. I am...
1
by: new | last post by:
I have data for each week in a single table. I need to export this data to a separate flat file for each week. Any ideas? DB2 SQL Query export to flat files as a function of data on each record
3
by: anuanusha29 | last post by:
Hi, I am kinda new to programming and am in the process of implementing an application which is expected to parse/read data from flat files and display errors to the user about missing/invalid...
0
by: dave32079 | last post by:
I am moving data from Oracle to MS SQL 2000. I have been given the Oracle table structures and about 130 CVS files to populate these structures, 1 file per table. I have converted the table...
0
by: Stylus Studio | last post by:
Dear Comp.Text.XML Community, Attend a webinar event on how to modernize your EDI infrastructure and exploit XML technologies. Topic: Exploiting EDI and Flat Files in an XML World Date &...
3
by: stephen | last post by:
I had to use use ssis 2005 in a short project recently & had little time to work it out. I was importing a whole bunch of flat files into SQL Server tables with many derived columns and...
3
by: Tim | last post by:
Hi Folks, I'm used to a UNLOAD command that allows me to dump to a named flat file the results of any SELECT statement. Hence one can build a single SQL file which contains multiple SQL...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.