Inorder to handle large files over 100 MB, is there a way in dotnet to open
only certain bytes of the file by buffer ?
It is shaky if the entire file is opened and splitted. Instead is there a
way or any alternatives on opening small chunk of the file. 10 1608
I don't think there is any different between opening a 1M file or 100M file
if you use a stream. Streams are designed with buffers and will only read
in as much information as you want. So, you could open your file using
FileStream and seek to a specified location, then read as many bytes as you
want.
Now, if you need more complex processing like getting the 200th row of some
file that are delimited by newlines, then that is something that you might
have to process yourself or rely on something like a Stream reader.
"shiva" <sh***@discussi ons.microsoft.c om> wrote in message
news:C7******** *************** ***********@mic rosoft.com... Inorder to handle large files over 100 MB, is there a way in dotnet to open only certain bytes of the file by buffer ?
It is shaky if the entire file is opened and splitted. Instead is there a way or any alternatives on opening small chunk of the file.
Thank you for the feedback Sir. Well, i am just trying to print a 30MB file
and its shaky. It gets hogged sometimes amd prints sometimes as if it works
when we have luck. Unless or otherwise something is wrong with the following
code. There is lot more transactions to do than just printing, but it is
skaky right here when printing.
Private Sub Button1_Click(B yVal sender As Object, ByVal e As
System.EventArg s) Handles Button1.Click
'Open a file for reading
Dim FILENAME As String
Dim FileContents As String
Dim iCount As Integer
iCount = 1
FILENAME = "c:\\Filereader \\File1.txt"
Try
'Get a StreamReader class that can be used to read the file
Dim objStreamReader As StreamReader
objStreamReader = File.OpenText(F ILENAME)
'Read Line By Line
While True
FileContents = CType(objStream Reader.ReadLine , String)
If Not FileContents = Nothing Then
Response.Write( iCount & "Begin Line1" & "<br>")
Response.Write( FileContents)
Response.Write( "<br>" & iCount & "End Line1" & "<br>")
iCount = iCount + 1
'lblRawOutput.T ext &= FileContents & vbCrLf
Else : Exit While
End If
End While
'Set the text of the file to a Web control
'lblRawOutput.T ext = contents
objStreamReader .Close()
Catch ex As Exception
Response.Write( ex.Message)
End Try
End Sub
"Peter Rilling" wrote: I don't think there is any different between opening a 1M file or 100M file if you use a stream. Streams are designed with buffers and will only read in as much information as you want. So, you could open your file using FileStream and seek to a specified location, then read as many bytes as you want.
Now, if you need more complex processing like getting the 200th row of some file that are delimited by newlines, then that is something that you might have to process yourself or rely on something like a Stream reader.
"shiva" <sh***@discussi ons.microsoft.c om> wrote in message news:C7******** *************** ***********@mic rosoft.com... Inorder to handle large files over 100 MB, is there a way in dotnet to open only certain bytes of the file by buffer ?
It is shaky if the entire file is opened and splitted. Instead is there a way or any alternatives on opening small chunk of the file.
Thank you for the feedback Sir. Well, i am just trying to print a 30MB file
and its shaky. It gets hogged sometimes amd prints sometimes as if it works
when we have luck. Unless or otherwise something is wrong with the following
code. There is lot more transactions to do than just printing, but it is
skaky right here when printing. -
Private Sub Button1_Click(ByVal sender As Object, ByVal e As
-
System.EventArgs) Handles Button1.Click
-
'Open a file for reading
-
Dim FILENAME As String
-
Dim FileContents As String
-
Dim iCount As Integer
-
iCount = 1
-
-
FILENAME = "c:\\Filereader\\File1.txt"
-
-
Try
-
-
'Get a StreamReader class that can be used to read the file
-
Dim objStreamReader As StreamReader
-
objStreamReader = File.OpenText(FILENAME)
-
-
'Read Line By Line
-
While True
-
FileContents = CType(objStreamReader.ReadLine, String)
-
If Not FileContents = Nothing Then
-
Response.Write(iCount & "Begin Line1" & "<br>")
-
Response.Write(FileContents)
-
Response.Write("<br>" & iCount & "End Line1" & "<br>")
-
iCount = iCount + 1
-
'lblRawOutput.Text &= FileContents & vbCrLf
-
Else : Exit While
-
End If
-
End While
-
-
'Set the text of the file to a Web control
-
'lblRawOutput.Text = contents
-
-
objStreamReader.Close()
-
-
Catch ex As Exception
-
Response.Write(ex.Message)
-
End Try
-
-
End Sub
-
-
"Peter Rilling" wrote: I don't think there is any different between opening a 1M file or 100M file if you use a stream. Streams are designed with buffers and will only read in as much information as you want. So, you could open your file using FileStream and seek to a specified location, then read as many bytes as you want.
Now, if you need more complex processing like getting the 200th row of some file that are delimited by newlines, then that is something that you might have to process yourself or rely on something like a Stream reader.
"shiva" <sh***@discussi ons.microsoft.c om> wrote in message news:C7******** *************** ***********@mic rosoft.com... Inorder to handle large files over 100 MB, is there a way in dotnet to open only certain bytes of the file by buffer ?
It is shaky if the entire file is opened and splitted. Instead is there a way or any alternatives on opening small chunk of the file.
Thank you for the feedback Sir. Well, i am just trying to print a 30MB file
and its shaky. It gets hogged sometimes amd prints sometimes as if it works
when we have luck. Unless or otherwise something is wrong with the following
code. There is lot more transactions to do than just printing, but it is
skaky right here when printing. -
Private Sub Button1_Click(ByVal sender As Object, ByVal e As
-
System.EventArgs) Handles Button1.Click
-
'Open a file for reading
-
Dim FILENAME As String
-
Dim FileContents As String
-
Dim iCount As Integer
-
iCount = 1
-
-
FILENAME = "c:\\Filereader\\File1.txt"
-
-
Try
-
-
'Get a StreamReader class that can be used to read the file
-
Dim objStreamReader As StreamReader
-
objStreamReader = File.OpenText(FILENAME)
-
-
'Read Line By Line
-
While True
-
FileContents = CType(objStreamReader.ReadLine, String)
-
If Not FileContents = Nothing Then
-
Response.Write(iCount & "Begin Line1" & "<br>")
-
Response.Write(FileContents)
-
Response.Write("<br>" & iCount & "End Line1" & "<br>")
-
iCount = iCount + 1
-
'lblRawOutput.Text &= FileContents & vbCrLf
-
Else : Exit While
-
End If
-
End While
-
-
'Set the text of the file to a Web control
-
'lblRawOutput.Text = contents
-
-
objStreamReader.Close()
-
-
Catch ex As Exception
-
Response.Write(ex.Message)
-
End Try
-
-
End Sub
-
-
"Peter Rilling" wrote: I don't think there is any different between opening a 1M file or 100M file if you use a stream. Streams are designed with buffers and will only read in as much information as you want. So, you could open your file using FileStream and seek to a specified location, then read as many bytes as you want.
Now, if you need more complex processing like getting the 200th row of some file that are delimited by newlines, then that is something that you might have to process yourself or rely on something like a Stream reader.
"shiva" <sh***@discussi ons.microsoft.c om> wrote in message news:C7******** *************** ***********@mic rosoft.com... Inorder to handle large files over 100 MB, is there a way in dotnet to open only certain bytes of the file by buffer ?
It is shaky if the entire file is opened and splitted. Instead is there a way or any alternatives on opening small chunk of the file.
Shiva,
Are you sure you want it byte by byte, used are often Unicodecharters which
used two bytes.
Than the encoding class will probably one of the first things too look at. http://msdn.microsoft.com/library/de...classtopic.asp
I hope this helps?
Copr
"shiva" <sh***@discussi ons.microsoft.c om> Inorder to handle large files over 100 MB, is there a way in dotnet to
open only certain bytes of the file by buffer ?
It is shaky if the entire file is opened and splitted. Instead is there a way or any alternatives on opening small chunk of the file.
Shiva,
Tell us first what type of file it is, text, a kind of own database, are
there char in it etc. or maybe it are pictures. Now it is just shooting in
the dark.
Cor
"shiva" <sh***@discussi ons.microsoft.c om> wrote in message
news:B9******** *************** ***********@mic rosoft.com... Thank you for the feedback Sir. Well, i am just trying to print a 30MB file and its shaky. It gets hogged sometimes amd prints sometimes as if it works when we have luck. Unless or otherwise something is wrong with the following code. There is lot more transactions to do than just printing, but it is skaky right here when printing.
Private Sub Button1_Click(B yVal sender As Object, ByVal e As System.EventArg s) Handles Button1.Click 'Open a file for reading Dim FILENAME As String Dim FileContents As String Dim iCount As Integer iCount = 1
FILENAME = "c:\\Filereader \\File1.txt"
Try
'Get a StreamReader class that can be used to read the file Dim objStreamReader As StreamReader objStreamReader = File.OpenText(F ILENAME)
'Read Line By Line While True FileContents = CType(objStream Reader.ReadLine , String)
ReadLine returns a string already, so there is no reason to
re-caste it.
If Not FileContents = Nothing Then Response.Write( iCount & "Begin Line1" & "<br>") Response.Write( FileContents) Response.Write( "<br>" & iCount & "End Line1" & "<br>") iCount = iCount + 1 'lblRawOutput.T ext &= FileContents & vbCrLf Else : Exit While End If End While
It doesn't look like your speed issues are due to the way you are
working with the files, but rather with what you are doing with
all the data.
What is this Response object you are working with? I'm assuming
it's something like a web response. If this is the case then it
can take quite a while to send the 30 MB of data via the network.
Even on a local network it can take a while depending on network
configuration and current traffic loads.
Also you have
'lblRawOutput.T ext &= FileContents & vbCrLf
commented out. However, if your tests were done with this
uncommented it could have a drastic performance hit for large
files, as the larger a string gets, the longer it takes to
concatenate more data on to the end of it.
Andrew Faust
"shiva" <sh***@discussi ons.microsoft.c om> wrote in message
news:53******** *************** ***********@mic rosoft.com... Dear Cor,
For example if the file is a 30MB file, i am looking for opening for 1MB at a time.
You can't open only 1 MB at a time. You either have a file open
or you don't. Size is irrelevant. However, you can read in to
memory only a small portion at a time.
Andrew Faust
Shiva,
What is than the reason you do than not proces it line by line? http://msdn.microsoft.com/library/de...xtFromFile.asp
Cor
"shiva" <sh***@discussi ons.microsoft.c om> schreef in bericht
news:9D******** *************** ***********@mic rosoft.com... Hi Cor,
Its a regular text file, but the formats are pretty complex like a EDI
file. There are no pictures or any kind of picture data in the file. Its a regular file
Header1<Informa tion>endofline Detail1<Informa tion>endofline Detail2<Informa tion>endofline Header2<Informa tion>endofline Detail1<Informa tion>endofline Detail2<Informa tion>endofline Detail3<Informa tion>endofline Header3<Informa tion>endofline Detail1<Informa tion>endofline Header4<Informa tion>endofline Detail1<Informa tion>endofline Detail2<Informa tion>endofline Detail3<Informa tion>endofline
I understand if i read only a 1MB out of the 30MB i may not get an entire record, but i can come up with some other algorithm for that.
But the main question is, is there a way i can open 1MB out of the 30MB.
Thank you Shiva "Cor Ligthert" wrote:
Shiva,
Tell us first what type of file it is, text, a kind of own database, are there char in it etc. or maybe it are pictures. Now it is just shooting
in the dark.
Cor This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: matt |
last post by:
I have compiled some code, some written by me, some compiled from
various sources online, and basically i've got a very simple flat file
photo gallery. An upload form, to upload the photos and give them a
caption, storing the caption and filename in a text file. It's a bit
buggy when removing the photos and captions from the file, and also in
displaying them on the delete page. you can see it in action at
www.4am.com.au/gallery/upload.php...
|
by: Dave Smithz |
last post by:
Hi There,
I have a PHP script that sends an email with attachment and works great when
provided the path to the file to send.
However this file needs to be on the same server as the script.
I want to develop a webpage where people can send attachments that are
stored on their local PC.
|
by: Joseph |
last post by:
Hi,
I'm having bit of questions on recursive pointer. I have following
code that supports upto 8K files but when i do a file like 12K i get a
segment fault. I Know it is in this line of code. How do i make the
last pointer in the indirect sector that has another level of indirect
pointer, and be defined recursively to support infinite large files?
-code-
|
by: StGo |
last post by:
How can i read/write file's custom attributs(like subject,author...) in
C#???
Thanks :))
|
by: Lokkju |
last post by:
I am pretty much lost here - I am trying to create a managed c++
wrapper for this dll, so that I can use it from c#/vb.net, however, it
does not conform to any standard style of coding I have seen. It is
almost like it is trying to implement it's own COM interfaces...
below is the header, and a link to the dll+code:
Zip file with header, example, and DLL:...
| |
by: Sky Sigal |
last post by:
I have created an IHttpHandler that waits for uploads as attachments for a
webmail interface, and saves it to a directory that is defined in
config.xml.
My question is the following:
assuming that this is suppossed to end up as a component for others to use,
and therefore I do NOT have access to their global.cs::Session_End()
how do I cleanup files that were uploaded -- but obviously left stranded
when the users aborted/gave up writting...
|
by: Roy |
last post by:
Hi,
I have a problem that I have been working with for a while.
I need to be able from server side (asp.net) to detect that the file i'm
streaming down to the client is saved completely/succsessfully on the
client's computer before updating some metadata on the server (file
downloaded date for instance)
However,
All examples i have tried, and all examples I have found that other people
says works - doesn't work for me :-(
|
by: Shapper |
last post by:
Hello,
I created a script to upload a file.
To determine the file type I am using userPostedFile.ContentType.
For example, for a png image I get "image/png".
My questions are:
1. Where can I find a list of all the types which can be returned?
For example, if it is a Word document will it return "document/doc".
|
by: troutbum |
last post by:
I am experiencing problems when one user has a document open through a
share pointing to the web site. I use the dsolefile to read the
contents of a particular directory and then display them in a
datalist. When the next user selects trys to run the page, the page
fails and I get a generic error message from the stack trace. I am
assuming that the document properties cannot be read when a file is
open, but it worked well in asp.
...
|
by: thjwong |
last post by:
I'm using WinXP with Microsoft Visual C++ .NET
69462-006-3405781-18776, Microsoft Development Environment 2003 Version
7.1.3088, Microsoft .NET Framework 1.1 Version 1.1.4322 SP1
Most developers said to me that they have no problem doing that, but
the following project file is said to be corrupted while opening in the
IDE, it is the project file of NT xemacs BETA 21.5.24,
http://ftp.xemacs.org/pub/beta/xemacs-21.5.24.tar.gz (under the...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
| |
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |