Hi,
I have a piece of code that needs to be able to modify a few bytes towards the end of a file. The problem is that the files are huge. Up to 100+ Gb.
I need the operation to be as fast as possible but after hours of Googeling, it looks like .Net is rather limited here???
I have mostly been trying using System.IO.FileStream and know of no other methods. A "reverse" filestream would do but I have know idea how to create one (write from the end instead of the beginning).
Here is sort of what I do: -
static void Main(string[] args)
-
{
-
//Just to simulate a large file
-
int size = 1000 * 1024 * 1024;
-
string filename = "blah.dat";
-
FileStream fs = new FileStream(filename, FileMode.Create);
-
fs.SetLength(size);
-
fs.Close();
-
-
//Modify the last byte
-
byte c = 255;
-
fs = new FileStream(filename, FileMode.Open);
-
-
//If I comment this out, the modification happens instantly
-
fs.Seek(fs.Length - 1, SeekOrigin.Begin);
-
fs.WriteByte(c);
-
-
//Now, since I am modifying the last byte, this last step is extremely slow on large files (as slow as copying the entire file)
-
fs.Close();
-
}
-
5 14334
I believe that this is happening because your asking the FileStream to seek to the end of the file, however if you seek using an offset of -1, it should seek backwards from the end of the file.
hope this helps.
Aimee.
@AmzBee
Tried it but I get an IOException trying to seek with a negative value.
"An attempt was made to move the file pointer before the beginning of the file."
I just tested it, works fine :), heres how i achieved it: -
private void button1_Click(object sender, EventArgs e)
-
{
-
OpenFileDialog dlg = new OpenFileDialog();
-
if (dlg.ShowDialog() == DialogResult.OK)
-
{
-
using (FileStream f = new FileStream(dlg.FileName,
-
FileMode.Open,
-
FileAccess.Read))
-
{
-
f.Seek(-1, SeekOrigin.End);
-
f.ReadByte();
-
string result = String.Format("Position: {0} \r\n" +
-
"Length Of File: {1}",
-
fs.Position.ToString(),
-
fs.Length.ToString());
-
MessageBox.Show(result);
-
}
-
}
-
}
-
@AmzBee
Right, so seeking itself is not the issue. Reading the end of a huge file is fast. It's writing to the end of the file that is super slow. The time is spent when you close the filestream.
fs.Seek(-1, SeekOrigin.End);
and
fs.Seek(fs.Length - 1, SeekOrigin.Begin);
is both very fast.
Sign in to post your reply or Sign up for a free account.
Similar topics
by: Stephane |
last post by:
Hello,
I need to allow an end user to upload video files on a server and in the
same time to put the file name and a few infos in a database. It must be
very simple for the end user, and...
|
by: Paul |
last post by:
Hi all
I have a sorting problem, but my experience with Python is rather
limited (3 days), so I am running this by the list first.
I have a large database of 15GB, consisting of 10^8 entries...
|
by: hakhan |
last post by:
Hello,
I need to store huge(+/- 100MB) data. Furthermore, my GUI application
must select data portions from these huge data files in order to do
some post-processing. I wonder in which format I...
|
by: Jan |
last post by:
Is there a program that easily let me view huge files without being slow?
Norton Commander for DOS was able to immediately view any file of any size
scrolling from beginning or end of file. No time...
|
by: Mario Rodriguez |
last post by:
HI people, there is an alternative way to upload files than the ASP.NET
native way? Is OK for small files but It doesn't seems to be convenient for
an application where there is a lot of paralell...
|
by: Jeff |
last post by:
Our company has to find a way to transfer huge files, target is maximum
of 6GB, between sites so we are looking to find the best way to do it.
There are lots of criteria we need but the most...
|
by: s99999999s2003 |
last post by:
hi
i wrote some code to compare 2 files. One is the base file, the other
file i got from somewhere. I need to compare this file against the
base,
eg base file
abc
def
ghi
eg another file
|
by: John Redmond |
last post by:
From time to time, there have been posts asking about options for
transforming really huge xml files. I might just have a solution for
this in my new app (Lexxia for Linux and Windows). Providing...
|
by: Mohamed Yousef |
last post by:
Thanks all ,
but there is still something i forget to state -sorry - all
communication will be via Http with a server
so data is received via Http
so local network solutions won't work
the...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
| |