473,785 Members | 2,919 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Handling large files in encryption prog

I'm currently writing an encryption program designed to read in a file (and
encrypt it) by splitting it in to 128 byte block segments and encrypting
each segment iteratively. The problem is is that it works fine up to sizes
of approx 8.5MB (albeit the program slows down throughout the process) and
then crashes at 8.5 leaving me to have to reboot my computer (linux).

Here is the pseudo-code:

whilst not eof
{
read in 128 bytes from file, stick in to inBytes[]
encrypt inBytes[], stick reult in to outBytes[];
write outBytes to disk[];

iterate put stream pointer by 128;
iterate get stream pointer by 128;
}// end of encryption process

delete[] inBytes;delete[] outBytes;

Do you think its a memory (de)allocation issue with inBytes and outBytes. If
so, does any one have any suggestions how I can sort it out?

Thanks
Ben
Jul 22 '05 #1
2 2061
BrianJones wrote:

I'm currently writing an encryption program designed to read in a file (and
encrypt it) by splitting it in to 128 byte block segments and encrypting
each segment iteratively. The problem is is that it works fine up to sizes
of approx 8.5MB (albeit the program slows down throughout the process) and
then crashes at 8.5 leaving me to have to reboot my computer (linux).

Here is the pseudo-code:


pseudo code is of no use in diagnosing a problem in your real code.

The fact that your program slows down is an inidcation that something
is wrong. My guess would be: the encrypt function allocates memory which is
never released. Thus memory leaks get bigger over time and eventually your
program runs out of memory. But that's just a guess.

--
Karl Heinz Buchegger
kb******@gascad .at
Jul 22 '05 #2

"BrianJones " <br***@jones161 1.fsnet.co.uk> wrote in message
news:cd******** **@news5.svr.po l.co.uk...
I'm currently writing an encryption program designed to read in a file (and encrypt it) by splitting it in to 128 byte block segments and encrypting
each segment iteratively. The problem is is that it works fine up to sizes
of approx 8.5MB (albeit the program slows down throughout the process) and
then crashes at 8.5 leaving me to have to reboot my computer (linux).

Here is the pseudo-code:

whilst not eof
{
read in 128 bytes from file, stick in to inBytes[]
encrypt inBytes[], stick reult in to outBytes[];
write outBytes to disk[];

iterate put stream pointer by 128;
iterate get stream pointer by 128;
}// end of encryption process

delete[] inBytes;delete[] outBytes;

Do you think its a memory (de)allocation issue with inBytes and outBytes. If so, does any one have any suggestions how I can sort it out?


Well I wonder why you are allocating memory at all, 128 bytes is small
enough that you should just declare those arrays on the stack.

What does

iterate put stream pointer by 128;
iterate get stream pointer by 128;

mean? You are reading and writing files sequentially, there is no need to
adjust the get or put positions. If that is what you are doing then I would
think that could cause a big slow down in your program.

If that doesn't help I suggest you post some real code.

john
Jul 22 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
1444
by: Florian Lindner | last post by:
Hello, is there a python library available for handling GNU parameters. For example: prog --filename=foo.bar is the same as prog -f foo.bar or prog --execute == prog -e
24
7303
by: Joerg Schuster | last post by:
Hello, I am looking for a method to "shuffle" the lines of a large file. I have a corpus of sorted and "uniqed" English sentences that has been produced with (1): (1) sort corpus | uniq > corpus.uniq corpus.uniq is 80G large. The fact that every sentence appears only
18
3186
by: JKop | last post by:
Here's what I know so far: You have a C++ project. You have source files in it. When you go to compile it, first thing the preprocessor sticks the header files into each source file. So now you have your ".cpp" files all ready, without any "#include" or "#define" in them. Let's assume that there's 2 source files in this project, "a.cpp" and
4
2005
by: LBX | last post by:
I am looking for C++ libraries that handle very large integers. The purpose of the large integers are to handle RSA algorithms. So I need to be able to multiply, power, and modulus. Having one that is fast will really help. Please provide links to libraries or to places that can assist in programming the library. ----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
6
2382
by: Ashok | last post by:
hi, i want to know how to make a specific type of file open in an application i developed in python when the user clicks on the file.(in windows) for eg. a .txt file when clicked opens in notepad, a .doc file when clicked opens in MS-word. In the same way i want to make a .xyz file open in the application i developed when clicked. thanks in advance for any advice.
1
5998
by: Bart | last post by:
Dear all, I would like to encrypt a large amount of data by using public/private keys, but I read on MSDN: "Symmetric encryption is performed on streams and is therefore useful to encrypt large amounts of data. Asymmetric encryption is performed on a small number of bytes and is therefore only useful for small amounts of data." There is not possibility to do it? I have tried to encrypt a 300kB file by RSA Algorithm, but I received...
6
4847
by: Thomas Due | last post by:
Hi, I am writing an ASP.NET project where I allow users to upload files to the server. I have changed to web.config to allow a total file size of 100MB. My problem is that if the total file size exceed this amount the page simply stops works, and I get a "Server unreachable" error. Is there any way to handle this, so I am able to present the user with a nice error message telling him that the maximum file size has been exceeded.
0
2062
by: balaji krishna | last post by:
Hi, I need to handle the return set from COBOL stored procedure from my invoking Java program. I do not know, how many rows the stored proc SQL fetches.I have declared the cursor in that proc, but i don't know how to return the rows the cursor has opened and I don't know how to handle the return set from the proc in my java code. My main problem with that proc is that whether I can retun the result set from the proc without closing the cursor...
0
944
by: =?ISO-8859-15?Q?Bernard_Delm=E9e?= | last post by:
Hello, We have a processing chain automatically handling email attachments in python, which works well. We'd like to add support for encrypted files. We're using python 2.3 on HP-UX. I've checked the online 2.5 documentation, and my understanding is that the standard library still only has support for messages digests, not full encryption of content. The stock answer seems to be to use the "Python Cryptography Toolkit" by AM Kuchlin. It...
0
9643
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9480
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10319
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10147
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10087
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8971
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6737
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5380
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
2
3645
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.