473,386 Members | 2,078 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

Handling large files in encryption prog

I'm currently writing an encryption program designed to read in a file (and
encrypt it) by splitting it in to 128 byte block segments and encrypting
each segment iteratively. The problem is is that it works fine up to sizes
of approx 8.5MB (albeit the program slows down throughout the process) and
then crashes at 8.5 leaving me to have to reboot my computer (linux).

Here is the pseudo-code:

whilst not eof
{
read in 128 bytes from file, stick in to inBytes[]
encrypt inBytes[], stick reult in to outBytes[];
write outBytes to disk[];

iterate put stream pointer by 128;
iterate get stream pointer by 128;
}// end of encryption process

delete[] inBytes;delete[] outBytes;

Do you think its a memory (de)allocation issue with inBytes and outBytes. If
so, does any one have any suggestions how I can sort it out?

Thanks
Ben
Jul 22 '05 #1
2 2037
BrianJones wrote:

I'm currently writing an encryption program designed to read in a file (and
encrypt it) by splitting it in to 128 byte block segments and encrypting
each segment iteratively. The problem is is that it works fine up to sizes
of approx 8.5MB (albeit the program slows down throughout the process) and
then crashes at 8.5 leaving me to have to reboot my computer (linux).

Here is the pseudo-code:


pseudo code is of no use in diagnosing a problem in your real code.

The fact that your program slows down is an inidcation that something
is wrong. My guess would be: the encrypt function allocates memory which is
never released. Thus memory leaks get bigger over time and eventually your
program runs out of memory. But that's just a guess.

--
Karl Heinz Buchegger
kb******@gascad.at
Jul 22 '05 #2

"BrianJones" <br***@jones1611.fsnet.co.uk> wrote in message
news:cd**********@news5.svr.pol.co.uk...
I'm currently writing an encryption program designed to read in a file (and encrypt it) by splitting it in to 128 byte block segments and encrypting
each segment iteratively. The problem is is that it works fine up to sizes
of approx 8.5MB (albeit the program slows down throughout the process) and
then crashes at 8.5 leaving me to have to reboot my computer (linux).

Here is the pseudo-code:

whilst not eof
{
read in 128 bytes from file, stick in to inBytes[]
encrypt inBytes[], stick reult in to outBytes[];
write outBytes to disk[];

iterate put stream pointer by 128;
iterate get stream pointer by 128;
}// end of encryption process

delete[] inBytes;delete[] outBytes;

Do you think its a memory (de)allocation issue with inBytes and outBytes. If so, does any one have any suggestions how I can sort it out?


Well I wonder why you are allocating memory at all, 128 bytes is small
enough that you should just declare those arrays on the stack.

What does

iterate put stream pointer by 128;
iterate get stream pointer by 128;

mean? You are reading and writing files sequentially, there is no need to
adjust the get or put positions. If that is what you are doing then I would
think that could cause a big slow down in your program.

If that doesn't help I suggest you post some real code.

john
Jul 22 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Florian Lindner | last post by:
Hello, is there a python library available for handling GNU parameters. For example: prog --filename=foo.bar is the same as prog -f foo.bar or prog --execute == prog -e
24
by: Joerg Schuster | last post by:
Hello, I am looking for a method to "shuffle" the lines of a large file. I have a corpus of sorted and "uniqed" English sentences that has been produced with (1): (1) sort corpus | uniq >...
18
by: JKop | last post by:
Here's what I know so far: You have a C++ project. You have source files in it. When you go to compile it, first thing the preprocessor sticks the header files into each source file. So now...
4
by: LBX | last post by:
I am looking for C++ libraries that handle very large integers. The purpose of the large integers are to handle RSA algorithms. So I need to be able to multiply, power, and modulus. Having one...
6
by: Ashok | last post by:
hi, i want to know how to make a specific type of file open in an application i developed in python when the user clicks on the file.(in windows) for eg. a .txt file when clicked opens in notepad,...
1
by: Bart | last post by:
Dear all, I would like to encrypt a large amount of data by using public/private keys, but I read on MSDN: "Symmetric encryption is performed on streams and is therefore useful to encrypt large...
6
by: Thomas Due | last post by:
Hi, I am writing an ASP.NET project where I allow users to upload files to the server. I have changed to web.config to allow a total file size of 100MB. My problem is that if the total file size...
0
by: balaji krishna | last post by:
Hi, I need to handle the return set from COBOL stored procedure from my invoking Java program. I do not know, how many rows the stored proc SQL fetches.I have declared the cursor in that proc, but i...
0
by: =?ISO-8859-15?Q?Bernard_Delm=E9e?= | last post by:
Hello, We have a processing chain automatically handling email attachments in python, which works well. We'd like to add support for encrypted files. We're using python 2.3 on HP-UX. I've...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.