By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,629 Members | 1,222 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,629 IT Pros & Developers. It's quick & easy.

String to binary and back

P: 18
Hello, so now that I am getting the data from the db - I have another problem to tackle. I need to save the data in binary format in order to save space as normal text files are getting to be in order of gigs.

However, I am having issues related to this. I discovered the binascii module and it seemed to be what I needed - especially a2b_uu() function. However, that only takes lines of 45 bytes long and that seemed to complex to work around. So, I tried a2b_base64() but that seems to do .. well nothing.

Here is an example code snippet that I attempted (mind you my knowledge on binary encoding is rudimentary at best):

line = b64.b64encode((str(row[0])+","+str(row[1])+","+str(row[2])+"\n"))
f.write(b.a2b_base64(line))

But unfortunately this seems to just make normal text files unlike the a2b_uu() function that made binary data. I tried to find good tutorials on this subject but everything seemed too complicated. I just want to have smaller data files. Any suggestions?
Nov 30 '06 #1
Share this Question
Share on Google+
5 Replies


bvdet
Expert Mod 2.5K+
P: 2,851
Expand|Select|Wrap|Line Numbers
  1. pickle.Pickler(file_name, 1).dump(data)
Expand|Select|Wrap|Line Numbers
  1. pickle.Pickler(file_name).dump(data)
The former will give you somewhat smaller files compared to the latter.
Nov 30 '06 #2

bartonc
Expert 5K+
P: 6,596
Hello, so now that I am getting the data from the db - I have another problem to tackle. I need to save the data in binary format in order to save space as normal text files are getting to be in order of gigs.

However, I am having issues related to this. I discovered the binascii module and it seemed to be what I needed - especially a2b_uu() function. However, that only takes lines of 45 bytes long and that seemed to complex to work around. So, I tried a2b_base64() but that seems to do .. well nothing.

Here is an example code snippet that I attempted (mind you my knowledge on binary encoding is rudimentary at best):

line = b64.b64encode((str(row[0])+","+str(row[1])+","+str(row[2])+"\n"))
f.write(b.a2b_base64(line))

But unfortunately this seems to just make normal text files unlike the a2b_uu() function that made binary data. I tried to find good tutorials on this subject but everything seemed too complicated. I just want to have smaller data files. Any suggestions?
But why, if the data is in a db, do you want to write data files?
Nov 30 '06 #3

bartonc
Expert 5K+
P: 6,596
Expand|Select|Wrap|Line Numbers
  1. pickle.Pickler(file_name, 1).dump(data)
Expand|Select|Wrap|Line Numbers
  1. pickle.Pickler(file_name).dump(data)
The former will give you somewhat smaller files compared to the latter.
If your code is something like
row = curs.fetchone()
you could pickle the whole result set by
)[/code]
Expand|Select|Wrap|Line Numbers
  1. pickle.dump(curs.fetchall())
Nov 30 '06 #4

P: 18
Well their are few reasons for writing to a file but the primary reason is that I want to take the data I am interested in to another computer for data mining purposes. The hardware that has the DB is used by lot of other people and it is not part of a network for security reasons. If I have the data on a file I can give it to some of my students for their projects in the future too. Unfortunately, no one knows the password for the OMS to the oracle database and apparently the export wizard only works if you are logged into OMS instead of a stand alone console.

I am sure their is a better way to do this but this seems to be working and I always wanted to learn Python!
Dec 1 '06 #5

bartonc
Expert 5K+
P: 6,596
Well their are few reasons for writing to a file but the primary reason is that I want to take the data I am interested in to another computer for data mining purposes. The hardware that has the DB is used by lot of other people and it is not part of a network for security reasons. If I have the data on a file I can give it to some of my students for their projects in the future too. Unfortunately, no one knows the password for the OMS to the oracle database and apparently the export wizard only works if you are logged into OMS instead of a stand alone console.

I am sure their is a better way to do this but this seems to be working and I always wanted to learn Python!
It's a worthy endevor and I applaud you sir. Of course, with a handle like "PythonNewbie", I was expecting less forthought and trying to avert a misstep. If that pickle thing doesn't work out for you, I'm sure we can come up with something better. Your replies are welcome,
Barton
Dec 1 '06 #6

Post your reply

Sign in to post your reply or Sign up for a free account.