473,386 Members | 1,693 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

Postgres BYTEA problems

I've searched throughout the python website and can't find an answer.

My problem:
After getting a BYTEA from a postgres database using Pygresql call:
Expand|Select|Wrap|Line Numbers
  1. x = Db.query("SELECT seq FROM sequence \
  2. WHERE version = \'NT_004350.16\' \
  3. AND startloc = 1 \
  4. AND endloc = 1120")
  5. y = x.getresult()[0][1]
  6.  
'y' ends up containing a string of characters. THe problem is that
many of these characters should be escaped characters. However,
somewhere in the process the escape character itself is interpreted as
a character that must be escaped.
so if I should be getting
"\x00\x01"
I'll get
"\\x00\\x01"

My Thoughts:
1) Is there a way to somehow get a 'raw' uninterpreted input to the
string?
2) Can I possibly get the BYTEA results back as one very long Long?
3) I can just accept that this is the way I'm going to get my results
and try to convert the the string into something more usable.
-Any ideas how I might go about doing this?

Thanks for any help you can provide.

--Steve
Jul 18 '05 #1
1 3265
Ivan Voras <ivoras@__geri.cc.fer.hr> wrote in message news:<c2**********@bagan.srce.hr>...
Steve wrote:
a character that must be escaped.
so if I should be getting
"\x00\x01"
I'll get
"\\x00\\x01"


Are you sure that it's the dbapi driver that is doing it, or maybe the
data was written to the database in that way (e.g. not chr(00)+chr(01)
but '\' 'x' '0' '0' ...)?

Thanks for responding Ivan

I'm pretty sure that the table is beeing fed chr(00) + chr(01). I'm
not the one generating the database(it's written in Java) however I've
seen the code and it appears to be entering in the data correctly.

Recently, to test the problem, I've gone through and inserted a string
in python that needed escaping something like this:
Db.insertable("steves",[['\001'],["test"],["\\001\\002"]])
y = Db.query("SELECT * FROM steves").getresult()
print y

[('\001',), ('test',), ('\\001\\002',)]

And I got the double escape problem on both y[0] and y[2].
Actually, looking at my testing I got an idea. Do you think this
could have to do with the double escaping one must do while working
with other postgres interfaces(namely psql). Let me clarify that
question, I'm wondering if this double escape thing is actually an
artifact of postgres and not a problem with the pg module.

My current solution is to do this simple loop that replaces all //###
with the char replacemet like so:

def no_escape(x):
i, ret = x.find('\\'), ''
while i != -1:
if x[i+1].isdigit(): //number is in octal
y = x[i+1:i+4]
ret += x[:i] + struct.pack('B',string.atoi(y,8))
x = x[i+4:]
i = x.find('\\')

return ret + x

But, of course, this solution is a little less than ideal.

Any thoughts?

--Steve
Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: mPath Records | last post by:
Hello, I hope someone can help me with this problem - I'm getting nowhere fast and have been working on it for a week. All I'm trying to do is upload an image, store the image in a bytea...
0
by: Ravi | last post by:
Hi, I'm looking for a Postgres connection Python module that has good support for the BYTEA type. By good support I mean that -it should have a function to escape the raw binary data as...
2
by: bissatch | last post by:
Hi, I am trying to write script that is run when a form is submitted. The form contains an image input field and when submitted, the image is uploaded, resized and added as binary information to...
48
by: Edwin Quijada | last post by:
Hi !! Everybody I am developing app using Delphi and I have a question: I have to save pictures into my database. Each picture has 20 o 30k aprox. What is the way more optimus? That 's table will...
1
by: Matthew Hixson | last post by:
I am currently working on a Java web application in which we are making use of the JDBC driver for Postgres 7.4.1. Part of our application allows the administrators to manage a large number of...
16
by: Michal Hlavac | last post by:
Hello, I am working on web portal. There are some ads. We have about 200 000 ads. Every ad have own directory called ID, where is 5 subdirectories with various sizes of 5 images. Filesystem...
2
by: Carlos | last post by:
Do I need to use the -b option in pg_dump to dump bytea fields? For a while now I have been routinely using the -b option in pg_dump to back up, restore, and copy my databases because I thought...
6
by: Együd Csaba | last post by:
Hi All, I'd like to dump out my database using plain text format. Everything is ok, but the bytea filds. Restoring the dump file (using <<psql -f LO_TRY_INSERT_BYTEA.sql tmp7>>) it sends an...
0
by: NM | last post by:
Hello, I've got a problem inserting binary objects into the postgres database. I have binary objects (e.g. images or smth else) of any size which I want to insert into the database. Funny is it...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.