473,748 Members | 10,737 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Postgres BYTEA problems

I've searched throughout the python website and can't find an answer.

My problem:
After getting a BYTEA from a postgres database using Pygresql call:
Expand|Select|Wrap|Line Numbers
  1. x = Db.query("SELECT seq FROM sequence \
  2. WHERE version = \'NT_004350.16\' \
  3. AND startloc = 1 \
  4. AND endloc = 1120")
  5. y = x.getresult()[0][1]
  6.  
'y' ends up containing a string of characters. THe problem is that
many of these characters should be escaped characters. However,
somewhere in the process the escape character itself is interpreted as
a character that must be escaped.
so if I should be getting
"\x00\x01"
I'll get
"\\x00\\x01 "

My Thoughts:
1) Is there a way to somehow get a 'raw' uninterpreted input to the
string?
2) Can I possibly get the BYTEA results back as one very long Long?
3) I can just accept that this is the way I'm going to get my results
and try to convert the the string into something more usable.
-Any ideas how I might go about doing this?

Thanks for any help you can provide.

--Steve
Jul 18 '05 #1
1 3292
Ivan Voras <ivoras@__geri. cc.fer.hr> wrote in message news:<c2******* ***@bagan.srce. hr>...
Steve wrote:
a character that must be escaped.
so if I should be getting
"\x00\x01"
I'll get
"\\x00\\x01 "


Are you sure that it's the dbapi driver that is doing it, or maybe the
data was written to the database in that way (e.g. not chr(00)+chr(01)
but '\' 'x' '0' '0' ...)?

Thanks for responding Ivan

I'm pretty sure that the table is beeing fed chr(00) + chr(01). I'm
not the one generating the database(it's written in Java) however I've
seen the code and it appears to be entering in the data correctly.

Recently, to test the problem, I've gone through and inserted a string
in python that needed escaping something like this:
Db.insertable(" steves",[['\001'],["test"],["\\001\\002 "]])
y = Db.query("SELEC T * FROM steves").getres ult()
print y

[('\001',), ('test',), ('\\001\\002',)]

And I got the double escape problem on both y[0] and y[2].
Actually, looking at my testing I got an idea. Do you think this
could have to do with the double escaping one must do while working
with other postgres interfaces(name ly psql). Let me clarify that
question, I'm wondering if this double escape thing is actually an
artifact of postgres and not a problem with the pg module.

My current solution is to do this simple loop that replaces all //###
with the char replacemet like so:

def no_escape(x):
i, ret = x.find('\\'), ''
while i != -1:
if x[i+1].isdigit(): //number is in octal
y = x[i+1:i+4]
ret += x[:i] + struct.pack('B' ,string.atoi(y, 8))
x = x[i+4:]
i = x.find('\\')

return ret + x

But, of course, this solution is a little less than ideal.

Any thoughts?

--Steve
Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2887
by: mPath Records | last post by:
Hello, I hope someone can help me with this problem - I'm getting nowhere fast and have been working on it for a week. All I'm trying to do is upload an image, store the image in a bytea column of a postrgresql database, pull the image out of the datbase, and verify that the image is still the same and it can be viewed by a browser - that's all. The upload part works fine... I'm having problems getting out what I put in. The file...
0
1426
by: Ravi | last post by:
Hi, I'm looking for a Postgres connection Python module that has good support for the BYTEA type. By good support I mean that -it should have a function to escape the raw binary data as appropriate. -it must not choke on some rather large 20-30MB files. -Retrieval using Select statements should be transparent (or pretty simple)
2
9308
by: bissatch | last post by:
Hi, I am trying to write script that is run when a form is submitted. The form contains an image input field and when submitted, the image is uploaded, resized and added as binary information to a db table. Please note, I am using a PostgreSQL database I have written all the code out below that deals with the submission processing:
48
11909
by: Edwin Quijada | last post by:
Hi !! Everybody I am developing app using Delphi and I have a question: I have to save pictures into my database. Each picture has 20 o 30k aprox. What is the way more optimus? That 's table will have 500000 records around. Somebody said the best way to do that was encoder the picture to field bytea but I dont know about this. Another way is save the path to the picture file but I dont like so much because I need to write to disk by OS...
1
6280
by: Matthew Hixson | last post by:
I am currently working on a Java web application in which we are making use of the JDBC driver for Postgres 7.4.1. Part of our application allows the administrators to manage a large number of small images, most of them not exceeding 5KB. There is about a gigabyte of these small files. We're currently storing the files on disk and the other information about the file in the database (historical reasons that I won't complain about here)....
16
8227
by: Michal Hlavac | last post by:
Hello, I am working on web portal. There are some ads. We have about 200 000 ads. Every ad have own directory called ID, where is 5 subdirectories with various sizes of 5 images. Filesystem is too slow. But I don't know, if I store these images into postgres, performace will grow. Second question is, what kind of hardware I need for storing in DB. Now
2
2679
by: Carlos | last post by:
Do I need to use the -b option in pg_dump to dump bytea fields? For a while now I have been routinely using the -b option in pg_dump to back up, restore, and copy my databases because I thought that this was necessary to dump bytea fields. I am not using other types of large objects in my database. Recently, I tried to dump a database with this option and the dump failed; however I can dump the database without the -b option and my...
6
3541
by: Együd Csaba | last post by:
Hi All, I'd like to dump out my database using plain text format. Everything is ok, but the bytea filds. Restoring the dump file (using <<psql -f LO_TRY_INSERT_BYTEA.sql tmp7>>) it sends an error message like psql:LO_TRY_INSERT_BYTEA.sql:14: ERROR: Bad input string for type bytea Trying dumping with copy commands it sends this error: psql:LO_TRY_COPY_BYTEA.sql:13: lost synchronization with server, resetting
0
2705
by: NM | last post by:
Hello, I've got a problem inserting binary objects into the postgres database. I have binary objects (e.g. images or smth else) of any size which I want to insert into the database. Funny is it works for files larger than 8000 Bytes. If a file is less than 1000 Bytes I get the following message: Error message: --invalid input syntax for type oid: "\074\077......";
0
8991
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8830
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9541
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9370
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8242
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
4602
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4874
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3312
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2215
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.