Problem with addslashes

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Hello folks,

I need some help/advice FAST.

I have problems with addslashes on my web-servers. After uploading a file, I
read the uploaded file, use addslashes on the read data and then insert it
into a blob field in a MySQL database.

The problem is that this works fine on my internal test web-server (running
under RedHat 7.3). But on my production web-server (running Fedora Core 4)
it "fails". The same function create different outputs on the two servers.
The data on the prod server is corrupted.

I added code to the upload script that dumps the data returned by addslashes
to a file. I then compared this file created on the two servers (based on
the same uploaded document). They are different both in content and length.

Both servers use the same version of Apache, PHP and MySQL. The only
difference is the OS.

I use PHP ver 4.3.9, Apache ver 2.0.52 and MySQL ver 4.1.8

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[Willem Bogaerts]

This feature is called "magic quotes". It is a setting in PHP.INI.

--
Willem Bogaerts

Application smith
Kratz B.V.
http://www.kratz.nl/

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Willem Bogaerts wrote:

This feature is called "magic quotes". It is a setting in PHP.INI.

I use the same php.ini file on both servers. The current settings for all
'magic' keywords:

magic_quotes_gpc = On
magic_quotes_runtime = Off
magic_quotes_sybase = Off

Is this the correct settings or?

I also tried to install php 4.4.5, since I found that there might be a bug
in addslashes. It did not help, but I now get the message "The uploaded
file was only partially uploaded" when uploading a large file (112 Kb). A
Smaller file (69Kb) works OK.

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Willem Bogaerts wrote:

This feature is called "magic quotes". It is a setting in PHP.INI.

I have checked this a bit and it does not affect the content of the uploaded
file.
IMHO, I can't see that this has anything to do with my problem.

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Jørn Dahl-Stamnes wrote:

Hello folks,

I need some help/advice FAST.

Here are parts of the upload script that handles the file. If you find any
errors, please let me know.

It seems like it works OK for smaller files (~70 Kb), but not for larger
files (~112 Kb). I have checked that the amount of memory my PHP script can
use is suffisient.
<?php
static $Upload_Errmsgs = array (
1 ="The uploaded file exceeds the upload_max_filesize directive in
php.ini.",
2 ="The uploaded file exceeds the MAX_FILE_SIZE directive that was
specified in the HTML form.",
3 ="The uploaded file was only partially uploaded.",
4 ="No file was uploaded."
);

if (0 != $_FILES['upfile']['error']) return "Error code:
{$Upload_Errmsgs[$_FILES['upfile']['error']]}.";
$max_size = 8* 1024 * 1024; // 8 Mb max size.
if ($max_size < $_FILES['upfile']['size'] || 0 == $_FILES['upfile'
['size']) return "Too big!";

$uploadfile = tempnam ("/tmp","att");
if (move_the_file($uploadfile))
{
$fsize = filesize ($uploadfile);
$bfh = @fopen ($uploadfile,"rb");
if ($bfh)
{
$att_data = @fread ($bfh,$fsize);
@fclose ($bfh);
$att_data = addslashes ($att_data);
} else
return "Error: Could not open uploaded file.";

$query = "INSERT INTO attachments (filetype,filedata) VALUE
('{$_FILES['upfile']['type']}','$att_data')";
mysql_query ($query,$dbh);
unlink ($uploadfile);
}

function move_the_file ($uploadfile)
{
$ok_status = false;
if (is_uploaded_file($_FILES['upfile']['tmp_name']))
{
$ok_status = @move_uploaded_file ($_FILES['upfile']['tmp_name']
$uploadfile);
}
return $ok_status;
}
?>

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[Willem Bogaerts]

The problem is that this works fine on my internal test web-server (running

under RedHat 7.3). But on my production web-server (running Fedora Core 4)
it "fails". The same function create different outputs on the two servers.
The data on the prod server is corrupted.

I don't know what you mean by corrupted. There's an awful lot of ways a
file can be corrupted. Can this be en encoding problem? If the
corruption occurs at special characters (accented characters, euro sign,
etc.) and you see an "A" with an accent or tilde, then you are seeing
utf-8 rendered as latin1 (or derivatives). If you are seeing question
marks, the opposite is the case.

Hope this helps.
--
Willem Bogaerts

Application smith
Kratz B.V.
http://www.kratz.nl/

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Willem Bogaerts wrote:

>The problem is that this works fine on my internal test web-server
(running under RedHat 7.3). But on my production web-server (running
Fedora Core 4) it "fails". The same function create different outputs on
the two servers. The data on the prod server is corrupted.

I don't know what you mean by corrupted. There's an awful lot of ways a
file can be corrupted. Can this be en encoding problem? If the
corruption occurs at special characters (accented characters, euro sign,
etc.) and you see an "A" with an accent or tilde, then you are seeing
utf-8 rendered as latin1 (or derivatives). If you are seeing question
marks, the opposite is the case.

I have tested this by uploading MS Word documents to the web-server. When
downloading the data again, the client browser (FireFox) starts MS Words.
MS Word claims that the document is invalid.

The scripts works for some Word documents, but not other. The same happens
for PDF documents created with Open Office (reading MS Word docs and
exporting it to pdf files).

Also, I cannot understand why the data output from the addslashes is
different on two machiens that use the same version of PHP when the input
is the same file. It should be the same, or?

One thing I have not thought about, is the character set used in the
database. The table is created as this:

DROP TABLE IF EXISTS attachments;
CREATE TABLE attachments (
id MEDIUMINT UNSIGNED NOT NULL AUTO_INCREMENT,
title CHAR(32) NOT NULL,
filetype CHAR(32) NOT NULL,
filedata MEDIUMBLOB NOT NULL,
PRIMARY KEY (id)
) ENGINE=MyISAM DEFAULT CHARACTER SET latin1 COLLATE latin1_danish_ci
CHECKSUM=1;

Perhaps I should alter the table and let it use utf-8 (ENGINE=MyISAM DEFAULT
CHARACTER SET utf-8...)?

And thanks for replying :-)

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[Willem Bogaerts]

>>The problem is that this works fine on my internal test web-server

>>(running under RedHat 7.3). But on my production web-server (running
Fedora Core 4) it "fails". The same function create different outputs on
the two servers. The data on the prod server is corrupted.

I don't know what you mean by corrupted. There's an awful lot of ways a
file can be corrupted. Can this be en encoding problem? If the
corruption occurs at special characters (accented characters, euro sign,
etc.) and you see an "A" with an accent or tilde, then you are seeing
utf-8 rendered as latin1 (or derivatives). If you are seeing question
marks, the opposite is the case.

I have tested this by uploading MS Word documents to the web-server. When
downloading the data again, the client browser (FireFox) starts MS Words.
MS Word claims that the document is invalid.

The scripts works for some Word documents, but not other. The same happens
for PDF documents created with Open Office (reading MS Word docs and
exporting it to pdf files).

Also, I cannot understand why the data output from the addslashes is
different on two machiens that use the same version of PHP when the input
is the same file. It should be the same, or?

One thing I have not thought about, is the character set used in the
database. The table is created as this:

DROP TABLE IF EXISTS attachments;
CREATE TABLE attachments (
id MEDIUMINT UNSIGNED NOT NULL AUTO_INCREMENT,
title CHAR(32) NOT NULL,
filetype CHAR(32) NOT NULL,
filedata MEDIUMBLOB NOT NULL,
PRIMARY KEY (id)
) ENGINE=MyISAM DEFAULT CHARACTER SET latin1 COLLATE latin1_danish_ci
CHECKSUM=1;

Perhaps I should alter the table and let it use utf-8 (ENGINE=MyISAM DEFAULT
CHARACTER SET utf-8...)?

Before you do that (and frankly, you shouldn't, as you are dealing with
binary data), check the output of

SHOW VARIABLES LIKE '%char%'

on both MySQL servers. Is there any difference?

--
Willem Bogaerts

Application smith
Kratz B.V.
http://www.kratz.nl/

[Jerry Stuckle]

Jørn Dahl-Stamnes wrote:

Hello folks,

I need some help/advice FAST.

I have problems with addslashes on my web-servers. After uploading a file, I
read the uploaded file, use addslashes on the read data and then insert it
into a blob field in a MySQL database.

The problem is that this works fine on my internal test web-server (running
under RedHat 7.3). But on my production web-server (running Fedora Core 4)
it "fails". The same function create different outputs on the two servers.
The data on the prod server is corrupted.

I added code to the upload script that dumps the data returned by addslashes
to a file. I then compared this file created on the two servers (based on
the same uploaded document). They are different both in content and length.

Both servers use the same version of Apache, PHP and MySQL. The only
difference is the OS.

I use PHP ver 4.3.9, Apache ver 2.0.52 and MySQL ver 4.1.8

Well, first of all, you shouldn't be using addslashes. You should be
using mysql_real_escape_string(), especially since this is binary data.

And when you say the data is different - exactly what is different about
the two? There could be lots of things going on here, and without
knowing what's different, it's impossible to say.

For instance - once uploaded, are the files exactly the same (before you
do anything to them)?

Also, what charset is being used by MySQL?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Willem Bogaerts wrote:

Before you do that (and frankly, you shouldn't, as you are dealing with
binary data), check the output of

SHOW VARIABLES LIKE '%char%'
on both MySQL servers. Is there any difference?

No, they both show the same.

After I posted my previous message, I realized that BLOB does not use
character sets. BLOB is pure binary.

My PHP code and SQL table is more or less identical to the guide found at
http://www.php-mysql-tutorial.com/php-mysql-upload.php

However, when I downloaded the SAME file from the client to the test- and
test-server, the size of the file is different by 7 bytes! That sounds
strange to me.

I also tried to fire up my cold standby prod-server, same shit as on the
prod-server. The standby prod-server also run RH 7.3, which is what the
test-server does.

I have tried to debug this for a month now, trying all kinds of stunts - no
luck. I'm beginning to get rather frustrated!!

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Jerry Stuckle wrote:

Well, first of all, you shouldn't be using addslashes. You should be
using mysql_real_escape_string(), especially since this is binary data.

Try that to - does not work.

And when you say the data is different - exactly what is different about
the two? There could be lots of things going on here, and without
knowing what's different, it's impossible to say.

I downloaded the same file from the same client to both the test- and the
prod-server. The filesize is 7 bytes larger on the prod-server. Doing a
diff on the two files (from the prod and the test-server) creates A LOT of
output.

For instance - once uploaded, are the files exactly the same (before you
do anything to them)?

Exactly the same.

Also, what charset is being used by MySQL?

The data is stored in BLOB, which is, according to the manuals, pure binary.
No character set is used. Please correct me if I'm wrong.
I'm using "DEFAULT CHARACTER SET latin1 COLLATE latin1_danish_ci" when
creating the table.

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Jerry Stuckle wrote:

And when you say the data is different - exactly what is different about
the two? There could be lots of things going on here, and without
knowing what's different, it's impossible to say.

I just did another test. First I modified the upload script so that the
uploaded file is not deleted after the data has been inserted into the SQL
server.
From the same client using the same file, I uploaded the file to both the
test and the prod server. Then I compared the two files with the original
files:

size name
-----------------------
114176 org_file.doc
114182 prod_file.doc
114176 test_file.doc

As you can see, the file uploaded to the prod-server is larger. Doing a
'diff -a org_file.doc prod_file.doc x' and then analyzing the output, I
found that there is 6 different sections found.

If you like to take a look at the files, you can download them at:

http://www.dahl-stamnes.net/dahls/Misc/files.tar.gz

(note, they are in norwegian... ;-)

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[Jerry Stuckle]

Jørn Dahl-Stamnes wrote:

Jerry Stuckle wrote:

>Well, first of all, you shouldn't be using addslashes. You should be
using mysql_real_escape_string(), especially since this is binary data.

Try that to - does not work.

>And when you say the data is different - exactly what is different about
the two? There could be lots of things going on here, and without
knowing what's different, it's impossible to say.

I downloaded the same file from the same client to both the test- and the
prod-server. The filesize is 7 bytes larger on the prod-server. Doing a
diff on the two files (from the prod and the test-server) creates A LOT of
output.

>For instance - once uploaded, are the files exactly the same (before you
do anything to them)?

Exactly the same.

>Also, what charset is being used by MySQL?

The data is stored in BLOB, which is, according to the manuals, pure binary.
No character set is used. Please correct me if I'm wrong.
I'm using "DEFAULT CHARACTER SET latin1 COLLATE latin1_danish_ci" when
creating the table.

I guess first of all I'm confused. Are you DOWNloading or UPloading the
files? There is a difference - and it's important!

And when the files get to the servers, you indicate there is a seven
byte difference. So it seems the problem is occurring even before MySQL
gets involved.

So, to start, how are you downloading/uploading the files to the server?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Jerry Stuckle]

Jørn Dahl-Stamnes wrote:

Jerry Stuckle wrote:

>And when you say the data is different - exactly what is different about
the two? There could be lots of things going on here, and without
knowing what's different, it's impossible to say.

I just did another test. First I modified the upload script so that the
uploaded file is not deleted after the data has been inserted into the SQL
server.
From the same client using the same file, I uploaded the file to both the
test and the prod server. Then I compared the two files with the original
files:

size name
-----------------------
114176 org_file.doc
114182 prod_file.doc
114176 test_file.doc

As you can see, the file uploaded to the prod-server is larger. Doing a
'diff -a org_file.doc prod_file.doc x' and then analyzing the output, I
found that there is 6 different sections found.

If you like to take a look at the files, you can download them at:

http://www.dahl-stamnes.net/dahls/Misc/files.tar.gz

(note, they are in norwegian... ;-)

(Our messages crossed on the internet :-) ).

Ok, so the problem has nothing to do with MySQL - but the upload.
What's different between the two which would account for this?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[=?ISO-8859-1?Q?J=F8rn?= Dahl-Stamnes]

Jerry Stuckle wrote:

Jørn Dahl-Stamnes wrote:

>Jerry Stuckle wrote:

>>Well, first of all, you shouldn't be using addslashes. You should be
using mysql_real_escape_string(), especially since this is binary data.

Try that to - does not work.

>>And when you say the data is different - exactly what is different about
the two? There could be lots of things going on here, and without
knowing what's different, it's impossible to say.

I downloaded the same file from the same client to both the test- and the
prod-server. The filesize is 7 bytes larger on the prod-server. Doing a
diff on the two files (from the prod and the test-server) creates A LOT
of output.

>>For instance - once uploaded, are the files exactly the same (before you
do anything to them)?

Exactly the same.

>>Also, what charset is being used by MySQL?

The data is stored in BLOB, which is, according to the manuals, pure
binary. No character set is used. Please correct me if I'm wrong.
I'm using "DEFAULT CHARACTER SET latin1 COLLATE latin1_danish_ci" when
creating the table.

I guess first of all I'm confused. Are you DOWNloading or UPloading the
files? There is a difference - and it's important!

I'm uploading the file from the client to the server.

However... I began to suspect that the problem was in the network. It turned
out to be a problem in my firewall, since both the prod server and the cold
backup for the prod server was behind a fire wall (client -fw -server),
while the test-server was on the same net as the client.

So, after modifying the firewall, it worked out just fine!!

Then, my code is OK and the problem has been solved. Thanks a lot to all
replies.

--
Jørn Dahl-Stamnes
http://www.dahl-stamnes.net/dahls/

[Toby A Inkster]

Jørn Dahl-Stamnes wrote:

However, when I downloaded the SAME file from the client to the test- and
test-server, the size of the file is different by 7 bytes! That sounds
strange to me.

You should attempt to find out what those differences are. If you can do
that, then you might be able to figure out what's causing them.

My advice would be to try uploading a reasonably big ASCII document (e.g.
a large HTML file would do nicely) and then retrieve the file from your
database. Compare "before" and "after" with a "diff" tool. (Most Linux/UNIX
boxes should have at least one diff tool installed by default. Usually
/usr/bin/diff)

If there are no differences, try it again with a few non-ASCII characters
present (e.g. French accents, one or two Katakana characters, the euro
sign, etc).

Figure out what characters are being changed by the process; then you will
have a good clue as to what is causing them to change.

--
Toby A Inkster BSc (Hons) ARCS
Contact Me ~ http://tobyinkster.co.uk/contact
Geek of ~ HTML/SQL/Perl/PHP/Python*/Apache/Linux

* = I'm getting there!