473,320 Members | 1,861 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Real-time compression for mysql udf_lzo

I found this old thread while looking for some type of compression
solution myself. Long story short, I contacted Lester and ended up
writing an LZO solution just as Mark had mentioned below. Thought this
could also benifit someone out there so I'm reposting. Here's a link
to the source code:

http://www.servangle.net/udf_lzo/

BTW, Lester got his zlib implementation working as well. Hopefully
these will both be posted on MySQL website UDF section soon.

-John R.

--------------- original message ---------------
From: Mark Hazen (ma*************@hotmail.com)
Subject: Re: Mysql compress_gz()/uncompress_gz() user defined
functions
Newsgroups: mailing.database.mysql
Date: 2001-12-18 15:55:02 PST
I think this guy has made a noteworthy contribution. I've been
looking for
something like this for a while. Sure, there are ways to do it outside
of
MySQL but I always prefer working within MySQL. His function is great
for
storing big BLOB data that never needs to be searched on. I might add
this
suggestion though. Zlib compression is slow compared to LZO (search
LZO
compression on Google). Sure, Zlib isn't a dog and compresses better
than
LZO, but lzo is lightning fast for situations where the time to
compress is
very important. A function to compress_lzo and uncompress_lzo would
be
awesome.

Mark
"Lester Hightower" <hi******@TheAIMSGroup.com> wrote in message
news:9v***********@FreeBSD.csie.NCTU.edu.tw...
After sending my earlier email regarding compress_gz()/uncompress_gz() in
mysqld, I discovered user defined functions -- something I had never used
in mysqld until today.

So, I took a couple of hours and wrote a Mysql UDF to do what I requested:

# gcc -shared -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr/local/mysql\""
-DDAT
ADIR="\"/usr/local/mysql/var\""
-DSHAREDIR="\"/usr/local/mysql/share/mysql\"
" -DHAVE_CONFIG_H -I./../include -I./../regex -I. -I../include -I..
-I. -I/u
sr/local/mysql/include -O3 -DDBUG_OFF -export-dynamic -fPIC
-fno-implicit-te
mplates -o mysqld_udf_zlib.so mysqld_udf_zlib.cc /usr/lib/libz.a
# cp ./mysqld_udf_zlib.so /usr/lib; ldconfig

# mysql mysql
mysql> CREATE FUNCTION compress_gz RETURNS STRING SONAME "mysqld_udf_zlib.so"; CREATE FUNCTION uncompress_gz RETURNS STRING
SONAME
"mysqld_udf_zlib.so"; Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

mysql> select compress_gz('lester');
+-----------------------+
| compress_gz('lester') | +-----------------------+
| xËI-.I-a |
+-----------------------+
1 row in set (0.00 sec)

mysql> select uncompress_gz(compress_gz('lester'));
+--------------------------------------+
| uncompress_gz(compress_gz('lester')) | +--------------------------------------+
| lester |
+--------------------------------------+
1 row in set (0.01 sec)

mysql> drop function compress_gz; drop function uncompress_gz;
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

It is late and I am going home for the evening. Tomorrow I will review
the code carefully, clean it up a bit, comment it well, and will then be
happy to contribute it to MySQL AB, if you guys are interested in having
it in the contributed software page. Is there a procedure for contribing
the code to you guys, or just email it to the mysql list, or what?

--
Lester H. Hightower TheAIMSGroup.com
Chief Technology Officer
The Advanced Integrated Management Solutions Group
---------------------------------------------------------------------
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)

To request this thread, e-mail <my***************@lists.mysql.com>
To unsubscribe, e-mail <my**********************************************@ lists.mysql.com>
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php

Jul 19 '05 #1
0 1174

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Aaron W. West | last post by:
Timings... sometimes there are almost too many ways to do the same thing. The only significant findings I see from all the below timings is: 1) Integer math is generally fastest, naturally....
2
by: cwdjr | last post by:
Real One has a page to copy on their site that detects if the browser of a viewer of a page has Real One installed. The page is located at...
4
by: Allan Adler | last post by:
I'm trying to reinstall RedHat 7.1 Linux on a PC that was disabled when I tried to upgrade from RH7.1 to RH9. This is presenting lots of unexpected difficulties. Apart from wanting to keep the old...
10
by: Pavils Jurjans | last post by:
Hallo, It is know issue that due to the fact that computer has to store the real numbers in limited set of bytes, thus causing a minor imprecision from the decimal value that likely was stored....
17
by: David Scemama | last post by:
Hi, I'm writing a program using VB.NET that needs to communicate with a DOS Pascal program than cannot be modified. The communication channel is through some file databases, and I have a huge...
5
by: Henry Wu | last post by:
Hi, now that in .NET everything is on millimeter, I was wondering how can one convert Pixel to Millimeter and any user's screen/monitor. I saw the following code on how to convert pixel to...
0
by: support | last post by:
Veteran Real Estate Investor Shares some of his best Insider Secrets for successful investments! www.RealEstateBeginners.ws Have you ever wondered about investing in real estate? Maybe one...
12
by: Raymond Hettinger | last post by:
I am evaluating a request for an alternate version of itertools.izip() that has a None fill-in feature like the built-in map function: >>> map(None, 'abc', '12345') # demonstrate map's None...
16
by: DirtyHarry | last post by:
Good day everyone. This sounds like a stupid question, but I became just curious yesterday, and I looked up several textbooks. However, no textbooks on computer language (that I have ) mentioned...
2
by: Tim | last post by:
Folks, Can anyone thow some clarifying light on the following? I have come across a column with the same name and same data contents defined on different tables, on some the column is defined...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.