473,372 Members | 1,245 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,372 software developers and data experts.

Real-time compression for mysql udf_lzo

I found this old thread while looking for some type of compression
solution myself. Long story short, I contacted Lester and ended up
writing an LZO solution just as Mark had mentioned below. Thought this
could also benifit someone out there so I'm reposting. Here's a link
to the source code:


BTW, Lester got his zlib implementation working as well. Hopefully
these will both be posted on MySQL website UDF section soon.

-John R.

--------------- original message ---------------
From: Mark Hazen (ma*************@hotmail.com)
Subject: Re: Mysql compress_gz()/uncompress_gz() user defined
Newsgroups: mailing.database.mysql
Date: 2001-12-18 15:55:02 PST
I think this guy has made a noteworthy contribution. I've been
looking for
something like this for a while. Sure, there are ways to do it outside
MySQL but I always prefer working within MySQL. His function is great
storing big BLOB data that never needs to be searched on. I might add
suggestion though. Zlib compression is slow compared to LZO (search
compression on Google). Sure, Zlib isn't a dog and compresses better
LZO, but lzo is lightning fast for situations where the time to
compress is
very important. A function to compress_lzo and uncompress_lzo would

"Lester Hightower" <hi******@TheAIMSGroup.com> wrote in message
After sending my earlier email regarding compress_gz()/uncompress_gz() in
mysqld, I discovered user defined functions -- something I had never used
in mysqld until today.

So, I took a couple of hours and wrote a Mysql UDF to do what I requested:

# gcc -shared -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/usr/local/mysql\""
" -DHAVE_CONFIG_H -I./../include -I./../regex -I. -I../include -I..
-I. -I/u
sr/local/mysql/include -O3 -DDBUG_OFF -export-dynamic -fPIC
mplates -o mysqld_udf_zlib.so mysqld_udf_zlib.cc /usr/lib/libz.a
# cp ./mysqld_udf_zlib.so /usr/lib; ldconfig

# mysql mysql
mysql> CREATE FUNCTION compress_gz RETURNS STRING SONAME "mysqld_udf_zlib.so"; CREATE FUNCTION uncompress_gz RETURNS STRING
"mysqld_udf_zlib.so"; Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

mysql> select compress_gz('lester');
| compress_gz('lester') | +-----------------------+
| xI-.I-a |
1 row in set (0.00 sec)

mysql> select uncompress_gz(compress_gz('lester'));
| uncompress_gz(compress_gz('lester')) | +--------------------------------------+
| lester |
1 row in set (0.01 sec)

mysql> drop function compress_gz; drop function uncompress_gz;
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

It is late and I am going home for the evening. Tomorrow I will review
the code carefully, clean it up a bit, comment it well, and will then be
happy to contribute it to MySQL AB, if you guys are interested in having
it in the contributed software page. Is there a procedure for contribing
the code to you guys, or just email it to the mysql list, or what?

Lester H. Hightower TheAIMSGroup.com
Chief Technology Officer
The Advanced Integrated Management Solutions Group
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)

To request this thread, e-mail <my***************@lists.mysql.com>
To unsubscribe, e-mail <my**********************************************@ lists.mysql.com>
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php

Jul 19 '05 #1
0 1692

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: Aaron W. West | last post by:
Timings... sometimes there are almost too many ways to do the same thing. The only significant findings I see from all the below timings is: 1) Integer math is generally fastest, naturally....
by: cwdjr | last post by:
Real One has a page to copy on their site that detects if the browser of a viewer of a page has Real One installed. The page is located at...
by: Allan Adler | last post by:
I'm trying to reinstall RedHat 7.1 Linux on a PC that was disabled when I tried to upgrade from RH7.1 to RH9. This is presenting lots of unexpected difficulties. Apart from wanting to keep the old...
by: Pavils Jurjans | last post by:
Hallo, It is know issue that due to the fact that computer has to store the real numbers in limited set of bytes, thus causing a minor imprecision from the decimal value that likely was stored....
by: David Scemama | last post by:
Hi, I'm writing a program using VB.NET that needs to communicate with a DOS Pascal program than cannot be modified. The communication channel is through some file databases, and I have a huge...
by: Henry Wu | last post by:
Hi, now that in .NET everything is on millimeter, I was wondering how can one convert Pixel to Millimeter and any user's screen/monitor. I saw the following code on how to convert pixel to...
by: support | last post by:
Veteran Real Estate Investor Shares some of his best Insider Secrets for successful investments! www.RealEstateBeginners.ws Have you ever wondered about investing in real estate? Maybe one...
by: Raymond Hettinger | last post by:
I am evaluating a request for an alternate version of itertools.izip() that has a None fill-in feature like the built-in map function: >>> map(None, 'abc', '12345') # demonstrate map's None...
by: DirtyHarry | last post by:
Good day everyone. This sounds like a stupid question, but I became just curious yesterday, and I looked up several textbooks. However, no textbooks on computer language (that I have ) mentioned...
by: Tim | last post by:
Folks, Can anyone thow some clarifying light on the following? I have come across a column with the same name and same data contents defined on different tables, on some the column is defined...
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.