473,568 Members | 2,795 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Real-time compression for mysql udf_lzo

I found this old thread while looking for some type of compression
solution myself. Long story short, I contacted Lester and ended up
writing an LZO solution just as Mark had mentioned below. Thought this
could also benifit someone out there so I'm reposting. Here's a link
to the source code:

http://www.servangle.net/udf_lzo/

BTW, Lester got his zlib implementation working as well. Hopefully
these will both be posted on MySQL website UDF section soon.

-John R.

--------------- original message ---------------
From: Mark Hazen (ma************ *@hotmail.com)
Subject: Re: Mysql compress_gz()/uncompress_gz() user defined
functions
Newsgroups: mailing.databas e.mysql
Date: 2001-12-18 15:55:02 PST
I think this guy has made a noteworthy contribution. I've been
looking for
something like this for a while. Sure, there are ways to do it outside
of
MySQL but I always prefer working within MySQL. His function is great
for
storing big BLOB data that never needs to be searched on. I might add
this
suggestion though. Zlib compression is slow compared to LZO (search
LZO
compression on Google). Sure, Zlib isn't a dog and compresses better
than
LZO, but lzo is lightning fast for situations where the time to
compress is
very important. A function to compress_lzo and uncompress_lzo would
be
awesome.

Mark
"Lester Hightower" <hi******@TheAI MSGroup.com> wrote in message
news:9v******** ***@FreeBSD.csi e.NCTU.edu.tw.. .
After sending my earlier email regarding compress_gz()/uncompress_gz() in
mysqld, I discovered user defined functions -- something I had never used
in mysqld until today.

So, I took a couple of hours and wrote a Mysql UDF to do what I requested:

# gcc -shared -DMYSQL_SERVER -DDEFAULT_MYSQL_ HOME="\"/usr/local/mysql\""
-DDAT
ADIR="\"/usr/local/mysql/var\""
-DSHAREDIR="\"/usr/local/mysql/share/mysql\"
" -DHAVE_CONFIG_H -I./../include -I./../regex -I. -I../include -I..
-I. -I/u
sr/local/mysql/include -O3 -DDBUG_OFF -export-dynamic -fPIC
-fno-implicit-te
mplates -o mysqld_udf_zlib .so mysqld_udf_zlib .cc /usr/lib/libz.a
# cp ./mysqld_udf_zlib .so /usr/lib; ldconfig

# mysql mysql
mysql> CREATE FUNCTION compress_gz RETURNS STRING SONAME "mysqld_udf_zli b.so"; CREATE FUNCTION uncompress_gz RETURNS STRING
SONAME
"mysqld_udf_zli b.so"; Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

mysql> select compress_gz('le ster');
+-----------------------+
| compress_gz('le ster') | +-----------------------+
| xËI-.I-a |
+-----------------------+
1 row in set (0.00 sec)

mysql> select uncompress_gz(c ompress_gz('les ter'));
+--------------------------------------+
| uncompress_gz(c ompress_gz('les ter')) | +--------------------------------------+
| lester |
+--------------------------------------+
1 row in set (0.01 sec)

mysql> drop function compress_gz; drop function uncompress_gz;
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

It is late and I am going home for the evening. Tomorrow I will review
the code carefully, clean it up a bit, comment it well, and will then be
happy to contribute it to MySQL AB, if you guys are interested in having
it in the contributed software page. Is there a procedure for contribing
the code to you guys, or just email it to the mysql list, or what?

--
Lester H. Hightower TheAIMSGroup.co m
Chief Technology Officer
The Advanced Integrated Management Solutions Group
---------------------------------------------------------------------
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)

To request this thread, e-mail <my************ ***@lists.mysql .com>
To unsubscribe, e-mail <my************ *************** *************** ****@lists.mysq l.com>
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php

Jul 19 '05 #1
0 1710

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
2036
by: Aaron W. West | last post by:
Timings... sometimes there are almost too many ways to do the same thing. The only significant findings I see from all the below timings is: 1) Integer math is generally fastest, naturally. Bigint math isn't much slower, for integers that all fit within an integer. 2) Converting float to varchar is relatively slow, and should be avoided...
2
1942
by: cwdjr | last post by:
Real One has a page to copy on their site that detects if the browser of a viewer of a page has Real One installed. The page is located at...
4
2286
by: Allan Adler | last post by:
I'm trying to reinstall RedHat 7.1 Linux on a PC that was disabled when I tried to upgrade from RH7.1 to RH9. This is presenting lots of unexpected difficulties. Apart from wanting to keep the old model T in shape, I'm treating this as a learning experience. Right now, I'm trying to gain more control over the installation CD. By that I mean, I...
10
2681
by: Pavils Jurjans | last post by:
Hallo, It is know issue that due to the fact that computer has to store the real numbers in limited set of bytes, thus causing a minor imprecision from the decimal value that likely was stored. I don't know, how dotnet framework stored doubles, but it's certain, that if I store, say, 1.234567 in some real variable, it is stored in memory...
17
4351
by: David Scemama | last post by:
Hi, I'm writing a program using VB.NET that needs to communicate with a DOS Pascal program than cannot be modified. The communication channel is through some file databases, and I have a huge problem writing VB Double values to the file so as the Pascal program can read them as Pascal Real values. I've managed to find the algorithm to...
5
2717
by: Henry Wu | last post by:
Hi, now that in .NET everything is on millimeter, I was wondering how can one convert Pixel to Millimeter and any user's screen/monitor. I saw the following code on how to convert pixel to millimeter. It's done on pascal/delphi, but can be very easily read & converted to VB ..NET. However my question is what is the difference between "real"...
0
1554
by: support | last post by:
Veteran Real Estate Investor Shares some of his best Insider Secrets for successful investments! www.RealEstateBeginners.ws Have you ever wondered about investing in real estate? Maybe one sleepless night you tuned in to one of those infomercials that promises you the moon. You know the ones we're talking about. They
12
2072
by: Raymond Hettinger | last post by:
I am evaluating a request for an alternate version of itertools.izip() that has a None fill-in feature like the built-in map function: >>> map(None, 'abc', '12345') # demonstrate map's None fill-in feature The movitation is to provide a means for looping over all data elements when the input lengths are unequal. The question of the...
16
10839
by: DirtyHarry | last post by:
Good day everyone. This sounds like a stupid question, but I became just curious yesterday, and I looked up several textbooks. However, no textbooks on computer language (that I have ) mentioned this. So I am asking to you, gurus... Is there any particular reason to call 'float' instead of 'real'?
2
7326
by: Tim | last post by:
Folks, Can anyone thow some clarifying light on the following? I have come across a column with the same name and same data contents defined on different tables, on some the column is defined as a FLOAT on others it is a REAL. (Don't ask me why, it's inherited, legacy and due for removal once we have absorbed all the good bits into the...
0
7693
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7604
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8117
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
0
6275
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5498
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5217
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3651
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3631
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1207
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.