471,334 Members | 1,398 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,334 software developers and data experts.

Python,SWIG and libjvm

Hello,
I'm not sure if this the correct list but here goes (and sorry for the
noise). I've been attempting to wrap python around libhdfs.
So far so good (i've attached the SWIG template at the end). The
compilation works without errors and the shared objects do have
references to all the functions.

However, when importing into python

import pyhdfs

i get the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/home/sguha/tmp/b/hdfs.py", line 7, in ?
import _hdfs
ImportError: libjvm.so: cannot open shared object file: No such file
or directory

However, libjvm.so is located in /home/sguha/mine/jdk1.6.0_02/jre/lib/
amd64/server which is present in the PYTHONPATH and sys.path.
I can understand it complaining <i>while</iloading the libjvm.so
but python doesn't locate it even when in the path.

As an aside, the build command for hdfs.c has " -ljvm " - this
should affect SWIG, right? Meaning if my program (e.g) links to a
library and i want a wrapper around my program (to python) i need SWIG
templates for my headers only and not for the linked library...right?
( I guess the answer is i dont otherwise that would be quite a
nightmare of work)

Thank you for your time
Saptarshi
Attachments:
SWIG template
%module pyhdfs

%{
#include "hdfs.h"
%}

%include "hdfs.h"

Sep 20 '07 #1
3 2742
On Sep 20, 7:00 pm, sapsi <saptarshi.g...@gmail.comwrote:
Hello,
I'm not sure if this the correct list but here goes (and sorry for the
noise). I've been attempting to wrap python around libhdfs.
So far so good (

I should point out that libhdfs is a c library to Hadoop Distributed
FileSystem.
Thank you
Saptarshi
Sep 21 '07 #2
On Sep 20, 8:22 pm, sapsi <saptarshi.g...@gmail.comwrote:
On Sep 20, 7:00 pm, sapsi <saptarshi.g...@gmail.comwrote:
Hello,
I'm not sure if this the correct list but here goes (and sorry for the
noise). I've been attempting to wrap python around libhdfs.
So far so good (

I should point out that libhdfs is a c library to Hadoop Distributed
FileSystem.
Thank you
Saptarshi
Hmm,here goes, you need -Wl,--rpart -Wl,path-to-jvm in the LDFLAGS in
the compile process.
HTH
Saptarshi

Sep 21 '07 #3
On Sep 21, 9:00 am, sapsi <saptarshi.g...@gmail.comwrote:
Hello,
I'm not sure if this the correct list but here goes (and sorry for the
noise). I've been attempting to wrap python around libhdfs.
So far so good (i've attached the SWIG template at the end). The
compilation works without errors and the shared objects do have
references to all the functions.

However, when importing into python

import pyhdfs

i get the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/home/sguha/tmp/b/hdfs.py", line 7, in ?
import _hdfs
ImportError: libjvm.so: cannot open shared object file: No such file
or directory

However, libjvm.so is located in /home/sguha/mine/jdk1.6.0_02/jre/lib/
amd64/server which is present in the PYTHONPATH and sys.path.
I can understand it complaining <i>while</iloading the libjvm.so
but python doesn't locate it even when in the path.

As an aside, the build command for hdfs.c has " -ljvm " - this
should affect SWIG, right? Meaning if my program (e.g) links to a
library and i want a wrapper around my program (to python) i need SWIG
templates for my headers only and not for the linked library...right?
( I guess the answer is i dont otherwise that would be quite a
nightmare of work)

Thank you for your time
Saptarshi

Attachments:
SWIG template
%module pyhdfs

%{
#include "hdfs.h"
%}

%include "hdfs.h"
Your libjvm.so isn't being found because it isn't in a standard system
library directory. You have a few choices to solve this:

1. Install libjvm.so into /lib or /usr/lib. You may be able to get
away with putting it in /usr/local/lib if your OS is setup to look
there.

2. If a Linux/Solaris box, do:

LD_LIBRARY_PATH=/home/sguha/mine/jdk1.6.0_02/jre/lib/amd64/server
export LD_LIBRARY_PATH

then run your application. This will tell you application to look in
that directory for the shared library at run time.

3. If on Linux/Solaris, compile that library directory into the Python
module .so by setting before you compile your Python module:

LD_RUN_PATH=/home/sguha/mine/jdk1.6.0_02/jre/lib/amd64/server
export LD_RUN_PATH

This only needs to be done at compile time, but means your module will
always expect it to be in that location.

Graham

Sep 21 '07 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

5 posts views Thread by Gary | last post: by
13 posts views Thread by Roy Smith | last post: by
1 post views Thread by Arthur Chereau | last post: by
reply views Thread by Claudio Grondi | last post: by
1 post views Thread by Java and Swing | last post: by
2 posts views Thread by ajikoe | last post: by
6 posts views Thread by matey | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.