Unable to call an external C routine from a Parallel Routine

[ray.wurlod]

The libraries need to be found by other processes. The directories in which the libraries live need to be recorded in the "shared library path" environment variable (LD_LIBRARY_PATH, LIBPATH or SHLIB_PATH, depending on which UNIX you're on).
This variable needs to be set in the dsenv file, which is executed by all DataStage processes.

[Eric]

How did you compile the 'C' code?

if you did not compile the code on the same platform etc .. then you would expect this error.

[banactp]

Hello,

I'm getting the same error. The .a and .o files are in place, their folder is in the LD_LIBRARY_PATH (set in the dsenv file), and I'm compiling on the same platform that I'm running on.

The commands I'm using for the build on my Solaris 5.8 system are:

.../SunCompilers/bin/CC -c -DSOLARIS_CC -g -features=castop,rtti -DSOLARIS58 -I[a bunch of include dirs] source.C

followed by:

/usr/ccs/bin/ar r libSOLARIS_source.a source.o

I've tried using both the object and the library in my routine definition - both methods result in the same error. Has anyone figured this one out?

TIA, tpb

[Eric]

If the job compiles (use force compile to be sure) then DataStage can find the library file and I expect the problem will be with the way the compile of the library was performed.

You could Look at the compiler options in the DataStage Administrator and then use these options when compiling the External C code, or look at the example Makefile in your DataStage Media disk, /Samples/TrxExternalFunctions/<Platform>

[richdhan]

Hi,

I have seen a similar kind of error during runtime and a force compile fixed the problem.

Did you do any upgrade or plug-in installation?

HTH
Rich

[banactp]

Update:

I recompiled everything exactly as specified in the examples from the Ascential installation disk. I also rebuilt the routine definition and jobs from scratch.

Still getting the same error.

There is nothing unusual about our installation as far as I know. Our administrator has been looking into the problem, but doesn't have any answers either. We are all newbies here.

tpb
DataStage newbie

[T42]

Did you "distribute_component" the library?

[dsxuserrio]

Try the following ( may sound low tech )
1. If you made any changes to dsenv ,stop and start your datastage services.

2. Make sure the library you built has the method/function.

3. Put the library in a standard library path which you know for sure.

[Eric]

I have given the exact library path (-/home_dir/somename where the routine is existing), while creating the routine. I set environment variable LD_LIBRARY_PATH to

/home_dir/somename:/apps/Ascential/DataStage/PXEngine/lib:/opt/oracle/product/9.2.0.1/lib32:/apps/Ascential/DataStage/PXEngine/user_lib:/export/home/db2inst1/sqllib/lib

I have my .so file both in /home_dir/somename and /apps/Ascential/DataStage/PXEngine/lib also

The *.so file needs to be in the location specified in the PXRoutine definition. Also if this rouinte is a Library (dynamic) then the Path to
the routine (as used in the PXRoutine definition) must be included in
your LD_LIBRARY_PATH.

No other paths need to be added to LD_LIBRARY_PATH, try looking at the runtime environement used by the job which is shown within the Director Log.

[aesguerra]

Hi,

You can try using the ldd utility on the file to see what libraries it links to at runtime.

Cheers!

[banactp]

After many hours of tribulation, I have finally managed to successfully call an external routine from PX! :D

It appears that the following requirements must be observed:

• 1. You may not use an underscore ('_') in the name of the library or object file that contains your routine.
  
  2. You must modify the job's $LD_LIBRARY_PATH environment variable through the Job Parameters dialog so that the path includes the directory in which your routine's library or object file is located.
  
  3. The external C/C++ routine must return one of the 11 data types allowed by DataStage. These are:
  • char and unsigned char
    char *
    double
    float
    int and unsigned int
    long and unsigned long
    short and unsigned short
  Note that the const modifier is not supported in the return type. You must cast away the const modifier if it exists in your code. Passing a static value or a pointer to a static value is OK.
  
  4. Input and output arguments of DataStage type "long" or "unsigned long" must be declared "long long" or "unsigned long long" in your code. Input arguments of DataStage type "char" must be defined as "signed char" in your code. At least, this is true on my Solaris system.

I also switched to using the Studio 8 CC compiler on my system instead of the default compiler, but I am not sure that was necessary.

I sincerely hope this post spares others from a similar experience.

tpb