Parallel Routine for DSSetEnvVar

devidotcom · Post by **devidotcom** » Mon Mar 31, 2008 4:01 am

Hi All,

I have a requirement of updating the environment parameter defined at project level. I need to build a parallel routine of the same so that it can be used in a parallel job.
I have referred the advanced PX manul for the same and know how to create a parallel routine.
But does any one have a tested code for DSSetEnvVar parallel routine already.

Thank you
Devi

ray.wurlod · Post by **ray.wurlod** » Mon Mar 31, 2008 3:24 pm

If you call it from a job sequence or a before/after subroutine you use a server routine, not a parallel routine. Do you really want to set the environment variable for every row processed?

devidotcom · Post by **devidotcom** » Mon Mar 31, 2008 10:14 pm

Thanks Ray for your reply.

I am following Joshy approach to generate a surrogate key where in he mentioned we will need a parallel routine in the transformer stage to update the last maximum number generated in the surrogate key.

http://blogs.ittoolbox.com/dw/soa/archi ... 82#1954763

I have a doubt in his approach too. If for some reason the job gets aborted will the surrogate key be generated correctly as there is a possibility that the parameter got set but the job aborted. Then in the next run the parameter value will be the new value.

Devi

devidotcom · Post by **devidotcom** » Mon Apr 07, 2008 10:11 pm

Could anyone help me on this post.

ray.wurlod · Post by **ray.wurlod** » Mon Apr 07, 2008 10:18 pm

That would depend entirely on what the routine does, so there's no ready answer to what is at best a hypothetical question.

Having said that, it does not matter that there are "gaps" in sequences of surrogate key values. Surrogate keys provide uniqueness and nothing else - by definition.

devidotcom · Post by **devidotcom** » Tue Apr 08, 2008 1:35 am

[quote="ray.wurlod"]That would depend entirely on what the routine does, so there's no ready answer to what is at best a hypothetical question.

Having said that, it does not matter that there are "gaps" in sequences o ...[/quote]

Thanks Ray for your reply.
I tried writing the following code for this routine

#include<iostream.h>
#include<string.h>
#include "/u001/Ascential/DataStage/DSEngine/include/dsapi.h"

int DSSetEnv(char* ProjName, char* val, char* ProjParm)
{
char *EnvVarName;
char *Value;
char *Project;
int retval;
DSPROJECT hProject;

EnvVarName=ProjParm;
Value=val;
Project=ProjName;

hProject=DSOpenProject(Project);

retval = DSSetEnvVar(hProject, EnvVarName, Value);
return retval;
}

I get the following errors

Transformer_21: Failed to load the library "V0S21_Surrogate_Key_Transformer_21.o"; either the directory containing the library file
is not on the library search path, or the library was compiled on a system
that is incompatible with this system: Could not load "V0S21_Surrogate_Key_Transformer_21": rtld: 0712-001 Symbol DSOpenProjectEx was referenced
from module /u001/Ascential/DataStage/Projects/EDQS_CIS/RT_BP200.O/V0S21_Surrogate_Key_Transformer_21.o(), but a runtime definition
of the symbol was not found.
rtld: 0712-001 Symbol DSSetEnvVar was referenced
from module /u001/Ascential/DataStage/Projects/EDQS_CIS/RT_BP200.O/V0S21_Surrogate_Key_Transformer_21.o(), but a runtime definition
of the symbol was not found.. [transform/transform.C:1520]

I tried all the possible things I can
1. Force compile the job
2. Checked my environment path under LD_LIBRARY_PATH
LD_LIBRARY_PATH=/db2home/db2cli/sqllib/lib:/usr/lpp/tdodbc/odbc/lib:/users/apps/edwdev/mitrd01

3. Routine has the following settings
Routine name: SetEnvVar
Category: WFIS
External SubRoutine name:DSSetEnv
Library path: /users/apps/edwdev/mitrd01/SetEnvPar.o
Object type: Object
Type: External function

4. In administrator the
APT_COMPILEOPT=-O -c -qspill=32704 -I $EDWCMNDIR/DataStage/src
APT_COMPILER=/usr/vacpp/bin/xlC_r

I compiled the code as follows

/usr/vacpp/bin/xlC_r -O -c -qspill=32704 -I $EDWCMNDIR/DataStage/src SetEnvPar.cc

/usr/vacpp/bin/xlC_r -G SetEnvPar.so SetEnvPar.o

I tried searching the forum for help but I could not get much info to solve the issue.

devidotcom · Post by **devidotcom** » Tue Apr 08, 2008 4:57 am

Am I doing right?

DS_FocusGroup · Post by **DS_FocusGroup** » Wed Apr 09, 2008 11:42 pm

i think he did mention in his post that if the job gets aborted it wont effect on the SK

DS_FocusGroup · Post by **DS_FocusGroup** » Wed Apr 09, 2008 11:43 pm

i think he did mention in his post that if the job gets aborted it wont effect on the SK

devidotcom · Post by **devidotcom** » Thu Apr 10, 2008 2:23 am

Hi All,

Any one yet has some suggestions on this.

Thanks

devidotcom · Post by **devidotcom** » Wed Apr 16, 2008 7:47 am

I tried the following:

/usr/vacpp/bin/xlC_r -b loadmap:a.txt SetEnvPar.cc

Output is
ld: 0711-317 ERROR: Undefined symbol: .main
ld: 0711-344 See the loadmap file a.txt for more information.

a.txt contains
"a.txt" 43 lines, 2275 characters
(ld): halt 4
(ld): setfflag 4
(ld): savename a.out
(ld): filelist 7 1
(ld): i /lib/crt0.o
(ld): i SetEnvPar.o
(ld): lib /usr/vac/lib/libxlopt.a
(ld): lib /usr/vacpp/lib/libC.a
(ld): lib /usr/lib/libpthreads.a
(ld): lib /usr/lib/libm.a
(ld): lib /usr/lib/threads/libc.a
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[shr_32.imp]: 402
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[shr2_32.imp]: 42
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[shr3_32.imp]: 33
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[ansi_32.imp]: 2360
LIBRARY: Shared object libC.a[ansi_32.o]: 2592 symbols imported.
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[shr_64.imp]: 0
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[shr2_64.imp]: 0
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[shr3_64.imp]: 0
LIBRARY: Symbols imported from import file /usr/vacpp/lib/libC.a[ansi_64.imp]: 0
LIBRARY: Shared object libpthreads.a[shr_comm.o]: 176 symbols imported.
LIBRARY: Shared object libpthreads.a[shr_xpg5.o]: 162 symbols imported.
LIBRARY: Shared object libc.a[shr.o]: 2865 symbols imported.
LIBRARY: Shared object libc.a[meth.o]: 2 symbols imported.
LIBRARY: Shared object libc.a[posix_aio.o]: 20 symbols imported.
LIBRARY: Shared object libc.a[aio.o]: 14 symbols imported.
LIBRARY: Shared object libc.a[pse.o]: 5 symbols imported.
LIBRARY: Shared object libc.a[dl.o]: 4 symbols imported.
LIBRARY: Shared object libc.a[pty.o]: 1 symbols imported.
FILELIST: Number of previously inserted files processed: 7
(ld): resolve
RESOLVE: 104 of 11933 symbols were kept.
(ld): addgl /usr/lib/glink.o
ADDGL: Glink code added for 18 symbols.
(ld): er full
ld: 0711-318 ERROR: Undefined symbols were found.
The following symbols are in error:
Symbol Inpndx TY CL Source-File(Object-File) OR Import-File{Shared-object}
RLD: Address Section Rld-type Referencing Symbol
----------------------------------------------------------------------------------------------
.main [8] ER PR crt0main.s(/lib/crt0.o)
00000098 .text R_RBR [30] .__start
ER: The return code is 8.

The code for SetEnvPar.cc is as follows:
#include<iostream.h>
#include<string.h>
#include "/u001/Ascential/DataStage/DSEngine/include/dsapi.h"

int DSSetEnv(char* ProjName, char* val, char* ProjParm)
{
char *EnvVarName;
char *Value;
char *Project;
int retval;
DSPROJECT hProject;

EnvVarName=ProjParm;
Value=val;
Project=ProjName;

hProject=DSOpenProject(Project);

retval = DSSetEnvVar(hProject, EnvVarName, Value);
return retval;
}

Any help please apprepriated

devidotcom · Post by **devidotcom** » Wed Apr 16, 2008 11:20 pm

If I use a basic transformer to set the environment project defined variable what will be the impact on performance.

I even do not see the server routine of DSSetEnvPar to call it in the basic transformer

devidotcom · Post by **devidotcom** » Tue Apr 22, 2008 2:11 am

I tried running the job with the transformer stage having the routine. Yet get the same error

Trn_Fetch_Max_Value: Failed to load the library "V15S0_EDQS_CIS_POP_COUNT_DS_Trn_Fetch_Max_Value.o"; either the directory containing the library file
is not on the library search path, or the library was compiled on a system
that is incompatible with this system: Could not load "V15S0_EDQS_CIS_POP_COUNT_DS_Trn_Fetch_Max_Value": rtld: 0712-001 Symbol DSOpenProjectEx was referenced
from module /u001/Ascential/DataStage/Projects/EDQS_CIS/RT_BP270.O/V15S0_EDQS_CIS_POP_COUNT_DS_Trn_Fetch_Max_Value.o(), but a runtime definition
of the symbol was not found.
rtld: 0712-001 Symbol DSSetEnvVar was referenced
from module /u001/Ascential/DataStage/Projects/EDQS_CIS/RT_BP270.O/V15S0_EDQS_CIS_POP_COUNT_DS_Trn_Fetch_Max_Value.o(), but a runtime definition
of the symbol was not found.. [transform/transform.C:1520]

Any help on this? Tried searching help online too...

devidotcom · Post by **devidotcom** » Tue Apr 29, 2008 3:42 am

Any help on this?

hsbc_ds_dev · Post by **hsbc_ds_dev** » Mon Sep 22, 2008 12:09 am

Hi Devi,

Did you get your problem resolved?

Regards,
A DS Developer