how to lookup a unix file through C program through routine

[somesh2suma]

hi all,

i have a requirement to lookup a file through placed on unix from C program.
Let me explain the example below:

Lookup_codes.txt contains:
IN
UK
HK
US
EN

like this i have 250 country codes available in individual lines.

My C program Lookup_county.cpp should contain:
1 or 2 parameters like 2 digit country code and the unix file_name as 2nd parameter.

so, if the 1st parameter value is available in lookup_codes.txt file then it should return the same code as it is to datastage
otherwise(else)
C program should return "XX" to the datastage.

How to proceed with this and how to pass the directory path of the unix file to C program
please help me with this problem.

[chulett]

Why in the world use 'a C program' and reinvent the wheel? Why not simply use a Lookup stage?

[somesh2suma]

oh i forgot to tell that this small c program is called inside another routine many times. and the input and output values of this lookup c program is supplied by the external main function runtime based on other values.

i can handle the main function but problem is with lookup with unix file. So please let me know only this functionality later i will handle things.

[chulett]

Can't help with the C code but still wondering if you can't simply do your check pre-call and pass the result in? And in a way this really isn't a "lookup" in the sense most might think but rather more of a simple existence check with no additional information being 'looked up'.

Can't imagine it's all that hard to read those values in (or pass them in) and then just see if you get a substring match or not from it.

[somesh2suma]

yes you are exactly correct, its not a actual lookup but just existence check.

ok now i have kept all my requirements into this blog:

Routine name:ORIG_BANK_BIC

1) Initialize the varaibles as given to ""
2)If _FLAG[IN_COV] == 1 then --- this variable is set in jbTXN8952WireTxnTfm1 i.e. svFlagInCov
3)execute function Lookup_BIC passing AC_SEQ_ACU_ID_ORD as parameter and assign the
return value to _BIC variable -- the above field is obtained from the i/p file inward_cover record type AC_SEQB_ACU_ID_ORD column
4) If the return value from function is not = 'XXXXXXXXX' then
5) execute function LOOKUP_BIC_TO_ISO_COUNTRY passing _BIC set above as parameter and assign
return value to _ORIG_BANK_ISO variable.



6)If _FLAG[ADD] == 1 then --- this variable is set in jbTXN8952WireTxnTfm1 i.e. svFlagAdd
7)execute function Lookup_BIC passing AD_SWA_CD_ORD_BNK as parameter and assign the
return value to _BIC variable -- the above field is obtained from the i/p file add_pay record type
If the return value from function is not = 'XXXXXXXXX' then
9) execute function LOOKUP_BIC_TO_ISO_COUNTRY passing -BIC set above as parameter and assign
return value to _ORIG_BANK_ISO variable.



10) IF (IN_FTG_CD_052 == 'A') --- this field is from i/p file inward record type
7)execute function Lookup_BIC passing IN_SWA_CD_BNK_ORD as parameter and assign the
return value to _BIC variable -- the above field is obtained from the i/p file inward record type
9) execute function LOOKUP_BIC_TO_ISO_COUNTRY passing -BIC set above as parameter and assign
return value to _ORIG_BANK_ISO variable.


10) IF (IN_SWA_CD_BNK_ORD is null or blank)
11)execute function Lookup_BIC passing IN_SWA_CD_FUL_DST as parameter and assign the
return value to _BIC variable -- the above field is obtained from the i/p file inward record type
12) execute function LOOKUP_BIC_TO_ISO_COUNTRY passing _BIC set above as parameter and assign
return value to _ORIG_BANK_ISO variable.



13) IF (IN_FTG_CD_053 == 'A') --- this field is from i/p file inward record type
14) assign 'XXXXXXXX' to _BIC variable
15) execute function Lookup_BIC passing IN_SWA_CD_SBC as parameter and assign the
return value to _FINT_BIC variable -- the above field is obtained from the i/p file inward record type
16)execute function LOOKUP_BIC_TO_ISO_COUNTRY passing _FINT_BIC set above as parameter
and assign result to variable _FINT_BANK_ISO
17) Assign _FINT_BANK_ISO set above to _ORIG_BANK_ISO variable
18)execute function Lookup_BIC passing IN_SWA_CD_FUL_DST as parameter and assign the
return value to _SINT_BIC variable -- the above field is obtained from the i/p file inward record type
19)execute function LOOKUP_BIC_TO_ISO_COUNTRY passing _SINT_BIC set above as parameter
and assign result to variable _SINT_BANK_ISO



20) execute function Lookup_BIC passing IN_SWA_CD_FUL_DST as parameter and assign the
return value to _SENDER variable -- the above field is obtained from the i/p file inward record type
21) assign 'XXXXXXXX' to _BIC variable
22)execute function LOOKUP_BIC_TO_ISO_COUNTRY passing _SENDER set above as parameter
and assign result to variable _ORIG_BANK_ISO
23) Assign _SENDER set above to _FINT_BIC variable
24) Assign _ORIG_BANK_ISO set above to _FINT_BANK_ISO variable

where
ORIG_BANK_BIC is a routine in which 2 functions a)Lookup_bic b)LOOKUP_BIC_TO_ISO_COUNTRY are called several times.


_FLAG[IN_COV],_FLAG[ADD],IN_FTG_CD_052 ,(IN_FTG_CD_053 ) are the stage variables which are already defined in transformer.

and remaining others which has "_" like _bic are C variables.

please let me know if this can be done through any way either with routines, datastage itself or through job control etc.

so the above scenario of looking up the file in unix is required for LOOKUP_BIC_TO_ISO_COUNTRY function.
please give some suggestions regarding this.

[Sainath.Srinivasan]

Are they mutually exclusive conditions ? Have you tried grouping the lookups / reference activities ?

Did you give any thought about using shared containers ?

Both C and C++ provides many features to scan and search through data residing in various media.

Simple one will be to read in the list and browse through each time.

In C, you can also do system(unixCommand).

[somesh2suma]

hi all,

i am almost done with my coding but struck at some big error. let me explain clearly.
the below code is working fine with a main function but its not working if i want to change the function name to create a routine.

it gives this error:
ld: 0711 - 317 ERROR: undefined symbol:.main
ld: 0711 - 345 use the -bloadmap or -bnoquiet option to obtain more information.

#include <stdio.h>
#include <stdlib.h>

char *IsoCountry(char *);

char *IsoCountry(char *iso)
{
char rc;
char code[1];

File *fp=fopen("/home/routines/unix_file.txt","rb");

if (fp==NULL)
{
perror("failed to open this file");
return EXIT_FAILURE;
}

while(!feof(fp))
{
rc=fgetc(fp);
if (code[0]==toupper(rc))
{
rc=fgetc(fp);
if (code[1]==toupper(rc))
{
return iso;
break;
}
}
}
return "XX";
fclose(fp);
}

This code is working and it will lookup the country code if main is not used otherwise it will give the above error message.

please help me with this issue.

[Kryt0n]

You would be better off reading a line at a time rather than a character.

As for your issue, how did you compile? There's a guide somewhere on how to create a routine (search DSX forums as someone provided a link once or twice), did you follow that?

The error implies either you are trying to create an executable without a main function or trying to create a library with a main function (I don't know enough about C to be sure which way it is going)

[somesh2suma]

hey thanks,

As you said it was my mistake, i didn't use the compiler options.
Now its working fine.

[ray.wurlod]

Please mark this thread as Resolved.

Do you propose to rewrite the whole of DataStage, or just the Lookup stage?

[Sainath.Srinivasan]

If you are going to use file pointers with open and close each time, think about running 1 million (or billion) rows through it.

Also what happens if you call the function in parallel ?

[somesh2suma]

If you are going to use file pointers with open and close each time, think about running 1 million (or billion) rows through it.
So please suggest me something which can overcome this performance issue.

Also what happens if you call the function in parallel ? ...

Sorry i didn't understand this!! please let me know clearly.

Do you propose to rewrite the whole of DataStage, or just the Lookup stage? ...

Sorry i didn't understand even this!! please let me know clearly.

[ray.wurlod]

Your function mimics what the Lookup stage does - at least one of the things that it does. So why are you going to the trouble of writing the function? Surely it's unnecessary.

My question was a cynical response to your reinvention of this particular wheel.

[somesh2suma]

Hi ray,
i think you didn't read my whole requirement. I accept the code which i have written is just a basic functionality of lookup, but it has a dependency because the input for this is given based on some validity condition from another routine and the output from this routine will be used to take some other action.

i got an idea of using shared containers before but that would be again a tedious task as you can see my whole requirement at the top mentioned above.

So please suggest me even if by using a shared container can we improve the performance, if yes give some hints on that cos i know very less about shared containers.

[ray.wurlod]

Do a regular lookup against lookup_codes.txt. Outputs from the Lookup stage contain:
(0) the found country code
(1) the source lookup code, which is passed through a Column Generator stage to replace it with "XX"
These two outputs then run into a Funnel stage to return them to a single stream.

Create the file name as a job parameter or, if it must be different for each country, a separate lookup against file_names.txt with similar conditions as above, perhaps conditional on the country code not being "XX".

No need for shared containers, no need for routines.