Help needed in converting code which is in DS Server routine

rapaka89 · Post by **rapaka89** » Fri Feb 04, 2011 7:52 am

OutputFile = LogPath:'/':CrossReferenceTable:'_':NaturalKeyColumnName:'_':JobPID:StagePID:'.dati' 
HashFile1 = LogPath:'/':DB2Schema:'_':CrossReferenceTable:'_':NaturalKeyColumnName:'_':JobPID:'_':StagePID:'_temp1' 
HashFile2 = LogPath:'/':DB2Schema:'_':CrossReferenceTable:'_':NaturalKeyColumnName:'_':JobPID:'_':StagePID:'_temp' 


UNIXcmd = 'cat ':TDPATHbin:'/SurrKeyGen_Index_csv | grep -e ':LogSubject:' | grep -e ':NaturalKeyColumnName' ' 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
IF ReturnCode = 0 
THEN 
UNIXcmd = 'cat ':TDPATHbin:'/SurrKeyGen_Index_csv | grep -e ':LogSubject:' | grep -e ':NaturalKeyColumnName:' |cut -d "," -f2 ' 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
InputFileName = ScreenOutput<1> 

UNIXcmd = 'cat ':TDPATHbin:'/SurrKeyGen_Index_csv | grep -e ':LogSubject:' | grep -e ':NaturalKeyColumnName:' |cut -d "," -f4 ' 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
KeyPos = ScreenOutput<1> 

UNIXcmd = "echo '$KeyPos' | awk -F ":" '{print NF}' " 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
NoofFields = ScreenOutput<1> 


n=1 
CALL DSExecute("UNIX", " while '$n' <= '$NoofFields' ", ScreenOutput,ReturnCode) 
IF ReturnCode<1> = 0 
THEN 
UNIXcmd = "echo '$KeyPos' | cut -d ":" -f'$n' " 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
NaturalKeyPos = ScreenOutput<1> 

UNIXcmd = " echo ':NaturalKeyPos:' | grep '|' " 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
IF ReturnCode <> 0 
THEN 
GoTo OutLabel 

END 
ELSE 

UNIXcmd = "echo '$NaturalKeyPos' | awk -F '|' '{print NF}' " 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
NoofColField = ScreenOutput<1> 

x=2 
UNIXcmd = "echo '$NaturalKeyPos' | cut -d '|' -f1 " 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
y = ScreenOutput<1> 

CALL DSExecute("UNIX", " while '$x' <= '$NoofColField' ", ScreenOutput,ReturnCode) 
If ReturnCode<1> = 0 
THEN 
UNIXcmd = "echo '$NaturalKeyPos' | cut -d '|' -f'$x' " 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
z = ScreenOutput<1> 

UNIXcmd = 'cat 'TDPATHbin:'/SurrKeyGen_Index_csv | cut -d ,-f"$y","$z" | sed ''s/,//g'' ' 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
OutputFile = ScreenOutput<1> 

x += 1 
END 

n += 1 
END 
END 
END 

OutLabel: 
UNIXcmd = 'cat 'LogPath:'/':InputFileName:' | cut -d "':InputFileDelimiter:'" -f ':NaturalKeyPos:' | sed "s/\"//g" | sed /^$/d > ':OutputFile 
CALL DSExecute("UNIX", UNIXcmd, ScreenOutput,ReturnCode) 
If ReturnCode <> 0 THEN CALL DSLogFatal('Error when parsing the input file. ReturnCode = ':ReturnCode:', Command = ':UNIXcmd, ProgramName) 
** open output file 
OPENSEQ OutputFile TO OUTPUTFILE THEN 
CALL DSLogInfo('Opened Output File: ':OutputFile, ProgramName) 
END 
ELSE CALL DSLogFatal('Could not open Output File: ':OutputFile, ProgramName) 


** close output file 
CLOSE OUTPUTFILE

===========================================
This is sort of big code which I want to convert but the above is not working at all..
Problems I am facing -
Unable to assign the output of a UNIXcmd to a variable ..so that I can use it in another UNIX command

For ur convenience
TDPATHbin, LogSubject, NaturalKeyColumnName are coming from the Job itself and SurrKeyGen_Index_csv is a file from which I am greping subject and natural key column.
Let me know if more details are required

chulett · Post by **chulett** » Fri Feb 04, 2011 7:58 am

Why two posts on the same subject? How is this different from your other post here?

viewtopic.php?t=138503

Never mind the fact that we have no idea what you mean by 'convert' in this context or specifically what part of this large mess you are having an issue with. Other than what's in your other post, of course.

rapaka89 · Post by **rapaka89** » Sat Feb 05, 2011 1:30 am

chulett wrote:Why two posts on the same subject? How is this different from your other post here?

viewtopic.php?t=138503

Never mind the fact that we have no idea what you mean by 'convert' in this context or specifically what part of this large mess you are having an issue with. Other than what's in your other post, of course.

the problem I am facing with the code is assigning the output from a unix command to a variable.. in the last post its very small code ..
but my actual problem lies with the cide I have mentioned above..
in which I got struck..and I m not getting desired output as I feel I have missed out variouss link in between and cant able to run the above code.

kindly help me to complete this code

If you require I have its code in UNIX shell script which I am trying to convert it in DS basic language

index_file format (SUBJECT,FILE,KEY,KEYPOSITION)
e.g.
money,mon.csv,rupees,1:3

file.csv
123,iiuuiiu,43847,hfhuhi
344,hurhghh,34783,kjfrjj

Code: Select all

	keyposition=`cat ${SCRIPTPATH}/index_file |grep ${SUBJECT} |grep ${KEY}| cut -d , -f4 | sed 's/ //g'`
	echo "Keyposition=$keyposition"
	nooffields=`echo $kp | awk -F ":" '{print NF}'`
	echo "nooffields=$nooffields"

n=1
while [ "$n" -le $nooffields ]
do
	columnkeyposition=`echo "$keyposition" | cut -d ":" -f"$n"`
	echo "columnkeyposition=$columnkeyposition"
	delimiterpresent=`echo "$columnkeyposition" | grep "|" `
if [ "`echo $?`" != "0" ]
then
	echo " passing the keyposition=$columnkeyposition for generating keys "

OUTFILE=`cat ${PATH}/${FILENAME}|cut -f"$columnkeyposition" -d ','`
else
	noofcolumnfield=`echo $columnkeyposition | awk -F "|" '{print NF}'`
	echo "noofcolumnfield=$noofcolumnfield"

x=2
	y=`echo "$columnkeyposition" | cut -d "|" -f1`
while [ "$x" -le $noofcolumnfield ]
do
	z=`echo "$colkp" | cut -d "|" -f"$x"`
	echo "z=$z"
	y=`cat ${SCRIPTPATH}/${FILENAME} | cut -f"$y","$z" -d ',' | sed 's/,//g'`
	echo "y=$y"
	echo " Assigning the value of y to outfile "
let "x+=1"
done
fi
let "n+=1"
done

OUTPUT
for KEYPOSITION 1
123
344
for KEYPOSITION 3
43847
34783

And I want to pass the vaue of keyposition in another script which will generate system keys

My requirement is to convert above shell script into a datastage server routine language which is a Basic Language.

hope you get my requirement

ray.wurlod · Post by **ray.wurlod** » Sat Feb 05, 2011 2:37 am

Does "1:3" mean "1, 2 and 3" or (as suggested by your expected output) just "1 and 3"? In that case, how would you specify "1, 2 and 3"?

It seems to me that you don't need DataStage BASIC at all - this could very simply be done with a DataStage job that simply drops all but the desired columns. Am I misreading this?

Of course you could convert this to code, but I wonder why? And, given that a DataStage BASIC routine has a single output value, what would you expect to see there if your routine had to process the entire file?

A bit more thought is needed, I suspect.

rapaka89 · Post by **rapaka89** » Sat Feb 05, 2011 4:35 pm

ray.wurlod wrote:Does "1:3" mean "1, 2 and 3" or (as suggested by your expected output) just "1 and 3"? In that case, how would you specify "1, 2 and 3"?

It seems to me that you don't need DataStage BASIC at all - ...

no actally : is a delimitter for seperating 1 and 3...
1 is a key position in a file so as 3

ray.wurlod · Post by **ray.wurlod** » Sat Feb 05, 2011 7:35 pm

OK, one question down, many doubtless to follow.

Where do variables LogPath, CrossReferenceTable, JobPID, StagePID, DB2Schema, NaturalKeyColumnName, TDPATHbin, LogSubject get their values initialized?

You can't call the while command by itself in a call to DSExecute. You would have to build the entire shell script loop in a single string separated by field marks.

Can you please, as we've already asked a number of times, describe the purpose of this routine/script in English? That is, can you document its specification/design?

rapaka89 · Post by **rapaka89** » Sun Feb 06, 2011 11:08 am

ray.wurlod wrote:OK, one question down, many doubtless to follow.

Where do variables LogPath, CrossReferenceTable, JobPID, StagePID, DB2Schema, NaturalKeyColumnName, TDPATHbin, LogSubject get their values initializ ...

well they are coming from the job parameter itself..

ray.wurlod · Post by **ray.wurlod** » Sun Feb 06, 2011 12:19 pm

That's why I always name job parameters beginning with "jp" - it makes it obvious in expressions and routines what they are.

Next question. Do you need to understand the shell script, to understand the DataStage BASIC code, or to get something working that is not producing the expected result? Please indicate the expected and actual result, and how you tested this.

rapaka89 · Post by **rapaka89** » Sun Feb 06, 2011 10:16 pm

ray.wurlod wrote:That's why I always name job parameters beginning with "jp" - it makes it obvious in expressions and routines what they are.

Next question. Do you need to understand the shell script, to understand the DataStage BASIC code, or to get something working that is not producing the expected result? Please indicate the expected and actual result, and how you tested this.

I only wrote the shell script so I know what it does

let me give ou clear picture
I have a index file which conatin (SUBJECT,FILE,KEY,KEYPOSITION)
e.g.
money,mon.csv,rupees,1:3
insur,LIC.dat,dollar,5:6

Actually in datastage jobs.. few of the times natural key comes in two different field so the fourth column of index file gives you the position of natural key in the file present in second column

for e.g
for first record key column for mon.csv is 1 and 3

File format of mon.csv
12345,ghghgh,678,fof90
98766,oeoeoo,543,huh3

so the expected result of the script is

initially we are grepping subject name (money) & keycolumn(rupee) for getting the exact record for a particular module, then through the script I will seperate the two key column and pass the first key that is 1 to OUTPUT FILE after that pass the second key position that is 3

key position 1 output -
12345
98766
key position 3 output -
678
543

So I have tried to convert it in basic language
but it seems its not at all working... hope the above is clear with you

ray.wurlod · Post by **ray.wurlod** » Sun Feb 06, 2011 11:23 pm

rapaka89 wrote:...
File format of mon.csv
12345,ghghgh,678,fof90
98766,oeoeoo,543,huh3

so the expected result of the script is

initially ...

Did you leave something important out? Perhaps you can go back and edit your post (don't bother to create a new one!).

rapaka89 · Post by **rapaka89** » Mon Feb 07, 2011 3:54 am

ray.wurlod wrote:
rapaka89 wrote:...
File format of mon.csv
12345,ghghgh,678,fof90
98766,oeoeoo,543,huh3

so the expected result of the script is

initially ...
Did you leave something important out? Perhaps you can go back and edit your post (don't bother to create a new one!).

All the bits are mentioned in above chain.. please let me know if you feel the above is not clear with you

ray.wurlod · Post by **ray.wurlod** » Mon Feb 07, 2011 11:51 am

So, based on the part that I quoted, the expected result is nothing?

rapaka89 · Post by **rapaka89** » Tue Feb 08, 2011 2:34 am

ray.wurlod wrote:So, based on the part that I quoted, the expected result is nothing? ...

Out put
key position 1 output -
12345
98766
key position 3 output -
678
543

which is cominmg from the mon.csv
file mentioned

ray.wurlod · Post by **ray.wurlod** » Tue Feb 08, 2011 12:12 pm

We're getting there.

Your subject is "converting" the routine code. To what do you need to convert it?

rapaka89 · Post by **rapaka89** » Wed Feb 09, 2011 3:27 am

ray.wurlod wrote:We're getting there.

Your subject is "converting" the routine code. To what do you need to convert it? ...

Well I have tried to make the routine code which is mentioned above but that code is not working.Thats the big probs