Get the Input link record count and pass/insert into target

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
gnan_gun
Participant
Posts: 30
Joined: Thu Aug 02, 2007 5:31 am
Location: Mumbai, India

Get the Input link record count and pass/insert into target

Post by gnan_gun »

I have a job like
Oracle(Source) ---> Lookup ---> Join ---> Transformer ---> Oracle(Target)
My requirement is i need get the input link record count and pass that count value into Oracle(target) table as Total_input_records.
I have written routine in Server using Linkinfo. Its working fine. but in Parallel I am unable to write/modify.
Is there any functions available in PX with out writing PX routine to get input link record count.
Please help me to resolve this.
Thanks in advance.

--Gnan
gnan_gun
Participant
Posts: 30
Joined: Thu Aug 02, 2007 5:31 am
Location: Mumbai, India

Re: Get the Input link record count and pass/insert into tar

Post by gnan_gun »

Any Replies with PX routine or alternative of PX routine to achieve this??
mk_ds09
Participant
Posts: 72
Joined: Sun Jan 25, 2009 4:50 pm
Location: Pune

Post by mk_ds09 »

If you want to the input number of records only which is from oracle table, you can write the before job routine to call the unix script.

In the script
count = select count(*) from the table_name

update table_2 set total_input_records = count

-------------------------

Hope this helps..

----------------------

MK
samsuf2002
Premium Member
Premium Member
Posts: 397
Joined: Wed Apr 12, 2006 2:28 pm
Location: Tennesse

Post by samsuf2002 »

Why not use @INROWNUM in the transformer for target rowcount column.....
hi sam here
mk_ds09
Participant
Posts: 72
Joined: Sun Jan 25, 2009 4:50 pm
Location: Pune

Post by mk_ds09 »

Why not use @INROWNUM in the transformer for target rowcount column.....

@INROWNUM gives the number to each row that is passing through the tramsformer.. however the issue with this one is that if job is executing on multiple nodes, the @INROWNUM will give differenet numbers on the rows on each nodes.

If there are 2 nodes with two rows ..

it will give 1 and 2 for 1st and 2nd row as they are executing on node1
again it will give 1 and 2 to 3rd and 4th row as they are excuting on the node2.

so this will not give the correct count of the records ..

hope this helps

--------------------------

MK[/i]
mspanda
Participant
Posts: 32
Joined: Tue May 09, 2006 6:22 am
Location: bangalore

Post by mspanda »

Hi
call a unix script from datastage job(from before job subroutine) by passing the parameter like <projecetname> <jobname> <stagename> <linkname>

script should contain the following command.
dsjob -linkinfo <projecetname> <jobname> <stagename> <linkname> |head -1 | awk '{ print $5 }'>/path/outputfile.txt.

This /path/outputfile.txt file will be source and oracle will be target.Then insert.
Hop this will resolve the issue.

Thanks,
mspanda
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Use agg stage and get max(rownum)
gnan_gun
Participant
Posts: 30
Joined: Thu Aug 02, 2007 5:31 am
Location: Mumbai, India

Post by gnan_gun »

I dont want to modify/add the stages in the job. This has to be solved by PX routine or Unix script(After Job Subroutine).

mspanda--> U r correct. But has to implement this command in unix script. Thank for ur reply.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

OK... if you "have to" do this in a script then, as noted, you would use dsjob with the -linkinfo option.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply