Generating Sequence numbers in Parallel Transormer
Moderators: chulett, rschirm, roy
Generating Sequence numbers in Parallel Transormer
Hi All,
Scenario 1. Generating Sequence numbers by using Surrogate key option with NextSurrogateKey() utility in Parallel Transform stage.
Scenario 2. calling System varibales @OUTROWNUM in output derivation.
in both Scenario's DataStage job is generating even numbers, I know this is because of more than one partition.
Is there-a-way to generate sequence numbers with-in the Transformer stage, other than using Surrogate Key Generator stage itself.
Any idea is great
Thanks
srimitta
Scenario 1. Generating Sequence numbers by using Surrogate key option with NextSurrogateKey() utility in Parallel Transform stage.
Scenario 2. calling System varibales @OUTROWNUM in output derivation.
in both Scenario's DataStage job is generating even numbers, I know this is because of more than one partition.
Is there-a-way to generate sequence numbers with-in the Transformer stage, other than using Surrogate Key Generator stage itself.
Any idea is great
Thanks
srimitta
Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.
By William A.Foster
By William A.Foster
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Use a stage variable initialized to the partition number (or, perhaps, (next available key value + partition number)) and incrementing by partition count. There are system variables that yield these partition values.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Thanks Ray for te response.
@PARTITIONNUM
@INROWNUM
@NUMPARTITIONS
@OUTROWNUM
Can you help me in how to use these varibales in Transformer stage to get right sequence.
Thanks
Srimitta
@PARTITIONNUM
@INROWNUM
@NUMPARTITIONS
@OUTROWNUM
Can you help me in how to use these varibales in Transformer stage to get right sequence.
Thanks
Srimitta
Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.
By William A.Foster
By William A.Foster
Thanks Ray I got it working
In output derivation
(NextSurrogateKey() + @PARTITIONNUM) / @NUMPARTITIONS
In output derivation
(NextSurrogateKey() + @PARTITIONNUM) / @NUMPARTITIONS
Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.
By William A.Foster
By William A.Foster
Oops I could generate sequence numbers from one partition only,
any what changes do I need to print right sequnce.
any what changes do I need to print right sequnce.
Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.
By William A.Foster
By William A.Foster
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Transformer input link is reading 7707 rows and writing same number.of (7707) rows into DataSet, but when I try to view data I could see only 50% of total input records.
I checked job log for errors, didn't find any.
Do you think CODE is wrong! or am I missing something here.
Thanks
srimitta
I checked job log for errors, didn't find any.
Do you think CODE is wrong! or am I missing something here.
Thanks
srimitta
Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.
By William A.Foster
By William A.Foster
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Code: Select all
Partitions:
# Node Records Blocks Bytes
0 node1 3609 38 4893804
1 node2 3703 39 5021268
Any idea if I have to view all rows in DataSet what is that I need to look into.
Thanks
srimita
Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.
By William A.Foster
By William A.Foster
And one more thing is Data in DataSet not in Sequence order it starts from 1 and after 96 again row starts from 3704 and ends with 3799 and sratrs from 97 ends with 192.
Sequence no changes app after every 95 rows.
Any idea hwat's going-on and how to staight-up this.
Thanks
srimitta
Sequence no changes app after every 95 rows.
Any idea hwat's going-on and how to staight-up this.
Thanks
srimitta
Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.
By William A.Foster
By William A.Foster