I have job design where i need to sort the data based on timestamp column(microsecond) and after sort apply transormation logic in transformer which should pick the max timestamp and populate for rest of records.
Stage veriable value is:-
Code: Select all
If srt_to_tfm.LastChgDateTime>= svEFFDT then srt_to_tfm.LastChgDateTime else svEFFDT
Instead of getting max timestamp for all input records,i am getting wrong output.
It seems timestamp column is sorted on each partition and stage veriable is picking that partitions max timestamp value and for next partition some other value,but i want max timestamp value for all input records.
This job is working filr on single node.
Kindly help me with some solution to this issue.
Thanks