My gut feel has been that this is going to require looping, however I'm unsure where to begin. And have yet to use looping, but have done plenty of reading about it.
The input data comes in something like this:
This can be done thru a single threaded job.
Pass Input to a transformer. Use stage variables to detect key(Product) change. Transformer will have three output links:
- Link 1: Constraint: key-change; Output: Append 'PRODUCT' to Product-num.
- Link 2: Constraint: None; Output: Append 'WAREHOUSE|' to warehouse-num.
- Link 3: Constraint: key-change and rec-no> 1; Output: "EOR".
Combine output from transformer using Funnel stage in sequential mode. (append another "EOR" to filer-footer).
However.... Upon trying to implement something similar I ran into an issue with the funnel. First off it requires that all the column names are the same, which I guess I can just name them all the same without any side effects. But the sequential mode funnel would not meet my needs. As it would do all of input 1, then input 2, then input 3.
If Continuous funnel is also not giving desired result, then another option could be to pass columns 'Product-num and link-num (1, 2, 3 etc)' from transformer to the output. Sort 'collected' records from transformer using composite key Product-num and link-num.
I think what @rkashyap is getting at is that you might need to add an extra columns to you outputs that can be used to determine sort order.
This could be as simple as the key column AND some sort of counter. Then you can set your funnel to sort the data. After the funnel you would drop the unrequired columns.
Awesome, thanks for the input everybody!!! It helped a ton.
I solved it by sorting it in the funnel as suggested. And added 2 extra columns to sort it correctly. I added the Product# to all the links and sorted it, then a dummy column to keep the links in the correct order, "A", "B", "C".
Thanks again
taylor.hermann wrote:Awesome, thanks for the input everybody!!! It helped a ton.
I solved it by sorting it in the funnel as suggested. And added 2 extra columns to sort it correctly. I added the Product# to all the links and sorted it, then a dummy column to keep the links in the correct order, "A", "B", "C".
Thanks again