Filter ranges of records in a file based on search strings

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
jerome_rajan
Premium Member
Premium Member
Posts: 376
Joined: Sat Jan 07, 2012 12:25 pm
Location: Piscataway

Filter ranges of records in a file based on search strings

Post by jerome_rajan »

Dear All,

We have a file that serves as kind of a master and holds data that should go into two different target tables with completely different metadata. Our source file looks something like

Code: Select all

<Line1>
a,b,c,d
g,h,i,j
<line2>
1,2
3,4
5,6
<line3>
garbage
<line1>
f,g,h,k
<line2>
1,2
7,9
<line3>
garbage
<line1>
...and so on

The idea here is to pick data lying between <line1> and <line2> and load into one Target(say A) and data between <line2> and <line3> to another target(say B). The data between <line3> and <line1> should be ignored.

I've started by reading each line into a single column. I'm trying to use the Transformer Loop to identify and bifurcate the data into two different links based on the logic mentioned above.

While I try, can someone please assist me in the looping logic?

Thank you.
Jerome
Data Integration Consultant at AWS
Connect With Me On LinkedIn

Life is really simple, but we insist on making it complicated.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I don't see a need for any kind of 'looping' here, just stage variable(s) to tract the current target based on the tags seen and then appropriate constraints on the output links.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply