Need help with Parsing

[rajeshknl]

Hello my sample data looks like below


|77.2.0.153.1.1|77.2.0.153.1.1|0x3|0|0|1 --------------Header for each row
|0|68------------Part of Header
|7|52|47408---------Data & the Format is Table_No|Column_No|Value
|7|53|0
|129|66|6 34 59 34 34 44 28
|129|67|6 34 59 34 34 44 28
|129|68|6 0 0 0 0 0 0
|129|69|6 0 0 0 0 0 0
|129|70|6 0 0 0 0 0 0

I have to concatenate all the rows with same Table_No along with Header and load it as one record. The Header and Data is repititive. I mean i can have multiple Headers and corresponding Data in the same file.

I want to achieve this within Datastage. If not possible with only datastage i want use very minimal scripting.

[ray.wurlod]

If the data are sorted so that each header and associated detail lines are adjacent (and on the same partition), you can do it in stage variables in a Transformer stage.

You need to ability to detect the header rows, presumably using some characteristic of their structure. You preserve these in stage variables, only updating the same when a new header line comes in. You constrain the output of the Transformer to detail lines, which you have enriched with header information from the stage variables.