Replicating Input Records based on a column value

in_finity307 · Post by **in_finity307** » Thu May 21, 2009 6:54 am

Hi,

I have an input source that has a field A. My requirement is that if the length of If the field A is longer than 132 then split the line in as many records to ensure that tha length is not longer than 132 .

Can you tell me the way in which this can be implemented in Datastage. I am not using Datastage in Unix environment, so a unix shell script can not be used.

One other way I guess would be to use a Build - Op stage but I am not sure how that stage is used.

I am working on Datastage 8.0

chulett · Post by **chulett** » Thu May 21, 2009 7:46 am

Just an aside - the EE installs the MKS Toolkit on your Windows server so you do in fact have the ability to run a 'UNIX' script. FYI.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Thu May 21, 2009 8:29 am

What is the size of the input column? Or is it unbounded ?

It it from seq file or some other source?

Who provides it to you?

If you know the max length, you can split the rows in a transformer and gather with a funnel.

Seq file can be resolved using awk script in source.

Others will need buildOp.

in_finity307 · Post by **in_finity307** » Fri May 22, 2009 1:56 am

Can someone give the Build-op code for this logic?

The length of the input column is not known. Its a free text column.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Fri May 22, 2009 2:37 am

Have you tried awk script?

in_finity307 · Post by **in_finity307** » Fri May 22, 2009 3:09 am

No, the data is coming from a database.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Fri May 22, 2009 3:15 am

Then surely it must have a max length.

in_finity307 · Post by **in_finity307** » Fri May 22, 2009 3:27 am

The maximum length in the database is defined as 2000

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Fri May 22, 2009 4:35 am

Then join with a row generator to get a cartesian product.

This can be used in substring later.