Retrieve from Unix file

yxie · Post by **yxie** » Tue Jul 22, 2008 6:04 pm

I am trying to retrieve data from unix file through Sequential File Stage, the layout of file is like following

1....2 ......3 ........165......200....257
-------------------------------------------
1; 235; 9090;.... 0000;
2; 353; 8849;.... 3666;... .... ....7780;
3; 536; 4672;.... 5745;... 8383;

it has about 257 columns with ; delimited, my problem is that only those records with data in all 257 fields will show up, which means in above example record 1 and 3 won't be retrieved, even I set each column nullable in table definition. The server is Linux redhat.

Please give any adv how to retrieve the entile data set
Thanks a lot

ray.wurlod · Post by **ray.wurlod** » Tue Jul 22, 2008 6:10 pm

Pre-process the file so that each line has 256 ";" characters. You could even do this in DataStage; read the line as a single VarChar field, add the appropriate number of semi-colons

Code: Select all

InField : Str(";",256-Count(InField,";"))

then re-parse using Column Import stage.

yxie · Post by **yxie** » Wed Jul 23, 2008 9:16 am

Thanks Ray, Could you tell me where should I put this code to, Sequential file stage?

ray.wurlod · Post by **ray.wurlod** » Wed Jul 23, 2008 3:12 pm

Transformer stage downstream of the Sequential File stage.