Column Import Issue

[pravin1581]

Hi All,

We are facing an issue with the column import stage. The input file has one column which contains the 36 rows , where the data is pipe delimited.
Through the column import stage it is getting divided into 2 output cols based on the pipe delimiter. The output file should contain 36 rows but only 9 rows are getting populated in the output file, we are using 2 node config file, changing the partiiton type to Entire in the Column Import Stage, changes the row count to 18 but the records are getting repeated,the same 9 records are getting repeated twice.

Thanks in advance.

[Minhajuddin]

Column import stage doesn't need any keyed partitioning because it doesn't work on any keys. So, you don't even need "Entire Partitioning". And if Entire partitioning is used, then you'll get one record repeated as many times as there are nodes in your config file.

So, now about your problem. You said you want 36 rows in the output, Now, how many rows do you have in the input? Column import stage is not supposed to change the number of rows.

[pravin1581]

Column import stage doesn't need any keyed partitioning because it doesn't work on any keys. So, you don't even need "Entire Partitioning". And if Entire partitioning is used, then you'll get one record repeated as many times as there are nodes in your config file.

So, now about your problem. You said you want 36 rows in the output, Now, how many rows do you have in the input? Column import stage is not supposed to change the number of rows.

Thanks for the reply, there are 36 rows in the input.

[pravin1581]

Any suggestions.

[OddJob]

Have you checked Director log to see if any input records are being rejected? Try introducing a reject link on the Column Import stage and forcing rejected output to this link (to a peek stage) by setting 'Reject Mode' to 'Output'.

[Minhajuddin]

Do any of the rows have a NULL value in the column which you are splitting?