Can we have mulitple rejects pointing to same physical file

[akash_nitj]

Hi Techno's
Is it possible to have a single physical file for the mulitple reject links in a Parallel Job. If yes any limitations on its use ?

TIA
Akash

[ArndW]

That would work if you just push your errors out to a database table and make sure you choose a unique key. Sequential files are 1WnR and I don't think that Datasets are any help there. You could use a parallel merge stage for distinct error flows within the job as well.

[T42]

Yes. You would need to be using some form of funnel/join/merge/whatever solution to collect all the reject links, and output the results to a sequential file.

[memrinal]

If you have several Sequential File Stages writing into the same file in the same job, then that may lead to completely inconsistent result depeding on the partitioning selected by you.
So its better to colletc all the rejects into a funnel and then write to the file.

[mujeebur]

You can implement this using Switch with multiple if else clauses and out put the records which ever you dont want to process.

[rsrikant]

Hi Akash,

It is very much possible to have a single sequential file for multiple rejects using funnel.

But make sure that all the reject links have save same metadata and collect them in a funnel and pass them to the sequential file.

To identify the reject link from which the row is rejected (this is sometimes difficult since all the reject links have the same metadata) you can add a col generator stage between the reject link and the funnel. Add one column in the metadata (say 'RejectLink') and hard code the value using field properties of the column. Do this for each reject link.

So, after running the job if any rows are rejected, you can view the data in the sequential file and from the value in 'RejectLink' column you can identify from which link that record came.

Regards,
Srikanth Reddy