DSXchange

I have a job where the data from the funnel stage goes to a sort stage. The job executes fine. The link from the funnel stage to the sort stage shows a row count which means data has moved. However, data does not go through the sort stage onwards. This same problem happens when I replace the sort stage with a copy or a transformer. However, if I directly write this data to a sequential file, the file DOES get the data. I also turned off RCP.

Does anybody have an idea?

Check if you try to push a nullable column to non nullable column. this happens often. It happened to me, it shows as writing and but nothing is written.

so my suggestion is to check the null values.

All the incoming and outgoing columns to/from the funnel stage are non-nullable. This obviously means that the incoming columns to the sort stage are non-nullable as well.

Hi,

What is your director log showing. Does it show any warnings of records being dropped because of nullability.

The metadata might be not null for all the columns from one stage to another but the value for one of the columns coming in can be null for which you have to do null handling.

HTH
--Rich

Very strange! I removed all the stages after the funnel. Then I added a sequential file stage as a direct output of the funnel. I then added all the previous stages one by one between the funnel and the sequential file stage. It worked!