External Filter Output Column question

[Bob Kendall]

I've written a unix C program to be used as an external filter. The columns it returns are different than the columns fed to it (and many more rows are produced than are supplied to the filter).

When I run the DS job, the external filter is called three times and, based on debug msgs I've written to stderr that show up in the director log, I'm getting the output I want. But the file output stage that follows the filter is failing because it can't find one of the columns I supplied as input to the filter. I'm not outputting this column and it isn't in the output column list for the filter and the Runtime Column Propogation check box for the filter stage is not checked.

On a side note, the column being complained about is also one that I didn't want in my input to the filter, but DS insisted on supplying it. I finally gave up and accepted in on input, even though I don't use it.

Does anyone have any idea why DS is upset about the lack of a column that isn't in the output list from the filter?

Thanks,
Bob Kendall

[I_Server_Whale]

hello Bob,
Are you using the "generated query" or "user defined query" if the source is a database table. I think it the meta-data problem. Check your meta-data defined for that table and the query if it is a db table.

Let me know!

Naveen.

[Bob Kendall]

The data is all flat file, no database involved. I defined the meta data from COBOL copy books.

Thanks,
Bob

[ray.wurlod]

There's not really enough information to diagnose this one. I presume you're using an External Filter stage. Can you post the exact error or warning message that was generated?

DataStage will normally insist on supplying a column because that column appears in the Columns grid. Have you tried removing it from there?

Do you have runtime column propagation enabled (which can generate columns apparently from thin air)? If so, try disabling it.