I've written a unix C program to be used as an external filter. The columns it returns are different than the columns fed to it (and many more rows are produced than are supplied to the filter).
When I run the DS job, the external filter is called three times and, based on debug msgs I've written to stderr that show up in the director log, I'm getting the output I want. But the file output stage that follows the filter is failing because it can't find one of the columns I supplied as input to the filter. I'm not outputting this column and it isn't in the output column list for the filter and the Runtime Column Propogation check box for the filter stage is not checked.
On a side note, the column being complained about is also one that I didn't want in my input to the filter, but DS insisted on supplying it. I finally gave up and accepted in on input, even though I don't use it.
Does anyone have any idea why DS is upset about the lack of a column that isn't in the output list from the filter?
Thanks,
Bob Kendall
External Filter Output Column question
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 6
- Joined: Wed Jul 13, 2005 5:11 pm
- Location: El Dorado Hills, CA
-
- Premium Member
- Posts: 1255
- Joined: Wed Feb 02, 2005 11:54 am
- Location: United States of America
-
- Participant
- Posts: 6
- Joined: Wed Jul 13, 2005 5:11 pm
- Location: El Dorado Hills, CA
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There's not really enough information to diagnose this one. I presume you're using an External Filter stage. Can you post the exact error or warning message that was generated?
DataStage will normally insist on supplying a column because that column appears in the Columns grid. Have you tried removing it from there?
Do you have runtime column propagation enabled (which can generate columns apparently from thin air)? If so, try disabling it.
DataStage will normally insist on supplying a column because that column appears in the Columns grid. Have you tried removing it from there?
Do you have runtime column propagation enabled (which can generate columns apparently from thin air)? If so, try disabling it.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.