Deriving value for a new column using a Modify stage
Posted: Tue Oct 12, 2004 12:55 am
Hi All,
I need some help on the usage of modify stage. Let me first explain the scenario.
We have two fields in the input say F1 and F2. We want a third field F3 in the output(in addition to F1 and F2) whose value is decided as follows :
F3 = F1 if F1 is not null and
F3 = NULL if F1 is null.
The above operation can be done in a transformer but we want to avoid using a transformer in the job.
So we were trying to use a combination of filter and modify stages. We are filtering the null records and using a modify stage for the filtered and the rejected records(from the filter stage)we are adding the third field F3. Finally using a funnel stage, we are collecting the records into a sequential file.
The modify stage has the following specification :
KEEP F1,F2
SetNull(F3:string)
When we are trying to do a validate run for this job, it fails. Can somebody guide me on how to use SetNull function to assign field F3 a value of null? But first, can we use a modify stage to define a new column, which is not present in the input, and derive a value for it?
Thanks in Advance.
I need some help on the usage of modify stage. Let me first explain the scenario.
We have two fields in the input say F1 and F2. We want a third field F3 in the output(in addition to F1 and F2) whose value is decided as follows :
F3 = F1 if F1 is not null and
F3 = NULL if F1 is null.
The above operation can be done in a transformer but we want to avoid using a transformer in the job.
So we were trying to use a combination of filter and modify stages. We are filtering the null records and using a modify stage for the filtered and the rejected records(from the filter stage)we are adding the third field F3. Finally using a funnel stage, we are collecting the records into a sequential file.
The modify stage has the following specification :
KEEP F1,F2
SetNull(F3:string)
When we are trying to do a validate run for this job, it fails. Can somebody guide me on how to use SetNull function to assign field F3 a value of null? But first, can we use a modify stage to define a new column, which is not present in the input, and derive a value for it?
Thanks in Advance.