Modify Stage Issue

rajudx · Post by **rajudx** » Tue Apr 17, 2012 4:31 am

Hi,

In modify stage we have used one column for dropping before moving to comparison and while running job giving below error message.

DROP chgdate

Error when checking operator: Could not find input field.

Input table having above mentioned column.

Please let me know what is the issue with modify stage on above mentioned syntax.

Thanks&Regards,
Raju

jwiles · Post by **jwiles** » Tue Apr 17, 2012 8:32 am

There is not an issue with the modify stage--the issue is that the field chgdate does not exist within the virtual record when it enters the modify stage.

You state that the input table has the column:
1) Does the job read that column from the table?
2) If so, what happens to the data between when the table is read and the modify stage? That is, what processing occurs? Sorts? Transformers? Something else?

Regards,

rajudx · Post by **rajudx** » Tue Apr 17, 2012 10:21 am

We are reading metadata dynamically from ODBC stage and in job modify stage already drooping one column successfully.
When I tired with any new column from table then job is failing with Error "when checking operator: Could not find input field"
The column which job able to process successfully that is first column in table and other columns are failing with above mentioned error.

ODBC------>MODFIY----->column export stage

Thanks,
Raju

jwiles · Post by **jwiles** » Tue Apr 17, 2012 12:20 pm

Still, the issue is straightforward: The columns you are attempting to drop in the modify do not exist.

Are you performing a table read in the ODBC stage, or a SELECT * FROM tablename query? Runtime column propagation is enabled on the output of ODBC?

You can add the $OSH_PRINT_SCHEMAS environment variable to your and set it to True. This will show the columns being output by the ODBC stage. From there you can verify whether or not the columns you believe should be there (the ones you are trying to drop) actually are there.

Regards,

rajudx · Post by **rajudx** » Wed Apr 18, 2012 4:22 am

We are using read method on ODBC stage Read method as "Table".
Issue is if the column will not be specified as drop column in modify stage then datastage job able to complete successfully and not sure that time how the column passing modify stage and going into "column export stage" .if still column not coming from ODBC stage job should fail when running without drop statement.RCP has been enabled in all ODBC stages.

Thanks,
Raju.

jwiles · Post by **jwiles** » Wed Apr 18, 2012 7:56 am

Have you done everything you can to validate that the column name in your DROP statement exactly matches (ignoring case) the column name being output from the source ODBC stage? If Modify can't find the column by the name you specified, then it simply isn't there. Could one or more tables have the column name misspelled perhaps? It's been known to happen...

if still column not coming from ODBC stage job should fail when running without drop statement

What failure do you expect in this case?

Regards,

priyadarshikunal · Post by **priyadarshikunal** » Wed Apr 18, 2012 9:25 am

jwiles wrote:You can add the $OSH_PRINT_SCHEMAS environment variable to your and set it to True. This will show the columns being output by the ODBC stage. From there you can verify whether or not the columns you believe should be there (the ones you are trying to drop) actually are there.

Did you perform this step?? This should give you a clue what you have been missing.

rahcomp · Post by **rahcomp** » Wed Apr 18, 2012 2:46 pm

Could you make sure if the input column passed to the modify is not nullable.

jwiles · Post by **jwiles** » Wed Apr 18, 2012 2:59 pm

Could you make sure if the input column passed to the modify is not nullable.

Irrelevant to the problem being encountered.

Regards,

rajudx · Post by **rajudx** » Fri Apr 20, 2012 5:30 am

Thanks for all for provding suggestion on this.

All column are defined as "Not Null" on database and also I have double checked columns name which mentioned in modify stage and those are same which are there on database.
I have requested to admin team set $OSH_PRINT_SCHEMAS parameter to "true" and once it will be update then I will check ODBC log column information.
My only confusion thing is if ODBC not sending the column information then Job should be fail when running job without adding column name in modify stage.

Thanks,
Raju.

jwiles · Post by **jwiles** » Fri Apr 20, 2012 7:53 am

My only confusion thing is if ODBC not sending the column information then Job should be fail when running job without adding column name in modify stage.

Again, why do you think there should be a failure? If a column is not in the output from the ODBC stage, and the modify statement does not attempt to drop a column that does not exist (this is how I interpret your statement), why would it fail?

Are you unable to add $OSH_PRINT_SCHEMAS to the job directly? You really don't want or need to set it at the project level. It can add a lot of extra output to the job logs....

Regards,

SURA · Post by **SURA** » Sun Apr 22, 2012 5:23 pm

Create the copy of the job and use the peek stage and check whether you are getting that column in the output.

DSXchange

Modify Stage Issue

Modify Stage Issue

Re: Modify Stage Issue