DSXchange

Hi all,

I am loading millions of records into a SQL SERVER table. For each of the rejected record, I get an entry in the log and the log gets filled and creates problems, so I wanted to restrict the number of entries written to the log. Can you please tell me how to restrict this?

Also, I am using a dataset to load the rejeceted records, but rejected records are not captured into the DataSet. When I try to view the records in the DataSet, it says, a new column called, "sqlcode" is present in the dataset which is not present in the table and yes a new column is there.

Can any one clear this?
thanks!

The only correct way is to get rid of the warning message. An alternate way is to demote or ignore the warning message. This can be done with 7.5 PX DataStage for PX jobs. See the directory tab Tools -> Message Handler Environment or look into the DataStage Manager Guide, 14-1 "Ma naging Message Handlers"

And for the second, turn off the RCP.

sqlcode has the code from the database for the reason of rejection. You can limit the warnings by setting the number of warnings during run time. Similar to 'dsjob -warn' option through command line.

What DSguru2B was referring is to stop the job after 'n' warning. Which is similar to DSJ_LIMITWARN option available in DSSetJobLimit function.

You can drop 'sqlcode' column using Modify Stage or Copy Stage ( which is better)

Thanks

Sqlcode is unseen column, which will be propagated only if RCP is enabled.

Not correct. The sqlcode column appears by virtue of the fact that it's on a reject link from a database stage type.

My mistake I was under the impression, that the sqlcode is propagated only when RPC is turned on. But its not the case.
As mentioned, Copy stage can be used as an intermediate stage to drop the column. By the way, if its reject table, adding Sqlcode as a part of metadata will add more information on what reason, the records got rejected.