Limit the entry in the log file

vij · Post by **vij** » Tue Feb 27, 2007 4:28 am

Hi all,

I am loading millions of records into a SQL SERVER table. For each of the rejected record, I get an entry in the log and the log gets filled and creates problems, so I wanted to restrict the number of entries written to the log. Can you please tell me how to restrict this?

Also, I am using a dataset to load the rejeceted records, but rejected records are not captured into the DataSet. When I try to view the records in the DataSet, it says, a new column called, "sqlcode" is present in the dataset which is not present in the table and yes a new column is there.

Can any one clear this?
thanks!

ArndW · Post by **ArndW** » Tue Feb 27, 2007 4:36 am

The only correct way is to get rid of the warning message. An alternate way is to demote or ignore the warning message. This can be done with 7.5 PX DataStage for PX jobs. See the directory tab Tools -> Message Handler Environment or look into the DataStage Manager Guide, 14-1 "Ma naging Message Handlers"

kumar_s · Post by **kumar_s** » Tue Feb 27, 2007 4:59 am

And for the second, turn off the RCP.

DSguru2B · Post by **DSguru2B** » Tue Feb 27, 2007 9:41 am

sqlcode has the code from the database for the reason of rejection. You can limit the warnings by setting the number of warnings during run time. Similar to 'dsjob -warn' option through command line.

kumar_s · Post by **kumar_s** » Tue Feb 27, 2007 5:38 pm

What DSguru2B was referring is to stop the job after 'n' warning. Which is similar to DSJ_LIMITWARN option available in DSSetJobLimit function.

swades · Post by **swades** » Tue Feb 27, 2007 5:59 pm

You can drop 'sqlcode' column using Modify Stage or Copy Stage ( which is better)

Thanks

kumar_s · Post by **kumar_s** » Tue Feb 27, 2007 6:03 pm

Sqlcode is unseen column, which will be propagated only if RCP is enabled.

ray.wurlod · Post by **ray.wurlod** » Tue Feb 27, 2007 7:45 pm

Not correct. The sqlcode column appears by virtue of the fact that it's on a reject link from a database stage type.

kumar_s · Post by **kumar_s** » Tue Feb 27, 2007 8:10 pm

My mistake

I was under the impression, that the sqlcode is propagated only when RPC is turned on. But its not the case.
As mentioned, Copy stage can be used as an intermediate stage to drop the column. By the way, if its reject table, adding Sqlcode as a part of metadata will add more information on what reason, the records got rejected.