DataStage Job Failure

[RajVenugopalan]

DataStage job fails randomly after certain row limit with an error mentioning "error reading the source file : Invalid argument". If I restart the job again it fails at a different row that is greater than the one during the previous run with the same error message.

Here is a sample error message that I receive from the job.

Jb_ELIGIBILITY_CCS_GHPP..Trn1: At row 61963853, link "Elig_IN"
Error reading D:/DHSDATA-FTP\ELIG_DF\ELIG.D201005: Invalid argument

When I try to see the range of records from the source file using the Start and End Record # in CFF stage, I could view those records without any issues. And visually I don't see any errors in the record. The source file has about 100 million records.

This job has a cff stage as a source (EBCDIC file), goes through a transformer and loads a Teradata table (using Multiload).

This job has been running fine in 7.5 and 8.1. But for the past 3 months we are having this issue.

Things we did trying to resolve this issue:
1. Restarted the ETL server - This helped once. But the next time even after the restart the job failed.
2. Contacted IBM Tech support - They were not able to duplicate the issue, since we could not provide the source file has PHI (protected health information) and hence could not resolve the issue.
3. Tried to bring the checkpoint in Teradata Stage from 500,000 to 100,000. - This again worked one time, the next time the job failed with the same error message.

I would appreciate if anyone can suggest any modifications to the job or let me know if anyone else faced the same issue.

Thanks
Raj

[chulett]

Build a job to redact/mask the PHI elements and create a new file. Verify new file has the same issue. Send new file to IBM Support.