Sequential file stage with FILE PATTERN option

[rohanf]

Hi,

I have some 80 to 90 Datastage jobs. Each jobs creates a set of error files all with the same file structure across all 90 jobs. All these files created in seperate jobs are created in a single folder but with a common file pattern say "*_Seq_Err.txt".

When I run a job that loads these error files into a table I use the file pattern functionality of the sequential file stage.
I get below warnings(To be treated as failures as each error record is crucial to log):
---------------------------------------------------------------------
seqf_ERROR_FILE_PATTERN,0: Field "SOURCE_KEY_VALUE" has import error and no default value; data: {O B P}, at offset: 58
seqf_ERROR_FILE_PATTERN,0: Import warning at record 2578051.
seqf_ERROR_FILE_PATTERN,0: Import unsuccessful at record 2578051.

seqf_ERROR_FILE_PATTERN,0: Bad trailing quote character at field "CREATE_DATE_TIME"; looking for '34' but found '.', at offset: 389
seqf_ERROR_FILE_PATTERN,0: Import warning at record 17666949.
seqf_ERROR_FILE_PATTERN,0: Import unsuccessful at record 17666949.

seqf_ERROR_FILE_PATTERN,0: Field "CREATE_DATE_TIME" with 'delim=end' did not consume entire input, at offset: 265
seqf_ERROR_FILE_PATTERN,0: Import warning at record 28277572.
seqf_ERROR_FILE_PATTERN,0: Import unsuccessful at record 28277572.
--------------------------------------------------------------------------

However out of 200 error files created using different jobs, I have no clue from these warnings that which error files are causing these warnings.

I certainly hope there is an easier way than to open each of 200 error file and check the File structure.

Thanks in advance,
Rohan

[rohanf]

Eagerly awaiting response

Thanks in Advance
Rohan

[mikegohl]

Did you consider using the reject link to capture the import errors?

[dsedi]

Hi,

I have some 80 to 90 Datastage jobs. Each jobs creates a set of error files all with the same file structure across all 90 jobs.

If you are very sure that your file structure is same across all Jobs then the warning issue is from every file...
so check what datatypes you have in your table and what is the data you are trying to import.

Thanks.

[rohanf]

Thanks,

dsedi,
Not all files have the wrong pattern. Only reason why some of the files have different structure was because the jobs were created by different developers.

Michael,
I have used a reject link to Sequential file stage as suggested by you and am able to identify the files causeing the error.

I am marking this post as resolved.

Thanks again
Rohan