sequential file read running if the wrong delimeter

suresh_dsx · Post by **suresh_dsx** » Thu Apr 07, 2011 6:37 am

Source is the comma delimited file.
Sample data:
"Emp name","Empno","sal"
"xxx","101","2000"
"bbb","102","4000"
"aaa","103","3000
I have given the below properties in the sequential file stage.

Record level

final delimter=end
Record delimiter=UNIX newline

Field defaults

Dlimeter=Comma
Quote=double

I am able to run the job. It is working. Some reason they have provided wrong delimiter in the file.

Sample data:
"Emp name" "Empno" "sal"
"xxx" "101" "2000"
"bbb" "102" "4000"
"aaa" "103" "3000"

even though i am having different delimeter in the source file. Still my job running fine.
But I want to fail my job if the file having wrong delimiter other than comma delimiter. It is not failing and running fine.

Any help greatly appreciated.

Thanks

GJ_Stage · Post by **GJ_Stage** » Thu Apr 07, 2011 8:01 am

Hi,

It looks default parameter is space , that is the reason it is accepting. If you want to abort/fail job then you need to check the file after the second quote shoud be comma.

ray.wurlod · Post by **ray.wurlod** » Thu Apr 07, 2011 3:24 pm

My guess is that it would read the entire line into the first field, if there are no commas in the line but your metadata prescribe comma as the field delimiter.

suresh_dsx · Post by **suresh_dsx** » Thu Apr 07, 2011 11:22 pm

We have specified the below options in the sequential file stage.

Record level

final delimter=end
Record delimiter=UNIX newline

Field defaults

Dlimeter=Comma
Quote=double

Actual process expected:
i am sure i have to get the file with comma delimeter, so that i have used comma as field delimeter in the sequential file stage.

when i run the job with space delimeted file, it is running fine. as per the requirement it should get fail.

Additional information: when i view the data from sequential file stage, i am able to view the data for all the fields not as a single column.

Any help greatly appriciated.
Thanks

GJ_Stage · Post by **GJ_Stage** » Fri Apr 08, 2011 2:46 am

As I told earlier in my reply - defauly delimiter is space So it will work fine without fail. If you wanted to fail you need to do explicit check in transforamtion stage.

GJ_Stage · Post by **GJ_Stage** » Fri Apr 08, 2011 4:32 am

What I am trying to say is , Before you read the file with delimiter - Read each line as one record and check if anything otherthan comma after the second quote then fail the job.
I have tested the same (file without comma) and it is loading so I put the above condition and fail the job.

chulett · Post by **chulett** » Fri Apr 08, 2011 6:54 am

suresh_dsx wrote:Finally, The sequential fine having any delimiter (space, comma, pipe), it is working fine.

That just seems... odd to me and not any kind of behaviour I recall seeing in the past. I would have said the same thing Ray did.

suresh_dsx also wrote:So can I treat this is a bug in data stage V8.1 version. Do I need to ask IBM for patch or am I need to add any options in the sequential file stage to control.

Only way to know is to ask, let us know what you find out.