Remove "," from a file to read each field

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
harshada
Premium Member
Premium Member
Posts: 92
Joined: Tue May 29, 2007 8:40 am

Remove "," from a file to read each field

Post by harshada »

Hi,

I have a file which has fields seperated by ","(double quotes comma double quotes.

So data is like
"LAA","1644/10","+Jul","","","SURNAME-","FORENAME","1 FLOATED ONE "

I want to read each record in a single column - col1 and then cut the fields according to "," and do some validations on them. I am not reading it as a normal file as the data to be validated can come in any field like field 3 or field 4 . This 3 or 4 i am getting from a reference table.

I tried performing

Field(INP_3.col1, ('","'), INP_3.amend_flag_pos) where INP_3.amend_flag_pos is the position of the amend flag coming in from a reference table which is currently 3. So my output should be--------- +Jul
But my output is coming as --------------- ,"+Jul(comma double quote +Jul).

All the columns I cut using the field command has a ," before the actual data. Can anyone help me get the clean data.

Thanks
Harshada
Roopanwita
Participant
Posts: 125
Joined: Mon Sep 11, 2006 4:22 am
Location: India

Post by Roopanwita »

It is better to read different columns in sequential file only...Then after that in the transfomer you can validate the records.. :)

sequential file ---> FOrmat-->Field Defaults ---> Quote..

Thanks, :wink:
harshada
Premium Member
Premium Member
Posts: 92
Joined: Tue May 29, 2007 8:40 am

Post by harshada »

I cannot do that as I also have to count the number of fields in each record, so I am reading it in a single column. Also there are many fields , I cannot perform all the validations on all the fields tats why I have to first cut the field (e.g 3rd field) and then perform validations on it.
mystuff
Premium Member
Premium Member
Posts: 200
Joined: Wed Apr 11, 2007 2:06 pm

Post by mystuff »

Try to trim out the double quotes first and then use comma as the field separator
ZDC
Premium Member
Premium Member
Posts: 15
Joined: Fri Nov 23, 2007 5:06 am

Post by ZDC »

It looks like your record will have minimum 4 cols ..
If so you can read your file using following schema in seq file stage
Col1
Col2
Col3
Col4
RemCol
(?)
Zero Degree Consulting
Post Reply