Sequential File read problem

mydsworld · Post by **mydsworld** » Mon Apr 30, 2007 12:18 am

Hi,

While trying to use a Sequential file stage (.CSV file) as input with delimiter 'comma' and Quote 'none', I am facing the following problem :

For records like :

XYZ,GHT,UJY - the stage reads the input data properly ( three fields with values XYZ,GHT,UJY respectively)

However some of the records are appearing in the file like this :

"""XYZ,GHT,UJY""","""IJY""" - In this case data is should be read as two fields 'XYZ,GHT,UJY' and 'IJY'

But it is reading them into four fields XYZ,GHT,UJY and IJY respectively.

Please advise.

ray.wurlod · Post by **ray.wurlod** » Mon Apr 30, 2007 1:06 am

The data are being parsed correctly according to your metadata. You specify quote=none, so the quote characters are treated as ordinary characters. You now have four comma-delimited fields. That's what you've described to DataStage.

If your data are coming in in inconsistent format, harrass your data provider. Or pre-process the data, so that there are no delimiter characters in the data. Otherwise read the entire line as a single string and parse it yourself. You will probably need a routine (or maybe Column Import stage) at some point.

mydsworld · Post by **mydsworld** » Mon Apr 30, 2007 2:08 am

But to read data in two fields 'XYZ,GHT,UJY' and 'IJY' from input data stream """XYZ,GHT,UJY""","""IJY""" (that also contain data like XYZ,GHT,UJY ) what should be the delimiter and quote values.

ray.wurlod · Post by **ray.wurlod** » Mon Apr 30, 2007 5:21 am

If you import the data rows as single strings (VarChar) the delimiter and quote characters are irrelevant. Each should be set to none.

DSguru2B · Post by **DSguru2B** » Mon Apr 30, 2007 7:48 am

If your data is quoted then provide the quote character. This will pick up the data as you want. Since it is a csv file. With quote character set to none, each comma will be treated as a selimiter.