Reading of csv file using Seq File Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
krisp321
Participant
Posts: 39
Joined: Thu Mar 04, 2010 12:39 am

Reading of csv file using Seq File Stage

Post by krisp321 »

Hi I want to read .csv file data using seq. file stage, but I have an issue here

Data is as follows (presume all are varchar datatypes)

100,200,300,A,"34,00","64,000","0,23"
200,200,500,B,"22,000","11,0"12,00"

Here is the issue, because the values within double quotes contains commas, it is treating as delimiter value, though it is single value


Please suggest me how to handle this
Final Delimiter=end

Field Level
Delimiter=comma
Quote=double


Thanks in advance
anbu
Premium Member
Premium Member
Posts: 596
Joined: Sat Feb 18, 2006 2:25 am
Location: india

Post by anbu »

If you define Quote as double, then you are fine. Were you able to run your successfully?
You are the creator of your destiny - Swami Vivekananda
krisp321
Participant
Posts: 39
Joined: Thu Mar 04, 2010 12:39 am

Post by krisp321 »

anbu wrote:If you define Quote as double, then you are fine. Were you able to run your successfully?
Hi
Quote=double also I have done it before which I already mentioned in my first post.

Its not going to work, because within quotes , comma is there, which is getting treated as delimiter and my values is getting truncated

Looking for some better solution
anbu
Premium Member
Premium Member
Posts: 596
Joined: Sat Feb 18, 2006 2:25 am
Location: india

Post by anbu »

Your quote setting should work. Truncation problem is due to your length defined for that field. Increase your length, then you should be able to read the data.
You are the creator of your destiny - Swami Vivekananda
krisp321
Participant
Posts: 39
Joined: Thu Mar 04, 2010 12:39 am

Post by krisp321 »

anbu wrote:Your quote setting should work. Truncation problem is due to your length defined for that field. Increase your length, then you should be able to read the data.
Hi
thanks for ur reply. its not the problem with length, its varchar with no length specification.

You can check from ur end placing the same data in my first posting
it wont work if there is comma within double quotes

Really appreciate if someone can give an affective solution.
I am not expecting simple new datastage level solutions
krisp321
Participant
Posts: 39
Joined: Thu Mar 04, 2010 12:39 am

Re: Reading of csv file using Seq File Stage

Post by krisp321 »

Please read my posting clearly before suggestion
I am saying comma within quotes is getting treated as delimiter which should not..THATS MY ONLY CONCERN
PLEASE DONT SUGGEST, LENGHT PROBLEM OR SOME TRUNCATION PROBLEM.
krisp321 wrote:Hi I want to read .csv file data using seq. file stage, but I have an issue here

Data is as follows (presume all are varchar datatypes)

100,200,300,A,"34,00","64,000","0,23"
200,200,500,B,"22,000","11,0"12,00"

Here is the issue, because the values within double quotes contains commas, it is treating as delimiter value, though it is single value


Please suggest me how to handle this
Final Delimiter=end

Field Level
Delimiter=comma
Quote=double


Thanks in advance
anbu
Premium Member
Premium Member
Posts: 596
Joined: Sat Feb 18, 2006 2:25 am
Location: india

Post by anbu »

I just noticed problem in your data. Correct your data
100,200,300,A,"34,00","64,000","0,23"
200,200,500,B,"22,000","11,0","12,00"
You are the creator of your destiny - Swami Vivekananda
nitkuar
Participant
Posts: 46
Joined: Mon Jun 23, 2008 3:09 am

Re: Reading of csv file using Seq File Stage

Post by nitkuar »

krisp321 wrote:Please read my posting clearly before suggestion
I am saying comma within quotes is getting treated as delimiter which should not..THATS MY ONLY CONCERN
PLEASE DONT SUGGEST, LENGHT PROBLEM OR SOME TRUNCATION PROBLEM.
this is not the way of asking for favour... this is a professional forum, please think before you post something this. :shock:
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

See if a server Sequential File stage - either in a server job or in a server shared container - can read this file properly. You can use a server shared container in a parallel job.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
krisp321
Participant
Posts: 39
Joined: Thu Mar 04, 2010 12:39 am

Post by krisp321 »

anbu wrote:I just noticed problem in your data. Correct your data
100,200,300,A,"34,00","64,000","0,23"
200,200,500,B,"22,000","11,0","12,00"
Sorry, thats my typing error, data in file is correct with double quotes and having a comma in it

Thanks
krisp321
Participant
Posts: 39
Joined: Thu Mar 04, 2010 12:39 am

Post by krisp321 »

ray.wurlod wrote:See if a server Sequential File stage - either in a server job or in a server shared container - can read this file properly. You can use a server shared container in a parallel job. ...
Excellent ray, it worked. Really appreciate that.
Thanks
Post Reply