To read all data from an CSV file and then populate into xml

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ujala
Participant
Posts: 45
Joined: Mon Jun 21, 2010 2:51 pm
Location: Chennai

To read all data from an CSV file and then populate into xml

Post by ujala »

Hello all,
I have an requirement where an csv file data should be read such that initial few columns will be fixed and the rest of the columns might be added or removed and any column data may contain comma(,) into it.

So i need to get the following:
1. Fixed columns in one link and
2. The columns which might be added or removed into other link with different delimiter retaining the comma in any of the column values.

For Ex:
column in file is Id,N,N1,N2,N3

col 'Id' and 'N' will be fixed and col 'N1','N2' can be 'N1' or 'N1','N2','N3'...

Input data:
Source col- Id,N,N1,N2
source val- 1,a,"b,d",c

Required Target data to be as
first link Data - Id,N
1,a
other link data - NewCol
b,d/c
In the above expression delimiter comma(,) is replaced by / but comma(,) inside the b,d is retained because it is a single column value.

Please get my any solution for the above stated problem.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

This is easily handled using a server Sequential File stage, which has (among other things) a "Missing Column" rule for each column.

You can use this stage in a parallel job be encasing it in a server Shared Container.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ujala
Participant
Posts: 45
Joined: Mon Jun 21, 2010 2:51 pm
Location: Chennai

Post by ujala »

It would be of great help to me if you could uncheck the premium content and we are informed not to use any server jobs.

It would also help if i can get an unix script with support pattern matching and replace only the comma in the string and not the delimiter.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

ujala wrote:It would be of great help to me if you could uncheck the premium content.
That's just not going to happen. Premium content is the mechanism by which DSXchange is funded. Were it not for the premium content fees, DSXchange could not exist.
Premium membership is not expensive - less than 30c (Rs12 approx) per day. And well worth the investment.
ujala wrote:...we are informed not to use any server jobs.
So use a server Shared Container in the parallel job as I advised.
And resist stupid requirements.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ujala
Participant
Posts: 45
Joined: Mon Jun 21, 2010 2:51 pm
Location: Chennai

Post by ujala »

Well thanks a lot for your reply.

I will plan to become an premium member asap.

And i can't use server jobs as the client is not going to allow us to deploy server jobs in their instance.

However i got the solution using awk script.

Thanks again for all your replies.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Note that I did not suggest using a server job.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply