I have looked around on google and this forum, as well as the documentation, and experimenting for myself, and not found a solution. If there is no DataStage native solution, I would like confirmation that that is the case. I do not want to see a command-line scripting solution to this problem.
It seems like the Datastage parallel job file input step cannot handle general Excel CSV files. For instance, the below.
30-40 ABC+C,TV,MXN,0.08026,160701,,,SPACE,"20""",,,Cable / Satellite,,"97,253.20","7,805.13",,0,"97,253.20","7,805.13",2,1/6/14,1/12/14,January,"24,313.30","24,313.30","1,951.28","1,951.28",2014
Delimiter for field "Burst_Spend_NNMC_USD" not found; input: {2 0 1 4}, at offset: 368
There are commas embedded in the fields, and also double quotes. Setting the file as double-quoted will fail on the embedded quotes, setting it as not double-quoted will fail on the embedded commas. How can I read a line like this in correctly?
CSV file with embedded quotes and commas
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 39
- Joined: Tue Apr 15, 2014 9:14 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Had you searched more diligently you would have found that the parallel Sequential File stage is really bad at this whereas the server Sequential File stage is brilliant with these and other aberrations (such as missing columns).
Therefore best advice is to use a server Sequential File stage, either in a server job or in a server Shared Container in a parallel job.
Therefore best advice is to use a server Sequential File stage, either in a server job or in a server Shared Container in a parallel job.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: