Hi All
My sample file qtest.csv in unix has two lines ( though it is CSV it really is delimited by a |)
"arvind"|"tes"ting"
"newtest"|"testing"
I am trying to read it via a sequential file stage in a PX job
The record delimiter is set to be a Unix new line
Delimiter is set to |
Quote is set to double
When i view the data in the sequential stage i only can see as follows
arvind|tes
newtest|testing
In other words How can i specify to datastage that the field is enclosed in double quotes ( and can contain a double quote as part of the data ) and fields are delimited by |
I want to avoid manually massaging the data outsite to replace " in the data with some other character/s
Am I missing something basic here.
Has someone else ran into this problem before
arvind
Double quotes in data
Moderators: chulett, rschirm, roy
The stage is behaving exactly how you asked it to behave. The very first double quote it finds, after the initial quote, is where the data ends. I don't see any other way, other than massaging the file prehand. Or set the quote character to none and handle it within the job.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Participant
- Posts: 55
- Joined: Tue Sep 20, 2005 10:58 am
Thanks for your suggestions
Hi
Thanks for your responses
Actually in our situation since we have | as a delimiter and the fact that we can get pipe's in our data means, I cannot set the quotes to none
The quotes attribute set to double allows us to have pipes in the data
But it looks like i have to do some transformation of the datafile before it hits datastage to handle our situation or change the delimiter to something other than pipe
Thanks again for your help
consider this post as closed
arvind
Thanks for your responses
Actually in our situation since we have | as a delimiter and the fact that we can get pipe's in our data means, I cannot set the quotes to none
The quotes attribute set to double allows us to have pipes in the data
But it looks like i have to do some transformation of the datafile before it hits datastage to handle our situation or change the delimiter to something other than pipe
Thanks again for your help
consider this post as closed
arvind
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Just ran into the same situation, although the embedded double quote is escaped by another double quote (which by the way, is CSV standard).
http://en.wikipedia.org/wiki/Comma-separated_values
DSEE does not seem to be able to handle an escaped, double-quote situation like
"1234","1XYEU74GB6UA51754","1599","Great car, New 18"" Chrome Wheels!","60606"
If you say quoted none, then you have to ensure you have no embedded commas in text fields (which is a long-shot).
http://en.wikipedia.org/wiki/Comma-separated_values
DSEE does not seem to be able to handle an escaped, double-quote situation like
"1234","1XYEU74GB6UA51754","1599","Great car, New 18"" Chrome Wheels!","60606"
If you say quoted none, then you have to ensure you have no embedded commas in text fields (which is a long-shot).