Read decimal directly from csv

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Joshi
Premium Member
Premium Member
Posts: 17
Joined: Mon Aug 18, 2003 11:59 pm
Location: Germany

Read decimal directly from csv

Post by Joshi »

Hello,

I am trying to read a decimal column from csv as decimal. The values have a format like this: From -1,23 up to -123.456.789,12

I defined the decimal separator and tried various decimal data type definitions (length, scale) but I don't get the "view data" to run. I get the error:

Field xxx has import error and no default value; data: {- 2 7 1 . 7 8 7 , 9 4}, at offset: 37

Any idea?

Thanks in advance,

Joshi
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The difficulty here is that the input value has both a decimal separator (the comma), and a thousands-separator (the period). The input definition for the decimal data type only allows for specifying the decimal separator and any values with both will unfortunately not be parsed.

You will need to read this value in as a string and then parse it within DataStage inside a transform stage as I don't know of any practicable way to do it in the source sequential stage.

If your input data has no "." periods apart from those used as thousands-separator then you could add a filter to remove them in your input sequential file stage and then could set the decimal separator to "," and it would read in your data correctly.
Post Reply