Datastage converting field to scientific notation

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Nicole
Premium Member
Premium Member
Posts: 93
Joined: Mon Oct 22, 2007 12:55 pm
Location: Albany
Contact:

Datastage converting field to scientific notation

Post by Nicole »

My job contains a counter - the counter was up to 1450955

Once it hit the following number it converted it to the following number in what I believe is scientific notation. Is there a setting or option that would stop DS from doing this?

1.45096e+06
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

What's the Precision setting? What's the data type?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Nicole
Premium Member
Premium Member
Posts: 93
Joined: Mon Oct 22, 2007 12:55 pm
Location: Albany
Contact:

Post by Nicole »

Hi Ray -

The datatype is varchar.

The field originates as a stage variable - which I increment by one for every record. The precision is set to 255 in the stage variable properties.

Just to give you all the details I can - another step is padding the 7 character field with 0's

Str('0', 7-Len(COUNTER)):COUNTER

From what i've told you, do you see anything?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

I'm surprised that incrementing a VarChar works at all - after all there's no implicit conversion from string to any kind of numeric data type. Is the stage variable also VarChar?

Are you using a stage type (such as Aggregator) that generates dfloat by default?

Try defining this field as int32 (or uint32 or int64 or uint64) throughout. Integers are never represented using scientific notation.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Nicole
Premium Member
Premium Member
Posts: 93
Joined: Mon Oct 22, 2007 12:55 pm
Location: Albany
Contact:

Post by Nicole »

I double checked and the COUNTER variable is definitely a VARCHAR. Is it possible it did an implicit conversion when I add 1 to it? 'COUNTER + 1' is done in the derivation of the stage variable area.
Then another derivation is done. Str('0', 7-Len(COUNTER)):COUNTER

Is it possible that in any of these derivations is it converted to a numeric? The data below is what the job did to the following data. Obviously, the last one is what gave me the issue. I'm going to create a copy of the job and modify it and see if I change this behavior. It can't be that hard to figure out hopefully.

0999998
0999999
001e+06
Nicole
Premium Member
Premium Member
Posts: 93
Joined: Mon Oct 22, 2007 12:55 pm
Location: Albany
Contact:

Post by Nicole »

Ok, after trying a couple different things - I decided to change the counter to an integer - just to see what happened - and of course, that worked!

I don't fully understand what went wrong, but after pointing out that it shouldn't be adding 1 to a varchar field, I thought i'd try it as an int. Everything still worked ok, so I think this was the solution.

Thanks for talking me through it!

Nicole
Post Reply