DSXchange

Posted: **Thu May 28, 2015 3:02 am**

I am trying to create a generic job to remove duplicates based on a key column and a timestamp column.

The key column can potentially be any numeric type, so I am creating another column in a string format and importing the key column as well as retaining the original key column

The modify statement is thus

Code: Select all

CLE_JOB :String= #pKeyColumn#
#pKeycolumn#=#pKeyColumn#

However, I get the following error

Code: Select all

modify(1),0: Fatal Error: Invalid Julian day

This is odd as I only am modifying a number not a date/timestamp

Code: Select all

  CLE_DECIBEL: int64;

There are other fields in the incoming schema which are timestamps with microseconds, but this should not make any difference as they are not impacted by the modify.

What do I need to do to get this through the modify ?

Posted: **Thu May 28, 2015 3:51 am**

Can you please post all the specifications in your Modify stage?

Also, please verify that the metadata for any column mentioned in a KEEP specification is identical on input and output link?

Posted: **Thu May 28, 2015 3:54 am**

A specification that specifies that the output is a string must also specify the size of that string (for Char data type) or the maximum size of the string (for VarChar data type).

Posted: **Thu May 28, 2015 8:22 am**

This issue was resolved. It was actually an error in a sort stage which would not allow a timestamp with microseconds as a sort key.

Converted it to a string in transformer and then it worked.

DSXchange

Invalid Julian Day with Modify

Invalid Julian Day with Modify