Compress Stage making Timestamp field invalid.

[sridharvis]

Hi,
I am using Compress & Expand Stage for Datasets, since we have do not have much space in server.
But ini using that when we compress a Dataset using Compress Stage and then in the subsequent job when we use that compressed Dataset followed by Expand Stage. Then in the output from Expand Dataset we are facing a issue that , the 1st field with TimeStamp as DataType is getting corrupted and coming as '************' asteriks. We say that only the 1st TimeStamp field because other Timestamp field are coming out fine.

1st Job :

Dataset1-----> Compress-------->Dataset2

2nd Job :

Dataset2-----> Compress-------->Dataset3

We had tested this using both the GZIP & COMPRESS utility that comes with Compress & Expand Stage. But the issue still persist.

E,g:
Source Dataset1:
Col1,Col2,Col3,Col4,Col5,COl6
-----------------------------------------------------------------------
Deb,123,2009-08-08 03:06:04,Seal,2005-01-01 02:56:54,India

Target Dataset3:
Col1,Col2,Col3,Col4,Col5,COl6
-----------------------------------------------------------------------
Deb,123,************,Seal,2005-01-01 02:56:54,India


Anyone who has already faced this issue?

[chulett]

So, Job 2 actually has an Expand stage in it, yes? Your "picture" shows Compress for both, so best to just go back and edit that post to correct it.

Any idea what the actual 'corrupted' timestamp value is? Perhaps add a link to a Peek stage to find out. And double-check that your metadata for that field is in fact correct in both jobs.

[ArndW]

In order for a timestamp to be compressed, it needs to be converted to a string. Likewise, the string value is decompressed and then put into a timestamp format. I would wager that the problem here is the default string formatting of the timestamp field; since the "*"s you see usually come when a format cannot be parsed.
Is the date format used always the same, perhaps your first column definitions are different?