DSXchange

ArndW wrote:What is the datatype of the EFFECTIVE_TIME and TO_TIME columns? Can you convert these explicitly to timestamp before the filter stage?

DataStage timestamp. All three are DS timestamp.

Hi folks, Peek stage shows this data: RECORDED_TIME:2008-09-23 11:36:00 EFFECTIVE_TIME:2008-09-20 03:14:00 TO_TIME:2008-09-20 14:10:00 Filter stage with this where clause: RECORDED_TIME BETWEEN EFFECTIVE_TIME AND TO_TIME or with this where clause: RECORDED_TIME >= EFFECTIVE_TIME AND RECORDED_TIME < ...

If you select the option "multiple rows returned from link" it will work like a join stage. If you did not select that option the lookup stage will drop duplicate records based on key which in your case RefKey Oh duh. I see that. I had the drop down blank. It seems to be working now. than...

Hi guys, I am trying this idea of doing a lookup on the key, and then using a filter to apply the date logic, but I don't think the lookup is working. If the reference returns more than one row, does it operate like a join, or does it ignore the duplicates? My rowcount going in is the same as the ro...

kris007 wrote:And feed the output link of the LookUp Stage to a Copy Stage or Sequential file stage with path /dev/null

Does this encourage garbage collection?

Thanks for the clarification. I'm submitting a ticket to see what IBM has to say. In the meantime I think I'll try your idea of using the lookup to match the keys. Up until now, I've always had unique rows in the lookup, so I guess you're saying multiple rows from the reference link just lead to mul...

You can join just the StreamKey and RefKey and bring all the columns downstream into a Transformer or Filter Stage and then apply the logic StreamDate is between RefDate1 and RefDate2 That should work fine. Hi thanks. Joining the StreamKey and the RefKey requires sorting and repartitioning the stre...

Hi folks, there are many range lookup problems posted here, so I realize I am adding to a pile, but I haven't seen this particular issue resolved so I am posting it before submitting a ticket. I have a Varchar Key and a date on the stream: StreamKey StreamDate I have the same key, and a pair of date...

Hi folks, in a transformer derivation I find that this works fine: TimestampToString(lnk_lkp_ip_flowsheet_rows.RECORDED_TIME,"%yyyy-%mm-%dd %hh:%nn:%ss") But this: TimestampToString(lnk_lkp_ip_flowsheet_rows.RECORDED_TIME,"%hh:%nn:%ss") throws the following: APT_CombinedOperatorC...

Hi folks, increasing the warning limit in Director fixed the problem. Thanks to kris007 and chulett for that answer.

Thanks,
Doug

Is JobMonApp running or disabled? Are you collecting performance statistics when running from Director? Not sure about the former. I'm not sure how to check this. For the latter, I do not have the jobs set to record performance data on the job properties execution tab. Regarding the problem startin...

Awesome. Thanks Ray. I suppose creating jobs that don't abort is an uncontroversial "best practice"

Disable RCP to see whether that resolves the issue. If it does, try to discover why the additional, unspecified columns might have raised an interrupt signal. Hi, RCP is not enabled. I'm not sure that these warnings have anything to do with it. Again, the jobs run just fine when launched from Desig...

Is this a Job Sequence or just a parallel job? Are you getting any warnings before you get the SIGINT error. Can you post any warnings from director log? Couple of reasons why SIGINT error occurs is when someone tries to stop the job or if the number of warnings exceed the limit mentioned in the Jo...

anbu wrote:Search the forum with the keyword "SIGINT"

Maybe someone has a better answer?

Thanks,
Doug