DSXchange

Ok, so you only what columns you require once the job has started running. In that case you will need to pivot your data, whilst at the same time adding column which contains your column names. This can be achieved in a variety of ways: 1. Use a peek stage with an output link. Set the peek to return...

So you know before you run the job which columns are required for reference? In that case I would just use a modify stage and set a KEEP specification in which you specify your column names to keep. You would pass these column names via a parameter which you populate as the job starts. OR You might ...

You could also leverage "round robin" as a partitioning method if you wanted the split to be across all nodes, be it three or whatever. I was going to suggest this - but the OP seemed to want the data in 3 distinct links rather than 3 nodes - this is why I am asking why the data is being ...

Of course there is a stage - the transformer stage. There is also a switch stage - but that would also use the same criteria as a transformer stage (I think). Then again - I cannot think of a reason why you would want to split your input rows without some sort of criteria - for me, splitting of rows...

You could try following the suggestion here:
http://www-01.ibm.com/support/docview.w ... wg21662699

Are you using multiple nodes to process? If you just want to split them evenly then I would use the Mod function on the Input Row number in a transformer stage variable. You can then equate the constraint on each output link accordingly. eg set stage variable svOutputLink as Mod(@INROWNUM,3) Set out...

Do you have any criteria for splitting your records, or do you just want them split equally?

What is the target after the split? Are you processing each stream of data separately afterwards? Ie what are you trying to achieve?

Couple of questions: Does each input row potentially have a different set of reference column requirements? If so - how do you determine from your input row which columns need to be referenced? Ie in your example you suggest that your input row requires an output of only cols A,B and E? Does your jo...

You don't have to have Transformer2 set to RCP - I was just pointing out that is probably why it wasn't compiling when you tried to surface column B. So you have two options 1. Either surface column B using a copy/modify stage prior to the Transformer2, then set the transformer to RCP off and output...

So if I understand correctly from your Shared Container output you have "surfaced" the columns A,C,D? But B remains hidden. Transformer2 would then have columns A,C,D specifically surfaced also? It sounds as though you have RCP turned off in your Transformer2 (hence why it won't compile). ...

Not sure how to convert in SQL first. Use case when? You would just put the syntax around the field that you wish to convert, so your select statement would become (and add the column alias afterwards else datastage tends to get confused): Select key, code, to_number(replace(value,',','')) value fr...

Is the purpose of the job to only get the last row of an input file?

If so, just use the unix cmd tail -1 on the filter in the sequential file stage.

If your source is oracle you could convert it to a number prior to output?

eg use the following in your select clause for the given column where INPUT is your varchar input value.

That syntax should work for any of the values that you provided.

Is it possible that your sequence is generating the same invocation id for multiple jobs?

I think what @rkashyap is getting at is that you might need to add an extra columns to you outputs that can be used to determine sort order. This could be as simple as the key column AND some sort of counter. Then you can set your funnel to sort the data. After the funnel you would drop the unrequir...