eostic wrote:I haven't used the RowMerger as much as the RowSplitter, but it might be worth looking at the history....these stages were not designed to perform any meaningful manipulation of the row (emphasis added) --- but to dynamically re-assign metadata (in memory) to an existing row that is coming down-stream. If it doesn't do partial row merging, it wasn't short-sighted -- it simply wasn't the goal of the Stage ---- these were designed to provide an in-memory solution to a technique that is often used --- writing a file to disk on an input link so that it could be read via output link with entirely alternative metadata (yes this is the technique I'm currently using since I can't get Row Merger to do what I want), and written as part of the distribution of RTI because the Web Services architecture prevents a job from having a passive stage mid-stream dropping data to disk.
That being said, a partial merge would be useful....
...on the actual topic, I'm not entirely understanding your requirement. Couldn't you just re-order the columns as necessary in a Transformer immediately before the RowMerger, and then use RowMerger as designed? (No because the RowMerger includes 1) the key fields and 2) "insignificant variables" in the merged column. If I then need to parse out the undesired variables, at this point I might as well roll my own concatenation code)
Ernie
Hi Ernie,
Re: the origin of the stage design, I still find it shortsighted that I can't select the columns to merge. The DS developers would still have met their design goal (as you stated it above) by making the stage more flexible.
Re: the context of what I'm trying to do, see:
viewtopic.php?t=125895&highlight=server%20merge, scroll down to vmcburney's post
viewtopic.php?t=125989&highlight=
viewtopic.php?p=320343#320343
In summary, I'm trying to build a concatenated list of variables, delimited by some delimiter, with nulls converted to empty, to feed to CRC32. This is all to implement Changed Data Capture (CDC) and SCD Type 2. My input data does not have the keys all at the front of the line, and the variables of interest are non-contiguous.
And perhaps all of this goes away if we upgrade to 8.x and have the SCD stage, but that's moot at present.
Thanks...