DSXchange

The job score doesn't show you what is happening within the transformer performing the conversion, only the input and outputs. So while the scores may be identical, the operation could be considerably different. The outstanding questions are: 1) Is the source data guaranteed to be 100% valid and in ...

You need to more clearly define what the end results you need are for certain input conditions: 1) You have an input record which has no match for either REF 1 or REF 2. 2) You have an input record which has a match for REF 1 but not on REF 2. 3) You have an input record which has a match for REF 2 ...

Has the client provided a method for identifying the new rows other than comparing the data row-to-row? Are there any unique keys available?

Regards,

That would be the $APT_IMPORT_PATTERN_USES_FILESET environment variable.

Regards,

A custom operator would fit the bill. The Custom Operator Reference document will give you a lot of information. Here is the version for IS 8.7. You can also Google/Bing/whatever for it for other releases.

Regards,

Remember to mark the topic as resolved!

Regards,

Been able to do that as long as I've been doing parallel (7.0)

Explain what you mean by "handle first row". What do you think you need to do differently with it? Is your requirement to start with a particular value for the generated key, or just to generate unique values? If the former, you could modify the Initial Value logic in my earlier post: svKe...

To avoid any repartition. No need. The engine will only insert a partition in front of a stage which would require it (sorts, joins, merges, lookups and other key-requiring stages), if it feels that it's required. Won't happen with a transformer except in certain situations (such as if it's followi...

svKey Initial value: @PARTITIONNUM svKey derivation: if keyChange=1 then svKey+@NUMPARTITIONS else svKey Assuming you have properly partitioned and sorted your data, each combination of Key1 and Key2 values will be given a unique generated key value. Optionally, you could just use the surrogate key ...

Does your job contain stages running in parallel mode after the transformer? If so, DataStage partitioning the data,most likely with RoundRobin paritioning. If you do nothing to control the order the records are written to the output, the order is at the mercy of when a block of records arrives at t...

Review the following presentation regarding how to perform that migration. It details the steps you need to take.

Relocating the XMETA repository for Information Server 8.0.1 and 8.1

Regards,

What's the purpose of the join stage? What will it provide? Based upon your examples, I think a simple key-change-based calculation using stage variables within a transformer is all that's necessary: Input--->Sort--->Transformer--->Output Sort: Partition on Account, Sort on account/Installment#, gen...

Have you examined the unzipped data from the larger (>4GB) files to verify that the data is in the expected format?

Perhaps the larger files contain multiple smaller files which are also zipped?

Regards,

The method Ray mentioned (updating value files) is the proper and best way to accomplish this task when using parameter sets to pass your parameter values. You can certainly use whatever method you desire to create or modify a value file dynamically (shell script/awk/sed/perl/etc.), then pass that v...