DSXchange

Wow, how did you do that!? It's as though you're looking over my shoulder. Yes, I had missed that the input column was Unicode and the dimension column wasn't. But correcting this oversight still didn't fix the problem. Stranger still, it still shouldn't find a match when running on 1 node. I'm gett...

Thanks for the response Craig, but it didn't fix things. The Score now shows: . . . ds2: {op1[4p] (parallel copyPlaceholder4) eAny=>eCollectAny op2[4p] (parallel copyPlaceholder4:RF02Before-FILTERED)} ds3: {op1[4p] (parallel copyPlaceholder4) eAny=>eCollectAny op7[4p] (parallel APT_TransformOperator...

Problem: - When rerunning the job with no changes to the source data, the SCD stage identifies all rows as new inserts. - This problem does not occur when running on a single node. SCD configuration: - Input partitioning tab: The dimension link partitioning=Auto; the input link partition=Hash on BA...

As stated, our workaround was to split the transformer into 2 stages. However, we encountered another problem, to do with the last 6 Text stage variables, which was resolved by replacing the "setnull()" with a text string (e.g. "NULL FLAG"), and in the output derivation use: If s...

Thanks Eric. We're also using AIX, so the octal code will do very nicely.

Yes, we did something like this, using a Unix script. Basically, as James suggested, the script loops through a set of dsjob commands. e.g. 1. Build a file with a record for each of the parameters to be used for the dsjob command. 2. Loop for each of the records. 2a. Build & submit the dsjob com...

Requirement: We need to read a text file that has a record for each of the tables to be loaded. The file can have comment lines, which begin with a #. e.g. #Batch Code, Source File, Schema File, Target Table, Key Field(s) #PRS tables PRS,PRS-EMPLOYEE-*-EXTRACT.DAT,EmployeeFile.schema,EMPLOYEE,EMP_I...

If it were a cartesian join then we'd have 500,000,000 rows. :) Yes, thanks qt_ky. SURA made that point too. The 20K was just some test data I had lying around. It is also indicative of the numbers we can expect in production, so you're right that in this case, Clear ing the partitioning will have n...

Thanks Ray. Yes, we have a SMP environment. Further investigation showed the the job that failed was appending to a dataset that had "Preserve Partition" set to true. i.e. orchadmin lp MetricsObsolete.ds 2>/dev/null | grep "Preserve" returns Preserve Partitioning: true So a bette...

Problem: If we run the batchflow with a different number of nodes, some jobs fail with the message: There are irreconcilable constraints on the number of partitions of an operator: parallel copyPlaceholder05. The number of partitions is already constrained to 2, but an eSame partitioned input virtu...

Good idea. Thanks James.

Requirement: We have a job that reads a dataset, makes the required changes, and overwites the dataset. To do this, we write to a temporary dataset, and after the job completes successfully, we "rename" the temporary dataset to the proper dataset name (using orchadmin copy ). Problem: We ...

Well, I do not know the cause of the 262276 error, but I recreated the DB2 Connector stage (i.e. deleted and re-added the stage), and now the bulk load works fine. So my guess is it was something hanging around from when the stage was defined with a write mode of "Insert". The job now runs...

Thanks Michael. It's good to know we're on the right track. Now we just need to get the bulk load working! [Edit: the job fails with the 262276 message before writing the message file] Since we saved so much time by dropping indexes, we plan to use the "do not update table indexes" option,...

Requirement: We are trying to improve the performance of a simple load job: Sequential File -> Transformer (adding some metadata columns) -> Db2Connector The job was taking ~15 minutes to load 4,117,660 rows (the average rowsize is 244 bytes). The job was run using 2 nodes, which looks like it runs...