DSXchange

Thanks for weighing in, Ernie. File sizes: good job (no change in performance), bad job (fourth pair is where the volume increase performance change occurred). 33K 76K 65K 150K 45K 101K 451K 1000K 690K 1560K For giggles, I checked the XMLInput stage's Input/Advanced tab and found: Buffer mode -- def...

I'll try to be as brief as possible, so do expect me to respond to requests for more details. Both jobs have the same general design: XML input, transformers to create mainframe-format rows including headers and trailers, a final funnel and FTP to mainframe. The input formats are different, but foll...

eostic wrote:...mostly because the processing time for parsing the information is probably not where your bottleneck is.

Ernie

I'm about to post an XML performance problem, and it looks like parsing is where our bottleneck is.

There's only one safe way to do this. In Designer copy each sequence and job, rename the copies to what you want (_T, or whatever), compile the jobs and elevate them up to your test and production regions. Once you are satisfied that they work properly, delete the "old" versions. I've lear...

Ravi.K wrote:Use the below thread for XML file reading at Parallel Jobs.

viewtopic.php?t=138188&highlight=XML

That was the exact thread I found and used to help me get started using XML input.

Without knowing the details, the following criti... um, commentary should be taken with a grain of salt. Number one: Using a single dataset to contain variable-length/blocked records with multiple data definitions (schemas, copybooks) is insane. It's lazy design to avoid making the coders do standar...

Remember: I'm using 7x, not 8x, so if 8x is different I'll be wrong... That said, may I state the obvious? The sequential file stage(s) that read(s) the file(s) will simply ignore (as in not pass through the output link) the first record. It doesn't matter what's in that record. The attribute in thi...

I'm using 7x, and not yet familiar with differences with 8x, but here's how I'd do it:

This is a delimiter problem, and the root cause is the rules (or lack of them) at the data source. If the main record delimiter is a comma, parse into last-name and first-middle. If there will always be a single space between the first name and middle name (full or initial), reparse first-middle int...

I'm using PX, not Server. That could be a difference.

I compile all code for QA using the multi-job compiler tool in Manager.

I hope that helps.

Looks like you have a major mystery there, Martina. :( I can only speculate on the possible issues. At this point, I agree with Craig that you would be better served to open a problem ticket. In the meantime, I suggest reviewing the environment settings by doing a complete dump of them from Administ...

Martina, The "executable" code is all text-based except for those stages that go through the C compile (like transformer). You see the result of that in binary code sections in the executable dsx. Please note: you don't change the output format. The dsx is the same whether for design, exec...

Martina, I suggest that you investigate the executable code, not the Designer code. For example: Compile and execute new code in QA (reverify success), then export the QA executable. Compile and execute the new code in Production (reverify failure), then export the Production executable. A compare o...

Point taken, Craig. Permit me a moment of cynicism, though: in the ETL world, where does DataStage stand? Is it at risk of losing market share, or would it remain competitive without what the competitors have? If the reality is the latter, I don't see them being motivated to get started. Before I me...

The Server product has always had an interactive debugger, thought the same would make it into the PX side of the house eventually but for whatever reason it hasn't happened yet. :( Considering the functionality DS offers, I was honestly surprised that PX did not have interactive debugging. Debug s...