Changes to sequences ignored after import and compile

centennium · Post by **centennium** » Fri Jul 22, 2011 2:40 am

Hi all,

We have a Development, QA and Production environment. Changes to datastage jobs and sequences are done in development, then exported/imported to QA, tested, then again exported/imported into Production. This has been running fine for years. Yesterdays change did not work well, though. We have changed some things in some of our sequences, tested, exported, imported into Production and Compiled. Now when running the sequence, the changes are ignored and the jobs and sequences in the moved sequence that was in production before the move runs. I mean: it seems like datastage ignores the changes and still runs the old sequence, as if no import of the new design has been done.

Anyone experienced this?

Thanks in Advance,

Martina

senthilmp · Post by **senthilmp** » Fri Jul 22, 2011 2:51 am

Seems like the sequence is corrupted, try deleting the old sequence and then move it as if you are moving newly

centennium · Post by **centennium** » Fri Jul 22, 2011 3:13 am

Thanks Senthil

We already tried that, but it did not help.

Regards,

Martina

centennium · Post by **centennium** » Sat Jul 23, 2011 11:52 am

Update:
Even when deleting the sequence and recreating it directly in our production environment, the changes are not recognized, but the old sequence runs. Our conclusion is that changes to sequences are ignored when the name of the sequence is the same. Only sequences with a new name run as expected.

Can anyone point me in the right direction of what we can try to solve this issue?

Thanks in advance,

Martina

chulett · Post by **chulett** » Sat Jul 23, 2011 1:51 pm

Have you looked at the Sequence jobs in Production to see if your actual changes made it there? How exactly are you exporting/importing your jobs? A 'normal' import from the GUI automatically deletes the original job before importing the replacement so it doesn't surprise me that doing that manually before the import had no impact.

centennium · Post by **centennium** » Sun Jul 24, 2011 3:00 am

Hi Craig,

Yes, the changes are there in production. The new jobs as well as the sequences with the new jobs in them. But after compiling and running, the sequence runs as if the new jobs have not been added to the sequence. We're using Manager for export/import actions.

Unfortunately cannot read your post after that, although my company pays for my premium membership, the 'Premium content button' shows in your answer.

Thanks and Regards,

Martina

chulett · Post by **chulett** » Sun Jul 24, 2011 6:23 am

That would mean your membership has expired and the date is listed with your member information on the 'Home' page. For your rather odd issue - have you gotten your official support provider involved yet?

centennium · Post by **centennium** » Sun Jul 24, 2011 12:04 pm

Hi Craig,

Yes, I have now seen it, membership expired about 10 days ago. Will fix that shortly. We have not gotten hold of official support. Do you know if the environment being virtualized can cause this behaviour? Or should we try a reindex all?

Thanks in Advance,

Martina

chulett · Post by **chulett** » Sun Jul 24, 2011 2:59 pm

I honestly have no idea what would cause the behaviour you are reporting. Reindexing can't hurt but I'd be surprised if it changes the behaviour... but then I've been surprised before.

FranklinE · Post by **FranklinE** » Mon Jul 25, 2011 2:47 pm

Martina,

I suggest that you investigate the executable code, not the Designer code.

For example: Compile and execute new code in QA (reverify success), then export the QA executable. Compile and execute the new code in Production (reverify failure), then export the Production executable. A compare of QA (works) and Production (doesn't work) executables may point you to the problem. This compare may not work for the binary code.

A respectful commentary on your shop's policies: We here never import Designer code to Production. We import only executables from QA. I'm convinced that this is the best approach to maintaining the integrity of the Production environment. You might consider that as an alternative to your current practice, at least in getting the correct code into Production.

I hasten to add that our practice doesn't prevent corruption of code. In my experience it does make it easier to find the corrupted code. Good luck.

centennium · Post by **centennium** » Tue Jul 26, 2011 9:36 am

Hello Franklin,

Thank you for your answer. Can you help me a couple of steps further? We normally export the design to a .dsx file and then import it to the next environment. Now we have exported the executable into a .dsx. However, how should we compare the executable codes? Can the .dsx files be opened in a text editor or something? First I tried to export to xml, however the option to export the executable is greyed out when export target is xml.....

One more question: you say you only export and import the executable codes. Does that mean you cannot see the changes in the design when you open those jobs and sequences in designer? Or don't you open jobs and sequences in Designer in your production environment at all?

Thanks in advance,

Martina

FranklinE · Post by **FranklinE** » Tue Jul 26, 2011 10:28 am

Martina,

The "executable" code is all text-based except for those stages that go through the C compile (like transformer). You see the result of that in binary code sections in the executable dsx. Please note: you don't change the output format. The dsx is the same whether for design, executable or both.

Those binary sections make compares difficult. Depending on the size of the file, I recommend deleting the binary code sections before trying to do compares. I've used Windows and Unix file compare utilities with good results, but only after removing the binary code. (And don't try to reimport the edited dsx files.

)

You are correct, we don't see design code in production, and I don't mind telling you it sometimes makes tracking problems difficult. The benefit, which I find a good trade, is that the executable code I run in QA is identical to the code I run in production, and problems are generally either system environment (configuration, etc.) or insufficient test coverage in QA.

The most efficient method to export executables: In Manager, open the Jobs tree and click/highlight any folder that contains jobs and/or sequences. The Export menu will have an Executables choice, clicking on that will open a new window with a checkbox tree display of your project. You can check entire folders or individual jobs, and there's a counter on the right that tells you how many your picked. Please note: If you don't have your project organized in folders and sub-folders, you won't get much advantage using this.

Code: Select all

Jobs
    Project folder
        Sub-project/sub-section folder
            Jobs/sequencers

My current project has seven sub-sections. I keep two main folders, one for jobs and one for sequencers, with section folders under each one. I recently had a design change that affected all the sequencers, so exporting executables was just one click.

centennium · Post by **centennium** » Wed Jul 27, 2011 2:38 am

Hi Franklin,

Even by looking at the size of the .dsx files containing the executables from QA and PROD, we can tell the PROD one has not been updated because of the size of the PROD file alone.

Our next try will be to export both design and executable from QA and import that into PROD, then see if our change is being picked up.

Do you know of any reason why updating the executable in PROD by compiling the imported Design all of a sudden would not work anymore?

Thanks in Advance,

Martina

chulett · Post by **chulett** » Wed Jul 27, 2011 7:19 am

As I've said before, any reason for this would come from your official support provider. Open a case, get them involved.

FranklinE · Post by **FranklinE** » Wed Jul 27, 2011 7:40 am

Looks like you have a major mystery there, Martina.

I can only speculate on the possible issues. At this point, I agree with Craig that you would be better served to open a problem ticket.

In the meantime, I suggest reviewing the environment settings by doing a complete dump of them from Administrator (there's a dsadmin command line you can use, page 7-157 in the Parallel Job Advanced guide) and compare for differences between QA and Production. A significant difference in the executable sizes could point to compiler setting differences, though I don't know enough about the details to suggest what they might be.

My best guess is that you have environment/configuration issues. Only your support provider can determine that.