Changes to sequences ignored after import and compile
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 15
- Joined: Thu Mar 16, 2006 4:11 am
- Contact:
Changes to sequences ignored after import and compile
Hi all,
We have a Development, QA and Production environment. Changes to datastage jobs and sequences are done in development, then exported/imported to QA, tested, then again exported/imported into Production. This has been running fine for years. Yesterdays change did not work well, though. We have changed some things in some of our sequences, tested, exported, imported into Production and Compiled. Now when running the sequence, the changes are ignored and the jobs and sequences in the moved sequence that was in production before the move runs. I mean: it seems like datastage ignores the changes and still runs the old sequence, as if no import of the new design has been done.
Anyone experienced this?
Thanks in Advance,
Martina
We have a Development, QA and Production environment. Changes to datastage jobs and sequences are done in development, then exported/imported to QA, tested, then again exported/imported into Production. This has been running fine for years. Yesterdays change did not work well, though. We have changed some things in some of our sequences, tested, exported, imported into Production and Compiled. Now when running the sequence, the changes are ignored and the jobs and sequences in the moved sequence that was in production before the move runs. I mean: it seems like datastage ignores the changes and still runs the old sequence, as if no import of the new design has been done.
Anyone experienced this?
Thanks in Advance,
Martina
-
- Premium Member
- Posts: 15
- Joined: Thu Mar 16, 2006 4:11 am
- Contact:
-
- Premium Member
- Posts: 15
- Joined: Thu Mar 16, 2006 4:11 am
- Contact:
Update:
Even when deleting the sequence and recreating it directly in our production environment, the changes are not recognized, but the old sequence runs. Our conclusion is that changes to sequences are ignored when the name of the sequence is the same. Only sequences with a new name run as expected.
Can anyone point me in the right direction of what we can try to solve this issue?
Thanks in advance,
Martina
Even when deleting the sequence and recreating it directly in our production environment, the changes are not recognized, but the old sequence runs. Our conclusion is that changes to sequences are ignored when the name of the sequence is the same. Only sequences with a new name run as expected.
Can anyone point me in the right direction of what we can try to solve this issue?
Thanks in advance,
Martina
Have you looked at the Sequence jobs in Production to see if your actual changes made it there? How exactly are you exporting/importing your jobs? A 'normal' import from the GUI automatically deletes the original job before importing the replacement so it doesn't surprise me that doing that manually before the import had no impact.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 15
- Joined: Thu Mar 16, 2006 4:11 am
- Contact:
Hi Craig,
Yes, the changes are there in production. The new jobs as well as the sequences with the new jobs in them. But after compiling and running, the sequence runs as if the new jobs have not been added to the sequence. We're using Manager for export/import actions.
Unfortunately cannot read your post after that, although my company pays for my premium membership, the 'Premium content button' shows in your answer.
Thanks and Regards,
Martina
Yes, the changes are there in production. The new jobs as well as the sequences with the new jobs in them. But after compiling and running, the sequence runs as if the new jobs have not been added to the sequence. We're using Manager for export/import actions.
Unfortunately cannot read your post after that, although my company pays for my premium membership, the 'Premium content button' shows in your answer.
Thanks and Regards,
Martina
-
- Premium Member
- Posts: 15
- Joined: Thu Mar 16, 2006 4:11 am
- Contact:
Martina,
I suggest that you investigate the executable code, not the Designer code.
For example: Compile and execute new code in QA (reverify success), then export the QA executable. Compile and execute the new code in Production (reverify failure), then export the Production executable. A compare of QA (works) and Production (doesn't work) executables may point you to the problem. This compare may not work for the binary code.
A respectful commentary on your shop's policies: We here never import Designer code to Production. We import only executables from QA. I'm convinced that this is the best approach to maintaining the integrity of the Production environment. You might consider that as an alternative to your current practice, at least in getting the correct code into Production.
I hasten to add that our practice doesn't prevent corruption of code. In my experience it does make it easier to find the corrupted code. Good luck.
I suggest that you investigate the executable code, not the Designer code.
For example: Compile and execute new code in QA (reverify success), then export the QA executable. Compile and execute the new code in Production (reverify failure), then export the Production executable. A compare of QA (works) and Production (doesn't work) executables may point you to the problem. This compare may not work for the binary code.
A respectful commentary on your shop's policies: We here never import Designer code to Production. We import only executables from QA. I'm convinced that this is the best approach to maintaining the integrity of the Production environment. You might consider that as an alternative to your current practice, at least in getting the correct code into Production.
I hasten to add that our practice doesn't prevent corruption of code. In my experience it does make it easier to find the corrupted code. Good luck.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
-
- Premium Member
- Posts: 15
- Joined: Thu Mar 16, 2006 4:11 am
- Contact:
Hello Franklin,
Thank you for your answer. Can you help me a couple of steps further? We normally export the design to a .dsx file and then import it to the next environment. Now we have exported the executable into a .dsx. However, how should we compare the executable codes? Can the .dsx files be opened in a text editor or something? First I tried to export to xml, however the option to export the executable is greyed out when export target is xml.....
One more question: you say you only export and import the executable codes. Does that mean you cannot see the changes in the design when you open those jobs and sequences in designer? Or don't you open jobs and sequences in Designer in your production environment at all?
Thanks in advance,
Martina
Thank you for your answer. Can you help me a couple of steps further? We normally export the design to a .dsx file and then import it to the next environment. Now we have exported the executable into a .dsx. However, how should we compare the executable codes? Can the .dsx files be opened in a text editor or something? First I tried to export to xml, however the option to export the executable is greyed out when export target is xml.....
One more question: you say you only export and import the executable codes. Does that mean you cannot see the changes in the design when you open those jobs and sequences in designer? Or don't you open jobs and sequences in Designer in your production environment at all?
Thanks in advance,
Martina
Martina,
The "executable" code is all text-based except for those stages that go through the C compile (like transformer). You see the result of that in binary code sections in the executable dsx. Please note: you don't change the output format. The dsx is the same whether for design, executable or both.
Those binary sections make compares difficult. Depending on the size of the file, I recommend deleting the binary code sections before trying to do compares. I've used Windows and Unix file compare utilities with good results, but only after removing the binary code. (And don't try to reimport the edited dsx files. )
You are correct, we don't see design code in production, and I don't mind telling you it sometimes makes tracking problems difficult. The benefit, which I find a good trade, is that the executable code I run in QA is identical to the code I run in production, and problems are generally either system environment (configuration, etc.) or insufficient test coverage in QA.
The most efficient method to export executables: In Manager, open the Jobs tree and click/highlight any folder that contains jobs and/or sequences. The Export menu will have an Executables choice, clicking on that will open a new window with a checkbox tree display of your project. You can check entire folders or individual jobs, and there's a counter on the right that tells you how many your picked. Please note: If you don't have your project organized in folders and sub-folders, you won't get much advantage using this.
My current project has seven sub-sections. I keep two main folders, one for jobs and one for sequencers, with section folders under each one. I recently had a design change that affected all the sequencers, so exporting executables was just one click.
The "executable" code is all text-based except for those stages that go through the C compile (like transformer). You see the result of that in binary code sections in the executable dsx. Please note: you don't change the output format. The dsx is the same whether for design, executable or both.
Those binary sections make compares difficult. Depending on the size of the file, I recommend deleting the binary code sections before trying to do compares. I've used Windows and Unix file compare utilities with good results, but only after removing the binary code. (And don't try to reimport the edited dsx files. )
You are correct, we don't see design code in production, and I don't mind telling you it sometimes makes tracking problems difficult. The benefit, which I find a good trade, is that the executable code I run in QA is identical to the code I run in production, and problems are generally either system environment (configuration, etc.) or insufficient test coverage in QA.
The most efficient method to export executables: In Manager, open the Jobs tree and click/highlight any folder that contains jobs and/or sequences. The Export menu will have an Executables choice, clicking on that will open a new window with a checkbox tree display of your project. You can check entire folders or individual jobs, and there's a counter on the right that tells you how many your picked. Please note: If you don't have your project organized in folders and sub-folders, you won't get much advantage using this.
Code: Select all
Jobs
Project folder
Sub-project/sub-section folder
Jobs/sequencers
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
-
- Premium Member
- Posts: 15
- Joined: Thu Mar 16, 2006 4:11 am
- Contact:
Hi Franklin,
Even by looking at the size of the .dsx files containing the executables from QA and PROD, we can tell the PROD one has not been updated because of the size of the PROD file alone.
Our next try will be to export both design and executable from QA and import that into PROD, then see if our change is being picked up.
Do you know of any reason why updating the executable in PROD by compiling the imported Design all of a sudden would not work anymore?
Thanks in Advance,
Martina
Even by looking at the size of the .dsx files containing the executables from QA and PROD, we can tell the PROD one has not been updated because of the size of the PROD file alone.
Our next try will be to export both design and executable from QA and import that into PROD, then see if our change is being picked up.
Do you know of any reason why updating the executable in PROD by compiling the imported Design all of a sudden would not work anymore?
Thanks in Advance,
Martina
Looks like you have a major mystery there, Martina.
I can only speculate on the possible issues. At this point, I agree with Craig that you would be better served to open a problem ticket.
In the meantime, I suggest reviewing the environment settings by doing a complete dump of them from Administrator (there's a dsadmin command line you can use, page 7-157 in the Parallel Job Advanced guide) and compare for differences between QA and Production. A significant difference in the executable sizes could point to compiler setting differences, though I don't know enough about the details to suggest what they might be.
My best guess is that you have environment/configuration issues. Only your support provider can determine that.
I can only speculate on the possible issues. At this point, I agree with Craig that you would be better served to open a problem ticket.
In the meantime, I suggest reviewing the environment settings by doing a complete dump of them from Administrator (there's a dsadmin command line you can use, page 7-157 in the Parallel Job Advanced guide) and compare for differences between QA and Production. A significant difference in the executable sizes could point to compiler setting differences, though I don't know enough about the details to suggest what they might be.
My best guess is that you have environment/configuration issues. Only your support provider can determine that.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872