Modifying Job Export XML

Pavan_Yelugula · Post by **Pavan_Yelugula** » Mon Mar 14, 2011 6:03 am

Hi All
I have a quick question. If we do changes on the exported XML or DSX manually and import them in a server. Will the Jobs from the modified XML be stable or will i run in to some system errors.
The changes can be minor may be like the data type changes or major like making entries for new stages and changing transformations.

Is this advisable to make changes like this if not is there any particular reason why we shouldn't be meddling with these files

Thanks
Pavan

chulett · Post by **chulett** » Mon Mar 14, 2011 7:19 am

To me that would depend entirely on what exactly you are changing and if you get it 100% correct or not. Minor stuff, maybe, (although we may disagree on just what 'minor' means as I've destroyed a project importing a dsx file with a minor change) but I for one would never consider doing anything like "making entries for new stages" in the XML itself. The GUI and all of the validation it does is there for a reason, I'm curious under what circumstances you think taking this approach would be appropriate?

eostic · Post by **eostic** » Mon Mar 14, 2011 9:15 am

Reverse engineering, mass editing, and reporting techniques against DataStage exports (.dsx and .xml) have been going on since release 1. "Minor" and "Major" are entirely debatable, as Craig notes. I've seen subtle changes to things like the release number at the top, through wholesale changes of Stage Type, and edits only of special exports, such as TableDefs only...........any and all are fine....and at your own risk....just be sure you Test it thoroughly. The greater the type of change, the greater the planning, thinking, and risk -- and therefore the greater amount of testing that will be needed.

Ask yourself why you are doing it. If it's just to "make a subtle" change, at some point the benefits may be outweighed by the work needed to QA the solution. It may be simpler to just do the edits in the GUI. If you are building a solution that will help all your customers and provide new business for you, it's another story altogether (such as reverse engineering and job generation tooling that many of our partners have created over the years for various reasons). Of course, that requires lots more work, programming time, etc.

Ernie

Pavan_Yelugula · Post by **Pavan_Yelugula** » Tue Mar 15, 2011 3:33 am

Chulett
Thanks for the reply.

Ernie
Bang on the Money. we are actually trying to build some assets for creating some new business. it is a very intresting comment you made there about Job generation tooling. Is there some job generation tools you can point me to. Currently we are trying to create the XML ground up following the syntax which DS uses. I am not sure if there some documentation around the same. I gather from your comment that The approach we are taking is something that had been done before right by your partners. do correct me if i am wrong.

Thanks
Pavan

eostic · Post by **eostic** » Tue Mar 15, 2011 5:31 am

Over 11 years plus people have been doing this........many who monitor this forum too, no doubt.....

I have written many reporting solutions off of .dsx and .xml exports, but haven't been directly involved in reverse engineering efforts. I suspect that most efforts to do such are proprietary, since we don't formally document it.

Check out the consulting solutions and partners who offer conversion services. Some of them have code that probably generates .dsx's.

Ernie

chulett · Post by **chulett** » Tue Mar 15, 2011 7:15 am

As noted, there's no documentation so any job generation you end up doing will be via your own experimentation and discovery. To me that's an entirely different discussion from 'modifying job export xml' and something that's been going on forever as Ernie notes. I've personally been involved in a project where we took a basic job template and generated jobs from it in .dsx format using table metadata, ground out almost 500 jobs in only a few minutes once it was in place.

Pavan_Yelugula · Post by **Pavan_Yelugula** » Tue Mar 15, 2011 8:33 am

Ernie/Craig
Thanks for your replies and clarifications.

I am not sure why they don't have any documentation on the XML or DSX Generation. 500 jobs in a few minutes with a Basic template seems awesome to me

Never the less..i will be creating a lot of sample jobs and will study the XMLs that are being generated to see how i can build my functionality and jobs ground up.
The only bad thing is i will never be able to be confident of the Jobs that comes out of the XML's i created ground up till i see them and test them

Will be marking this as resolved. Thanks once again for all your inputs

Thanks
Pavan

ray.wurlod · Post by **ray.wurlod** » Tue Mar 15, 2011 4:26 pm

Keep in mind that importing from XML involves (a) converting from XML to DSX then (b) importing from DSX. So if you start playing around with the XML, you may also have to write your own XML to DSX functionality.