find/replace(Parameters) utility to promote to production

venkat n · Post by **venkat n** » Thu Jul 21, 2005 1:41 am

To be deployed for use DataStage code promotions. project will have a set of text files to run against code coming from Development: .dsx file
Here I want to change some pareamters form Development to QA/Test/production
I want to Find a utility to handle multiple find and replace combinations in .dsx file : 1. It needs to be free and run on Windows, where the DataStage DSX files are. It could be Windows freeware or some dos/Windows batch that calls Unix commands (you can get a free set of Unix commands to add on to Windows). 2. It needs to be able to process a list of global find and replace combinations that are defined in a text file, for example one find/replace combo on each line. 3. It needs to be able to process special characters including double quotes, as part of the find or replace string.

Any one please suggest me! on this issue

thanks in advance.....
N.Venkat krishna

Sreenivasulu · Post by **Sreenivasulu** » Thu Jul 21, 2005 2:29 am

I believe all dsx files can be viewed as txt file where an automated find/replace could work.

Regards
Sreenivasulu

ArndW · Post by **ArndW** » Thu Jul 21, 2005 2:37 am

Venkat,

as Sreenivasulu has stated, the .DSX files are text (even the binary object code is UUencoded to displayable characters.

Thus you can do all of your changes in the editor of your choice. I like EMACS (free dowloads available) which allows you to write quite nifty macros. You can also use VI as well, or any other editor. You could write a .BAT file or even a simple DataStage job to perform this modification for you. Another option is to export as an XML file and then there are numerous applications that will let you modify that structured file.

I think in your case you should write a DataStage job to do this for you.

venkat n · Post by **venkat n** » Thu Jul 21, 2005 3:34 am

Hi Thx for your responce especially for srinivasulu and arndw

but I need help to write a shell script to change the multiplea find/replace things
suppose i want to change src (dsn,UID,pwd) ,trg (dsn,UID,pwd)
while it is kick-off, for this shell script Just i gave only parameter values as arguments that reads and changes automatically.

where i can get EMACS (free dowloads) Please let me know the links or what ever you have regarding this

For this i want to have shell script can anybody help for this

ArndW · Post by **ArndW** » Thu Jul 21, 2005 3:39 am

Venkat,

I think EMACS might be overkill, it is a very powerful (but old) editor. You really should not write a shell script for this if you have DataStage. A simple job to read the .dsx file, do an EREPLACE() of your {search} and {replace} texts in a simple transform and write it to an output file. It shouldn't take more than an hour to write this job and debug it.

This is easier than writing and debugging a shell script. I doubt that someone will do this for you and post it here - although if you have a script that isn't working I am sure that any number of people will help you with that.

ArndW · Post by **ArndW** » Thu Jul 21, 2005 3:41 am

I forgot to add you should look for "WinEMACS download" in Google or your favorite search engine to get a number of sites where you can get this package.

venkat n · Post by **venkat n** » Thu Jul 21, 2005 3:45 am

Thankx ArndW

I will go through ur advice.

A simple job to read the .dsx file, do an EREPLACE() of your {search} and {replace} texts in a simple transform and write it to an output file. It shouldn't take more than an hour to write this job and debug it.

thanks for this IDea

roy · Post by **roy** » Thu Jul 21, 2005 4:12 am

Hi,
Just wanted to point out that if you used encripted passwords you'r out of luck there since you can't search/replace them!!!
the only thing I can think of capabale for mass updating your project/s wide paramters might be parameter manager (or hacking the repository to get it).

Having that said, I hope you implemented good practices hence use parameters so the location/s and occurances of actuall passwords if you have any should be minimal.

IHTH,

venkat n · Post by **venkat n** » Thu Jul 21, 2005 4:38 am

Thaks Ray,

We are using encrypting passwords only

kduke · Post by **kduke** » Thu Jul 21, 2005 7:50 am

This may be a feature in the next release of Parameter Manager where you could set all the parameter defaults. This would be called a Data Set. So if you have 3 Data Sets for Dev, Test and Production. When you promote your jobs from Dev to Test then you apply the Test Data Set.

This would for example change the Oracle connection information from DwhDev SID to DwhTst and the associated users and password.

Data Set Param Name Param Value
Test TargetSID DWHTST
Test TargetUser etltest
Test TargetPwd etltest (encrypted cannot view)

If this was a part Parameter Manager would this be something you would want to purchase?

A Data Set could also change your source on a conversion. So if you had multiple sources to convert then this would allow different groups of parameters to have different values at run time. This is a much more complex idea to visualize unless you had this need before. This is very powerful if you are doing data conversions. The need is less in a typical data warehouse environment.

All of us should have at least 3 environments like Development, Test and Production. If you are using Version Control then you should have to set the default parameter value only once unless it is a new job. It will not overwrite default values when promoting jobs. So on a new job you still need to set it.

Anyway if you would like to see this in Parameter Manager then let them know www.Tools4DataStage.com.

lebos · Post by **lebos** » Thu Jul 21, 2005 8:08 am

Actually you can use a text editor to change/replace encrypted passwords. We do it fairly often using UltraEdit (a Windows program).

The trick is to know what the encrypted passwords look like. This is easily accomplished by writing a dummy job which contains both the old and the new passwords as encrypted parameters using the defaults to contain your password values, exporting that job to a .dsx file, and displaying that .dsx file with your text editor. You will find the encrypted versions of the passwords. Now you can do a simple find/replace with these values on your real export file.

You should probably make a copy of your dsx file before doing any of this, just in case.

Larry