DSXchange

Posted: **Tue Jul 07, 2009 10:03 am**

In an earlier post I referred to a comprehensive 8.1 patch for IS Manager and istool. The good news is that it has resolved several of the issues that were preventing both tools from working properly.

However, the syntax on the istool command can be a bit tricky to figure out the first time because it uses a -datastage (-ds) option that has sub-options that must be passed properly. Because of this, I'm posting a few working examples of the istool command in operation. I'm just going to show using the "export" option since that is the most frequently used option and it will give you an idea of how the options work.

Note: In the examples below, I've updated it to show a three-tier architecture with a separate services tier (serviceshost) and an engine tier (enginehost). It can be a bit picky as to matching the engine host syntax (might have fully qualified .com name) and also is in upper case. If you are having issues matching the syntax, login to the InfoSphere Information Server Manager (the GUI front-end for istool) and use same syntax it displays for hostname after you connect.

First - here's some of the "help" options to show that there is help for each of the main istool operations (again - just showing export).

Code: Select all

$ pwd
/opt/IBM/InformationServer/Clients/istools/cli
$ ./istool -h
istool Command [ Command_Options ] [ Options ]

Possible values for: Command
    -                    : Provide usage information.
    help                 : Provide command-line help information.
                         : Istool Console.
    export               : Export to a file.
    import               : Import from a file.
    build package        : Build a Deployment Package.
    deploy package       : Deploy a Deployment Package.

To list Command_Options, enter:  istool Command -help

Possible values for: Options
    Long Name:         Short Name:     Description:
    -help            , -h              : print command usage
    -verbose         , -v              : display progress
    -silent          , -s              : silence command output
    -script          , -script         : file to read commands from.
 
Exit codes:
      0 - Success
      1 - Warning
      2 - Partial failure
      3 - Error reading from console
      4 - Invalid command history index
      5 - Error reading from script file
     11 - Invalid command syntax

$./istool export -help
istool export  [ Options ]

Possible values for: Options

    Long Name:         Short Name:     Description:
    -help           ,  -h              : print command usage
    -domain         ,  -dom            : Name of Information Server domain
    -username       ,  -u              : Name of user account on Information Server domain
    -password       ,  -p              : Password for Information Server account
    -verbose        ,  -v              : display progress
    -silent         ,  -s              : silence command output
    -archive        ,  -ar             : Archive file
    -preview        ,  -pre            : Preview export task without executing
    -abortIfError   ,  -abort          : Abort task on N errors
    -updatearchive  ,  -up             : Update any existing Archive file
    -datastage      ,  -ds             : DataStage Assets

            Options for: -datastage

              -includedependent      ,  -incdep    : Include dependent items
              -nodesign              ,  -nodes     : Exclude design items
              -includeexecutable     ,  -incexec   : Include executable items
              -base                  ,  -base      : Prefix for Paths
              DataStage item paths

Exit codes:
      0 - Success
      1 - Warning
      2 - Partial failure
      3 - Export failed
      4 - Export preview failed
      5 - Invalid archive file
     10 - Failed to connect to Information Server domain
     11 - Invalid command syntax

Now some working examples of an export. Note that I've "sanitized" the commands and output a bit to remove sensitive info. I'm also using the "-preview" option in a few cases that just shows what would be exported without actually doing the export. Also note that the suffixes are internal (ie: not part of the user supplied item names) - here are the ones I'm aware of:

qjb - Job Sequence
sjb - Server Job
pjb - Parallel Job
srt - Server routine
tbd - Table Definition
tfm - Transform

In the first example I'm just backing up a single category called "Batch":

Code: Select all

$ ./istool export -domain serviceshost:9080 -u dsadm -p password -ar test.isx -pre -ds '-base="ENGINEHOST/Dev_Project" Jobs/Batch/*.*'
Beginning Export Preview
 [1/2] ENGINEHOST/Dev_AdHoc/Jobs/Batch/SeqIBIInvoices.qjb
 [2/2] ENGINEHOST/Dev_AdHoc/Jobs/Batch/SeqIBIOrders.qjb
Previewed 2 assets

In the second example I use the "-incdep" option that includes dependent jobs and routines for that category.

Code: Select all

$ ./istool export -domain serviceshost:9080 -u dsadm -p password -ar test.isx -pre -ds '-incdep -base="ENGINEHOST/Dev_Project" Jobs/Batch/*.*'
Beginning Export Preview
 [1/9] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIInvoices.qjb
 [2/9] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIOrders.qjb
 [3/9] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmFileName.sjb
 [4/9] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmSelectDate.sjb
 [5/9] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBIInvoices.sjb
 [6/9] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISales.sjb
 [7/9] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISalesVoids.sjb
 [8/9] ENGINEHOST/Dev_Project/Routines/Job Control/CreateFileFromDate.srt
 [9/9] ENGINEHOST/Dev_Project/Routines/Job Control/SetDateFromInput.srt
Previewed 9 assets

In the final example I show how you'd backup all the jobs in a project including any routines they call:

Code: Select all

$ ./istool export -domain serviceshost:9080 -u dsadm -p password -ar test.isx -ds '-incdep  -base="ENGINEHOST/Dev_Project" Jobs/*/*.*'
Beginning Export
 [1/11] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIInvoices.qjb
 [2/11] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIOrders.qjb
 [3/11] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmFileName.sjb
 [4/11] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmSelectDate.sjb
 [5/11] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBIInvoices.sjb
 [6/11] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISales.sjb
 [7/11] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISalesVoids.sjb
 [8/11] ENGINEHOST/Dev_Project/Jobs/XArchive/SeqIBIInvoices_ParameterChg_20070404.qjb
 [9/11] ENGINEHOST/Dev_Project/Jobs/XArchive/SeqIBIOrders_ParameterChg_20070404.qjb
 [10/11] ENGINEHOST/Dev_Project/Routines/Job Control/CreateFileFromDate.srt
 [11/11] ENGINEHOST/Dev_Project/Routines/Job Control/SetDateFromInput.srt
Elapsed time: 00:00:18
Exported 11 assets
$ ls -l test.isx
-rw-r--r--  1 dsadm dstage 115147 Jul  6 13:53 test.isx

Hopefully that's helpful and keeps you from wasting time trying to figure out the syntax on the command!

[Edit - Andy Sorrell 04/29/2014]
Adding some stuff I keep getting asked about...

Below syntax shows backing up EVERYTHING in a project (all routines, jobs, etc.). Also shows syntax to use authfile with encrypted password. Due to poor design, you must still specify domain on command line, even with an .authfile (yes I've complained to IBM about it!).

Code: Select all

./istool export -domain serviceshost:9080 -authfile /home/dsadm/.authfile -silent -archive /backups/projectname.isx -datastage "-incdep -base=ENGINEHOST/Dev_Project */*.*"

The contents of the .authfile would be something like this:
user=dsadm
password={iisenc}encryption_text_here==
domain=servicehost:9080
server=enginehost

Posted: **Tue Jul 07, 2009 11:07 am**

Very nice, thank you Uncle Andy for taking point on that trail.

Posted: **Tue Jul 07, 2009 3:34 pm**

Thanks. This helps quite a bit. However, I'm getting an error when I run the export saying that the project does not exist, even though I know it does. I'm wondering if I have a typo or something wrong with my syntax. Can you spot anything?

Code: Select all

[dsadm@cpgapl65: /vend/IBM/InformationServer/v81/Clients/istools/cli] $ ./istool export -domain cpgapl65 -u dsadm -p password -ar test.isx -ds '- base="cpgapl65.xsr.domain.net/SANDBOX" Jobs/*/*.*'
Beginning Export
Export failed: Invalid asset identifier: Jobs/*/*.*. Invalid project name.

[dsadm@cpgapl65: /vend/IBM/InformationServer/v81/Clients/istools/cli] $ dsjob -lprojects
SANDBOX

Status code = 0

Posted: **Wed Jul 08, 2009 12:56 pm**

Two things - first of all I see a space before the word base - it should be "-base" not "- base".

Secondly - if you are in Unix and your password has any special characters like an @, & or $ you'll have to put a backslash in front of it (ie: \$) on the command line to submit it successfully.

Posted: **Thu Nov 19, 2009 9:28 pm**

Andy,

Code: Select all

$ ./istool export -domain dsdevbox -u dsadm -p password -ar test.isx -ds '-incdep  -base="DSDEVBOX.MMM.COM/Dev_Project" Jobs/*/*.*'

I tried,looks like its not doing the complete project export which I achieve
manually thro' old Export functionlity creating .dsx file by selecting all.

Need help to customise the above cole to do the complete project export.

Posted: **Tue Dec 15, 2009 1:47 am**

And I assume that we cannot import a .dsx file through this?

Posted: **Tue Dec 15, 2009 2:20 am**

That is correct. Deployment packages (which is what Information Server Manager (istool)) deals with, are different from DataStage export files.

Command line choices for importing from dsx files do exist, including dsjob, dscmdimport and dsimport.

Posted: **Fri Apr 02, 2010 2:16 pm**

thank you for the details on the utility "istool". Is it possible to encrypt password ?

Posted: **Mon Apr 05, 2010 11:16 am**

No method of encrypting passwords that I'm aware of. If your site uses individual login id's then you could

1) store the password in a file
2) make the file readable only by the dsadm id
3) use file descriptors on the command line to read the password from the file

something like:

Code: Select all

-u dsadm -p 8 8</dir/dir/pw_file

That should probably work.

{Edited - 09/19/13}

The istool command now supports "-authfile" which allows you to put the id and password in a protected file so they won't be displayed. The password can be encrypted via ASBNode/bin/encrypt.sh.

Posted: **Mon Apr 05, 2010 12:30 pm**

I wonder if Apple will sue IBM over use of its iStool product.

Posted: **Mon Apr 05, 2010 12:49 pm**

Probably send their stock right into the crapper...

Posted: **Mon Apr 05, 2010 8:35 pm**

Thomas Crapper?

Posted: **Wed Apr 14, 2010 2:19 pm**

I'm trying to create a backupScript.sh that contains the following.

Note: The tmpVar - srvName/awk pieces are just b/c I thought I might have had some case sensitivity issues, so if I don't have to use them, great, if I do, it's already done and working.

I've verified the project exists and that the username/password works when pulling up the project through the Designer. It may not have anything to do with the issue, but I am using LDAP for my authentication into the DataStage Designer.

I get the following errors:

Beginning Export Preview
Export preview failed: Project not found: qa_PRNAME

Code: Select all

tmpVar=`hostname`
srvName=`echo $tmpVar | awk '{print toupper($0)}'`
#echo $srvName

#From http://www.dsxchange.com/viewtopic.php?t=128126
./istool export -domain $srvName -u <username> -p <pw:includes @> -ar /home/dsadm/backups/testBackupWWW.isx -pre -ds '-base="$srvName.DOMAIN-NAME.COM/qa_PRNAME" Jobs/*.*'

Any help would be greatly appreciated. This is 8.1 with FP1.

Posted: **Thu Apr 15, 2010 1:06 am**

I use the syntax

Code: Select all

istool export -domain $srvName:9080 -username xx -password yy -archive /home/dsadm/backups/testBackupWWW.isx -ds '"$srvName.DOMAIN.NAME.COM/qa_PRNAME/*/*.*"

Posted: **Wed Jun 23, 2010 9:47 pm**

asorrell:: As you have pointed out earlier that if the password contains '$' a backslash needs to be in front of it.

I don't think it is needed.I'm using a password that contains a '$' and without the '/' it is working fine for me.

and I guess if I am using istool utility to export assets I must use the istool utility to import those.

DSXchange

istool syntax

istool syntax

Encrypt password