Developing a Batch File to Back up a complete proyect

J.Calvo · Post by **J.Calvo** » Wed Oct 14, 2009 3:35 am

Try Failed, I don't know what could be the problem...

One question, in the proyect path I have the common folders, and I have created in our server two folders witch contains for example connections and some job...

I think if I have selected a Proyect to Export, then, all the elements witch hang of it should be exported succesfully, and all the folders should be created automatically. It's Ok?

Other point: Is necesary that all the jobs witch I want to export have been compiled previously?

Thanks!

chulett · Post by **chulett** » Wed Oct 14, 2009 5:34 am

Second question first - no, they don't need to be compiled.

Now, for the first - by 'folders' do you mean Categories or do you mean literally folders/directories you created at the operating system level inside the project directory? If you mean categories, then yes an export always includes that information and will create the categories automatically during an import.

As to the export failure, can you find any pattern to the jobs that do not export? Anything they have in common? I'm afraid it is rapidly becoming time for you to pass this problem off to your official support provider, someone who can get access and 'eyes on' your installation if need be. I'm running out of ideas.

J.Calvo · Post by **J.Calvo** » Wed Oct 14, 2009 5:40 am

I mean categories. Thanks.

J.Calvo · Post by **J.Calvo** » Thu Oct 15, 2009 5:29 am

Hi, this problem continue pending...
I explain some data:

One of the folders witch haven't been imported is a folder named Conexiones (Is a categorie, a folder manually created with designer tool) in this folder i have 6 connections, if I select only dis folder and I Export it, Then all is correct, but if i Export The Entire proyect ... This folder not is imported.

chulett · Post by **chulett** » Thu Oct 15, 2009 5:53 am

Hmmm... so there's no jobs in the category, just connectors? Let me test that on 8.1 and see if it works there when I get a chance.

J.Calvo · Post by **J.Calvo** » Thu Oct 15, 2009 9:05 am

Ok Thanks!

chulett · Post by **chulett** » Thu Oct 15, 2009 5:26 pm

Not sure what to tell you other than it works fine in 8.1 for me. Created a new job category (even though I think you're really supposed to store connectors elsewhere) and created a new connector object there. Ran two full project exports from the command line - using dsexport and dscmdexport - and got everything in both, including all of the connectors from the project. Heck, the dsx files were even an identical byte count when they were done.

How did you verify yours were missing? Did you open up the dsx file in your editor of choice and look through the BEGIN DSDATACONNECTIONS section? It should be the last section in the file, btw - at least it was in mine:

Code: Select all

BEGIN DSDATACONNECTIONS
   BEGIN DSRECORD
      Identifier "Frodo"
      DateModified "2009-10-15"
      TimeModified "16.28.25"
      OLEType "CDataConnection"
      Readonly "0"
      Category "\\Jobs\\CraigH\\Connectors"
      ShortDesc "Test only"
      StageType "PxOracle"
      UserName "y"
      Password "LM;"
      ConnectionString "x"
   END DSRECORD
END DSDATACONNECTIONS

If you still have an issue then all I can suggest at this point is to contact your official support provider and open a case. The 8.0.x release was pretty darn buggy, perhaps this is a known issue with a patch available for it.

J.Calvo · Post by **J.Calvo** » Fri Oct 16, 2009 2:16 am

chulett wrote:Not sure what to tell you other than it works fine in 8.1 for me. Created a new job category (even though I think you're really supposed to store connectors elsewhere) and created a new connector object there. Ran two full project exports from the command line - using dsexport and dscmdexport - and got everything in both, including all of the connectors from the project. Heck, the dsx files were even an identical byte count when they were done.

How did you verify yours were missing? Did you open up the dsx file in your editor of choice and look through the BEGIN DSDATACONNECTIONS section? It should be the last section in the file, btw - at least it was in mine:
Code: Select all
BEGIN DSDATACONNECTIONS
   BEGIN DSRECORD
      Identifier "Frodo"
      DateModified "2009-10-15"
      TimeModified "16.28.25"
      OLEType "CDataConnection"
      Readonly "0"
      Category "\\Jobs\\CraigH\\Connectors"
      ShortDesc "Test only"
      StageType "PxOracle"
      UserName "y"
      Password "LM;"
      ConnectionString "x"
   END DSRECORD
END DSDATACONNECTIONS
If you still have an issue then all I can suggest at this point is to contact your official support provider and open a case. The 8.0.x release was pretty darn buggy, perhaps this is a known issue with a patch available for it.

Yes, I have verified this point previously, and the result is witch in the dsx file exists the connections in the section: BEGIN DSDATACONNECTIONS , but this isn't imported really.

chulett · Post by **chulett** » Fri Oct 16, 2009 6:59 am

And here I thought we were talking about an export problem.

J.Calvo wrote:Then this items are not read only items, but when i import the proyect i can see that some jobs are not been exported.

The Read Only Items wasn't the elements witch I have not found in the proyect export file. The problem may be other one.

So... all along they've been exporting and the problem is you can't import them?

J.Calvo · Post by **J.Calvo** » Fri Oct 16, 2009 8:58 am

chulett wrote:And here I thought we were talking about an export problem.

J.Calvo wrote:Then this items are not read only items, but when i import the proyect i can see that some jobs are not been exported.

The Read Only Items wasn't the elements witch I have not found in the proyect export file. The problem may be other one.
So... all along they've been exporting and the problem is you can't import them?

Finally, Elements witch not have been imported exists in the dsx file, but not have been imported...

chulett · Post by **chulett** » Fri Oct 16, 2009 10:08 am

[please disregard this post]

OK, you're correct in that they are not something that will import back from the GUI. When I selected my .dsx file from the Designer Import option and told it it do a 'selective' import, it brought in almost everything in the file but threw this message up as well:

20 mainframe objects were ignored.

No clue why it would say that as there's no "mainframe" anything here and this isn't the 390 version of DataStage. I also don't know exactly what the 20 objects were yet, but it definitely excluded the Connector objects in the export file for some silly ass reason.

Why? I have no idea, someone would need to ping IBM on that one - unless somebody out there in the home audience knows the answer? Is this something that can be solved by working from the command line? Or via the istool tool?

chulett · Post by **chulett** » Fri Oct 16, 2009 1:54 pm

Well... rather than delete my previous post, let me simply print a retraction. I was wrong, the connectors are there in the list of things I can import, I just didn't see them the first time I looked because they were lost in my large list between the Stage Types and the Routines. They are there labelled as Data Connections and import just fine. [sigh]

So... not sure if that means it was an issue in 8.0.1 that was resolved in 8.1 or if something else is going on. Can you explain what steps you went through to determine you had this problem? Can you detail exactly what tool / process you are using for the import?

chulett · Post by **chulett** » Fri Oct 16, 2009 1:56 pm

... and I still have no idea what the '20 mainframe objects' might have been.

J.Calvo · Post by **J.Calvo** » Mon Oct 19, 2009 2:06 am

In the proyect named ETL_Desarrollo I have a tree like that:

ETL_Desarrollo (Proyect)
Conexiones (Folder)
Conexion_1 (Connection) --> Not is imported
Conexion_2 (Connection) --> Not is imported
Conexion_3 (Connection) --> Not is imported
Conexion_4 (Connection) --> Not is imported
Conexion_5 (Connection) --> Not is imported
Conexion_6 (Connection) --> Not is imported
Data Elements (Folder)
IMS Databases (Folder)
IMS Viewsets (Folder)
Jobs (Folder)
Volcados (Folder)
Volcado1 (Folder)
Job11 (Sequence Job)
Job12 (Server Job)
Job13 (Server Job)
Job14 (Server Job)
Job15 (Server Job)
Volcado2 (Folder)
Job21 (Sequence Job)
Job22 (Server Job)
Job23 (Server Job)
Job24 (Server Job)
Job25 (Server Job)
Volcado (Folder)
Job31 (Sequence Job)
Job32 (Server Job)
Job33 (Server Job)
Job34 (Server Job)
Job35 (Server Job)
Pruebas (Folder)
Volcado1_Pruebas (Folder)
Job11_Test (Sequence Job)
Job12_Test (Server Job)
Job13_Test (Server Job)
Job14_Test (Server Job)
Job15_Test (Server Job)
Prizing (Folder)
Job41 (Server Job) --> Not is imported
Job42 (Server Job) --> Not is imported
Volcado (Folder)
Job31 (Sequence Job)
Job32 (Server Job)
Job33 (Server Job)
Job34 (Server Job)
Job35 (Server Job)
Permanent (Folder)
Machine Profiles (Folder)
Match Specifications (Folder)
Parametros (Folder)
Routines (Folder)
Shared Containers (Folder)
Stage Types (Folder)
Standarization Rules (Folder)
Table Definitions (Folder)
Transforms (Folder)
WAVES Rules (Folder)

Really exists much more elements not imported....

chulett · Post by **chulett** » Mon Oct 19, 2009 7:30 am

chulett wrote:Can you detail exactly what tool / process you are using for the import?