PX Data Sets History

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
MAT
Participant
Posts: 65
Joined: Wed Mar 05, 2003 8:44 am
Location: Montréal, Canada

PX Data Sets History

Post by MAT »

Hi all,

I have a PX question. Is it possible to save a copy of a Data Set under a different name for historical purposes? I would like to be able for example to rename a Data Set named example.ds to example20030407.ds. I tried changing the name of the file, changing the names of the differents files composing the Data Set and chaging the names inside the master file but it does not work (Can't read : wrong Data Set format).

Thanks

MAT
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

Post by bigpoppa »

Well, you can't rename a .ds file because it is just a descriptor file that points to the actual data. The actual data sits in the DataSets directory (at the same level as the PXEngine directory). You can monitor this directory as PX writes out datasets. As the data files grow, you can get rough performance numbers by tracking time/bytes added.

If you delete a .ds file using the UNIX 'rm' command, the data files will still exist in the DataSet directory.

To properly remove a dataset, you must use the 'orchadmin' PX command line utility. The full command is 'orchadmin rm XXX.ds'.

Orchadmin may also have utilities to change the name of a dataset. If not, to transfer input.ds into output.ds, just write a simple OSH job :

copy < input.ds > output.ds

HTH,
BP
MAT
Participant
Posts: 65
Joined: Wed Mar 05, 2003 8:44 am
Location: Montréal, Canada

Post by MAT »

Thanks BP,

Being a newbie with PX, I do not know how to build OSH job, is there a document explaining how to do it somewhere?

Regards

MAT
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

Post by bigpoppa »

Well, you can do it thru the PX GUI, if you like. Just link a input dataset to a 'copy' stage. Then link the 'copy' stage to the output dataset. Compile and run!

To run the osh script, you would need to type:

osh "copy < in.ds > out.ds"

at the command line. This will only work if your environment is set up correctly.. meaning that the PX bin and libraries are in your PATH and LIBPATH env vars.

Each PX GUI job generates OSH code, similar to the way that PX Server jobs generate BASIC code. You can look thru the Projects directory on the server and try to find some OSH code in the RT_BP** or RT_SC** directories.

Please let me know if this is not enough info.

-BP
MAT
Participant
Posts: 65
Joined: Wed Mar 05, 2003 8:44 am
Location: Montréal, Canada

Post by MAT »

My goal is to write a routine that will do the copying for me. You think it's possible? How can I check if my librairies are set up correctly? I can see the env var but I don't know where the PX components are found. It is probably under the PXEngine directory but then again, I also have a PXEngine.600.1 directory. Can you tell me where to search?

Thanks

MAT
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

Post by bigpoppa »

I think that the PXEngine dir is linked to the other PXEngine dir.

When you run a PX GUI job, you see Environment Variable Settings in the job log. You'll see a couple of relevant environment variables including:

PATH
APT_CONFIG_FILE
APT_ORCHHOME
LIBPATH (or something like that)

The settings for these variables should be the same as the settings for the same variables when you run the 'env' command on the UNIX command line.

If your UNIX env vars do not match the ones you found in the job log, you need to modify your env vars.

Once these are set, you should be able to run the 'osh' command alone. If you get the message 'osh not found' or some kind of error, then you need to go back to your env vars and see where there is a disconnect.

-BP
MAT
Participant
Posts: 65
Joined: Wed Mar 05, 2003 8:44 am
Location: Montréal, Canada

Post by MAT »

Thanks a lot BP,
My env var were not set properly. It works now

MAT
Post Reply