DSXchange

Well, you can't rename a .ds file because it is just a descriptor file that points to the actual data. The actual data sits in the DataSets directory (at the same level as the PXEngine directory). You can monitor this directory as PX writes out datasets. As the data files grow, you can get rough per...

A common misconception about PX is that it has some control over an operating system's scheduler. However, PX only invokes parallel instances of processes; it allows the OS scheduler to schedule the processes on the CPUs. For example, if a 4-way PX configuration file is set, in general, PX will sche...

From what has been posted, it seems like AxcelPath allows a user to do parallel dsrunjobs on a particular job. This is **radically** different from what PX does. While running a DS Server job in parallel harnesses the power of partitioned parallelism, it does not harness the power of pipeline parall...

DataStage,

I do not know what AxcelPath is so I can't answer your question. If I had more info on AxcelPath, I might be able to give you some info. Do you have a link for AxcelPath?

Thanks,
BP

Product: DS Server 6.0 Server: Sun Solaris Database: DB2 A couple of questions: 1. Can I use DataStage to call a DB2 stored procedure? I see a lot of discussion threads on Oracle stored procs, but none for DB2. 2. When doing a simple extract of a large volume of data thru DS, will it make a differen...

MAT, 'dsrecords' is a PX utility. Another useful utility is 'orchadmin'. If your env vars are set up properly, you should be able run the 'osh' command from the command line. If you can run that command, then you can run 'orchadmin' and 'dsrecords'. By running 'orchadmin', you will see a list of all...

MAT, I have seen PX generate statistics thru the graphical monitor, but I have never tried using the GetLinkInfo command. The behavior you have described sounds like a bug. If you need to get the # of records in a PX dataset, you can run the following command in the post-processing of your job: dsre...

Hi.

I'm currently reading non-partitioned tables from a db and assigning surrogate keys to the incoming data within DS Server 6.0. I am wondering what's going to happen to the surrogate-key assignment jobs if these tables ever become partitioned within in the db. Anyone know?

TIA,
BP

Creating a unique index while running in parallel can be a challenge. The easiest way to do this is to: Collect parallel streams into a single stream and use a midstream generator or transformer to generate the index field. This method introduces a bottleneck in your parallel data flow because it ad...

The compile flags are customizable properties of the Transformer stage. Turning off the full-debugging/trace flag can drastically reduce Transform compile times.

-
BP

The PX transformer stage generates C Code that you can view if you can find the appropriate RT_SC## directory in Projects dir on the server. When your PX transformer is not compiling, you can check the generated C code. Occasionally, you will find that the code has been generated incorrectly (a bug!...

Several of the environment variables that PX uses start with the prefix 'APT'. This stands for Applied Parallel Technologies.. the name of the company that originally built the PX Engine. Applied Parallel Technologies became Torrent Systems, and then Ascential bought Torrent and turned Torrent's Orc...

Ray, You're right. PX doesn't have a real need for hashed files. Data flows through PX jobs in a much different way than it flows through DS Server jobs. PX starts by loading as much data into memory as possible. Data is transferred between stages in buffers. When a buffer fills up, PX processes it ...

In designing the ETL process flows, I have the choice of running DS jobs in two ways: 1. I can run each set of related Extraction-Transform-Load processes as a single batch job. OR 2. I can batch all of the Extractions, do a checkpoint, then batch all of the Transforms, do a checkpoint, etc.. I pref...

I remember that there was an unresolved bug where PX would read incorrect dates when reading large volumes of data off a certain database (can't remember if it was DB2). How many records are you trying to import? Are you certain that you haven't set the 1901 date as the default (you can double click...