DSXchange

Check this out.... http://dsrealtime.wordpress.com/2010/07/02/what-are-shared-table-definitions/ Ernie Hi ernie, one quick question, referring to your article and the list of import methods you describe, what is the behavior of MW when a table is imported that already exists? In the case of the DS ...

Hi Ernie, I still don't know how to connect Cognos to MW. It looks like the easiest way is to go through Cognos Go! In the Business Glossary setup. We are getting the gosearch function installed on our Cognos server for this purpose. We don't have Fast Track (IBM gave us the impression that this was...

Hi folks, I am a little confused about the role of Table Definitions in MW. While they are critical in DataStage, and I can see them in MW, and I can see what stages depend on them in Workbench, there is no option to show lineage or impacts on them in WB. Furthermore, they don't appear in any impact...

Hi folks, what is the recommended workflow with MW and DataStage? Do I import Oracle tables into MW and then cascade them down to DataStage? Do all imports and updates happen directly on MW? Or do I import into DataStage and share them up to MW? Seems this doesn't work because updates to Table Defs ...

Ray, thanks so much for these recommendations. I'll look into it.

Doug

mhester wrote:Doug,

GREAT POST. Thanks a lot.

You can certainly tune memory in the Sort stage, up or down, but if the system is already swapping heavily I'd be more inclined to figure out what's causing that. Is it genuine swap (to system swap space) or is it usage of the scratchdisk resources mentioned in your configuration file? If the latte...

ray.wurlod wrote:Never encountered it, but can you source the dsenv script in the script that cron runs?

Hi Ray, normally yes, but in this case, the cron entry is created by Director.
Doug

Hi folks, everything runs fine firing things off from Director manually, but scheduled jobs fail with Error loading connector library libccora10g.so I chased this down to the cron process requiring the LD_LIBRARY_PATH specified in $DSHOME/dsenv, but I'm not sure where to go from here. Has anyone sol...

Hi folks, any idea why simple jobs of the form Oracle Connector->Sort->Dataset consume 3.2GB of swap space per instance, independent of the amount of data they are transferring? Perhaps more simply, are there parameters I can tune (like limit MB in the sort stage) to reduce this footprint? Thanks, D...

Hi folks, I have a bunch of very simple extract jobs. OracleConnector->Sort->DataSet. With 16 partitions, I am finding that each one of these jobs requires about 3.2GB of swap space (based on using vmstat while the jobs are running). At this rate I consume all 30GB of my "available" swap a...

Providing SAME partition would never creates this warning. But when you dont want the same partition as the previous stage? When you end up in re-partitioning? What I find is that if any input requires a repartition, then all of the inputs must have the flag cleared, even if they are set to SAME, o...

I don't understand why the flag has to be cleared on all inputs. Seems to me any inputs with a partitioning strategy of SAME should not lead to the warning, but they do.

Hi folks, I think I have this solved. Apparently the error message is correct. If you are doing a range lookup, the reference link must be sorted on the lookup keys. Partition entire first, then sort.

Doug

add $OSH_PRINT_SCHEMAS to your job and look at the actual schemas going into your filter stage are they actually timestamp? RECORDED_TIME: nullable timestamp; EFFECTIVE_TIME: timestamp; TO_TIME: nullable timestamp; Doing the obvious and removing "nullable" led to a null error, so I think ...