DSXchange

Hi, The amount of data that can be stored in each destination data file is limited by the characteristics of the file system and the amount of free disk space available. In your case, there is no problem until and unless, if there is an issue with space. You can access that Dataset as a source in ot...

Hi,

In the Datastage job, we can include comments by using Annotation Boxes.

In all stages, we have Descrption filed for all columns. Where you can write, your column derivation details.

Hi, You cannot use the UNIX rm command to delete a data set because DataStage represents a single data set with multiple files. Using rm simply removes the descriptor file, leaving the much larger data files behind. So, you can use Data set management in Designer or Manager to delete the entire data...

Hi,

Are you tried by switching off the RCP ? Try with this option.

One more solution is use modify stage to drop a column, which is not required for further processing.

Hi All, First thing is How link sorting will differ from External sorting stage? As per my knowledge, both will use the Tsort operator and both will use the same disk for sorting purpose. Until and unless, you have a specific requirement to use sort stage ( Ex: implementing some logic by using Chang...

Hi,

As per Parallel Job Advance guid, it is suggested that, don't go for sparse lookup if your reference data is more. In your case, its 30 billion records. There will be a effect on performance if you use a sparse lookup.

Hi,

In Parallel it is not possible to use the same dataset as a Source and Target. You required to create a new job for this functionality. Where as incase of server we have HASH file facility to perform the same.

Hi, Possible solution for this is to change the column name of any one link i.e either in left or right . You can perform this operation by using copy stage or in any stage before your join link. One more solution for this is Disable RCP at the job level. It will not throw any warning. One more bett...

1001 question about preserve partitioning. My friend, search is the best friend for you. Your saying error, but I think it is a warning. Error and Warning is different. Preserve partitioning about the flow of your data. What is your source, is it sequential file ? *********** Thanks for info,will k...

Hi Nageshsunkoji, I am only using one DataSet, but referring to it multiple times by using multiple stages. Each stage points to the same underlying dataset in the OS. No need to copy anything. I just needed to know that I could access the same dataset multiple times in the same job. Rob W Thats ok...

Thanks, Arnd. No internal locking for reading or anything like that? Rob hi, Rather than going for multiple datsets .The better approach is, use one Dataset in your job and use copy stage in the down stream and make that many copies to filter your data. It will save your memory and increase the per...

1001 question about preserve partitioning. My friend, search is the best friend for you.

Your saying error, but I think it is a warning. Error and Warning is different.

Preserve partitioning about the flow of your data. What is your source, is it sequential file ?

I think, it is not possible to do this directly in the where condition of filter stage. There should be a intermediate dataset/sequentail file to store the result of your SQL and then perform filter on the required column.

Hi,

First thing is the datatype of lookup key should be the same in both reference and driving link.
Second thing is check the lookup condition status. If it is Fail, then your job will abort. Select the other options like Drop,Output and continue. For Output you have to provide reject link.

Hi,

you can acheive the same thing by using Unix script or unix funations. Call the same in the After Job subroutine of the job.

Checking the header and position of the column is quite possible with the shell script.