DataCompress in DataStage

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
basu.ds
Participant
Posts: 118
Joined: Tue Feb 06, 2007 12:59 am
Location: Bangalore

DataCompress in DataStage

Post by basu.ds »

Hi ,
Can any one help me on this how to archive Data using DataStage.and how to compress data in Database level is there any data compress or Archive concept in Datastage?
Thanks
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

DataStage Server has no builtin compression functionality, but you can always shell out and execute UNIX or Windows commands such as gzip, compress or the command-line version of WinZip or WinRar or pkzip to mention just a few
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Some database products, such as DB2 version 9, have inherent data compression capability.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
basu.ds
Participant
Posts: 118
Joined: Tue Feb 06, 2007 12:59 am
Location: Bangalore

Post by basu.ds »

thanks a lot for responses.
reddysrc
Participant
Posts: 39
Joined: Mon May 30, 2005 3:19 pm

Re: DataCompress in DataStage

Post by reddysrc »

here is the unix shell script for archival & compress the data files.


#!/bin/ksh
filename=/../../LocationFileList.txt ## this file containg files names wich need to archive & compress
filedir=/directory location
backupdir=/ ## where you want to put your archived files

## *** ARCHIVING Processing *** #####
cd $filedir
for i in `cat $filename | grep "^[A]" | cut -f1`;
do
act_filename=$i


DateTime=`date "+%Y%m%d%0H%M%S"`


mv $i $backupdir/$i"_"$DateTime

BackupPathAndName=$backupdir/$i"_"$DateTime

compress -v $BackupPathAndName

RC=$?
if [ $RC -ne 0 ]
then
echo "The file could not be Archived.\nPlease check the directory path."
# exit 139
fi

done

echo "Files have been archived and cleaned successfully"


## *** Versioning Process *** ##
if [ "$NumberOfBackups" = "" ]
then
NumberOfBackups=10
fi

cd $backupdir
for i in `cat $filename | grep "^[A]" | cut -f3`;
do

FileToBackupName=$i
VersionCount=`ls -l $FileToBackupName* | wc -l`

while [ $VersionCount -gt $NumberOfBackups ]
do
OldestFile=`ls -rt $FileToBackupName* | head -1`
rm -f $OldestFile

RC=$?
if [ $RC != 0 ]
then
echo "The file could not be deleted.\nPlease check the directory path."
# exit 139
fi
VersionCount=`expr $VersionCount - 1`
echo "\n Old versions (more than 10) removed in Backup Directory\n"

done
done
exit 0


Let me know you have any issues.

Thanks
Rama
Post Reply