Hi ,
Can any one help me on this how to archive Data using DataStage.and how to compress data in Database level is there any data compress or Archive concept in Datastage?
Thanks
DataCompress in DataStage
Moderators: chulett, rschirm, roy
DataStage Server has no builtin compression functionality, but you can always shell out and execute UNIX or Windows commands such as gzip, compress or the command-line version of WinZip or WinRar or pkzip to mention just a few
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Re: DataCompress in DataStage
here is the unix shell script for archival & compress the data files.
#!/bin/ksh
filename=/../../LocationFileList.txt ## this file containg files names wich need to archive & compress
filedir=/directory location
backupdir=/ ## where you want to put your archived files
## *** ARCHIVING Processing *** #####
cd $filedir
for i in `cat $filename | grep "^[A]" | cut -f1`;
do
act_filename=$i
DateTime=`date "+%Y%m%d%0H%M%S"`
mv $i $backupdir/$i"_"$DateTime
BackupPathAndName=$backupdir/$i"_"$DateTime
compress -v $BackupPathAndName
RC=$?
if [ $RC -ne 0 ]
then
echo "The file could not be Archived.\nPlease check the directory path."
# exit 139
fi
done
echo "Files have been archived and cleaned successfully"
## *** Versioning Process *** ##
if [ "$NumberOfBackups" = "" ]
then
NumberOfBackups=10
fi
cd $backupdir
for i in `cat $filename | grep "^[A]" | cut -f3`;
do
FileToBackupName=$i
VersionCount=`ls -l $FileToBackupName* | wc -l`
while [ $VersionCount -gt $NumberOfBackups ]
do
OldestFile=`ls -rt $FileToBackupName* | head -1`
rm -f $OldestFile
RC=$?
if [ $RC != 0 ]
then
echo "The file could not be deleted.\nPlease check the directory path."
# exit 139
fi
VersionCount=`expr $VersionCount - 1`
echo "\n Old versions (more than 10) removed in Backup Directory\n"
done
done
exit 0
Let me know you have any issues.
Thanks
Rama
#!/bin/ksh
filename=/../../LocationFileList.txt ## this file containg files names wich need to archive & compress
filedir=/directory location
backupdir=/ ## where you want to put your archived files
## *** ARCHIVING Processing *** #####
cd $filedir
for i in `cat $filename | grep "^[A]" | cut -f1`;
do
act_filename=$i
DateTime=`date "+%Y%m%d%0H%M%S"`
mv $i $backupdir/$i"_"$DateTime
BackupPathAndName=$backupdir/$i"_"$DateTime
compress -v $BackupPathAndName
RC=$?
if [ $RC -ne 0 ]
then
echo "The file could not be Archived.\nPlease check the directory path."
# exit 139
fi
done
echo "Files have been archived and cleaned successfully"
## *** Versioning Process *** ##
if [ "$NumberOfBackups" = "" ]
then
NumberOfBackups=10
fi
cd $backupdir
for i in `cat $filename | grep "^[A]" | cut -f3`;
do
FileToBackupName=$i
VersionCount=`ls -l $FileToBackupName* | wc -l`
while [ $VersionCount -gt $NumberOfBackups ]
do
OldestFile=`ls -rt $FileToBackupName* | head -1`
rm -f $OldestFile
RC=$?
if [ $RC != 0 ]
then
echo "The file could not be deleted.\nPlease check the directory path."
# exit 139
fi
VersionCount=`expr $VersionCount - 1`
echo "\n Old versions (more than 10) removed in Backup Directory\n"
done
done
exit 0
Let me know you have any issues.
Thanks
Rama