Hi ,
I wrote the algorithm for my logic below
Step1:
scheduling team will enter 3 parameters(Company,entity,datetimestamp) while running the DS job.
Step2:
we have to capture these papameters and we have to check in UNIX in specific floder(/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/)
is there any zip file name matching with above parameters Ex:company_entity_datetimestamp.zip(IBM_product_200606271614.zip)
Step3:
if parameters matches with the zip file name then we need to unzip in the same location.
Step4:
After unzip we will find more than 3 DAT files
Step5:
we have to combine all the DAT files into single file XML file
Step5:
After making into single file we need to remove white spaces.
Step6:
Finally we will use this XML file as input for datastage xml input stage.
please help to write code in UNIX for the above logic as i am still in learning stage in UNIX
Thaks in advance,
Chris
UNIX Code needed for complex logic
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 194
- Joined: Thu May 11, 2006 9:42 am
Step1:
scheduling team will enter 3 parameters(Company,entity,datetimestamp) while running the DS job.
Step2:
we have to capture these papameters and we have to check in UNIX in specific floder(/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/)
is there any zip file name matching with above parameters Ex:company_entity_datetimestamp.zip(IBM_product_200606271614.zip)
when you get those three parameters , concatenate them together and store it in a variable. Something like
Step3:
if parameters matches with the zip file name then we need to unzip in the same location.
You can either use the unzip or uncompress command to unzip the file
Step4:
After unzip we will find more than 3 DAT files
Again utilize the -f <file> functionality in unix to find those 3 DAT files
Step5:
we have to combine all the DAT files into single file XML file
once you have found those DAT files use the cat command to build your XML file. SOmething like
Step5:
After making into single file we need to remove white spaces.
use the sed functionality to remove the white spaces. Something like
Step6:
Finally we will use this XML file as input for datastage xml input stage.
this you can easily do using the the builtin stages.
If you are confused about any command that i have mentioned, just google it, you will get enough examples and explanations to get your buggy rolling.
Regards,
scheduling team will enter 3 parameters(Company,entity,datetimestamp) while running the DS job.
Step2:
we have to capture these papameters and we have to check in UNIX in specific floder(/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/)
is there any zip file name matching with above parameters Ex:company_entity_datetimestamp.zip(IBM_product_200606271614.zip)
when you get those three parameters , concatenate them together and store it in a variable. Something like
Code: Select all
FILE="/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/$1_$2_$3.zip"
export FILE
if [ -f $FILE ]
go to step3 else do what you have to
if parameters matches with the zip file name then we need to unzip in the same location.
You can either use the unzip or uncompress command to unzip the file
Step4:
After unzip we will find more than 3 DAT files
Again utilize the -f <file> functionality in unix to find those 3 DAT files
Step5:
we have to combine all the DAT files into single file XML file
once you have found those DAT files use the cat command to build your XML file. SOmething like
Code: Select all
if [ -f file1.DAT -a -f file2.DAT -a file3.DAT]
then
cat file1.DAT file2.DAT file3.DAT finalfile.XML
else
do what you have to do if all three files not found.
After making into single file we need to remove white spaces.
use the sed functionality to remove the white spaces. Something like
Code: Select all
cat finalfile.XML | sed 's/ //g' > NoSpaceFile.XML
Finally we will use this XML file as input for datastage xml input stage.
this you can easily do using the the builtin stages.
If you are confused about any command that i have mentioned, just google it, you will get enough examples and explanations to get your buggy rolling.
Regards,
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Try these
CODE:
#! /bin/ksh
set -vx
#Name of Script: chrisjones
COMPANY=$1
ENTITY=$2
PROCDTE=$3
SOURCE_PATH=/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/
chmod 770 ${SOURCE_PATH}
if [ -f ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip ];then
RETURN_CDE=$?
unzip -p ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip
else
echo 'File does not exits: $COMPANY_$ENTITY_PROCDTE.zip '
$RETURN_CDE
exit -9999
fi
#Combine all the DAT files into single file XML file
if [ -f ${SOURCE_PATH}/*.DAT ];then
for i in files1.DAT files2.DAT files3.DAT . . . . .
do
RETURN_CDE=$?
cat $i > ThreeFile.XML
done
else
echo 'DAT files does not exits'
$RETURN_CDE
exit -9999
fi
#remove white spaces from DATFile.XML and send it to NoSpaceFile.XML
if [ -f ${SOURCE_PATH}/DATFile.XML ];then
RETURN_CDE=$?
cat DATFile.XML | sed 's/ //g' > NoSpaceFile.XML
else
echo DATFile.XML files does not exits'
$RETURN_CDE
exit -9999
fi
#Now you have your final file NoSpaceFile.XML you may choose to remove
#These files
rm -f ${SOURCE_PATH}/files1.DAT
rm -f ${SOURCE_PATH}/files2.DAT
rm -f ${SOURCE_PATH}/files3.DAT
.
.
.
rm -f ${SOURCE_PATH}/DATFile.XML
exit 0
You run this scripts w/ three paramaters (chrisjones IBM product 200606271614)
I hope this will give you a good start. Re check all the syntax if are correct!!
Bryceson
CODE:
#! /bin/ksh
set -vx
#Name of Script: chrisjones
COMPANY=$1
ENTITY=$2
PROCDTE=$3
SOURCE_PATH=/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/
chmod 770 ${SOURCE_PATH}
if [ -f ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip ];then
RETURN_CDE=$?
unzip -p ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip
else
echo 'File does not exits: $COMPANY_$ENTITY_PROCDTE.zip '
$RETURN_CDE
exit -9999
fi
#Combine all the DAT files into single file XML file
if [ -f ${SOURCE_PATH}/*.DAT ];then
for i in files1.DAT files2.DAT files3.DAT . . . . .
do
RETURN_CDE=$?
cat $i > ThreeFile.XML
done
else
echo 'DAT files does not exits'
$RETURN_CDE
exit -9999
fi
#remove white spaces from DATFile.XML and send it to NoSpaceFile.XML
if [ -f ${SOURCE_PATH}/DATFile.XML ];then
RETURN_CDE=$?
cat DATFile.XML | sed 's/ //g' > NoSpaceFile.XML
else
echo DATFile.XML files does not exits'
$RETURN_CDE
exit -9999
fi
#Now you have your final file NoSpaceFile.XML you may choose to remove
#These files
rm -f ${SOURCE_PATH}/files1.DAT
rm -f ${SOURCE_PATH}/files2.DAT
rm -f ${SOURCE_PATH}/files3.DAT
.
.
.
rm -f ${SOURCE_PATH}/DATFile.XML
exit 0
You run this scripts w/ three paramaters (chrisjones IBM product 200606271614)
I hope this will give you a good start. Re check all the syntax if are correct!!
Bryceson
-
- Participant
- Posts: 194
- Joined: Thu May 11, 2006 9:42 am
Thanks
Hi Bryceson,
Thanks for your help.
One doubt
In the given path "/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/ "
we will have multiple zip files ,so we need to read each and every zip file name(IBM_Product_2000612.zip)and we have to check whether this matches with given parameter .
so please let me know how to read zip file name from data folder.
Thanks,
Chris.
SOURCE_PATH=/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/
chmod 770 ${SOURCE_PATH}
if [ -f ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip ];then
Thanks for your help.
One doubt
In the given path "/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/ "
we will have multiple zip files ,so we need to read each and every zip file name(IBM_Product_2000612.zip)and we have to check whether this matches with given parameter .
so please let me know how to read zip file name from data folder.
Thanks,
Chris.
SOURCE_PATH=/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/
chmod 770 ${SOURCE_PATH}
if [ -f ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip ];then
Bryceson wrote:Try these
CODE:
#! /bin/ksh
set -vx
#Name of Script: chrisjones
COMPANY=$1
ENTITY=$2
PROCDTE=$3
SOURCE_PATH=/opt/ibm/data/Projects/WORKSPACE/lnapps/BI/data/
chmod 770 ${SOURCE_PATH}
if [ -f ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip ];then
RETURN_CDE=$?
unzip -p ${SOURCE_PATH}/$COMPANY_$ENTITY_PROCDTE.zip
else
echo 'File does not exits: $COMPANY_$ENTITY_PROCDTE.zip '
$RETURN_CDE
exit -9999
fi
#Combine all the DAT files into single file XML file
if [ -f ${SOURCE_PATH}/*.DAT ];then
for i in files1.DAT files2.DAT files3.DAT . . . . .
do
RETURN_CDE=$?
cat $i > ThreeFile.XML
done
else
echo 'DAT files does not exits'
$RETURN_CDE
exit -9999
fi
#remove white spaces from DATFile.XML and send it to NoSpaceFile.XML
if [ -f ${SOURCE_PATH}/DATFile.XML ];then
RETURN_CDE=$?
cat DATFile.XML | sed 's/ //g' > NoSpaceFile.XML
else
echo DATFile.XML files does not exits'
$RETURN_CDE
exit -9999
fi
#Now you have your final file NoSpaceFile.XML you may choose to remove
#These files
rm -f ${SOURCE_PATH}/files1.DAT
rm -f ${SOURCE_PATH}/files2.DAT
rm -f ${SOURCE_PATH}/files3.DAT
.
.
.
rm -f ${SOURCE_PATH}/DATFile.XML
exit 0
You run this scripts w/ three paramaters (chrisjones IBM product 200606271614)
I hope this will give you a good start. Re check all the syntax if are correct!!
Bryceson
Thanks,
Chris Jones
Chris Jones
Now you're talking . . . . Are all the files the same name with diff date??
CODE:
if [ -f ${SOURCE_PATH}/*.zip ];then
1. for i in files1.zip files2.zip files3.zip files4.zip . . . files15.zip
OR
2. for i in *.zip
do
RETURN_CDE=$?
unzip -p ${SOURCE_PATH}/$i
done
else
echo 'Zip Files does not exits'
$RETURN_CDE
exit -9999
fi
Good luck!!
Bryceson
CODE:
if [ -f ${SOURCE_PATH}/*.zip ];then
1. for i in files1.zip files2.zip files3.zip files4.zip . . . files15.zip
OR
2. for i in *.zip
do
RETURN_CDE=$?
unzip -p ${SOURCE_PATH}/$i
done
else
echo 'Zip Files does not exits'
$RETURN_CDE
exit -9999
fi
Good luck!!
Bryceson