Help needed on Requirement - File Processing

[wittyme]

I have a requirement where the source file is a fixed length file.

If the source file has data as only datetimestamp like 2012081503:44:54
then I should not process the file and should not go to next jobs and finish sequence as successfully and I have to process the source file only if there is anything else other than the timestamp (real data). Can this be achieved in sequence before I kick the jobs?

Just For Information, the date timestamp indicates that the file is empty and SAP team said their system cannot create an empty file and should have some character in it. Not sure how true it is?

[akarsh]

Hi,

Read the file and in a transformer check if it contains a date value in satage variable. if so move some flag value in seq file and in transformer use the property abort after no of rows as 1 row.

file -> Tfm- Actual file processing
|
seq file (which contain some flag, if so ABORT)

Hope u can abort the job for this requirement.

[wittyme]

The jobs needs to be completed successfully. Can I do this in sequence before any job run.

[ArndW]

Offhand I'd program this as a UNIX/DOS exit from the job sequence, using "wc -l" on the file to get the number of lines, or if the file really needs to be checked then a "sed" or "awk" command to parse out the date.

[wittyme]

any commands on Sed or Awk. I am searching online.

[bhasds]

As you said if there is no data then file contains DateTimestamp 20120821 03:44:54.

For this, we can do the following in the Job sequence-

Execute command-

Command-

cat <FileName> | nawk -F" " '{print $1}'


Trigger in Execute command acivity-

a)Link1-This can be mapped to the job to be triggered when there will be data in the file.

We can choose the expression type as custom-(conditional) and the expression is-

Code: Select all

Trim(Convert(@FM,"",Execute_Command_34.$CommandOutput )) <> Oconv(Date(),"DY") :Oconv(Date(),"DM[2]") :Oconv(Date(),"DD[2]")

b)Link2- This can be mapped to a routine activity where we can call the routine UtilityMessageToLog and give suitable
subject to the value expression in the Argument grid.


We can choose the expression type as custom-(conditional) and the expression is-

Code: Select all

Trim(Convert(@FM,"",Execute_Command_34.$CommandOutput )) = Oconv(Date(),"DY") :Oconv(Date(),"DM[2]") :Oconv(Date(),"DD[2]") 

So if the file has DateTimestamp then the trigger in execute command will trig the routine activity and if the file has data then the job will be triggered.

[wittyme]

I am using routine in the sequence.

Get head -1 and then get first 17 characters(YYYYMMDDHH:MM:SS)

See if there is any ":" if yes then don't go to next step.

I saw the datatype and in first 17 characters there should never be ":"

While doing this I figured out that if I run stat command the block size is always zero for the files with datetimestamp and always 17 bytes (0KB). I can check with file size too.

Which method is more appropriate?

[wittyme]

Hey Bhasds,

I will try that and see. That looks more promising

[bhasds]

Hi wittyme,

If your DateTimestamp format is YYYYMMDDHH:MM:SS then you can use the below command in execute command-

Code: Select all

cat <FileName> | cut -c  1-8

This will give you only the date and the rest of the thing of my previous post should work as it is.

[wittyme]

Thanks,

I can just use the condition if it has data and ignore the other link. I don't have to abort job if the file has datetimestamp. I just not have to process it

[bhasds]

Hi wittyme,

Yes, you can do that.My logic also not going to abort the job at any cost.If there is no file then it will not hit the job and finish the sequencer successfully.

[chulett]

Any chance you could just check for one row in the file? Or could you get a one row file that is legitimate data (i.e. not just a timestamp) ?

[wittyme]

I cannot do with one row as we can expect one row in the file.

Bhasds,
I cannot go with your logic too as in one of the file the first field is date.
I have to go with finding : in the date timestamp yyyymmddhh:mm:ss as from the data types we never get : in the first 16 characters.

I used a routine for that.

[bhasds]

Hi wittymy,

Its great that you have used routine. However, if you want to do it using sequence activity itself, you can still do it as below-

1.ExecuteCommand1 -

Command-

Code: Select all

echo `date +%Y%m%d%H:%M:%S`

2.ExecuteCommand2 -

Command-

Code: Select all

cat <FileName> | head -1 

Trigger in Execute command acivity-


We can choose the expression type as custom-(conditional) and the expression is-

Code: Select all

Trim(Convert(@FM,"",ExecuteCommand1.CommandOutput)) <> Trim(Convert(@FM,"",ExecuteCommand2.CommandOutput))

The above trigger will only check for DateTimestamp as a first record.