Getting a particular record (looping)

[wittyme]

I have a requirement where the source file has data like

#Z_ABC_FILE
#20121114123456
file1=ABC_SD_1235.txt
file2=
DONE
#Z_ABC_FILE
#20121114123456
file1=DEF_SD_4545.txt
file2=
DONE
#Z_ABC_FILE
#20121114123456
file1=ABC_SD_3454.txt
file2=
DONE
#Z_ABC_FILE
#20121114123456
file1=ABC_FI_3454.txt
file2=
DONE


How to get file1 values or that particular record. I want to know if it can be done with looping.

[SURA]

Do you need the value from file1 alone, down the line the list will increase?

[jwiles]

Wouldn't a simple if-then-else statement work, or is there some additional requirements not detailed in your question?

sv1: if left(inlink.record,6) = "file1" then logic_to_retrieve_the_value else ""

with a simple constraint to only output when data is retrieved?

sv1 <> ""

Regards,

[SURA]

Wouldn't a simple if-then-else statement work, or is there some additional requirements not detailed in your question?

Can't use the col[1,n] = "file1" in the downstream TFM ?

[jwiles]

Something wrong with left()?

Well, I guess you could get around a requirement like that by using filter and column import stages, or maybe just a sed or awk script in the filter option of sequential file stage.

Regards,

[bhasds]

Hi wittyme,

You may also use a constraint in the transformer-

Code: Select all

Field(ToT.Col1,"=",1) ="file1"

or
a grep command in the filter option of the sequential file stage

Code: Select all

grep "file1"

[bfennell]

I have a requirement where the source file has data like

(((sample data)))

How to get file1 values or that particular record. I want to know if it can be done with looping.

I notice in your sample data all fields appear all the time.
It looks like "DONE<NEWLINE>" is your record separator and
"<NEWLINE>" and "=" are your field separators. If my observations are correct, (consistent with all data samples you have) it may help you to look at your data this way and look for features in either a language like sed, awk, BASIC, java or a tool like SQLLDR or in DataStage itself that allows you to set your (multi-byte) record separators and set your field separators. You can then identify fields by field number (again if my observations are correct). You may also want to think in terms of set transformations or events instead of loops. The arrival of each record is a record event, and may be handled by a record handler. The resulting set is a group of records each a set of fields where each result record is transformed from an incoming record. OO languages like Java tend to think in the event model, algebraic languages like SQL tend to think in the set transformation model. This may help you find the language or tool features you want.

Sorry if my answer is less specific than you may have wanted, I am still learning DataStage myself.

Brian Fennell

[wittyme]

I read the sequential file as one record and filtered grep = "file1" and in transformer I wrote functions to get the value.

I didn't know in sequential file filter condition we can put grep="file1". Something new I learned

I have done this before using unix commands in sequence job but I want to use parallel job this time so put the question here. Thanks for help!

[bhasds]

Hi wittyme,

in transformer I wrote functions to get the value

If you are using transformer only to get the value of file1 then you can use the below in the filter option-

Code: Select all

grep "File1"|nawk -F"=" '{print $2}'

[ray.wurlod]

Yes, but why open another operating system shell when the Field() function within DataStage will do just as well, and more efficiently?

Constraint expression: Left(InLink.TheString,6) = "file1="
Derivation expression: Field(InLink.TheString, "=", 2, 9999)