Hi.
I need to know if I can pass an array as a parameter (from job to another job), and how.
The answer will be very appreciated.
Thanks in advance.
Guillermo P. Barsky
Buenos Aires - Argentina
Passing an array as parameter
Moderators: chulett, rschirm, roy
Only if you use a text delimiter and set the parameter type as STRING. You will have some limitation as to the size, of which I'm not sure.
I'm guessing from all of your questions that you are attempting to build your own job control. As to why you wish to pass an array from one job to the next, I don't know, and I'm scared to ask.
Kenneth Bland
I'm guessing from all of your questions that you are attempting to build your own job control. As to why you wish to pass an array from one job to the next, I don't know, and I'm scared to ask.
Kenneth Bland
Ken:
Glad of answering you.
My process consists of a job (CHKSAP) that checks in a SAP table (orders for ETL) if there is something to execute. If so, then the CHKSAP triggers the MasterJob job, which has the logic to process the orders. These orders are created in SAP by a SAP user. And for each order, a different MasterJob is triggered.
Inside job CHKSAP I am loading an array which mainly has:
- Code of entity (a code for Visa, Diners, Mastercard, etc).
- Name of the entity (a short description for the entity).
And all I wanted is to pass such array to all the MasterJobs that I have to execute.
Perhaps there is a function to convert an array to a string or to a list, and the inverse function to convert a string or a list to an array. Does it really exist ?
Thanks Ken. You, Kim, Ray and this forum are of a great help for me [^]
Bye [:)]
Guillermo P. Barsky
Buenos Aires - Argentina
Glad of answering you.
My process consists of a job (CHKSAP) that checks in a SAP table (orders for ETL) if there is something to execute. If so, then the CHKSAP triggers the MasterJob job, which has the logic to process the orders. These orders are created in SAP by a SAP user. And for each order, a different MasterJob is triggered.
Inside job CHKSAP I am loading an array which mainly has:
- Code of entity (a code for Visa, Diners, Mastercard, etc).
- Name of the entity (a short description for the entity).
And all I wanted is to pass such array to all the MasterJobs that I have to execute.
Perhaps there is a function to convert an array to a string or to a list, and the inverse function to convert a string or a list to an array. Does it really exist ?
Thanks Ken. You, Kim, Ray and this forum are of a great help for me [^]
Bye [:)]
Guillermo P. Barsky
Buenos Aires - Argentina
Well, you could always use the statement:
CONVERT @AM TO "|" IN YOURARRAY
This converts the attribute (row) marks in the array to pipes. You could also use the CHANGE function or the EREPLACE function. The point is to change @AM to something textual and then vice versa.
Why have an array when you could use discrete parameters?
Kenneth Bland
CONVERT @AM TO "|" IN YOURARRAY
This converts the attribute (row) marks in the array to pipes. You could also use the CHANGE function or the EREPLACE function. The point is to change @AM to something textual and then vice versa.
Why have an array when you could use discrete parameters?
Kenneth Bland
Ken:
I need an array because this array has codes for 44 credit cards, and about of 55 codes for banks (codes and descriptions).
Another way to solve this is to make a read of the array inside the MasterJob. But I wanted to avoid this, because this loading will be executing in every MasterJob that starts up. And there may be lot of MasterJobs running in a day (not all together; at the most about 4).
I hope this helps to figure out a good solution.
Bye, and have a nice week end.
Guillermo P. Barsky
Buenos Aires - Argentina
I need an array because this array has codes for 44 credit cards, and about of 55 codes for banks (codes and descriptions).
Another way to solve this is to make a read of the array inside the MasterJob. But I wanted to avoid this, because this loading will be executing in every MasterJob that starts up. And there may be lot of MasterJobs running in a day (not all together; at the most about 4).
I hope this helps to figure out a good solution.
Bye, and have a nice week end.
Guillermo P. Barsky
Buenos Aires - Argentina
Guillermo
Ken told you right. If you bring them in as one long string with a separator like '|'. Next convert the '|' into @AM then you have a dynamic array. Each field is addressable by Ary, Ary and so on. Instead if using () like Ary(1), dynamic arrays use .
Kim.
Kim Duke
DsWebMon - Monitor DataStage over the web
www.Duke-Consulting.com
Ken told you right. If you bring them in as one long string with a separator like '|'. Next convert the '|' into @AM then you have a dynamic array. Each field is addressable by Ary, Ary and so on. Instead if using () like Ary(1), dynamic arrays use .
Kim.
Kim Duke
DsWebMon - Monitor DataStage over the web
www.Duke-Consulting.com
Here's your solution:
You can setup a named COMMON and put variables there. In your first transformer, in the before-transformer routine box, put a subroutine call. This subroutine defines a named COMMON and has a variable that is your array. Your subroutine will open either a sequential (my preference) or hash file and read into the array your data. You must do it in a transformer before subroutine and NOT the job before subroutine, as they don't share the same memory. Something like the following:
COMMON /YourArrayLabel/ YourArray
ArrayFile = Arg1
OpenFailed = @TRUE
OPEN ArrayFile TO F.FILE Then
OpenFailed = @FALSE
End
If OpenFailed Then
Call DSLogFatal("Unable to open the file ":ArrayFile , "Msg")
End Else
*
* Select your rows and read them into your array
*
End
Now write a function that simply declares that same named COMMON and returns the array. In the transformer that needs access to this array, simply use the function like a variable. Or, if you want, write a function that has this same named COMMON and can just simply do whatever array searching required.
COMMON /YourArrayLabel/ YourArray
Ans = YourArray
Or if you want to process logic in a function:
COMMON /YourArrayLabel/ YourArray
CreditCardCode = Arg1
Locate CreditCardCode IN YourArray SETTING PSN Then
ValueICareAbout = YourArray ;* Get corresponding value in array
End Else
ValueICareAbout = -1 ;* Not found default value
End
Ans = ValueICareAbout
The overhead in doing this your jobs is negligible. The initial read of values into the array is minor. The trick will be dynamically generating the file that all of the jobs will use.
Kenneth Bland
You can setup a named COMMON and put variables there. In your first transformer, in the before-transformer routine box, put a subroutine call. This subroutine defines a named COMMON and has a variable that is your array. Your subroutine will open either a sequential (my preference) or hash file and read into the array your data. You must do it in a transformer before subroutine and NOT the job before subroutine, as they don't share the same memory. Something like the following:
COMMON /YourArrayLabel/ YourArray
ArrayFile = Arg1
OpenFailed = @TRUE
OPEN ArrayFile TO F.FILE Then
OpenFailed = @FALSE
End
If OpenFailed Then
Call DSLogFatal("Unable to open the file ":ArrayFile , "Msg")
End Else
*
* Select your rows and read them into your array
*
End
Now write a function that simply declares that same named COMMON and returns the array. In the transformer that needs access to this array, simply use the function like a variable. Or, if you want, write a function that has this same named COMMON and can just simply do whatever array searching required.
COMMON /YourArrayLabel/ YourArray
Ans = YourArray
Or if you want to process logic in a function:
COMMON /YourArrayLabel/ YourArray
CreditCardCode = Arg1
Locate CreditCardCode IN YourArray SETTING PSN Then
ValueICareAbout = YourArray ;* Get corresponding value in array
End Else
ValueICareAbout = -1 ;* Not found default value
End
Ans = ValueICareAbout
The overhead in doing this your jobs is negligible. The initial read of values into the array is minor. The trick will be dynamically generating the file that all of the jobs will use.
Kenneth Bland
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The function for converting a pipe-delimited array to a field-mark delimited array is, as Ken suggested, Convert (assuming there are no pipe characters in your data). The function for converting back again is also convert.
DynArray = Convert("|", @AM, PipeArray)
PipeArray = Convert(@AM, "|", DynArray)
You can access individual items in a dynamic array using the angle-bracket notation, as Ken also suggested.
You can access individual items in any delimited list using the Field function. For example:
SecondCardNo = Field(PipeArray, "|", 2, 1)
Everything you want to do can be done with any delimited string as the "array".
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
PS Avoid using COMMON if you are planning to migrate server jobs into the Parallel Extender environment, or even if you are planning to use inter-process row buffering in server jobs (DS 6.x and later).
DynArray = Convert("|", @AM, PipeArray)
PipeArray = Convert(@AM, "|", DynArray)
You can access individual items in a dynamic array using the angle-bracket notation, as Ken also suggested.
You can access individual items in any delimited list using the Field function. For example:
SecondCardNo = Field(PipeArray, "|", 2, 1)
Everything you want to do can be done with any delimited string as the "array".
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
PS Avoid using COMMON if you are planning to migrate server jobs into the Parallel Extender environment, or even if you are planning to use inter-process row buffering in server jobs (DS 6.x and later).
Thanks to all for your cooperation.
What about using a list instead an array ? As I could see, I can pass a list, and I can address a specific element of the list. As far as I know, I have the following elements to handle a list:
1) To populate the list, I use namelist = code of bank, and this will add each code of bank that I need.
2) With dcount(namelist, @FM) I can retrieve the number of elements in the list.
3) And to retrieve a list, I can make a FOR-NEXT cycle looking for namelist.
4) Is there any way to delete elements from the list ?
Are all the points that I'm saying right ?
Guillermo P. Barsky
Buenos Aires - Argentina
What about using a list instead an array ? As I could see, I can pass a list, and I can address a specific element of the list. As far as I know, I have the following elements to handle a list:
1) To populate the list, I use namelist = code of bank, and this will add each code of bank that I need.
2) With dcount(namelist, @FM) I can retrieve the number of elements in the list.
3) And to retrieve a list, I can make a FOR-NEXT cycle looking for namelist.
4) Is there any way to delete elements from the list ?
Are all the points that I'm saying right ?
Guillermo P. Barsky
Buenos Aires - Argentina
3. Do not use a For-Next loop to search a list. Imagine a 2-D array, where row 1 is a series of infinite columns of codes and row 2 is a correlated/associate series of infinite columns of values. To locate code X in the list of codes in row 1, you use the LOCATE statement:
LOCATE X IN ARRAY SETTING POSITION THEN
CORRESPONDING_VALUE = ARRAY
END ELSE
CORRESPONDING_VALUE = ??? ;* NOT FOUND DEFAULT
END
4. To delete an element from an array, use the DEL command:
LOCATE X IN ARRAY SETTING POSITION THEN
DEL ARRAY ;* GET RID OF CODE
DEL ARRAY ;* GET RIDE OF VALUE
END ELSE
* NOTHING TO DELETE!
END
Kenneth Bland
LOCATE X IN ARRAY SETTING POSITION THEN
CORRESPONDING_VALUE = ARRAY
END ELSE
CORRESPONDING_VALUE = ??? ;* NOT FOUND DEFAULT
END
4. To delete an element from an array, use the DEL command:
LOCATE X IN ARRAY SETTING POSITION THEN
DEL ARRAY ;* GET RID OF CODE
DEL ARRAY ;* GET RIDE OF VALUE
END ELSE
* NOTHING TO DELETE!
END
Kenneth Bland
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There is a small logic fault in Ken's code, which will prevent it from working. LOCATE searches for a "whole element" match; as soon as you add a second value to the field, LOCATE won't find the first value. One alternative is to use FIND rather than LOCATE. The other alternative is to use two field-mark delimited arrays. Then Guillermo's points become as follows.
1. Add items to the arrays (which Guillermo is calling lists) using "-1" notation. This is correct, with the caveat that the arrays must be initialized to "".
CodeArray<-1> = NewCode
ValueArray<-1> = NewValue
2. Use LOCATE to search the list and insert elements, but ensure you maintain both lists equivalently (that is, always insert or replace or delete the same element in both lists).
3. The DEL statement is how you remove an element from a list. There is also a DELETE() function, but let's show the DEL statement.
LOCATE Code IN CodeArray SETTING FieldNo
Then
DEL CodeArray<FieldNo>
DEL ValueArray<FieldNo>
End
Else
* Code not found in list, nothing to delete.
* May need to log a warning.
End
4. You can use LOCATE and INS, rather than the "-1" notation, to preserve sorted order in a list. This means that subsequent searches will require on average N/2 list elements to be searched.
LOCATE NewCode IN CodeArray BY "AR" Setting FieldNo
Else
NULL ; * do nothing yet
End
* We're doing this INS whether the NewCode was found or not.
INS NewCode BEFORE CodeArray<FieldNo>
INS NewValue BEFORE ValueArray<FieldNo>
Then, to search (for example for deletion):
LOCATE Code IN CodeArray BY "AR" SETTING FieldNo
Then
DEL CodeArray<FieldNo>
DEL ValueArray<FieldNo>
End
Else
* Code not found in list, nothing to delete.
* May need to log a warning.
End
The "BY" clause asserts that the list is sorted. "AR" = "ascending, right-justified". You can also use "AL", "DL" or "DR".
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
1. Add items to the arrays (which Guillermo is calling lists) using "-1" notation. This is correct, with the caveat that the arrays must be initialized to "".
CodeArray<-1> = NewCode
ValueArray<-1> = NewValue
2. Use LOCATE to search the list and insert elements, but ensure you maintain both lists equivalently (that is, always insert or replace or delete the same element in both lists).
3. The DEL statement is how you remove an element from a list. There is also a DELETE() function, but let's show the DEL statement.
LOCATE Code IN CodeArray SETTING FieldNo
Then
DEL CodeArray<FieldNo>
DEL ValueArray<FieldNo>
End
Else
* Code not found in list, nothing to delete.
* May need to log a warning.
End
4. You can use LOCATE and INS, rather than the "-1" notation, to preserve sorted order in a list. This means that subsequent searches will require on average N/2 list elements to be searched.
LOCATE NewCode IN CodeArray BY "AR" Setting FieldNo
Else
NULL ; * do nothing yet
End
* We're doing this INS whether the NewCode was found or not.
INS NewCode BEFORE CodeArray<FieldNo>
INS NewValue BEFORE ValueArray<FieldNo>
Then, to search (for example for deletion):
LOCATE Code IN CodeArray BY "AR" SETTING FieldNo
Then
DEL CodeArray<FieldNo>
DEL ValueArray<FieldNo>
End
Else
* Code not found in list, nothing to delete.
* May need to log a warning.
End
The "BY" clause asserts that the list is sorted. "AR" = "ascending, right-justified". You can also use "AL", "DL" or "DR".
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
Ray, care to revisit my example again? [:)] When attribute 1 has a multivalue list of codes, using LOCATE with a specified attribute switches it to a multivalue search, instead of an attribute search. Were you to specify an attribute and a value, LOCATE does a subvalue search.
My example describes a 2-D matrix, not requiring 2 separate attribute delimited arrays. Most people think of rows as unique elements, whereby my example shows a pivoted thinking of columns as infinite unique elements. Example:
Attribute Column 1 Column 2 Column 3 Column 4
(Code) A B C D
(Value) 1 2 3 4
(Begin Date) 2003-01-01 2003-02-01 2003-03-01 2003-04-04
If you are searching for code "C", the syntax is:
LOCATE "C" IN ARRAY SETTING POSITION THEN
VALUE = ARRAY
BEGIN_DATE = ARRAY
END ELSE
VALUE = -1 ;* DEFAULT NOT FOUND
BEGIN_DATE = "1799-01-01" ;* DEFAULT NOT FOUND
END
The stated problem deals with being able to pass around an array. Instead I'm suggesting a matrix (2-D or 3-D array), because otherwise you have to pass around many arrays, given your example. Now of course, I hate matrices, something about 5 years of college math drove the love of mathematics out of my head. [xx(] So, for the stated problem of passing arrays in job parameters he should use a matrix to avoid lots of parameters. However, I hope he looks again at my COMMON approach, as passing arrays in parameters doesn't sound like a good idea.
Kenneth Bland
My example describes a 2-D matrix, not requiring 2 separate attribute delimited arrays. Most people think of rows as unique elements, whereby my example shows a pivoted thinking of columns as infinite unique elements. Example:
Attribute Column 1 Column 2 Column 3 Column 4
(Code) A B C D
(Value) 1 2 3 4
(Begin Date) 2003-01-01 2003-02-01 2003-03-01 2003-04-04
If you are searching for code "C", the syntax is:
LOCATE "C" IN ARRAY SETTING POSITION THEN
VALUE = ARRAY
BEGIN_DATE = ARRAY
END ELSE
VALUE = -1 ;* DEFAULT NOT FOUND
BEGIN_DATE = "1799-01-01" ;* DEFAULT NOT FOUND
END
The stated problem deals with being able to pass around an array. Instead I'm suggesting a matrix (2-D or 3-D array), because otherwise you have to pass around many arrays, given your example. Now of course, I hate matrices, something about 5 years of college math drove the love of mathematics out of my head. [xx(] So, for the stated problem of passing arrays in job parameters he should use a matrix to avoid lots of parameters. However, I hope he looks again at my COMMON approach, as passing arrays in parameters doesn't sound like a good idea.
Kenneth Bland