Firstly, my source table is not partitioned in any way. My understanding is that I still should be able to read the data in parallel.
I havce configured the Oracle connector in the following way:
Read mode: Select
Generate SQL at runtime: No
Select statement: SELECT <many fields> FROM <source_table> WHERE ROWNUM <= 10000
I have restricted the number of rows for the testing purposes
Enable partitioned reads: Yes
Partitioned reads method: Modulus
Not sure which method I should choose here
Column name for partitioned reads: ITEM_KEY
The field ITEM_KEY is Decimal(10)
When I run the job the following etries are in the log:
Code: Select all
The connector will use table <table_name> as input for the partitioned reads method.
The connector will run in parallel on 2 processing nodes.
The connector will use the following SELECT statement at runtime: SELECT <many fields> FROM <source_table> WHERE ROWNUM <= 10000
main_program: APT_SortedGroup2Operator::describeOperator nkeys: 7
...,0: Number of rows fetched on the current node: 10000.
...,1: Number of rows fetched on the current node: 10000.
Not sure whether the SQL statement should not be modified in some way, but maybe it happens only in a case of Generate SQL at runtime option set to Yes (which I'm planning to do in the future).
I would appreciate some help, also a bit more explanation how the partitioning works in a case of Oracle connector.
Thanks.