Getting the total number or records from a dataset
Moderators: chulett, rschirm, roy
Getting the total number or records from a dataset
Is there a command in orchadmin to get this? I would like to script this.
Thanks
Thanks
OK here's the output from orchadmin ll count but it seems to report record numbers in Kilobytes?? Kind of weird.
Let me know if you would know how to figure out the total number of records. Many Thanks!
Name: /loads/work/etl/outbound/datasets/cust_data_set.ds
Version: ORCHESTRATE V8.0.1 DM Block Format 6.
Time of Creation: 06/06/2008 11:40:46
Number of Partitions: 2
Number of Segments: 1
Valid Segments: 1
Preserve Partitioning: false
Segment Creation Time:
0: 06/06/2008 11:40:46
Partition 0
node : node1
records: 3120K
blocks : 41607
bytes : 5417M
files :
Segment 0 :
/loads/work/etl/outbound/datasets/cust_data_set.ds.srv_etl.mtl01dds01.yesbank.com.0000.0000.0000.274a.ca12a57e.0000.1578d353 5453M bytes
total : 5453M bytes
Partition 1
node : node2
records: 3120K
blocks : 41607
bytes : 5417M
files :
Segment 0 :
/loads/work/etl/outbound/datasets/cust_data_set.ds.srv_etl.mtl01dds01.yesbank.com.0000.0001.0000.274a.ca12a57e.0001.dcd6e488 5453M bytes
total : 5453M bytes
Totals:
records : 6240K
blocks : 83214
bytes : 10834M
filesize: 10907M
min part: 5453M
max part: 5453M
Schema:
record
( RECNUM: int64;
DATE_INSERTED: timestamp;
RECORD_CODE: int32;
BANK_NBR: string[max=4];
CUST_NAME_KEY: string[max=12];
TIE_BREAKER: int32;
CS_POINTER: string[max=2];
ZIP_CODE: string[max=9];
ZIP_CODE_DPBC: string[max=2];
VARB_LEN: int32;
LINE_CODE_1: string[max=1];
NAME_ADDR_LINE_1: string[max=40];
LINE_CODE_2: string[max=1];
NAME_ADDR_LINE_2: string[max=40];
LINE_CODE_3: string[max=1];
NAME_ADDR_LINE_3: string[max=40];
LINE_CODE_4: string[max=1];
NAME_ADDR_LINE_4: string[max=40];
LINE_CODE_5: string[max=1];
NAME_ADDR_LINE_5: string[max=40];
LINE_CODE_6: string[max=1];
NAME_ADDR_LINE_6: string[max=40];
GREETING: string[max=6];
COMPLIMENT: string[max=6];
ALT_BANK_NBR: string[max=4];
SSN_TAXID_NBR: int32;
ALT_NAME_KEY: string[max=12];
ALT_TIE_BREAKER: int32;
SSN_TAXID_CODE: string[max=1];
REG_P_CODE: int32;
HOME_PHONE: string[max=10];
WORK_PHONE: string[max=10];
DATE_OF_BIRTH: int32;
SPEC_HAND: string[max=1];
RESTRICTED: string[max=1];
REG_O_CODE: string[max=1];
GEOG_CODE: string[max=6];
LOCATION: string[max=3];
COUNTY: string[max=3];
USER_CODE1: string[max=2];
USER_CODE2: string[max=2];
USER_CODE3: string[max=2];
LAS_MAX_LIABILITY: decimal[11,2];
IRA_PART_ST: string[max=1];
IRA_DSTRB_WH_FED: string[max=1];
IRA_DSTRB_WH_OTH: string[max=1];
IRA_CONTRB_LIMIT: string[max=4];
PRV_TAXID_NBR: int32;
PRV_TAXID_CODE: string[max=1];
DT_LST_MAINT_TAX: int32;
DT_LST_MAINT: int32;
DT_ELIGIBLE_DELETE: int32;
USER_CODE4: string[max=2];
USER_CODE5: string[max=2];
USER_CODE6: string[max=2];
FORCE_NOTICE: string[max=1];
DATE_LAST_BNOTICE: int32;
NBR_BNOTICES: int32;
DATE_LAST_W9: int32;
NBR_W9S: int32;
INT_WH_FED: string[max=1];
INT_WH_OTH: string[max=1];
TIN_CERTIFIED: string[max=1];
DATE_CERTIFIED: int32;
DATE_REQ_WITHHOLDING: int32;
TAX_EXEMPT: string[max=1];
OCCUPATION: string[max=35];
COUNTRY: string[max=2];
ID_TYPE: string[max=1];
ID_ISSUED_BY: string[max=2];
ID_NUMBER: string[max=24];
REC_CREATE_PGM: string[max=8];
CUSTOMER_LEVEL: string[max=1];
CALL_AT_HOME: string[max=1];
CALL_AT_WORK: string[max=1];
COM_NBR_1: string[max=5];
COM_NBR_2: string[max=5];
COM_NBR_3: string[max=5];
CTR_EXEMPT: string[max=1];
CRA_GROSS_SALES: decimal[13,2];
CRA_GROSS_INCOME: decimal[13,2];
CRA_TOTAL_ASSETS: decimal[13,2];
CRA_TOTAL_LIAB: decimal[13,2];
CRA_NET_WORTH: decimal[13,2];
CRA_DATE_LAST_FIN_STMT: int32;
CRA_DATE_VERIFIED: int32;
CRA_FIN_STMT_REQUIRED: string[max=1];
DATE_ADDED: int32;
TIME_ADDED: int32;
USER_ADDED: string[max=8];
TERM_ADDED: string[max=4];
SIC: string[max=6];
DATE_FIRST_ROTH_IRA: int32;
DATE_FIRST_ROTH_CONV: int32;
DATE_FIRST_SIMPLE_IRA: int32;
RESIDENCY_CODE: string[max=2];
DATE_LAST_CREDIT_RPT: int32;
CUST_TYPE: string[max=1];
CREDIT_RATING: string[max=6];
CREDIT_RATED_BY: string[max=8];
OFFICER: string[max=8];
BRANCH: string[max=4];
FED_WTHLD_PERCENT: decimal[9,6];
ST_WTHLD_PERCENT: decimal[9,6];
IRA_CONTR_ANNUAL_MAX: decimal[13,2];
YEARS_OF_SCHOOL: int32;
MARITAL_STATUS: string[max=1];
GENDER: string[max=1];
RACE: int32;
OTHER_RACE_DESC: string[max=20];
FAX_NUMBER: string[max=10];
E_MAIL_ADDRESS: string[max=40];
AGES_OF_DEPENDENTS: string[max=40];
CUST_SOURCE: string[max=1];
ADDL_SOURCE_INFO: string[max=40];
SECURITY_ACCESS_CDE: string[max=20];
DATE_OF_DEATH: int32;
US_CITIZEN: string[max=1];
FRGN_STAT_CHANGE: string[max=1];
FRGN_STAT_US_INT: string[max=1];
FRGN_STAT_US_DIV: string[max=1];
FRGN_STAT_US_BROKER: string[max=1];
BUS_TYPE: string[max=1];
BUS_SUB_TYPE: string[max=1];
BUS_PHONE_NBR: string[max=10];
BUS_ALT_PHONE_NBR: string[max=10];
BUS_STARTED_DATE: int32;
BUS_RESOLUTION_DATE: int32;
BUS_WIRE_TRF_RES_DATE: int32;
BUS_INCORPORATION_DATE: int32;
BUS_INCORPORATION_ST: string[max=2];
BUS_SIG_OPEN_DEPOSIT: int32;
BUS_SIG_WD_DEP_FUNDS: int32;
BUS_SIG_BORROW_FUNDS: int32;
BUS_SIG_PLEDGE_COLL: int32;
BUS_SIG_OPEN_SAFE_DEP: int32;
BUS_SIG_RLSE_NOTICE: int32;
BUS_ORAL_IN_PERSON: string[max=1];
BUS_ORAL_BY_PHONE: string[max=1];
BUS_WRITING_ORIG_SIG: string[max=1];
BUS_FAX_TRANSMISSION: string[max=1];
BUS_ELEC_TRANSMISSION: string[max=1];
BUS_CALL_REPET: string[max=1];
BUS_CALL_NONREPET: string[max=1];
BUS_WTRF_SECURITY: string[max=1];
COMM_TRF_TYPE: string[max=2];
COMM_TRF_LIMIT: decimal[13,2];
COMM_TRF_DLY_LIMIT: decimal[13,2];
COMM_TRF_ADV_DAYS: int32;
DATE_LAST_CHEX_RPT: int32;
CHEX_RPT_BY: string[max=8];
DATE_LAST_ACTY: int32;
DATE_LAST_CONT: int32;
AFFILIATION_CODE: string[max=8];
WORK_PHONE_EXTN: int32;
ID_ISSUE_DATE: int32;
ID_EXP_DATE: int32;
ID_VRFD_IND: string[max=1];
ID_VRFD_DATE: int32;
ID_VRFD_OFFICER: string[max=8];
ID_VRFD_METH: int32;
ID_VRFD_DESC: string[max=40];
ID_GOV_LST_MTCH: string[max=1];
BEACON_SCORE: string[max=6];
FRAUD_ADV_INDEX: string[max=3];
FRAUD_VICTIM_IND: string[max=1];
SAFESCAN: string[max=1];
AMER_IND_AK_NATIVE: string[max=1];
ASIAN: string[max=1];
BLK_AFRICAN_AMER: string[max=1];
NATIVE_HI_PAC_ISL: string[max=1];
WHITE: string[max=1];
ETHNICITY: string[max=2];
NO_INFO_PROVIDED: string[max=1];
RISK_CLASS: string[max=8];
CLASS_DATE: int32;
CLASS_OFFICER: string[max=8];
MSA_CODE: int32;
NY_TIN: string[max=9];
DATE_LAST_ADDR_CHG: int32;
AML_RISK_LVL: int32;
TAX_PROOF_IND: string[max=1];
TF_FULL_NAME: string[max=40];
TF_CITY: string[max=40];
TF_STATE: string[max=2];
TF_SSN_TAXID: string[max=9];
)
Embedded ORCHESTRATE system configuration:
{
node "node1"
{
fastname "mtl01dds01.yesbank.com"
pools ""
resource disk "/loads/work/etl/outbound/datasets" {pools ""}
resource scratchdisk "/scratch" {pools ""}
}
node "node2"
{
fastname "mtl01dds01.yesbank.com"
pools ""
resource disk "/loads/work/etl/outbound/datasets" {pools ""}
resource scratchdisk "/scratch" {pools ""}
}
}
Let me know if you would know how to figure out the total number of records. Many Thanks!
Name: /loads/work/etl/outbound/datasets/cust_data_set.ds
Version: ORCHESTRATE V8.0.1 DM Block Format 6.
Time of Creation: 06/06/2008 11:40:46
Number of Partitions: 2
Number of Segments: 1
Valid Segments: 1
Preserve Partitioning: false
Segment Creation Time:
0: 06/06/2008 11:40:46
Partition 0
node : node1
records: 3120K
blocks : 41607
bytes : 5417M
files :
Segment 0 :
/loads/work/etl/outbound/datasets/cust_data_set.ds.srv_etl.mtl01dds01.yesbank.com.0000.0000.0000.274a.ca12a57e.0000.1578d353 5453M bytes
total : 5453M bytes
Partition 1
node : node2
records: 3120K
blocks : 41607
bytes : 5417M
files :
Segment 0 :
/loads/work/etl/outbound/datasets/cust_data_set.ds.srv_etl.mtl01dds01.yesbank.com.0000.0001.0000.274a.ca12a57e.0001.dcd6e488 5453M bytes
total : 5453M bytes
Totals:
records : 6240K
blocks : 83214
bytes : 10834M
filesize: 10907M
min part: 5453M
max part: 5453M
Schema:
record
( RECNUM: int64;
DATE_INSERTED: timestamp;
RECORD_CODE: int32;
BANK_NBR: string[max=4];
CUST_NAME_KEY: string[max=12];
TIE_BREAKER: int32;
CS_POINTER: string[max=2];
ZIP_CODE: string[max=9];
ZIP_CODE_DPBC: string[max=2];
VARB_LEN: int32;
LINE_CODE_1: string[max=1];
NAME_ADDR_LINE_1: string[max=40];
LINE_CODE_2: string[max=1];
NAME_ADDR_LINE_2: string[max=40];
LINE_CODE_3: string[max=1];
NAME_ADDR_LINE_3: string[max=40];
LINE_CODE_4: string[max=1];
NAME_ADDR_LINE_4: string[max=40];
LINE_CODE_5: string[max=1];
NAME_ADDR_LINE_5: string[max=40];
LINE_CODE_6: string[max=1];
NAME_ADDR_LINE_6: string[max=40];
GREETING: string[max=6];
COMPLIMENT: string[max=6];
ALT_BANK_NBR: string[max=4];
SSN_TAXID_NBR: int32;
ALT_NAME_KEY: string[max=12];
ALT_TIE_BREAKER: int32;
SSN_TAXID_CODE: string[max=1];
REG_P_CODE: int32;
HOME_PHONE: string[max=10];
WORK_PHONE: string[max=10];
DATE_OF_BIRTH: int32;
SPEC_HAND: string[max=1];
RESTRICTED: string[max=1];
REG_O_CODE: string[max=1];
GEOG_CODE: string[max=6];
LOCATION: string[max=3];
COUNTY: string[max=3];
USER_CODE1: string[max=2];
USER_CODE2: string[max=2];
USER_CODE3: string[max=2];
LAS_MAX_LIABILITY: decimal[11,2];
IRA_PART_ST: string[max=1];
IRA_DSTRB_WH_FED: string[max=1];
IRA_DSTRB_WH_OTH: string[max=1];
IRA_CONTRB_LIMIT: string[max=4];
PRV_TAXID_NBR: int32;
PRV_TAXID_CODE: string[max=1];
DT_LST_MAINT_TAX: int32;
DT_LST_MAINT: int32;
DT_ELIGIBLE_DELETE: int32;
USER_CODE4: string[max=2];
USER_CODE5: string[max=2];
USER_CODE6: string[max=2];
FORCE_NOTICE: string[max=1];
DATE_LAST_BNOTICE: int32;
NBR_BNOTICES: int32;
DATE_LAST_W9: int32;
NBR_W9S: int32;
INT_WH_FED: string[max=1];
INT_WH_OTH: string[max=1];
TIN_CERTIFIED: string[max=1];
DATE_CERTIFIED: int32;
DATE_REQ_WITHHOLDING: int32;
TAX_EXEMPT: string[max=1];
OCCUPATION: string[max=35];
COUNTRY: string[max=2];
ID_TYPE: string[max=1];
ID_ISSUED_BY: string[max=2];
ID_NUMBER: string[max=24];
REC_CREATE_PGM: string[max=8];
CUSTOMER_LEVEL: string[max=1];
CALL_AT_HOME: string[max=1];
CALL_AT_WORK: string[max=1];
COM_NBR_1: string[max=5];
COM_NBR_2: string[max=5];
COM_NBR_3: string[max=5];
CTR_EXEMPT: string[max=1];
CRA_GROSS_SALES: decimal[13,2];
CRA_GROSS_INCOME: decimal[13,2];
CRA_TOTAL_ASSETS: decimal[13,2];
CRA_TOTAL_LIAB: decimal[13,2];
CRA_NET_WORTH: decimal[13,2];
CRA_DATE_LAST_FIN_STMT: int32;
CRA_DATE_VERIFIED: int32;
CRA_FIN_STMT_REQUIRED: string[max=1];
DATE_ADDED: int32;
TIME_ADDED: int32;
USER_ADDED: string[max=8];
TERM_ADDED: string[max=4];
SIC: string[max=6];
DATE_FIRST_ROTH_IRA: int32;
DATE_FIRST_ROTH_CONV: int32;
DATE_FIRST_SIMPLE_IRA: int32;
RESIDENCY_CODE: string[max=2];
DATE_LAST_CREDIT_RPT: int32;
CUST_TYPE: string[max=1];
CREDIT_RATING: string[max=6];
CREDIT_RATED_BY: string[max=8];
OFFICER: string[max=8];
BRANCH: string[max=4];
FED_WTHLD_PERCENT: decimal[9,6];
ST_WTHLD_PERCENT: decimal[9,6];
IRA_CONTR_ANNUAL_MAX: decimal[13,2];
YEARS_OF_SCHOOL: int32;
MARITAL_STATUS: string[max=1];
GENDER: string[max=1];
RACE: int32;
OTHER_RACE_DESC: string[max=20];
FAX_NUMBER: string[max=10];
E_MAIL_ADDRESS: string[max=40];
AGES_OF_DEPENDENTS: string[max=40];
CUST_SOURCE: string[max=1];
ADDL_SOURCE_INFO: string[max=40];
SECURITY_ACCESS_CDE: string[max=20];
DATE_OF_DEATH: int32;
US_CITIZEN: string[max=1];
FRGN_STAT_CHANGE: string[max=1];
FRGN_STAT_US_INT: string[max=1];
FRGN_STAT_US_DIV: string[max=1];
FRGN_STAT_US_BROKER: string[max=1];
BUS_TYPE: string[max=1];
BUS_SUB_TYPE: string[max=1];
BUS_PHONE_NBR: string[max=10];
BUS_ALT_PHONE_NBR: string[max=10];
BUS_STARTED_DATE: int32;
BUS_RESOLUTION_DATE: int32;
BUS_WIRE_TRF_RES_DATE: int32;
BUS_INCORPORATION_DATE: int32;
BUS_INCORPORATION_ST: string[max=2];
BUS_SIG_OPEN_DEPOSIT: int32;
BUS_SIG_WD_DEP_FUNDS: int32;
BUS_SIG_BORROW_FUNDS: int32;
BUS_SIG_PLEDGE_COLL: int32;
BUS_SIG_OPEN_SAFE_DEP: int32;
BUS_SIG_RLSE_NOTICE: int32;
BUS_ORAL_IN_PERSON: string[max=1];
BUS_ORAL_BY_PHONE: string[max=1];
BUS_WRITING_ORIG_SIG: string[max=1];
BUS_FAX_TRANSMISSION: string[max=1];
BUS_ELEC_TRANSMISSION: string[max=1];
BUS_CALL_REPET: string[max=1];
BUS_CALL_NONREPET: string[max=1];
BUS_WTRF_SECURITY: string[max=1];
COMM_TRF_TYPE: string[max=2];
COMM_TRF_LIMIT: decimal[13,2];
COMM_TRF_DLY_LIMIT: decimal[13,2];
COMM_TRF_ADV_DAYS: int32;
DATE_LAST_CHEX_RPT: int32;
CHEX_RPT_BY: string[max=8];
DATE_LAST_ACTY: int32;
DATE_LAST_CONT: int32;
AFFILIATION_CODE: string[max=8];
WORK_PHONE_EXTN: int32;
ID_ISSUE_DATE: int32;
ID_EXP_DATE: int32;
ID_VRFD_IND: string[max=1];
ID_VRFD_DATE: int32;
ID_VRFD_OFFICER: string[max=8];
ID_VRFD_METH: int32;
ID_VRFD_DESC: string[max=40];
ID_GOV_LST_MTCH: string[max=1];
BEACON_SCORE: string[max=6];
FRAUD_ADV_INDEX: string[max=3];
FRAUD_VICTIM_IND: string[max=1];
SAFESCAN: string[max=1];
AMER_IND_AK_NATIVE: string[max=1];
ASIAN: string[max=1];
BLK_AFRICAN_AMER: string[max=1];
NATIVE_HI_PAC_ISL: string[max=1];
WHITE: string[max=1];
ETHNICITY: string[max=2];
NO_INFO_PROVIDED: string[max=1];
RISK_CLASS: string[max=8];
CLASS_DATE: int32;
CLASS_OFFICER: string[max=8];
MSA_CODE: int32;
NY_TIN: string[max=9];
DATE_LAST_ADDR_CHG: int32;
AML_RISK_LVL: int32;
TAX_PROOF_IND: string[max=1];
TF_FULL_NAME: string[max=40];
TF_CITY: string[max=40];
TF_STATE: string[max=2];
TF_SSN_TAXID: string[max=9];
)
Embedded ORCHESTRATE system configuration:
{
node "node1"
{
fastname "mtl01dds01.yesbank.com"
pools ""
resource disk "/loads/work/etl/outbound/datasets" {pools ""}
resource scratchdisk "/scratch" {pools ""}
}
node "node2"
{
fastname "mtl01dds01.yesbank.com"
pools ""
resource disk "/loads/work/etl/outbound/datasets" {pools ""}
resource scratchdisk "/scratch" {pools ""}
}
}
-
- Participant
- Posts: 467
- Joined: Tue Mar 20, 2007 6:36 am
- Location: Chennai
- Contact:
There is an easier way to get the count of records in a dataset.
You can run:
This would give you the number of records in the dataset. We use this on a Windows server, I am not sure if this is available in DS on unix.
You can run:
Code: Select all
$APT_ORCHHOME/bin/dsrecords <absolute_path_of_dataset>
Minhajuddin
<a href="http://feeds.feedburner.com/~r/MyExperi ... ~6/2"><img src="http://feeds.feedburner.com/MyExperienc ... lrow.3.gif" alt="My experiences with this DLROW" border="0"></a>
<a href="http://feeds.feedburner.com/~r/MyExperi ... ~6/2"><img src="http://feeds.feedburner.com/MyExperienc ... lrow.3.gif" alt="My experiences with this DLROW" border="0"></a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 467
- Joined: Tue Mar 20, 2007 6:36 am
- Location: Chennai
- Contact:
ray.wurlod wrote:The dsrecords command is available on all platforms. It has an undocumeted -n option for returning just the number (not the "records" word). ...
Ray, How did you learn about so many hidden tricks ...
What's the secret?
Minhajuddin
<a href="http://feeds.feedburner.com/~r/MyExperi ... ~6/2"><img src="http://feeds.feedburner.com/MyExperienc ... lrow.3.gif" alt="My experiences with this DLROW" border="0"></a>
<a href="http://feeds.feedburner.com/~r/MyExperi ... ~6/2"><img src="http://feeds.feedburner.com/MyExperienc ... lrow.3.gif" alt="My experiences with this DLROW" border="0"></a>
-
- Premium Member
- Posts: 730
- Joined: Tue Nov 04, 2008 10:14 am
- Location: Bangalore
well i tried the dsrecords -n <dataset>
It gave me this error
Could not load program dsrecords:
Dependent module liborchaix3.a(libtemp.o) could not be loaded.
Could not load module liborchaix3.a(libtemp.o).
System error: No such file or directory
I have sourced the dsenv file and added the path of dsrecords to the PATH variable and then ran the command.... did i miss any thing?
It gave me this error
Could not load program dsrecords:
Dependent module liborchaix3.a(libtemp.o) could not be loaded.
Could not load module liborchaix3.a(libtemp.o).
System error: No such file or directory
I have sourced the dsenv file and added the path of dsrecords to the PATH variable and then ran the command.... did i miss any thing?
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
When libraries can't be found, check the shared library search list. This is the environment variable LD_LIBRARY_PATH, LIBPATH or SHLIB_PATH, depending on which operating system you're using.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.