Getting the total number or records from a dataset

seanc217 · Post by **seanc217** » Fri Jun 06, 2008 11:09 am

Is there a command in orchadmin to get this? I would like to script this.

Thanks

ArndW · Post by **ArndW** » Fri Jun 06, 2008 11:19 am

I'm not at a place where I can check now, but there are a couple of lines of output in "orchadmin ll {dataset}" that will give you this information. Once you have the exact line, you can select a less comprehensive orchadmin describe option.

seanc217 · Post by **seanc217** » Fri Jun 06, 2008 11:29 am

OK here's the output from orchadmin ll count but it seems to report record numbers in Kilobytes?? Kind of weird.

Let me know if you would know how to figure out the total number of records. Many Thanks!

Name: /loads/work/etl/outbound/datasets/cust_data_set.ds
Version: ORCHESTRATE V8.0.1 DM Block Format 6.
Time of Creation: 06/06/2008 11:40:46
Number of Partitions: 2
Number of Segments: 1
Valid Segments: 1
Preserve Partitioning: false
Segment Creation Time:
0: 06/06/2008 11:40:46

Partition 0
node : node1
records: 3120K
blocks : 41607
bytes : 5417M
files :
Segment 0 :
/loads/work/etl/outbound/datasets/cust_data_set.ds.srv_etl.mtl01dds01.yesbank.com.0000.0000.0000.274a.ca12a57e.0000.1578d353 5453M bytes
total : 5453M bytes
Partition 1
node : node2
records: 3120K
blocks : 41607
bytes : 5417M
files :
Segment 0 :
/loads/work/etl/outbound/datasets/cust_data_set.ds.srv_etl.mtl01dds01.yesbank.com.0000.0001.0000.274a.ca12a57e.0001.dcd6e488 5453M bytes
total : 5453M bytes

Totals:
records : 6240K
blocks : 83214
bytes : 10834M
filesize: 10907M
min part: 5453M
max part: 5453M

Schema:
record
( RECNUM: int64;
DATE_INSERTED: timestamp;
RECORD_CODE: int32;
BANK_NBR: string[max=4];
CUST_NAME_KEY: string[max=12];
TIE_BREAKER: int32;
CS_POINTER: string[max=2];
ZIP_CODE: string[max=9];
ZIP_CODE_DPBC: string[max=2];
VARB_LEN: int32;
LINE_CODE_1: string[max=1];
NAME_ADDR_LINE_1: string[max=40];
LINE_CODE_2: string[max=1];
NAME_ADDR_LINE_2: string[max=40];
LINE_CODE_3: string[max=1];
NAME_ADDR_LINE_3: string[max=40];
LINE_CODE_4: string[max=1];
NAME_ADDR_LINE_4: string[max=40];
LINE_CODE_5: string[max=1];
NAME_ADDR_LINE_5: string[max=40];
LINE_CODE_6: string[max=1];
NAME_ADDR_LINE_6: string[max=40];
GREETING: string[max=6];
COMPLIMENT: string[max=6];
ALT_BANK_NBR: string[max=4];
SSN_TAXID_NBR: int32;
ALT_NAME_KEY: string[max=12];
ALT_TIE_BREAKER: int32;
SSN_TAXID_CODE: string[max=1];
REG_P_CODE: int32;
HOME_PHONE: string[max=10];
WORK_PHONE: string[max=10];
DATE_OF_BIRTH: int32;
SPEC_HAND: string[max=1];
RESTRICTED: string[max=1];
REG_O_CODE: string[max=1];
GEOG_CODE: string[max=6];
LOCATION: string[max=3];
COUNTY: string[max=3];
USER_CODE1: string[max=2];
USER_CODE2: string[max=2];
USER_CODE3: string[max=2];
LAS_MAX_LIABILITY: decimal[11,2];
IRA_PART_ST: string[max=1];
IRA_DSTRB_WH_FED: string[max=1];
IRA_DSTRB_WH_OTH: string[max=1];
IRA_CONTRB_LIMIT: string[max=4];
PRV_TAXID_NBR: int32;
PRV_TAXID_CODE: string[max=1];
DT_LST_MAINT_TAX: int32;
DT_LST_MAINT: int32;
DT_ELIGIBLE_DELETE: int32;
USER_CODE4: string[max=2];
USER_CODE5: string[max=2];
USER_CODE6: string[max=2];
FORCE_NOTICE: string[max=1];
DATE_LAST_BNOTICE: int32;
NBR_BNOTICES: int32;
DATE_LAST_W9: int32;
NBR_W9S: int32;
INT_WH_FED: string[max=1];
INT_WH_OTH: string[max=1];
TIN_CERTIFIED: string[max=1];
DATE_CERTIFIED: int32;
DATE_REQ_WITHHOLDING: int32;
TAX_EXEMPT: string[max=1];
OCCUPATION: string[max=35];
COUNTRY: string[max=2];
ID_TYPE: string[max=1];
ID_ISSUED_BY: string[max=2];
ID_NUMBER: string[max=24];
REC_CREATE_PGM: string[max=8];
CUSTOMER_LEVEL: string[max=1];
CALL_AT_HOME: string[max=1];
CALL_AT_WORK: string[max=1];
COM_NBR_1: string[max=5];
COM_NBR_2: string[max=5];
COM_NBR_3: string[max=5];
CTR_EXEMPT: string[max=1];
CRA_GROSS_SALES: decimal[13,2];
CRA_GROSS_INCOME: decimal[13,2];
CRA_TOTAL_ASSETS: decimal[13,2];
CRA_TOTAL_LIAB: decimal[13,2];
CRA_NET_WORTH: decimal[13,2];
CRA_DATE_LAST_FIN_STMT: int32;
CRA_DATE_VERIFIED: int32;
CRA_FIN_STMT_REQUIRED: string[max=1];
DATE_ADDED: int32;
TIME_ADDED: int32;
USER_ADDED: string[max=8];
TERM_ADDED: string[max=4];
SIC: string[max=6];
DATE_FIRST_ROTH_IRA: int32;
DATE_FIRST_ROTH_CONV: int32;
DATE_FIRST_SIMPLE_IRA: int32;
RESIDENCY_CODE: string[max=2];
DATE_LAST_CREDIT_RPT: int32;
CUST_TYPE: string[max=1];
CREDIT_RATING: string[max=6];
CREDIT_RATED_BY: string[max=8];
OFFICER: string[max=8];
BRANCH: string[max=4];
FED_WTHLD_PERCENT: decimal[9,6];
ST_WTHLD_PERCENT: decimal[9,6];
IRA_CONTR_ANNUAL_MAX: decimal[13,2];
YEARS_OF_SCHOOL: int32;
MARITAL_STATUS: string[max=1];
GENDER: string[max=1];
RACE: int32;
OTHER_RACE_DESC: string[max=20];
FAX_NUMBER: string[max=10];
E_MAIL_ADDRESS: string[max=40];
AGES_OF_DEPENDENTS: string[max=40];
CUST_SOURCE: string[max=1];
ADDL_SOURCE_INFO: string[max=40];
SECURITY_ACCESS_CDE: string[max=20];
DATE_OF_DEATH: int32;
US_CITIZEN: string[max=1];
FRGN_STAT_CHANGE: string[max=1];
FRGN_STAT_US_INT: string[max=1];
FRGN_STAT_US_DIV: string[max=1];
FRGN_STAT_US_BROKER: string[max=1];
BUS_TYPE: string[max=1];
BUS_SUB_TYPE: string[max=1];
BUS_PHONE_NBR: string[max=10];
BUS_ALT_PHONE_NBR: string[max=10];
BUS_STARTED_DATE: int32;
BUS_RESOLUTION_DATE: int32;
BUS_WIRE_TRF_RES_DATE: int32;
BUS_INCORPORATION_DATE: int32;
BUS_INCORPORATION_ST: string[max=2];
BUS_SIG_OPEN_DEPOSIT: int32;
BUS_SIG_WD_DEP_FUNDS: int32;
BUS_SIG_BORROW_FUNDS: int32;
BUS_SIG_PLEDGE_COLL: int32;
BUS_SIG_OPEN_SAFE_DEP: int32;
BUS_SIG_RLSE_NOTICE: int32;
BUS_ORAL_IN_PERSON: string[max=1];
BUS_ORAL_BY_PHONE: string[max=1];
BUS_WRITING_ORIG_SIG: string[max=1];
BUS_FAX_TRANSMISSION: string[max=1];
BUS_ELEC_TRANSMISSION: string[max=1];
BUS_CALL_REPET: string[max=1];
BUS_CALL_NONREPET: string[max=1];
BUS_WTRF_SECURITY: string[max=1];
COMM_TRF_TYPE: string[max=2];
COMM_TRF_LIMIT: decimal[13,2];
COMM_TRF_DLY_LIMIT: decimal[13,2];
COMM_TRF_ADV_DAYS: int32;
DATE_LAST_CHEX_RPT: int32;
CHEX_RPT_BY: string[max=8];
DATE_LAST_ACTY: int32;
DATE_LAST_CONT: int32;
AFFILIATION_CODE: string[max=8];
WORK_PHONE_EXTN: int32;
ID_ISSUE_DATE: int32;
ID_EXP_DATE: int32;
ID_VRFD_IND: string[max=1];
ID_VRFD_DATE: int32;
ID_VRFD_OFFICER: string[max=8];
ID_VRFD_METH: int32;
ID_VRFD_DESC: string[max=40];
ID_GOV_LST_MTCH: string[max=1];
BEACON_SCORE: string[max=6];
FRAUD_ADV_INDEX: string[max=3];
FRAUD_VICTIM_IND: string[max=1];
SAFESCAN: string[max=1];
AMER_IND_AK_NATIVE: string[max=1];
ASIAN: string[max=1];
BLK_AFRICAN_AMER: string[max=1];
NATIVE_HI_PAC_ISL: string[max=1];
WHITE: string[max=1];
ETHNICITY: string[max=2];
NO_INFO_PROVIDED: string[max=1];
RISK_CLASS: string[max=8];
CLASS_DATE: int32;
CLASS_OFFICER: string[max=8];
MSA_CODE: int32;
NY_TIN: string[max=9];
DATE_LAST_ADDR_CHG: int32;
AML_RISK_LVL: int32;
TAX_PROOF_IND: string[max=1];
TF_FULL_NAME: string[max=40];
TF_CITY: string[max=40];
TF_STATE: string[max=2];
TF_SSN_TAXID: string[max=9];
)

Embedded ORCHESTRATE system configuration:
{
node "node1"
{
fastname "mtl01dds01.yesbank.com"
pools ""
resource disk "/loads/work/etl/outbound/datasets" {pools ""}
resource scratchdisk "/scratch" {pools ""}
}
node "node2"
{
fastname "mtl01dds01.yesbank.com"
pools ""
resource disk "/loads/work/etl/outbound/datasets" {pools ""}
resource scratchdisk "/scratch" {pools ""}
}
}

ArndW · Post by **ArndW** » Fri Jun 06, 2008 11:34 am

If you call up orchadmin with no options you will see a "describe" section and I think one of the options there is to disable the pretty-printing of numbers so that you get actual record counts.

seanc217 · Post by **seanc217** » Fri Jun 06, 2008 11:41 am

OK Thanks!

I got it the command you have to use is:

orchadmin describe -d -l <your dataset file>

Worked great.

Thanks Again!

Minhajuddin · Post by **Minhajuddin** » Fri Jun 06, 2008 1:01 pm

There is an easier way to get the count of records in a dataset.
You can run:

Code: Select all

$APT_ORCHHOME/bin/dsrecords <absolute_path_of_dataset>

This would give you the number of records in the dataset. We use this on a Windows server, I am not sure if this is available in DS on unix.

ray.wurlod · Post by **ray.wurlod** » Sun Jun 08, 2008 11:49 pm

The dsrecords command is available on all platforms. It has an undocumeted -n option for returning just the number (not the "records" word).

Minhajuddin · Post by **Minhajuddin** » Mon Jun 09, 2008 10:43 am

ray.wurlod wrote:The dsrecords command is available on all platforms. It has an undocumeted -n option for returning just the number (not the "records" word). ...

Ray, How did you learn about so many hidden tricks ...
What's the secret?

zulfi123786 · Post by **zulfi123786** » Tue Feb 16, 2010 1:01 am

well i tried the dsrecords -n <dataset>

It gave me this error

Could not load program dsrecords:
Dependent module liborchaix3.a(libtemp.o) could not be loaded.
Could not load module liborchaix3.a(libtemp.o).
System error: No such file or directory

I have sourced the dsenv file and added the path of dsrecords to the PATH variable and then ran the command.... did i miss any thing?

gssr · Post by **gssr** » Tue Feb 16, 2010 1:09 am

System error: No such file or directory

The path or the dataset is missing!!

ray.wurlod · Post by **ray.wurlod** » Tue Feb 16, 2010 2:30 am

When libraries can't be found, check the shared library search list. This is the environment variable LD_LIBRARY_PATH, LIBPATH or SHLIB_PATH, depending on which operating system you're using.

chulett · Post by **chulett** » Tue Feb 16, 2010 6:22 am

It usually means you didn't source the dsenv file. Can you show us the actual command you ran to do that?