bitfields - extract bit fields from given variable to create new variables

bitfields - Extracts bit fields from a given variable to create new variables

SYNOPSIS

bitfields  [ parameter=value ]  [ inputfile ... ]

Parameters are: bit_variable, last_rec_dim, field_names, field_units, convert_funcs, field_offsets, field_lengths, fields_per_rec, field_steps, data_types, bad_values, scale_factors, scale_offsets

DESCRIPTION

bitfields extracts bit fields from a given variable to create new variables in the same dataset. bitfields is intended as a tool to explore telemetry header variables. Up to 16 new variables can be created at a time.

Bit fields are extracted from records. A variable record is all variable data where the first k dimensions are fixed, e.g., all data with coordinates (I1, I2, ..., Ik, *, *, ... *). Normally, bit fields are extracted from a 2-D variable, and k is 1. k is specified by the last_rec_dim parameter.

The variables created by bitfields either have k or k+1 dimensions. The first case occurs when there is only one instance of a given bit field per record. The second case occurs when there is more than one instance of a given bit field per record. bitfields can only work with multiple instances that are evenly spaced, e.g.

instance 1 bits = [offset        , ..., offset          + length -1]
instance 2 bits = [offset +1*step, ..., offset + 1*step + length -1]
instance 3 bits = [offset +2*step, ..., offset + 2*step + length -1]
 ...

The following conversion functions are supplied for processing bit fields.

     u	- unsigned
     ru	- byte-reversed, unsigned
     fu	- bit-flipped, unsigned
     rfu	- byte-reversed, fu

     s0	- signed snn...nn, where s=0 means positive
     rs0	- byte-reversed, s0
     s1	- signed snn...nn, where s=1 means negative
     rs1	- byte-reversed, s1

     c1	- one's complement
     rc1	- byte-reversed, c1
     c2	- two's complement
     rc2	- byte-reversed, c2

     fs0	- bit-flipped, s0
     rfs0	- byte-reversed, fs0
     fs1	- bit-flipped, s1
     rfs1	- byte-reversed, fs1

Bit-flipped means binary complement, e.g., 1 is mapped to 0, and 0 is mapped to 1. Byte-reversed means the bytes are in the opposite order. The most common case of byte-reversing is byte swapping. Note, a byte-reversed bit field must have a length that is a multiple of 8.

A given bit field is (1) byte-reversed if necessary, (2) bit-flipped if necessary, and then (3) converted to a number.

A given bit field cannot be wider than 32 bits. If the bit field represents an unsigned integer, it cannot be wider than 31 bits. This is because TeraScan datasets do not support and unsigned long integer datatype.

bitfields operates in place, i.e., variables are created in the input datasets. bitfields can be run several times on the same input dataset, provided the names of the variables to create are different each time.

PARAMETERS

bit_variable: Variable from which bit field data is to be extracted. There is no default. Wildcard expressions cannot be used.

last_rec_dim: One-relative dimension number defining the record structure of the bit_variable. The default is 1, meaning the first dimension. last_rec_dim cannot be greater than the number of dimensions of the bit_variable.

field_names: Names of the variables to create in the input dataset. There is not default. Up to 16 names can be specified.

field_units: Units for each of the variables to be created. The default for each new variable is "-", which is interpreted as no units.

convert_funcs: Conversion functions for processing the bit fields for each of the new variables. These functions are listed above. There is no default.

field_offsets: For each new variable, the 1-relative start bit of the first bit field instance in each record. The first bit in each record is 1. There is no default.

field_lengths: For each new variable, the length of the corresponding bit fields. Note the restrictions on bit field lengths described above. There is no default.

fields_per_rec: For each new variable, the number of bit field instances per record. The default is 1 is one for each new variable.

field_steps: For each new variable, the number of bits separating the starts of two bit field instances. The default is 0 for each new variable. This parameter is not used at all if fields_per_rec=1 for each new variable.

data_types: The output datatypes for each new variable, either byte, short or long. The default for each new variable is computed by bitfields based on conversion function and bit field length.

bad_values: Missing (bad) values for each variable to be created. The default is 0 for each new variable.

scale_factors
scale_offsets

Scaling information for each new variable. These parameters are applied as follows:

true value = bit field (stored) value * scale_factor + scale_offset

The default scale_factor for each new variable is 1. The default scale_offset for each new variable is 0.

EXAMPLES

In the following example, XXX is a dataset containing a 10x5 2-D short integer variable xxx. The contents of xxx are first displayed below using dumpvar. Then, bitfields is used to break each record into 20, 4-bit quantities. The result is a 10x20 byte valued variable yyy which is added to the dataset XXX.

% dumpvar XXX
include_vars   : char(255) ? []
line_per_elem  : char(  3) ? [yes] no
dim_starts     : int (  5) ? []
dim_counts     : int (  5) ? []
list_dims      : char(  3) ? [yes]
hex_output     : char(  3) ? [no] yes
printout       : char(  3) ? [no]
Dumpvar  XXX  xxx  Page 1

 1  0000 0001 0002 0003 0004
 2  0100 0101 0102 0103 0104
 3  0200 0201 0202 0203 0204
 4  0300 0301 0302 0303 0304
 5  0400 0401 0402 0403 0404
 6  0500 0501 0502 0503 0504
 7  0600 0601 0602 0603 0604
 8  0700 0701 0702 0703 0704
 9  0800 0801 0802 0803 0804
10  0900 0901 0902 0903 0904

% bitfields XXX
bit_variable   : char( 31) ? xxx
last_rec_dim   : int       ? [1]
field_names    : char(255) ? yyy
field_units    : char( 31) ? [-]
convert_funcs  : char(  4) ? u
field_offsets  : int       ? 1
field_lengths  : int       ? 4
fields_per_rec : int       ? [1] 20
field_steps    : int       ? [4]
data_types     : char(  5) ? [byte]
bad_values     : real      ? [0] 255
scale_factors  : real      ? [1]
scale_offsets  : real      ? [0]

% dumpvar XXX
include_vars   : char(255) ? [] yyy
line_per_elem  : char(  3) ? [yes] n
dim_starts     : int (  5) ? []
dim_counts     : int (  5) ? []
list_dims      : char(  3) ? [yes]
hex_output     : char(  3) ? [no] y
printout       : char(  3) ? [no]
Dumpvar  XXX  yyy  Page 1

 1  00 00 00 00 00 00 00 01 00 00 00 02 00 00 00 03 00 00 00 04
 2  00 01 00 00 00 01 00 01 00 01 00 02 00 01 00 03 00 01 00 04
 3  00 02 00 00 00 02 00 01 00 02 00 02 00 02 00 03 00 02 00 04
 4  00 03 00 00 00 03 00 01 00 03 00 02 00 03 00 03 00 03 00 04
 5  00 04 00 00 00 04 00 01 00 04 00 02 00 04 00 03 00 04 00 04
 6  00 05 00 00 00 05 00 01 00 05 00 02 00 05 00 03 00 05 00 04
 7  00 06 00 00 00 06 00 01 00 06 00 02 00 06 00 03 00 06 00 04
 8  00 07 00 00 00 07 00 01 00 07 00 02 00 07 00 03 00 07 00 04
 9  00 08 00 00 00 08 00 01 00 08 00 02 00 08 00 03 00 08 00 04
10  00 09 00 00 00 09 00 01 00 09 00 02 00 09 00 03 00 09 00 04

NOTES

The rules by which bitfields computes the default datatype for a given variable is as follows:

# bits     signed      type
------     ------      ----
<= 16        yes       short
>  16        yes       long
<=  8        no        byte
<  16        no        short
>= 16        no        long