process

process.aws

process.L0toL1

process.L1toL2

process.L2toL3

postprocess

postprocess.csv2bufr

qc

qc.persistence

get

get.get

tx

tx.tx

AWS Level 0 (L0) data transmission fetching module

class tx.tx.EmailMessage(email_msg, sender_name)

Bases: SbdMessage

Email message object

checkEmail(email_msg)

Check if email is Message object

checkSender(sender_name)

Check email message from field matches sender name or names

getEmailBody()

Get email message body

getEmailInfo()

Parse message in email object

getIMEI()

Get modem identifier from email subject string

tx.tx.GFP2toDEC(Bytes)

Two-bit decoder

Parameters

Bytes (list) – List of two values

Returns

Decoded value

Return type

float

tx.tx.GLI4toDEC(Bytes)

Four-bit decoder

Parameters

Bytes (list) – List of four values

Returns

Decoded value

Return type

float

class tx.tx.L0tx(email_msg, format_file=pypromice.resources.DEFAULT_PAYLOAD_FORMATS_PATH, type_file=pypromice.resources.DEFAULT_PAYLOAD_TYPES_PATH, sender_name=['sbdservice', 'ice@geus.dk', 'emailrelay@konectgds.com'], UnixEpochOffset=0, CRbasicEpochOffset=631152000)

Bases: EmailMessage, PayloadFormat

L0 tranmission data object

check2BitNAN(msg, type_letter, letter_flag=['g', 'n', 'e'], nan_value=8191)

Check if byte is a 2-bit NAN. This occurs when the GPS data is not available and the logger sends a 2-bytes NAN instead of a 4-bytes value

checkByte(b)

Check byte format against payload formatter object

checkLength()
checkPayload()

Check message payload

getByteValue(ValueBytesCount, BinaryMessage, idx)

Get values from byte range in binary message

getDataLine()

Get data line from transmission message

Returns

Dataline string if found

Return type

str or None

getFirstByte()

Get first byte in payload

getFormat()

Get binary format type from first byte in payload

Returns

  • bval (int or None) – Format value

  • bfor (str or None) – Format string characters

  • bname (str or None) – Format name

  • blength (int or None) – Expected format length

  • bidx (int) – Format index

  • bool – Valid format flag

isDiagnostics(DataLine)

Flag if message is diagnostics

isObservations(DataLine)

Flag if message is observations

isSummer(DataLine)

Flag if message is summer message

isWatsonObservation(DataLine)

Flag if message is Watson River measurement

isWithInstance(DataLine)

Flag if message is with instance

updateByteCounter(value)

Update byte counter for decoding message

writeEntry(entry, i)

Write out comma-formatted data entry from message

class tx.tx.PayloadFormat(format_file, type_file)

Bases: object

Payload formatter object

readFile(in_file)

Read lines from file

Parameters

in_file (str) – Input file path

Returns

lines – List of file line contents

Return type

list

readFormatter(in_file, delimiter=',')

Read payload formatter from file. Outputted dictionary set as key[number]: [expected_length, format_characters, description]. Flag column (info[4]) used to signify if entry should be written to output

Parameters

  • in_file (str) – Input file path

  • delimiter (str, optional) – File delimiter. The default is “,”

Returns

payload_fmt – Payload format information

Return type

dict

readType(in_file, delimiter=',')

Read payload type setter from file. Outputted dictionary set as key[type_letter]: number_of_bytes

Parameters

  • in_file (str) – Input file path

  • delimiter (str, optional) – File delimiter. The default is “,”

Returns

payload_typ – Payload type information

Return type

dict

tx.tx.RAWtoSTR(Bytes)

Byte-to-string decoder

Parameters

Bytes (list) – List of values

Return type

Decoded string characters

class tx.tx.SbdMessage(content, attach, imei)

Bases: object

SBD transmission message object

checkAttachment(attach)

Check if attachment is present in email.message.Message object

checkAttachmentName(attach_file)

Check if attachment is .sbd file

getKeyValue(content, seps, key, integer=True)

Get attribute from email via keyword

getLocation(content, seps=' ', key='Unit Location')

Get latitude longitude unit location from email message

getPayloadFromEmail(attach, message_size)

Get Sbd payload from email object

getPayloadFromFile(attach)

Read Sbd payload from .sbd file

getStatus(content, seps1=': ', seps2=' ', key='Session Status')

Get session status from email message

tx.tx.findLine(content, key)

Find keyword in line

Parameters

  • content (str) – String to find keyword in

  • key (str) – Keyword to find in string

Returns

line – Line that keyword appears on

Return type

str

tx.tx.getMail(mail_server, last_uid=1)

Retrieve new mail

Parameters

  • mail_server (imaplib.IMAP_SSL) – Mail server object

  • last_uid (int, optional) – Mail uid to start retrieval from. The default is 1.

Yields

  • str – Mail uid

  • str – Mail message

tx.tx.loadMsg(fname)

Load .msg email file into format compatible with EmailMessage and SbdMessage objects

Parameters

fname (str) – File path to .msg file

Returns

Email message object

Return type

email.message.Message

tx.tx.parseValue(line, seps)

Parse last value from line according to separating characters

Parameters

  • line (str) – String to split

  • sep (str) – Separator characters to split line by

Returns

value – Value extracted from line

Return type

str

tx.tx.readSBD(sbd_file)

Read encoded .sbd transmission file

Parameters

sbd_file (str) – Filepath to encoded .sbd file

Returns

data – Transmission message byte object

Return type

bytes

tx.tx.saveMsg(msg, fname)

Save email message object to .msg file

Parameters

  • msg (email.message.Message) – Email object to save to file

  • fname (str) – File path to outputted .msg file