Welcome to CORK and BPR documentation!¶

Verify that your software is still current¶

[mheesema@sciserv C-tools]$ svn log -r base:head

Simple health check and download¶

1. Make sure laptop clock is synchronized (see Sync to NTP)

2. Create a working directory (see Setup working directory)

3. Connect instrument to computer (see Connecting the Instrument)

4. Turn on communication and power after connection is established make sure power and comms are off before ODI connector is removed from parking position

5. Run mlterm, making sure to use a reasonable log-file name (see Starting mlterm)

6. Get instrument stats F

   1. Take note of clock drift (do not syncronize the clock, later on)
   2. Take note of predicted download times

7. Get basic logger settings I

   1. Take note of download baud-rate (115200 for NEPTUNE instruments)
   2. Take note of control baud-rate (115200 for NEPTUNE instruments)

8. Consider to increase download speed (see Increase download speed)

9. Download data D, making sure to use a reasonable file name

10. Get instrument stats F again

11. Deploy instrument and quit Z

    PI might suggests real-time monitoring while following next steps

12. Backup data (*.raw) to USB key (see Backup data)

13. Check data integrity

    1. run mlbin xxxx.raw

       strips off file system and generate xxxx.bin (there should be no errors)

    2. run dumpBin.py -a -n -t -s xxxx.bin > xxxx.dmp

       Does some integrity checking and creates ASCII dump

    3. run tail xxxxx.dmp to see statistics

       Discuss with PI

14. Contact PI and discuss whether to clear memory

15. Backup *.log files and sent *.log and *.raw files to PI

16. Make sure the communication and power lines are turned off before the under water mateable connector is disconnected.

Linux¶

Working with the HP-Mini s that are used to download data from CORK s and BPR s requires some basic Linux skills. This section provides some recipes for common tasks.

Linux session with terminal window.

Open a terminal window¶

Click on the terminal icon

to open a command-line terminal window. If needed you can create tabs with additional terminals from the File menu.

Instrument setup¶

mlterm¶

Data post-processing¶

After downloading raw data from an instrument (see Simple health check and download) the data has to post-processed.

First, mlbin has to be applied (mlbin filename.raw) to turn the *.raw file into a *.bin file. The *.raw files still contain the filesystem of the flash card and recovery information in case there were errors in the data transmission. mlbin strips off the filesystem and tries to fix any errors that might have occured. The *.bin file only contains data records in binary counts.

mldat9 (mldat9 filename.bin) is the standart utility to calibrate *.bin data and write the results into a *.dat ASCII file. In order to use mldat9 you have to cd into the directory with the *.bin file and make sure there is also a current set of calibration files in the directory (see Setup working directory). While running mldat9 you will be ask whether the temperatures are measured using a platinum sensor or a thermistor. Platinum sensors are mostly used in CORKs and BPRs installed past 2009 and have two-digit serial numbers, while themistors have one digit serial numbers. The serial numbers are shown in the sites.txt calibratrion file (see mlterm calibration files) and should be noted in the respective *.par file under ~/corkobservatory/ParameterFiles. Furthermore, you are asked whether the Paroscientific gauge measures pressure and temperature or pressure only. If you do not know the answer you can lookup the gauge serial numbers that are mentionend in sites.txt in parosci.txt and check whether U0, Y1, Y2 are zero (pressure only) or non zeros (pressure and temperature). Then you have to select a data format (typically `` 4. Parosci temperature compensated pressure``) and a site description which is a line from the site.txt calibration file. The site description has the following format:

mldat9 automatically removes spikes from the paros pressure and temperature channels that get sometimes generated by the measurement system and is convinient to use, but it has certain limitations:

1. mldat9 has problems with partial records that can sometimes occur especially on systems that occasionally switch between battery and cabled mode.
2. mldat9 cannot cope with instruments that have a mix between gauges that measure temperature and pressure and pressure only. These configurations exist especially for CORKs which have many gauges and are designed to operated in cabled mode.

To cope with these problems two Python programs were developed that can be used instead of mldat9 (see Install dependencies for Python scripts). The first script dumpBin.py can be used fix *.bin files by eliminating partial records, remove spikes, find problems, and dump the binary counts into an ASCII format that is human readable and can be calibrated using a calibrateLogfile.py.

A common workflow to process filename.bin using these scripts would be:

1. Fix data problems using dumpBin.py -n -s -a -t -b filename_fixed.bin filename.bin > filename.dmp
   • filename_fixed.bin will contain all complete data records
   • filename.dmp will contain ASCII representation of binary data and debugging information
2. Dump despiked data (only works with clean data) for calibration ``dumpBin.py -d filename_fixed.bin > filename_despiked.dmp
3. Calibrate the data calibrateLogfile.py -c calibrationName filename_despiked.dmp > filename_despiked.dat
   • calibrateLogfile.py also works with data from stdin, so the last two steps can be combined using a pipe: dumpBin.py -d filename_fixed.bin | calibrateLogfile.py -c calibrationName > filename_despiked.dat
   • Run calibrateLogfile.py -l to get a list of available calibrations. New calibrations can be defined in calibrateData.py.

Run dumpBin.py --help and calibrateLogfile.py --help to study all available the commandline options.

Check out the CORK software repository¶

CORK software, calibration and parameter files are maintainded at the CORK Observatory Software Repository.

It is best to have svn installed to check out the code. Check if svn is installed by running svn --help. If there is not help displayed you have to install svn e.g. by running sodu yum install subversion.

Now, go into the home directory of the cork user (cd ~) and run

svn co https://svn.code.sf.net/p/corkobservatory/code/ corkobservatory

This will create a corkobservatory directory that contains program source codes, calibration information, etc. To update these files to the latest version you can cd into the corkobservatory directory and run svn update at any time when you have an internet connection.

Compile and install mlterm, mlbin, mldat9¶

mlterm, mlbin and mldat9 are the core utilities (programmed in C) to setup CORKs and download the data, strip unneccessary filesystem information from the downloaded *.raw files, and to calibrate the data contained in the *.bin files, respectively. In some special cases, when there are problems with data consistency or with special instrument setups you need to use some Python scripts, that will be discussed later, instead of mldat9.

Since the core utilities are supplied as C source code, you need to have a C-compiler and basic development tools installed. cd into the mlterm, mlbin, and mldat directories under ~/corkobservatory/C-tools and run

1. make
2. sudo make install

in each directory to compile and install the programs individually.

NOTE: there is also a variant of mlterm mlterm_vpn that gets automatically installed with mlterm that is for use with instruments connected over connections with high latency (e.g. TCP/IP, VPN,...).

Install RS-422 adapter driver if necessary¶

Many USB/RS-422 adapters run under Linux out of the box, but the Moxa UPort-1130, which we found is the most reliable adapter, requires the compilation/installation of a seperate driver. We had problems in the past, because the driver was not compatible with current Linux distributions, but fortunately, Moxa did update the driver in summer 2012.

So if you want to use a Moxa adapter make sure to download and install the latest diver either from their website or use the copy that is stored in the software repository (~/corkobservatory/ComputerSetup/drivers/MoxaUPort). Follow the instructions in the readme.txt file under “Module driver configuration”.

If the driver install succeeded running mlterm with the Moxa UPort plugged into a USB port should start an mlterm session with mlterm trying to connect to the instrument. If the adapter is not recognized by the OS mlterm will fail with the following message:

/dev/ttyUSB0: No such file or directory

Install dependencies for Python scripts¶

The Python scripts need

1. Python (tested with 2.6 and 2.7)
2. numpy
3. matplotlib (optionally for plotting)

So first check if Python is already installed python --version and install it with e.g. sudo yum install python if not. Now you can try to install the numpy and matplotlib packages of your distribution e.g.:

• sudo yum install python-numpy
• sudo yum install python-matplotlib

or install easy_install e.g. sudo yum install python-setuptools and then run:

• sudo easy_install numpy
• sudo easy_install matplotlib

Now, cd into the ~/corkobservatory/ComputerSetup/scripts directory (see Check out the CORK software repository for how to set it up) and run ./createPythonToolsSymLinks.bash to add symbolic links to the python scripts dumpBin.py and calibrateLogfile.py to your path. If everything worked out, running dumpBin.py --help and calibrateLogfile.py --help will display the respective help information.