Blog

Gordon, June 25

I did some by-hand rsyncs of part of today's data from sites 2-4, using the rsync -z (compress) option:

The downloads were happening simultaneously. These speeds look good, 40-60 kbyte/sec is acceptable.

June 24

Kurt, Tom, Dan Rajewski, Gordon

ncar2 is set up.

sonic: 1119
krypton: 1393
prop: 00319 (determined by a process of elimination, not noted down at setup time)
baro: B3
2m: TRH27
10m: TRH25

Verified that prop direction read 0 when vane held along boom.

Dan and Gordon shot the azimuths, using the same procedure as at ncar4 today:

Sonic:
357.3
356.9
357.7
356.7
356.4
357.2

Average: 357.0

Prop:
93.3
93.4
93.2
93.4
93.6
93.1

Average: 93.3

Gordon, Dan Rajewski June 24 10:55 CDT

Used the data scope to shoot the wind sensor azimuth, holding down the green button to average the bearings.All readings wrt magnetic north. Each person shot 3 values.

Sonic azimuths, looking into the array, positioning the datascope so that the top, white transducer arm of the sonic was centered on the mast:

355.4
353.4
353.2
353.0
355.5
355.1

Average: 354.3

Prop azimuths:
92.5
92.3
92.4
92.6
92.9
93.0

Average: 92.6

Left 12:00 CDT

ncar4 stopped reporting data around 15:30 CDT on June 23rd. This was around the time that I was hacking on ncar3 and ncar4 from the motel in Ames.

On 6/24 around 10:55 Dan and I visited the site. The system was spewing messages to the console port, the primary error was "ntpd page allocation error", indicating that the system had run out of memory. All voltages look OK.

Around the time it died, I was testing a "sv" command to show the system voltage from the power mote. For some weird reason I couldn't get a

mote_dump | fgrep | awk

pipeline to work, it would never show data. What finally worked was

mote_dump | awk

. Seems like a bug in the busybox linux. Perhaps in all the hacking I started an awk command which somehow ate up the memory.

So we just power cycled the system, and monitored it.

free -m

shows 52 Mbytes free.

Later I realized we had seen this once at playa1 during PCAPS: https://wiki.ucar.edu/x/nJhCB Wouild be interesting to check if it is the same box. The box was not replaced during PCAPS and ran fine the rest of the project.

A dsm_server for CWEX11 is running on grapegarden to feed the data for cockpit. If you have the access in EOL network, you are able to run cockpit and see the real-time data plots.  

Right now, we see the data from ncar3.

The two instructions to setup a dsm or a laptop and operate a computer are on EOL CWEX11 website.

http://www.eol.ucar.edu/deployment/educational-deployments/CWEX11/cwex11

Users need login-id and password to the instructions. The documents contain key information; please keep it safely.

June 23, 16:30 CDT

Tom, Kurt and Dan Rajewski are at site 3 in order to remove the krypton 1393 and install a Licor 7500 (S/N 813).

Dan wants Licors at 1 and 3, and kryptons at site 2 and 4.

They dropped the tower sometime after 15:00 CDT and raised it again at 16:05. The drop time is a guess.

Jun 22, 5:46 pm CDT

Low clouds, intermittent light rain, winds from the WSW

ncar 4 is up

CSAT3: 0671
krypton: 1390
prop: 00318, propeller: 53678
baro: B10
2m: TRH26
10m: TRH15

Verified that prop direction read 0 when vane held along boom.
Plan to deploy the props with the boom direction set to 0 (BD000)
and correct the directions in software.

Jun 22, 12:30 CDT

Steady light rain

ncar3 is up.

CSAT3: 1119 was originally installed on Jun 22, then removed and replaced by 539 on Jun 23
krypton:1393 (this was removed the next day on Jun 23)
Licor: 813 (installed Jun 23)
2m: TRH18
10m: TH19
Barometer:B5
Prop: 00002, propeller: 64662

ToDos writeups for ISU:

• How to download configuration to router
• How to clean krypton
• How to check voltage at serial port, and check/change serial port fuse
• How to lower and raise mast
• contacts with phone numbers
• How to login to station, run ds, kv, rs, etc
• How to monitor daily plots

Station maintenance kit:

• 4 ft ladder
• tools: 7/16, 1/2, 9/16, diag cutters, hex wrenches, std & phillips, jeweler's screwdriver, needle nose and std pliers,
• fuses are in boxes
• serial port meter box
• system load ammeter
• DVM
• Q-tips
• distilled H2O
• winch, cordless drill, batteries, charger, inverter
• zip ties
• 4 spare 12V batteries
• battery charger (power supply)
• sonic socks
• console cable
• cellular modem
• compass

At 2:25 pm MDT ncar1 was brought up outside on the demo tripod near the horseshoe pits. It is sending data over cellular, with all sensors connected.

64 byte ping times to ncar1 from an EOL host look good, around 100 msec:

ping ncar1.dyndns.org
PING ncar1.dyndns.org (75.220.191.57) 56(84) bytes of data.
64 bytes from 57.sub-75-220-191.myvzw.com (75.220.191.57): icmp_req=1 ttl=52 time=108 ms
64 bytes from 57.sub-75-220-191.myvzw.com (75.220.191.57): icmp_req=2 ttl=52 time=114 ms
64 bytes from 57.sub-75-220-191.myvzw.com (75.220.191.57): icmp_req=3 ttl=52 time=105 ms
64 bytes from 57.sub-75-220-191.myvzw.com (75.220.191.57): icmp_req=4 ttl=52 time=88.6 ms
64 bytes from 57.sub-75-220-191.myvzw.com (75.220.191.57): icmp_req=5 ttl=52 time=118 ms
64 bytes from 57.sub-75-220-191.myvzw.com (75.220.191.57): icmp_req=6 ttl=52 time=79.2 ms

ping times to ncar4, inside the cal lab are more like 500 msec. This is the only modem activated in the lab.

ping ncar4.dyndns.org
PING ncar4.dyndns.org (75.196.178.174) 56(84) bytes of data.
64 bytes from 174.sub-75-196-178.myvzw.com (75.196.178.174): icmp_req=1 ttl=45 time=402 ms
64 bytes from 174.sub-75-196-178.myvzw.com (75.196.178.174): icmp_req=2 ttl=45 time=402 ms
64 bytes from 174.sub-75-196-178.myvzw.com (75.196.178.174): icmp_req=3 ttl=45 time=562 ms
64 bytes from 174.sub-75-196-178.myvzw.com (75.196.178.174): icmp_req=4 ttl=45 time=522 ms
64 bytes from 174.sub-75-196-178.myvzw.com (75.196.178.174): icmp_req=5 ttl=45 time=662 ms

Jun 9 15:30 MDT

Since Friday, Jun 3 the four systems have been setup in the cal lab. In order to get cellular signal, external cellular antennas are mounted in the window, at a spacing of about 4 feet apart.

The bulk of the sensors were added on Monday, Jun 6, and by Tuesday we had all sensors connected, except for the leaf wetness sensor.

Encountered several issues

• loose RS232 connection on Licor 7500 (serial number ?)
• fix loose RS232 connection on prop vane (serial number ?)
• many reboots on ncar4, which is getting its power from aux1 on ncar2.

Cellular comms are not great. Using a browser to look at the router web page, shows that the cellular signals are good (70 - 80 %), but the throughput is not great - probably because they are interfering with each other. The connection to ncar2 and ncar4 has been the worst. Chris also noticed that their modems seem hotter to the touch.

Copying files with rsync was problematic. It worked well from 1 and 3, not so good from 2 and terribly from 4.
rsync from 4 showed about 3-4 kB/sec, and the copying of a file would generally fail after 15 minutes or so with a timeout error, even though the rsync timeout was set to 600 seconds.

So, I shut down modems 1-3 and then the rsync connection to 4 started working well, 40 kB/sec with no timeouts. So it seems that the cellular comm problems are related to the fact that the transmissions are interfering.

Would be nice to be able to test the systems at spacings more similar to what they will be in the field (or at least some thing like 50'). We'll see if that can be done.

More on ncar4 reboots:

• 6/10: Still giving problems
• 6/10: Tightened Viper power connector screw terminals. Still died.
• Noticed that for the past day the system is dying at 4 minutes after the hour. Previous to that it was dying about every 15 minutes. Every 15 minutes the system checks for internet connectivity, and after several minutes of trying will power down/up the router and modem. Perhaps that causes a power brownout and the system reboots.
• 6/10 12:36 MDT: switched to its own power supply instead of being daisy chained from aux1 on ncar2.

Sonics:  0539(Licor), 0537(Licor), 0671, 1119, 0853SPARE, 0855SPARE(Licor)

Licors:  0813, 1167

Kryptons:  1390, 1393, 1389(spare):  Added Kryptons(1.0cm) 1525, 1394

Props:  00002, 00319, 000318, 000315, 38770SPARE

Barometers:  B10, B5, B3, B9, B6SPARE

TRHs:  017, 021, 025, 027, 019, 024, 013, 018

dsms:  V2, V12, V14, V1, V6SPARE

Verizon:  R1/M1, R2/M2, R3/M3, R4/M4

GPS:  19S025786, 19S024353, 19S023925, 19S023685, 19S025780SPARE

Vbatt:  ID37, ID34, ID38, ID32

We want to sample the 7500's at 20 Hz. The record lengths in terse mode (no labels) are about 49 bytes long. This is a throughput of 49 bytes/sample * 20 sample/sec * (10 serial bits/byte) = 9800 bits/sec. This exceeds 9600, so we'll set the baud rate to 19200.

To change the configuration of the licor:

• adn
• minicom ttyS8 (start minicom at 9600 baud)

ctrl-A F   (control-A F sends BREAK, which puts the Licor in command mode at 9600 baud)

(Outputs (RS232 (EOL "0A") (Labels FALSE) (DiagRec FALSE) (Ndx FALSE) (Aux FALSE) (Cooler FALSE)))

ctrl-A F

(Outputs (RS232 (CO2Raw TRUE) (CO2D TRUE) (H2ORaw TRUE) (H2OD TRUE) (Temp  TRUE) (Pres TRUE) (DiagVal TRUE)))

ctrl-A F

(Outputs (BW 20) (Delay 0) (RS232 (Freq 20.0)  (Baud 19200)))

ctrl-A F

(Outputs (Dac1 (Source NONE)(Zero 0)(Full 5)) (Dac2 (Source NONE)(Zero 0)(Full 5)))

ctrl-A Q    (quit minicom)

An "Output" statement must occur all on one line, so they've been split into 3 lines above. After sending the Baud 19200 command you will see garbage data in minicom, because the baud rate has changed. You can change the minicom rate to 19200 with ctrl-A P F, or just exit minicom and see if the data system is receiving the data, as follows:

• aup
• wait a few seconds
• rs 8

The Licor data should look like so:

249\t0.08451\t14.7814\t0.04891\t349.722\t25.91\t82.6\t\n
249\t0.08450\t14.7759\t0.04889\t349.021\t25.87\t82.6\t\n
249\t0.08449\t14.7741\t0.04884\t349.602\t25.95\t82.6\t\n
249\t0.08450\t14.7807\t0.04892\t349.765\t25.95\t82.6\t\n
249\t0.08452\t14.7761\t0.04891\t349.665\t25.91\t82.6\t\n
249\t0.08452\t14.7775\t0.04887\t349.152\t25.91\t82.6\t\n
249\t0.08452\t14.7764\t0.04890\t349.480\t25.89\t82.6\t\n
249\t0.08451\t14.7803\t0.04895\t349.623\t25.93\t82.6\t\n
249\t0.08451\t14.7790\t0.04897\t349.508\t25.93\t82.6\t\n
ctrl-D