Blog from November, 2018

Teardown, Day 4

Continued mild temperatures (40s), making the gully somewhat muddy.  Indeed, we couldn't drive out the normal way with a loaded pickup, so we drove out through the former SODAR trailer access.

We're almost done. Today, removed almost everything from the field, including the concrete blocks that we returned to the company that rented them to us.  Thanks to Ken for again allowing us to use the backhoe and his trailer for the concrete block removal.  We did a walk-through and found mostly minor bits of trash.  We left the Wisconsin temperature sticks in place in the field. 

Tomorrow:

  • We need to secure the base trailer for pickup by Trailer Transit at noon.
  • Lou will call to arrange for pickup of the porta-pottys.
  • We plan to pick up the Penske and load it (stuff is now just at the storage locker and at Ken's).

We will all travel home on Sunday.  (Bob leaves tomorrow.)  ISS is done, and Andy is flying back as I write this, though their trailer pickup isn't until Sunday.

Teardown, Day 3

Still warmer, somewhat cloudy. I no longer felt comfortable taking the rental car into the field.

Another amazing day for the crew!  All towers down.  All sensors packed.  Base trailer mostly packed.  Remaining tasks:

  • remove concrete blocks (anchor points)
  • remove all tower hardware and solar panels from the field (probably will use pickup to stage to the road, then load the Penske later.
  • Penske loading to clean up the storage locker and stuff that we've staged at the base trailer.
  • final sweep for experiment residue.  (Noctis was cleaning up PI residue again today, and should be mostly done.)
  • prep base trailer for transport (remove porch).

The ISS crew completed teardown of the field site, including bringing the sodar trailer to Ken's.  They just have to finish packing at Ken's.

Soil notes:

  • again, kept soils together with the mote.  Mote 32 was from the corn site, mote 31 was grass
  • during soil removal, checked that Gsoil white dots were up and that EC-5s were positioned lettering up (wrong, according to an email from Meter to Hendrik!)

One teardown issue:  Bob noticed that the stub cable on several screw anchors (including both on one init guy wire) had partially slipped over the cap of the anchor.  This is potentially very dangerous.  We will need to double-check these anchors in the future.




Teardown, day 2

Warmer, maybe up to 40 today and sunny in the morning.  This was bad because the ground is starting to get muddy, though we could still drive in.  Nevertheless....

All towers are now stripped, with most sensors put in boxes.  Kurt & Bob took care of upper convergence and other clean-up tasks while Dan and I stripped release and init.  The only sensors still deployed are the grass soil suite.  All of the 10' towers are down.  Noctis had a crew that removed a lot of the PI residue (his power system; electric fence materials; fog generator; etc.). Now, the field has only:

  • 10&20m towers (bare)
  • solar panels
  • grass soils
  • the PI's release platform
  • the Wisconsin temperature poles

We rushed since we are worried about field access as the weather warms, and rain is predicted to arrive soon.  The base trailer is a big mess now, but we'll have time to sort it out in the coming days.

Again, we're very pleased by this progress!

Tomorrow the towers will start coming down.  Friday, we hope to remove the concrete anchor blocks.

P.S. After various travel delays, the ISS crew also was onsite today and (at least) packed the 449 antenna.


Teardown, day 1

snow flurries, and about 0.5" on the ground.  Cloudy, temp about 25F with moderate wind.  Ground is frozen for first 2cm, so vehicles could drive into field. It was a bit hard to chip through the crust, but the ground was soft to remove soil sensors.

Kurt & Bob Wiley tore down all 6 pressure sites.

Dan & Steve (with some help from Kurt & Bob) removed all sensors/cables/batteries/aux towers from lconv, leaving the stripped 20m tower and solar panels.

Moved all sensors back to base.  Infrastructure still in the field.

We took a lunch break to warm up and refuel (ourselves).

Labeled batteries:

  • Golf batteries: LC0'm, LC1'm (0 connected to charger)
  • Other 2 lconvm: LC2m, LC3m
  • Other site batteries: LC0, LC1 (only 2?)
  • Also labeled power monitors as LCm and LC.  Note that LCm had a non-sunsaver controller.

Decided not to record sensor serial numbers, since should already know them.  I wasn't sure about rad/soils, so:

  • left all soybean soil sensors cabled together to mote 21
  • 2794(?) NR01 came from soybeans, placed in middle, next-to-bottom row in black case
  • NR01 from 20m.lconv is in the right, next-to-bottom row in black case.

All in all, we're pleased with how the day went.

Weather: Today was snowy/icy and cold, with light winds and overcast skies. 

  • All sites made it through the night with no evident problems. The new golf cart batteries seemed to do great overnight, maintaining the highest charge. 
  • The project has officially ended, so we removed the external hard drive from ustar, dropped the truck off at the base trailer, and locked up the base. I have the truck keys with me on the way back to Boulder.
  • The network will remain up until teardown.
Status: 14 November

Weather: Today began mostly sunny and clear, with breezy winds and cold temps. Partly cloudy skies moved in from the south by around noon. Tonight a front will bring a wintery mix through the day tomorrow. 

  • Lconvm failed this morning shortly after the sun began charging the batteries (around 0800 CST), but otherwise everything made it through the night. It appears that the fuse yesterday was not the issue. Our plan was to test every link in the power "chain" if the station was still down when we arrived. Unfortunately, the site was back up by the time we got to it. It appeared as though the increase in cloud cover could have been impacting the power, with the Sunsaver shutting down the power when the sun was fully out. The Sunsaver was blinking all three LED's in sequence (red - yellow - green) and had the charging light solid red. According to the manual, this indicates a failed temperature sensor on the Sunsaver, which is a critical error that would shut the power off. We decided to replace the Sunsaver with an older version we had in the base trailer, which seemed to solve the problem. We gave it about 40 minutes without incident before deciding to change the batteries to the new 6 V golf cart batteries.
  • Lconvm officially has the golf cart batteries, wired in series, as of about 1600 CST. The batteries should have been 90% charged. We did blow the fuse on the power monitor board, but after replacing this everything seemed to work.
  • We left the ATV parked behind the solar panels at uconv. We feel it should be safe there and is mostly out of sight.
  • We noticed no other problems. Given the forecast conditions the next 2 night, April is calling off any chance of ops through the end of the project.


Status: 13 November

Weather: This morning began overcast before clearing out by noon. Breezy most of the day and very very cold. The forecast for tomorrow is similar to today with clear skies and slightly higher temperatures.

  • Multiple sites did not make it through the night again, as we feared due to the weather yesterday. Lconvm failed around 1730 CST, although we suspect this was likely due to a loose fuse on the power monitor board. Init1/t failed at 0217, lconv1/t failed just before 0400 (could have been due to the lconvm Ubiquity being down because voltage was 11.8 at the time), and rel1/t failed around 0617. Initm, relm, and uconv1 made it through the night. Other than these power issues, no obvious issues were noted today. 
  • Terry visited us today, taking a tour of the sites and providing valuable input on some of our issues. Based on his input and suggestion, we will purchase two 6V golf cart batteries and wire them in series at lconvm. We will get these batteries tomorrow from a local battery and golf cart specialty store (very conveniently exactly the store we needed) and see how this site makes it through the next week and a half.
  • At the end of the day we noticed that all of lconv had been dropping in and out all day. As we troubleshot the issue we determined that lconv1/t were operating just fine and had complete data files. Lconvm/2 both had many data files being created with only a few seconds/minutes of data in each. We think that the power was out between 1230 and 1600, perhaps intermittently, which prevented the data from lconv1/t from being sent via the network. We didn't notice any obvious reason for this power outage, although I noted that the fuse on the power monitor board seemed a bit loose and I pushed it in completely. The power had definitely returned to the station before this, but since then we have had no issues. It also seems like the station was in and out in a similar fashion yesterday, so it is plausible that we loosened the fuse when we were working on the site that morning. We left unsatisfied, but unsure what else we could do given that everything was working fine at the time.
  • Just a note, the power cord situation in the SODAR trailer is a mess. 

There is an IOP scheduled tonight beginning at 0200. 

Status: 12 November

Weather: Overcast, cold, and breezy today, with intermittent snow showers developing by about 1100 CST. Tomorrow is expected to be sunny and clear, but remain below freezing all day.

  • There again were no obvious issues noted with the data today. 
  • Due to the lack of sunshine today we decided to attempt to charge lconv and rel. We used the charger from the base trailer for the first time and had interesting results. It appeared to charge the batteries very quickly, from 35% charge to full within 10 minutes. The charger, multimeter, and Sunsaver all agreed that the batteries began drained and were fully (or at least mostly) charged within minutes. We found this hard to believe, but found nothing that suggested the batteries were not full. We left satisfied that the batteries had enough charge to make it through the night. 

There is no IOP tonight, but one is scheduled starting at 0200 on Wednesday morning. 

Status: 11 November

Weather: Mostly sunny skies again today, with some clouds rolling in just after sunset. Light winds much of the day and a bit warmer than the past couple of days. Tonight a cold front should arrive, bringing overcast conditions and potential for snow showers throughout the day tomorrow. 

Tasking:

  • There were no obvious issues noted with the data today. 
  • When driving by P4 this morning, Ned noticed that the tubing was disconnected from the pressure port. We have no idea when this could have happened. We replaced the tubing and zip tie. 

There will be another IOP tonight starting at 1900. No IOP is scheduled for tomorrow.

Status: 10 November

Weather: Sunny and clear skies all day, with breezy conditions until the afternoon. Very cold, with temperatures around 22 F this morning and a high around freezing. Tomorrow looks to be clear again and slightly warmer, with a small chance of snow showers overnight into Monday. 

Tasking:

  • The morning data check again revealed no obvious issues, but yet again not all of the stations made it through the night. Lconv failed first, around 01:30 CST, followed by rel prior to 0600. Uconv made it through the night and we think Init likely did as well, but we didn't look very closely at all of the data files. Ned and I suspect that the cold is increasing the load, as evidenced in nchart plots of Iload, and it is possibly from the TRH fans. We also wonder if the fan bearings going bad may increase the power consumption of the TRH's. 
  • We turned on Ubiquity data streams and left them on. 
  • The 4 additional TRH's that Dan sent have arrived and we assembled them this morning.
  • Replaced the 1.5-m TRH at Init. We decided not to replace anything else due to the bitter cold this morning. We believe there are a couple of TRH fans that are going bad. 

There will be an IOP tonight starting at 1700, and another is scheduled for Sunday at 1900 CST.

Status: 9 November

Weather: Overcast with light snow in the morning, then overcast through the afternoon becoming partly cloudy by sunset. Very windy throughout the day, increasing through the early evening. Saturday is expected to be sunny.

Tasking:

  • The morning data check revealed no obvious issues. 
  • We are concerned the batteries will not make it through the night, since there was very little solar getting through today. We will test it to see if the batteries can make it, as they should according to Steve's calculations.
  • No site visit today otherwise.

There will not be an IOP tonight, but there is one scheduled for early evening on Saturday.

Status: 8 November

Weather: Mostly cloudy this morning, but mostly cleared out by noon. Breezy winds and cold temperatures persisted throughout the day. A system is expected to bring rain/snow overnight (1-2 in. possible), tapering off by late morning, but remaining overcast all day. Saturday is expected to be sunny.

Tasking:

  • The morning data check revealed that no data was making it to Boulder since about 1700 CST last night, thus no QC tables or plots were available. Steve fixed the issue around 1030 this morning. Between cockpit and ncharts we did not find any obvious issues with any sensors. 
  • We decided to charge the batteries at lconv using our generator and 2 chargers. We disconnected the solar and measured the voltage on each bank of batteries. The batteries as lconvm showed less charge (around 63% and 12.0 V  according to the charger) than those at lconvt (around 75% and 12.3 V). We charged the lconvm batteries for over 2 hours and couldn't determine if they ever reached a full charge. At about 2.5 hrs the charger showed 100% charge, but the Sunsaver still had the yellow light illuminated. Upon disconnecting and reconnecting the charger is showed significantly less charge and lower voltage than before, which led us to determine that we don't really trust this particular charger. The lconvt batteries did appear to reach sufficient charge to illuminate the green LED on the Sunsaver after about 2 hours. After this we gave up and reconnected the batteries to the solar panels.
  • The sun started to show through the clouds near 1000 CST. After maybe an hour of this direct sunlight we tested the batteries at rel and found that the green LED's on the Sunsavers were illuminated and the batteries showed a charge above 12.8 V. From this we determined that the solar panels must be charging the batteries significantly faster than our chargers. Eventually we determined that all of the batteries connected to solar were "fully" charged by noon (or earlier), after only a couple of hours of good sunshine. 
  • We are not confident that our backup plan of manually charging each set of batteries on cloudy days is feasible. 
  • Ned noticed that the TRH fans sound like they are going bad at 1.5-m on relm and initb. 
  • We also noticed that rel1 2-D Gill is connected to port USB0, while the TRH is in port USB2. This was reversed on the sensor list on the Wiki, which has been updated.
  • I returned the Ubiquities on all towers to the 3-hr reporting schedule to try to save some power. This was done just after 1600 CST.

There will not be an IOP tonight, due to the weather. Hopefully the batteries will hold enough charge to make it through Friday night, as tomorrow should be very cloudy.

Status: 7 November

Weather: Mostly clear today, with plenty of sunshine, but cold and breezy until late afternoon. Tomorrow should be partly to mostly cloudy with similar daytime highs as today and light winds. Rain is expected after 1800 and should transition to snow after midnight.

Tasking:

  • The morning data check revealed no obvious issues. Given the amount of sunshine we had yesterday we expect that the batteries should be fully charged. 
  • We fixed the generator. The choke line had become disconnected, preventing the choke from engaging to allow us to start the engine.
  • At 1230 CST we turned on continuous broadcasting of the tower data through the Ubiquities. This was done to enable us to check for instrument issues, as well as provide data for tonight's operations. We are not planning on turning this off.
  • Ned noticed that the TRH fan was failing at 0.2-m on rel1. Since we had no spares, he swapped it with the 1.5-m TRH from Initb. It sounds like a few other fans are going bad and the QC plots suggest this could already be impacting the T data. Relt 20-m TRH also seems to have failed or be failing.

There will be an IOP tonight, but it could end early if winds pick up. IOP is scheduled for 2100 start time. 

Status: 6 November

Weather: Mostly cloudy this morning with gradual clearing by midday today. Cool and very windy through the afternoon. Tomorrow's forecast is for mostly sunny skies and cooler temperatures, with gusty conditions much of the day. 

Tasking:

  • The morning data check is considerably more challenging now that data is only reported every 3 hours. At first glance, everything looked normal, however under closer inspection it seems that at least some towers did not make it through the night. We weren't able to confirm if all towers went down, as relm and uconv1 both reported at all Ubiquity "windows" through the night. Plotting battery voltage suggests that lconvt/lconv1 failed before 0300, relt/rel1 failed around 0305, lconvm/lconv2 failed around 0320, and init failed sometime between 0330-0600 CST. My guess is that relm also failed after 0630, as it showed a voltage just about 11.5 V at that time and the sun was not yet rising. Conversely, uconv may have held on throughout the night since it showed a voltage of 11.87 at that time. The batteries did receive some solar charging yesterday, all peaking above 13.5 V just before noon before dropping off fairly rapidly after noon.
  • The initial plan for today was to use our generator to power 2 battery chargers that Ned purchased, but a) we couldn't start our generator, and b) by the time we hooked the chargers up to batteries at rel using April's generator, the sun had started to poke through the clouds enough to make it difficult to discern what impact (if any) the chargers were having. By about 1200 we had tested voltages at all battery banks and determined that all were nearly fully charged at 13+ volts. This indicates that the solar panels were charging the batteries and thus the impact of our charger was unclear.
  • We further tested the charge of the batteries at init and removed one to test with the charger in the base trailer. The test gave us no further insights. The DSM on initm was powered off during this time, approx. 1400-1500.
  • We also twisted ratchet straps at most side towers in order to reduce the vibrations of the straps in these strong winds. Our efforts seemed to be successful in doing so.
  • No other issues were noted at this time.

It appears there will be no IOP tonight, but one is tentatively scheduled for tomorrow night beginning at 2100 CST.

Status: 5 November

Weather: Mostly cloudy today, with afternoon rain showers. Cool and windy again. It should be less cloudy tomorrow, but cooler. 

Tasking:

  • The morning data check revealed that no QC table was generated today. The 18-00 UTC table from yesterday showed some NA's across all data on several towers, but not on all towers. Further inspection showed that data dropped out as early as ~2030 CST at lconv and ~2300 at rel, which brought down the Ubiquity network and ended all data transmission. This was clearly another power failure, as yesterday was a completely cloudy day. Plotting battery voltage and Icharge over the last 4 days shows a pattern of subdued charging during the day on these cloudy days and battery voltages dropping to 11.6 V before the data drops out at night. This is consistent with what we saw before the last power failure. The decision has been made to stop streaming data in real-time over the Ubiquities in favor of periodic data transmissions (every 3 hours).   
  • No other issues were noted at this time.

There will be no IOP tonight.