Blog from July, 2023

Status 31 July

A smidge cooler.  Calm in the morning, breezy in the afternoon

Onsite: Matt, Mack, Steve

In the ongoing quest to get beautiful vertical profiles, we have extended the 7m boom away from the trailer tower, arguing that blockage by the tower might reduce the measured winds (even upstream).  Mack brought out some booms that had been used with the scaffolding towers that are a bit bigger, longer, and stiffer.  After reviewing several options this morning, we came up with the Mack "V".  It required a bit of hole drilling, but nothing major.  We found enough hardware to assemble everything (using 6 clamps at one level on the tt).  The tower was down from 1225-1730.  We were a bit delayed by a poorly-seated D15 connector on the nano.  Note that U-bolts now holding the EC-100 need to be loosened in order to open the EC100 box (done to avoid stretching cables).  Before extending back up, we also releveled the trailer that seemed to be a bit off.

Also unplugged t38's Victron from 1000 to 1455 to help diagnose t38's low-voltage issues.


Status 30 July

Scattered high cloud all day.  Pretty normal.

Onsite: Matt, Gary, Steve (Gary leaves (sob) and Mack arrives (yay!) tonight.

Just a few tasks:

  • Matt shot a bunch more boom angles
  • Steve and Chenning redid DTS power, adding a battery on the load side.  So far, it has stayed up, but we'll see.  I'm considering adding a DSM to the DTS to log the Victron output – this might be useful in general to understand Victron operation.
  • Steve fixed the 7V problem on the hot film and plugged probes in at the lowest 3 levels.  Gary gave a quick demo and started up data acquisition – the beginning of SHINE!
  • Steve reseated the Victron USB connectors on t38, but it is highly unlikely that this will fix the issues with that DSM.  We'll try one or two things more before replacing it.

Another data analysis nugget:  I created a histogram of wind directions and see that even for spd>2 m/s and afternoon (12:00-20:00), NNW dominates.  SSE is a clear secondary maximum.  By my count, we have almost 14 hours of good conditions so far in this project.

This entry has been added because of questions concerning the DTS fiber and the power required by the electronics.

The Vicrons (Solar battery chargers / managers) will provide a maximum charge current of 20A to the any batteries being charged. Assuming we are using 12V batteries in parallel, then the maximum instantaneous power generated is 240 Watts.  Assuming a 12 hour solar cycle, the Victon, solar panel, battery setup, should be able to provide ~ 120 Watts of continuous power (day and night).

This power number could be increased if the batteries are put in series configuration to produce a 24V battery (however the load would have to be able to use 24 volts instead of 12 volts).  In this case the Victon would still charge at 20A, producing up to 480 Watts of instantaneous power, and yielding ~240 Watts of continuous power (day and night).  This does assume that the batteries are well matched and correctly handle a series configuration.


Our batteries are 100 AH batteries (1200 Watt hours, or 50 Watts of continuous power 24/7).  If we are using 3 batteries in parallel, they should provide about 150 watts of continuous power.


The peak production of a single solar panel (we have 2 panels on a frame)  is about 150 Watts.  If we de-rate this to 120 Watts.  We still should generally not require more than 2 solar panel (one frame) when we have the batteries connected in the 12 volt configuration.  The power from additional panels will not be used, however non-optimal orientation, or cloudy / rainy days will reduce power generation and could warrant additional panels added to the system.

Status 29 July

Scattered clouds with the usual temperatures and winds

Onsite: Gary, Matt, Steve (also for ISS)

Started the day by seeing the Solar Concentrator turn on at sunrise.

Outside work:

  • Matt started the Leica scans that show us pretty close in setting stuff up.  Boom angles are generally within 1 degree of design.  tt heights are within 35cm of designed.
  • I dressed the new t0 with hotfilm cables.  Later, Gary installed some probes.

Trailer work:

  • Found the bug in profile.plot() that wasn't averaging data, causing odd profile plots.  Now we begin to see that indeed 7m seems to be a bit low, presumably due to flow blockage by the lower tt tower structure.  Mack is bringing a couple of the long scaffolding booms from CME04 that we'll try to install on Monday.  Reran webplots back to 16 July.
  • Tested the spare CSAT3 some more and had 4 periods where up to 10 samples were dropped, during the heat of the day.  I guess I'll declare it a working spare.
  • Matt did dry soil core weighings for t49 and t23.  Still haven't taken t2.
  • Also did a quick analysis of w'w' across the array and confirmed that the different models of sonics see different amounts of variance.  There is good agreement within each type.  Variances from the CSAT3As are the highest, CSAT3Bs a bit lower by about 5%, and CSAT3s even lower by about 15%.  I note that this is the order of the sampling rate, and thus the bandwidth.  Not sure yet if something else is going on.



Status 28 July

"a mostly sunny sky" day.  Occasional gusty winds.  (We need to create a wind rose of our data sometime.)

Onsite: Gary, Matt, me.  Justin and Delia left this morning.  Thanks for your help!

A pretty quiet day.

  • Replaced the quad-disk-probe support tubing with a shorter one at 0.5m.t0.  Now all QDPs should be in the same relative position.
  • Matt and I took soil samples at t49 and t23 (only) because we only had 2 working tins.  We undercut the samples (though they may not have needed it, with slightly firmer sand/rocks at the bottom of the core), opened up the sampler, manually scooped out the top 2cm of the core, placed a 3cm ring in the sample tin, poured the sandy soil to fill the 3cm ring, checked that this was about the same 3cm in the corer, then removed the ring from the sample tin.  Rather strange, and subject to inaccuracies, but I think the best we could do with soil that is all sand.
  • Changed the Ubiquiti omni antenna with a 120-degree sector antenna, that resulted in a reported factor of 4 speedup in throughput.  With this, decided not to try to readjust t0's nanobeam position.
  • I spent the rest of the day trying to understand the profile jumps that we still see in the data, even after moving both tt and t0 so that they are now only ~60m apart.  Part of the issue is that we have been looking at spd computed from the 5-minute mean u and v, rather than the average of instantaneous spd.  The "spdprof" webplots now are supposed to be the latter (still not sure this is working), but we still see a jump.  I'll keep on looking at the data, but am increasingly becoming convinced to try a longer boom on tt, at least at 7m (though the profiles seem odd higher up as well).  BTW, I also see kinks in the CFACT profiles, taken similarly, that at the time I blamed on inhomogeneity of the site.
  • Tested the "spare" CSAT3 some more – in this case putting it outside to see if its prior failures were heat related.  In an afternoon of heat, it had about 6 seconds of data flagged as path speed of sound do not agree, but otherwise was okay.  If someone was desperate, we could try again to deploy it.

The best part was an overflight by an F-117 that circled our site this morning at reasonably low altitude while we were there.  It was being piloted by Matt's cousin's husband, who knew we were there.

Daily status 27 Jul

Another hot, cloudless, day.  Perfect for a long day out in the field....

Onsite: Gary, Justin, Matt, me.

Most everything was working (until we killed t0).  t38 also had a hiccup today, that Gary probably revived.

Spent the entire day moving t0 to a new tower about 100m to the east of the main tower line, in an attempt to get perfect vertical profiles with tt.  At the old t0, we mounted our last spare CSAT3A as 4m.t0p, though I don't think I got the configuration right yet.  I'll be adding this to qctables, etc.

In an attempt to think through the profile problem further, Chris and I had a discussion about trying to use longer booms at at least 7m on tt.  Another option would be to raise the new 20' t0 to 30' and put 7m on it.  Keep those ideas coming, folks!

P.S. The t0p problem was that its EC100 also was set to ASCII.  Grrr.  I connected over minicom and changed it back to binary.  Also updated stats_5min.xml on ustar to include it.  And qctables now displays t0p.

Daily status 26 July

Another scattered cumulus day with south winds in the afternoon

Bill left today.  Onsite: Justin, Gary, Matt, Delia, me

Biggest event was a virtual tour for the ASP Colloquium from here that went pretty well.  As that was finishing, Mark from the FBO and his grandson showed up and also got a tour of the trailers.

The only data issue is the GPS on ttb that seems to lose satellite reception, presumably since it is nominally under the ttt DSM and thus is blocked from seeing the sky.  We're considering just taking the entire ttb DSM and putting it on the trailer bed.

Delia and Gary made tweaks to her code to use the DSM WiFi and configuration switches and installed it on t0b.  We'll get to play with it now.

Gary is also trying to track down a bug that disconnects the USB and doesn't bring the USB stick back up, just on t38.  He installed code that seems to bring the stick back up, but still doesn't know the source of this problem on just one DSM.

I fired up the CSAT3 sonic that we removed because it was giving error code 31(?) on every sample and thus reporting nans.  This seems to mean that the signals aren't right and that the path speed of sound values don't agree.  On the bench in the trailer it works just fine.  I wonder if heat got to it.  Maybe it is half of a spare?  It seems odd to send it to Campbell saying that it works fine....

The big question is whether we'll move the t0 profile measurements to another tower (either t49 or a new t50 near tt.  I haven't convinced myself using the data that this will help, but Chenning seems to really want t50.  Hopefully, I'll see something in the data soon to justify this move.

P.S. After dinner, went back out to the site to confirm that the tt beacon light is visible.  There are brighter red lights around the airport, but tt's light definitely is there.  Also saw 2 satellites, despite a bunch of clouds and a half moon.


Horizontal array data

Yet another slightly worrying artifact in the data is the turbulence across the horizontal array.  In the web plot here (a new type of plot I created for this project), I see streaks where every third(?) sonic has slightly higher vertical velocity variance than its neighbors.  I actually designed the deployment array so that we could check that this type of artifact hopefully didn't exist.  The conclusion would be that one of the types of CSAT sonic reads higher vertical velocities than another.  This could be in part related to the sample rate, since the CSAT3A and 3Bs are running at higher rates that the CSAT3s and thus might see a bit more velocity variance contributed by high frequencies.  It is also possible that something more subtle is going on.  We should probably start running some power spectra from adjacent sonics.

Daily: 25 July

Scattered clouds throughout the day, with the usual afternoon winds.

Onsite: Justin, Gary, Delia, Matt, and a lot of help from Chenning!

Everything looked good this morning, except for GPS.ttb.  Not sure why it stopped getting satellites again.

Visited with Mark at the FBO who stated that the space between the runways was actually preferable to our original location since the patterns are flown to the outside of the runway locations.  Thus, we immediately set out to move tt.  10:00-11:30 lowered and packed up tt.  11:30-12:00 drove it to its new location. 12:00-13:30 did about half of the set up.  This took a while since one of the pins that lock an outrigger leg became unscrewed in a way that the teardown crew would not have been able to retract it.  We ended up having 7 of us pull the entire outrigger leg out (both sections) so we could rotate it to let gravity help release the pin.  It worked!  After this, we took a lunch break, then returned from 14:30-16:00 to level, finish hooking up, install the Ott, and raise the tower.  It seemed not to go up quite as far before the guy wires tightened up, so we let the guy wire tensions dictate the height.  To the best of my ability with a boy-scout compass, the trailer is oriented at 248 degrees magnetic, 260 degrees true, just like it was at the old location.  (Now that Matt is here, we'll soon begin the full instrument survey with the Leica.)

Unfortunately, from the plot below with the first amount of data (from the good southerly wind direction), we still don't have perfect vertical profiles.  There still is a jump between the 4m and 7m observations.  According to our plan, this suggests that we should either move the t0 lower profile sensors to tt or to a new tower even closer to tt.  I was really hoping this wouldn't be necessary.  I suggest we wait for at least a full day's of data to come in before making this decision.

t47 Victron restarted

The t47 victron stopped reporting about 5pm today (about the time that Gary and Bill were adding a webcam near it).  ddn/dup brought it back.

Daily status 24 Jul

Overcast, occasionally rainy, some big gusts in the afternoon associated with thunderstorms

On site: Gary, Justin, Steve.  Matt and Delia arrived in the evening – welcome!!

Started the morning going through yesterday's punch list:

  • swapped batteries at t44, found an issue with solar panel cabling that we fixed.  Assembled the charging system to recharge the old t44 batteries.
  • checked t41 power, but everything seemed okay.  Its voltages and currents haven't changed.
  • Lowered tt from about 10:15-11:30.  Pirga was set to external barometer, but the PTB110 was reading 500mb flat.  Changed the EC100 to use the "internal barometer" and now all is good.

This got us to a status of everything working!

Spent the rest of the time working on options for t50/tt.  The options appear to be:

  • Build another tower near tt and move t0 sensors to it.  The local surface is rougher, so the profile won't represent the horizontal array.  Possibly reinstrument the old t0 with 2D sonics.
  • Move tt to a location to the NNE of t49.  The surface and topography are essentially the same as for t0.  We should do something (black paint/wrap in foam?) to the lower section of tt to avoid generating specular reflections if REAL scans it.  We need to check with Mark if this is acceptable to air traffic, but he had left for the day by the time we checked.  We'll talk with him tomorrow.
  • In the case that we move tt, we might move the t0 sensors to t49 to be closer to tt.  Chenning is pretty insistent that we should build another tower (t50) close to the new tt location, but that would require an extra sonic and I don't see how a change of 110m would make any difference.  Chenning has agreed (I think) that we could look at the profile data after the tt move to see if a t0 move is needed.

I appreciated Will's comments that pointed us to the 2 spare base plates, among other things.  I also tried plotting profiles (just wind speed so far) for only southerly winds and confirmed that profile differences are not just due to a sampling of poor wind directions.  We have a persistent problem even for the best wind directions, unfortunately.

With the disposition of t0 still unknown, it didn't make sense to work on the hot-film set-up.

So, some issues still in the air, but hopefully we'll have a resolution tomorrow....


Tower 50?

As in past projects, going back to CASES99 and most recently at CFACT, I split the vertical profile into a short tower and a tall tower, to avoid the influence of the tall tower's superstructure on the airflow sensed by the lower sensors.  For M2HATS, I placed the short tower near the horizontal array to reduce the number of CSATs by one, with 4m.t0 doing double duty for the vertical and horizontal array.  However, due to the airport glide slope restrictions, tt ended up almost a mile away.  Due to the overall homogeneity of the site and the length of the fetch for the tt sensors, I thought that this separation would be acceptable.

However, after walking around, Jielun commented on the differences in vegetation height at these two locations and Chenning commented today about the difference in terrain height (and maybe slope) at these two locations.  The profile webplots that I just got going today confirm their suspicions: there are definite jumps in the combined wind speed and temperature profiles.

Chenning suggests that we add a short profile tower near tt to create mean velocity profile at the same location.  His suggestion was to use our 2-D sonics (wind observers) for this – I'll assume at 1, 2, 3, 4m.  This would require:

  • Another 15' tower.  I assume (though I didn't look) that we have at least a 10' and 5' section left over.  I don't know if we have a spare base plate.  I assume that we have enough ratchet straps and stakes for another set.
  • A box of (4) wind observers would have to be shipped from Boulder
  • We should have enough extra booms.  The wind observers have compatible cassettes (though we've never put them in a vertical profile before, surprisingly).  We have 2 spare tower clamps, I think.  We could probably improvise another 2, or have two more (the last 2 new, or 2 very old style) also sent from Boulder.
  • We could create another DSM from a spare or just plug these sensors into unused ports on ttt – we should have enough long serial cables (4x35m?) for this.
  • The extra power draw on tt shouldn't be too large.
  • All of this begs the question of whether we should also add corresponding TRHs.  We have 4 SHTs.  I haven't looked but assume we have 4 spare housings.  We have 4 spare TRH booms.  I don't know how many extra cell clamps we have – there are a few.

I'd appreciate comments from anybody about this project addition!


1 Comment  · 
t44 down

It appears that t44 went down about 8:30 tonight (while we were at dinner).  Vbatt shows that power crashed (last reading was 10.8V).  I'm not sure why, and I guess I'll have to do a better job of monitoring power!  (I had been paying attention to t0 and t47, that have larger-than-normal loads.).

P.S. It appears that this site has not been charging at all – either a bad Victron (or setting) or a bad connection to the solar panels.  In either case, should be an easy fix.  We'll want to swap in a set of charged batteries to jump-start it.  This will also give me an excuse to set up the charging system in the base.



Operations, day 1!

Weather: Initially calm and warm, then windy and overcast in the afternoon

Staff: Just me and Gary.  Justin is taking a well-deserved day off.

There were only 2 items on our punch list: fixing the t7 sonic and the 28m ec150/ec100 config.

  • I swapped our only full spare sonic (the new CSAT3B, after the usual mounting modifications) at t7.  It took a while before I remembered to update the config on ustar, so data were a bit late to get added to the plots.  Now all sonics are working!
  • Gary also noted that ttt had stopped collecting data and restarted it this morning.
  • I wanted to have the Clemson students help with lowering the ttt tower, but by the time they were available (after the 3? PM radiosonde), I judged the winds to be too high for a new crew to do this task.  We'll wait until tomorrow.

I spent most of the day working on qctables and webplots and now have all running as I intended – not to say we can't add more plots.

The students arrived back from yesterday's trip to Vegas with a 1000W pure sine inverter, which Chenning tested and worked fine.  They installed it (though deciding to connect directly to the battery, rather than the Victron load output) and it seems to be running.  I may still add a solar panel or two to his setup.  (Just trying to decide if one should be added to t0, before committing more to Chenning.). I'm still not sure if he has the DTS fiber running or not.

The students (with Bill) launched 2 sondes today.

Sounds like everyone was somewhat happy with the way things are working for our first day.  I certainly am.

Tomorrow's tasks:

  • get t44 power back
  • also check t41 power, since its Vbatt is lowest
  • change P config on 28m.tt ec100
  • hot-film:
    • finish support installation
    • add battery+shorting caps for further testing
    • check that probe is still intact after yesterday's dust storms


Burn-in Day 2

Yet another day in the desert.

ISFS staff: Justin, Steve, Gary (mostly).  Many ISFS set-up staff departed in the wee hours.

Today mostly was working through the rest of the punch list from yesterday:

  • Sonics
    • t26 and t9 EC100s were reinstalled, now in binary mode, and were working
    • t2 had the EC100 green connector pulled and came up
    • 7m.t0 also had the EC100 green connector pulled.  We used the ladder as an extension ladder to access the 7m level on tt – a bit dicey, but easier than pulling down the tower for this simple fix.
    • t7: moved this CSAT3 to another port and are seeing messages, but there always are error flags (9, 13, 31) and thus no good data.  I'll do a better visual inspection of the head tomorrow.  If no obvious fix, we'll have to replace it with our 1 spare (the new CSAT3B that Tony got working yesterday)
  • TRH
    • t2 was missing the SHT probe, then still wasn't reporting data.  Found that one of the Bulgin pins on the housing was pushed in.  Pulling it out (and feeling it click) got it going.
    • 7m.t0 had a blown fuse on ttb, DSM1.  Replacing it got it going.  Note that this is the "chewed cable" – maybe that is how the fuse blew?
  • Nano
    • Yesterday, we had isolated this problem to the internal wiring within the EC100 box.  Today, found that the pins in the Bulgin connector were rotated about 180 degrees.  (pin 1 was in 5, pin 5 was in 1, pin 6 was in 2).  I swapped these, but managed to break two of the tiny wires in the process.  Just made a big solder blog to hold them together, since I didn't immediately locate stuff to re-terminate Bulgins.  Seemed to work and was reinstalled.
  • Hot films (Gary spent a bunch of time on this and may want to write his own blog (or edit here))
    • Jielun looked at the data that Gary provided yesterday and sees that the signal hits a noise floor at 200 Hz.  This is probably usable, but we'd like lower noise.  After a bunch of (probably unrelated) effort, Gary is now taking another run with the hot-film applying a low-pass filter at 1kHz (I think).
    • Gary had noticed strange power behavior, with both(?) t0b and t0t Pis reporting low voltage and going through repeated boot cycles.  We looked at the Victron through its App and once saw it say that it briefly turned its output (load power) off, but not nearly as often as the Pis reboots.  Power levels went up to 56W (and idled at 35W), but still should be within Victron capabilities.  At one point, we got cycling to stop by pulling power to some of the CTAs (hot-film electronics), but the total power before was less than 3W, so this shouldn't have made a difference.  We left with things seeming to be happy again, with each DSM powered from the Victron using a splitter and only one CTA powered up (through t0b's AUX port).  We'll try introducing a 4-way splitter tomorrow, though I'm not sure where it ended up after clean-up.
    • We remounted the hot-film bridge boxes that might let us access the 4m level with existing cables and re-routed (temporarily) the cablesN
    • We mounted the 0.5m probe support

Tomorrow is supposed to be the start of operations.  Nearly everything of ours is running, except:

  • The above-mentioned sonic at t7
  • The EC150 at 28m on tt (of course) has never worked.  It has Pirga set at 500mb, rather than reading a sensor.  This is easy to fix using ECmon, but appears to be impossible over the serial interface.  Guess the tower will be coming down after all!
  • Adding more hot-films.  We'll actually delay a day or three before installing more probes themselves until we get a feel for how long the one survives, but will get the rest of the cabling set up.
  • Chenning's DTS still has power issues.  His students went to Vegas today to attempt to get a better inverter.  We'll help him get it installed.

I'm pretty happy with this status!  Thanks everyone for getting us to this point!