This is a place where we can provide information (via the comments) that relates to realtime data QC issues. One benefit is to consolidate all data QC relevant issues in one place, rather than looking piecemeal through the blog. Let's try it out and see if this works.
103 Comments
Jacquelyn Witte
Pressures at site c
P.0.8m.c and P.1.0m.c has been varying by as much as 60-70 hPa!! Looks like it's resolving itself in the last few hours though.
Don't panic -
P.c at 0.8m, 0.9m, 1.0m, 1.1m at capped off and left for the PI’s to connect an inlet. Hopefully, we’ll see a change in the data?? These lower level pressure sensors are to measure the pressure as the snow accumulates.
Jacquelyn Witte
Site c for center
Jacquelyn Witte
Site ue for upwind east
Jacquelyn Witte
Steve answered all my questions. All these missing variables are artifacts. Good to know and ignore if they remain in NCharts.
Jacquelyn Witte
h2o.5m.c and co2.5m.c not reporting since 2022-Nov-02 16:27:30
Maybe related:
sonic values at 5m at c site stopped reporting since 2022-Nov-03 03:33:30.
Jacquelyn Witte
I'm sure some of you are aware of this:
Data reporting stopped on NCharts and QCTables since the morning of Nov 6.
---------------
As of this afternoon - real-time data are reporting on NCharts and QC Tables/Plots.
Thank you Gary!
Jacquelyn Witte
All Snow Pillow Loads and SWE at p3.c are recording zero's (or nan's).
lexc at p3.c stopped reporting on NCharts 8 Nov ~03:07
--------------------------------------
Steve did a pio and brought data back online.
Jacquelyn Witte
All snow pillows are not reporting:
I haven't yet learned how to power cycle the ports that the sensors are on. Can anyone who knows please do it?
Gary Granger
I cycled power on the snow pillows. However, only pillow 1 was not reporting. Note in the qctable above that SWE is working for pillows 2-4, so the missing Load for all pillows is something else, admittedly mysterious. Load is definitely being recorded and reported at the DSM.
Also, it's unfortunate that we see NAs even when a pillow is working, since the pillows are only reporting in every other 5-minute period, but I'm not sure it's worth doing anything about that.
Jacquelyn Witte
Looks like the sonic at 5m.c (u.5m.c, v.5m.c, w.5m.c, spd.5m.c, dir.5m.c, etc) has followed the irga (h2o.5m.c, co2.5m.c ) and had been giving nan's since 9 Nov. I suspect the entire CSAT/IRGA needs to be replaced.
Steve Oncley
I'm cautiously optimistic that this problem is <just> the EC100 and that the heads are fine...
Jacquelyn Witte
No data has been recorded on NCharts and Tables/Plots since 13 Nov ~3pm local.
Network disruption? Nagios is not responding ...
In other news:
Gary Granger
There were some network disruptions yesterday. dsm_server has been restarted on barolo, so data are flowing again.
Isabel Suhr
Still no new netcdfs an hour after you restarted. I saw that dsm_server had been restarted but that the two statsproc instances had been running since the 9th. Managed to restart the statsprocs but only by using kill -9 (I don't remember having to use -9 on statsproc before, that's usually just dsm_server), looks like we're getting netCDFs again.
Jacquelyn Witte
Tsfc.Ap at all sites are recording nan's
Jacquelyn Witte
h2o.15m.c has been reporting nan's since 15 Nov on QCTables *BUT* there is data showing up on NCharts. I cleared my browsing history but it's still nan's on QCTables.
QCTables is behaving strangely. Tables are missing the color coating for min/max/median/nan's values. Looking at it on the Safari browser shows the same.
Steve Oncley
The nan's on 15m are because all the humidities are now lower (due to colder weather) and 15m has now drifted negative. However, the signal variations track the other .c h2o values, so I'm calling this just a bias, that won't affect the flux calculation. Also, a visual inspection by Will who is now on the tower didn't show unusual dirt on the optics. Thus, I've suggested to the team now at the site that they leave this alone. I will next update the xml to widen the "acceptable" h2o range.
Steve Oncley
Yesterday (11/27) in the morning (0130-0900) many of the h2os went nuts, the worst were the lower sensors (1m). There is just a slight increase in Wetness during this period, hinting that this was due to frost. I'll try to check if (internal) heating is enabled on these sensors.
Steve Oncley
One spike on Vtherm.c (apogee) at 21:30 last night (11/28).
Snow overnight (also in Boulder) has caused sonic 1m.uw to go out and ec150 at 1m.ue to drop some data. Further checking shows that 1m.uw diagnostic bit 0 is being set, causing NIDAS to set data values to NA. I think this bit is "signal amplitude too low", due to snow blocking the acoustic signal. I haven't even tried pio, since I'm guessing that the snow will unblock itself with time. Also note that the values h2o&co2 are fine from the EC150, but are showing as NA since they are in the same stats_proc group as the sonic w, which is NA.
Steve Oncley
The 1m.uw sonic came back about 15:00 yesterday (11/29).
Matt Paulus
It looks like the 12m TRH fan may have died this afternoon (11/29). Rfan shows ~0 since 12:27 pm (preceded by a spike) and there is a noticeable, corresponding increase in temperature after this time. We may want to plan a TRH housing swap for our next maintenance visit.
Steve Oncley
I just tried a pio power cycle, which didn't fix the fan. I think that -99 means that the TRH PIC didn't receive any revolutions counts from the fan within a certain time period.
The below plot of the raw data, shows that the drop was pretty instantaneous, with a large value followed by zero values, then an occasional attempt by the fan to restart.
Matt Paulus
No data has been recorded on NCharts or QCTables/Plots since about noon on 1 Dec. Nagios is showing all green for the network and the data appear to be making it to Boulder. The most recent netcdf file is from noon on the 1st, so I'm guessing this is a statsproc issue.
Carol Ruchti
Sorry to have looked through the SOS data bit late today. I did just check the data over the weekend as well.
Rfan.12m.c has been turned off as Steve noted. You can see the temperatures continue to be too warm for T.12m.c over the last 6 days.
As for the SWE issues at c that Steve was seeing Nov 21 or about 2 weeks ago, it seems like that been resolved now that it is colder and snow covered. All snow pillows have been working well as far as I can tell.
One potentially naive question.... what's happening when co2 and h2o are negative? Is that an instrument issue or something else? Is this a frost issue that's going to persist through the winter like Steve was suggesting in his comment on 11/27? Here's an associated plot for site c for the last few days.
Carol Ruchti
All c02 and h20 sensors were not recording over night 12/6/22 - 12/7/22
Do we expect this to continue as it gets colder overnight with frost overnight and melting in the sun?
Also, Rfan.12m.c is alive again? RPM is almost 20,000?
You can see something happened with the fan for the Temperature data as well.
Otherwise, another calm and cloudy day at SOS!
Carol Ruchti
Rfan.12m.c is now reaching over 100K RPM as of the afternoon of 12/8/2022
Carol Ruchti
Continues with 400K+ RPM overnight and into 12/9/2022
Steve Oncley
Clearly, something is wrong with the reporting of fan RPM. However, the temperatures and RH now align with the neighbors, so I assume that the fan is operating sufficiently. This sensor is on the list for Chris and Will to replace next week.
Steve Oncley
Chris and Antonio replaced this on 21 Dec, so this is no longer a problem.
They didn't bring the sensor back with them (would have had to ski it out), so we don't yet know the nature of the problem or what caused it. They did identify the bad sensor as TRH057.
Steve Oncley
Something has happened to our calculation of Tsfc.ap in the last 2 days, which is now NA despite values of Vtherm and Vpile that are similar to previous days. I'll work on this tomorrow. However, I note that the PIs have questioned this calculation, so some work on this was needed anyway.
Steve Oncley
This problem seems to have fixed itself overnight. I still haven't revisited the code.
Steve Oncley
Reading the manual, I found some unit conversions in the calculation that weren't explained on the datasheet – Vpile is assumed in mV and TD is assumed in degC when used to compute m and b. I've made these changes, so we'll see what happens...
Jacquelyn Witte
Since the 2022 Dec 21-22 snow storm, uw Tsnow measurements at the lowest heights (0.4, 0.5, 0.6, 0.7 m) have altered. Measurements seem invariant probably in response to being buried by the blowing snow reported by Steve. Chris reported variable snow cover so from looking at NCharts I’m assuming site d was not particularly impacted by the snow storm, vis-a-vis Tsnow.
Jacquelyn Witte
Looks like more snow has accumulated at site uw up to 1m since ~ new years.
Steve Oncley
Having hopefully fixed the Tsfc.Ap calculation, I see differences with the Tsfc from the Kipp&Zonen and NR01 values. A tiny bit of this is that our code assumes an emissivity of 0.98, that hasn't (yet) been applied to the Apogee values. Nevertheless, the differences are still rather large. Thus, I've just inserted our Table Mountain-derived corrections for the K&Z Rsw and Rpile values into cal_files (and changed sensor_catalog to use them), and restarted the statsproc's. (We would have needed to do this anyway sometime.). We'll see if the comparison improves....
Matt Paulus
Since yesterday (12/27) around 10:42 am, there have been a number of data dropouts (I count 11) evident in both ncharts and qctables. They mostly seem to last for 15-20 minutes, although the longest one yesterday lasted 50 minutes. The most recent gap was today from 10:02-10:17 am.
Also, T.3m seems high today (higher than the rest, including 2m and 4m TRH's), but Rfan looks fine. Maybe drifting/accumulating snow is doing interesting things to the profile? It's very evident in the profile plots on qc plots.
Steve Oncley
I saw those gaps and don't understand them either. One was me restarting statsproc (to implement the Rsw/Rpile cal file), but I discovered that statsproc hadn't been running at the time anyway.
Steve Oncley
Rsw.in.uw has been quite a bit lower than Rsw.in.d during the last 2 days, presumably because of snow accumulation on the dome.
The corresponding wetness values are a bit hard to interpret. They were higher than the nominal 0.255 the night of 12/26-27, when Rsw looked okay. Wetness then dropped to 0.255 during the day, then increased with snowfall starting about 15:30 on 12/27. As I write this, Rsw.in.uw looks much better, though Wetness is still a bit high at 0.258.
My interpretation is that the wetness sensor gets frost and thus reads slightly wet even with the NR01 domes mostly clear. Thus, we can use a tight threshold of 0.256 for Wetness for the NR01 readings, but may throw out some good data.
Steve Oncley
Today, Rsw.in is lower than Rsw.out for both the K&Z and NR01. Usually, this means that the incoming dome is covered by snow, which is expected for the NR01. However, I also see this on the K&Z today, despite the supposedly heated airflow. Presumably, whatever snow accumulation happened last night has overcome the CVF4's ventillation capabilities. Interestingly, the K&Z and NR01 agreed very well in the first half of the morning, despite the significantly different ventillation (none for the NR01). They are now diverging as the day progresses.
Jacquelyn Witte
This should be noted in the data report that Rsw.in < Rsw.out during snow events (NR01 at uw). In particular, I see this pattern most apparent at night - concurrent with the camera documenting snowfall. This happens to a lesser degree with the K&Z (Rsw.9m at d) and in the early morning hours.
So far - a Wetness threshold of 0.265 or 0.2625 looks reasonable.
Jacquelyn Witte
IRGA’s at all sites are recording bad data (10m and 15m at site c coming on briefly) since 02 Jan. We’ve had more snow storms coming through since new years. We certainly had another snow event since yesterday into overnight. Irgadiag diagnostic gives a constant value of 847870 - if that means anything.
Jacquelyn Witte
Gas analyzers are all back online as of last night (huzzah!), except at 1m.uw which we already know is buried under snow. See webcam image.
Steve Oncley
Now that I've lowered the snow pillow data rate to only one sample every 17min, the points disappear from ncharts unless you really zoom in in time. If you hover over them, you can see that they are there. Sorry about that!
Steve Oncley
With some sunny weather, it appears that the K&Z Rsw.in finally cleared of snow about 11:20 yesterday (4 Jan). We'll still check it on our next visit.
Steve Oncley
...and, just about now (14:00, 5 Jan), the NR01 Rsw.in looks like it is clear (though there may be some surrounding snow that is reflecting more light into it)....
Steve Oncley
It was fun while it lasted... New snow overnight (Jan 5-6) covered both radiometers again. Sigh.
Carol Ruchti
Hey everyone! I'm at AMS like most of you as well this week. I'll try my best to do some brief QC of the data at the beginning and end of the day. I can take a more through look on Friday when I'll have more time. Please let me know if there's anything specific that you all are checking besides Rsw and snow pillows. It seems like those have been causing the most trouble recently. As Steve mentioned, DSM server was down Jan. 7th 17:42 MST to Jan. 8th 11 AM MST.
Over the weekend, looks like no new snow, cold, with low winds, and finally some clear skies with Rsw values increasing and RH values decreasing.
Carol Ruchti
I think the DSM Server went down again at about 5 pm MST yesterday Tues. 1/10/23. We're not getting any updated in QC Tables, NCharts, etc.
Gary Granger
Thanks, I restarted it.
Carol Ruchti
Great, thanks Gary!
Jacquelyn Witte
10 Jan - sonics/irga's at ue, uw, and d were moved from 1m to 2m See blog post. We will have to remember to change the variable naming convention from '_1m_' to '_2m_' at these sites in the netcdf files.
Carol Ruchti
Great point Jacquie, and thanks for adding this change to instrument heights here as well!
Carol Ruchti
Short data outages January 12th 14:57 MST to 15:47 MST, and 16:22 MST to 16:47 MST.
Jacquelyn Witte
Blowing snow, or snow, since 15 Jan - see latest webcam images. A snow storm is forecast for today into tomorrow. Looks like it's already in the high country.
Sonics at the lowest meters are not registering, or flakey as of this morning. IRGA's are bad values through 10m since yesterday.
You can observe and follow this intermittency in NCharts
Jacquelyn Witte
The crew disconnected the EC100 at 1m.c Jan 19th. No data has been flowing since then - refer to Steve's blog. What are the plans for this sensor?
Steve Oncley
This sensor is now placed (but not connected to anything) on 2m.uw, facing west. In the Spring, when the snowpack allows, it should be re-placed and re-connected at 1m.c. I could remove it from QCtables, but then would have to remember to add it back later.
Carol Ruchti
Whiteout conditions at the site this afternoon..... likely snowing with RH values in the 90's
Otherwise, everything is looking great in the data the last few days!
Carol Ruchti
Is anyone else having trouble loading ncharts? - http://datavis.eol.ucar.edu/ncharts/projects/SOS
That page works, but I can't load a dataset...
Carol Ruchti
I should mention I can see data fine on the QCdata page - https://archive.eol.ucar.edu/docs/isf/projects/SOS/isfs/qcdata/
Carol Ruchti
All fixed! Thanks to Erik for looking into it, and Ted did a reboot on datavis. This was likely caused due to turbulence the morning of 2/16 with networking maintenance.
Carol Ruchti
Alright, it's been a pretty quiet week of SOS data QC. Happy to hear Isabel and Tony are enjoying a very warm -10°C thanks to clear skies the last 2 days.
One thing I've noticed that's a bit odd is the T.3m.c values are something like 1-2°C warmer that all the other levels during the day while the sun is out and there's almost not wind.... like the last 2 days (2/16 and 2/17). Could there be an issue with the Rfan.3m.c, but I don't see anything weird with the RPMs on that fan. Something similar to below can be seen on other days with clear skies and low winds where T.3m.c is always the warmest. Maybe others already have an answers as to why this happens that I can't think of.....
Finally, here's a really cool feature I saw from yesterday! Look at the 5°C temperature increase at all levels right at sunset last night (2/16).
Steve Oncley
Pillow #4 started reading 0 about midnight last night (Mar 21-22). Power cycling just now seems to have brought it back. This is the first failure we've seen since implementing Gary's script to do this automatically.
Gary Granger
Hmm, the script only checks if a pillow is not responding, it does not check for zero values. I guess it needs to check for zero? Or should we go back to cycling power daily "just in case" there's some data problem that we can't detect automatically?
Steve Oncley
I wouldn't panic yet, with one failure in 2? months. It is an easy failure to spot by eye (I saw it in QCtables). We'll just have to keep an eye on it again.
Carol Ruchti
Alright, I think there's a few different things of note. Some of these happened last week during Chris's visit and after the March 11th snow storm. I tried to add the updates the Google Sheet, but please have a look since I'm not sure I have all of this right.
As Chris noted, the uw 2m sonic was moved to 2.5 meters on 3/14.
After the March 11th snow, the T.1m.c and RH.1m.c have been buried in the snow.
As Chris also mentioned, the Tsnow.0.4m.uw and Tsnow.0.5m.uw also haven't really been working since the March 11th storm reporting bad values. Tsnow.0.5m.uw completely stopped reporting as of today 3/20.
Finally, it's also looking like the 1m.d sensors (wind, co2, h20, etc) aren't reporting as of this morning at ~14 UTC 3/20. I'm have to check this again tomorrow.
Steve Oncley
Chris mentioned that 1m.d was very close to being buried, and there has been even more snow since he left. Our plan is just to let it get buried and then hopefully come back soon as the snowpack decreases.
Carol Ruchti
I'd say 1m.d instruments are officially buried in the snow as of yesterday (3/20).
Carol Ruchti
Due to more snow over the last 24 hours, now the 1m.ue sensors also appear to be buried in the snow and the various (wind, c02, h20, etc) aren't reporting as of ~18 UTC today (3/22).
Carol Ruchti
I'm not sure if this has already been discussed, but I did notice this morning that the h2o.10m.c values all seem to be +2 higher than the h2o.10m values on the other towers. This seems to be true for basically the whole of the campaign, so it's nothing new. I didn't put it in the Google Sheet, but feel free to add it if needed.
Currently, all of the *.1m.c, *.1m.d, *.1m.ue instruments are still buried in the snow, and it looks like there is more snow forecasted for tonight. I won't be surprised if the *.1m.uw get buried in the snow over the weekend.
Finally SWE.p3.c is not reporting as of this morning.
Steve Oncley
Thanks for catching the snow pillow outage, Carol! This was pillow #3, the previous outage was pillow #4. I've just cycled power to the pillows and they all are back.
Incidentally, it is an outage (I looked at the data – no reply from pillow #3 after 12Z), rather than a report of 0 values, so Gary's script should have detected it.
Jacquelyn Witte
In conversation Steve mentioned the inversion in the Tsoil - specifically at 18.1, 9.4, 20 cm.
I think the expectation is that this should be a linear decline, like this
If this is the case, by my estimate, the inflection started around 5-6 Dec 2022 and doesn’t relate to anything noted in the blog. If this is not real, how do we diagnose this? Post-calibrations? Visual inspection during teardown?
Steve Oncley
Seeing this, I had intended to be sure to get a good post-calibration for each of these Tsoil probes. However, another approach is to look at the time series data during freeze/thaw events. We would expect the soil temperature to remain approximately constant for, say, several hours at least while extra heat is absorbed/released due to the latent heat of fusion during such events. From these data, we should be able to tell what the Tsoil probe was reading when the temperature really was zero.
Jacquelyn Witte
Looking at the time series of the ‘Load’ variables, I notice Load.3.p4.c and Load.4.p1.c have been slowly creeping up and are separating from the pack. I don’t have a feel for these measurements or location so I thought I’d mention this for someone else to chime in. By my estimates, this slow increase becomes apparent around 27 March.
Steve Oncley
I'm not an expert either, but it is possible that this is real, with snow becoming more compact and filling in with more snow above at one corner of the pillow. Definitely worth watching, but not much we can do if it is an instrument issue.
Steve Oncley
At about 10AM today (3/31), we lost Tsnow.1m.uw. Tsnow.0.4 and 0.5.d also are now reading positive.
Jacquelyn Witte
Yup!
Matt Paulus
A few issues I've noticed this week, none of which require intervention...
Tsnow.1.2.d started reporting bad values on 10 Apr around 1947. About 12 hours prior it started creeping towards positive values.
Tsnow0.4.uw started reporting bad values as of 0505 on 7 Apr.
Gsoil.d started reading exactly 0 on 12 Apr. This came after a positive spike on 9 Apr.
There were 2 short data outages between 1000-1200 on 12 Apr. Not sure what happened, but it affected the whole network.
Jacquelyn Witte
Tsnow.0.7m.uw has been bad since 15 April 03:17:30 MT
Jacquelyn Witte
The evolution of bad Tsnow's, so far:
Tsnow.0.4m.d - 2023-04-07 04:32:30
Tsnow.0.4m.uw - 2023-03-11 11:07:30
Tsnow.0.5m.uw - 2023-03-11 11:07:30
Tsnow.0.7m.uw - 2023-04-15 03:17:30
Tsnow.1.0m.uw - 2023-03-31 09:47:30
Tsnow.1.2m.uw - 2023-04-10 19:47:30
Tsnow.1.4m.uw - 2023-04-18 14:52:30 (new)
I don’t know enough to interpret these differences (exposure vs non-exposure I'd guess) so I’ll just note this observation. There are periods where Tsnow.d measures much larger amplitudes than uw at heights 1.1m, 1.2m, 1.3m. Tsnow.d time series tend to be self-consistent and that is not so true for Tsnow.uw.
Otherwise, I'm not seeing anything unusual to comment on that hasn't been mentioned.
Jacquelyn Witte
Bad Tsnow.0.5m.d since 2023-04-25 23:36:40
Carol Ruchti
There was a minor outage with the wind 1m.c variables overnight between 11 pm - 10 am MST 4/26. Looks like it's working fine this afternoon, but the Sonic 1m.c does look a little buried in the snow.
Carol Ruchti
All looks good with the SOS data the last few days! Good luck to everyone with the last month of operations
Jacquelyn Witte
Bad Tsoil.30.6cm.d since 3 May.
Jacquelyn Witte
Bad Tsoil.28.1cm.d since 2023-05-03 21:47:30
Jacquelyn Witte
What's happening with the sonic/irga's today?
Jacquelyn Witte
All near surface Tsoils (0.6, 1.9, 3.1, 4.4 cm) have been above 0.0degC since ~5 May. Presumably, we can bring back Qsoil if this trend continues.
Jacquelyn Witte
Radiometer measurements at site d are not being recorded on NCharts or QC tables since yesterday ~ 11 May 16:47:30
Steve Oncley
Tsnow.0.4m.d appears to have revived itself on 14 May. The ones that appear bad at the moment are all on uw at:
0.4, 0.5, 0.7, 1.0, 1.2, 1.3.
Tsnow.1.4m.uw is intermittently good.
Carol Ruchti
I would say Tsnow.1.4m.uw is good for the last few days.
Carol Ruchti
Radiometer.9m.d is not reporting as of ~21:00 MDT on Wed. 5/31. Rsw, Rlw, Rpile, and Tcase at 9m.d are missing values all day for 6/1.
Steve Oncley
cycling power to the radiometer mote via "pio" seems to have restarted everything. Odd. We'll have to keep watching it. Thanks, Carol!
Carol Ruchti
Steve Oncley Similar outages starting yesterday 6/4 for Rsw, Rlw, Rpile, and Tcase at 9m.d. As of today 6/5, all those variables are still not reporting.
Carol Ruchti
To add, this plot is for the last 8 days showing when the 9m.d radiometer has not been working. Also, I did add this to the Data Issue Tracker.
Carol Ruchti
Many different instruments on all towers are not reporting as of 9:30 am MDT on 6/2. See QC Table
P.10m.d, P.10m.ue, P.10m.uw, P.20m.c, spd.*, w.*, tc.*, h2o.*, co2.*, Load.*, SWE.*
Carol Ruchti
It appears ~half of those instruments are reporting again this afternoon and more are coming online.
Carol Ruchti
Since ~18:00 MDT on Saturday 6/3, the SF_avg.2m.ue and Spd_avg.2m.ue have not been reporting.
Carol Ruchti
P.0.8m.c, P.0.9m.c, P.1.0m.c, P.1.1m.c
spd.1m.d, spd.1m.ue, spd.1m.uw
w.1m.d, w.1m.ue, w.1m.uw
tc.1m.d, tc.1m.ue, tc.1m.uw
h2o.1m.d, h2o.1m.ue, h2o.1m.uw
co2.1m.d, co2.1m.ue, co2.1m.uw
All of the instruments are missing values since ~8:00 am MDT on Wed. 6/7
Carol Ruchti
spd.1m.*, w.1m.*, tc.1m.*, h2o.1m.*, co2.1m.* instrument are now reporting again as of ~11 am MDT on Fri. 6/9
P.0.8m.c, P.0.9m.c, P.1.0m.c, P.1.1m.c continue to have missing values.
Steve Oncley
I had made changes to the configuration back in Boulder in March to rename the 1m to 2m (or 2.5m) when we raised them. Apparently, we never restarted various processes to have these changes take effect with the real-time data until earlier this week (by which time, we had lowered the sonics back). I made the config changes setting the sonics back to 1m and restarted these processes this morning, which brought them "back".
At the same time in March, I also changed the P. names, since they clearly weren't being used at 0.8-1.1m. In this case, the best fix is to change the qctables R code to keep up with the name change. I'll do that soon.
Sorry for the confusion.
Carol Ruchti
Ah ok, thanks for explaining Steve!
Carol Ruchti
As of Monday 6/19 at ~18:00 MDT, the Load variable and corresponding SWE and SWE_calc variables are missing. It looks all of the snow has been melted for something like 3 weeks, but observations were still being recorded this whole time.
Carol Ruchti
I think the DSM Server went down again at about 10 am MT yesterday Tues. 6/20/23. We're not getting any updated in QC Tables, NCharts, etc.
Gary Granger
Teardown has begun. The site was shut down yesterday, so I turned off dsm_server and all related SOS realtime processing.