A couple of times on this project the desktop on the data manager has gone black. It looks very disturbing if no other windows were open, because except for the mouse pointer on the screen, it looks like the desktop or the monitor crashed.
The workaround is to restart the plasma-desktop program like so:
Alt-F2 to open the run dialog, or use a terminal window. Run 'plasma-desktop'.
The plasma desktop should restart, and all the desktop background, icons, and toolbars should return.
There is a JIRA issue for this: https://jira.eol.ucar.edu/browse/ISS-555.
Profiler serial moments stopped on Friday, at 2017-01-20,09:58. I checked on the profiler PC with VNC, but there was nothing in the event logs to specifically indicate a reboot or a pop error. In the application event log, rsyncserver and norton ghost started up around 10:00:06. In the system event log, there appears to be a windows startup message at 09:59:53. So I think the PC rebooted, but I cannot find out why.
After investigating the logs, I tried restarting POP with the desktop icon. It counted down, and then nothing happened, the VNC screen did not change, and no moments data were recorded by the data manager. Also, I could no longer connect with VNC.
While connected over VNC, I added the data manager (192.168.0.56) as the primary time server to symmtime.
So yesterday, 2017-01-21 around noon, I visited the profiler site. The profiler PC was not hung, but POP was not running, and the VNC server had stopped with an error dialog. So I restarted POP with the desktop icon. Later I realized I had not restarted VNC, so I quit POP, restarted VNC from the Control Panel->Admin Tools->Services dialog, then restarted POP. When I left, everything was working.
I've created a JIRA issue for the intermittent reboots: https://jira.eol.ucar.edu/browse/ISS-554.
Around 01-18-2017 09:25:59 the profiler PC stopped responding to pings from the data manager, then the pings recovered a minute later. Soon after there were nagios alerts that the profiler data had stopped. I suspect the profiler PC just rebooted for some unknown reason, and after that POP did not restart. (I assume POP is not setup to restart automatically.) Around 19:30 I connected to the profiler PC with VNC and was able to start POP manually, and now all the nagios checks are green again.