2023-11-19 09:51:34,728 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.7 seconds late 2023-11-19 09:51:35,343 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 09:51:35,344 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 09:51:18 2023-11-19 09:51:35,344 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 09:56:23 2023-11-19 09:51:35,344 INFO 1947952:repeater task 3dayplots_os150nb starting 11.3 seconds late 2023-11-19 09:51:38,687 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 09:51:38,687 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 09:56:24 2023-11-19 09:51:38,688 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.6 seconds late 2023-11-19 09:51:39,687 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 09:51:39,688 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:39,688 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.6 seconds late 2023-11-19 09:51:39,854 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 09:51:39,854 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:39,855 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.8 seconds late 2023-11-19 09:51:41,113 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 09:51:41,114 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:41,114 INFO 1947952:repeater task beamdiagnostics_os150nb starting 22.1 seconds late 2023-11-19 09:51:42,380 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 09:51:42,380 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:42,380 INFO 1947952:repeater task lastensq_os38nb starting 17.4 seconds late 2023-11-19 09:51:44,591 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 09:51:44,591 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 09:51:20 2023-11-19 09:51:44,591 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 09:56:25 2023-11-19 09:51:44,592 INFO 1947952:repeater task quick_os38nb starting 19.6 seconds late 2023-11-19 09:51:49,754 INFO 1947952:repeater task quick_os38nb ended after 5.2 seconds 2023-11-19 09:51:49,754 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 09:51:20 2023-11-19 09:51:49,754 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 09:56:25 2023-11-19 09:51:49,755 INFO 1947952:repeater task lastfew_vec_os38nb starting 24.8 seconds late 2023-11-19 09:51:50,388 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 09:51:50,388 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 09:51:20 2023-11-19 09:51:50,389 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 09:56:25 2023-11-19 09:51:50,389 INFO 1947952:repeater task 3dayplots_os38nb starting 26.3 seconds late 2023-11-19 09:51:53,797 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 09:51:53,797 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 09:56:24 2023-11-19 09:51:53,797 INFO 1947952:repeater task plot_tsstats_os38nb starting 34.7 seconds late 2023-11-19 09:51:54,821 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 09:51:54,821 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:54,822 INFO 1947952:repeater task grab_lastpings_os38nb starting 35.8 seconds late 2023-11-19 09:51:54,990 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 09:51:54,990 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:54,991 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.9 seconds late 2023-11-19 09:51:56,275 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 09:51:56,275 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:56,275 INFO 1947952:repeater task beamdiagnostics_os38nb starting 37.2 seconds late 2023-11-19 09:51:57,476 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 09:51:57,476 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:57,477 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 38.2 seconds late 2023-11-19 09:51:58,842 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-19 09:51:58,842 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:51:58,843 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 39.6 seconds late 2023-11-19 09:52:00,128 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 09:52:00,128 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:52:00,129 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 40.9 seconds late 2023-11-19 09:52:01,428 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 09:52:01,429 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:52:01,429 INFO 1947952:repeater task plot_hbins starting 42.2 seconds late 2023-11-19 09:52:05,789 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 09:52:05,789 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-19 09:52:05,790 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 10:06:19 2023-11-19 09:52:10,796 INFO 1947952:repeater task 3dayplots_wh300 starting 46.7 seconds late 2023-11-19 09:52:14,650 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 09:52:14,650 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 09:56:24 2023-11-19 09:52:24,670 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-19 09:52:27,731 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 09:52:27,732 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:52:19 2023-11-19 09:52:27,732 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 09:54:24 2023-11-19 09:52:27,732 INFO 1947952:repeater task quick_wh300 starting 3.7 seconds late 2023-11-19 09:52:35,004 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 09:52:35,004 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:52:19 2023-11-19 09:52:35,005 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 09:54:24 2023-11-19 09:52:35,005 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-19 09:52:35,641 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 09:52:35,641 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:52:19 2023-11-19 09:52:35,641 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 09:54:24 2023-11-19 09:54:25,949 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-19 09:54:28,750 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 09:54:28,750 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:54:19 2023-11-19 09:54:28,750 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 09:56:24 2023-11-19 09:54:28,751 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-19 09:54:36,470 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 09:54:36,471 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:54:19 2023-11-19 09:54:36,471 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 09:56:24 2023-11-19 09:54:36,471 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2023-11-19 09:54:37,107 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 09:54:37,108 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:54:19 2023-11-19 09:54:37,108 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 09:56:24 2023-11-19 09:56:27,413 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.4 seconds late 2023-11-19 09:56:30,196 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-19 09:56:30,196 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 10:06:24 2023-11-19 09:56:30,196 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.1 seconds late 2023-11-19 09:56:32,909 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 09:56:32,909 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 10:06:24 2023-11-19 09:56:32,909 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.9 seconds late 2023-11-19 09:56:35,924 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 09:56:35,924 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 10:06:24 2023-11-19 09:56:35,925 INFO 1947952:repeater task lastensq_wh300 starting 11.9 seconds late 2023-11-19 09:56:38,877 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 09:56:38,877 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:56:19 2023-11-19 09:56:38,878 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 09:58:24 2023-11-19 09:56:38,878 INFO 1947952:repeater task quick_wh300 starting 14.9 seconds late 2023-11-19 09:56:46,173 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 09:56:46,173 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:56:19 2023-11-19 09:56:46,174 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 09:58:24 2023-11-19 09:56:46,174 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.2 seconds late 2023-11-19 09:56:46,820 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 09:56:46,821 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:56:19 2023-11-19 09:56:46,821 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 09:58:24 2023-11-19 09:56:46,821 INFO 1947952:repeater task 3dayplots_wh300 starting 22.8 seconds late 2023-11-19 09:56:50,715 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 09:56:50,715 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:01:24 2023-11-19 09:56:50,717 INFO 1947952:repeater task lastensq_os150nb starting 27.7 seconds late 2023-11-19 09:56:52,913 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 09:56:52,914 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 09:56:18 2023-11-19 09:56:52,914 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:01:23 2023-11-19 09:56:52,914 INFO 1947952:repeater task quick_os150nb starting 29.9 seconds late 2023-11-19 09:56:57,434 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 09:56:57,434 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 09:56:18 2023-11-19 09:56:57,434 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:01:23 2023-11-19 09:56:57,435 INFO 1947952:repeater task lastfew_vec_os150nb starting 34.4 seconds late 2023-11-19 09:56:58,060 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 09:56:58,060 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 09:56:18 2023-11-19 09:56:58,060 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:01:23 2023-11-19 09:56:58,061 INFO 1947952:repeater task 3dayplots_os150nb starting 34.0 seconds late 2023-11-19 09:57:01,457 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 09:57:01,457 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:01:24 2023-11-19 09:57:01,458 INFO 1947952:repeater task lastensq_os38nb starting 36.5 seconds late 2023-11-19 09:57:03,662 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 09:57:03,663 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 09:56:20 2023-11-19 09:57:03,663 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:01:25 2023-11-19 09:57:03,663 INFO 1947952:repeater task quick_os38nb starting 38.7 seconds late 2023-11-19 09:57:08,729 INFO 1947952:repeater task quick_os38nb ended after 5.1 seconds 2023-11-19 09:57:08,730 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 09:56:20 2023-11-19 09:57:08,730 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:01:25 2023-11-19 09:57:08,730 INFO 1947952:repeater task lastfew_vec_os38nb starting 43.7 seconds late 2023-11-19 09:57:09,350 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 09:57:09,351 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 09:56:20 2023-11-19 09:57:09,351 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:01:25 2023-11-19 09:57:09,351 INFO 1947952:repeater task 3dayplots_os38nb starting 45.3 seconds late 2023-11-19 09:57:12,777 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 09:57:12,777 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:01:24 2023-11-19 09:57:12,778 INFO 1947952:repeater task daily_netcdf starting 48.7 seconds late 2023-11-19 09:57:13,684 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 09:57:13,685 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 10:06:24 2023-11-19 09:58:28,872 INFO 1947952:repeater task lastensq_wh300 starting 4.9 seconds late 2023-11-19 09:58:31,871 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 09:58:31,871 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:58:19 2023-11-19 09:58:31,871 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:00:24 2023-11-19 09:58:31,872 INFO 1947952:repeater task quick_wh300 starting 7.9 seconds late 2023-11-19 09:58:39,274 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 09:58:39,274 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:58:19 2023-11-19 09:58:39,274 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:00:24 2023-11-19 09:58:39,275 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.3 seconds late 2023-11-19 09:58:39,927 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 09:58:39,927 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 09:58:19 2023-11-19 09:58:39,927 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:00:24 2023-11-19 10:00:25,212 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-19 10:00:28,560 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-19 10:00:28,560 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:00:19 2023-11-19 10:00:28,560 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:02:24 2023-11-19 10:00:28,560 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-19 10:00:36,215 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 10:00:36,216 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:00:19 2023-11-19 10:00:36,216 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:02:24 2023-11-19 10:00:36,216 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2023-11-19 10:00:36,865 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:00:36,866 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:00:19 2023-11-19 10:00:36,866 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:02:24 2023-11-19 10:01:27,001 INFO 1947952:repeater task 3dayplots_wh300 starting 3.0 seconds late 2023-11-19 10:01:30,970 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 10:01:30,971 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:06:24 2023-11-19 10:01:30,972 INFO 1947952:repeater task lastensq_os150nb starting 8.0 seconds late 2023-11-19 10:01:33,457 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-19 10:01:33,458 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:01:18 2023-11-19 10:01:33,458 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:06:23 2023-11-19 10:01:33,458 INFO 1947952:repeater task quick_os150nb starting 10.5 seconds late 2023-11-19 10:01:38,387 INFO 1947952:repeater task quick_os150nb ended after 4.9 seconds 2023-11-19 10:01:38,387 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:01:18 2023-11-19 10:01:38,387 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:06:23 2023-11-19 10:01:38,388 INFO 1947952:repeater task lastfew_vec_os150nb starting 15.4 seconds late 2023-11-19 10:01:39,003 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:01:39,003 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:01:18 2023-11-19 10:01:39,003 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:06:23 2023-11-19 10:01:39,004 INFO 1947952:repeater task 3dayplots_os150nb starting 15.0 seconds late 2023-11-19 10:01:42,391 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:01:42,391 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:06:24 2023-11-19 10:01:42,392 INFO 1947952:repeater task lastensq_os38nb starting 17.4 seconds late 2023-11-19 10:01:44,511 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 10:01:44,511 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:01:20 2023-11-19 10:01:44,511 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:06:25 2023-11-19 10:01:44,512 INFO 1947952:repeater task quick_os38nb starting 19.5 seconds late 2023-11-19 10:01:49,722 INFO 1947952:repeater task quick_os38nb ended after 5.2 seconds 2023-11-19 10:01:49,722 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:01:20 2023-11-19 10:01:49,723 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:06:25 2023-11-19 10:01:49,723 INFO 1947952:repeater task lastfew_vec_os38nb starting 24.7 seconds late 2023-11-19 10:01:50,352 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:01:50,353 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:01:20 2023-11-19 10:01:50,353 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:06:25 2023-11-19 10:01:50,353 INFO 1947952:repeater task 3dayplots_os38nb starting 26.3 seconds late 2023-11-19 10:01:53,860 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 10:01:53,860 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:06:24 2023-11-19 10:02:28,937 INFO 1947952:repeater task lastensq_wh300 starting 4.9 seconds late 2023-11-19 10:02:32,069 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 10:02:32,070 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:02:19 2023-11-19 10:02:32,070 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:04:24 2023-11-19 10:02:32,070 INFO 1947952:repeater task quick_wh300 starting 8.1 seconds late 2023-11-19 10:02:39,345 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 10:02:39,345 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:02:19 2023-11-19 10:02:39,345 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:04:24 2023-11-19 10:02:39,346 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.3 seconds late 2023-11-19 10:02:39,979 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:02:39,979 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:02:19 2023-11-19 10:02:39,979 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:04:24 2023-11-19 10:04:25,236 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-19 10:04:28,352 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 10:04:28,352 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:04:19 2023-11-19 10:04:28,352 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:06:24 2023-11-19 10:04:28,353 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-19 10:04:35,610 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 10:04:35,611 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:04:19 2023-11-19 10:04:35,611 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:06:24 2023-11-19 10:04:35,611 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2023-11-19 10:04:36,239 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:04:36,240 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:04:19 2023-11-19 10:04:36,240 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:06:24 2023-11-19 10:06:21,522 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.5 seconds late 2023-11-19 10:06:22,759 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 10:06:22,759 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:22,759 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.7 seconds late 2023-11-19 10:06:22,928 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 10:06:22,928 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:22,929 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.9 seconds late 2023-11-19 10:06:24,163 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-19 10:06:24,163 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:24,164 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.1 seconds late 2023-11-19 10:06:25,477 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 10:06:25,477 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:25,477 INFO 1947952:repeater task lastensq_os150nb starting 2.5 seconds late 2023-11-19 10:06:28,178 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-19 10:06:28,180 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:06:18 2023-11-19 10:06:28,180 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:11:23 2023-11-19 10:06:28,180 INFO 1947952:repeater task quick_os150nb starting 5.2 seconds late 2023-11-19 10:06:32,782 INFO 1947952:repeater task quick_os150nb ended after 4.6 seconds 2023-11-19 10:06:32,782 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:06:18 2023-11-19 10:06:32,782 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:11:23 2023-11-19 10:06:32,783 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.8 seconds late 2023-11-19 10:06:33,405 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:06:33,405 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:06:18 2023-11-19 10:06:33,405 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:11:23 2023-11-19 10:06:33,406 INFO 1947952:repeater task 3dayplots_os150nb starting 9.4 seconds late 2023-11-19 10:06:36,806 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:06:36,806 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:11:24 2023-11-19 10:06:36,807 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.8 seconds late 2023-11-19 10:06:37,800 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 10:06:37,800 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:37,801 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.7 seconds late 2023-11-19 10:06:37,967 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 10:06:37,968 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:37,968 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.9 seconds late 2023-11-19 10:06:39,192 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 10:06:39,192 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:39,193 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.1 seconds late 2023-11-19 10:06:40,466 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 10:06:40,466 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:40,466 INFO 1947952:repeater task lastensq_os38nb starting 15.5 seconds late 2023-11-19 10:06:42,813 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-19 10:06:42,813 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:06:20 2023-11-19 10:06:42,813 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:11:25 2023-11-19 10:06:42,814 INFO 1947952:repeater task quick_os38nb starting 17.8 seconds late 2023-11-19 10:06:47,560 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 10:06:47,560 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:06:20 2023-11-19 10:06:47,560 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:11:25 2023-11-19 10:06:47,561 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.6 seconds late 2023-11-19 10:06:48,181 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:06:48,181 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:06:20 2023-11-19 10:06:48,182 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:11:25 2023-11-19 10:06:48,182 INFO 1947952:repeater task 3dayplots_os38nb starting 24.1 seconds late 2023-11-19 10:06:51,565 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 10:06:51,566 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:11:24 2023-11-19 10:06:51,566 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.5 seconds late 2023-11-19 10:06:52,594 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 10:06:52,594 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:52,595 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.5 seconds late 2023-11-19 10:06:52,766 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 10:06:52,766 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:52,766 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.7 seconds late 2023-11-19 10:06:53,986 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 10:06:53,987 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:53,987 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.9 seconds late 2023-11-19 10:06:55,229 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 10:06:55,229 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:06:55,229 INFO 1947952:repeater task daily_netcdf starting 31.2 seconds late 2023-11-19 10:06:56,081 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 10:06:56,081 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:06:56,081 INFO 1947952:repeater task ggatime_diagnostics starting 37.0 seconds late 2023-11-19 10:07:12,464 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-19 10:07:12,465 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:07:12,465 INFO 1947952:repeater task ggatime_diagnostics starting 53.4 seconds late 2023-11-19 10:07:28,186 INFO 1947952:repeater task ggatime_diagnostics ended after 15.7 seconds 2023-11-19 10:07:28,187 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:07:28,187 INFO 1947952:repeater task ggatime_diagnostics starting 69.1 seconds late 2023-11-19 10:07:43,825 INFO 1947952:repeater task ggatime_diagnostics ended after 15.6 seconds 2023-11-19 10:07:43,825 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:07:43,825 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 84.6 seconds late 2023-11-19 10:07:45,178 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-19 10:07:45,178 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:07:45,178 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 85.9 seconds late 2023-11-19 10:07:46,493 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 10:07:46,493 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:07:46,493 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 87.2 seconds late 2023-11-19 10:07:47,768 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 10:07:47,768 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:07:47,769 INFO 1947952:repeater task plot_hbins starting 88.5 seconds late 2023-11-19 10:07:52,028 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 10:07:52,029 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 10:07:52,029 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 10:21:19 2023-11-19 10:07:57,033 INFO 1947952:repeater task noisy_rawtemps_os150 starting 93.0 seconds late 2023-11-19 10:07:59,990 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-19 10:07:59,991 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:07:59,991 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 95.9 seconds late 2023-11-19 10:08:02,747 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 10:08:02,747 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:08:02,748 INFO 1947952:repeater task noisy_rawtemps_os38 starting 98.7 seconds late 2023-11-19 10:08:05,805 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 10:08:05,805 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:08:05,805 INFO 1947952:repeater task lastensq_wh300 starting 101.8 seconds late 2023-11-19 10:08:08,850 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 10:08:08,851 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:06:19 2023-11-19 10:08:08,851 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:08:24 2023-11-19 10:08:08,851 INFO 1947952:repeater task quick_wh300 starting 104.9 seconds late 2023-11-19 10:08:15,707 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-19 10:08:15,708 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:06:19 2023-11-19 10:08:15,708 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:08:24 2023-11-19 10:08:15,708 INFO 1947952:repeater task lastfew_vec_wh300 starting 111.7 seconds late 2023-11-19 10:08:16,415 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 10:08:16,415 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:06:19 2023-11-19 10:08:16,415 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:08:24 2023-11-19 10:08:16,416 INFO 1947952:repeater task 3dayplots_wh300 starting 112.4 seconds late 2023-11-19 10:08:20,512 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.1 seconds 2023-11-19 10:08:20,513 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:11:24 2023-11-19 10:08:25,522 INFO 1947952:repeater task lastensq_wh300 starting 1.5 seconds late 2023-11-19 10:08:28,652 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 10:08:28,653 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:08:19 2023-11-19 10:08:28,653 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:10:24 2023-11-19 10:08:28,653 INFO 1947952:repeater task quick_wh300 starting 4.7 seconds late 2023-11-19 10:08:35,790 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-19 10:08:35,791 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:08:19 2023-11-19 10:08:35,791 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:10:24 2023-11-19 10:08:35,791 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-19 10:08:36,423 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:08:36,423 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:08:19 2023-11-19 10:08:36,424 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:10:24 2023-11-19 10:10:26,708 INFO 1947952:repeater task lastensq_wh300 starting 2.7 seconds late 2023-11-19 10:10:29,543 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:10:29,544 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:10:19 2023-11-19 10:10:29,544 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:12:24 2023-11-19 10:10:29,544 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-19 10:10:36,650 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-19 10:10:36,651 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:10:19 2023-11-19 10:10:36,651 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:12:24 2023-11-19 10:10:36,651 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2023-11-19 10:10:37,277 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:10:37,277 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:10:19 2023-11-19 10:10:37,277 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:12:24 2023-11-19 10:11:27,404 INFO 1947952:repeater task 3dayplots_wh300 starting 3.4 seconds late 2023-11-19 10:11:31,285 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 10:11:31,285 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:11:31,287 INFO 1947952:repeater task lastensq_os150nb starting 8.3 seconds late 2023-11-19 10:11:33,779 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-19 10:11:33,779 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:11:18 2023-11-19 10:11:33,779 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:16:23 2023-11-19 10:11:33,780 INFO 1947952:repeater task quick_os150nb starting 10.8 seconds late 2023-11-19 10:11:38,198 INFO 1947952:repeater task quick_os150nb ended after 4.4 seconds 2023-11-19 10:11:38,199 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:11:18 2023-11-19 10:11:38,199 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:16:23 2023-11-19 10:11:38,199 INFO 1947952:repeater task lastfew_vec_os150nb starting 15.2 seconds late 2023-11-19 10:11:38,818 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:11:38,818 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:11:18 2023-11-19 10:11:38,818 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:16:23 2023-11-19 10:11:38,819 INFO 1947952:repeater task 3dayplots_os150nb starting 14.8 seconds late 2023-11-19 10:11:42,160 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 10:11:42,160 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:11:42,161 INFO 1947952:repeater task lastensq_os38nb starting 17.2 seconds late 2023-11-19 10:11:44,513 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-19 10:11:44,514 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:11:20 2023-11-19 10:11:44,514 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:16:25 2023-11-19 10:11:44,515 INFO 1947952:repeater task quick_os38nb starting 19.5 seconds late 2023-11-19 10:11:49,472 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 10:11:49,473 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:11:20 2023-11-19 10:11:49,473 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:16:25 2023-11-19 10:11:49,473 INFO 1947952:repeater task lastfew_vec_os38nb starting 24.5 seconds late 2023-11-19 10:11:50,092 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:11:50,093 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:11:20 2023-11-19 10:11:50,093 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:16:25 2023-11-19 10:11:50,093 INFO 1947952:repeater task 3dayplots_os38nb starting 26.0 seconds late 2023-11-19 10:11:53,534 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 10:11:53,534 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:12:28,619 INFO 1947952:repeater task lastensq_wh300 starting 4.6 seconds late 2023-11-19 10:12:31,658 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 10:12:31,658 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:12:19 2023-11-19 10:12:31,658 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:14:24 2023-11-19 10:12:31,659 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-19 10:12:39,159 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 10:12:39,159 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:12:19 2023-11-19 10:12:39,159 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:14:24 2023-11-19 10:12:39,160 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.2 seconds late 2023-11-19 10:12:39,791 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:12:39,791 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:12:19 2023-11-19 10:12:39,791 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:14:24 2023-11-19 10:14:25,092 INFO 1947952:repeater task lastensq_wh300 starting 1.1 seconds late 2023-11-19 10:14:28,027 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 10:14:28,028 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:14:19 2023-11-19 10:14:28,028 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:14:28,028 INFO 1947952:repeater task quick_wh300 starting 4.0 seconds late 2023-11-19 10:14:34,846 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-19 10:14:34,846 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:14:19 2023-11-19 10:14:34,846 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:14:34,847 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2023-11-19 10:14:35,488 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:14:35,489 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:14:19 2023-11-19 10:14:35,489 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:16:24 2023-11-19 10:16:25,771 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.7 seconds late 2023-11-19 10:16:28,648 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 10:16:28,648 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:16:28,649 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 4.6 seconds late 2023-11-19 10:16:31,438 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 10:16:31,438 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:16:31,439 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.4 seconds late 2023-11-19 10:16:34,477 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 10:16:34,477 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:16:34,477 INFO 1947952:repeater task lastensq_wh300 starting 10.5 seconds late 2023-11-19 10:16:37,524 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 10:16:37,525 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:16:19 2023-11-19 10:16:37,525 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:18:24 2023-11-19 10:16:37,525 INFO 1947952:repeater task quick_wh300 starting 13.5 seconds late 2023-11-19 10:16:44,892 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 10:16:44,892 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:16:19 2023-11-19 10:16:44,893 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:18:24 2023-11-19 10:16:44,893 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.9 seconds late 2023-11-19 10:16:45,542 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:16:45,542 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:16:19 2023-11-19 10:16:45,542 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:18:24 2023-11-19 10:16:45,542 INFO 1947952:repeater task 3dayplots_wh300 starting 21.5 seconds late 2023-11-19 10:16:49,406 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 10:16:49,407 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:21:24 2023-11-19 10:16:49,408 INFO 1947952:repeater task lastensq_os150nb starting 26.4 seconds late 2023-11-19 10:16:51,966 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-19 10:16:51,966 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:16:18 2023-11-19 10:16:51,967 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:21:23 2023-11-19 10:16:51,967 INFO 1947952:repeater task quick_os150nb starting 29.0 seconds late 2023-11-19 10:16:56,441 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 10:16:56,442 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:16:18 2023-11-19 10:16:56,442 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:21:23 2023-11-19 10:16:56,442 INFO 1947952:repeater task lastfew_vec_os150nb starting 33.4 seconds late 2023-11-19 10:16:57,067 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:16:57,067 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:16:18 2023-11-19 10:16:57,067 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:21:23 2023-11-19 10:16:57,068 INFO 1947952:repeater task 3dayplots_os150nb starting 33.0 seconds late 2023-11-19 10:17:00,461 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:17:00,461 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:21:24 2023-11-19 10:17:00,463 INFO 1947952:repeater task lastensq_os38nb starting 35.5 seconds late 2023-11-19 10:17:02,882 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-19 10:17:02,883 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:16:20 2023-11-19 10:17:02,883 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:21:25 2023-11-19 10:17:02,883 INFO 1947952:repeater task quick_os38nb starting 37.9 seconds late 2023-11-19 10:17:07,612 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 10:17:07,613 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:16:20 2023-11-19 10:17:07,613 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:21:25 2023-11-19 10:17:07,613 INFO 1947952:repeater task lastfew_vec_os38nb starting 42.6 seconds late 2023-11-19 10:17:08,237 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:17:08,237 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:16:20 2023-11-19 10:17:08,237 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:21:25 2023-11-19 10:17:08,237 INFO 1947952:repeater task 3dayplots_os38nb starting 44.2 seconds late 2023-11-19 10:17:11,651 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 10:17:11,652 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:21:24 2023-11-19 10:17:11,653 INFO 1947952:repeater task daily_netcdf starting 47.6 seconds late 2023-11-19 10:17:12,441 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 10:17:12,441 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:18:27,631 INFO 1947952:repeater task lastensq_wh300 starting 3.6 seconds late 2023-11-19 10:18:30,754 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 10:18:30,755 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:18:19 2023-11-19 10:18:30,755 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:20:24 2023-11-19 10:18:30,755 INFO 1947952:repeater task quick_wh300 starting 6.8 seconds late 2023-11-19 10:18:37,898 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-19 10:18:37,898 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:18:19 2023-11-19 10:18:37,898 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:20:24 2023-11-19 10:18:37,899 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2023-11-19 10:18:38,553 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 10:18:38,553 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:18:19 2023-11-19 10:18:38,553 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:20:24 2023-11-19 10:20:28,834 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-19 10:20:31,632 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:20:31,633 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:20:19 2023-11-19 10:20:31,633 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:22:24 2023-11-19 10:20:31,633 INFO 1947952:repeater task quick_wh300 starting 7.6 seconds late 2023-11-19 10:20:38,630 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 10:20:38,630 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:20:19 2023-11-19 10:20:38,630 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:22:24 2023-11-19 10:20:38,630 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.6 seconds late 2023-11-19 10:20:39,287 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 10:20:39,287 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:20:19 2023-11-19 10:20:39,287 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:22:24 2023-11-19 10:21:19,385 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.3 seconds late 2023-11-19 10:21:20,625 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 10:21:20,625 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:20,625 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.6 seconds late 2023-11-19 10:21:20,795 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 10:21:20,795 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:20,795 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.7 seconds late 2023-11-19 10:21:22,077 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 10:21:22,077 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:22,077 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.0 seconds late 2023-11-19 10:21:23,377 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 10:21:23,377 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:23,377 INFO 1947952:repeater task lastensq_os150nb starting 0.4 seconds late 2023-11-19 10:21:25,982 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-19 10:21:25,983 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:21:18 2023-11-19 10:21:25,983 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:26:23 2023-11-19 10:21:25,983 INFO 1947952:repeater task quick_os150nb starting 3.0 seconds late 2023-11-19 10:21:30,222 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 10:21:30,223 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:21:18 2023-11-19 10:21:30,223 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:26:23 2023-11-19 10:21:30,223 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.2 seconds late 2023-11-19 10:21:30,853 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:21:30,854 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:21:18 2023-11-19 10:21:30,854 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:26:23 2023-11-19 10:21:30,854 INFO 1947952:repeater task 3dayplots_os150nb starting 6.8 seconds late 2023-11-19 10:21:34,214 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:21:34,214 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:21:34,215 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.2 seconds late 2023-11-19 10:21:35,222 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 10:21:35,222 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:35,223 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.2 seconds late 2023-11-19 10:21:35,429 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 10:21:35,429 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:35,430 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.4 seconds late 2023-11-19 10:21:36,712 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 10:21:36,712 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:36,713 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.7 seconds late 2023-11-19 10:21:37,998 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 10:21:37,998 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:37,998 INFO 1947952:repeater task lastensq_os38nb starting 13.0 seconds late 2023-11-19 10:21:40,321 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-19 10:21:40,321 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:21:20 2023-11-19 10:21:40,322 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:26:25 2023-11-19 10:21:40,322 INFO 1947952:repeater task quick_os38nb starting 15.3 seconds late 2023-11-19 10:21:45,269 INFO 1947952:repeater task quick_os38nb ended after 4.9 seconds 2023-11-19 10:21:45,270 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:21:20 2023-11-19 10:21:45,270 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:26:25 2023-11-19 10:21:45,270 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.3 seconds late 2023-11-19 10:21:45,895 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:21:45,895 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:21:20 2023-11-19 10:21:45,895 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:26:25 2023-11-19 10:21:45,896 INFO 1947952:repeater task 3dayplots_os38nb starting 21.8 seconds late 2023-11-19 10:21:49,365 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 10:21:49,365 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:21:49,365 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.3 seconds late 2023-11-19 10:21:50,387 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 10:21:50,387 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:50,388 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.3 seconds late 2023-11-19 10:21:50,555 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 10:21:50,555 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:50,556 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.5 seconds late 2023-11-19 10:21:51,833 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 10:21:51,833 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:51,833 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.8 seconds late 2023-11-19 10:21:53,103 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 10:21:53,103 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:53,104 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 33.8 seconds late 2023-11-19 10:21:54,450 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 10:21:54,450 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:54,451 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 35.2 seconds late 2023-11-19 10:21:55,737 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 10:21:55,737 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:55,738 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 36.5 seconds late 2023-11-19 10:21:57,058 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 10:21:57,058 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:21:57,059 INFO 1947952:repeater task plot_hbins starting 37.8 seconds late 2023-11-19 10:22:01,340 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 10:22:01,340 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 10:22:01,340 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 10:36:19 2023-11-19 10:22:06,348 INFO 1947952:repeater task 3dayplots_wh300 starting 42.3 seconds late 2023-11-19 10:22:10,141 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 10:22:10,141 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:22:25,170 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-19 10:22:28,126 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 10:22:28,127 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:22:19 2023-11-19 10:22:28,127 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:24:24 2023-11-19 10:22:28,127 INFO 1947952:repeater task quick_wh300 starting 4.1 seconds late 2023-11-19 10:22:35,027 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-19 10:22:35,028 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:22:19 2023-11-19 10:22:35,028 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:24:24 2023-11-19 10:22:35,028 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-19 10:22:35,671 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:22:35,672 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:22:19 2023-11-19 10:22:35,672 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:24:24 2023-11-19 10:24:25,960 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-19 10:24:28,965 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 10:24:28,966 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:24:19 2023-11-19 10:24:28,966 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:24:28,966 INFO 1947952:repeater task quick_wh300 starting 5.0 seconds late 2023-11-19 10:24:36,320 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 10:24:36,320 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:24:19 2023-11-19 10:24:36,320 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:24:36,321 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2023-11-19 10:24:36,957 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:24:36,958 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:24:19 2023-11-19 10:24:36,958 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:26:24 2023-11-19 10:26:27,245 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.2 seconds late 2023-11-19 10:26:30,101 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 10:26:30,102 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:26:30,102 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.1 seconds late 2023-11-19 10:26:32,856 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 10:26:32,857 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:26:32,857 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.8 seconds late 2023-11-19 10:26:35,832 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 10:26:35,832 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:26:35,832 INFO 1947952:repeater task lastensq_wh300 starting 11.8 seconds late 2023-11-19 10:26:38,669 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:26:38,669 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:26:19 2023-11-19 10:26:38,669 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:28:24 2023-11-19 10:26:38,670 INFO 1947952:repeater task quick_wh300 starting 14.7 seconds late 2023-11-19 10:26:45,866 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 10:26:45,867 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:26:19 2023-11-19 10:26:45,867 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:28:24 2023-11-19 10:26:45,867 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.9 seconds late 2023-11-19 10:26:46,504 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:26:46,505 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:26:19 2023-11-19 10:26:46,505 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:28:24 2023-11-19 10:26:46,505 INFO 1947952:repeater task 3dayplots_wh300 starting 22.5 seconds late 2023-11-19 10:26:50,405 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 10:26:50,405 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:31:24 2023-11-19 10:26:50,407 INFO 1947952:repeater task lastensq_os150nb starting 27.4 seconds late 2023-11-19 10:26:53,123 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-19 10:26:53,124 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:26:18 2023-11-19 10:26:53,124 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:31:23 2023-11-19 10:26:53,124 INFO 1947952:repeater task quick_os150nb starting 30.1 seconds late 2023-11-19 10:26:58,027 INFO 1947952:repeater task quick_os150nb ended after 4.9 seconds 2023-11-19 10:26:58,028 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:26:18 2023-11-19 10:26:58,028 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:31:23 2023-11-19 10:26:58,028 INFO 1947952:repeater task lastfew_vec_os150nb starting 35.0 seconds late 2023-11-19 10:26:58,653 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:26:58,653 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:26:18 2023-11-19 10:26:58,653 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:31:23 2023-11-19 10:26:58,653 INFO 1947952:repeater task 3dayplots_os150nb starting 34.6 seconds late 2023-11-19 10:27:02,034 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:27:02,035 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:31:24 2023-11-19 10:27:02,036 INFO 1947952:repeater task lastensq_os38nb starting 37.0 seconds late 2023-11-19 10:27:04,312 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-19 10:27:04,313 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:26:20 2023-11-19 10:27:04,313 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:31:25 2023-11-19 10:27:04,313 INFO 1947952:repeater task quick_os38nb starting 39.3 seconds late 2023-11-19 10:27:09,299 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 10:27:09,300 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:26:20 2023-11-19 10:27:09,300 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:31:25 2023-11-19 10:27:09,300 INFO 1947952:repeater task lastfew_vec_os38nb starting 44.3 seconds late 2023-11-19 10:27:09,917 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:27:09,917 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:26:20 2023-11-19 10:27:09,918 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:31:25 2023-11-19 10:27:09,918 INFO 1947952:repeater task 3dayplots_os38nb starting 45.9 seconds late 2023-11-19 10:27:13,331 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 10:27:13,331 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:31:24 2023-11-19 10:27:13,333 INFO 1947952:repeater task daily_netcdf starting 49.3 seconds late 2023-11-19 10:27:14,183 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 10:27:14,183 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:28:24,375 INFO 1947952:repeater task lastensq_wh300 starting 0.4 seconds late 2023-11-19 10:28:27,575 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 10:28:27,575 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:28:19 2023-11-19 10:28:27,576 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:30:24 2023-11-19 10:28:27,576 INFO 1947952:repeater task quick_wh300 starting 3.6 seconds late 2023-11-19 10:28:34,422 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-19 10:28:34,422 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:28:19 2023-11-19 10:28:34,422 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:30:24 2023-11-19 10:28:34,423 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-19 10:28:35,050 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:28:35,051 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:28:19 2023-11-19 10:28:35,051 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:30:24 2023-11-19 10:30:25,346 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-19 10:30:28,215 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 10:30:28,215 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:30:19 2023-11-19 10:30:28,215 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:32:24 2023-11-19 10:30:28,216 INFO 1947952:repeater task quick_wh300 starting 4.2 seconds late 2023-11-19 10:30:35,249 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 10:30:35,250 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:30:19 2023-11-19 10:30:35,250 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:32:24 2023-11-19 10:30:35,250 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2023-11-19 10:30:35,879 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:30:35,880 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:30:19 2023-11-19 10:30:35,880 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:32:24 2023-11-19 10:31:26,017 INFO 1947952:repeater task 3dayplots_wh300 starting 2.0 seconds late 2023-11-19 10:31:30,012 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 10:31:30,013 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:31:30,014 INFO 1947952:repeater task lastensq_os150nb starting 7.0 seconds late 2023-11-19 10:31:32,583 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-19 10:31:32,584 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:31:18 2023-11-19 10:31:32,584 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:36:23 2023-11-19 10:31:32,584 INFO 1947952:repeater task quick_os150nb starting 9.6 seconds late 2023-11-19 10:31:36,534 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-19 10:31:36,534 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:31:18 2023-11-19 10:31:36,534 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:36:23 2023-11-19 10:31:36,535 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.5 seconds late 2023-11-19 10:31:37,155 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:31:37,156 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:31:18 2023-11-19 10:31:37,156 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:36:23 2023-11-19 10:31:37,156 INFO 1947952:repeater task 3dayplots_os150nb starting 13.1 seconds late 2023-11-19 10:31:40,532 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:31:40,532 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:31:40,534 INFO 1947952:repeater task lastensq_os38nb starting 15.5 seconds late 2023-11-19 10:31:42,804 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-19 10:31:42,804 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:31:20 2023-11-19 10:31:42,804 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:36:25 2023-11-19 10:31:42,805 INFO 1947952:repeater task quick_os38nb starting 17.8 seconds late 2023-11-19 10:31:47,700 INFO 1947952:repeater task quick_os38nb ended after 4.9 seconds 2023-11-19 10:31:47,700 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:31:20 2023-11-19 10:31:47,700 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:36:25 2023-11-19 10:31:47,701 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.7 seconds late 2023-11-19 10:31:48,315 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:31:48,316 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:31:20 2023-11-19 10:31:48,316 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:36:25 2023-11-19 10:31:48,316 INFO 1947952:repeater task 3dayplots_os38nb starting 24.3 seconds late 2023-11-19 10:31:51,737 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 10:31:51,737 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:32:26,818 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-19 10:32:29,636 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:32:29,636 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:32:19 2023-11-19 10:32:29,636 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:34:24 2023-11-19 10:32:29,636 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-19 10:32:36,775 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-19 10:32:36,776 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:32:19 2023-11-19 10:32:36,776 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:34:24 2023-11-19 10:32:36,776 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-19 10:32:37,409 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:32:37,409 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:32:19 2023-11-19 10:32:37,409 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:34:24 2023-11-19 10:34:27,703 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-19 10:34:30,461 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:34:30,462 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:34:19 2023-11-19 10:34:30,462 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:34:30,462 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-19 10:34:37,748 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 10:34:37,748 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:34:19 2023-11-19 10:34:37,748 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:34:37,749 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-19 10:34:38,377 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:34:38,378 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:34:19 2023-11-19 10:34:38,378 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:36:24 2023-11-19 10:36:23,665 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.6 seconds late 2023-11-19 10:36:24,901 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 10:36:24,902 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:24,902 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.9 seconds late 2023-11-19 10:36:25,072 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 10:36:25,072 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:25,072 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.0 seconds late 2023-11-19 10:36:26,420 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 10:36:26,420 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:26,420 INFO 1947952:repeater task beamdiagnostics_wh300 starting 7.4 seconds late 2023-11-19 10:36:27,626 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-19 10:36:27,626 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:27,627 INFO 1947952:repeater task lastensq_os150nb starting 4.6 seconds late 2023-11-19 10:36:29,817 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 10:36:29,818 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:36:18 2023-11-19 10:36:29,818 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:41:23 2023-11-19 10:36:29,818 INFO 1947952:repeater task quick_os150nb starting 6.8 seconds late 2023-11-19 10:36:34,110 INFO 1947952:repeater task quick_os150nb ended after 4.3 seconds 2023-11-19 10:36:34,110 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:36:18 2023-11-19 10:36:34,110 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:41:23 2023-11-19 10:36:34,111 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.1 seconds late 2023-11-19 10:36:34,730 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:36:34,730 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:36:18 2023-11-19 10:36:34,730 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:41:23 2023-11-19 10:36:34,731 INFO 1947952:repeater task 3dayplots_os150nb starting 10.7 seconds late 2023-11-19 10:36:38,178 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:36:38,178 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:41:24 2023-11-19 10:36:38,179 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.1 seconds late 2023-11-19 10:36:39,189 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 10:36:39,189 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:39,189 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.1 seconds late 2023-11-19 10:36:39,359 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 10:36:39,359 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:39,360 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.3 seconds late 2023-11-19 10:36:40,600 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 10:36:40,601 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:40,601 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.6 seconds late 2023-11-19 10:36:41,858 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 10:36:41,859 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:41,859 INFO 1947952:repeater task lastensq_os38nb starting 16.9 seconds late 2023-11-19 10:36:44,155 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-19 10:36:44,155 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:36:20 2023-11-19 10:36:44,155 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:41:25 2023-11-19 10:36:44,155 INFO 1947952:repeater task quick_os38nb starting 19.2 seconds late 2023-11-19 10:36:49,171 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 10:36:49,172 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:36:20 2023-11-19 10:36:49,172 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:41:25 2023-11-19 10:36:49,172 INFO 1947952:repeater task lastfew_vec_os38nb starting 24.2 seconds late 2023-11-19 10:36:49,779 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:36:49,779 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:36:20 2023-11-19 10:36:49,779 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:41:25 2023-11-19 10:36:49,780 INFO 1947952:repeater task 3dayplots_os38nb starting 25.7 seconds late 2023-11-19 10:36:53,248 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 10:36:53,248 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:41:24 2023-11-19 10:36:53,248 INFO 1947952:repeater task plot_tsstats_os38nb starting 34.2 seconds late 2023-11-19 10:36:54,277 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 10:36:54,278 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:54,278 INFO 1947952:repeater task grab_lastpings_os38nb starting 35.2 seconds late 2023-11-19 10:36:54,449 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 10:36:54,449 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:54,449 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.4 seconds late 2023-11-19 10:36:55,701 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 10:36:55,701 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:55,701 INFO 1947952:repeater task beamdiagnostics_os38nb starting 36.6 seconds late 2023-11-19 10:36:56,968 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 10:36:56,968 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:36:56,969 INFO 1947952:repeater task daily_netcdf starting 32.9 seconds late 2023-11-19 10:36:57,812 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 10:36:57,812 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:36:57,812 INFO 1947952:repeater task ggatime_diagnostics starting 38.8 seconds late 2023-11-19 10:37:14,448 INFO 1947952:repeater task ggatime_diagnostics ended after 16.6 seconds 2023-11-19 10:37:14,449 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:37:14,449 INFO 1947952:repeater task ggatime_diagnostics starting 55.4 seconds late 2023-11-19 10:37:30,463 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-19 10:37:30,464 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:37:30,464 INFO 1947952:repeater task ggatime_diagnostics starting 71.4 seconds late 2023-11-19 10:37:46,527 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-19 10:37:46,527 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:37:46,527 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 87.3 seconds late 2023-11-19 10:37:47,904 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-19 10:37:47,905 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:37:47,905 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 88.6 seconds late 2023-11-19 10:37:49,146 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.2 seconds 2023-11-19 10:37:49,146 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:37:49,147 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 89.9 seconds late 2023-11-19 10:37:50,427 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 10:37:50,427 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:37:50,427 INFO 1947952:repeater task plot_hbins starting 91.2 seconds late 2023-11-19 10:37:54,726 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 10:37:54,726 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 10:37:54,726 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 10:51:19 2023-11-19 10:37:54,727 INFO 1947952:repeater task report_generator starting 95.5 seconds late 2023-11-19 10:38:00,905 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 6.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-19 10:38:00,905 INFO 1947952:repeater task report_generator ended after 6.2 seconds 2023-11-19 10:38:00,905 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/19 11:36:19 2023-11-19 10:38:05,909 INFO 1947952:repeater task noisy_rawtemps_os150 starting 101.9 seconds late 2023-11-19 10:38:08,754 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-19 10:38:08,754 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:38:08,754 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 104.7 seconds late 2023-11-19 10:38:11,451 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 10:38:11,451 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:38:11,452 INFO 1947952:repeater task noisy_rawtemps_os38 starting 107.4 seconds late 2023-11-19 10:38:14,490 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 10:38:14,490 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:38:14,491 INFO 1947952:repeater task lastensq_wh300 starting 110.5 seconds late 2023-11-19 10:38:17,480 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 10:38:17,480 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:36:19 2023-11-19 10:38:17,481 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:38:24 2023-11-19 10:38:17,481 INFO 1947952:repeater task quick_wh300 starting 113.5 seconds late 2023-11-19 10:38:25,706 INFO 1947952:repeater task quick_wh300 ended after 8.2 seconds 2023-11-19 10:38:25,707 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:36:19 2023-11-19 10:38:25,707 INFO 1947952:repeater task quick_wh300: added 1 extra increments to time_to_run 2023-11-19 10:38:25,707 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:40:29 2023-11-19 10:38:25,707 INFO 1947952:repeater task lastfew_vec_wh300 starting 121.7 seconds late 2023-11-19 10:38:26,337 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:38:26,337 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:36:19 2023-11-19 10:38:26,337 INFO 1947952:repeater task lastfew_vec_wh300: added 1 extra increments to time_to_run 2023-11-19 10:38:26,337 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:40:29 2023-11-19 10:38:26,338 INFO 1947952:repeater task 3dayplots_wh300 starting 122.3 seconds late 2023-11-19 10:38:30,280 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 10:38:30,280 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:41:24 2023-11-19 10:38:35,291 INFO 1947952:repeater task lastensq_wh300 starting 11.3 seconds late 2023-11-19 10:38:38,519 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 10:38:38,520 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:38:19 2023-11-19 10:38:38,520 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:40:24 2023-11-19 10:40:28,815 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-19 10:40:31,759 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 10:40:31,760 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:40:19 2023-11-19 10:40:31,760 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:42:24 2023-11-19 10:40:31,760 INFO 1947952:repeater task quick_wh300 starting 2.8 seconds late 2023-11-19 10:40:39,578 INFO 1947952:repeater task quick_wh300 ended after 7.8 seconds 2023-11-19 10:40:39,579 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:40:19 2023-11-19 10:40:39,579 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:42:24 2023-11-19 10:40:39,579 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2023-11-19 10:40:40,217 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:40:40,217 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:40:19 2023-11-19 10:40:40,218 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:42:24 2023-11-19 10:41:25,329 INFO 1947952:repeater task 3dayplots_wh300 starting 1.3 seconds late 2023-11-19 10:41:29,232 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 10:41:29,233 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:41:29,234 INFO 1947952:repeater task lastensq_os150nb starting 6.2 seconds late 2023-11-19 10:41:31,684 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-19 10:41:31,684 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:41:18 2023-11-19 10:41:31,684 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:46:23 2023-11-19 10:41:31,684 INFO 1947952:repeater task quick_os150nb starting 8.7 seconds late 2023-11-19 10:41:36,063 INFO 1947952:repeater task quick_os150nb ended after 4.4 seconds 2023-11-19 10:41:36,063 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:41:18 2023-11-19 10:41:36,064 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:46:23 2023-11-19 10:41:36,064 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.1 seconds late 2023-11-19 10:41:36,680 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:41:36,681 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:41:18 2023-11-19 10:41:36,681 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:46:23 2023-11-19 10:41:36,681 INFO 1947952:repeater task 3dayplots_os150nb starting 12.6 seconds late 2023-11-19 10:41:40,119 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:41:40,120 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:41:40,121 INFO 1947952:repeater task lastensq_os38nb starting 15.1 seconds late 2023-11-19 10:41:42,662 INFO 1947952:repeater task lastensq_os38nb ended after 2.5 seconds 2023-11-19 10:41:42,662 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:41:20 2023-11-19 10:41:42,663 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:46:25 2023-11-19 10:41:42,663 INFO 1947952:repeater task quick_os38nb starting 17.7 seconds late 2023-11-19 10:41:47,911 INFO 1947952:repeater task quick_os38nb ended after 5.2 seconds 2023-11-19 10:41:47,912 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:41:20 2023-11-19 10:41:47,912 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:46:25 2023-11-19 10:41:47,912 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.9 seconds late 2023-11-19 10:41:48,531 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:41:48,531 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:41:20 2023-11-19 10:41:48,532 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:46:25 2023-11-19 10:41:48,532 INFO 1947952:repeater task 3dayplots_os38nb starting 24.5 seconds late 2023-11-19 10:41:51,962 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 10:41:51,962 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:42:27,048 INFO 1947952:repeater task lastensq_wh300 starting 3.0 seconds late 2023-11-19 10:42:29,926 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 10:42:29,926 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:42:19 2023-11-19 10:42:29,927 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:44:24 2023-11-19 10:42:29,927 INFO 1947952:repeater task quick_wh300 starting 5.9 seconds late 2023-11-19 10:42:37,354 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 10:42:37,355 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:42:19 2023-11-19 10:42:37,355 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:44:24 2023-11-19 10:42:37,355 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2023-11-19 10:42:38,007 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 10:42:38,007 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:42:19 2023-11-19 10:42:38,007 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:44:24 2023-11-19 10:44:28,303 INFO 1947952:repeater task lastensq_wh300 starting 4.3 seconds late 2023-11-19 10:44:31,131 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:44:31,131 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:44:19 2023-11-19 10:44:31,131 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:44:31,132 INFO 1947952:repeater task quick_wh300 starting 7.1 seconds late 2023-11-19 10:44:38,288 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 10:44:38,288 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:44:19 2023-11-19 10:44:38,288 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:44:38,289 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-19 10:44:38,917 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:44:38,917 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:44:19 2023-11-19 10:44:38,917 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:46:24 2023-11-19 10:46:24,209 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.2 seconds late 2023-11-19 10:46:27,023 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-19 10:46:27,024 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:46:27,024 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.0 seconds late 2023-11-19 10:46:29,768 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 10:46:29,768 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:46:29,769 INFO 1947952:repeater task noisy_rawtemps_os38 starting 5.7 seconds late 2023-11-19 10:46:32,876 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 10:46:32,876 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:46:32,877 INFO 1947952:repeater task lastensq_wh300 starting 8.9 seconds late 2023-11-19 10:46:35,734 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 10:46:35,735 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:46:19 2023-11-19 10:46:35,735 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:48:24 2023-11-19 10:46:35,735 INFO 1947952:repeater task quick_wh300 starting 11.7 seconds late 2023-11-19 10:46:43,177 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 10:46:43,177 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:46:19 2023-11-19 10:46:43,177 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:48:24 2023-11-19 10:46:43,178 INFO 1947952:repeater task lastfew_vec_wh300 starting 19.2 seconds late 2023-11-19 10:46:43,813 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:46:43,813 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:46:19 2023-11-19 10:46:43,813 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:48:24 2023-11-19 10:46:43,814 INFO 1947952:repeater task 3dayplots_wh300 starting 19.8 seconds late 2023-11-19 10:46:47,719 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 10:46:47,719 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:51:24 2023-11-19 10:46:47,720 INFO 1947952:repeater task lastensq_os150nb starting 24.7 seconds late 2023-11-19 10:46:50,300 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-19 10:46:50,301 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:46:18 2023-11-19 10:46:50,301 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:51:23 2023-11-19 10:46:50,301 INFO 1947952:repeater task quick_os150nb starting 27.3 seconds late 2023-11-19 10:46:54,800 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 10:46:54,800 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:46:18 2023-11-19 10:46:54,800 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:51:23 2023-11-19 10:46:54,800 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.8 seconds late 2023-11-19 10:46:55,417 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:46:55,417 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:46:18 2023-11-19 10:46:55,417 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:51:23 2023-11-19 10:46:55,418 INFO 1947952:repeater task 3dayplots_os150nb starting 31.4 seconds late 2023-11-19 10:46:58,819 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:46:58,819 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:51:24 2023-11-19 10:46:58,821 INFO 1947952:repeater task lastensq_os38nb starting 33.8 seconds late 2023-11-19 10:47:01,059 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 10:47:01,060 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:46:20 2023-11-19 10:47:01,060 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:51:25 2023-11-19 10:47:01,060 INFO 1947952:repeater task quick_os38nb starting 36.1 seconds late 2023-11-19 10:47:06,465 INFO 1947952:repeater task quick_os38nb ended after 5.4 seconds 2023-11-19 10:47:06,466 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:46:20 2023-11-19 10:47:06,466 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:51:25 2023-11-19 10:47:06,466 INFO 1947952:repeater task lastfew_vec_os38nb starting 41.5 seconds late 2023-11-19 10:47:07,095 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:47:07,095 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:46:20 2023-11-19 10:47:07,095 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:51:25 2023-11-19 10:47:07,096 INFO 1947952:repeater task 3dayplots_os38nb starting 43.0 seconds late 2023-11-19 10:47:10,634 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 10:47:10,634 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:51:24 2023-11-19 10:47:10,635 INFO 1947952:repeater task daily_netcdf starting 46.6 seconds late 2023-11-19 10:47:11,580 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 10:47:11,580 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:48:26,757 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-19 10:48:29,929 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 10:48:29,930 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:48:19 2023-11-19 10:48:29,930 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:50:24 2023-11-19 10:48:29,930 INFO 1947952:repeater task quick_wh300 starting 5.9 seconds late 2023-11-19 10:48:37,680 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 10:48:37,680 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:48:19 2023-11-19 10:48:37,681 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:50:24 2023-11-19 10:48:37,681 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-19 10:48:38,306 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:48:38,307 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:48:19 2023-11-19 10:48:38,307 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:50:24 2023-11-19 10:50:28,600 INFO 1947952:repeater task lastensq_wh300 starting 4.6 seconds late 2023-11-19 10:50:31,411 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:50:31,412 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:50:19 2023-11-19 10:50:31,412 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:52:24 2023-11-19 10:50:31,412 INFO 1947952:repeater task quick_wh300 starting 7.4 seconds late 2023-11-19 10:50:38,608 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 10:50:38,608 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:50:19 2023-11-19 10:50:38,608 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:52:24 2023-11-19 10:50:38,609 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.6 seconds late 2023-11-19 10:50:39,236 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:50:39,236 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:50:19 2023-11-19 10:50:39,236 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:52:24 2023-11-19 10:51:19,349 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.3 seconds late 2023-11-19 10:51:20,572 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 10:51:20,572 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:20,573 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.5 seconds late 2023-11-19 10:51:20,740 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 10:51:20,740 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:20,741 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.7 seconds late 2023-11-19 10:51:22,022 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 10:51:22,022 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:22,023 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.0 seconds late 2023-11-19 10:51:23,309 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 10:51:23,309 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:23,309 INFO 1947952:repeater task lastensq_os150nb starting 0.3 seconds late 2023-11-19 10:51:25,569 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 10:51:25,569 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:51:18 2023-11-19 10:51:25,569 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 10:56:23 2023-11-19 10:51:25,570 INFO 1947952:repeater task quick_os150nb starting 2.6 seconds late 2023-11-19 10:51:29,706 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 10:51:29,707 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:51:18 2023-11-19 10:51:29,707 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 10:56:23 2023-11-19 10:51:29,707 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.7 seconds late 2023-11-19 10:51:30,318 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:51:30,319 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:51:18 2023-11-19 10:51:30,319 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 10:56:23 2023-11-19 10:51:30,319 INFO 1947952:repeater task 3dayplots_os150nb starting 6.3 seconds late 2023-11-19 10:51:33,748 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:51:33,748 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:51:33,749 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.7 seconds late 2023-11-19 10:51:34,741 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 10:51:34,741 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:34,741 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.7 seconds late 2023-11-19 10:51:34,905 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 10:51:34,906 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:34,906 INFO 1947952:repeater task beamdiagnostics_os150nb starting 15.9 seconds late 2023-11-19 10:51:36,220 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 10:51:36,220 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:36,221 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.2 seconds late 2023-11-19 10:51:37,495 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 10:51:37,496 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:37,496 INFO 1947952:repeater task lastensq_os38nb starting 12.5 seconds late 2023-11-19 10:51:39,671 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 10:51:39,672 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:51:20 2023-11-19 10:51:39,672 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 10:56:25 2023-11-19 10:51:39,672 INFO 1947952:repeater task quick_os38nb starting 14.7 seconds late 2023-11-19 10:51:44,271 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-19 10:51:44,271 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:51:20 2023-11-19 10:51:44,271 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 10:56:25 2023-11-19 10:51:44,271 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.3 seconds late 2023-11-19 10:51:44,889 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:51:44,890 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:51:20 2023-11-19 10:51:44,890 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 10:56:25 2023-11-19 10:51:44,890 INFO 1947952:repeater task 3dayplots_os38nb starting 20.8 seconds late 2023-11-19 10:51:48,328 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 10:51:48,328 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:51:48,329 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.3 seconds late 2023-11-19 10:51:49,366 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 10:51:49,366 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:49,367 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.3 seconds late 2023-11-19 10:51:49,533 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 10:51:49,533 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:49,534 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.5 seconds late 2023-11-19 10:51:50,785 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 10:51:50,786 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:50,786 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.7 seconds late 2023-11-19 10:51:52,094 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 10:51:52,095 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:52,096 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 32.8 seconds late 2023-11-19 10:51:53,392 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 10:51:53,392 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:53,392 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 34.1 seconds late 2023-11-19 10:51:54,664 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 10:51:54,665 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:54,665 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 35.4 seconds late 2023-11-19 10:51:55,893 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-19 10:51:55,894 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:51:55,894 INFO 1947952:repeater task plot_hbins starting 36.6 seconds late 2023-11-19 10:52:00,240 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 10:52:00,240 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 10:52:00,240 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 11:06:19 2023-11-19 10:52:05,246 INFO 1947952:repeater task 3dayplots_wh300 starting 41.2 seconds late 2023-11-19 10:52:09,270 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 10:52:09,270 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:52:24,302 INFO 1947952:repeater task lastensq_wh300 starting 0.3 seconds late 2023-11-19 10:52:27,012 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 10:52:27,013 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:52:19 2023-11-19 10:52:27,013 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:54:24 2023-11-19 10:52:27,013 INFO 1947952:repeater task quick_wh300 starting 3.0 seconds late 2023-11-19 10:52:34,345 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 10:52:34,346 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:52:19 2023-11-19 10:52:34,346 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:54:24 2023-11-19 10:52:34,346 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-19 10:52:34,981 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:52:34,982 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:52:19 2023-11-19 10:52:34,982 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:54:24 2023-11-19 10:54:25,268 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-19 10:54:28,436 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 10:54:28,437 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:54:19 2023-11-19 10:54:28,437 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:54:28,437 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-19 10:54:36,096 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 10:54:36,097 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:54:19 2023-11-19 10:54:36,097 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:54:36,097 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2023-11-19 10:54:36,743 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:54:36,743 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:54:19 2023-11-19 10:54:36,744 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:56:24 2023-11-19 10:56:27,037 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.0 seconds late 2023-11-19 10:56:29,928 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 10:56:29,929 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 11:06:24 2023-11-19 10:56:29,929 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.9 seconds late 2023-11-19 10:56:32,664 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 10:56:32,664 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 11:06:24 2023-11-19 10:56:32,664 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.6 seconds late 2023-11-19 10:56:35,762 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 10:56:35,763 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 11:06:24 2023-11-19 10:56:35,763 INFO 1947952:repeater task lastensq_wh300 starting 11.8 seconds late 2023-11-19 10:56:38,742 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 10:56:38,743 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:56:19 2023-11-19 10:56:38,743 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 10:58:24 2023-11-19 10:56:38,743 INFO 1947952:repeater task quick_wh300 starting 14.7 seconds late 2023-11-19 10:56:46,390 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 10:56:46,390 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:56:19 2023-11-19 10:56:46,390 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 10:58:24 2023-11-19 10:56:46,390 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.4 seconds late 2023-11-19 10:56:47,031 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 10:56:47,032 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:56:19 2023-11-19 10:56:47,032 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 10:58:24 2023-11-19 10:56:47,032 INFO 1947952:repeater task 3dayplots_wh300 starting 23.0 seconds late 2023-11-19 10:56:50,997 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 10:56:50,997 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:01:24 2023-11-19 10:56:50,998 INFO 1947952:repeater task lastensq_os150nb starting 28.0 seconds late 2023-11-19 10:56:53,154 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 10:56:53,155 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:56:18 2023-11-19 10:56:53,155 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:01:23 2023-11-19 10:56:53,155 INFO 1947952:repeater task quick_os150nb starting 30.2 seconds late 2023-11-19 10:56:57,495 INFO 1947952:repeater task quick_os150nb ended after 4.3 seconds 2023-11-19 10:56:57,496 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:56:18 2023-11-19 10:56:57,496 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:01:23 2023-11-19 10:56:57,496 INFO 1947952:repeater task lastfew_vec_os150nb starting 34.5 seconds late 2023-11-19 10:56:58,125 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 10:56:58,126 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 10:56:18 2023-11-19 10:56:58,126 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:01:23 2023-11-19 10:56:58,126 INFO 1947952:repeater task 3dayplots_os150nb starting 34.1 seconds late 2023-11-19 10:57:01,478 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 10:57:01,478 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:01:24 2023-11-19 10:57:01,479 INFO 1947952:repeater task lastensq_os38nb starting 36.5 seconds late 2023-11-19 10:57:03,643 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 10:57:03,644 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:56:20 2023-11-19 10:57:03,644 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:01:25 2023-11-19 10:57:03,644 INFO 1947952:repeater task quick_os38nb starting 38.6 seconds late 2023-11-19 10:57:08,281 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-19 10:57:08,282 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:56:20 2023-11-19 10:57:08,282 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:01:25 2023-11-19 10:57:08,282 INFO 1947952:repeater task lastfew_vec_os38nb starting 43.3 seconds late 2023-11-19 10:57:08,901 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 10:57:08,902 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 10:56:20 2023-11-19 10:57:08,902 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:01:25 2023-11-19 10:57:08,902 INFO 1947952:repeater task 3dayplots_os38nb starting 44.9 seconds late 2023-11-19 10:57:12,468 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.6 seconds 2023-11-19 10:57:12,469 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:01:24 2023-11-19 10:57:12,470 INFO 1947952:repeater task daily_netcdf starting 48.4 seconds late 2023-11-19 10:57:13,290 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 10:57:13,291 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 11:06:24 2023-11-19 10:58:28,485 INFO 1947952:repeater task lastensq_wh300 starting 4.5 seconds late 2023-11-19 10:58:31,281 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 10:58:31,282 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:58:19 2023-11-19 10:58:31,282 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:00:24 2023-11-19 10:58:31,282 INFO 1947952:repeater task quick_wh300 starting 7.3 seconds late 2023-11-19 10:58:38,644 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 10:58:38,645 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:58:19 2023-11-19 10:58:38,645 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:00:24 2023-11-19 10:58:38,645 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.6 seconds late 2023-11-19 10:58:39,309 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 10:58:39,310 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 10:58:19 2023-11-19 10:58:39,310 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:00:24 2023-11-19 11:00:24,579 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-19 11:00:27,195 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 11:00:27,195 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:00:19 2023-11-19 11:00:27,195 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:02:24 2023-11-19 11:00:27,196 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-19 11:00:34,482 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:00:34,482 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:00:19 2023-11-19 11:00:34,482 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:02:24 2023-11-19 11:00:34,483 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-19 11:00:35,123 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:00:35,124 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:00:19 2023-11-19 11:00:35,124 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:02:24 2023-11-19 11:01:25,239 INFO 1947952:repeater task 3dayplots_wh300 starting 1.2 seconds late 2023-11-19 11:01:29,165 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 11:01:29,166 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:06:24 2023-11-19 11:01:29,167 INFO 1947952:repeater task lastensq_os150nb starting 6.2 seconds late 2023-11-19 11:01:31,512 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 11:01:31,512 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:01:18 2023-11-19 11:01:31,512 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:06:23 2023-11-19 11:01:31,513 INFO 1947952:repeater task quick_os150nb starting 8.5 seconds late 2023-11-19 11:01:35,715 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 11:01:35,716 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:01:18 2023-11-19 11:01:35,716 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:06:23 2023-11-19 11:01:35,716 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.7 seconds late 2023-11-19 11:01:36,338 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:01:36,339 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:01:18 2023-11-19 11:01:36,339 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:06:23 2023-11-19 11:01:36,339 INFO 1947952:repeater task 3dayplots_os150nb starting 12.3 seconds late 2023-11-19 11:01:39,716 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:01:39,717 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:06:24 2023-11-19 11:01:39,718 INFO 1947952:repeater task lastensq_os38nb starting 14.7 seconds late 2023-11-19 11:01:41,831 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 11:01:41,832 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:01:20 2023-11-19 11:01:41,832 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:06:25 2023-11-19 11:01:41,832 INFO 1947952:repeater task quick_os38nb starting 16.8 seconds late 2023-11-19 11:01:46,378 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-19 11:01:46,378 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:01:20 2023-11-19 11:01:46,378 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:06:25 2023-11-19 11:01:46,379 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.4 seconds late 2023-11-19 11:01:46,992 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:01:46,993 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:01:20 2023-11-19 11:01:46,993 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:06:25 2023-11-19 11:01:46,993 INFO 1947952:repeater task 3dayplots_os38nb starting 22.9 seconds late 2023-11-19 11:01:50,443 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:01:50,443 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:06:24 2023-11-19 11:02:25,531 INFO 1947952:repeater task lastensq_wh300 starting 1.5 seconds late 2023-11-19 11:02:28,266 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 11:02:28,266 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:02:19 2023-11-19 11:02:28,266 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:04:24 2023-11-19 11:02:28,267 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-19 11:02:35,139 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-19 11:02:35,139 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:02:19 2023-11-19 11:02:35,139 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:04:24 2023-11-19 11:02:35,140 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-19 11:02:35,776 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:02:35,777 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:02:19 2023-11-19 11:02:35,777 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:04:24 2023-11-19 11:04:26,075 INFO 1947952:repeater task lastensq_wh300 starting 2.1 seconds late 2023-11-19 11:04:29,197 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 11:04:29,198 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:04:19 2023-11-19 11:04:29,198 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:06:24 2023-11-19 11:04:29,198 INFO 1947952:repeater task quick_wh300 starting 5.2 seconds late 2023-11-19 11:04:36,410 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 11:04:36,410 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:04:19 2023-11-19 11:04:36,410 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:06:24 2023-11-19 11:04:36,411 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2023-11-19 11:04:37,047 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:04:37,048 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:04:19 2023-11-19 11:04:37,048 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:06:24 2023-11-19 11:06:22,316 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.3 seconds late 2023-11-19 11:06:23,550 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 11:06:23,550 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:23,550 INFO 1947952:repeater task grab_lastpings_wh300 starting 4.5 seconds late 2023-11-19 11:06:23,718 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 11:06:23,718 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:23,718 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.7 seconds late 2023-11-19 11:06:25,007 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 11:06:25,007 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:25,008 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.0 seconds late 2023-11-19 11:06:26,285 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 11:06:26,285 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:26,285 INFO 1947952:repeater task lastensq_os150nb starting 3.3 seconds late 2023-11-19 11:06:28,536 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 11:06:28,536 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:06:18 2023-11-19 11:06:28,536 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:11:23 2023-11-19 11:06:28,537 INFO 1947952:repeater task quick_os150nb starting 5.5 seconds late 2023-11-19 11:06:32,704 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 11:06:32,704 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:06:18 2023-11-19 11:06:32,705 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:11:23 2023-11-19 11:06:32,705 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.7 seconds late 2023-11-19 11:06:33,312 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:06:33,313 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:06:18 2023-11-19 11:06:33,313 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:11:23 2023-11-19 11:06:33,313 INFO 1947952:repeater task 3dayplots_os150nb starting 9.3 seconds late 2023-11-19 11:06:36,720 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:06:36,720 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:11:24 2023-11-19 11:06:36,721 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.7 seconds late 2023-11-19 11:06:37,725 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 11:06:37,725 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:37,726 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.7 seconds late 2023-11-19 11:06:37,891 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 11:06:37,891 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:37,892 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.8 seconds late 2023-11-19 11:06:39,138 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 11:06:39,138 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:39,138 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.1 seconds late 2023-11-19 11:06:40,416 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 11:06:40,417 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:40,417 INFO 1947952:repeater task lastensq_os38nb starting 15.4 seconds late 2023-11-19 11:06:42,542 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 11:06:42,542 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:06:20 2023-11-19 11:06:42,542 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:11:25 2023-11-19 11:06:42,543 INFO 1947952:repeater task quick_os38nb starting 17.5 seconds late 2023-11-19 11:06:46,927 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-19 11:06:46,927 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:06:20 2023-11-19 11:06:46,928 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:11:25 2023-11-19 11:06:46,928 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.9 seconds late 2023-11-19 11:06:47,540 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:06:47,540 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:06:20 2023-11-19 11:06:47,540 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:11:25 2023-11-19 11:06:47,541 INFO 1947952:repeater task 3dayplots_os38nb starting 23.5 seconds late 2023-11-19 11:06:50,926 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:06:50,926 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:11:24 2023-11-19 11:06:50,926 INFO 1947952:repeater task plot_tsstats_os38nb starting 31.9 seconds late 2023-11-19 11:06:51,955 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 11:06:51,955 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:51,956 INFO 1947952:repeater task grab_lastpings_os38nb starting 32.9 seconds late 2023-11-19 11:06:52,120 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 11:06:52,121 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:52,121 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.1 seconds late 2023-11-19 11:06:53,297 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 11:06:53,297 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:53,297 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.2 seconds late 2023-11-19 11:06:54,550 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 11:06:54,550 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:06:54,551 INFO 1947952:repeater task daily_netcdf starting 30.5 seconds late 2023-11-19 11:06:55,353 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 11:06:55,354 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:06:55,354 INFO 1947952:repeater task ggatime_diagnostics starting 36.3 seconds late 2023-11-19 11:07:11,596 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-19 11:07:11,596 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:07:11,597 INFO 1947952:repeater task ggatime_diagnostics starting 52.5 seconds late 2023-11-19 11:07:27,224 INFO 1947952:repeater task ggatime_diagnostics ended after 15.6 seconds 2023-11-19 11:07:27,225 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:07:27,225 INFO 1947952:repeater task ggatime_diagnostics starting 68.2 seconds late 2023-11-19 11:07:43,090 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 11:07:43,090 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:07:43,091 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 83.8 seconds late 2023-11-19 11:07:44,424 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 11:07:44,424 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:07:44,424 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 85.2 seconds late 2023-11-19 11:07:45,723 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 11:07:45,723 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:07:45,723 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 86.5 seconds late 2023-11-19 11:07:47,020 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 11:07:47,021 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:07:47,021 INFO 1947952:repeater task plot_hbins starting 87.7 seconds late 2023-11-19 11:07:51,469 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 11:07:51,469 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-19 11:07:51,470 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 11:21:19 2023-11-19 11:07:56,473 INFO 1947952:repeater task noisy_rawtemps_os150 starting 92.4 seconds late 2023-11-19 11:07:59,352 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 11:07:59,353 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:07:59,353 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 95.3 seconds late 2023-11-19 11:08:02,212 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-19 11:08:02,212 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:08:02,212 INFO 1947952:repeater task noisy_rawtemps_os38 starting 98.2 seconds late 2023-11-19 11:08:05,348 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 11:08:05,348 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:08:05,349 INFO 1947952:repeater task lastensq_wh300 starting 101.3 seconds late 2023-11-19 11:08:08,365 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 11:08:08,365 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:06:19 2023-11-19 11:08:08,365 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:08:24 2023-11-19 11:08:08,365 INFO 1947952:repeater task quick_wh300 starting 104.4 seconds late 2023-11-19 11:08:15,652 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:08:15,652 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:06:19 2023-11-19 11:08:15,652 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:08:24 2023-11-19 11:08:15,653 INFO 1947952:repeater task lastfew_vec_wh300 starting 111.7 seconds late 2023-11-19 11:08:16,283 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:08:16,283 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:06:19 2023-11-19 11:08:16,283 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:08:24 2023-11-19 11:08:16,284 INFO 1947952:repeater task 3dayplots_wh300 starting 112.2 seconds late 2023-11-19 11:08:20,526 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.2 seconds 2023-11-19 11:08:20,527 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:11:24 2023-11-19 11:08:25,534 INFO 1947952:repeater task lastensq_wh300 starting 1.5 seconds late 2023-11-19 11:08:28,365 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:08:28,365 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:08:19 2023-11-19 11:08:28,365 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:10:24 2023-11-19 11:08:28,366 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-19 11:08:35,666 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:08:35,667 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:08:19 2023-11-19 11:08:35,667 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:10:24 2023-11-19 11:08:35,667 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-19 11:08:36,327 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 11:08:36,328 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:08:19 2023-11-19 11:08:36,328 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:10:24 2023-11-19 11:10:26,624 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-19 11:10:29,404 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:10:29,405 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:10:19 2023-11-19 11:10:29,405 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:12:24 2023-11-19 11:10:29,405 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-19 11:10:37,133 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 11:10:37,133 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:10:19 2023-11-19 11:10:37,134 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:12:24 2023-11-19 11:10:37,134 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2023-11-19 11:10:37,773 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:10:37,773 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:10:19 2023-11-19 11:10:37,773 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:12:24 2023-11-19 11:11:27,992 INFO 1947952:repeater task 3dayplots_wh300 starting 3.9 seconds late 2023-11-19 11:11:31,954 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 11:11:31,954 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:11:31,956 INFO 1947952:repeater task lastensq_os150nb starting 9.0 seconds late 2023-11-19 11:11:34,298 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 11:11:34,298 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:11:18 2023-11-19 11:11:34,298 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:16:23 2023-11-19 11:11:34,299 INFO 1947952:repeater task quick_os150nb starting 11.3 seconds late 2023-11-19 11:11:39,007 INFO 1947952:repeater task quick_os150nb ended after 4.7 seconds 2023-11-19 11:11:39,008 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:11:18 2023-11-19 11:11:39,008 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:16:23 2023-11-19 11:11:39,008 INFO 1947952:repeater task lastfew_vec_os150nb starting 16.0 seconds late 2023-11-19 11:11:39,625 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:11:39,625 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:11:18 2023-11-19 11:11:39,625 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:16:23 2023-11-19 11:11:39,626 INFO 1947952:repeater task 3dayplots_os150nb starting 15.6 seconds late 2023-11-19 11:11:43,022 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:11:43,022 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:11:43,024 INFO 1947952:repeater task lastensq_os38nb starting 18.0 seconds late 2023-11-19 11:11:45,192 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 11:11:45,193 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:11:20 2023-11-19 11:11:45,193 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:16:25 2023-11-19 11:11:45,193 INFO 1947952:repeater task quick_os38nb starting 20.2 seconds late 2023-11-19 11:11:49,966 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 11:11:49,966 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:11:20 2023-11-19 11:11:49,967 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:16:25 2023-11-19 11:11:49,967 INFO 1947952:repeater task lastfew_vec_os38nb starting 25.0 seconds late 2023-11-19 11:11:50,584 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:11:50,585 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:11:20 2023-11-19 11:11:50,585 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:16:25 2023-11-19 11:11:50,585 INFO 1947952:repeater task 3dayplots_os38nb starting 26.5 seconds late 2023-11-19 11:11:54,009 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:11:54,010 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:12:24,085 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-19 11:12:26,807 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 11:12:26,808 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:12:19 2023-11-19 11:12:26,808 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:14:24 2023-11-19 11:12:26,808 INFO 1947952:repeater task quick_wh300 starting 2.8 seconds late 2023-11-19 11:12:34,255 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 11:12:34,256 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:12:19 2023-11-19 11:12:34,256 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:14:24 2023-11-19 11:12:34,256 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-19 11:12:34,896 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:12:34,896 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:12:19 2023-11-19 11:12:34,896 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:14:24 2023-11-19 11:14:25,192 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-19 11:14:28,303 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 11:14:28,304 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:14:19 2023-11-19 11:14:28,304 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:14:28,304 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-19 11:14:35,804 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 11:14:35,805 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:14:19 2023-11-19 11:14:35,805 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:14:35,805 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-19 11:14:36,437 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:14:36,437 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:14:19 2023-11-19 11:14:36,437 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:16:24 2023-11-19 11:16:26,717 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.7 seconds late 2023-11-19 11:16:29,593 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 11:16:29,593 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:16:29,594 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.5 seconds late 2023-11-19 11:16:32,308 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 11:16:32,309 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:16:32,309 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.3 seconds late 2023-11-19 11:16:35,378 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 11:16:35,378 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:16:35,378 INFO 1947952:repeater task lastensq_wh300 starting 11.4 seconds late 2023-11-19 11:16:38,486 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 11:16:38,486 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:16:19 2023-11-19 11:16:38,486 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:18:24 2023-11-19 11:16:38,487 INFO 1947952:repeater task quick_wh300 starting 14.5 seconds late 2023-11-19 11:16:46,070 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 11:16:46,071 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:16:19 2023-11-19 11:16:46,071 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:18:24 2023-11-19 11:16:46,071 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.1 seconds late 2023-11-19 11:16:46,709 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:16:46,709 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:16:19 2023-11-19 11:16:46,709 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:18:24 2023-11-19 11:16:46,710 INFO 1947952:repeater task 3dayplots_wh300 starting 22.7 seconds late 2023-11-19 11:16:50,661 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 11:16:50,661 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:21:24 2023-11-19 11:16:50,662 INFO 1947952:repeater task lastensq_os150nb starting 27.7 seconds late 2023-11-19 11:16:53,288 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-19 11:16:53,289 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:16:18 2023-11-19 11:16:53,289 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:21:23 2023-11-19 11:16:53,289 INFO 1947952:repeater task quick_os150nb starting 30.3 seconds late 2023-11-19 11:16:57,648 INFO 1947952:repeater task quick_os150nb ended after 4.4 seconds 2023-11-19 11:16:57,648 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:16:18 2023-11-19 11:16:57,648 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:21:23 2023-11-19 11:16:57,649 INFO 1947952:repeater task lastfew_vec_os150nb starting 34.6 seconds late 2023-11-19 11:16:58,274 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:16:58,275 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:16:18 2023-11-19 11:16:58,275 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:21:23 2023-11-19 11:16:58,275 INFO 1947952:repeater task 3dayplots_os150nb starting 34.2 seconds late 2023-11-19 11:17:01,658 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:17:01,658 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:21:24 2023-11-19 11:17:01,660 INFO 1947952:repeater task lastensq_os38nb starting 36.7 seconds late 2023-11-19 11:17:04,011 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-19 11:17:04,012 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:16:20 2023-11-19 11:17:04,012 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:21:25 2023-11-19 11:17:04,012 INFO 1947952:repeater task quick_os38nb starting 39.0 seconds late 2023-11-19 11:17:09,434 INFO 1947952:repeater task quick_os38nb ended after 5.4 seconds 2023-11-19 11:17:09,434 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:16:20 2023-11-19 11:17:09,434 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:21:25 2023-11-19 11:17:09,435 INFO 1947952:repeater task lastfew_vec_os38nb starting 44.4 seconds late 2023-11-19 11:17:10,052 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:17:10,053 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:16:20 2023-11-19 11:17:10,053 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:21:25 2023-11-19 11:17:10,053 INFO 1947952:repeater task 3dayplots_os38nb starting 46.0 seconds late 2023-11-19 11:17:13,479 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:17:13,479 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:21:24 2023-11-19 11:17:13,481 INFO 1947952:repeater task daily_netcdf starting 49.4 seconds late 2023-11-19 11:17:14,357 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 11:17:14,357 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:18:24,516 INFO 1947952:repeater task lastensq_wh300 starting 0.5 seconds late 2023-11-19 11:18:27,262 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 11:18:27,263 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:18:19 2023-11-19 11:18:27,263 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:20:24 2023-11-19 11:18:27,263 INFO 1947952:repeater task quick_wh300 starting 3.3 seconds late 2023-11-19 11:18:35,081 INFO 1947952:repeater task quick_wh300 ended after 7.8 seconds 2023-11-19 11:18:35,081 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:18:19 2023-11-19 11:18:35,081 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:20:24 2023-11-19 11:18:35,082 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-19 11:18:35,721 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:18:35,722 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:18:19 2023-11-19 11:18:35,722 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:20:24 2023-11-19 11:20:26,012 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-19 11:20:28,804 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:20:28,805 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:20:19 2023-11-19 11:20:28,805 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:22:24 2023-11-19 11:20:28,805 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-19 11:20:37,153 INFO 1947952:repeater task quick_wh300 ended after 8.3 seconds 2023-11-19 11:20:37,154 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:20:19 2023-11-19 11:20:37,154 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:22:24 2023-11-19 11:20:37,154 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2023-11-19 11:20:37,788 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:20:37,788 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:20:19 2023-11-19 11:20:37,788 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:22:24 2023-11-19 11:21:22,909 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.9 seconds late 2023-11-19 11:21:24,144 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 11:21:24,144 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:24,144 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.1 seconds late 2023-11-19 11:21:24,315 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 11:21:24,315 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:24,315 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.3 seconds late 2023-11-19 11:21:25,553 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-19 11:21:25,554 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:25,554 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.5 seconds late 2023-11-19 11:21:26,829 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 11:21:26,830 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:26,830 INFO 1947952:repeater task lastensq_os150nb starting 3.8 seconds late 2023-11-19 11:21:29,527 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-19 11:21:29,527 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:21:18 2023-11-19 11:21:29,527 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:26:23 2023-11-19 11:21:29,528 INFO 1947952:repeater task quick_os150nb starting 6.5 seconds late 2023-11-19 11:21:34,045 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 11:21:34,046 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:21:18 2023-11-19 11:21:34,046 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:26:23 2023-11-19 11:21:34,046 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.0 seconds late 2023-11-19 11:21:34,657 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:21:34,657 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:21:18 2023-11-19 11:21:34,658 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:26:23 2023-11-19 11:21:34,658 INFO 1947952:repeater task 3dayplots_os150nb starting 10.6 seconds late 2023-11-19 11:21:38,002 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 11:21:38,003 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:21:38,003 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.0 seconds late 2023-11-19 11:21:39,001 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 11:21:39,001 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:39,003 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.0 seconds late 2023-11-19 11:21:39,170 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 11:21:39,170 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:39,171 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.1 seconds late 2023-11-19 11:21:40,427 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 11:21:40,428 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:40,428 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.4 seconds late 2023-11-19 11:21:41,680 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 11:21:41,680 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:41,681 INFO 1947952:repeater task lastensq_os38nb starting 16.7 seconds late 2023-11-19 11:21:43,905 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 11:21:43,905 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:21:20 2023-11-19 11:21:43,905 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:26:25 2023-11-19 11:21:43,906 INFO 1947952:repeater task quick_os38nb starting 18.9 seconds late 2023-11-19 11:21:48,711 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 11:21:48,712 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:21:20 2023-11-19 11:21:48,712 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:26:25 2023-11-19 11:21:48,712 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.7 seconds late 2023-11-19 11:21:49,335 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:21:49,336 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:21:20 2023-11-19 11:21:49,336 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:26:25 2023-11-19 11:21:49,336 INFO 1947952:repeater task 3dayplots_os38nb starting 25.3 seconds late 2023-11-19 11:21:52,739 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:21:52,739 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:21:52,740 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.7 seconds late 2023-11-19 11:21:53,757 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 11:21:53,757 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:53,757 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.7 seconds late 2023-11-19 11:21:53,920 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 11:21:53,921 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:53,921 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.9 seconds late 2023-11-19 11:21:55,199 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 11:21:55,199 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:55,200 INFO 1947952:repeater task beamdiagnostics_os38nb starting 36.1 seconds late 2023-11-19 11:21:56,476 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 11:21:56,476 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:56,478 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 37.2 seconds late 2023-11-19 11:21:57,770 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 11:21:57,770 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:57,771 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 38.5 seconds late 2023-11-19 11:21:59,026 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 11:21:59,027 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:21:59,027 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 39.8 seconds late 2023-11-19 11:22:00,302 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 11:22:00,302 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:22:00,303 INFO 1947952:repeater task plot_hbins starting 41.0 seconds late 2023-11-19 11:22:04,590 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 11:22:04,590 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 11:22:04,590 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 11:36:19 2023-11-19 11:22:09,595 INFO 1947952:repeater task 3dayplots_wh300 starting 45.5 seconds late 2023-11-19 11:22:13,522 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 11:22:13,523 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:22:28,554 INFO 1947952:repeater task lastensq_wh300 starting 4.6 seconds late 2023-11-19 11:22:31,572 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 11:22:31,572 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:22:19 2023-11-19 11:22:31,572 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:24:24 2023-11-19 11:22:31,572 INFO 1947952:repeater task quick_wh300 starting 7.6 seconds late 2023-11-19 11:22:38,842 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:22:38,843 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:22:19 2023-11-19 11:22:38,843 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:24:24 2023-11-19 11:22:38,843 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-19 11:22:39,475 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:22:39,475 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:22:19 2023-11-19 11:22:39,475 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:24:24 2023-11-19 11:24:24,759 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-19 11:24:27,734 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 11:24:27,735 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:24:19 2023-11-19 11:24:27,735 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:24:27,735 INFO 1947952:repeater task quick_wh300 starting 3.7 seconds late 2023-11-19 11:24:34,865 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-19 11:24:34,865 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:24:19 2023-11-19 11:24:34,865 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:24:34,866 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2023-11-19 11:24:35,499 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:24:35,499 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:24:19 2023-11-19 11:24:35,499 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:26:24 2023-11-19 11:26:25,857 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.8 seconds late 2023-11-19 11:26:28,759 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 11:26:28,760 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:26:28,760 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 4.7 seconds late 2023-11-19 11:26:31,514 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 11:26:31,515 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:26:31,515 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.5 seconds late 2023-11-19 11:26:34,556 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 11:26:34,557 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:26:34,557 INFO 1947952:repeater task lastensq_wh300 starting 10.6 seconds late 2023-11-19 11:26:37,146 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 11:26:37,147 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:26:19 2023-11-19 11:26:37,147 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:28:24 2023-11-19 11:26:37,147 INFO 1947952:repeater task quick_wh300 starting 13.1 seconds late 2023-11-19 11:26:44,170 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 11:26:44,170 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:26:19 2023-11-19 11:26:44,170 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:28:24 2023-11-19 11:26:44,171 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.2 seconds late 2023-11-19 11:26:44,809 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:26:44,809 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:26:19 2023-11-19 11:26:44,809 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:28:24 2023-11-19 11:26:44,810 INFO 1947952:repeater task 3dayplots_wh300 starting 20.8 seconds late 2023-11-19 11:26:48,707 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 11:26:48,707 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:31:24 2023-11-19 11:26:48,709 INFO 1947952:repeater task lastensq_os150nb starting 25.7 seconds late 2023-11-19 11:26:50,918 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 11:26:50,918 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:26:18 2023-11-19 11:26:50,919 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:31:23 2023-11-19 11:26:50,919 INFO 1947952:repeater task quick_os150nb starting 27.9 seconds late 2023-11-19 11:26:54,949 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-19 11:26:54,950 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:26:18 2023-11-19 11:26:54,950 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:31:23 2023-11-19 11:26:54,950 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.0 seconds late 2023-11-19 11:26:55,561 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:26:55,562 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:26:18 2023-11-19 11:26:55,562 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:31:23 2023-11-19 11:26:55,562 INFO 1947952:repeater task 3dayplots_os150nb starting 31.5 seconds late 2023-11-19 11:26:58,973 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:26:58,973 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:31:24 2023-11-19 11:26:58,974 INFO 1947952:repeater task lastensq_os38nb starting 34.0 seconds late 2023-11-19 11:27:01,102 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 11:27:01,103 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:26:20 2023-11-19 11:27:01,103 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:31:25 2023-11-19 11:27:01,103 INFO 1947952:repeater task quick_os38nb starting 36.1 seconds late 2023-11-19 11:27:05,548 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-19 11:27:05,548 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:26:20 2023-11-19 11:27:05,548 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:31:25 2023-11-19 11:27:05,549 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.5 seconds late 2023-11-19 11:27:06,155 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:27:06,156 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:26:20 2023-11-19 11:27:06,156 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:31:25 2023-11-19 11:27:06,156 INFO 1947952:repeater task 3dayplots_os38nb starting 42.1 seconds late 2023-11-19 11:27:09,588 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:27:09,588 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:31:24 2023-11-19 11:27:09,589 INFO 1947952:repeater task daily_netcdf starting 45.5 seconds late 2023-11-19 11:27:10,450 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 11:27:10,450 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:28:25,660 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-19 11:28:28,378 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 11:28:28,379 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:28:19 2023-11-19 11:28:28,379 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:30:24 2023-11-19 11:28:28,379 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-19 11:28:35,604 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 11:28:35,604 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:28:19 2023-11-19 11:28:35,604 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:30:24 2023-11-19 11:28:35,605 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2023-11-19 11:28:36,252 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:28:36,252 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:28:19 2023-11-19 11:28:36,252 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:30:24 2023-11-19 11:30:26,584 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-19 11:30:29,441 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 11:30:29,442 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:30:19 2023-11-19 11:30:29,442 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:32:24 2023-11-19 11:30:29,442 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-19 11:30:36,789 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:30:36,789 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:30:19 2023-11-19 11:30:36,789 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:32:24 2023-11-19 11:30:36,790 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-19 11:30:37,421 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:30:37,421 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:30:19 2023-11-19 11:30:37,422 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:32:24 2023-11-19 11:31:27,544 INFO 1947952:repeater task 3dayplots_wh300 starting 3.5 seconds late 2023-11-19 11:31:31,552 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 11:31:31,552 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:31:31,553 INFO 1947952:repeater task lastensq_os150nb starting 8.6 seconds late 2023-11-19 11:31:33,739 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 11:31:33,740 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:31:18 2023-11-19 11:31:33,740 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:36:23 2023-11-19 11:31:33,740 INFO 1947952:repeater task quick_os150nb starting 10.7 seconds late 2023-11-19 11:31:37,776 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-19 11:31:37,776 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:31:18 2023-11-19 11:31:37,776 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:36:23 2023-11-19 11:31:37,777 INFO 1947952:repeater task lastfew_vec_os150nb starting 14.8 seconds late 2023-11-19 11:31:38,396 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:31:38,397 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:31:18 2023-11-19 11:31:38,397 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:36:23 2023-11-19 11:31:38,397 INFO 1947952:repeater task 3dayplots_os150nb starting 14.3 seconds late 2023-11-19 11:31:41,915 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-19 11:31:41,915 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:31:41,917 INFO 1947952:repeater task lastensq_os38nb starting 16.9 seconds late 2023-11-19 11:31:44,019 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 11:31:44,020 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:31:20 2023-11-19 11:31:44,020 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:36:25 2023-11-19 11:31:44,020 INFO 1947952:repeater task quick_os38nb starting 19.0 seconds late 2023-11-19 11:31:48,637 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-19 11:31:48,637 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:31:20 2023-11-19 11:31:48,637 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:36:25 2023-11-19 11:31:48,638 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.6 seconds late 2023-11-19 11:31:49,245 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:31:49,246 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:31:20 2023-11-19 11:31:49,246 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:36:25 2023-11-19 11:31:49,246 INFO 1947952:repeater task 3dayplots_os38nb starting 25.2 seconds late 2023-11-19 11:31:52,678 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:31:52,679 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:32:27,734 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-19 11:32:30,511 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:32:30,511 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:32:19 2023-11-19 11:32:30,511 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:34:24 2023-11-19 11:32:30,512 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-19 11:32:37,803 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:32:37,804 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:32:19 2023-11-19 11:32:37,804 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:34:24 2023-11-19 11:32:37,804 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.8 seconds late 2023-11-19 11:32:38,435 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:32:38,435 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:32:19 2023-11-19 11:32:38,435 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:34:24 2023-11-19 11:34:28,718 INFO 1947952:repeater task lastensq_wh300 starting 4.7 seconds late 2023-11-19 11:34:31,506 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:34:31,506 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:34:19 2023-11-19 11:34:31,506 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:34:31,507 INFO 1947952:repeater task quick_wh300 starting 7.5 seconds late 2023-11-19 11:34:38,787 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:34:38,788 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:34:19 2023-11-19 11:34:38,788 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:34:38,788 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-19 11:34:39,425 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:34:39,425 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:34:19 2023-11-19 11:34:39,425 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:36:24 2023-11-19 11:36:19,669 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.6 seconds late 2023-11-19 11:36:20,912 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 11:36:20,912 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:20,912 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.9 seconds late 2023-11-19 11:36:21,189 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.3 seconds 2023-11-19 11:36:21,189 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:21,190 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.1 seconds late 2023-11-19 11:36:22,664 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.5 seconds 2023-11-19 11:36:22,664 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:22,664 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.6 seconds late 2023-11-19 11:36:23,977 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 11:36:23,977 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:23,978 INFO 1947952:repeater task lastensq_os150nb starting 1.0 seconds late 2023-11-19 11:36:26,289 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 11:36:26,289 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:36:18 2023-11-19 11:36:26,289 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:41:23 2023-11-19 11:36:26,290 INFO 1947952:repeater task quick_os150nb starting 3.3 seconds late 2023-11-19 11:36:30,429 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 11:36:30,429 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:36:18 2023-11-19 11:36:30,430 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:41:23 2023-11-19 11:36:30,430 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.4 seconds late 2023-11-19 11:36:31,047 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:36:31,048 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:36:18 2023-11-19 11:36:31,048 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:41:23 2023-11-19 11:36:31,048 INFO 1947952:repeater task 3dayplots_os150nb starting 7.0 seconds late 2023-11-19 11:36:34,490 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:36:34,491 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:41:24 2023-11-19 11:36:34,491 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.4 seconds late 2023-11-19 11:36:35,492 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 11:36:35,492 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:35,493 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.4 seconds late 2023-11-19 11:36:35,660 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 11:36:35,661 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:35,661 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.6 seconds late 2023-11-19 11:36:36,921 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 11:36:36,921 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:36,921 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.9 seconds late 2023-11-19 11:36:38,174 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 11:36:38,174 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:38,175 INFO 1947952:repeater task lastensq_os38nb starting 13.2 seconds late 2023-11-19 11:36:40,363 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 11:36:40,364 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:36:20 2023-11-19 11:36:40,364 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:41:25 2023-11-19 11:36:40,364 INFO 1947952:repeater task quick_os38nb starting 15.4 seconds late 2023-11-19 11:36:45,140 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 11:36:45,141 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:36:20 2023-11-19 11:36:45,141 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:41:25 2023-11-19 11:36:45,141 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.1 seconds late 2023-11-19 11:36:45,761 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:36:45,761 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:36:20 2023-11-19 11:36:45,761 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:41:25 2023-11-19 11:36:45,762 INFO 1947952:repeater task 3dayplots_os38nb starting 21.7 seconds late 2023-11-19 11:36:49,163 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:36:49,163 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:41:24 2023-11-19 11:36:49,164 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.1 seconds late 2023-11-19 11:36:50,197 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 11:36:50,197 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:50,197 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.1 seconds late 2023-11-19 11:36:50,365 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 11:36:50,365 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:50,365 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.3 seconds late 2023-11-19 11:36:51,624 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 11:36:51,624 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:51,625 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.6 seconds late 2023-11-19 11:36:52,936 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 11:36:52,937 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:36:52,937 INFO 1947952:repeater task daily_netcdf starting 28.9 seconds late 2023-11-19 11:36:53,780 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 11:36:53,781 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:36:53,781 INFO 1947952:repeater task ggatime_diagnostics starting 34.7 seconds late 2023-11-19 11:37:09,639 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 11:37:09,640 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:37:09,640 INFO 1947952:repeater task ggatime_diagnostics starting 50.6 seconds late 2023-11-19 11:37:25,384 INFO 1947952:repeater task ggatime_diagnostics ended after 15.7 seconds 2023-11-19 11:37:25,384 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:37:25,385 INFO 1947952:repeater task ggatime_diagnostics starting 66.3 seconds late 2023-11-19 11:37:41,418 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-19 11:37:41,419 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:37:41,419 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 82.1 seconds late 2023-11-19 11:37:42,826 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-19 11:37:42,826 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:37:42,826 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 83.6 seconds late 2023-11-19 11:37:44,146 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 11:37:44,146 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:37:44,146 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 84.9 seconds late 2023-11-19 11:37:45,424 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 11:37:45,425 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:37:45,425 INFO 1947952:repeater task plot_hbins starting 86.2 seconds late 2023-11-19 11:37:49,792 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 11:37:49,793 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-19 11:37:49,793 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 11:51:19 2023-11-19 11:37:49,793 INFO 1947952:repeater task report_generator starting 90.5 seconds late 2023-11-19 11:37:56,191 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 6.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-19 11:37:56,191 INFO 1947952:repeater task report_generator ended after 6.4 seconds 2023-11-19 11:37:56,191 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/19 12:36:19 2023-11-19 11:38:01,193 INFO 1947952:repeater task noisy_rawtemps_os150 starting 97.1 seconds late 2023-11-19 11:38:04,081 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 11:38:04,081 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:38:04,082 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 100.0 seconds late 2023-11-19 11:38:06,773 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 11:38:06,773 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:38:06,773 INFO 1947952:repeater task noisy_rawtemps_os38 starting 102.7 seconds late 2023-11-19 11:38:09,747 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 11:38:09,748 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:38:09,748 INFO 1947952:repeater task lastensq_wh300 starting 105.7 seconds late 2023-11-19 11:38:12,453 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 11:38:12,454 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:36:19 2023-11-19 11:38:12,454 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:38:24 2023-11-19 11:38:12,455 INFO 1947952:repeater task quick_wh300 starting 108.5 seconds late 2023-11-19 11:38:19,702 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 11:38:19,702 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:36:19 2023-11-19 11:38:19,702 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:38:24 2023-11-19 11:38:19,703 INFO 1947952:repeater task lastfew_vec_wh300 starting 115.7 seconds late 2023-11-19 11:38:20,343 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:38:20,344 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:36:19 2023-11-19 11:38:20,344 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:38:24 2023-11-19 11:38:20,344 INFO 1947952:repeater task 3dayplots_wh300 starting 116.3 seconds late 2023-11-19 11:38:24,484 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.1 seconds 2023-11-19 11:38:24,484 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:41:24 2023-11-19 11:38:29,494 INFO 1947952:repeater task lastensq_wh300 starting 5.5 seconds late 2023-11-19 11:38:32,449 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 11:38:32,450 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:38:19 2023-11-19 11:38:32,450 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:40:24 2023-11-19 11:38:32,450 INFO 1947952:repeater task quick_wh300 starting 8.5 seconds late 2023-11-19 11:38:39,897 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 11:38:39,898 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:38:19 2023-11-19 11:38:39,898 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:40:24 2023-11-19 11:38:39,898 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.9 seconds late 2023-11-19 11:38:40,545 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:38:40,546 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:38:19 2023-11-19 11:38:40,546 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:40:24 2023-11-19 11:40:25,808 INFO 1947952:repeater task lastensq_wh300 starting 1.8 seconds late 2023-11-19 11:40:28,623 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:40:28,623 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:40:19 2023-11-19 11:40:28,623 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:42:24 2023-11-19 11:40:28,624 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-19 11:40:36,008 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 11:40:36,008 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:40:19 2023-11-19 11:40:36,008 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:42:24 2023-11-19 11:40:36,009 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2023-11-19 11:40:36,652 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:40:36,652 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:40:19 2023-11-19 11:40:36,652 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:42:24 2023-11-19 11:41:26,789 INFO 1947952:repeater task 3dayplots_wh300 starting 2.7 seconds late 2023-11-19 11:41:30,709 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 11:41:30,709 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:41:30,711 INFO 1947952:repeater task lastensq_os150nb starting 7.7 seconds late 2023-11-19 11:41:33,012 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 11:41:33,013 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:41:18 2023-11-19 11:41:33,013 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:46:23 2023-11-19 11:41:33,013 INFO 1947952:repeater task quick_os150nb starting 10.0 seconds late 2023-11-19 11:41:37,010 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-19 11:41:37,011 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:41:18 2023-11-19 11:41:37,011 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:46:23 2023-11-19 11:41:37,011 INFO 1947952:repeater task lastfew_vec_os150nb starting 14.0 seconds late 2023-11-19 11:41:37,629 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:41:37,629 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:41:18 2023-11-19 11:41:37,629 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:46:23 2023-11-19 11:41:37,629 INFO 1947952:repeater task 3dayplots_os150nb starting 13.6 seconds late 2023-11-19 11:41:41,020 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:41:41,021 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:41:41,022 INFO 1947952:repeater task lastensq_os38nb starting 16.0 seconds late 2023-11-19 11:41:43,136 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 11:41:43,137 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:41:20 2023-11-19 11:41:43,137 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:46:25 2023-11-19 11:41:43,138 INFO 1947952:repeater task quick_os38nb starting 18.1 seconds late 2023-11-19 11:41:48,263 INFO 1947952:repeater task quick_os38nb ended after 5.1 seconds 2023-11-19 11:41:48,263 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:41:20 2023-11-19 11:41:48,264 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:46:25 2023-11-19 11:41:48,264 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.3 seconds late 2023-11-19 11:41:48,874 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:41:48,875 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:41:20 2023-11-19 11:41:48,875 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:46:25 2023-11-19 11:41:48,875 INFO 1947952:repeater task 3dayplots_os38nb starting 24.8 seconds late 2023-11-19 11:41:52,354 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 11:41:52,354 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:42:27,430 INFO 1947952:repeater task lastensq_wh300 starting 3.4 seconds late 2023-11-19 11:42:30,371 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 11:42:30,371 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:42:19 2023-11-19 11:42:30,371 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:44:24 2023-11-19 11:42:30,372 INFO 1947952:repeater task quick_wh300 starting 6.4 seconds late 2023-11-19 11:42:37,190 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-19 11:42:37,190 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:42:19 2023-11-19 11:42:37,191 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:44:24 2023-11-19 11:42:37,191 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2023-11-19 11:42:37,830 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:42:37,830 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:42:19 2023-11-19 11:42:37,831 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:44:24 2023-11-19 11:44:28,118 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-19 11:44:30,957 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:44:30,957 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:44:19 2023-11-19 11:44:30,957 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:44:30,958 INFO 1947952:repeater task quick_wh300 starting 7.0 seconds late 2023-11-19 11:44:38,321 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 11:44:38,321 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:44:19 2023-11-19 11:44:38,321 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:44:38,321 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-19 11:44:38,956 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:44:38,956 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:44:19 2023-11-19 11:44:38,956 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:46:24 2023-11-19 11:46:24,261 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.2 seconds late 2023-11-19 11:46:27,109 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-19 11:46:27,110 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:46:27,110 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.1 seconds late 2023-11-19 11:46:29,827 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 11:46:29,828 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:46:29,828 INFO 1947952:repeater task noisy_rawtemps_os38 starting 5.8 seconds late 2023-11-19 11:46:32,914 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 11:46:32,914 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:46:32,914 INFO 1947952:repeater task lastensq_wh300 starting 8.9 seconds late 2023-11-19 11:46:35,739 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 11:46:35,739 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:46:19 2023-11-19 11:46:35,739 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:48:24 2023-11-19 11:46:35,740 INFO 1947952:repeater task quick_wh300 starting 11.7 seconds late 2023-11-19 11:46:42,788 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 11:46:42,789 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:46:19 2023-11-19 11:46:42,789 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:48:24 2023-11-19 11:46:42,789 INFO 1947952:repeater task lastfew_vec_wh300 starting 18.8 seconds late 2023-11-19 11:46:43,420 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:46:43,421 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:46:19 2023-11-19 11:46:43,421 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:48:24 2023-11-19 11:46:43,421 INFO 1947952:repeater task 3dayplots_wh300 starting 19.4 seconds late 2023-11-19 11:46:47,220 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 11:46:47,220 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:51:24 2023-11-19 11:46:47,222 INFO 1947952:repeater task lastensq_os150nb starting 24.2 seconds late 2023-11-19 11:46:49,573 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-19 11:46:49,574 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:46:18 2023-11-19 11:46:49,574 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:51:23 2023-11-19 11:46:49,574 INFO 1947952:repeater task quick_os150nb starting 26.6 seconds late 2023-11-19 11:46:53,816 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 11:46:53,816 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:46:18 2023-11-19 11:46:53,817 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:51:23 2023-11-19 11:46:53,817 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.8 seconds late 2023-11-19 11:46:54,428 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:46:54,428 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:46:18 2023-11-19 11:46:54,428 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:51:23 2023-11-19 11:46:54,428 INFO 1947952:repeater task 3dayplots_os150nb starting 30.4 seconds late 2023-11-19 11:46:57,709 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 11:46:57,709 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:51:24 2023-11-19 11:46:57,710 INFO 1947952:repeater task lastensq_os38nb starting 32.7 seconds late 2023-11-19 11:46:59,846 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 11:46:59,847 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:46:20 2023-11-19 11:46:59,847 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:51:25 2023-11-19 11:46:59,847 INFO 1947952:repeater task quick_os38nb starting 34.8 seconds late 2023-11-19 11:47:04,099 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-19 11:47:04,100 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:46:20 2023-11-19 11:47:04,100 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:51:25 2023-11-19 11:47:04,100 INFO 1947952:repeater task lastfew_vec_os38nb starting 39.1 seconds late 2023-11-19 11:47:04,711 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:47:04,711 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:46:20 2023-11-19 11:47:04,711 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:51:25 2023-11-19 11:47:04,712 INFO 1947952:repeater task 3dayplots_os38nb starting 40.7 seconds late 2023-11-19 11:47:08,155 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:47:08,155 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:51:24 2023-11-19 11:47:08,156 INFO 1947952:repeater task daily_netcdf starting 44.1 seconds late 2023-11-19 11:47:08,958 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 11:47:08,958 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:48:24,156 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-19 11:48:27,145 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 11:48:27,145 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:48:19 2023-11-19 11:48:27,145 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:50:24 2023-11-19 11:48:27,146 INFO 1947952:repeater task quick_wh300 starting 3.1 seconds late 2023-11-19 11:48:34,473 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:48:34,474 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:48:19 2023-11-19 11:48:34,474 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:50:24 2023-11-19 11:48:34,474 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-19 11:48:35,112 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:48:35,112 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:48:19 2023-11-19 11:48:35,113 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:50:24 2023-11-19 11:50:25,391 INFO 1947952:repeater task lastensq_wh300 starting 1.4 seconds late 2023-11-19 11:50:28,064 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 11:50:28,064 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:50:19 2023-11-19 11:50:28,064 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:52:24 2023-11-19 11:50:28,065 INFO 1947952:repeater task quick_wh300 starting 4.1 seconds late 2023-11-19 11:50:35,907 INFO 1947952:repeater task quick_wh300 ended after 7.8 seconds 2023-11-19 11:50:35,907 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:50:19 2023-11-19 11:50:35,907 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:52:24 2023-11-19 11:50:35,907 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-19 11:50:36,548 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:50:36,548 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:50:19 2023-11-19 11:50:36,548 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:52:24 2023-11-19 11:51:21,681 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.6 seconds late 2023-11-19 11:51:22,926 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 11:51:22,926 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:22,927 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.9 seconds late 2023-11-19 11:51:23,099 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 11:51:23,099 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:23,100 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.0 seconds late 2023-11-19 11:51:24,439 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 11:51:24,439 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:24,439 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.4 seconds late 2023-11-19 11:51:25,745 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 11:51:25,745 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:25,746 INFO 1947952:repeater task lastensq_os150nb starting 2.7 seconds late 2023-11-19 11:51:27,973 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 11:51:27,973 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:51:18 2023-11-19 11:51:27,973 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 11:56:23 2023-11-19 11:51:27,974 INFO 1947952:repeater task quick_os150nb starting 5.0 seconds late 2023-11-19 11:51:32,507 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 11:51:32,507 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:51:18 2023-11-19 11:51:32,507 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 11:56:23 2023-11-19 11:51:32,508 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.5 seconds late 2023-11-19 11:51:33,119 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:51:33,120 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:51:18 2023-11-19 11:51:33,120 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 11:56:23 2023-11-19 11:51:33,120 INFO 1947952:repeater task 3dayplots_os150nb starting 9.1 seconds late 2023-11-19 11:51:36,532 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:51:36,532 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:51:36,532 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.5 seconds late 2023-11-19 11:51:37,529 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 11:51:37,529 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:37,530 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.5 seconds late 2023-11-19 11:51:37,698 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 11:51:37,698 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:37,698 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.6 seconds late 2023-11-19 11:51:39,032 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 11:51:39,032 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:39,033 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.0 seconds late 2023-11-19 11:51:40,284 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 11:51:40,284 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:40,284 INFO 1947952:repeater task lastensq_os38nb starting 15.3 seconds late 2023-11-19 11:51:42,486 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 11:51:42,486 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:51:20 2023-11-19 11:51:42,486 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 11:56:25 2023-11-19 11:51:42,487 INFO 1947952:repeater task quick_os38nb starting 17.5 seconds late 2023-11-19 11:51:46,806 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-19 11:51:46,806 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:51:20 2023-11-19 11:51:46,806 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 11:56:25 2023-11-19 11:51:46,806 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.8 seconds late 2023-11-19 11:51:47,411 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:51:47,412 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:51:20 2023-11-19 11:51:47,412 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 11:56:25 2023-11-19 11:51:47,412 INFO 1947952:repeater task 3dayplots_os38nb starting 23.4 seconds late 2023-11-19 11:51:50,848 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:51:50,849 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:51:50,849 INFO 1947952:repeater task plot_tsstats_os38nb starting 31.8 seconds late 2023-11-19 11:51:51,883 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 11:51:51,884 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:51,884 INFO 1947952:repeater task grab_lastpings_os38nb starting 32.8 seconds late 2023-11-19 11:51:52,055 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 11:51:52,055 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:52,056 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.0 seconds late 2023-11-19 11:51:53,292 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 11:51:53,292 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:53,292 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.2 seconds late 2023-11-19 11:51:54,495 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 11:51:54,495 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:54,497 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 35.2 seconds late 2023-11-19 11:51:55,785 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 11:51:55,786 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:55,786 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 36.5 seconds late 2023-11-19 11:51:57,053 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 11:51:57,054 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:57,054 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 37.8 seconds late 2023-11-19 11:51:58,362 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 11:51:58,363 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:51:58,363 INFO 1947952:repeater task plot_hbins starting 39.1 seconds late 2023-11-19 11:52:02,679 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 11:52:02,679 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 11:52:02,679 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 12:06:19 2023-11-19 11:52:07,687 INFO 1947952:repeater task 3dayplots_wh300 starting 43.6 seconds late 2023-11-19 11:52:11,573 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 11:52:11,573 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:52:26,606 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-19 11:52:29,689 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 11:52:29,690 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:52:19 2023-11-19 11:52:29,690 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:54:24 2023-11-19 11:52:29,690 INFO 1947952:repeater task quick_wh300 starting 5.7 seconds late 2023-11-19 11:52:37,193 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 11:52:37,194 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:52:19 2023-11-19 11:52:37,194 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:54:24 2023-11-19 11:52:37,194 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2023-11-19 11:52:37,838 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:52:37,838 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:52:19 2023-11-19 11:52:37,838 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:54:24 2023-11-19 11:54:28,096 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-19 11:54:30,981 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 11:54:30,981 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:54:19 2023-11-19 11:54:30,981 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:54:30,982 INFO 1947952:repeater task quick_wh300 starting 7.0 seconds late 2023-11-19 11:54:38,289 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 11:54:38,289 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:54:19 2023-11-19 11:54:38,289 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:54:38,290 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-19 11:54:38,916 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:54:38,916 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:54:19 2023-11-19 11:54:38,916 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:56:24 2023-11-19 11:56:24,176 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.1 seconds late 2023-11-19 11:56:26,968 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-19 11:56:26,968 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 12:06:24 2023-11-19 11:56:26,969 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 2.9 seconds late 2023-11-19 11:56:29,703 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 11:56:29,703 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 12:06:24 2023-11-19 11:56:29,703 INFO 1947952:repeater task noisy_rawtemps_os38 starting 5.7 seconds late 2023-11-19 11:56:32,764 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 11:56:32,764 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 12:06:24 2023-11-19 11:56:32,764 INFO 1947952:repeater task lastensq_wh300 starting 8.8 seconds late 2023-11-19 11:56:35,898 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 11:56:35,898 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:56:19 2023-11-19 11:56:35,898 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 11:58:24 2023-11-19 11:56:35,898 INFO 1947952:repeater task quick_wh300 starting 11.9 seconds late 2023-11-19 11:56:43,008 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-19 11:56:43,008 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:56:19 2023-11-19 11:56:43,008 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 11:58:24 2023-11-19 11:56:43,009 INFO 1947952:repeater task lastfew_vec_wh300 starting 19.0 seconds late 2023-11-19 11:56:43,641 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:56:43,641 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:56:19 2023-11-19 11:56:43,642 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 11:58:24 2023-11-19 11:56:43,642 INFO 1947952:repeater task 3dayplots_wh300 starting 19.6 seconds late 2023-11-19 11:56:47,462 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 11:56:47,462 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:01:24 2023-11-19 11:56:47,463 INFO 1947952:repeater task lastensq_os150nb starting 24.5 seconds late 2023-11-19 11:56:49,606 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-19 11:56:49,606 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:56:18 2023-11-19 11:56:49,607 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:01:23 2023-11-19 11:56:49,607 INFO 1947952:repeater task quick_os150nb starting 26.6 seconds late 2023-11-19 11:56:53,689 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 11:56:53,689 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:56:18 2023-11-19 11:56:53,689 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:01:23 2023-11-19 11:56:53,690 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.7 seconds late 2023-11-19 11:56:54,302 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 11:56:54,303 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 11:56:18 2023-11-19 11:56:54,303 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:01:23 2023-11-19 11:56:54,303 INFO 1947952:repeater task 3dayplots_os150nb starting 30.3 seconds late 2023-11-19 11:56:57,697 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 11:56:57,697 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:01:24 2023-11-19 11:56:57,699 INFO 1947952:repeater task lastensq_os38nb starting 32.7 seconds late 2023-11-19 11:56:59,822 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 11:56:59,822 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:56:20 2023-11-19 11:56:59,822 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:01:25 2023-11-19 11:56:59,823 INFO 1947952:repeater task quick_os38nb starting 34.8 seconds late 2023-11-19 11:57:04,577 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 11:57:04,577 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:56:20 2023-11-19 11:57:04,579 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:01:25 2023-11-19 11:57:04,579 INFO 1947952:repeater task lastfew_vec_os38nb starting 39.6 seconds late 2023-11-19 11:57:05,185 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 11:57:05,186 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 11:56:20 2023-11-19 11:57:05,186 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:01:25 2023-11-19 11:57:05,186 INFO 1947952:repeater task 3dayplots_os38nb starting 41.1 seconds late 2023-11-19 11:57:08,622 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 11:57:08,622 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:01:24 2023-11-19 11:57:08,624 INFO 1947952:repeater task daily_netcdf starting 44.6 seconds late 2023-11-19 11:57:09,464 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 11:57:09,464 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 12:06:24 2023-11-19 11:58:24,647 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-19 11:58:27,381 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 11:58:27,382 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:58:19 2023-11-19 11:58:27,382 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:00:24 2023-11-19 11:58:27,382 INFO 1947952:repeater task quick_wh300 starting 3.4 seconds late 2023-11-19 11:58:34,420 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 11:58:34,421 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:58:19 2023-11-19 11:58:34,421 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:00:24 2023-11-19 11:58:34,421 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-19 11:58:35,050 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 11:58:35,051 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 11:58:19 2023-11-19 11:58:35,051 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:00:24 2023-11-19 12:00:25,405 INFO 1947952:repeater task lastensq_wh300 starting 1.4 seconds late 2023-11-19 12:00:28,295 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 12:00:28,295 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:00:19 2023-11-19 12:00:28,295 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:02:24 2023-11-19 12:00:28,296 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-19 12:00:35,288 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 12:00:35,289 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:00:19 2023-11-19 12:00:35,289 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:02:24 2023-11-19 12:00:35,289 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2023-11-19 12:00:35,941 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 12:00:35,941 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:00:19 2023-11-19 12:00:35,941 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:02:24 2023-11-19 12:01:26,060 INFO 1947952:repeater task 3dayplots_wh300 starting 2.0 seconds late 2023-11-19 12:01:30,034 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 12:01:30,034 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:06:24 2023-11-19 12:01:30,035 INFO 1947952:repeater task lastensq_os150nb starting 7.0 seconds late 2023-11-19 12:01:32,251 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 12:01:32,252 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:01:18 2023-11-19 12:01:32,252 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:06:23 2023-11-19 12:01:32,252 INFO 1947952:repeater task quick_os150nb starting 9.3 seconds late 2023-11-19 12:01:36,466 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 12:01:36,466 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:01:18 2023-11-19 12:01:36,466 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:06:23 2023-11-19 12:01:36,467 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.5 seconds late 2023-11-19 12:01:37,063 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:01:37,064 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:01:18 2023-11-19 12:01:37,064 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:06:23 2023-11-19 12:01:37,064 INFO 1947952:repeater task 3dayplots_os150nb starting 13.0 seconds late 2023-11-19 12:01:40,468 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:01:40,468 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:06:24 2023-11-19 12:01:40,470 INFO 1947952:repeater task lastensq_os38nb starting 15.5 seconds late 2023-11-19 12:01:42,580 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 12:01:42,581 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:01:20 2023-11-19 12:01:42,581 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:06:25 2023-11-19 12:01:42,581 INFO 1947952:repeater task quick_os38nb starting 17.6 seconds late 2023-11-19 12:01:47,190 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-19 12:01:47,190 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:01:20 2023-11-19 12:01:47,190 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:06:25 2023-11-19 12:01:47,191 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.2 seconds late 2023-11-19 12:01:47,826 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:01:47,826 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:01:20 2023-11-19 12:01:47,826 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:06:25 2023-11-19 12:01:47,827 INFO 1947952:repeater task 3dayplots_os38nb starting 23.8 seconds late 2023-11-19 12:01:51,267 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:01:51,267 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:06:24 2023-11-19 12:02:26,331 INFO 1947952:repeater task lastensq_wh300 starting 2.3 seconds late 2023-11-19 12:02:29,503 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 12:02:29,503 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:02:19 2023-11-19 12:02:29,503 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:04:24 2023-11-19 12:02:29,504 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-19 12:02:37,059 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 12:02:37,060 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:02:19 2023-11-19 12:02:37,060 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:04:24 2023-11-19 12:02:37,060 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2023-11-19 12:02:37,704 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:02:37,705 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:02:19 2023-11-19 12:02:37,705 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:04:24 2023-11-19 12:04:28,010 INFO 1947952:repeater task lastensq_wh300 starting 4.0 seconds late 2023-11-19 12:04:31,093 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 12:04:31,093 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:04:19 2023-11-19 12:04:31,093 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:06:24 2023-11-19 12:04:31,094 INFO 1947952:repeater task quick_wh300 starting 7.1 seconds late 2023-11-19 12:04:38,348 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 12:04:38,348 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:04:19 2023-11-19 12:04:38,349 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:06:24 2023-11-19 12:04:38,349 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-19 12:04:38,985 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:04:38,985 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:04:19 2023-11-19 12:04:38,985 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:06:24 2023-11-19 12:06:19,241 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.2 seconds late 2023-11-19 12:06:20,476 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 12:06:20,477 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:20,477 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.4 seconds late 2023-11-19 12:06:20,645 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 12:06:20,646 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:20,646 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.6 seconds late 2023-11-19 12:06:21,990 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 12:06:21,991 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:21,991 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.9 seconds late 2023-11-19 12:06:23,408 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.4 seconds 2023-11-19 12:06:23,408 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:23,408 INFO 1947952:repeater task lastensq_os150nb starting 0.4 seconds late 2023-11-19 12:06:25,752 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 12:06:25,752 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:06:18 2023-11-19 12:06:25,752 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:11:23 2023-11-19 12:06:25,753 INFO 1947952:repeater task quick_os150nb starting 2.8 seconds late 2023-11-19 12:06:29,874 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 12:06:29,875 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:06:18 2023-11-19 12:06:29,875 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:11:23 2023-11-19 12:06:29,875 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.9 seconds late 2023-11-19 12:06:30,484 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:06:30,485 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:06:18 2023-11-19 12:06:30,485 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:11:23 2023-11-19 12:06:30,485 INFO 1947952:repeater task 3dayplots_os150nb starting 6.4 seconds late 2023-11-19 12:06:33,896 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:06:33,896 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:11:24 2023-11-19 12:06:33,896 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.8 seconds late 2023-11-19 12:06:34,895 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 12:06:34,895 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:34,896 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.8 seconds late 2023-11-19 12:06:35,067 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 12:06:35,067 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:35,068 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.0 seconds late 2023-11-19 12:06:36,316 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 12:06:36,316 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:36,316 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.3 seconds late 2023-11-19 12:06:37,595 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 12:06:37,595 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:37,596 INFO 1947952:repeater task lastensq_os38nb starting 12.6 seconds late 2023-11-19 12:06:39,704 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 12:06:39,704 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:06:20 2023-11-19 12:06:39,705 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:11:25 2023-11-19 12:06:39,705 INFO 1947952:repeater task quick_os38nb starting 14.7 seconds late 2023-11-19 12:06:44,394 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 12:06:44,395 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:06:20 2023-11-19 12:06:44,395 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:11:25 2023-11-19 12:06:44,395 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.4 seconds late 2023-11-19 12:06:44,998 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:06:44,999 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:06:20 2023-11-19 12:06:44,999 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:11:25 2023-11-19 12:06:44,999 INFO 1947952:repeater task 3dayplots_os38nb starting 20.9 seconds late 2023-11-19 12:06:48,430 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:06:48,430 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:11:24 2023-11-19 12:06:48,431 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.4 seconds late 2023-11-19 12:06:49,457 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 12:06:49,457 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:49,458 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.4 seconds late 2023-11-19 12:06:49,626 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 12:06:49,626 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:49,627 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.6 seconds late 2023-11-19 12:06:50,876 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 12:06:50,876 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:50,877 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.8 seconds late 2023-11-19 12:06:52,081 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 12:06:52,081 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:06:52,082 INFO 1947952:repeater task daily_netcdf starting 28.0 seconds late 2023-11-19 12:06:52,903 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 12:06:52,903 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:06:52,903 INFO 1947952:repeater task ggatime_diagnostics starting 33.9 seconds late 2023-11-19 12:07:09,154 INFO 1947952:repeater task ggatime_diagnostics ended after 16.3 seconds 2023-11-19 12:07:09,155 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:07:09,155 INFO 1947952:repeater task ggatime_diagnostics starting 50.1 seconds late 2023-11-19 12:07:25,080 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 12:07:25,080 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:07:25,081 INFO 1947952:repeater task ggatime_diagnostics starting 66.0 seconds late 2023-11-19 12:07:41,101 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-19 12:07:41,101 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:07:41,101 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 81.8 seconds late 2023-11-19 12:07:42,476 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-19 12:07:42,476 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:07:42,476 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 83.2 seconds late 2023-11-19 12:07:43,787 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 12:07:43,787 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:07:43,788 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 84.5 seconds late 2023-11-19 12:07:45,057 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 12:07:45,057 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:07:45,058 INFO 1947952:repeater task plot_hbins starting 85.8 seconds late 2023-11-19 12:07:49,323 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 12:07:49,323 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 12:07:49,323 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 12:21:19 2023-11-19 12:07:54,329 INFO 1947952:repeater task noisy_rawtemps_os150 starting 90.3 seconds late 2023-11-19 12:07:57,172 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-19 12:07:57,172 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:07:57,173 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 93.1 seconds late 2023-11-19 12:07:59,923 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 12:07:59,923 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:07:59,923 INFO 1947952:repeater task noisy_rawtemps_os38 starting 95.9 seconds late 2023-11-19 12:08:03,011 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 12:08:03,012 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:08:03,012 INFO 1947952:repeater task lastensq_wh300 starting 99.0 seconds late 2023-11-19 12:08:05,765 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 12:08:05,765 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:06:19 2023-11-19 12:08:05,765 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:08:24 2023-11-19 12:08:05,766 INFO 1947952:repeater task quick_wh300 starting 101.8 seconds late 2023-11-19 12:08:13,054 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 12:08:13,055 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:06:19 2023-11-19 12:08:13,055 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:08:24 2023-11-19 12:08:13,055 INFO 1947952:repeater task lastfew_vec_wh300 starting 109.1 seconds late 2023-11-19 12:08:13,691 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:08:13,691 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:06:19 2023-11-19 12:08:13,691 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:08:24 2023-11-19 12:08:13,692 INFO 1947952:repeater task 3dayplots_wh300 starting 109.6 seconds late 2023-11-19 12:08:17,578 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 12:08:17,578 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:11:24 2023-11-19 12:08:27,594 INFO 1947952:repeater task lastensq_wh300 starting 3.6 seconds late 2023-11-19 12:08:30,801 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 12:08:30,801 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:08:19 2023-11-19 12:08:30,801 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:10:24 2023-11-19 12:08:30,802 INFO 1947952:repeater task quick_wh300 starting 6.8 seconds late 2023-11-19 12:08:38,067 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 12:08:38,068 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:08:19 2023-11-19 12:08:38,068 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:10:24 2023-11-19 12:08:38,068 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.1 seconds late 2023-11-19 12:08:38,701 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:08:38,701 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:08:19 2023-11-19 12:08:38,701 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:10:24 2023-11-19 12:10:28,998 INFO 1947952:repeater task lastensq_wh300 starting 5.0 seconds late 2023-11-19 12:10:31,785 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 12:10:31,786 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:10:19 2023-11-19 12:10:31,786 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:12:24 2023-11-19 12:10:31,786 INFO 1947952:repeater task quick_wh300 starting 7.8 seconds late 2023-11-19 12:10:39,148 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 12:10:39,148 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:10:19 2023-11-19 12:10:39,148 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:12:24 2023-11-19 12:10:39,149 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.1 seconds late 2023-11-19 12:10:39,778 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:10:39,779 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:10:19 2023-11-19 12:10:39,779 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:12:24 2023-11-19 12:11:24,896 INFO 1947952:repeater task 3dayplots_wh300 starting 0.8 seconds late 2023-11-19 12:11:28,840 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 12:11:28,840 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:11:28,841 INFO 1947952:repeater task lastensq_os150nb starting 5.8 seconds late 2023-11-19 12:11:31,113 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 12:11:31,114 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:11:18 2023-11-19 12:11:31,114 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:16:23 2023-11-19 12:11:31,114 INFO 1947952:repeater task quick_os150nb starting 8.1 seconds late 2023-11-19 12:11:35,113 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-19 12:11:35,114 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:11:18 2023-11-19 12:11:35,114 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:16:23 2023-11-19 12:11:35,114 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.1 seconds late 2023-11-19 12:11:35,726 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:11:35,727 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:11:18 2023-11-19 12:11:35,727 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:16:23 2023-11-19 12:11:35,727 INFO 1947952:repeater task 3dayplots_os150nb starting 11.7 seconds late 2023-11-19 12:11:39,112 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:11:39,113 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:11:39,114 INFO 1947952:repeater task lastensq_os38nb starting 14.1 seconds late 2023-11-19 12:11:41,227 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 12:11:41,228 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:11:20 2023-11-19 12:11:41,228 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:16:25 2023-11-19 12:11:41,228 INFO 1947952:repeater task quick_os38nb starting 16.2 seconds late 2023-11-19 12:11:45,557 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-19 12:11:45,557 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:11:20 2023-11-19 12:11:45,557 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:16:25 2023-11-19 12:11:45,558 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.6 seconds late 2023-11-19 12:11:46,158 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:11:46,158 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:11:20 2023-11-19 12:11:46,158 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:16:25 2023-11-19 12:11:46,159 INFO 1947952:repeater task 3dayplots_os38nb starting 22.1 seconds late 2023-11-19 12:11:49,590 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:11:49,590 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:12:24,674 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-19 12:12:27,401 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 12:12:27,401 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:12:19 2023-11-19 12:12:27,401 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:14:24 2023-11-19 12:12:27,402 INFO 1947952:repeater task quick_wh300 starting 3.4 seconds late 2023-11-19 12:12:34,398 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 12:12:34,399 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:12:19 2023-11-19 12:12:34,399 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:14:24 2023-11-19 12:12:34,399 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-19 12:12:35,043 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:12:35,044 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:12:19 2023-11-19 12:12:35,044 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:14:24 2023-11-19 12:14:25,332 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-19 12:14:28,378 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 12:14:28,433 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:14:19 2023-11-19 12:14:28,433 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:14:28,433 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-19 12:14:35,795 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 12:14:35,795 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:14:19 2023-11-19 12:14:35,795 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:14:35,796 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-19 12:14:36,443 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:14:36,444 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:14:19 2023-11-19 12:14:36,444 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:16:24 2023-11-19 12:16:26,729 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.7 seconds late 2023-11-19 12:16:29,686 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-19 12:16:29,686 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:16:29,686 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.6 seconds late 2023-11-19 12:16:32,392 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 12:16:32,392 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:16:32,393 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.3 seconds late 2023-11-19 12:16:35,498 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 12:16:35,498 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:16:35,499 INFO 1947952:repeater task lastensq_wh300 starting 11.5 seconds late 2023-11-19 12:16:38,319 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 12:16:38,319 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:16:19 2023-11-19 12:16:38,319 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:18:24 2023-11-19 12:16:38,320 INFO 1947952:repeater task quick_wh300 starting 14.3 seconds late 2023-11-19 12:16:45,148 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-19 12:16:45,148 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:16:19 2023-11-19 12:16:45,148 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:18:24 2023-11-19 12:16:45,148 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.1 seconds late 2023-11-19 12:16:45,784 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:16:45,785 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:16:19 2023-11-19 12:16:45,785 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:18:24 2023-11-19 12:16:45,785 INFO 1947952:repeater task 3dayplots_wh300 starting 21.7 seconds late 2023-11-19 12:16:49,592 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 12:16:49,592 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:21:24 2023-11-19 12:16:49,593 INFO 1947952:repeater task lastensq_os150nb starting 26.6 seconds late 2023-11-19 12:16:51,813 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 12:16:51,814 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:16:18 2023-11-19 12:16:51,814 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:21:23 2023-11-19 12:16:51,814 INFO 1947952:repeater task quick_os150nb starting 28.8 seconds late 2023-11-19 12:16:56,186 INFO 1947952:repeater task quick_os150nb ended after 4.4 seconds 2023-11-19 12:16:56,186 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:16:18 2023-11-19 12:16:56,186 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:21:23 2023-11-19 12:16:56,187 INFO 1947952:repeater task lastfew_vec_os150nb starting 33.2 seconds late 2023-11-19 12:16:56,819 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:16:56,819 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:16:18 2023-11-19 12:16:56,819 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:21:23 2023-11-19 12:16:56,820 INFO 1947952:repeater task 3dayplots_os150nb starting 32.8 seconds late 2023-11-19 12:17:00,302 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-19 12:17:00,302 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:21:24 2023-11-19 12:17:00,303 INFO 1947952:repeater task lastensq_os38nb starting 35.3 seconds late 2023-11-19 12:17:02,462 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 12:17:02,463 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:16:20 2023-11-19 12:17:02,463 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:21:25 2023-11-19 12:17:02,463 INFO 1947952:repeater task quick_os38nb starting 37.5 seconds late 2023-11-19 12:17:07,436 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 12:17:07,436 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:16:20 2023-11-19 12:17:07,436 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:21:25 2023-11-19 12:17:07,437 INFO 1947952:repeater task lastfew_vec_os38nb starting 42.4 seconds late 2023-11-19 12:17:08,035 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:17:08,036 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:16:20 2023-11-19 12:17:08,036 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:21:25 2023-11-19 12:17:08,036 INFO 1947952:repeater task 3dayplots_os38nb starting 44.0 seconds late 2023-11-19 12:17:11,863 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.8 seconds 2023-11-19 12:17:11,863 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:21:24 2023-11-19 12:17:11,864 INFO 1947952:repeater task daily_netcdf starting 47.8 seconds late 2023-11-19 12:17:12,914 INFO 1947952:repeater task daily_netcdf ended after 1.0 seconds 2023-11-19 12:17:12,914 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:18:28,104 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-19 12:18:31,387 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-19 12:18:31,388 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:18:19 2023-11-19 12:18:31,388 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:20:24 2023-11-19 12:18:31,388 INFO 1947952:repeater task quick_wh300 starting 7.4 seconds late 2023-11-19 12:18:39,223 INFO 1947952:repeater task quick_wh300 ended after 7.8 seconds 2023-11-19 12:18:39,223 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:18:19 2023-11-19 12:18:39,224 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:20:24 2023-11-19 12:18:39,224 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.2 seconds late 2023-11-19 12:18:39,862 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:18:39,863 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:18:19 2023-11-19 12:18:39,863 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:20:24 2023-11-19 12:20:25,146 INFO 1947952:repeater task lastensq_wh300 starting 1.1 seconds late 2023-11-19 12:20:28,131 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 12:20:28,132 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:20:19 2023-11-19 12:20:28,132 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:22:24 2023-11-19 12:20:28,132 INFO 1947952:repeater task quick_wh300 starting 4.1 seconds late 2023-11-19 12:20:35,796 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 12:20:35,796 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:20:19 2023-11-19 12:20:35,796 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:22:24 2023-11-19 12:20:35,797 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-19 12:20:36,434 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:20:36,435 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:20:19 2023-11-19 12:20:36,435 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:22:24 2023-11-19 12:21:21,556 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.5 seconds late 2023-11-19 12:21:22,783 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 12:21:22,783 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:22,783 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.7 seconds late 2023-11-19 12:21:22,951 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 12:21:22,951 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:22,951 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.9 seconds late 2023-11-19 12:21:24,264 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 12:21:24,264 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:24,264 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.2 seconds late 2023-11-19 12:21:25,543 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 12:21:25,543 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:25,543 INFO 1947952:repeater task lastensq_os150nb starting 2.5 seconds late 2023-11-19 12:21:28,243 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-19 12:21:28,244 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:21:18 2023-11-19 12:21:28,244 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:26:23 2023-11-19 12:21:28,244 INFO 1947952:repeater task quick_os150nb starting 5.2 seconds late 2023-11-19 12:21:32,754 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 12:21:32,755 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:21:18 2023-11-19 12:21:32,755 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:26:23 2023-11-19 12:21:32,755 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.8 seconds late 2023-11-19 12:21:33,364 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:21:33,364 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:21:18 2023-11-19 12:21:33,365 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:26:23 2023-11-19 12:21:33,365 INFO 1947952:repeater task 3dayplots_os150nb starting 9.3 seconds late 2023-11-19 12:21:36,751 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:21:36,751 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:21:36,751 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.7 seconds late 2023-11-19 12:21:37,784 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 12:21:37,784 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:37,785 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.7 seconds late 2023-11-19 12:21:37,952 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 12:21:37,952 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:37,953 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.9 seconds late 2023-11-19 12:21:39,206 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 12:21:39,207 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:39,207 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.2 seconds late 2023-11-19 12:21:40,415 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 12:21:40,415 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:40,416 INFO 1947952:repeater task lastensq_os38nb starting 15.4 seconds late 2023-11-19 12:21:42,571 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 12:21:42,571 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:21:20 2023-11-19 12:21:42,571 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:26:25 2023-11-19 12:21:42,572 INFO 1947952:repeater task quick_os38nb starting 17.6 seconds late 2023-11-19 12:21:47,539 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 12:21:47,539 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:21:20 2023-11-19 12:21:47,539 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:26:25 2023-11-19 12:21:47,540 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.5 seconds late 2023-11-19 12:21:48,150 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:21:48,151 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:21:20 2023-11-19 12:21:48,151 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:26:25 2023-11-19 12:21:48,151 INFO 1947952:repeater task 3dayplots_os38nb starting 24.1 seconds late 2023-11-19 12:21:51,545 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:21:51,545 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:21:51,546 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.5 seconds late 2023-11-19 12:21:52,563 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 12:21:52,563 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:52,564 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.5 seconds late 2023-11-19 12:21:52,731 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 12:21:52,732 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:52,732 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.7 seconds late 2023-11-19 12:21:53,962 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 12:21:53,962 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:53,963 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.9 seconds late 2023-11-19 12:21:55,160 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 12:21:55,160 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:55,161 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 35.9 seconds late 2023-11-19 12:21:56,467 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 12:21:56,467 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:56,467 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 37.2 seconds late 2023-11-19 12:21:57,730 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 12:21:57,731 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:57,731 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 38.5 seconds late 2023-11-19 12:21:58,997 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 12:21:58,998 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:21:58,998 INFO 1947952:repeater task plot_hbins starting 39.7 seconds late 2023-11-19 12:22:03,283 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 12:22:03,283 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 12:22:03,283 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 12:36:19 2023-11-19 12:22:08,293 INFO 1947952:repeater task 3dayplots_wh300 starting 44.2 seconds late 2023-11-19 12:22:12,202 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 12:22:12,202 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:22:27,238 INFO 1947952:repeater task lastensq_wh300 starting 3.2 seconds late 2023-11-19 12:22:30,072 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 12:22:30,072 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:22:19 2023-11-19 12:22:30,072 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:24:24 2023-11-19 12:22:30,073 INFO 1947952:repeater task quick_wh300 starting 6.1 seconds late 2023-11-19 12:22:37,223 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-19 12:22:37,223 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:22:19 2023-11-19 12:22:37,223 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:24:24 2023-11-19 12:22:37,224 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2023-11-19 12:22:37,862 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:22:37,862 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:22:19 2023-11-19 12:22:37,862 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:24:24 2023-11-19 12:24:28,148 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-19 12:24:30,807 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 12:24:30,808 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:24:19 2023-11-19 12:24:30,808 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:24:30,808 INFO 1947952:repeater task quick_wh300 starting 6.8 seconds late 2023-11-19 12:24:37,698 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-19 12:24:37,698 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:24:19 2023-11-19 12:24:37,698 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:24:37,699 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-19 12:24:38,335 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:24:38,335 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:24:19 2023-11-19 12:24:38,335 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:26:24 2023-11-19 12:26:23,606 INFO 1947952:repeater task lastensq_os150nb starting 0.6 seconds late 2023-11-19 12:26:26,512 INFO 1947952:repeater task lastensq_os150nb ended after 2.9 seconds 2023-11-19 12:26:26,512 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:26:18 2023-11-19 12:26:26,512 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:31:23 2023-11-19 12:26:26,513 INFO 1947952:repeater task quick_os150nb starting 3.5 seconds late 2023-11-19 12:26:31,080 INFO 1947952:repeater task quick_os150nb ended after 4.6 seconds 2023-11-19 12:26:31,081 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:26:18 2023-11-19 12:26:31,081 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:31:23 2023-11-19 12:26:31,081 INFO 1947952:repeater task lastfew_vec_os150nb starting 8.1 seconds late 2023-11-19 12:26:31,695 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:26:31,695 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:26:18 2023-11-19 12:26:31,695 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:31:23 2023-11-19 12:26:31,696 INFO 1947952:repeater task 3dayplots_os150nb starting 7.6 seconds late 2023-11-19 12:26:35,108 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:26:35,108 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:31:24 2023-11-19 12:26:35,110 INFO 1947952:repeater task lastensq_os38nb starting 10.1 seconds late 2023-11-19 12:26:37,383 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-19 12:26:37,384 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:26:20 2023-11-19 12:26:37,384 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:31:25 2023-11-19 12:26:37,384 INFO 1947952:repeater task quick_os38nb starting 12.4 seconds late 2023-11-19 12:26:42,462 INFO 1947952:repeater task quick_os38nb ended after 5.1 seconds 2023-11-19 12:26:42,462 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:26:20 2023-11-19 12:26:42,462 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:31:25 2023-11-19 12:26:42,463 INFO 1947952:repeater task lastfew_vec_os38nb starting 17.5 seconds late 2023-11-19 12:26:43,077 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:26:43,078 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:26:20 2023-11-19 12:26:43,078 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:31:25 2023-11-19 12:26:43,078 INFO 1947952:repeater task 3dayplots_os38nb starting 19.0 seconds late 2023-11-19 12:26:46,490 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:26:46,490 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:31:24 2023-11-19 12:26:46,491 INFO 1947952:repeater task daily_netcdf starting 22.4 seconds late 2023-11-19 12:26:47,338 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 12:26:47,338 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:26:52,345 INFO 1947952:repeater task noisy_rawtemps_os150 starting 28.3 seconds late 2023-11-19 12:26:55,277 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 12:26:55,277 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:26:55,277 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 31.2 seconds late 2023-11-19 12:26:57,929 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 12:26:57,930 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:26:57,930 INFO 1947952:repeater task noisy_rawtemps_os38 starting 33.9 seconds late 2023-11-19 12:27:01,006 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 12:27:01,006 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:27:01,007 INFO 1947952:repeater task lastensq_wh300 starting 37.0 seconds late 2023-11-19 12:27:03,637 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 12:27:03,638 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:26:19 2023-11-19 12:27:03,638 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:28:24 2023-11-19 12:27:03,638 INFO 1947952:repeater task quick_wh300 starting 39.6 seconds late 2023-11-19 12:27:11,064 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 12:27:11,065 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:26:19 2023-11-19 12:27:11,065 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:28:24 2023-11-19 12:27:11,065 INFO 1947952:repeater task lastfew_vec_wh300 starting 47.1 seconds late 2023-11-19 12:27:11,707 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:27:11,708 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:26:19 2023-11-19 12:27:11,708 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:28:24 2023-11-19 12:27:11,708 INFO 1947952:repeater task 3dayplots_wh300 starting 47.7 seconds late 2023-11-19 12:27:15,521 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 12:27:15,521 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:31:24 2023-11-19 12:28:25,704 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-19 12:28:28,417 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 12:28:28,417 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:28:19 2023-11-19 12:28:28,417 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:30:24 2023-11-19 12:28:28,418 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-19 12:28:35,937 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 12:28:35,938 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:28:19 2023-11-19 12:28:35,938 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:30:24 2023-11-19 12:28:35,938 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-19 12:28:36,584 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:28:36,584 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:28:19 2023-11-19 12:28:36,584 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:30:24 2023-11-19 12:30:26,882 INFO 1947952:repeater task lastensq_wh300 starting 2.9 seconds late 2023-11-19 12:30:29,513 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 12:30:29,513 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:30:19 2023-11-19 12:30:29,513 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:32:24 2023-11-19 12:30:29,514 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-19 12:30:36,450 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-19 12:30:36,450 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:30:19 2023-11-19 12:30:36,450 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:32:24 2023-11-19 12:30:36,450 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2023-11-19 12:30:37,078 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:30:37,078 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:30:19 2023-11-19 12:30:37,078 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:32:24 2023-11-19 12:31:27,192 INFO 1947952:repeater task 3dayplots_wh300 starting 3.1 seconds late 2023-11-19 12:31:31,056 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 12:31:31,057 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:31:31,058 INFO 1947952:repeater task lastensq_os150nb starting 8.1 seconds late 2023-11-19 12:31:33,360 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 12:31:33,361 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:31:18 2023-11-19 12:31:33,361 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:36:23 2023-11-19 12:31:33,361 INFO 1947952:repeater task quick_os150nb starting 10.4 seconds late 2023-11-19 12:31:37,304 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-19 12:31:37,305 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:31:18 2023-11-19 12:31:37,305 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:36:23 2023-11-19 12:31:37,305 INFO 1947952:repeater task lastfew_vec_os150nb starting 14.3 seconds late 2023-11-19 12:31:37,911 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:31:37,912 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:31:18 2023-11-19 12:31:37,912 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:36:23 2023-11-19 12:31:37,912 INFO 1947952:repeater task 3dayplots_os150nb starting 13.9 seconds late 2023-11-19 12:31:41,286 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:31:41,287 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:31:41,288 INFO 1947952:repeater task lastensq_os38nb starting 16.3 seconds late 2023-11-19 12:31:43,444 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 12:31:43,445 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:31:20 2023-11-19 12:31:43,445 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:36:25 2023-11-19 12:31:43,445 INFO 1947952:repeater task quick_os38nb starting 18.4 seconds late 2023-11-19 12:31:47,918 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-19 12:31:47,919 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:31:20 2023-11-19 12:31:47,919 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:36:25 2023-11-19 12:31:47,919 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.9 seconds late 2023-11-19 12:31:48,530 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:31:48,530 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:31:20 2023-11-19 12:31:48,530 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:36:25 2023-11-19 12:31:48,531 INFO 1947952:repeater task 3dayplots_os38nb starting 24.5 seconds late 2023-11-19 12:31:51,944 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:31:51,944 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:32:27,036 INFO 1947952:repeater task lastensq_wh300 starting 3.0 seconds late 2023-11-19 12:32:29,819 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 12:32:29,819 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:32:19 2023-11-19 12:32:29,819 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:34:24 2023-11-19 12:32:29,820 INFO 1947952:repeater task quick_wh300 starting 5.8 seconds late 2023-11-19 12:32:37,178 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 12:32:37,178 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:32:19 2023-11-19 12:32:37,178 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:34:24 2023-11-19 12:32:37,178 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2023-11-19 12:32:37,808 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:32:37,809 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:32:19 2023-11-19 12:32:37,809 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:34:24 2023-11-19 12:34:28,103 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-19 12:34:30,589 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-19 12:34:30,589 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:34:19 2023-11-19 12:34:30,589 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:34:30,589 INFO 1947952:repeater task quick_wh300 starting 6.6 seconds late 2023-11-19 12:34:37,608 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 12:34:37,609 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:34:19 2023-11-19 12:34:37,609 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:34:37,609 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.6 seconds late 2023-11-19 12:34:38,231 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:34:38,231 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:34:19 2023-11-19 12:34:38,232 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:36:24 2023-11-19 12:36:23,522 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.5 seconds late 2023-11-19 12:36:24,917 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2023-11-19 12:36:24,917 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:24,917 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.9 seconds late 2023-11-19 12:36:25,090 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 12:36:25,091 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:25,091 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.0 seconds late 2023-11-19 12:36:26,426 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 12:36:26,427 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:26,427 INFO 1947952:repeater task beamdiagnostics_wh300 starting 7.4 seconds late 2023-11-19 12:36:27,680 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 12:36:27,680 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:27,680 INFO 1947952:repeater task lastensq_os150nb starting 4.7 seconds late 2023-11-19 12:36:30,011 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 12:36:30,012 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:36:18 2023-11-19 12:36:30,012 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:41:23 2023-11-19 12:36:30,012 INFO 1947952:repeater task quick_os150nb starting 7.0 seconds late 2023-11-19 12:36:34,056 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-19 12:36:34,057 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:36:18 2023-11-19 12:36:34,057 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:41:23 2023-11-19 12:36:34,057 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.1 seconds late 2023-11-19 12:36:34,661 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:36:34,662 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:36:18 2023-11-19 12:36:34,662 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:41:23 2023-11-19 12:36:34,662 INFO 1947952:repeater task 3dayplots_os150nb starting 10.6 seconds late 2023-11-19 12:36:38,067 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:36:38,067 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:41:24 2023-11-19 12:36:38,068 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.0 seconds late 2023-11-19 12:36:39,051 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 12:36:39,051 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:39,051 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.0 seconds late 2023-11-19 12:36:39,226 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 12:36:39,226 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:39,226 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.2 seconds late 2023-11-19 12:36:40,486 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 12:36:40,487 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:40,487 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.4 seconds late 2023-11-19 12:36:41,725 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 12:36:41,725 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:41,725 INFO 1947952:repeater task lastensq_os38nb starting 16.7 seconds late 2023-11-19 12:36:43,893 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 12:36:43,893 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:36:20 2023-11-19 12:36:43,894 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:41:25 2023-11-19 12:36:43,894 INFO 1947952:repeater task quick_os38nb starting 18.9 seconds late 2023-11-19 12:36:48,374 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-19 12:36:48,374 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:36:20 2023-11-19 12:36:48,374 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:41:25 2023-11-19 12:36:48,374 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.4 seconds late 2023-11-19 12:36:48,980 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:36:48,981 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:36:20 2023-11-19 12:36:48,981 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:41:25 2023-11-19 12:36:48,981 INFO 1947952:repeater task 3dayplots_os38nb starting 24.9 seconds late 2023-11-19 12:36:52,382 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:36:52,382 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:41:24 2023-11-19 12:36:52,382 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.3 seconds late 2023-11-19 12:36:53,414 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 12:36:53,415 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:53,415 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.4 seconds late 2023-11-19 12:36:53,584 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 12:36:53,584 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:53,584 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.5 seconds late 2023-11-19 12:36:54,827 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 12:36:54,827 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:54,828 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.8 seconds late 2023-11-19 12:36:56,061 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 12:36:56,061 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:36:56,062 INFO 1947952:repeater task daily_netcdf starting 32.0 seconds late 2023-11-19 12:36:56,906 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 12:36:56,907 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:36:56,907 INFO 1947952:repeater task ggatime_diagnostics starting 37.9 seconds late 2023-11-19 12:37:13,183 INFO 1947952:repeater task ggatime_diagnostics ended after 16.3 seconds 2023-11-19 12:37:13,183 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:37:13,184 INFO 1947952:repeater task ggatime_diagnostics starting 54.1 seconds late 2023-11-19 12:37:29,094 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 12:37:29,094 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:37:29,094 INFO 1947952:repeater task ggatime_diagnostics starting 70.0 seconds late 2023-11-19 12:37:45,099 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-19 12:37:45,100 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:37:45,100 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 85.8 seconds late 2023-11-19 12:37:46,451 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-19 12:37:46,452 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:37:46,452 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 87.2 seconds late 2023-11-19 12:37:47,641 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.2 seconds 2023-11-19 12:37:47,641 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:37:47,641 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 88.4 seconds late 2023-11-19 12:37:48,908 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 12:37:48,908 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:37:48,908 INFO 1947952:repeater task plot_hbins starting 89.6 seconds late 2023-11-19 12:37:53,168 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 12:37:53,168 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 12:37:53,169 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 12:51:19 2023-11-19 12:37:53,169 INFO 1947952:repeater task report_generator starting 93.9 seconds late 2023-11-19 12:37:59,631 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 6.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-19 12:37:59,632 INFO 1947952:repeater task report_generator ended after 6.5 seconds 2023-11-19 12:37:59,632 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/19 13:36:19 2023-11-19 12:38:04,637 INFO 1947952:repeater task noisy_rawtemps_os150 starting 100.6 seconds late 2023-11-19 12:38:07,532 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 12:38:07,532 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:38:07,532 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 103.5 seconds late 2023-11-19 12:38:10,257 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 12:38:10,257 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:38:10,258 INFO 1947952:repeater task noisy_rawtemps_os38 starting 106.2 seconds late 2023-11-19 12:38:13,293 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 12:38:13,293 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:38:13,293 INFO 1947952:repeater task lastensq_wh300 starting 109.3 seconds late 2023-11-19 12:38:15,929 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 12:38:15,929 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:36:19 2023-11-19 12:38:15,929 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:38:24 2023-11-19 12:38:15,930 INFO 1947952:repeater task quick_wh300 starting 111.9 seconds late 2023-11-19 12:38:24,084 INFO 1947952:repeater task quick_wh300 ended after 8.2 seconds 2023-11-19 12:38:24,085 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:36:19 2023-11-19 12:38:24,085 INFO 1947952:repeater task quick_wh300: added 1 extra increments to time_to_run 2023-11-19 12:38:24,085 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:40:29 2023-11-19 12:38:24,085 INFO 1947952:repeater task lastfew_vec_wh300 starting 120.1 seconds late 2023-11-19 12:38:24,850 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.8 seconds 2023-11-19 12:38:24,851 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:36:19 2023-11-19 12:38:24,851 INFO 1947952:repeater task lastfew_vec_wh300: added 1 extra increments to time_to_run 2023-11-19 12:38:24,851 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:40:29 2023-11-19 12:38:24,851 INFO 1947952:repeater task 3dayplots_wh300 starting 120.8 seconds late 2023-11-19 12:38:28,811 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 12:38:28,811 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:41:24 2023-11-19 12:38:33,821 INFO 1947952:repeater task lastensq_wh300 starting 9.8 seconds late 2023-11-19 12:38:37,081 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-19 12:38:37,081 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:38:19 2023-11-19 12:38:37,081 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:40:24 2023-11-19 12:40:27,384 INFO 1947952:repeater task lastensq_wh300 starting 3.4 seconds late 2023-11-19 12:40:30,464 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 12:40:30,465 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:40:19 2023-11-19 12:40:30,465 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:42:24 2023-11-19 12:40:30,465 INFO 1947952:repeater task quick_wh300 starting 1.5 seconds late 2023-11-19 12:40:38,051 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 12:40:38,051 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:40:19 2023-11-19 12:40:38,052 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:42:24 2023-11-19 12:40:38,052 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.1 seconds late 2023-11-19 12:40:38,685 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:40:38,685 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:40:19 2023-11-19 12:40:38,685 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:42:24 2023-11-19 12:41:23,790 INFO 1947952:repeater task lastensq_os150nb starting 0.8 seconds late 2023-11-19 12:41:26,227 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-19 12:41:26,227 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:41:18 2023-11-19 12:41:26,227 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:46:23 2023-11-19 12:41:26,228 INFO 1947952:repeater task quick_os150nb starting 3.2 seconds late 2023-11-19 12:41:30,719 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 12:41:30,719 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:41:18 2023-11-19 12:41:30,719 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:46:23 2023-11-19 12:41:30,720 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.7 seconds late 2023-11-19 12:41:31,328 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:41:31,328 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:41:18 2023-11-19 12:41:31,328 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:46:23 2023-11-19 12:41:31,329 INFO 1947952:repeater task 3dayplots_os150nb starting 7.3 seconds late 2023-11-19 12:41:34,716 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:41:34,716 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:41:34,717 INFO 1947952:repeater task lastensq_os38nb starting 9.7 seconds late 2023-11-19 12:41:36,682 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 12:41:36,682 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:41:20 2023-11-19 12:41:36,682 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:46:25 2023-11-19 12:41:36,682 INFO 1947952:repeater task quick_os38nb starting 11.7 seconds late 2023-11-19 12:41:41,335 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 12:41:41,336 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:41:20 2023-11-19 12:41:41,336 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:46:25 2023-11-19 12:41:41,336 INFO 1947952:repeater task lastfew_vec_os38nb starting 16.3 seconds late 2023-11-19 12:41:41,956 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:41:41,957 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:41:20 2023-11-19 12:41:41,957 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:46:25 2023-11-19 12:41:41,957 INFO 1947952:repeater task 3dayplots_os38nb starting 17.9 seconds late 2023-11-19 12:41:45,397 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:41:45,397 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:41:50,407 INFO 1947952:repeater task 3dayplots_wh300 starting 26.4 seconds late 2023-11-19 12:41:54,408 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 12:41:54,408 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:42:24,470 INFO 1947952:repeater task lastensq_wh300 starting 0.5 seconds late 2023-11-19 12:42:27,489 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 12:42:27,489 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:42:19 2023-11-19 12:42:27,489 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:44:24 2023-11-19 12:42:27,490 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-19 12:42:34,304 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-19 12:42:34,304 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:42:19 2023-11-19 12:42:34,304 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:44:24 2023-11-19 12:42:34,304 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-19 12:42:34,942 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:42:34,943 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:42:19 2023-11-19 12:42:34,943 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:44:24 2023-11-19 12:44:25,243 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-19 12:44:28,324 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 12:44:28,325 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:44:19 2023-11-19 12:44:28,325 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:44:28,325 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-19 12:44:35,547 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 12:44:35,548 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:44:19 2023-11-19 12:44:35,548 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:44:35,548 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.5 seconds late 2023-11-19 12:44:36,182 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:44:36,183 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:44:19 2023-11-19 12:44:36,183 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:46:24 2023-11-19 12:46:26,493 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.4 seconds late 2023-11-19 12:46:29,404 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 12:46:29,404 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:46:29,404 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.4 seconds late 2023-11-19 12:46:32,148 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 12:46:32,148 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:46:32,149 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.1 seconds late 2023-11-19 12:46:35,086 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 2.9 seconds 2023-11-19 12:46:35,087 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:46:35,087 INFO 1947952:repeater task lastensq_wh300 starting 11.1 seconds late 2023-11-19 12:46:37,977 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 12:46:37,977 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:46:19 2023-11-19 12:46:37,977 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:48:24 2023-11-19 12:46:37,978 INFO 1947952:repeater task quick_wh300 starting 14.0 seconds late 2023-11-19 12:46:45,267 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 12:46:45,268 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:46:19 2023-11-19 12:46:45,268 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:48:24 2023-11-19 12:46:45,268 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.3 seconds late 2023-11-19 12:46:45,911 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:46:45,911 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:46:19 2023-11-19 12:46:45,912 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:48:24 2023-11-19 12:46:45,912 INFO 1947952:repeater task 3dayplots_wh300 starting 21.9 seconds late 2023-11-19 12:46:49,833 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 12:46:49,833 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:51:24 2023-11-19 12:46:49,835 INFO 1947952:repeater task lastensq_os150nb starting 26.8 seconds late 2023-11-19 12:46:52,472 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-19 12:46:52,472 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:46:18 2023-11-19 12:46:52,472 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:51:23 2023-11-19 12:46:52,472 INFO 1947952:repeater task quick_os150nb starting 29.5 seconds late 2023-11-19 12:46:56,750 INFO 1947952:repeater task quick_os150nb ended after 4.3 seconds 2023-11-19 12:46:56,751 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:46:18 2023-11-19 12:46:56,751 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:51:23 2023-11-19 12:46:56,751 INFO 1947952:repeater task lastfew_vec_os150nb starting 33.8 seconds late 2023-11-19 12:46:57,370 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:46:57,371 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:46:18 2023-11-19 12:46:57,371 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:51:23 2023-11-19 12:46:57,371 INFO 1947952:repeater task 3dayplots_os150nb starting 33.3 seconds late 2023-11-19 12:47:00,736 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:47:00,736 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:51:24 2023-11-19 12:47:00,737 INFO 1947952:repeater task lastensq_os38nb starting 35.7 seconds late 2023-11-19 12:47:03,097 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-19 12:47:03,098 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:46:20 2023-11-19 12:47:03,098 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:51:25 2023-11-19 12:47:03,098 INFO 1947952:repeater task quick_os38nb starting 38.1 seconds late 2023-11-19 12:47:08,299 INFO 1947952:repeater task quick_os38nb ended after 5.2 seconds 2023-11-19 12:47:08,299 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:46:20 2023-11-19 12:47:08,299 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:51:25 2023-11-19 12:47:08,300 INFO 1947952:repeater task lastfew_vec_os38nb starting 43.3 seconds late 2023-11-19 12:47:08,911 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:47:08,911 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:46:20 2023-11-19 12:47:08,911 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:51:25 2023-11-19 12:47:08,911 INFO 1947952:repeater task 3dayplots_os38nb starting 44.9 seconds late 2023-11-19 12:47:12,340 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 12:47:12,340 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:51:24 2023-11-19 12:47:12,341 INFO 1947952:repeater task daily_netcdf starting 48.3 seconds late 2023-11-19 12:47:13,148 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 12:47:13,149 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:48:28,332 INFO 1947952:repeater task lastensq_wh300 starting 4.3 seconds late 2023-11-19 12:48:31,544 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 12:48:31,544 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:48:19 2023-11-19 12:48:31,544 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:50:24 2023-11-19 12:48:31,545 INFO 1947952:repeater task quick_wh300 starting 7.5 seconds late 2023-11-19 12:48:39,243 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 12:48:39,244 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:48:19 2023-11-19 12:48:39,244 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:50:24 2023-11-19 12:48:39,244 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.2 seconds late 2023-11-19 12:48:39,879 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:48:39,880 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:48:19 2023-11-19 12:48:39,880 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:50:24 2023-11-19 12:50:25,174 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-19 12:50:28,503 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-19 12:50:28,503 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:50:19 2023-11-19 12:50:28,504 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:52:24 2023-11-19 12:50:28,504 INFO 1947952:repeater task quick_wh300 starting 4.5 seconds late 2023-11-19 12:50:35,951 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 12:50:35,951 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:50:19 2023-11-19 12:50:35,951 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:52:24 2023-11-19 12:50:35,951 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2023-11-19 12:50:36,591 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:50:36,592 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:50:19 2023-11-19 12:50:36,592 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:52:24 2023-11-19 12:51:21,721 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.7 seconds late 2023-11-19 12:51:22,950 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 12:51:22,950 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:22,951 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.9 seconds late 2023-11-19 12:51:23,119 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 12:51:23,120 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:23,120 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.1 seconds late 2023-11-19 12:51:24,413 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 12:51:24,413 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:24,414 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.4 seconds late 2023-11-19 12:51:25,693 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 12:51:25,694 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:25,694 INFO 1947952:repeater task lastensq_os150nb starting 2.7 seconds late 2023-11-19 12:51:28,148 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-19 12:51:28,148 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:51:18 2023-11-19 12:51:28,148 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 12:56:23 2023-11-19 12:51:28,148 INFO 1947952:repeater task quick_os150nb starting 5.1 seconds late 2023-11-19 12:51:32,714 INFO 1947952:repeater task quick_os150nb ended after 4.6 seconds 2023-11-19 12:51:32,715 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:51:18 2023-11-19 12:51:32,715 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 12:56:23 2023-11-19 12:51:32,715 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.7 seconds late 2023-11-19 12:51:33,331 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:51:33,331 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:51:18 2023-11-19 12:51:33,331 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 12:56:23 2023-11-19 12:51:33,332 INFO 1947952:repeater task 3dayplots_os150nb starting 9.3 seconds late 2023-11-19 12:51:36,701 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:51:36,701 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:51:36,702 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.7 seconds late 2023-11-19 12:51:37,694 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 12:51:37,695 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:37,695 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.6 seconds late 2023-11-19 12:51:37,861 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 12:51:37,861 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:37,862 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.8 seconds late 2023-11-19 12:51:39,148 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 12:51:39,148 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:39,149 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.1 seconds late 2023-11-19 12:51:40,418 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 12:51:40,418 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:40,418 INFO 1947952:repeater task lastensq_os38nb starting 15.4 seconds late 2023-11-19 12:51:42,803 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-19 12:51:42,803 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:51:20 2023-11-19 12:51:42,804 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 12:56:25 2023-11-19 12:51:42,804 INFO 1947952:repeater task quick_os38nb starting 17.8 seconds late 2023-11-19 12:51:48,066 INFO 1947952:repeater task quick_os38nb ended after 5.3 seconds 2023-11-19 12:51:48,066 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:51:20 2023-11-19 12:51:48,066 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 12:56:25 2023-11-19 12:51:48,067 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.1 seconds late 2023-11-19 12:51:48,686 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:51:48,687 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:51:20 2023-11-19 12:51:48,687 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 12:56:25 2023-11-19 12:51:48,687 INFO 1947952:repeater task 3dayplots_os38nb starting 24.6 seconds late 2023-11-19 12:51:52,148 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 12:51:52,149 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:51:52,149 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.1 seconds late 2023-11-19 12:51:53,181 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 12:51:53,182 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:53,182 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.1 seconds late 2023-11-19 12:51:53,357 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 12:51:53,357 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:53,358 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.3 seconds late 2023-11-19 12:51:54,622 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 12:51:54,622 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:54,622 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.6 seconds late 2023-11-19 12:51:55,861 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 12:51:55,861 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:55,863 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 36.6 seconds late 2023-11-19 12:51:57,178 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 12:51:57,178 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:57,178 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 37.9 seconds late 2023-11-19 12:51:58,454 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 12:51:58,454 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:58,455 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 39.2 seconds late 2023-11-19 12:51:59,727 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 12:51:59,727 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:51:59,727 INFO 1947952:repeater task plot_hbins starting 40.5 seconds late 2023-11-19 12:52:04,237 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 12:52:04,237 INFO 1947952:repeater task plot_hbins ended after 4.5 seconds 2023-11-19 12:52:04,238 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 13:06:19 2023-11-19 12:52:09,243 INFO 1947952:repeater task 3dayplots_wh300 starting 45.2 seconds late 2023-11-19 12:52:13,160 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 12:52:13,160 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:52:28,194 INFO 1947952:repeater task lastensq_wh300 starting 4.2 seconds late 2023-11-19 12:52:31,480 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-19 12:52:31,481 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:52:19 2023-11-19 12:52:31,481 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:54:24 2023-11-19 12:52:31,481 INFO 1947952:repeater task quick_wh300 starting 7.5 seconds late 2023-11-19 12:52:38,828 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 12:52:38,829 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:52:19 2023-11-19 12:52:38,829 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:54:24 2023-11-19 12:52:38,829 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-19 12:52:39,464 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:52:39,465 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:52:19 2023-11-19 12:52:39,465 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:54:24 2023-11-19 12:54:24,728 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-19 12:54:27,703 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 12:54:27,704 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:54:19 2023-11-19 12:54:27,704 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:54:27,704 INFO 1947952:repeater task quick_wh300 starting 3.7 seconds late 2023-11-19 12:54:35,280 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 12:54:35,280 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:54:19 2023-11-19 12:54:35,281 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:54:35,281 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2023-11-19 12:54:35,905 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:54:35,906 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:54:19 2023-11-19 12:54:35,906 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:56:24 2023-11-19 12:56:26,185 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.1 seconds late 2023-11-19 12:56:29,115 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 12:56:29,115 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 13:06:24 2023-11-19 12:56:29,115 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.1 seconds late 2023-11-19 12:56:31,988 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-19 12:56:31,988 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 13:06:24 2023-11-19 12:56:31,988 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.9 seconds late 2023-11-19 12:56:34,928 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 2.9 seconds 2023-11-19 12:56:34,928 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 13:06:24 2023-11-19 12:56:34,928 INFO 1947952:repeater task lastensq_wh300 starting 10.9 seconds late 2023-11-19 12:56:38,025 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 12:56:38,026 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:56:19 2023-11-19 12:56:38,026 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 12:58:24 2023-11-19 12:56:38,026 INFO 1947952:repeater task quick_wh300 starting 14.0 seconds late 2023-11-19 12:56:45,223 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 12:56:45,224 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:56:19 2023-11-19 12:56:45,224 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 12:58:24 2023-11-19 12:56:45,224 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.2 seconds late 2023-11-19 12:56:45,855 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:56:45,855 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:56:19 2023-11-19 12:56:45,855 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 12:58:24 2023-11-19 12:56:45,856 INFO 1947952:repeater task 3dayplots_wh300 starting 21.8 seconds late 2023-11-19 12:56:49,719 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 12:56:49,719 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:01:24 2023-11-19 12:56:49,720 INFO 1947952:repeater task lastensq_os150nb starting 26.7 seconds late 2023-11-19 12:56:51,774 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-19 12:56:51,774 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:56:18 2023-11-19 12:56:51,775 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:01:23 2023-11-19 12:56:51,775 INFO 1947952:repeater task quick_os150nb starting 28.8 seconds late 2023-11-19 12:56:55,886 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 12:56:55,887 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:56:18 2023-11-19 12:56:55,887 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:01:23 2023-11-19 12:56:55,887 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.9 seconds late 2023-11-19 12:56:56,486 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 12:56:56,486 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 12:56:18 2023-11-19 12:56:56,486 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:01:23 2023-11-19 12:56:56,487 INFO 1947952:repeater task 3dayplots_os150nb starting 32.4 seconds late 2023-11-19 12:56:59,909 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 12:56:59,909 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:01:24 2023-11-19 12:56:59,911 INFO 1947952:repeater task lastensq_os38nb starting 34.9 seconds late 2023-11-19 12:57:02,173 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-19 12:57:02,173 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:56:20 2023-11-19 12:57:02,173 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:01:25 2023-11-19 12:57:02,174 INFO 1947952:repeater task quick_os38nb starting 37.2 seconds late 2023-11-19 12:57:06,904 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 12:57:06,904 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:56:20 2023-11-19 12:57:06,904 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:01:25 2023-11-19 12:57:06,905 INFO 1947952:repeater task lastfew_vec_os38nb starting 41.9 seconds late 2023-11-19 12:57:07,530 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 12:57:07,530 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 12:56:20 2023-11-19 12:57:07,530 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:01:25 2023-11-19 12:57:07,530 INFO 1947952:repeater task 3dayplots_os38nb starting 43.5 seconds late 2023-11-19 12:57:11,010 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 12:57:11,010 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:01:24 2023-11-19 12:57:11,011 INFO 1947952:repeater task daily_netcdf starting 47.0 seconds late 2023-11-19 12:57:11,839 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 12:57:11,840 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 13:06:24 2023-11-19 12:58:27,027 INFO 1947952:repeater task lastensq_wh300 starting 3.0 seconds late 2023-11-19 12:58:29,707 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 12:58:29,707 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:58:19 2023-11-19 12:58:29,707 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:00:24 2023-11-19 12:58:29,708 INFO 1947952:repeater task quick_wh300 starting 5.7 seconds late 2023-11-19 12:58:37,076 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 12:58:37,077 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:58:19 2023-11-19 12:58:37,077 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:00:24 2023-11-19 12:58:37,077 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2023-11-19 12:58:37,720 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 12:58:37,720 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 12:58:19 2023-11-19 12:58:37,721 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:00:24 2023-11-19 13:00:28,022 INFO 1947952:repeater task lastensq_wh300 starting 4.0 seconds late 2023-11-19 13:00:30,833 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 13:00:30,833 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:00:19 2023-11-19 13:00:30,833 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:02:24 2023-11-19 13:00:30,834 INFO 1947952:repeater task quick_wh300 starting 6.8 seconds late 2023-11-19 13:00:38,369 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 13:00:38,369 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:00:19 2023-11-19 13:00:38,369 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:02:24 2023-11-19 13:00:38,370 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.4 seconds late 2023-11-19 13:00:39,002 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:00:39,002 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:00:19 2023-11-19 13:00:39,002 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:02:24 2023-11-19 13:01:24,112 INFO 1947952:repeater task 3dayplots_wh300 starting 0.1 seconds late 2023-11-19 13:01:28,038 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:01:28,038 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:06:24 2023-11-19 13:01:28,040 INFO 1947952:repeater task lastensq_os150nb starting 5.0 seconds late 2023-11-19 13:01:30,346 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 13:01:30,346 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:01:18 2023-11-19 13:01:30,346 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:06:23 2023-11-19 13:01:30,347 INFO 1947952:repeater task quick_os150nb starting 7.3 seconds late 2023-11-19 13:01:34,477 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 13:01:34,477 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:01:18 2023-11-19 13:01:34,477 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:06:23 2023-11-19 13:01:34,478 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.5 seconds late 2023-11-19 13:01:35,083 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:01:35,083 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:01:18 2023-11-19 13:01:35,083 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:06:23 2023-11-19 13:01:35,084 INFO 1947952:repeater task 3dayplots_os150nb starting 11.0 seconds late 2023-11-19 13:01:38,479 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 13:01:38,479 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:06:24 2023-11-19 13:01:38,480 INFO 1947952:repeater task lastensq_os38nb starting 13.5 seconds late 2023-11-19 13:01:40,567 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 13:01:40,567 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:01:20 2023-11-19 13:01:40,567 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:06:25 2023-11-19 13:01:40,568 INFO 1947952:repeater task quick_os38nb starting 15.6 seconds late 2023-11-19 13:01:45,318 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 13:01:45,319 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:01:20 2023-11-19 13:01:45,319 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:06:25 2023-11-19 13:01:45,319 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.3 seconds late 2023-11-19 13:01:45,926 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:01:45,927 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:01:20 2023-11-19 13:01:45,927 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:06:25 2023-11-19 13:01:45,927 INFO 1947952:repeater task 3dayplots_os38nb starting 21.9 seconds late 2023-11-19 13:01:49,358 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:01:49,358 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:06:24 2023-11-19 13:02:24,435 INFO 1947952:repeater task lastensq_wh300 starting 0.4 seconds late 2023-11-19 13:02:27,512 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 13:02:27,512 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:02:19 2023-11-19 13:02:27,512 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:04:24 2023-11-19 13:02:27,512 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-19 13:02:35,072 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 13:02:35,073 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:02:19 2023-11-19 13:02:35,073 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:04:24 2023-11-19 13:02:35,073 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-19 13:02:35,718 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:02:35,718 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:02:19 2023-11-19 13:02:35,718 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:04:24 2023-11-19 13:04:26,023 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-19 13:04:29,431 INFO 1947952:repeater task lastensq_wh300 ended after 3.4 seconds 2023-11-19 13:04:29,431 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:04:19 2023-11-19 13:04:29,431 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:06:24 2023-11-19 13:04:29,431 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-19 13:04:36,932 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 13:04:36,933 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:04:19 2023-11-19 13:04:36,933 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:06:24 2023-11-19 13:04:36,933 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-19 13:04:37,567 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:04:37,567 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:04:19 2023-11-19 13:04:37,567 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:06:24 2023-11-19 13:06:22,841 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.8 seconds late 2023-11-19 13:06:24,073 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 13:06:24,073 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:24,073 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.0 seconds late 2023-11-19 13:06:24,241 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 13:06:24,241 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:24,241 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.2 seconds late 2023-11-19 13:06:25,538 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 13:06:25,538 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:25,539 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.5 seconds late 2023-11-19 13:06:26,968 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.4 seconds 2023-11-19 13:06:26,968 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:26,969 INFO 1947952:repeater task lastensq_os150nb starting 4.0 seconds late 2023-11-19 13:06:29,415 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-19 13:06:29,415 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:06:18 2023-11-19 13:06:29,416 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:11:23 2023-11-19 13:06:29,416 INFO 1947952:repeater task quick_os150nb starting 6.4 seconds late 2023-11-19 13:06:33,807 INFO 1947952:repeater task quick_os150nb ended after 4.4 seconds 2023-11-19 13:06:33,807 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:06:18 2023-11-19 13:06:33,807 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:11:23 2023-11-19 13:06:33,808 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.8 seconds late 2023-11-19 13:06:34,418 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:06:34,418 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:06:18 2023-11-19 13:06:34,418 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:11:23 2023-11-19 13:06:34,419 INFO 1947952:repeater task 3dayplots_os150nb starting 10.4 seconds late 2023-11-19 13:06:37,757 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 13:06:37,757 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:11:24 2023-11-19 13:06:37,758 INFO 1947952:repeater task plot_tsstats_os150nb starting 18.7 seconds late 2023-11-19 13:06:38,748 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 13:06:38,749 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:38,749 INFO 1947952:repeater task grab_lastpings_os150nb starting 19.7 seconds late 2023-11-19 13:06:38,917 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 13:06:38,917 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:38,917 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.9 seconds late 2023-11-19 13:06:40,181 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 13:06:40,181 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:40,181 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.1 seconds late 2023-11-19 13:06:41,458 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 13:06:41,458 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:41,459 INFO 1947952:repeater task lastensq_os38nb starting 16.5 seconds late 2023-11-19 13:06:43,698 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 13:06:43,698 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:06:20 2023-11-19 13:06:43,698 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:11:25 2023-11-19 13:06:43,698 INFO 1947952:repeater task quick_os38nb starting 18.7 seconds late 2023-11-19 13:06:49,053 INFO 1947952:repeater task quick_os38nb ended after 5.4 seconds 2023-11-19 13:06:49,054 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:06:20 2023-11-19 13:06:49,054 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:11:25 2023-11-19 13:06:49,054 INFO 1947952:repeater task lastfew_vec_os38nb starting 24.1 seconds late 2023-11-19 13:06:49,664 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:06:49,664 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:06:20 2023-11-19 13:06:49,664 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:11:25 2023-11-19 13:06:49,664 INFO 1947952:repeater task 3dayplots_os38nb starting 25.6 seconds late 2023-11-19 13:06:52,994 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.3 seconds 2023-11-19 13:06:52,994 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:11:24 2023-11-19 13:06:52,995 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.9 seconds late 2023-11-19 13:06:54,020 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 13:06:54,021 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:54,021 INFO 1947952:repeater task grab_lastpings_os38nb starting 35.0 seconds late 2023-11-19 13:06:54,190 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 13:06:54,190 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:54,190 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.1 seconds late 2023-11-19 13:06:55,523 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 13:06:55,523 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:55,523 INFO 1947952:repeater task beamdiagnostics_os38nb starting 36.5 seconds late 2023-11-19 13:06:56,739 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 13:06:56,739 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:06:56,739 INFO 1947952:repeater task daily_netcdf starting 32.7 seconds late 2023-11-19 13:06:57,594 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 13:06:57,594 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:06:57,594 INFO 1947952:repeater task ggatime_diagnostics starting 38.5 seconds late 2023-11-19 13:07:14,201 INFO 1947952:repeater task ggatime_diagnostics ended after 16.6 seconds 2023-11-19 13:07:14,202 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:07:14,202 INFO 1947952:repeater task ggatime_diagnostics starting 55.1 seconds late 2023-11-19 13:07:30,110 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 13:07:30,111 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:07:30,111 INFO 1947952:repeater task ggatime_diagnostics starting 71.1 seconds late 2023-11-19 13:07:46,027 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 13:07:46,028 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:07:46,028 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 86.8 seconds late 2023-11-19 13:07:47,474 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-19 13:07:47,475 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:07:47,476 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 88.2 seconds late 2023-11-19 13:07:48,741 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 13:07:48,741 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:07:48,742 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 89.5 seconds late 2023-11-19 13:07:50,013 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 13:07:50,013 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:07:50,013 INFO 1947952:repeater task plot_hbins starting 90.7 seconds late 2023-11-19 13:07:54,359 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 13:07:54,359 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 13:07:54,359 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 13:21:19 2023-11-19 13:07:59,365 INFO 1947952:repeater task noisy_rawtemps_os150 starting 95.3 seconds late 2023-11-19 13:08:02,183 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-19 13:08:02,183 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:08:02,183 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 98.1 seconds late 2023-11-19 13:08:04,931 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 13:08:04,931 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:08:04,932 INFO 1947952:repeater task noisy_rawtemps_os38 starting 100.9 seconds late 2023-11-19 13:08:07,970 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 13:08:07,970 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:08:07,970 INFO 1947952:repeater task lastensq_wh300 starting 104.0 seconds late 2023-11-19 13:08:11,025 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 13:08:11,025 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:06:19 2023-11-19 13:08:11,025 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:08:24 2023-11-19 13:08:11,026 INFO 1947952:repeater task quick_wh300 starting 107.0 seconds late 2023-11-19 13:08:18,609 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 13:08:18,610 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:06:19 2023-11-19 13:08:18,610 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:08:24 2023-11-19 13:08:18,610 INFO 1947952:repeater task lastfew_vec_wh300 starting 114.6 seconds late 2023-11-19 13:08:19,245 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:08:19,245 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:06:19 2023-11-19 13:08:19,246 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:08:24 2023-11-19 13:08:19,246 INFO 1947952:repeater task 3dayplots_wh300 starting 115.2 seconds late 2023-11-19 13:08:23,117 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:08:23,117 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:11:24 2023-11-19 13:08:28,126 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-19 13:08:31,105 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 13:08:31,106 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:08:19 2023-11-19 13:08:31,106 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:10:24 2023-11-19 13:08:31,106 INFO 1947952:repeater task quick_wh300 starting 7.1 seconds late 2023-11-19 13:08:38,361 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 13:08:38,361 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:08:19 2023-11-19 13:08:38,361 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:10:24 2023-11-19 13:08:38,362 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.4 seconds late 2023-11-19 13:08:38,995 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:08:38,996 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:08:19 2023-11-19 13:08:38,996 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:10:24 2023-11-19 13:10:24,246 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-19 13:10:27,231 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 13:10:27,231 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:10:19 2023-11-19 13:10:27,231 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:12:24 2023-11-19 13:10:27,232 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-19 13:10:34,486 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 13:10:34,487 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:10:19 2023-11-19 13:10:34,487 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:12:24 2023-11-19 13:10:34,487 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-19 13:10:35,119 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:10:35,119 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:10:19 2023-11-19 13:10:35,120 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:12:24 2023-11-19 13:11:25,245 INFO 1947952:repeater task 3dayplots_wh300 starting 1.2 seconds late 2023-11-19 13:11:29,140 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:11:29,141 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:11:29,142 INFO 1947952:repeater task lastensq_os150nb starting 6.1 seconds late 2023-11-19 13:11:31,491 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 13:11:31,492 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:11:18 2023-11-19 13:11:31,492 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:16:23 2023-11-19 13:11:31,492 INFO 1947952:repeater task quick_os150nb starting 8.5 seconds late 2023-11-19 13:11:35,653 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 13:11:35,653 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:11:18 2023-11-19 13:11:35,653 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:16:23 2023-11-19 13:11:35,654 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.7 seconds late 2023-11-19 13:11:36,253 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:11:36,254 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:11:18 2023-11-19 13:11:36,254 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:16:23 2023-11-19 13:11:36,254 INFO 1947952:repeater task 3dayplots_os150nb starting 12.2 seconds late 2023-11-19 13:11:39,646 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 13:11:39,646 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:11:39,647 INFO 1947952:repeater task lastensq_os38nb starting 14.6 seconds late 2023-11-19 13:11:41,782 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 13:11:41,783 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:11:20 2023-11-19 13:11:41,783 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:16:25 2023-11-19 13:11:41,783 INFO 1947952:repeater task quick_os38nb starting 16.8 seconds late 2023-11-19 13:11:46,186 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-19 13:11:46,187 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:11:20 2023-11-19 13:11:46,187 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:16:25 2023-11-19 13:11:46,188 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.2 seconds late 2023-11-19 13:11:46,796 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:11:46,796 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:11:20 2023-11-19 13:11:46,796 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:16:25 2023-11-19 13:11:46,797 INFO 1947952:repeater task 3dayplots_os38nb starting 22.7 seconds late 2023-11-19 13:11:50,162 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:11:50,162 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:12:25,242 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-19 13:12:28,287 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 13:12:28,287 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:12:19 2023-11-19 13:12:28,287 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:14:24 2023-11-19 13:12:28,288 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-19 13:12:35,479 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 13:12:35,479 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:12:19 2023-11-19 13:12:35,479 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:14:24 2023-11-19 13:12:35,479 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.5 seconds late 2023-11-19 13:12:36,113 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:12:36,113 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:12:19 2023-11-19 13:12:36,114 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:14:24 2023-11-19 13:14:26,400 INFO 1947952:repeater task lastensq_wh300 starting 2.4 seconds late 2023-11-19 13:14:29,639 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-19 13:14:29,640 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:14:19 2023-11-19 13:14:29,640 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:14:29,640 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-19 13:14:36,841 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 13:14:36,842 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:14:19 2023-11-19 13:14:36,842 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:14:36,842 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-19 13:14:37,475 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:14:37,476 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:14:19 2023-11-19 13:14:37,476 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:16:24 2023-11-19 13:16:27,766 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.7 seconds late 2023-11-19 13:16:30,664 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 13:16:30,664 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:16:30,665 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.6 seconds late 2023-11-19 13:16:33,355 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 13:16:33,355 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:16:33,356 INFO 1947952:repeater task noisy_rawtemps_os38 starting 9.3 seconds late 2023-11-19 13:16:36,426 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 13:16:36,426 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:16:36,426 INFO 1947952:repeater task lastensq_wh300 starting 12.4 seconds late 2023-11-19 13:16:39,039 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 13:16:39,040 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:16:19 2023-11-19 13:16:39,040 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:18:24 2023-11-19 13:16:39,040 INFO 1947952:repeater task quick_wh300 starting 15.0 seconds late 2023-11-19 13:16:46,524 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 13:16:46,524 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:16:19 2023-11-19 13:16:46,524 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:18:24 2023-11-19 13:16:46,525 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.5 seconds late 2023-11-19 13:16:47,157 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:16:47,158 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:16:19 2023-11-19 13:16:47,158 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:18:24 2023-11-19 13:16:47,158 INFO 1947952:repeater task 3dayplots_wh300 starting 23.1 seconds late 2023-11-19 13:16:51,060 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:16:51,060 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:21:24 2023-11-19 13:16:51,062 INFO 1947952:repeater task lastensq_os150nb starting 28.1 seconds late 2023-11-19 13:16:53,182 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-19 13:16:53,182 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:16:18 2023-11-19 13:16:53,182 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:21:23 2023-11-19 13:16:53,183 INFO 1947952:repeater task quick_os150nb starting 30.2 seconds late 2023-11-19 13:16:57,608 INFO 1947952:repeater task quick_os150nb ended after 4.4 seconds 2023-11-19 13:16:57,609 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:16:18 2023-11-19 13:16:57,609 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:21:23 2023-11-19 13:16:57,609 INFO 1947952:repeater task lastfew_vec_os150nb starting 34.6 seconds late 2023-11-19 13:16:58,212 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:16:58,213 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:16:18 2023-11-19 13:16:58,213 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:21:23 2023-11-19 13:16:58,213 INFO 1947952:repeater task 3dayplots_os150nb starting 34.2 seconds late 2023-11-19 13:17:01,607 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 13:17:01,607 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:21:24 2023-11-19 13:17:01,608 INFO 1947952:repeater task lastensq_os38nb starting 36.6 seconds late 2023-11-19 13:17:03,642 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 13:17:03,643 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:16:20 2023-11-19 13:17:03,643 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:21:25 2023-11-19 13:17:03,643 INFO 1947952:repeater task quick_os38nb starting 38.6 seconds late 2023-11-19 13:17:08,752 INFO 1947952:repeater task quick_os38nb ended after 5.1 seconds 2023-11-19 13:17:08,752 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:16:20 2023-11-19 13:17:08,752 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:21:25 2023-11-19 13:17:08,753 INFO 1947952:repeater task lastfew_vec_os38nb starting 43.8 seconds late 2023-11-19 13:17:09,367 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:17:09,368 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:16:20 2023-11-19 13:17:09,368 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:21:25 2023-11-19 13:17:09,368 INFO 1947952:repeater task 3dayplots_os38nb starting 45.3 seconds late 2023-11-19 13:17:12,815 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:17:12,816 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:21:24 2023-11-19 13:17:12,817 INFO 1947952:repeater task daily_netcdf starting 48.8 seconds late 2023-11-19 13:17:13,665 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 13:17:13,665 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:18:28,847 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-19 13:18:31,703 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 13:18:31,703 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:18:19 2023-11-19 13:18:31,704 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:20:24 2023-11-19 13:18:31,704 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-19 13:18:38,967 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 13:18:38,968 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:18:19 2023-11-19 13:18:38,968 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:20:24 2023-11-19 13:18:38,968 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.0 seconds late 2023-11-19 13:18:39,605 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:18:39,606 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:18:19 2023-11-19 13:18:39,606 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:20:24 2023-11-19 13:20:24,879 INFO 1947952:repeater task lastensq_wh300 starting 0.9 seconds late 2023-11-19 13:20:28,236 INFO 1947952:repeater task lastensq_wh300 ended after 3.4 seconds 2023-11-19 13:20:28,237 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:20:19 2023-11-19 13:20:28,237 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:22:24 2023-11-19 13:20:28,237 INFO 1947952:repeater task quick_wh300 starting 4.2 seconds late 2023-11-19 13:20:35,787 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 13:20:35,787 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:20:19 2023-11-19 13:20:35,787 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:22:24 2023-11-19 13:20:35,788 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-19 13:20:36,423 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:20:36,423 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:20:19 2023-11-19 13:20:36,423 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:22:24 2023-11-19 13:21:21,545 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.5 seconds late 2023-11-19 13:21:22,779 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 13:21:22,780 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:22,780 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.7 seconds late 2023-11-19 13:21:22,946 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 13:21:22,946 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:22,947 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.9 seconds late 2023-11-19 13:21:24,291 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 13:21:24,291 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:24,291 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.2 seconds late 2023-11-19 13:21:25,594 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 13:21:25,594 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:25,594 INFO 1947952:repeater task lastensq_os150nb starting 2.6 seconds late 2023-11-19 13:21:28,121 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-19 13:21:28,121 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:21:18 2023-11-19 13:21:28,121 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:26:23 2023-11-19 13:21:28,122 INFO 1947952:repeater task quick_os150nb starting 5.1 seconds late 2023-11-19 13:21:32,552 INFO 1947952:repeater task quick_os150nb ended after 4.4 seconds 2023-11-19 13:21:32,552 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:21:18 2023-11-19 13:21:32,552 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:26:23 2023-11-19 13:21:32,553 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.6 seconds late 2023-11-19 13:21:33,154 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:21:33,154 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:21:18 2023-11-19 13:21:33,154 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:26:23 2023-11-19 13:21:33,155 INFO 1947952:repeater task 3dayplots_os150nb starting 9.1 seconds late 2023-11-19 13:21:36,622 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-19 13:21:36,622 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:21:36,623 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.6 seconds late 2023-11-19 13:21:37,623 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 13:21:37,624 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:37,624 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.6 seconds late 2023-11-19 13:21:37,791 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 13:21:37,791 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:37,792 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.7 seconds late 2023-11-19 13:21:39,040 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 13:21:39,041 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:39,041 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.0 seconds late 2023-11-19 13:21:40,292 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 13:21:40,292 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:40,293 INFO 1947952:repeater task lastensq_os38nb starting 15.3 seconds late 2023-11-19 13:21:42,278 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 13:21:42,278 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:21:20 2023-11-19 13:21:42,278 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:26:25 2023-11-19 13:21:42,279 INFO 1947952:repeater task quick_os38nb starting 17.3 seconds late 2023-11-19 13:21:47,061 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 13:21:47,061 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:21:20 2023-11-19 13:21:47,061 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:26:25 2023-11-19 13:21:47,062 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.1 seconds late 2023-11-19 13:21:47,677 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:21:47,677 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:21:20 2023-11-19 13:21:47,677 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:26:25 2023-11-19 13:21:47,678 INFO 1947952:repeater task 3dayplots_os38nb starting 23.6 seconds late 2023-11-19 13:21:51,059 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:21:51,059 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:21:51,060 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.0 seconds late 2023-11-19 13:21:52,091 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 13:21:52,091 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:52,091 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.0 seconds late 2023-11-19 13:21:52,258 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 13:21:52,259 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:52,260 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.2 seconds late 2023-11-19 13:21:53,501 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 13:21:53,502 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:53,502 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.5 seconds late 2023-11-19 13:21:54,737 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 13:21:54,737 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:54,739 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 35.5 seconds late 2023-11-19 13:21:56,073 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 13:21:56,073 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:56,074 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 36.8 seconds late 2023-11-19 13:21:57,338 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 13:21:57,338 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:57,339 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 38.1 seconds late 2023-11-19 13:21:58,671 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 13:21:58,672 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:21:58,672 INFO 1947952:repeater task plot_hbins starting 39.4 seconds late 2023-11-19 13:22:02,939 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 13:22:02,939 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 13:22:02,939 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 13:36:19 2023-11-19 13:22:07,947 INFO 1947952:repeater task 3dayplots_wh300 starting 43.9 seconds late 2023-11-19 13:22:11,815 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:22:11,815 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:22:26,848 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-19 13:22:29,784 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 13:22:29,784 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:22:19 2023-11-19 13:22:29,785 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:24:24 2023-11-19 13:22:29,785 INFO 1947952:repeater task quick_wh300 starting 5.8 seconds late 2023-11-19 13:22:37,061 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 13:22:37,061 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:22:19 2023-11-19 13:22:37,061 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:24:24 2023-11-19 13:22:37,062 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2023-11-19 13:22:37,701 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:22:37,701 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:22:19 2023-11-19 13:22:37,701 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:24:24 2023-11-19 13:24:27,983 INFO 1947952:repeater task lastensq_wh300 starting 4.0 seconds late 2023-11-19 13:24:30,663 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 13:24:30,664 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:24:19 2023-11-19 13:24:30,664 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:24:30,664 INFO 1947952:repeater task quick_wh300 starting 6.7 seconds late 2023-11-19 13:24:38,249 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 13:24:38,249 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:24:19 2023-11-19 13:24:38,249 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:24:38,250 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-19 13:24:38,880 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:24:38,880 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:24:19 2023-11-19 13:24:38,880 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:26:24 2023-11-19 13:26:24,141 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.1 seconds late 2023-11-19 13:26:27,043 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 13:26:27,043 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:26:27,044 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.0 seconds late 2023-11-19 13:26:29,919 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-19 13:26:29,919 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:26:29,920 INFO 1947952:repeater task noisy_rawtemps_os38 starting 5.9 seconds late 2023-11-19 13:26:32,932 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 13:26:32,932 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:26:32,933 INFO 1947952:repeater task lastensq_wh300 starting 8.9 seconds late 2023-11-19 13:26:35,846 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 13:26:35,847 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:26:19 2023-11-19 13:26:35,847 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:28:24 2023-11-19 13:26:35,847 INFO 1947952:repeater task quick_wh300 starting 11.8 seconds late 2023-11-19 13:26:43,435 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 13:26:43,436 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:26:19 2023-11-19 13:26:43,436 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:28:24 2023-11-19 13:26:43,436 INFO 1947952:repeater task lastfew_vec_wh300 starting 19.4 seconds late 2023-11-19 13:26:44,068 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:26:44,069 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:26:19 2023-11-19 13:26:44,069 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:28:24 2023-11-19 13:26:44,069 INFO 1947952:repeater task 3dayplots_wh300 starting 20.0 seconds late 2023-11-19 13:26:47,958 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:26:47,959 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:31:24 2023-11-19 13:26:47,960 INFO 1947952:repeater task lastensq_os150nb starting 25.0 seconds late 2023-11-19 13:26:50,239 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 13:26:50,240 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:26:18 2023-11-19 13:26:50,240 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:31:23 2023-11-19 13:26:50,240 INFO 1947952:repeater task quick_os150nb starting 27.2 seconds late 2023-11-19 13:26:54,881 INFO 1947952:repeater task quick_os150nb ended after 4.6 seconds 2023-11-19 13:26:54,882 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:26:18 2023-11-19 13:26:54,882 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:31:23 2023-11-19 13:26:54,883 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.9 seconds late 2023-11-19 13:26:55,486 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:26:55,487 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:26:18 2023-11-19 13:26:55,487 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:31:23 2023-11-19 13:26:55,487 INFO 1947952:repeater task 3dayplots_os150nb starting 31.4 seconds late 2023-11-19 13:26:58,900 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 13:26:58,900 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:31:24 2023-11-19 13:26:58,901 INFO 1947952:repeater task lastensq_os38nb starting 33.9 seconds late 2023-11-19 13:27:01,046 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 13:27:01,046 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:26:20 2023-11-19 13:27:01,046 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:31:25 2023-11-19 13:27:01,047 INFO 1947952:repeater task quick_os38nb starting 36.0 seconds late 2023-11-19 13:27:06,158 INFO 1947952:repeater task quick_os38nb ended after 5.1 seconds 2023-11-19 13:27:06,158 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:26:20 2023-11-19 13:27:06,158 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:31:25 2023-11-19 13:27:06,159 INFO 1947952:repeater task lastfew_vec_os38nb starting 41.2 seconds late 2023-11-19 13:27:06,774 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:27:06,774 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:26:20 2023-11-19 13:27:06,774 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:31:25 2023-11-19 13:27:06,775 INFO 1947952:repeater task 3dayplots_os38nb starting 42.7 seconds late 2023-11-19 13:27:10,184 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:27:10,185 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:31:24 2023-11-19 13:27:10,186 INFO 1947952:repeater task daily_netcdf starting 46.1 seconds late 2023-11-19 13:27:11,054 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 13:27:11,054 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:28:26,236 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-19 13:28:29,723 INFO 1947952:repeater task lastensq_wh300 ended after 3.5 seconds 2023-11-19 13:28:29,724 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:28:19 2023-11-19 13:28:29,724 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:30:24 2023-11-19 13:28:29,724 INFO 1947952:repeater task quick_wh300 starting 5.7 seconds late 2023-11-19 13:28:36,956 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 13:28:36,956 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:28:19 2023-11-19 13:28:36,956 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:30:24 2023-11-19 13:28:36,957 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.0 seconds late 2023-11-19 13:28:37,591 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:28:37,592 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:28:19 2023-11-19 13:28:37,592 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:30:24 2023-11-19 13:30:27,891 INFO 1947952:repeater task lastensq_wh300 starting 3.9 seconds late 2023-11-19 13:30:31,029 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 13:30:31,029 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:30:19 2023-11-19 13:30:31,029 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:32:24 2023-11-19 13:30:31,029 INFO 1947952:repeater task quick_wh300 starting 7.0 seconds late 2023-11-19 13:30:38,830 INFO 1947952:repeater task quick_wh300 ended after 7.8 seconds 2023-11-19 13:30:38,830 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:30:19 2023-11-19 13:30:38,830 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:32:24 2023-11-19 13:30:38,830 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-19 13:30:39,457 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:30:39,458 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:30:19 2023-11-19 13:30:39,458 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:32:24 2023-11-19 13:31:24,567 INFO 1947952:repeater task 3dayplots_wh300 starting 0.5 seconds late 2023-11-19 13:31:28,632 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.1 seconds 2023-11-19 13:31:28,632 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:31:28,633 INFO 1947952:repeater task lastensq_os150nb starting 5.6 seconds late 2023-11-19 13:31:30,882 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 13:31:30,883 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:31:18 2023-11-19 13:31:30,883 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:36:23 2023-11-19 13:31:30,883 INFO 1947952:repeater task quick_os150nb starting 7.9 seconds late 2023-11-19 13:31:35,213 INFO 1947952:repeater task quick_os150nb ended after 4.3 seconds 2023-11-19 13:31:35,213 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:31:18 2023-11-19 13:31:35,213 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:36:23 2023-11-19 13:31:35,214 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.2 seconds late 2023-11-19 13:31:35,816 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:31:35,817 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:31:18 2023-11-19 13:31:35,817 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:36:23 2023-11-19 13:31:35,817 INFO 1947952:repeater task 3dayplots_os150nb starting 11.8 seconds late 2023-11-19 13:31:39,123 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 13:31:39,123 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:31:39,124 INFO 1947952:repeater task lastensq_os38nb starting 14.1 seconds late 2023-11-19 13:31:41,219 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 13:31:41,220 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:31:20 2023-11-19 13:31:41,220 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:36:25 2023-11-19 13:31:41,220 INFO 1947952:repeater task quick_os38nb starting 16.2 seconds late 2023-11-19 13:31:46,214 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 13:31:46,215 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:31:20 2023-11-19 13:31:46,215 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:36:25 2023-11-19 13:31:46,215 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.2 seconds late 2023-11-19 13:31:46,821 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:31:46,821 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:31:20 2023-11-19 13:31:46,821 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:36:25 2023-11-19 13:31:46,822 INFO 1947952:repeater task 3dayplots_os38nb starting 22.8 seconds late 2023-11-19 13:31:50,245 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:31:50,246 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:32:25,341 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-19 13:32:28,192 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 13:32:28,193 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:32:19 2023-11-19 13:32:28,193 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:34:24 2023-11-19 13:32:28,193 INFO 1947952:repeater task quick_wh300 starting 4.2 seconds late 2023-11-19 13:32:35,426 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 13:32:35,426 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:32:19 2023-11-19 13:32:35,427 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:34:24 2023-11-19 13:32:35,427 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-19 13:32:36,063 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:32:36,064 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:32:19 2023-11-19 13:32:36,064 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:34:24 2023-11-19 13:34:26,344 INFO 1947952:repeater task lastensq_wh300 starting 2.3 seconds late 2023-11-19 13:34:29,479 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 13:34:29,479 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:34:19 2023-11-19 13:34:29,479 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:34:29,480 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-19 13:34:37,151 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 13:34:37,151 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:34:19 2023-11-19 13:34:37,151 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:34:37,152 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2023-11-19 13:34:37,774 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:34:37,775 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:34:19 2023-11-19 13:34:37,775 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:36:24 2023-11-19 13:36:23,057 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.0 seconds late 2023-11-19 13:36:24,293 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 13:36:24,293 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:24,293 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.2 seconds late 2023-11-19 13:36:24,462 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 13:36:24,462 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:24,462 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.4 seconds late 2023-11-19 13:36:25,745 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 13:36:25,746 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:25,746 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.7 seconds late 2023-11-19 13:36:27,041 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 13:36:27,041 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:27,041 INFO 1947952:repeater task lastensq_os150nb starting 4.0 seconds late 2023-11-19 13:36:29,781 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-19 13:36:29,781 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:36:18 2023-11-19 13:36:29,781 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:41:23 2023-11-19 13:36:29,782 INFO 1947952:repeater task quick_os150nb starting 6.8 seconds late 2023-11-19 13:36:34,479 INFO 1947952:repeater task quick_os150nb ended after 4.7 seconds 2023-11-19 13:36:34,479 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:36:18 2023-11-19 13:36:34,479 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:41:23 2023-11-19 13:36:34,480 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.5 seconds late 2023-11-19 13:36:35,087 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:36:35,087 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:36:18 2023-11-19 13:36:35,088 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:41:23 2023-11-19 13:36:35,088 INFO 1947952:repeater task 3dayplots_os150nb starting 11.0 seconds late 2023-11-19 13:36:38,420 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 13:36:38,420 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:41:24 2023-11-19 13:36:38,421 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.4 seconds late 2023-11-19 13:36:39,423 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 13:36:39,424 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:39,424 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.4 seconds late 2023-11-19 13:36:39,591 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 13:36:39,592 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:39,592 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.5 seconds late 2023-11-19 13:36:40,860 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 13:36:40,860 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:40,861 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.8 seconds late 2023-11-19 13:36:42,134 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 13:36:42,134 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:42,134 INFO 1947952:repeater task lastensq_os38nb starting 17.1 seconds late 2023-11-19 13:36:44,510 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-19 13:36:44,510 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:36:20 2023-11-19 13:36:44,510 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:41:25 2023-11-19 13:36:44,510 INFO 1947952:repeater task quick_os38nb starting 19.5 seconds late 2023-11-19 13:36:49,411 INFO 1947952:repeater task quick_os38nb ended after 4.9 seconds 2023-11-19 13:36:49,412 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:36:20 2023-11-19 13:36:49,412 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:41:25 2023-11-19 13:36:49,412 INFO 1947952:repeater task lastfew_vec_os38nb starting 24.4 seconds late 2023-11-19 13:36:50,017 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:36:50,017 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:36:20 2023-11-19 13:36:50,018 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:41:25 2023-11-19 13:36:50,018 INFO 1947952:repeater task 3dayplots_os38nb starting 26.0 seconds late 2023-11-19 13:36:53,390 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:36:53,390 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:41:24 2023-11-19 13:36:53,390 INFO 1947952:repeater task plot_tsstats_os38nb starting 34.3 seconds late 2023-11-19 13:36:54,418 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 13:36:54,419 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:54,419 INFO 1947952:repeater task grab_lastpings_os38nb starting 35.4 seconds late 2023-11-19 13:36:54,588 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 13:36:54,588 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:54,589 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.5 seconds late 2023-11-19 13:36:55,842 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 13:36:55,842 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:55,843 INFO 1947952:repeater task beamdiagnostics_os38nb starting 36.8 seconds late 2023-11-19 13:36:57,085 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 13:36:57,086 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:36:57,086 INFO 1947952:repeater task daily_netcdf starting 33.0 seconds late 2023-11-19 13:36:57,922 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 13:36:57,923 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:36:57,923 INFO 1947952:repeater task ggatime_diagnostics starting 38.9 seconds late 2023-11-19 13:37:14,270 INFO 1947952:repeater task ggatime_diagnostics ended after 16.3 seconds 2023-11-19 13:37:14,270 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:37:14,271 INFO 1947952:repeater task ggatime_diagnostics starting 55.2 seconds late 2023-11-19 13:37:30,206 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 13:37:30,206 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:37:30,206 INFO 1947952:repeater task ggatime_diagnostics starting 71.2 seconds late 2023-11-19 13:37:46,206 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-19 13:37:46,207 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:37:46,207 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 86.9 seconds late 2023-11-19 13:37:47,552 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 13:37:47,552 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:37:47,553 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 88.3 seconds late 2023-11-19 13:37:48,812 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 13:37:48,812 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:37:48,812 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 89.5 seconds late 2023-11-19 13:37:50,098 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 13:37:50,099 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:37:50,099 INFO 1947952:repeater task plot_hbins starting 90.8 seconds late 2023-11-19 13:37:54,461 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 13:37:54,462 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-19 13:37:54,462 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 13:51:19 2023-11-19 13:37:54,462 INFO 1947952:repeater task report_generator starting 95.2 seconds late 2023-11-19 13:38:00,436 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-19 13:38:00,436 INFO 1947952:repeater task report_generator ended after 6.0 seconds 2023-11-19 13:38:00,438 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/19 14:36:19 2023-11-19 13:38:05,441 INFO 1947952:repeater task noisy_rawtemps_os150 starting 101.4 seconds late 2023-11-19 13:38:08,297 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 13:38:08,297 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:38:08,298 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 104.3 seconds late 2023-11-19 13:38:10,964 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 13:38:10,964 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:38:10,964 INFO 1947952:repeater task noisy_rawtemps_os38 starting 106.9 seconds late 2023-11-19 13:38:14,029 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 13:38:14,029 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:38:14,029 INFO 1947952:repeater task lastensq_wh300 starting 110.0 seconds late 2023-11-19 13:38:16,884 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 13:38:16,885 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:36:19 2023-11-19 13:38:16,885 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:38:24 2023-11-19 13:38:16,885 INFO 1947952:repeater task quick_wh300 starting 112.9 seconds late 2023-11-19 13:38:24,808 INFO 1947952:repeater task quick_wh300 ended after 7.9 seconds 2023-11-19 13:38:24,809 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:36:19 2023-11-19 13:38:24,809 INFO 1947952:repeater task quick_wh300: added 1 extra increments to time_to_run 2023-11-19 13:38:24,809 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:40:29 2023-11-19 13:38:24,809 INFO 1947952:repeater task lastfew_vec_wh300 starting 120.8 seconds late 2023-11-19 13:38:25,432 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:38:25,432 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:36:19 2023-11-19 13:38:25,432 INFO 1947952:repeater task lastfew_vec_wh300: added 1 extra increments to time_to_run 2023-11-19 13:38:25,432 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:40:29 2023-11-19 13:38:25,432 INFO 1947952:repeater task 3dayplots_wh300 starting 121.4 seconds late 2023-11-19 13:38:29,621 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.2 seconds 2023-11-19 13:38:29,621 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:41:24 2023-11-19 13:38:34,632 INFO 1947952:repeater task lastensq_wh300 starting 10.6 seconds late 2023-11-19 13:38:37,581 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 13:38:37,581 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:38:19 2023-11-19 13:38:37,581 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:40:24 2023-11-19 13:40:27,877 INFO 1947952:repeater task lastensq_wh300 starting 3.9 seconds late 2023-11-19 13:40:31,005 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 13:40:31,005 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:40:19 2023-11-19 13:40:31,005 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:42:24 2023-11-19 13:40:31,005 INFO 1947952:repeater task quick_wh300 starting 2.0 seconds late 2023-11-19 13:40:38,236 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 13:40:38,237 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:40:19 2023-11-19 13:40:38,237 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:42:24 2023-11-19 13:40:38,237 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2023-11-19 13:40:38,871 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:40:38,871 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:40:19 2023-11-19 13:40:38,872 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:42:24 2023-11-19 13:41:23,991 INFO 1947952:repeater task lastensq_os150nb starting 1.0 seconds late 2023-11-19 13:41:26,366 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-19 13:41:26,366 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:41:18 2023-11-19 13:41:26,366 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:46:23 2023-11-19 13:41:26,367 INFO 1947952:repeater task quick_os150nb starting 3.4 seconds late 2023-11-19 13:41:30,878 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 13:41:30,879 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:41:18 2023-11-19 13:41:30,879 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:46:23 2023-11-19 13:41:30,879 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.9 seconds late 2023-11-19 13:41:31,486 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:41:31,487 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:41:18 2023-11-19 13:41:31,487 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:46:23 2023-11-19 13:41:31,487 INFO 1947952:repeater task 3dayplots_os150nb starting 7.4 seconds late 2023-11-19 13:41:34,827 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 13:41:34,827 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:41:34,828 INFO 1947952:repeater task lastensq_os38nb starting 9.8 seconds late 2023-11-19 13:41:37,018 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 13:41:37,019 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:41:20 2023-11-19 13:41:37,019 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:46:25 2023-11-19 13:41:37,019 INFO 1947952:repeater task quick_os38nb starting 12.0 seconds late 2023-11-19 13:41:41,790 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 13:41:41,791 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:41:20 2023-11-19 13:41:41,791 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:46:25 2023-11-19 13:41:41,791 INFO 1947952:repeater task lastfew_vec_os38nb starting 16.8 seconds late 2023-11-19 13:41:42,403 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:41:42,403 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:41:20 2023-11-19 13:41:42,404 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:46:25 2023-11-19 13:41:42,404 INFO 1947952:repeater task 3dayplots_os38nb starting 18.4 seconds late 2023-11-19 13:41:45,813 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:41:45,814 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:41:50,824 INFO 1947952:repeater task 3dayplots_wh300 starting 26.8 seconds late 2023-11-19 13:41:54,669 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 13:41:54,670 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:42:24,752 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-19 13:42:27,381 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 13:42:27,381 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:42:19 2023-11-19 13:42:27,381 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:44:24 2023-11-19 13:42:27,381 INFO 1947952:repeater task quick_wh300 starting 3.4 seconds late 2023-11-19 13:42:34,606 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 13:42:34,606 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:42:19 2023-11-19 13:42:34,606 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:44:24 2023-11-19 13:42:34,606 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2023-11-19 13:42:35,239 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:42:35,239 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:42:19 2023-11-19 13:42:35,239 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:44:24 2023-11-19 13:44:25,527 INFO 1947952:repeater task lastensq_wh300 starting 1.5 seconds late 2023-11-19 13:44:28,281 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 13:44:28,282 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:44:19 2023-11-19 13:44:28,282 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:44:28,282 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-19 13:44:35,838 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 13:44:35,839 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:44:19 2023-11-19 13:44:35,839 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:44:35,839 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-19 13:44:36,471 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:44:36,471 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:44:19 2023-11-19 13:44:36,471 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:46:24 2023-11-19 13:46:26,757 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.7 seconds late 2023-11-19 13:46:29,803 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-19 13:46:29,803 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:46:29,804 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.8 seconds late 2023-11-19 13:46:32,573 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 13:46:32,573 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:46:32,574 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.5 seconds late 2023-11-19 13:46:35,635 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 13:46:35,636 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:46:35,636 INFO 1947952:repeater task lastensq_wh300 starting 11.6 seconds late 2023-11-19 13:46:38,075 INFO 1947952:repeater task lastensq_wh300 ended after 2.4 seconds 2023-11-19 13:46:38,075 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:46:19 2023-11-19 13:46:38,075 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:48:24 2023-11-19 13:46:38,076 INFO 1947952:repeater task quick_wh300 starting 14.1 seconds late 2023-11-19 13:46:45,100 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 13:46:45,101 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:46:19 2023-11-19 13:46:45,101 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:48:24 2023-11-19 13:46:45,101 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.1 seconds late 2023-11-19 13:46:45,731 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:46:45,731 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:46:19 2023-11-19 13:46:45,732 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:48:24 2023-11-19 13:46:45,732 INFO 1947952:repeater task 3dayplots_wh300 starting 21.7 seconds late 2023-11-19 13:46:49,557 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 13:46:49,558 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:51:24 2023-11-19 13:46:49,559 INFO 1947952:repeater task lastensq_os150nb starting 26.6 seconds late 2023-11-19 13:46:51,479 INFO 1947952:repeater task lastensq_os150nb ended after 1.9 seconds 2023-11-19 13:46:51,480 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:46:18 2023-11-19 13:46:51,480 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:51:23 2023-11-19 13:46:51,480 INFO 1947952:repeater task quick_os150nb starting 28.5 seconds late 2023-11-19 13:46:55,634 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 13:46:55,635 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:46:18 2023-11-19 13:46:55,635 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:51:23 2023-11-19 13:46:55,635 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.6 seconds late 2023-11-19 13:46:56,233 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:46:56,233 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:46:18 2023-11-19 13:46:56,233 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:51:23 2023-11-19 13:46:56,234 INFO 1947952:repeater task 3dayplots_os150nb starting 32.2 seconds late 2023-11-19 13:46:59,572 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 13:46:59,572 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:51:24 2023-11-19 13:46:59,574 INFO 1947952:repeater task lastensq_os38nb starting 34.6 seconds late 2023-11-19 13:47:01,541 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 13:47:01,541 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:46:20 2023-11-19 13:47:01,541 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:51:25 2023-11-19 13:47:01,542 INFO 1947952:repeater task quick_os38nb starting 36.5 seconds late 2023-11-19 13:47:06,255 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 13:47:06,256 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:46:20 2023-11-19 13:47:06,256 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:51:25 2023-11-19 13:47:06,256 INFO 1947952:repeater task lastfew_vec_os38nb starting 41.3 seconds late 2023-11-19 13:47:06,861 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:47:06,861 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:46:20 2023-11-19 13:47:06,862 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:51:25 2023-11-19 13:47:06,862 INFO 1947952:repeater task 3dayplots_os38nb starting 42.8 seconds late 2023-11-19 13:47:10,292 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:47:10,292 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:51:24 2023-11-19 13:47:10,293 INFO 1947952:repeater task daily_netcdf starting 46.2 seconds late 2023-11-19 13:47:11,169 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 13:47:11,170 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:48:26,369 INFO 1947952:repeater task lastensq_wh300 starting 2.4 seconds late 2023-11-19 13:48:29,293 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 13:48:29,294 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:48:19 2023-11-19 13:48:29,294 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:50:24 2023-11-19 13:48:29,294 INFO 1947952:repeater task quick_wh300 starting 5.3 seconds late 2023-11-19 13:48:36,749 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 13:48:36,749 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:48:19 2023-11-19 13:48:36,749 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:50:24 2023-11-19 13:48:36,749 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2023-11-19 13:48:37,379 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:48:37,380 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:48:19 2023-11-19 13:48:37,380 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:50:24 2023-11-19 13:50:27,664 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-19 13:50:30,763 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 13:50:30,763 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:50:19 2023-11-19 13:50:30,764 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:52:24 2023-11-19 13:50:30,764 INFO 1947952:repeater task quick_wh300 starting 6.8 seconds late 2023-11-19 13:50:37,919 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 13:50:37,919 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:50:19 2023-11-19 13:50:37,919 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:52:24 2023-11-19 13:50:37,920 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2023-11-19 13:50:38,547 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:50:38,548 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:50:19 2023-11-19 13:50:38,548 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:52:24 2023-11-19 13:51:23,663 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.6 seconds late 2023-11-19 13:51:24,898 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 13:51:24,898 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:24,898 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.8 seconds late 2023-11-19 13:51:25,087 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 13:51:25,087 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:25,088 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.0 seconds late 2023-11-19 13:51:26,385 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 13:51:26,385 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:26,386 INFO 1947952:repeater task beamdiagnostics_wh300 starting 7.3 seconds late 2023-11-19 13:51:27,668 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 13:51:27,669 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:27,669 INFO 1947952:repeater task lastensq_os150nb starting 4.7 seconds late 2023-11-19 13:51:29,721 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-19 13:51:29,722 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:51:18 2023-11-19 13:51:29,722 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 13:56:23 2023-11-19 13:51:29,722 INFO 1947952:repeater task quick_os150nb starting 6.7 seconds late 2023-11-19 13:51:34,298 INFO 1947952:repeater task quick_os150nb ended after 4.6 seconds 2023-11-19 13:51:34,299 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:51:18 2023-11-19 13:51:34,299 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 13:56:23 2023-11-19 13:51:34,299 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.3 seconds late 2023-11-19 13:51:34,905 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:51:34,906 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:51:18 2023-11-19 13:51:34,906 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 13:56:23 2023-11-19 13:51:34,906 INFO 1947952:repeater task 3dayplots_os150nb starting 10.9 seconds late 2023-11-19 13:51:38,331 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 13:51:38,332 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:51:38,332 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.3 seconds late 2023-11-19 13:51:39,331 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 13:51:39,331 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:39,332 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.3 seconds late 2023-11-19 13:51:39,500 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 13:51:39,500 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:39,500 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.4 seconds late 2023-11-19 13:51:40,753 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 13:51:40,753 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:40,754 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.7 seconds late 2023-11-19 13:51:42,008 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 13:51:42,008 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:42,008 INFO 1947952:repeater task lastensq_os38nb starting 17.0 seconds late 2023-11-19 13:51:44,078 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 13:51:44,079 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:51:20 2023-11-19 13:51:44,079 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 13:56:25 2023-11-19 13:51:44,079 INFO 1947952:repeater task quick_os38nb starting 19.1 seconds late 2023-11-19 13:51:48,509 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-19 13:51:48,510 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:51:20 2023-11-19 13:51:48,510 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 13:56:25 2023-11-19 13:51:48,510 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.5 seconds late 2023-11-19 13:51:49,120 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:51:49,120 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:51:20 2023-11-19 13:51:49,121 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 13:56:25 2023-11-19 13:51:49,121 INFO 1947952:repeater task 3dayplots_os38nb starting 25.1 seconds late 2023-11-19 13:51:52,565 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:51:52,565 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:51:52,566 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.5 seconds late 2023-11-19 13:51:53,591 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 13:51:53,591 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:53,592 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.5 seconds late 2023-11-19 13:51:53,760 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 13:51:53,760 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:53,760 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.7 seconds late 2023-11-19 13:51:55,030 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 13:51:55,030 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:55,031 INFO 1947952:repeater task beamdiagnostics_os38nb starting 36.0 seconds late 2023-11-19 13:51:56,210 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 13:51:56,210 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:56,211 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 36.9 seconds late 2023-11-19 13:51:57,507 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 13:51:57,507 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:57,507 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 38.2 seconds late 2023-11-19 13:51:58,784 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 13:51:58,784 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:51:58,785 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 39.5 seconds late 2023-11-19 13:52:00,057 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 13:52:00,057 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:52:00,057 INFO 1947952:repeater task plot_hbins starting 40.8 seconds late 2023-11-19 13:52:04,406 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 13:52:04,406 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 13:52:04,406 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 14:06:19 2023-11-19 13:52:09,413 INFO 1947952:repeater task 3dayplots_wh300 starting 45.4 seconds late 2023-11-19 13:52:13,318 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:52:13,318 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:52:28,354 INFO 1947952:repeater task lastensq_wh300 starting 4.4 seconds late 2023-11-19 13:52:31,309 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 13:52:31,309 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:52:19 2023-11-19 13:52:31,309 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:54:24 2023-11-19 13:52:31,310 INFO 1947952:repeater task quick_wh300 starting 7.3 seconds late 2023-11-19 13:52:38,958 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 13:52:38,959 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:52:19 2023-11-19 13:52:38,959 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:54:24 2023-11-19 13:52:38,959 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.0 seconds late 2023-11-19 13:52:39,597 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:52:39,598 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:52:19 2023-11-19 13:52:39,598 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:54:24 2023-11-19 13:54:24,850 INFO 1947952:repeater task lastensq_wh300 starting 0.9 seconds late 2023-11-19 13:54:27,568 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 13:54:27,569 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:54:19 2023-11-19 13:54:27,569 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:54:27,569 INFO 1947952:repeater task quick_wh300 starting 3.6 seconds late 2023-11-19 13:54:35,050 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 13:54:35,050 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:54:19 2023-11-19 13:54:35,052 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:54:35,052 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-19 13:54:35,690 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:54:35,690 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:54:19 2023-11-19 13:54:35,690 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:56:24 2023-11-19 13:56:25,957 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.9 seconds late 2023-11-19 13:56:28,832 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 13:56:28,833 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 14:06:24 2023-11-19 13:56:28,833 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 4.8 seconds late 2023-11-19 13:56:31,756 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-19 13:56:31,756 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 14:06:24 2023-11-19 13:56:31,757 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.7 seconds late 2023-11-19 13:56:34,784 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 13:56:34,785 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 14:06:24 2023-11-19 13:56:34,785 INFO 1947952:repeater task lastensq_wh300 starting 10.8 seconds late 2023-11-19 13:56:37,364 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 13:56:37,365 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:56:19 2023-11-19 13:56:37,365 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 13:58:24 2023-11-19 13:56:37,365 INFO 1947952:repeater task quick_wh300 starting 13.4 seconds late 2023-11-19 13:56:44,212 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-19 13:56:44,212 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:56:19 2023-11-19 13:56:44,212 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 13:58:24 2023-11-19 13:56:44,213 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.2 seconds late 2023-11-19 13:56:44,851 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:56:44,852 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:56:19 2023-11-19 13:56:44,852 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 13:58:24 2023-11-19 13:56:44,852 INFO 1947952:repeater task 3dayplots_wh300 starting 20.8 seconds late 2023-11-19 13:56:48,739 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 13:56:48,740 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:01:24 2023-11-19 13:56:48,741 INFO 1947952:repeater task lastensq_os150nb starting 25.7 seconds late 2023-11-19 13:56:50,891 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-19 13:56:50,891 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:56:18 2023-11-19 13:56:50,891 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:01:23 2023-11-19 13:56:50,892 INFO 1947952:repeater task quick_os150nb starting 27.9 seconds late 2023-11-19 13:56:54,813 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-19 13:56:54,813 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:56:18 2023-11-19 13:56:54,813 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:01:23 2023-11-19 13:56:54,814 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.8 seconds late 2023-11-19 13:56:55,404 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 13:56:55,405 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 13:56:18 2023-11-19 13:56:55,405 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:01:23 2023-11-19 13:56:55,405 INFO 1947952:repeater task 3dayplots_os150nb starting 31.4 seconds late 2023-11-19 13:56:58,814 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 13:56:58,814 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:01:24 2023-11-19 13:56:58,815 INFO 1947952:repeater task lastensq_os38nb starting 33.8 seconds late 2023-11-19 13:57:00,883 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 13:57:00,883 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:56:20 2023-11-19 13:57:00,883 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:01:25 2023-11-19 13:57:00,884 INFO 1947952:repeater task quick_os38nb starting 35.9 seconds late 2023-11-19 13:57:05,855 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 13:57:05,855 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:56:20 2023-11-19 13:57:05,855 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:01:25 2023-11-19 13:57:05,855 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.9 seconds late 2023-11-19 13:57:06,464 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 13:57:06,464 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 13:56:20 2023-11-19 13:57:06,464 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:01:25 2023-11-19 13:57:06,465 INFO 1947952:repeater task 3dayplots_os38nb starting 42.4 seconds late 2023-11-19 13:57:09,884 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 13:57:09,884 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:01:24 2023-11-19 13:57:09,886 INFO 1947952:repeater task daily_netcdf starting 45.8 seconds late 2023-11-19 13:57:10,700 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 13:57:10,700 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 14:06:24 2023-11-19 13:58:25,920 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-19 13:58:28,674 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 13:58:28,675 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:58:19 2023-11-19 13:58:28,675 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:00:24 2023-11-19 13:58:28,675 INFO 1947952:repeater task quick_wh300 starting 4.7 seconds late 2023-11-19 13:58:36,148 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 13:58:36,149 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:58:19 2023-11-19 13:58:36,149 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:00:24 2023-11-19 13:58:36,149 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2023-11-19 13:58:36,796 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 13:58:36,796 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 13:58:19 2023-11-19 13:58:36,796 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:00:24 2023-11-19 14:00:27,087 INFO 1947952:repeater task lastensq_wh300 starting 3.1 seconds late 2023-11-19 14:00:30,016 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 14:00:30,017 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:00:19 2023-11-19 14:00:30,017 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:02:24 2023-11-19 14:00:30,017 INFO 1947952:repeater task quick_wh300 starting 6.0 seconds late 2023-11-19 14:00:37,744 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 14:00:37,745 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:00:19 2023-11-19 14:00:37,745 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:02:24 2023-11-19 14:00:37,745 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-19 14:00:38,378 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:00:38,378 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:00:19 2023-11-19 14:00:38,378 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:02:24 2023-11-19 14:01:23,498 INFO 1947952:repeater task lastensq_os150nb starting 0.5 seconds late 2023-11-19 14:01:25,760 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 14:01:25,761 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:01:18 2023-11-19 14:01:25,761 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:06:23 2023-11-19 14:01:25,761 INFO 1947952:repeater task quick_os150nb starting 2.8 seconds late 2023-11-19 14:01:30,634 INFO 1947952:repeater task quick_os150nb ended after 4.9 seconds 2023-11-19 14:01:30,635 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:01:18 2023-11-19 14:01:30,635 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:06:23 2023-11-19 14:01:30,635 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.6 seconds late 2023-11-19 14:01:31,235 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:01:31,235 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:01:18 2023-11-19 14:01:31,235 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:06:23 2023-11-19 14:01:31,236 INFO 1947952:repeater task 3dayplots_os150nb starting 7.2 seconds late 2023-11-19 14:01:34,580 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-19 14:01:34,580 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:06:24 2023-11-19 14:01:34,581 INFO 1947952:repeater task lastensq_os38nb starting 9.6 seconds late 2023-11-19 14:01:36,691 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:01:36,691 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:01:20 2023-11-19 14:01:36,692 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:06:25 2023-11-19 14:01:36,692 INFO 1947952:repeater task quick_os38nb starting 11.7 seconds late 2023-11-19 14:01:41,768 INFO 1947952:repeater task quick_os38nb ended after 5.1 seconds 2023-11-19 14:01:41,768 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:01:20 2023-11-19 14:01:41,768 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:06:25 2023-11-19 14:01:41,769 INFO 1947952:repeater task lastfew_vec_os38nb starting 16.8 seconds late 2023-11-19 14:01:42,388 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:01:42,388 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:01:20 2023-11-19 14:01:42,388 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:06:25 2023-11-19 14:01:42,388 INFO 1947952:repeater task 3dayplots_os38nb starting 18.3 seconds late 2023-11-19 14:01:45,806 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 14:01:45,806 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:06:24 2023-11-19 14:01:50,815 INFO 1947952:repeater task 3dayplots_wh300 starting 26.8 seconds late 2023-11-19 14:01:54,699 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:01:54,700 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:06:24 2023-11-19 14:02:24,778 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-19 14:02:27,920 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 14:02:27,920 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:02:19 2023-11-19 14:02:27,921 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:04:24 2023-11-19 14:02:27,921 INFO 1947952:repeater task quick_wh300 starting 3.9 seconds late 2023-11-19 14:02:35,872 INFO 1947952:repeater task quick_wh300 ended after 8.0 seconds 2023-11-19 14:02:35,872 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:02:19 2023-11-19 14:02:35,872 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:04:24 2023-11-19 14:02:35,873 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-19 14:02:36,512 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:02:36,513 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:02:19 2023-11-19 14:02:36,513 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:04:24 2023-11-19 14:04:26,848 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-19 14:04:29,570 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:04:29,571 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:04:19 2023-11-19 14:04:29,571 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:06:24 2023-11-19 14:04:29,571 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-19 14:04:37,501 INFO 1947952:repeater task quick_wh300 ended after 7.9 seconds 2023-11-19 14:04:37,501 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:04:19 2023-11-19 14:04:37,502 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:06:24 2023-11-19 14:04:37,502 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.5 seconds late 2023-11-19 14:04:38,145 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:04:38,145 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:04:19 2023-11-19 14:04:38,145 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:06:24 2023-11-19 14:06:23,429 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.4 seconds late 2023-11-19 14:06:24,668 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 14:06:24,668 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:24,668 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.6 seconds late 2023-11-19 14:06:24,838 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 14:06:24,838 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:24,838 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.8 seconds late 2023-11-19 14:06:26,150 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 14:06:26,150 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:26,150 INFO 1947952:repeater task beamdiagnostics_wh300 starting 7.1 seconds late 2023-11-19 14:06:27,423 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 14:06:27,423 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:27,423 INFO 1947952:repeater task lastensq_os150nb starting 4.4 seconds late 2023-11-19 14:06:29,578 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 14:06:29,579 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:06:18 2023-11-19 14:06:29,579 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:11:23 2023-11-19 14:06:29,579 INFO 1947952:repeater task quick_os150nb starting 6.6 seconds late 2023-11-19 14:06:34,099 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 14:06:34,099 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:06:18 2023-11-19 14:06:34,099 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:11:23 2023-11-19 14:06:34,099 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.1 seconds late 2023-11-19 14:06:34,696 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:06:34,696 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:06:18 2023-11-19 14:06:34,696 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:11:23 2023-11-19 14:06:34,697 INFO 1947952:repeater task 3dayplots_os150nb starting 10.6 seconds late 2023-11-19 14:06:38,100 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 14:06:38,100 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:11:24 2023-11-19 14:06:38,100 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.0 seconds late 2023-11-19 14:06:39,104 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 14:06:39,104 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:39,104 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.1 seconds late 2023-11-19 14:06:39,272 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 14:06:39,272 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:39,272 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.2 seconds late 2023-11-19 14:06:40,574 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 14:06:40,575 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:40,575 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.5 seconds late 2023-11-19 14:06:41,866 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 14:06:41,866 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:41,866 INFO 1947952:repeater task lastensq_os38nb starting 16.9 seconds late 2023-11-19 14:06:43,830 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 14:06:43,830 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:06:20 2023-11-19 14:06:43,831 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:11:25 2023-11-19 14:06:43,831 INFO 1947952:repeater task quick_os38nb starting 18.8 seconds late 2023-11-19 14:06:48,209 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-19 14:06:48,209 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:06:20 2023-11-19 14:06:48,210 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:11:25 2023-11-19 14:06:48,210 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.2 seconds late 2023-11-19 14:06:48,820 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:06:48,821 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:06:20 2023-11-19 14:06:48,821 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:11:25 2023-11-19 14:06:48,821 INFO 1947952:repeater task 3dayplots_os38nb starting 24.8 seconds late 2023-11-19 14:06:52,136 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.3 seconds 2023-11-19 14:06:52,136 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:11:24 2023-11-19 14:06:52,137 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.1 seconds late 2023-11-19 14:06:53,153 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 14:06:53,153 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:53,153 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.1 seconds late 2023-11-19 14:06:53,319 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 14:06:53,319 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:53,320 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.3 seconds late 2023-11-19 14:06:54,583 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 14:06:54,584 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:54,584 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.5 seconds late 2023-11-19 14:06:55,832 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 14:06:55,832 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:06:55,832 INFO 1947952:repeater task daily_netcdf starting 31.8 seconds late 2023-11-19 14:06:56,698 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 14:06:56,699 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:06:56,699 INFO 1947952:repeater task ggatime_diagnostics starting 37.6 seconds late 2023-11-19 14:07:12,886 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-19 14:07:12,886 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:07:12,887 INFO 1947952:repeater task ggatime_diagnostics starting 53.8 seconds late 2023-11-19 14:07:28,824 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-19 14:07:28,825 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:07:28,825 INFO 1947952:repeater task ggatime_diagnostics starting 69.8 seconds late 2023-11-19 14:07:45,021 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-19 14:07:45,022 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:07:45,022 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 85.7 seconds late 2023-11-19 14:07:46,334 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 14:07:46,334 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:07:46,335 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 87.1 seconds late 2023-11-19 14:07:47,606 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 14:07:47,606 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:07:47,607 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 88.3 seconds late 2023-11-19 14:07:48,839 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-19 14:07:48,839 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:07:48,840 INFO 1947952:repeater task plot_hbins starting 89.6 seconds late 2023-11-19 14:07:53,067 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 14:07:53,068 INFO 1947952:repeater task plot_hbins ended after 4.2 seconds 2023-11-19 14:07:53,068 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 14:21:19 2023-11-19 14:07:58,073 INFO 1947952:repeater task noisy_rawtemps_os150 starting 94.0 seconds late 2023-11-19 14:08:00,963 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 14:08:00,964 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:08:00,964 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 96.9 seconds late 2023-11-19 14:08:03,733 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 14:08:03,733 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:08:03,734 INFO 1947952:repeater task noisy_rawtemps_os38 starting 99.7 seconds late 2023-11-19 14:08:06,779 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-19 14:08:06,779 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:08:06,779 INFO 1947952:repeater task lastensq_wh300 starting 102.8 seconds late 2023-11-19 14:08:09,819 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 14:08:09,820 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:06:19 2023-11-19 14:08:09,820 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:08:24 2023-11-19 14:08:09,820 INFO 1947952:repeater task quick_wh300 starting 105.8 seconds late 2023-11-19 14:08:17,227 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 14:08:17,228 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:06:19 2023-11-19 14:08:17,228 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:08:24 2023-11-19 14:08:17,228 INFO 1947952:repeater task lastfew_vec_wh300 starting 113.2 seconds late 2023-11-19 14:08:17,860 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:08:17,860 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:06:19 2023-11-19 14:08:17,860 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:08:24 2023-11-19 14:08:17,861 INFO 1947952:repeater task 3dayplots_wh300 starting 113.8 seconds late 2023-11-19 14:08:21,690 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-19 14:08:21,690 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:11:24 2023-11-19 14:08:26,698 INFO 1947952:repeater task lastensq_wh300 starting 2.7 seconds late 2023-11-19 14:08:29,525 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 14:08:29,525 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:08:19 2023-11-19 14:08:29,526 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:10:24 2023-11-19 14:08:29,526 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-19 14:08:37,439 INFO 1947952:repeater task quick_wh300 ended after 7.9 seconds 2023-11-19 14:08:37,439 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:08:19 2023-11-19 14:08:37,439 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:10:24 2023-11-19 14:08:37,440 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2023-11-19 14:08:38,067 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:08:38,068 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:08:19 2023-11-19 14:08:38,068 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:10:24 2023-11-19 14:10:28,363 INFO 1947952:repeater task lastensq_wh300 starting 4.4 seconds late 2023-11-19 14:10:31,206 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 14:10:31,206 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:10:19 2023-11-19 14:10:31,206 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:12:24 2023-11-19 14:10:31,207 INFO 1947952:repeater task quick_wh300 starting 7.2 seconds late 2023-11-19 14:10:39,398 INFO 1947952:repeater task quick_wh300 ended after 8.2 seconds 2023-11-19 14:10:39,398 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:10:19 2023-11-19 14:10:39,398 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:12:24 2023-11-19 14:10:39,399 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.4 seconds late 2023-11-19 14:10:40,021 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:10:40,022 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:10:19 2023-11-19 14:10:40,022 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:12:24 2023-11-19 14:11:25,132 INFO 1947952:repeater task 3dayplots_wh300 starting 1.1 seconds late 2023-11-19 14:11:29,032 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:11:29,033 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:11:29,034 INFO 1947952:repeater task lastensq_os150nb starting 6.0 seconds late 2023-11-19 14:11:31,583 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-19 14:11:31,584 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:11:18 2023-11-19 14:11:31,584 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:16:23 2023-11-19 14:11:31,584 INFO 1947952:repeater task quick_os150nb starting 8.6 seconds late 2023-11-19 14:11:36,324 INFO 1947952:repeater task quick_os150nb ended after 4.7 seconds 2023-11-19 14:11:36,325 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:11:18 2023-11-19 14:11:36,325 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:16:23 2023-11-19 14:11:36,325 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.3 seconds late 2023-11-19 14:11:36,934 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:11:36,934 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:11:18 2023-11-19 14:11:36,934 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:16:23 2023-11-19 14:11:36,935 INFO 1947952:repeater task 3dayplots_os150nb starting 12.9 seconds late 2023-11-19 14:11:40,342 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 14:11:40,342 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:11:40,343 INFO 1947952:repeater task lastensq_os38nb starting 15.3 seconds late 2023-11-19 14:11:42,387 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 14:11:42,398 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:11:20 2023-11-19 14:11:42,398 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:16:25 2023-11-19 14:11:42,399 INFO 1947952:repeater task quick_os38nb starting 17.4 seconds late 2023-11-19 14:11:47,528 INFO 1947952:repeater task quick_os38nb ended after 5.1 seconds 2023-11-19 14:11:47,529 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:11:20 2023-11-19 14:11:47,529 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:16:25 2023-11-19 14:11:47,529 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.5 seconds late 2023-11-19 14:11:48,139 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:11:48,140 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:11:20 2023-11-19 14:11:48,140 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:16:25 2023-11-19 14:11:48,140 INFO 1947952:repeater task 3dayplots_os38nb starting 24.1 seconds late 2023-11-19 14:11:51,578 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 14:11:51,578 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:12:26,658 INFO 1947952:repeater task lastensq_wh300 starting 2.7 seconds late 2023-11-19 14:12:29,757 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 14:12:29,757 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:12:19 2023-11-19 14:12:29,757 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:14:24 2023-11-19 14:12:29,758 INFO 1947952:repeater task quick_wh300 starting 5.8 seconds late 2023-11-19 14:12:37,352 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 14:12:37,352 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:12:19 2023-11-19 14:12:37,352 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:14:24 2023-11-19 14:12:37,353 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2023-11-19 14:12:37,977 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:12:37,978 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:12:19 2023-11-19 14:12:37,978 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:14:24 2023-11-19 14:14:28,260 INFO 1947952:repeater task lastensq_wh300 starting 4.3 seconds late 2023-11-19 14:14:31,341 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-19 14:14:31,341 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:14:19 2023-11-19 14:14:31,341 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:14:31,342 INFO 1947952:repeater task quick_wh300 starting 7.3 seconds late 2023-11-19 14:14:39,645 INFO 1947952:repeater task quick_wh300 ended after 8.3 seconds 2023-11-19 14:14:39,645 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:14:19 2023-11-19 14:14:39,645 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:14:39,645 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.6 seconds late 2023-11-19 14:14:40,272 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:14:40,272 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:14:19 2023-11-19 14:14:40,272 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:16:24 2023-11-19 14:16:25,545 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.5 seconds late 2023-11-19 14:16:28,430 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 14:16:28,430 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:16:28,431 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 4.4 seconds late 2023-11-19 14:16:31,216 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 14:16:31,217 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:16:31,217 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.2 seconds late 2023-11-19 14:16:34,308 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 14:16:34,308 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:16:34,309 INFO 1947952:repeater task lastensq_wh300 starting 10.3 seconds late 2023-11-19 14:16:37,329 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 14:16:37,330 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:16:19 2023-11-19 14:16:37,330 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:18:24 2023-11-19 14:16:37,330 INFO 1947952:repeater task quick_wh300 starting 13.3 seconds late 2023-11-19 14:16:45,480 INFO 1947952:repeater task quick_wh300 ended after 8.1 seconds 2023-11-19 14:16:45,480 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:16:19 2023-11-19 14:16:45,480 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:18:24 2023-11-19 14:16:45,481 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.5 seconds late 2023-11-19 14:16:46,123 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:16:46,124 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:16:19 2023-11-19 14:16:46,124 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:18:24 2023-11-19 14:16:46,124 INFO 1947952:repeater task 3dayplots_wh300 starting 22.1 seconds late 2023-11-19 14:16:50,074 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 14:16:50,075 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:21:24 2023-11-19 14:16:50,076 INFO 1947952:repeater task lastensq_os150nb starting 27.1 seconds late 2023-11-19 14:16:52,576 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-19 14:16:52,576 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:16:18 2023-11-19 14:16:52,576 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:21:23 2023-11-19 14:16:52,577 INFO 1947952:repeater task quick_os150nb starting 29.6 seconds late 2023-11-19 14:16:57,172 INFO 1947952:repeater task quick_os150nb ended after 4.6 seconds 2023-11-19 14:16:57,172 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:16:18 2023-11-19 14:16:57,172 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:21:23 2023-11-19 14:16:57,173 INFO 1947952:repeater task lastfew_vec_os150nb starting 34.2 seconds late 2023-11-19 14:16:57,789 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:16:57,790 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:16:18 2023-11-19 14:16:57,790 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:21:23 2023-11-19 14:16:57,790 INFO 1947952:repeater task 3dayplots_os150nb starting 33.7 seconds late 2023-11-19 14:17:01,191 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 14:17:01,192 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:21:24 2023-11-19 14:17:01,193 INFO 1947952:repeater task lastensq_os38nb starting 36.2 seconds late 2023-11-19 14:17:03,200 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 14:17:03,200 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:16:20 2023-11-19 14:17:03,201 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:21:25 2023-11-19 14:17:03,201 INFO 1947952:repeater task quick_os38nb starting 38.2 seconds late 2023-11-19 14:17:08,196 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-19 14:17:08,196 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:16:20 2023-11-19 14:17:08,196 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:21:25 2023-11-19 14:17:08,197 INFO 1947952:repeater task lastfew_vec_os38nb starting 43.2 seconds late 2023-11-19 14:17:08,810 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:17:08,811 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:16:20 2023-11-19 14:17:08,811 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:21:25 2023-11-19 14:17:08,811 INFO 1947952:repeater task 3dayplots_os38nb starting 44.8 seconds late 2023-11-19 14:17:12,232 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 14:17:12,233 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:21:24 2023-11-19 14:17:12,234 INFO 1947952:repeater task daily_netcdf starting 48.2 seconds late 2023-11-19 14:17:13,155 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 14:17:13,155 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:18:28,348 INFO 1947952:repeater task lastensq_wh300 starting 4.3 seconds late 2023-11-19 14:18:31,743 INFO 1947952:repeater task lastensq_wh300 ended after 3.4 seconds 2023-11-19 14:18:31,744 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:18:19 2023-11-19 14:18:31,744 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:20:24 2023-11-19 14:18:31,744 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-19 14:18:39,338 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 14:18:39,338 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:18:19 2023-11-19 14:18:39,339 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:20:24 2023-11-19 14:18:39,339 INFO 1947952:repeater task lastfew_vec_wh300 starting 15.3 seconds late 2023-11-19 14:18:39,969 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:18:39,970 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:18:19 2023-11-19 14:18:39,970 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:20:24 2023-11-19 14:20:25,259 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-19 14:20:27,985 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:20:27,985 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:20:19 2023-11-19 14:20:27,985 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:22:24 2023-11-19 14:20:27,986 INFO 1947952:repeater task quick_wh300 starting 4.0 seconds late 2023-11-19 14:20:35,667 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 14:20:35,667 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:20:19 2023-11-19 14:20:35,668 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:22:24 2023-11-19 14:20:35,668 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-19 14:20:36,303 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:20:36,303 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:20:19 2023-11-19 14:20:36,303 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:22:24 2023-11-19 14:21:21,419 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.4 seconds late 2023-11-19 14:21:22,656 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 14:21:22,656 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:22,656 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.6 seconds late 2023-11-19 14:21:22,825 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 14:21:22,826 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:22,826 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.8 seconds late 2023-11-19 14:21:24,126 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 14:21:24,126 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:24,127 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.1 seconds late 2023-11-19 14:21:25,464 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 14:21:25,465 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:25,465 INFO 1947952:repeater task lastensq_os150nb starting 2.5 seconds late 2023-11-19 14:21:27,790 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 14:21:27,790 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:21:18 2023-11-19 14:21:27,790 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:26:23 2023-11-19 14:21:27,791 INFO 1947952:repeater task quick_os150nb starting 4.8 seconds late 2023-11-19 14:21:31,974 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 14:21:31,975 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:21:18 2023-11-19 14:21:31,975 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:26:23 2023-11-19 14:21:31,975 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.0 seconds late 2023-11-19 14:21:32,599 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:21:32,599 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:21:18 2023-11-19 14:21:32,599 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:26:23 2023-11-19 14:21:32,600 INFO 1947952:repeater task 3dayplots_os150nb starting 8.5 seconds late 2023-11-19 14:21:35,986 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 14:21:35,986 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:21:35,987 INFO 1947952:repeater task plot_tsstats_os150nb starting 16.9 seconds late 2023-11-19 14:21:36,970 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 14:21:36,970 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:36,971 INFO 1947952:repeater task grab_lastpings_os150nb starting 17.9 seconds late 2023-11-19 14:21:37,137 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 14:21:37,137 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:37,138 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.1 seconds late 2023-11-19 14:21:38,404 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 14:21:38,404 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:38,405 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.4 seconds late 2023-11-19 14:21:39,613 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 14:21:39,614 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:39,614 INFO 1947952:repeater task lastensq_os38nb starting 14.6 seconds late 2023-11-19 14:21:41,678 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:21:41,679 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:21:20 2023-11-19 14:21:41,679 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:26:25 2023-11-19 14:21:41,680 INFO 1947952:repeater task quick_os38nb starting 16.7 seconds late 2023-11-19 14:21:46,230 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-19 14:21:46,230 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:21:20 2023-11-19 14:21:46,230 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:26:25 2023-11-19 14:21:46,231 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.2 seconds late 2023-11-19 14:21:46,832 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:21:46,832 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:21:20 2023-11-19 14:21:46,833 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:26:25 2023-11-19 14:21:46,833 INFO 1947952:repeater task 3dayplots_os38nb starting 22.8 seconds late 2023-11-19 14:21:50,219 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-19 14:21:50,220 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:21:50,220 INFO 1947952:repeater task plot_tsstats_os38nb starting 31.2 seconds late 2023-11-19 14:21:51,236 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 14:21:51,237 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:51,237 INFO 1947952:repeater task grab_lastpings_os38nb starting 32.2 seconds late 2023-11-19 14:21:51,402 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 14:21:51,403 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:51,403 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.4 seconds late 2023-11-19 14:21:52,679 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 14:21:52,679 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:52,679 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.6 seconds late 2023-11-19 14:21:53,897 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 14:21:53,897 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:53,898 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 34.6 seconds late 2023-11-19 14:21:55,240 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 14:21:55,241 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:55,241 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 36.0 seconds late 2023-11-19 14:21:56,494 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 14:21:56,494 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:56,495 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 37.2 seconds late 2023-11-19 14:21:57,747 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 14:21:57,748 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:21:57,748 INFO 1947952:repeater task plot_hbins starting 38.5 seconds late 2023-11-19 14:22:02,030 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 14:22:02,030 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 14:22:02,030 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 14:36:19 2023-11-19 14:22:07,036 INFO 1947952:repeater task 3dayplots_wh300 starting 43.0 seconds late 2023-11-19 14:22:10,928 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:22:10,928 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:22:25,958 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-19 14:22:28,667 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:22:28,667 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:22:19 2023-11-19 14:22:28,667 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:24:24 2023-11-19 14:22:28,668 INFO 1947952:repeater task quick_wh300 starting 4.7 seconds late 2023-11-19 14:22:36,222 INFO 1947952:repeater task quick_wh300 ended after 7.6 seconds 2023-11-19 14:22:36,222 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:22:19 2023-11-19 14:22:36,222 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:24:24 2023-11-19 14:22:36,223 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2023-11-19 14:22:36,857 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:22:36,858 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:22:19 2023-11-19 14:22:36,858 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:24:24 2023-11-19 14:24:27,148 INFO 1947952:repeater task lastensq_wh300 starting 3.1 seconds late 2023-11-19 14:24:29,943 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 14:24:29,944 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:24:19 2023-11-19 14:24:29,944 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:24:29,944 INFO 1947952:repeater task quick_wh300 starting 5.9 seconds late 2023-11-19 14:24:37,321 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 14:24:37,322 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:24:19 2023-11-19 14:24:37,322 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:24:37,322 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.3 seconds late 2023-11-19 14:24:37,948 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:24:37,948 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:24:19 2023-11-19 14:24:37,948 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:26:24 2023-11-19 14:26:23,214 INFO 1947952:repeater task lastensq_os150nb starting 0.2 seconds late 2023-11-19 14:26:25,539 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 14:26:25,540 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:26:18 2023-11-19 14:26:25,540 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:31:23 2023-11-19 14:26:25,540 INFO 1947952:repeater task quick_os150nb starting 2.5 seconds late 2023-11-19 14:26:29,663 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 14:26:29,664 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:26:18 2023-11-19 14:26:29,664 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:31:23 2023-11-19 14:26:29,664 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.7 seconds late 2023-11-19 14:26:30,278 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:26:30,279 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:26:18 2023-11-19 14:26:30,279 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:31:23 2023-11-19 14:26:30,279 INFO 1947952:repeater task 3dayplots_os150nb starting 6.2 seconds late 2023-11-19 14:26:33,862 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.6 seconds 2023-11-19 14:26:33,862 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:31:24 2023-11-19 14:26:33,863 INFO 1947952:repeater task lastensq_os38nb starting 8.9 seconds late 2023-11-19 14:26:35,827 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-19 14:26:35,827 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:26:20 2023-11-19 14:26:35,827 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:31:25 2023-11-19 14:26:35,828 INFO 1947952:repeater task quick_os38nb starting 10.8 seconds late 2023-11-19 14:26:40,362 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-19 14:26:40,363 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:26:20 2023-11-19 14:26:40,363 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:31:25 2023-11-19 14:26:40,363 INFO 1947952:repeater task lastfew_vec_os38nb starting 15.4 seconds late 2023-11-19 14:26:40,968 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:26:40,969 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:26:20 2023-11-19 14:26:40,969 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:31:25 2023-11-19 14:26:40,969 INFO 1947952:repeater task 3dayplots_os38nb starting 16.9 seconds late 2023-11-19 14:26:44,431 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 14:26:44,431 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:31:24 2023-11-19 14:26:44,432 INFO 1947952:repeater task daily_netcdf starting 20.4 seconds late 2023-11-19 14:26:45,250 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 14:26:45,251 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:26:50,257 INFO 1947952:repeater task noisy_rawtemps_os150 starting 26.2 seconds late 2023-11-19 14:26:53,213 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-19 14:26:53,213 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:26:53,214 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 29.2 seconds late 2023-11-19 14:26:56,005 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 14:26:56,006 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:26:56,006 INFO 1947952:repeater task noisy_rawtemps_os38 starting 32.0 seconds late 2023-11-19 14:26:59,087 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 14:26:59,088 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:26:59,088 INFO 1947952:repeater task lastensq_wh300 starting 35.1 seconds late 2023-11-19 14:27:01,574 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-19 14:27:01,576 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:26:19 2023-11-19 14:27:01,576 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:28:24 2023-11-19 14:27:01,576 INFO 1947952:repeater task quick_wh300 starting 37.6 seconds late 2023-11-19 14:27:08,610 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-19 14:27:08,611 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:26:19 2023-11-19 14:27:08,611 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:28:24 2023-11-19 14:27:08,611 INFO 1947952:repeater task lastfew_vec_wh300 starting 44.6 seconds late 2023-11-19 14:27:09,250 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:27:09,250 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:26:19 2023-11-19 14:27:09,250 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:28:24 2023-11-19 14:27:09,251 INFO 1947952:repeater task 3dayplots_wh300 starting 45.2 seconds late 2023-11-19 14:27:13,168 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:27:13,168 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:31:24 2023-11-19 14:28:28,362 INFO 1947952:repeater task lastensq_wh300 starting 4.4 seconds late 2023-11-19 14:28:30,885 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-19 14:28:30,885 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:28:19 2023-11-19 14:28:30,885 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:30:24 2023-11-19 14:28:30,886 INFO 1947952:repeater task quick_wh300 starting 6.9 seconds late 2023-11-19 14:28:38,785 INFO 1947952:repeater task quick_wh300 ended after 7.9 seconds 2023-11-19 14:28:38,785 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:28:19 2023-11-19 14:28:38,785 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:30:24 2023-11-19 14:28:38,785 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-19 14:28:39,427 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:28:39,427 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:28:19 2023-11-19 14:28:39,427 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:30:24 2023-11-19 14:30:24,693 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-19 14:30:27,508 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 14:30:27,508 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:30:19 2023-11-19 14:30:27,508 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:32:24 2023-11-19 14:30:27,509 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-19 14:30:35,047 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 14:30:35,047 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:30:19 2023-11-19 14:30:35,047 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:32:24 2023-11-19 14:30:35,047 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-19 14:30:35,689 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:30:35,689 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:30:19 2023-11-19 14:30:35,690 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:32:24 2023-11-19 14:31:25,821 INFO 1947952:repeater task 3dayplots_wh300 starting 1.8 seconds late 2023-11-19 14:31:29,744 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:31:29,745 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:31:29,746 INFO 1947952:repeater task lastensq_os150nb starting 6.7 seconds late 2023-11-19 14:31:31,937 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 14:31:31,937 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:31:18 2023-11-19 14:31:31,937 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:36:23 2023-11-19 14:31:31,937 INFO 1947952:repeater task quick_os150nb starting 8.9 seconds late 2023-11-19 14:31:35,835 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-19 14:31:35,836 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:31:18 2023-11-19 14:31:35,836 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:36:23 2023-11-19 14:31:35,837 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.8 seconds late 2023-11-19 14:31:36,454 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:31:36,455 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:31:18 2023-11-19 14:31:36,455 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:36:23 2023-11-19 14:31:36,455 INFO 1947952:repeater task 3dayplots_os150nb starting 12.4 seconds late 2023-11-19 14:31:39,832 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 14:31:39,833 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:31:39,834 INFO 1947952:repeater task lastensq_os38nb starting 14.8 seconds late 2023-11-19 14:31:41,934 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:31:41,934 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:31:20 2023-11-19 14:31:41,934 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:36:25 2023-11-19 14:31:41,935 INFO 1947952:repeater task quick_os38nb starting 16.9 seconds late 2023-11-19 14:31:46,607 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 14:31:46,607 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:31:20 2023-11-19 14:31:46,607 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:36:25 2023-11-19 14:31:46,608 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.6 seconds late 2023-11-19 14:31:47,224 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:31:47,225 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:31:20 2023-11-19 14:31:47,225 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:36:25 2023-11-19 14:31:47,225 INFO 1947952:repeater task 3dayplots_os38nb starting 23.2 seconds late 2023-11-19 14:31:50,743 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 14:31:50,743 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:32:25,824 INFO 1947952:repeater task lastensq_wh300 starting 1.8 seconds late 2023-11-19 14:32:28,498 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:32:28,498 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:32:19 2023-11-19 14:32:28,499 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:34:24 2023-11-19 14:32:28,499 INFO 1947952:repeater task quick_wh300 starting 4.5 seconds late 2023-11-19 14:32:35,720 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 14:32:35,721 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:32:19 2023-11-19 14:32:35,721 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:34:24 2023-11-19 14:32:35,721 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-19 14:32:36,358 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:32:36,358 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:32:19 2023-11-19 14:32:36,358 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:34:24 2023-11-19 14:34:26,624 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-19 14:34:29,300 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:34:29,300 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:34:19 2023-11-19 14:34:29,300 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:34:29,301 INFO 1947952:repeater task quick_wh300 starting 5.3 seconds late 2023-11-19 14:34:36,507 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 14:34:36,507 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:34:19 2023-11-19 14:34:36,507 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:34:36,508 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2023-11-19 14:34:37,141 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:34:37,141 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:34:19 2023-11-19 14:34:37,141 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:36:24 2023-11-19 14:36:22,413 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.4 seconds late 2023-11-19 14:36:23,649 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 14:36:23,649 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:23,650 INFO 1947952:repeater task grab_lastpings_wh300 starting 4.6 seconds late 2023-11-19 14:36:23,816 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 14:36:23,816 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:23,816 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.8 seconds late 2023-11-19 14:36:25,111 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 14:36:25,111 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:25,111 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.1 seconds late 2023-11-19 14:36:26,402 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 14:36:26,403 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:26,403 INFO 1947952:repeater task lastensq_os150nb starting 3.4 seconds late 2023-11-19 14:36:28,577 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 14:36:28,578 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:36:18 2023-11-19 14:36:28,578 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:41:23 2023-11-19 14:36:28,578 INFO 1947952:repeater task quick_os150nb starting 5.6 seconds late 2023-11-19 14:36:33,034 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 14:36:33,035 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:36:18 2023-11-19 14:36:33,035 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:41:23 2023-11-19 14:36:33,036 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.0 seconds late 2023-11-19 14:36:33,707 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.7 seconds 2023-11-19 14:36:33,707 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:36:18 2023-11-19 14:36:33,707 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:41:23 2023-11-19 14:36:33,708 INFO 1947952:repeater task 3dayplots_os150nb starting 9.7 seconds late 2023-11-19 14:36:37,162 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-19 14:36:37,163 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:41:24 2023-11-19 14:36:37,163 INFO 1947952:repeater task plot_tsstats_os150nb starting 18.1 seconds late 2023-11-19 14:36:38,164 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 14:36:38,165 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:38,165 INFO 1947952:repeater task grab_lastpings_os150nb starting 19.1 seconds late 2023-11-19 14:36:38,335 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 14:36:38,335 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:38,336 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.3 seconds late 2023-11-19 14:36:39,595 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 14:36:39,595 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:39,596 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.5 seconds late 2023-11-19 14:36:40,902 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 14:36:40,903 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:40,903 INFO 1947952:repeater task lastensq_os38nb starting 15.9 seconds late 2023-11-19 14:36:43,040 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:36:43,041 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:36:20 2023-11-19 14:36:43,041 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:41:25 2023-11-19 14:36:43,041 INFO 1947952:repeater task quick_os38nb starting 18.0 seconds late 2023-11-19 14:36:47,928 INFO 1947952:repeater task quick_os38nb ended after 4.9 seconds 2023-11-19 14:36:47,928 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:36:20 2023-11-19 14:36:47,928 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:41:25 2023-11-19 14:36:47,929 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.9 seconds late 2023-11-19 14:36:48,541 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:36:48,541 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:36:20 2023-11-19 14:36:48,541 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:41:25 2023-11-19 14:36:48,542 INFO 1947952:repeater task 3dayplots_os38nb starting 24.5 seconds late 2023-11-19 14:36:52,006 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 14:36:52,007 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:41:24 2023-11-19 14:36:52,007 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.0 seconds late 2023-11-19 14:36:53,040 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 14:36:53,040 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:53,041 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.0 seconds late 2023-11-19 14:36:53,209 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 14:36:53,209 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:53,210 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.2 seconds late 2023-11-19 14:36:54,469 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 14:36:54,469 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:54,470 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.4 seconds late 2023-11-19 14:36:55,700 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 14:36:55,700 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:36:55,700 INFO 1947952:repeater task daily_netcdf starting 31.6 seconds late 2023-11-19 14:36:56,519 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 14:36:56,519 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:36:56,520 INFO 1947952:repeater task ggatime_diagnostics starting 37.5 seconds late 2023-11-19 14:37:12,931 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-19 14:37:12,931 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:37:12,931 INFO 1947952:repeater task ggatime_diagnostics starting 53.9 seconds late 2023-11-19 14:37:28,587 INFO 1947952:repeater task ggatime_diagnostics ended after 15.7 seconds 2023-11-19 14:37:28,588 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:37:28,588 INFO 1947952:repeater task ggatime_diagnostics starting 69.5 seconds late 2023-11-19 14:37:44,656 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-19 14:37:44,656 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:37:44,657 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 85.4 seconds late 2023-11-19 14:37:46,003 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 14:37:46,004 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:37:46,004 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 86.7 seconds late 2023-11-19 14:37:47,282 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 14:37:47,282 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:37:47,283 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 88.0 seconds late 2023-11-19 14:37:48,488 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-19 14:37:48,489 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:37:48,489 INFO 1947952:repeater task plot_hbins starting 89.2 seconds late 2023-11-19 14:37:52,825 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 14:37:52,825 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 14:37:52,825 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 14:51:19 2023-11-19 14:37:52,826 INFO 1947952:repeater task report_generator starting 93.6 seconds late 2023-11-19 14:37:59,011 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 6.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-19 14:37:59,011 INFO 1947952:repeater task report_generator ended after 6.2 seconds 2023-11-19 14:37:59,011 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/19 15:36:19 2023-11-19 14:38:04,017 INFO 1947952:repeater task noisy_rawtemps_os150 starting 100.0 seconds late 2023-11-19 14:38:06,892 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 14:38:06,892 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:38:06,892 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 102.8 seconds late 2023-11-19 14:38:09,692 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 14:38:09,693 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:38:09,693 INFO 1947952:repeater task noisy_rawtemps_os38 starting 105.6 seconds late 2023-11-19 14:38:12,764 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 14:38:12,765 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:38:12,765 INFO 1947952:repeater task lastensq_wh300 starting 108.8 seconds late 2023-11-19 14:38:15,484 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:38:15,484 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:36:19 2023-11-19 14:38:15,485 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:38:24 2023-11-19 14:38:15,485 INFO 1947952:repeater task quick_wh300 starting 111.5 seconds late 2023-11-19 14:38:23,022 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 14:38:23,022 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:36:19 2023-11-19 14:38:23,022 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:38:24 2023-11-19 14:38:23,023 INFO 1947952:repeater task lastfew_vec_wh300 starting 119.0 seconds late 2023-11-19 14:38:23,651 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:38:23,651 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:36:19 2023-11-19 14:38:23,651 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:38:24 2023-11-19 14:38:23,652 INFO 1947952:repeater task 3dayplots_wh300 starting 119.6 seconds late 2023-11-19 14:38:27,636 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-19 14:38:27,636 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:41:24 2023-11-19 14:38:32,646 INFO 1947952:repeater task lastensq_wh300 starting 8.6 seconds late 2023-11-19 14:38:35,650 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 14:38:35,650 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:38:19 2023-11-19 14:38:35,651 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:40:24 2023-11-19 14:38:35,651 INFO 1947952:repeater task quick_wh300 starting 11.7 seconds late 2023-11-19 14:38:43,873 INFO 1947952:repeater task quick_wh300 ended after 8.2 seconds 2023-11-19 14:38:43,874 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:38:19 2023-11-19 14:38:43,874 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:40:24 2023-11-19 14:38:43,874 INFO 1947952:repeater task lastfew_vec_wh300 starting 19.9 seconds late 2023-11-19 14:38:44,506 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:38:44,506 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:38:19 2023-11-19 14:38:44,506 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:40:24 2023-11-19 14:40:24,756 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-19 14:40:27,584 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 14:40:27,584 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:40:19 2023-11-19 14:40:27,584 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:42:24 2023-11-19 14:40:27,585 INFO 1947952:repeater task quick_wh300 starting 3.6 seconds late 2023-11-19 14:40:34,920 INFO 1947952:repeater task quick_wh300 ended after 7.3 seconds 2023-11-19 14:40:34,920 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:40:19 2023-11-19 14:40:34,921 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:42:24 2023-11-19 14:40:34,921 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2023-11-19 14:40:35,553 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:40:35,553 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:40:19 2023-11-19 14:40:35,553 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:42:24 2023-11-19 14:41:25,677 INFO 1947952:repeater task 3dayplots_wh300 starting 1.6 seconds late 2023-11-19 14:41:29,605 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:41:29,605 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:41:29,606 INFO 1947952:repeater task lastensq_os150nb starting 6.6 seconds late 2023-11-19 14:41:31,868 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-19 14:41:31,869 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:41:18 2023-11-19 14:41:31,869 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:46:23 2023-11-19 14:41:31,869 INFO 1947952:repeater task quick_os150nb starting 8.9 seconds late 2023-11-19 14:41:35,955 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 14:41:35,955 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:41:18 2023-11-19 14:41:35,955 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:46:23 2023-11-19 14:41:35,956 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.0 seconds late 2023-11-19 14:41:36,561 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:41:36,561 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:41:18 2023-11-19 14:41:36,561 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:46:23 2023-11-19 14:41:36,562 INFO 1947952:repeater task 3dayplots_os150nb starting 12.5 seconds late 2023-11-19 14:41:39,938 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 14:41:39,938 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:41:39,939 INFO 1947952:repeater task lastensq_os38nb starting 14.9 seconds late 2023-11-19 14:41:42,066 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:41:42,067 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:41:20 2023-11-19 14:41:42,067 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:46:25 2023-11-19 14:41:42,067 INFO 1947952:repeater task quick_os38nb starting 17.1 seconds late 2023-11-19 14:41:46,768 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 14:41:46,769 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:41:20 2023-11-19 14:41:46,769 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:46:25 2023-11-19 14:41:46,769 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.8 seconds late 2023-11-19 14:41:47,394 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:41:47,394 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:41:20 2023-11-19 14:41:47,394 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:46:25 2023-11-19 14:41:47,395 INFO 1947952:repeater task 3dayplots_os38nb starting 23.3 seconds late 2023-11-19 14:41:50,900 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 14:41:50,900 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:42:25,988 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-19 14:42:29,009 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 14:42:29,009 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:42:19 2023-11-19 14:42:29,009 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:44:24 2023-11-19 14:42:29,010 INFO 1947952:repeater task quick_wh300 starting 5.0 seconds late 2023-11-19 14:42:36,838 INFO 1947952:repeater task quick_wh300 ended after 7.8 seconds 2023-11-19 14:42:36,839 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:42:19 2023-11-19 14:42:36,839 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:44:24 2023-11-19 14:42:36,839 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-19 14:42:37,471 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:42:37,472 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:42:19 2023-11-19 14:42:37,472 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:44:24 2023-11-19 14:44:27,764 INFO 1947952:repeater task lastensq_wh300 starting 3.8 seconds late 2023-11-19 14:44:30,489 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:44:30,490 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:44:19 2023-11-19 14:44:30,490 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:44:30,490 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-19 14:44:37,688 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 14:44:37,689 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:44:19 2023-11-19 14:44:37,689 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:44:37,689 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-19 14:44:38,323 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:44:38,323 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:44:19 2023-11-19 14:44:38,323 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:46:24 2023-11-19 14:46:23,706 INFO 1947952:repeater task lastensq_os150nb starting 0.7 seconds late 2023-11-19 14:46:25,948 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-19 14:46:25,948 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:46:18 2023-11-19 14:46:25,948 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:51:23 2023-11-19 14:46:25,949 INFO 1947952:repeater task quick_os150nb starting 2.9 seconds late 2023-11-19 14:46:30,063 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-19 14:46:30,063 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:46:18 2023-11-19 14:46:30,064 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:51:23 2023-11-19 14:46:30,064 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.1 seconds late 2023-11-19 14:46:30,666 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:46:30,666 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:46:18 2023-11-19 14:46:30,666 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:51:23 2023-11-19 14:46:30,666 INFO 1947952:repeater task 3dayplots_os150nb starting 6.6 seconds late 2023-11-19 14:46:34,387 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.7 seconds 2023-11-19 14:46:34,387 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:51:24 2023-11-19 14:46:34,388 INFO 1947952:repeater task lastensq_os38nb starting 9.4 seconds late 2023-11-19 14:46:36,449 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:46:36,449 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:46:20 2023-11-19 14:46:36,449 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:51:25 2023-11-19 14:46:36,449 INFO 1947952:repeater task quick_os38nb starting 11.4 seconds late 2023-11-19 14:46:40,829 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-19 14:46:40,829 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:46:20 2023-11-19 14:46:40,829 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:51:25 2023-11-19 14:46:40,830 INFO 1947952:repeater task lastfew_vec_os38nb starting 15.8 seconds late 2023-11-19 14:46:41,439 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:46:41,439 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:46:20 2023-11-19 14:46:41,439 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:51:25 2023-11-19 14:46:41,440 INFO 1947952:repeater task 3dayplots_os38nb starting 17.4 seconds late 2023-11-19 14:46:44,894 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 14:46:44,894 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:51:24 2023-11-19 14:46:44,895 INFO 1947952:repeater task daily_netcdf starting 20.8 seconds late 2023-11-19 14:46:45,730 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-19 14:46:45,730 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:46:50,733 INFO 1947952:repeater task noisy_rawtemps_os150 starting 26.7 seconds late 2023-11-19 14:46:53,715 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-19 14:46:53,715 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:46:53,715 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 29.7 seconds late 2023-11-19 14:46:56,454 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-19 14:46:56,454 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:46:56,454 INFO 1947952:repeater task noisy_rawtemps_os38 starting 32.4 seconds late 2023-11-19 14:46:59,568 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-19 14:46:59,569 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:46:59,569 INFO 1947952:repeater task lastensq_wh300 starting 35.6 seconds late 2023-11-19 14:47:02,214 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 14:47:02,214 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:46:19 2023-11-19 14:47:02,214 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:48:24 2023-11-19 14:47:02,215 INFO 1947952:repeater task quick_wh300 starting 38.2 seconds late 2023-11-19 14:47:09,913 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 14:47:09,914 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:46:19 2023-11-19 14:47:09,914 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:48:24 2023-11-19 14:47:09,914 INFO 1947952:repeater task lastfew_vec_wh300 starting 45.9 seconds late 2023-11-19 14:47:10,550 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:47:10,550 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:46:19 2023-11-19 14:47:10,550 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:48:24 2023-11-19 14:47:10,551 INFO 1947952:repeater task 3dayplots_wh300 starting 46.5 seconds late 2023-11-19 14:47:14,404 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:47:14,404 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:51:24 2023-11-19 14:48:24,591 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-19 14:48:27,604 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-19 14:48:27,604 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:48:19 2023-11-19 14:48:27,604 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:50:24 2023-11-19 14:48:27,605 INFO 1947952:repeater task quick_wh300 starting 3.6 seconds late 2023-11-19 14:48:35,624 INFO 1947952:repeater task quick_wh300 ended after 8.0 seconds 2023-11-19 14:48:35,625 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:48:19 2023-11-19 14:48:35,625 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:50:24 2023-11-19 14:48:35,625 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2023-11-19 14:48:36,258 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:48:36,258 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:48:19 2023-11-19 14:48:36,258 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:50:24 2023-11-19 14:50:26,554 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-19 14:50:29,456 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-19 14:50:29,456 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:50:19 2023-11-19 14:50:29,456 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:52:24 2023-11-19 14:50:29,457 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-19 14:50:36,891 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 14:50:36,892 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:50:19 2023-11-19 14:50:36,892 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:52:24 2023-11-19 14:50:36,892 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-19 14:50:37,526 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:50:37,526 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:50:19 2023-11-19 14:50:37,526 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:52:24 2023-11-19 14:51:22,633 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.6 seconds late 2023-11-19 14:51:23,866 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 14:51:23,866 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:23,866 INFO 1947952:repeater task grab_lastpings_wh300 starting 4.8 seconds late 2023-11-19 14:51:24,032 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 14:51:24,032 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:24,033 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.0 seconds late 2023-11-19 14:51:25,419 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.4 seconds 2023-11-19 14:51:25,419 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:25,419 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.4 seconds late 2023-11-19 14:51:26,690 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 14:51:26,691 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:26,691 INFO 1947952:repeater task lastensq_os150nb starting 3.7 seconds late 2023-11-19 14:51:28,817 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-19 14:51:28,818 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:51:18 2023-11-19 14:51:28,818 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 14:56:23 2023-11-19 14:51:28,818 INFO 1947952:repeater task quick_os150nb starting 5.8 seconds late 2023-11-19 14:51:33,363 INFO 1947952:repeater task quick_os150nb ended after 4.5 seconds 2023-11-19 14:51:33,364 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:51:18 2023-11-19 14:51:33,364 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 14:56:23 2023-11-19 14:51:33,364 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.4 seconds late 2023-11-19 14:51:33,969 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:51:33,969 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:51:18 2023-11-19 14:51:33,969 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 14:56:23 2023-11-19 14:51:33,970 INFO 1947952:repeater task 3dayplots_os150nb starting 9.9 seconds late 2023-11-19 14:51:37,463 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-19 14:51:37,464 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:51:37,464 INFO 1947952:repeater task plot_tsstats_os150nb starting 18.4 seconds late 2023-11-19 14:51:38,470 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 14:51:38,470 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:38,471 INFO 1947952:repeater task grab_lastpings_os150nb starting 19.4 seconds late 2023-11-19 14:51:38,641 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 14:51:38,641 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:38,642 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.6 seconds late 2023-11-19 14:51:39,911 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 14:51:39,912 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:39,912 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.9 seconds late 2023-11-19 14:51:41,176 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-19 14:51:41,176 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:41,176 INFO 1947952:repeater task lastensq_os38nb starting 16.2 seconds late 2023-11-19 14:51:43,287 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:51:43,287 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:51:20 2023-11-19 14:51:43,287 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 14:56:25 2023-11-19 14:51:43,288 INFO 1947952:repeater task quick_os38nb starting 18.3 seconds late 2023-11-19 14:51:48,108 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 14:51:48,108 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:51:20 2023-11-19 14:51:48,108 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 14:56:25 2023-11-19 14:51:48,109 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.1 seconds late 2023-11-19 14:51:48,729 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:51:48,729 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:51:20 2023-11-19 14:51:48,729 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 14:56:25 2023-11-19 14:51:48,730 INFO 1947952:repeater task 3dayplots_os38nb starting 24.7 seconds late 2023-11-19 14:51:52,191 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 14:51:52,192 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:51:52,192 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.1 seconds late 2023-11-19 14:51:53,234 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-19 14:51:53,234 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:53,234 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.2 seconds late 2023-11-19 14:51:53,399 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-19 14:51:53,399 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:53,400 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.3 seconds late 2023-11-19 14:51:54,703 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-19 14:51:54,704 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:54,704 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.7 seconds late 2023-11-19 14:51:55,869 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-19 14:51:55,869 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:55,871 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 36.6 seconds late 2023-11-19 14:51:57,141 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-19 14:51:57,141 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:57,142 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 37.9 seconds late 2023-11-19 14:51:58,425 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-19 14:51:58,425 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:58,426 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 39.2 seconds late 2023-11-19 14:51:59,678 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-19 14:51:59,678 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:51:59,679 INFO 1947952:repeater task plot_hbins starting 40.4 seconds late 2023-11-19 14:52:04,020 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-19 14:52:04,021 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-19 14:52:04,021 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/19 15:06:19 2023-11-19 14:52:09,029 INFO 1947952:repeater task 3dayplots_wh300 starting 45.0 seconds late 2023-11-19 14:52:12,883 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:52:12,883 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:52:27,914 INFO 1947952:repeater task lastensq_wh300 starting 3.9 seconds late 2023-11-19 14:52:30,648 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 14:52:30,649 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:52:19 2023-11-19 14:52:30,649 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:54:24 2023-11-19 14:52:30,649 INFO 1947952:repeater task quick_wh300 starting 6.6 seconds late 2023-11-19 14:52:38,311 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 14:52:38,312 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:52:19 2023-11-19 14:52:38,312 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:54:24 2023-11-19 14:52:38,312 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-19 14:52:38,947 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:52:38,947 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:52:19 2023-11-19 14:52:38,947 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:54:24 2023-11-19 14:54:24,224 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-19 14:54:27,034 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-19 14:54:27,034 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:54:19 2023-11-19 14:54:27,034 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:54:27,035 INFO 1947952:repeater task quick_wh300 starting 3.0 seconds late 2023-11-19 14:54:34,536 INFO 1947952:repeater task quick_wh300 ended after 7.5 seconds 2023-11-19 14:54:34,537 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:54:19 2023-11-19 14:54:34,537 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:54:34,537 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-19 14:54:35,173 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:54:35,174 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:54:19 2023-11-19 14:54:35,174 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:56:24 2023-11-19 14:56:25,465 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.4 seconds late 2023-11-19 14:56:28,330 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-19 14:56:28,331 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/19 15:06:24 2023-11-19 14:56:28,331 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 4.3 seconds late 2023-11-19 14:56:31,132 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-19 14:56:31,132 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/19 15:06:24 2023-11-19 14:56:31,132 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.1 seconds late 2023-11-19 14:56:34,354 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.2 seconds 2023-11-19 14:56:34,355 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/19 15:06:24 2023-11-19 14:56:34,355 INFO 1947952:repeater task lastensq_wh300 starting 10.4 seconds late 2023-11-19 14:56:36,972 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 14:56:36,973 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:56:19 2023-11-19 14:56:36,973 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 14:58:24 2023-11-19 14:56:36,973 INFO 1947952:repeater task quick_wh300 starting 13.0 seconds late 2023-11-19 14:56:44,164 INFO 1947952:repeater task quick_wh300 ended after 7.2 seconds 2023-11-19 14:56:44,165 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:56:19 2023-11-19 14:56:44,165 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 14:58:24 2023-11-19 14:56:44,165 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.2 seconds late 2023-11-19 14:56:44,795 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:56:44,795 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:56:19 2023-11-19 14:56:44,795 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 14:58:24 2023-11-19 14:56:44,796 INFO 1947952:repeater task 3dayplots_wh300 starting 20.7 seconds late 2023-11-19 14:56:48,717 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 14:56:48,718 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 15:01:24 2023-11-19 14:56:48,719 INFO 1947952:repeater task lastensq_os150nb starting 25.7 seconds late 2023-11-19 14:56:50,819 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-19 14:56:50,820 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:56:18 2023-11-19 14:56:50,820 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 15:01:23 2023-11-19 14:56:50,820 INFO 1947952:repeater task quick_os150nb starting 27.8 seconds late 2023-11-19 14:56:54,806 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-19 14:56:54,806 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:56:18 2023-11-19 14:56:54,807 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 15:01:23 2023-11-19 14:56:54,807 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.8 seconds late 2023-11-19 14:56:55,412 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 14:56:55,413 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 14:56:18 2023-11-19 14:56:55,413 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 15:01:23 2023-11-19 14:56:55,413 INFO 1947952:repeater task 3dayplots_os150nb starting 31.4 seconds late 2023-11-19 14:56:58,845 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 14:56:58,845 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 15:01:24 2023-11-19 14:56:58,847 INFO 1947952:repeater task lastensq_os38nb starting 33.8 seconds late 2023-11-19 14:57:00,993 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 14:57:00,995 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:56:20 2023-11-19 14:57:00,995 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 15:01:25 2023-11-19 14:57:00,995 INFO 1947952:repeater task quick_os38nb starting 36.0 seconds late 2023-11-19 14:57:05,715 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-19 14:57:05,716 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:56:20 2023-11-19 14:57:05,716 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 15:01:25 2023-11-19 14:57:05,716 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.7 seconds late 2023-11-19 14:57:06,353 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 14:57:06,353 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 14:56:20 2023-11-19 14:57:06,353 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 15:01:25 2023-11-19 14:57:06,354 INFO 1947952:repeater task 3dayplots_os38nb starting 42.3 seconds late 2023-11-19 14:57:09,918 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.6 seconds 2023-11-19 14:57:09,919 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 15:01:24 2023-11-19 14:57:09,920 INFO 1947952:repeater task daily_netcdf starting 45.9 seconds late 2023-11-19 14:57:10,781 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-19 14:57:10,781 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/19 15:06:24 2023-11-19 14:58:25,971 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-19 14:58:28,612 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-19 14:58:28,612 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:58:19 2023-11-19 14:58:28,612 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 15:00:24 2023-11-19 14:58:28,613 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-19 14:58:35,990 INFO 1947952:repeater task quick_wh300 ended after 7.4 seconds 2023-11-19 14:58:35,990 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:58:19 2023-11-19 14:58:35,990 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 15:00:24 2023-11-19 14:58:35,990 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2023-11-19 14:58:36,625 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 14:58:36,625 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 14:58:19 2023-11-19 14:58:36,625 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 15:00:24 2023-11-19 15:00:26,916 INFO 1947952:repeater task lastensq_wh300 starting 2.9 seconds late 2023-11-19 15:00:29,634 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 15:00:29,635 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:00:19 2023-11-19 15:00:29,635 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 15:02:24 2023-11-19 15:00:29,635 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-19 15:00:37,617 INFO 1947952:repeater task quick_wh300 ended after 8.0 seconds 2023-11-19 15:00:37,617 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:00:19 2023-11-19 15:00:37,617 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 15:02:24 2023-11-19 15:00:37,618 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.6 seconds late 2023-11-19 15:00:38,263 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 15:00:38,263 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:00:19 2023-11-19 15:00:38,263 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 15:02:24 2023-11-19 15:01:23,376 INFO 1947952:repeater task lastensq_os150nb starting 0.4 seconds late 2023-11-19 15:01:26,122 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-19 15:01:26,123 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 15:01:18 2023-11-19 15:01:26,123 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 15:06:23 2023-11-19 15:01:26,123 INFO 1947952:repeater task quick_os150nb starting 3.1 seconds late 2023-11-19 15:01:30,313 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-19 15:01:30,313 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 15:01:18 2023-11-19 15:01:30,313 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 15:06:23 2023-11-19 15:01:30,314 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.3 seconds late 2023-11-19 15:01:30,913 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-19 15:01:30,913 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 15:01:18 2023-11-19 15:01:30,914 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 15:06:23 2023-11-19 15:01:30,914 INFO 1947952:repeater task 3dayplots_os150nb starting 6.9 seconds late 2023-11-19 15:01:34,344 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 15:01:34,344 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 15:06:24 2023-11-19 15:01:34,346 INFO 1947952:repeater task lastensq_os38nb starting 9.3 seconds late 2023-11-19 15:01:36,453 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-19 15:01:36,453 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 15:01:20 2023-11-19 15:01:36,453 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 15:06:25 2023-11-19 15:01:36,454 INFO 1947952:repeater task quick_os38nb starting 11.5 seconds late 2023-11-19 15:01:41,237 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-19 15:01:41,238 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 15:01:20 2023-11-19 15:01:41,238 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 15:06:25 2023-11-19 15:01:41,238 INFO 1947952:repeater task lastfew_vec_os38nb starting 16.2 seconds late 2023-11-19 15:01:41,851 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-19 15:01:41,852 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 15:01:20 2023-11-19 15:01:41,852 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/19 15:06:25 2023-11-19 15:01:41,852 INFO 1947952:repeater task 3dayplots_os38nb starting 17.8 seconds late 2023-11-19 15:01:45,312 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-19 15:01:45,312 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/19 15:06:24 2023-11-19 15:01:50,319 INFO 1947952:repeater task 3dayplots_wh300 starting 26.3 seconds late 2023-11-19 15:01:54,204 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-19 15:01:54,204 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/19 15:06:24 2023-11-19 15:02:24,269 INFO 1947952:repeater task lastensq_wh300 starting 0.3 seconds late 2023-11-19 15:02:26,964 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-19 15:02:26,964 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:02:19 2023-11-19 15:02:26,965 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 15:04:24 2023-11-19 15:02:26,965 INFO 1947952:repeater task quick_wh300 starting 3.0 seconds late 2023-11-19 15:02:34,906 INFO 1947952:repeater task quick_wh300 ended after 7.9 seconds 2023-11-19 15:02:34,907 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:02:19 2023-11-19 15:02:34,907 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 15:04:24 2023-11-19 15:02:34,907 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2023-11-19 15:02:35,559 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-19 15:02:35,560 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:02:19 2023-11-19 15:02:35,560 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 15:04:24 2023-11-19 15:04:25,847 INFO 1947952:repeater task lastensq_wh300 starting 1.8 seconds late 2023-11-19 15:04:29,140 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-19 15:04:29,140 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:04:19 2023-11-19 15:04:29,140 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/19 15:06:24 2023-11-19 15:04:29,141 INFO 1947952:repeater task quick_wh300 starting 5.1 seconds late 2023-11-19 15:04:36,868 INFO 1947952:repeater task quick_wh300 ended after 7.7 seconds 2023-11-19 15:04:36,868 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:04:19 2023-11-19 15:04:36,868 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/19 15:06:24 2023-11-19 15:04:36,869 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-19 15:04:37,511 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-19 15:04:37,511 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/19 15:04:19 2023-11-19 15:04:37,511 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/19 15:06:24 2023-11-19 15:06:22,789 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.7 seconds late 2023-11-19 15:06:24,027 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-19 15:06:24,027 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:24,028 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.0 seconds late 2023-11-19 15:06:24,202 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-19 15:06:24,202 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:24,202 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.2 seconds late 2023-11-19 15:06:25,453 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 15:06:25,453 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:25,454 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.4 seconds late 2023-11-19 15:06:26,729 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-19 15:06:26,729 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:26,729 INFO 1947952:repeater task lastensq_os150nb starting 3.7 seconds late 2023-11-19 15:06:29,780 INFO 1947952:repeater task lastensq_os150nb ended after 3.1 seconds 2023-11-19 15:06:29,781 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 15:06:18 2023-11-19 15:06:29,781 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/19 15:11:23 2023-11-19 15:06:29,781 INFO 1947952:repeater task quick_os150nb starting 6.8 seconds late 2023-11-19 15:06:34,647 INFO 1947952:repeater task quick_os150nb ended after 4.9 seconds 2023-11-19 15:06:34,648 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 15:06:18 2023-11-19 15:06:34,648 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/19 15:11:23 2023-11-19 15:06:34,649 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.6 seconds late 2023-11-19 15:06:35,365 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.7 seconds 2023-11-19 15:06:35,365 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/19 15:06:18 2023-11-19 15:06:35,365 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/19 15:11:23 2023-11-19 15:06:35,365 INFO 1947952:repeater task 3dayplots_os150nb starting 11.3 seconds late 2023-11-19 15:06:38,720 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-19 15:06:38,721 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/19 15:11:24 2023-11-19 15:06:38,721 INFO 1947952:repeater task plot_tsstats_os150nb starting 19.7 seconds late 2023-11-19 15:06:39,712 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-19 15:06:39,713 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:39,713 INFO 1947952:repeater task grab_lastpings_os150nb starting 20.7 seconds late 2023-11-19 15:06:39,877 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-19 15:06:39,877 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:39,878 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.8 seconds late 2023-11-19 15:06:41,122 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 15:06:41,123 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:41,123 INFO 1947952:repeater task beamdiagnostics_os150nb starting 22.1 seconds late 2023-11-19 15:06:42,363 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-19 15:06:42,363 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/19 15:21:19 2023-11-19 15:06:42,364 INFO 1947952:repeater task lastensq_os38nb starting 17.4 seconds late 2023-11-19 15:06:44,583 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-19 15:06:44,584 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 15:06:20 2023-11-19 15:06:44,584 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/19 15:11:25 2023-11-19 15:06:44,584 INFO 1947952:repeater task quick_os38nb starting 19.6 seconds late 2023-11-19 15:06:47,306 INFO 1947952:incr_class translate exiting, hdg /home/data/SR2325/rbin/seapath_udp 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, sea /home/data/SR2325/rbin/seapath_udp 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, pmv /home/data/SR2325/rbin/hydrins_udp 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, hdg /home/data/SR2325/rbin/hydrins_udp 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, gps /home/data/SR2325/rbin/seapath_udp 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, gps /home/data/SR2325/rbin/trimble_udp 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, gps /home/data/SR2325/rbin/hydrins_udp 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, gps /home/data/SR2325/rbin/hemisphere 2023-11-19 15:06:47,307 INFO 1947952:incr_class translate exiting, gph /home/data/SR2325/rbin/trimble_udp 2023-11-19 15:06:47,308 INFO 1947952:incr_class translate exiting, gph /home/data/SR2325/rbin/hemisphere 2023-11-19 15:06:49,068 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-19 15:06:49,068 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/19 15:06:20 2023-11-19 15:06:49,069 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/19 15:11:25 2023-11-19 15:06:54,074 INFO 1947952:repeater Running task report_generator at exit 2023-11-19 15:06:54,074 INFO 1947952:repeater task report_generator starting -1765.2 seconds late