2024-11-21 22:36:30,626 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-21 22:36:30,626 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:36:30,627 INFO 3318922:repeater task lastensq_wh300 starting 11.6 seconds late 2024-11-21 22:36:33,774 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 22:36:33,774 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:36:14 2024-11-21 22:36:33,775 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:38:19 2024-11-21 22:36:33,775 INFO 3318922:repeater task quick_wh300 starting 14.8 seconds late 2024-11-21 22:36:39,381 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-21 22:36:39,381 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:36:14 2024-11-21 22:36:39,381 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:38:19 2024-11-21 22:36:39,382 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.4 seconds late 2024-11-21 22:36:40,412 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 22:36:40,412 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:36:14 2024-11-21 22:36:40,412 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:38:19 2024-11-21 22:36:40,413 INFO 3318922:repeater task 3dayplots_wh300 starting 20.4 seconds late 2024-11-21 22:36:46,779 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-21 22:36:46,779 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 22:41:19 2024-11-21 22:36:46,780 INFO 3318922:repeater task lastensq_os150nb starting 27.8 seconds late 2024-11-21 22:36:49,257 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-21 22:36:49,257 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:36:14 2024-11-21 22:36:49,258 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 22:41:19 2024-11-21 22:36:49,258 INFO 3318922:repeater task quick_os150nb starting 30.3 seconds late 2024-11-21 22:36:52,948 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-21 22:36:52,948 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:36:14 2024-11-21 22:36:52,948 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 22:41:19 2024-11-21 22:36:52,949 INFO 3318922:repeater task lastfew_vec_os150nb starting 33.9 seconds late 2024-11-21 22:36:53,952 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 22:36:53,952 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:36:14 2024-11-21 22:36:53,952 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 22:41:19 2024-11-21 22:36:53,952 INFO 3318922:repeater task 3dayplots_os150nb starting 34.0 seconds late 2024-11-21 22:36:59,913 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 22:36:59,913 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 22:41:19 2024-11-21 22:36:59,915 INFO 3318922:repeater task lastensq_os38nb starting 38.9 seconds late 2024-11-21 22:37:02,432 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-21 22:37:02,433 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:36:16 2024-11-21 22:37:02,433 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 22:41:21 2024-11-21 22:37:02,433 INFO 3318922:repeater task quick_os38nb starting 41.4 seconds late 2024-11-21 22:37:06,261 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-21 22:37:06,261 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:36:16 2024-11-21 22:37:06,261 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 22:41:21 2024-11-21 22:37:06,261 INFO 3318922:repeater task lastfew_vec_os38nb starting 45.3 seconds late 2024-11-21 22:37:07,260 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 22:37:07,261 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:36:16 2024-11-21 22:37:07,261 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 22:41:21 2024-11-21 22:37:07,261 INFO 3318922:repeater task 3dayplots_os38nb starting 47.3 seconds late 2024-11-21 22:37:13,293 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 22:37:13,293 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 22:41:19 2024-11-21 22:37:13,295 INFO 3318922:repeater task daily_netcdf starting 53.3 seconds late 2024-11-21 22:37:14,088 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 22:37:14,088 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:38:19,260 INFO 3318922:repeater task lastensq_wh300 starting 0.3 seconds late 2024-11-21 22:38:22,444 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 22:38:22,444 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:38:14 2024-11-21 22:38:22,444 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:40:19 2024-11-21 22:38:22,445 INFO 3318922:repeater task quick_wh300 starting 3.4 seconds late 2024-11-21 22:38:27,934 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 22:38:27,934 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:38:14 2024-11-21 22:38:27,934 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:40:19 2024-11-21 22:38:27,934 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.9 seconds late 2024-11-21 22:38:28,994 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 22:38:28,995 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:38:14 2024-11-21 22:38:28,995 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:40:19 2024-11-21 22:40:19,284 INFO 3318922:repeater task lastensq_wh300 starting 0.3 seconds late 2024-11-21 22:40:22,551 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 22:40:22,552 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:40:14 2024-11-21 22:40:22,552 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:42:19 2024-11-21 22:40:22,552 INFO 3318922:repeater task quick_wh300 starting 3.6 seconds late 2024-11-21 22:40:28,091 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 22:40:28,092 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:40:14 2024-11-21 22:40:28,092 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:42:19 2024-11-21 22:40:28,092 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.1 seconds late 2024-11-21 22:40:29,133 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 22:40:29,134 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:40:14 2024-11-21 22:40:29,134 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:42:19 2024-11-21 22:41:19,258 INFO 3318922:repeater task plot_tsstats_wh300 starting 4.3 seconds late 2024-11-21 22:41:20,689 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 22:41:20,689 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-21 22:41:20,689 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:20,689 INFO 3318922:repeater task grab_lastpings_wh300 starting 5.7 seconds late 2024-11-21 22:41:20,852 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-21 22:41:20,852 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:20,853 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.9 seconds late 2024-11-21 22:41:22,559 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 22:41:22,559 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:22,559 INFO 3318922:repeater task beamdiagnostics_wh300 starting 7.6 seconds late 2024-11-21 22:41:24,298 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 22:41:24,299 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:24,299 INFO 3318922:repeater task lastensq_os150nb starting 5.3 seconds late 2024-11-21 22:41:26,777 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-21 22:41:26,777 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:41:14 2024-11-21 22:41:26,777 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:41:26,778 INFO 3318922:repeater task quick_os150nb starting 7.8 seconds late 2024-11-21 22:41:30,337 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-21 22:41:30,337 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:41:14 2024-11-21 22:41:30,337 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:41:30,338 INFO 3318922:repeater task lastfew_vec_os150nb starting 11.3 seconds late 2024-11-21 22:41:31,340 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 22:41:31,340 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:41:14 2024-11-21 22:41:31,340 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:41:31,340 INFO 3318922:repeater task 3dayplots_os150nb starting 11.4 seconds late 2024-11-21 22:41:37,291 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 22:41:37,291 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:41:37,291 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.3 seconds late 2024-11-21 22:41:38,554 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 22:41:38,554 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-21 22:41:38,554 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:38,555 INFO 3318922:repeater task grab_lastpings_os150nb starting 23.6 seconds late 2024-11-21 22:41:38,712 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-21 22:41:38,713 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:38,713 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.7 seconds late 2024-11-21 22:41:40,407 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 22:41:40,407 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:40,408 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.4 seconds late 2024-11-21 22:41:42,132 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 22:41:42,132 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:42,132 INFO 3318922:repeater task lastensq_os38nb starting 21.1 seconds late 2024-11-21 22:41:44,675 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-21 22:41:44,675 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:41:16 2024-11-21 22:41:44,675 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 22:46:21 2024-11-21 22:41:44,676 INFO 3318922:repeater task quick_os38nb starting 23.7 seconds late 2024-11-21 22:41:48,397 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-21 22:41:48,398 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:41:16 2024-11-21 22:41:48,398 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 22:46:21 2024-11-21 22:41:48,398 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.4 seconds late 2024-11-21 22:41:49,454 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.1 seconds 2024-11-21 22:41:49,455 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:41:16 2024-11-21 22:41:49,455 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 22:46:21 2024-11-21 22:41:49,455 INFO 3318922:repeater task 3dayplots_os38nb starting 29.5 seconds late 2024-11-21 22:41:55,433 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 22:41:55,433 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:41:55,434 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.5 seconds late 2024-11-21 22:41:56,709 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 22:41:56,709 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-21 22:41:56,709 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:56,710 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.7 seconds late 2024-11-21 22:41:56,868 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-21 22:41:56,868 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:56,869 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.9 seconds late 2024-11-21 22:41:58,581 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-21 22:41:58,581 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:41:58,582 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.6 seconds late 2024-11-21 22:42:00,317 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-21 22:42:00,319 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 22:56:14 2024-11-21 22:42:00,320 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 45.2 seconds late 2024-11-21 22:42:02,154 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-21 22:42:02,154 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-21 22:42:02,154 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/21 22:56:15 2024-11-21 22:42:02,155 INFO 3318922:repeater task plot_hbins starting 47.0 seconds late 2024-11-21 22:42:06,289 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-21 22:42:06,289 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-21 22:42:06,290 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/21 22:56:15 2024-11-21 22:42:11,296 INFO 3318922:repeater task 3dayplots_wh300 starting 51.3 seconds late 2024-11-21 22:42:17,558 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 22:42:17,558 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:42:22,567 INFO 3318922:repeater task lastensq_wh300 starting 3.6 seconds late 2024-11-21 22:42:25,672 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 22:42:25,672 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:42:14 2024-11-21 22:42:25,672 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:44:19 2024-11-21 22:42:25,673 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-21 22:42:31,068 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 22:42:31,068 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:42:14 2024-11-21 22:42:31,068 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:44:19 2024-11-21 22:42:31,068 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-21 22:42:32,120 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 22:42:32,121 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:42:14 2024-11-21 22:42:32,121 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:44:19 2024-11-21 22:44:22,424 INFO 3318922:repeater task lastensq_wh300 starting 3.4 seconds late 2024-11-21 22:44:25,651 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 22:44:25,652 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:44:14 2024-11-21 22:44:25,652 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:44:25,652 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-21 22:44:31,274 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-21 22:44:31,274 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:44:14 2024-11-21 22:44:31,274 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:44:31,275 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-21 22:44:32,317 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 22:44:32,317 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:44:14 2024-11-21 22:44:32,317 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:46:19 2024-11-21 22:46:22,626 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.7 seconds late 2024-11-21 22:46:25,639 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2024-11-21 22:46:25,639 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:46:25,639 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.7 seconds late 2024-11-21 22:46:28,742 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-21 22:46:28,742 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:46:28,743 INFO 3318922:repeater task noisy_rawtemps_os38 starting 8.8 seconds late 2024-11-21 22:46:32,305 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-21 22:46:32,305 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:46:32,306 INFO 3318922:repeater task lastensq_wh300 starting 13.3 seconds late 2024-11-21 22:46:35,440 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 22:46:35,440 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:46:14 2024-11-21 22:46:35,441 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:48:19 2024-11-21 22:46:35,441 INFO 3318922:repeater task quick_wh300 starting 16.4 seconds late 2024-11-21 22:46:40,923 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 22:46:40,924 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:46:14 2024-11-21 22:46:40,924 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:48:19 2024-11-21 22:46:40,924 INFO 3318922:repeater task lastfew_vec_wh300 starting 21.9 seconds late 2024-11-21 22:46:41,987 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 22:46:41,987 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:46:14 2024-11-21 22:46:41,987 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:48:19 2024-11-21 22:46:41,987 INFO 3318922:repeater task 3dayplots_wh300 starting 22.0 seconds late 2024-11-21 22:46:48,386 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-21 22:46:48,386 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 22:51:19 2024-11-21 22:46:48,388 INFO 3318922:repeater task lastensq_os150nb starting 29.4 seconds late 2024-11-21 22:46:51,011 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-21 22:46:51,011 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:46:14 2024-11-21 22:46:51,011 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 22:51:19 2024-11-21 22:46:51,012 INFO 3318922:repeater task quick_os150nb starting 32.0 seconds late 2024-11-21 22:46:54,559 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-21 22:46:54,559 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:46:14 2024-11-21 22:46:54,559 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 22:51:19 2024-11-21 22:46:54,559 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.6 seconds late 2024-11-21 22:46:55,580 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 22:46:55,580 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:46:14 2024-11-21 22:46:55,580 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 22:51:19 2024-11-21 22:46:55,580 INFO 3318922:repeater task 3dayplots_os150nb starting 35.6 seconds late 2024-11-21 22:47:01,641 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-21 22:47:01,641 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 22:51:19 2024-11-21 22:47:01,643 INFO 3318922:repeater task lastensq_os38nb starting 40.6 seconds late 2024-11-21 22:47:04,253 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-21 22:47:04,253 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:46:16 2024-11-21 22:47:04,253 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 22:51:21 2024-11-21 22:47:04,254 INFO 3318922:repeater task quick_os38nb starting 43.3 seconds late 2024-11-21 22:47:08,031 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-21 22:47:08,032 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:46:16 2024-11-21 22:47:08,032 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 22:51:21 2024-11-21 22:47:08,032 INFO 3318922:repeater task lastfew_vec_os38nb starting 47.0 seconds late 2024-11-21 22:47:09,061 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 22:47:09,061 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:46:16 2024-11-21 22:47:09,062 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 22:51:21 2024-11-21 22:47:09,062 INFO 3318922:repeater task 3dayplots_os38nb starting 49.1 seconds late 2024-11-21 22:47:15,014 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 22:47:15,014 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 22:51:19 2024-11-21 22:47:15,015 INFO 3318922:repeater task daily_netcdf starting 55.0 seconds late 2024-11-21 22:47:15,818 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 22:47:15,818 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:48:20,990 INFO 3318922:repeater task lastensq_wh300 starting 2.0 seconds late 2024-11-21 22:48:24,086 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 22:48:24,086 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:48:14 2024-11-21 22:48:24,086 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:50:19 2024-11-21 22:48:24,087 INFO 3318922:repeater task quick_wh300 starting 5.1 seconds late 2024-11-21 22:48:29,540 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 22:48:29,540 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:48:14 2024-11-21 22:48:29,540 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:50:19 2024-11-21 22:48:29,541 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-21 22:48:30,608 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 22:48:30,608 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:48:14 2024-11-21 22:48:30,608 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:50:19 2024-11-21 22:50:20,908 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-21 22:50:24,156 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 22:50:24,157 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:50:14 2024-11-21 22:50:24,157 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:52:19 2024-11-21 22:50:24,157 INFO 3318922:repeater task quick_wh300 starting 5.2 seconds late 2024-11-21 22:50:29,677 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 22:50:29,677 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:50:14 2024-11-21 22:50:29,677 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:52:19 2024-11-21 22:50:29,678 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2024-11-21 22:50:30,766 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 22:50:30,766 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:50:14 2024-11-21 22:50:30,766 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:52:19 2024-11-21 22:51:20,910 INFO 3318922:repeater task 3dayplots_wh300 starting 0.9 seconds late 2024-11-21 22:51:27,240 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 22:51:27,240 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:51:27,241 INFO 3318922:repeater task lastensq_os150nb starting 8.2 seconds late 2024-11-21 22:51:29,770 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-21 22:51:29,770 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:51:14 2024-11-21 22:51:29,770 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:51:29,771 INFO 3318922:repeater task quick_os150nb starting 10.8 seconds late 2024-11-21 22:51:33,649 INFO 3318922:repeater task quick_os150nb ended after 3.9 seconds 2024-11-21 22:51:33,649 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:51:14 2024-11-21 22:51:33,649 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:51:33,649 INFO 3318922:repeater task lastfew_vec_os150nb starting 14.6 seconds late 2024-11-21 22:51:34,651 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 22:51:34,651 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:51:14 2024-11-21 22:51:34,651 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:51:34,652 INFO 3318922:repeater task 3dayplots_os150nb starting 14.7 seconds late 2024-11-21 22:51:40,554 INFO 3318922:repeater task 3dayplots_os150nb ended after 5.9 seconds 2024-11-21 22:51:40,554 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:51:40,556 INFO 3318922:repeater task lastensq_os38nb starting 19.6 seconds late 2024-11-21 22:51:43,071 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-21 22:51:43,071 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:51:16 2024-11-21 22:51:43,071 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 22:56:21 2024-11-21 22:51:43,072 INFO 3318922:repeater task quick_os38nb starting 22.1 seconds late 2024-11-21 22:51:46,909 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-21 22:51:46,910 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:51:16 2024-11-21 22:51:46,910 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 22:56:21 2024-11-21 22:51:46,910 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.9 seconds late 2024-11-21 22:51:47,946 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 22:51:47,946 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:51:16 2024-11-21 22:51:47,946 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 22:56:21 2024-11-21 22:51:47,946 INFO 3318922:repeater task 3dayplots_os38nb starting 28.0 seconds late 2024-11-21 22:51:54,047 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-21 22:51:54,047 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:52:19,113 INFO 3318922:repeater task lastensq_wh300 starting 0.1 seconds late 2024-11-21 22:52:22,189 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 22:52:22,190 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:52:14 2024-11-21 22:52:22,190 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:54:19 2024-11-21 22:52:22,190 INFO 3318922:repeater task quick_wh300 starting 3.2 seconds late 2024-11-21 22:52:27,398 INFO 3318922:repeater task quick_wh300 ended after 5.2 seconds 2024-11-21 22:52:27,398 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:52:14 2024-11-21 22:52:27,398 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:54:19 2024-11-21 22:52:27,399 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.4 seconds late 2024-11-21 22:52:28,429 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 22:52:28,429 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:52:14 2024-11-21 22:52:28,429 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:54:19 2024-11-21 22:54:23,760 INFO 3318922:repeater task lastensq_wh300 starting 4.8 seconds late 2024-11-21 22:54:27,160 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-21 22:54:27,160 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:54:14 2024-11-21 22:54:27,160 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:54:27,161 INFO 3318922:repeater task quick_wh300 starting 8.2 seconds late 2024-11-21 22:54:32,693 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 22:54:32,693 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:54:14 2024-11-21 22:54:32,693 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:54:32,694 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2024-11-21 22:54:33,730 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 22:54:33,730 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:54:14 2024-11-21 22:54:33,730 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:56:19 2024-11-21 22:56:19,044 INFO 3318922:repeater task lastensq_wh300 starting 0.0 seconds late 2024-11-21 22:56:22,212 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 22:56:22,213 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:56:14 2024-11-21 22:56:22,213 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 22:58:19 2024-11-21 22:56:22,213 INFO 3318922:repeater task quick_wh300 starting 3.2 seconds late 2024-11-21 22:56:27,441 INFO 3318922:repeater task quick_wh300 ended after 5.2 seconds 2024-11-21 22:56:27,442 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:56:14 2024-11-21 22:56:27,442 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 22:58:19 2024-11-21 22:56:27,442 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.4 seconds late 2024-11-21 22:56:28,483 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 22:56:28,483 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:56:14 2024-11-21 22:56:28,483 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 22:58:19 2024-11-21 22:56:28,484 INFO 3318922:repeater task 3dayplots_wh300 starting 8.5 seconds late 2024-11-21 22:56:34,878 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-21 22:56:34,878 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:01:19 2024-11-21 22:56:34,878 INFO 3318922:repeater task plot_tsstats_wh300 starting 19.9 seconds late 2024-11-21 22:56:36,293 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 22:56:36,293 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-21 22:56:36,293 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:36,293 INFO 3318922:repeater task grab_lastpings_wh300 starting 21.3 seconds late 2024-11-21 22:56:36,458 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-21 22:56:36,458 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:36,458 INFO 3318922:repeater task beamdiagnostics_wh300 starting 21.5 seconds late 2024-11-21 22:56:38,189 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 22:56:38,189 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:38,190 INFO 3318922:repeater task beamdiagnostics_wh300 starting 23.2 seconds late 2024-11-21 22:56:39,915 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 22:56:39,915 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:39,915 INFO 3318922:repeater task lastensq_os150nb starting 20.9 seconds late 2024-11-21 22:56:42,571 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-21 22:56:42,571 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:56:14 2024-11-21 22:56:42,571 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:01:19 2024-11-21 22:56:42,572 INFO 3318922:repeater task quick_os150nb starting 23.6 seconds late 2024-11-21 22:56:46,162 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-21 22:56:46,163 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:56:14 2024-11-21 22:56:46,163 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:01:19 2024-11-21 22:56:46,163 INFO 3318922:repeater task lastfew_vec_os150nb starting 27.2 seconds late 2024-11-21 22:56:47,179 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 22:56:47,179 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 22:56:14 2024-11-21 22:56:47,179 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:01:19 2024-11-21 22:56:47,180 INFO 3318922:repeater task 3dayplots_os150nb starting 27.2 seconds late 2024-11-21 22:56:53,199 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 22:56:53,200 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:01:19 2024-11-21 22:56:53,200 INFO 3318922:repeater task plot_tsstats_os150nb starting 38.2 seconds late 2024-11-21 22:56:54,446 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 22:56:54,446 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.2 seconds 2024-11-21 22:56:54,446 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:54,447 INFO 3318922:repeater task grab_lastpings_os150nb starting 39.5 seconds late 2024-11-21 22:56:54,605 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-21 22:56:54,605 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:54,606 INFO 3318922:repeater task beamdiagnostics_os150nb starting 39.6 seconds late 2024-11-21 22:56:56,358 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-21 22:56:56,358 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:56,358 INFO 3318922:repeater task beamdiagnostics_os150nb starting 41.4 seconds late 2024-11-21 22:56:58,060 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 22:56:58,061 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:56:58,061 INFO 3318922:repeater task lastensq_os38nb starting 37.1 seconds late 2024-11-21 22:57:00,660 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-21 22:57:00,661 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:56:16 2024-11-21 22:57:00,661 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:01:21 2024-11-21 22:57:00,661 INFO 3318922:repeater task quick_os38nb starting 39.7 seconds late 2024-11-21 22:57:04,549 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-21 22:57:04,549 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:56:16 2024-11-21 22:57:04,549 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:01:21 2024-11-21 22:57:04,550 INFO 3318922:repeater task lastfew_vec_os38nb starting 43.6 seconds late 2024-11-21 22:57:05,561 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 22:57:05,561 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 22:56:16 2024-11-21 22:57:05,562 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:01:21 2024-11-21 22:57:05,562 INFO 3318922:repeater task 3dayplots_os38nb starting 45.6 seconds late 2024-11-21 22:57:11,571 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 22:57:11,571 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:01:19 2024-11-21 22:57:11,572 INFO 3318922:repeater task plot_tsstats_os38nb starting 56.6 seconds late 2024-11-21 22:57:12,848 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 22:57:12,848 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-21 22:57:12,848 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:57:12,848 INFO 3318922:repeater task grab_lastpings_os38nb starting 57.9 seconds late 2024-11-21 22:57:13,009 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-21 22:57:13,009 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:57:13,009 INFO 3318922:repeater task beamdiagnostics_os38nb starting 58.0 seconds late 2024-11-21 22:57:14,653 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.6 seconds 2024-11-21 22:57:14,653 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:57:14,653 INFO 3318922:repeater task beamdiagnostics_os38nb starting 59.7 seconds late 2024-11-21 22:57:16,586 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.9 seconds 2024-11-21 22:57:16,587 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:11:14 2024-11-21 22:57:16,587 INFO 3318922:repeater task daily_netcdf starting 56.6 seconds late 2024-11-21 22:57:17,386 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 22:57:17,386 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 23:06:19 2024-11-21 22:57:17,387 INFO 3318922:repeater task ggatime_diagnostics starting 62.4 seconds late 2024-11-21 22:57:25,075 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-21 22:57:25,075 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/21 23:26:14 2024-11-21 22:57:25,075 INFO 3318922:repeater task ggatime_diagnostics starting 70.1 seconds late 2024-11-21 22:57:32,620 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-21 22:57:32,621 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/21 23:26:14 2024-11-21 22:57:32,621 INFO 3318922:repeater task ggatime_diagnostics starting 77.6 seconds late 2024-11-21 22:57:40,147 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-21 22:57:40,147 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/21 23:26:14 2024-11-21 22:57:40,147 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 85.0 seconds late 2024-11-21 22:57:41,997 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-21 22:57:41,997 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-21 22:57:41,997 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/21 23:11:15 2024-11-21 22:57:41,997 INFO 3318922:repeater task plot_hbins starting 86.9 seconds late 2024-11-21 22:57:46,146 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-21 22:57:46,146 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-21 22:57:46,147 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/21 23:11:15 2024-11-21 22:57:51,153 INFO 3318922:repeater task noisy_rawtemps_os150 starting 91.2 seconds late 2024-11-21 22:57:54,226 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-21 22:57:54,226 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/21 23:06:19 2024-11-21 22:57:54,226 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 94.3 seconds late 2024-11-21 22:57:57,329 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-21 22:57:57,329 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/21 23:06:19 2024-11-21 22:57:57,330 INFO 3318922:repeater task noisy_rawtemps_os38 starting 97.4 seconds late 2024-11-21 22:58:00,835 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-21 22:58:00,835 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 23:06:19 2024-11-21 22:58:20,891 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-21 22:58:24,234 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 22:58:24,234 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:58:14 2024-11-21 22:58:24,234 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:00:19 2024-11-21 22:58:24,235 INFO 3318922:repeater task quick_wh300 starting 5.2 seconds late 2024-11-21 22:58:29,807 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-21 22:58:29,807 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:58:14 2024-11-21 22:58:29,807 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:00:19 2024-11-21 22:58:29,808 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-21 22:58:30,849 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 22:58:30,849 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 22:58:14 2024-11-21 22:58:30,849 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:00:19 2024-11-21 23:00:21,163 INFO 3318922:repeater task lastensq_wh300 starting 2.2 seconds late 2024-11-21 23:00:24,639 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-21 23:00:24,640 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:00:14 2024-11-21 23:00:24,640 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:02:19 2024-11-21 23:00:24,640 INFO 3318922:repeater task quick_wh300 starting 5.6 seconds late 2024-11-21 23:00:29,858 INFO 3318922:repeater task quick_wh300 ended after 5.2 seconds 2024-11-21 23:00:29,858 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:00:14 2024-11-21 23:00:29,858 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:02:19 2024-11-21 23:00:29,859 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2024-11-21 23:00:30,913 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:00:30,914 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:00:14 2024-11-21 23:00:30,914 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:02:19 2024-11-21 23:01:21,057 INFO 3318922:repeater task 3dayplots_wh300 starting 1.1 seconds late 2024-11-21 23:01:27,355 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:01:27,355 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:01:27,356 INFO 3318922:repeater task lastensq_os150nb starting 8.4 seconds late 2024-11-21 23:01:29,962 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-21 23:01:29,962 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:01:14 2024-11-21 23:01:29,962 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:01:29,962 INFO 3318922:repeater task quick_os150nb starting 11.0 seconds late 2024-11-21 23:01:33,589 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-21 23:01:33,589 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:01:14 2024-11-21 23:01:33,589 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:01:33,590 INFO 3318922:repeater task lastfew_vec_os150nb starting 14.6 seconds late 2024-11-21 23:01:34,661 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-21 23:01:34,661 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:01:14 2024-11-21 23:01:34,661 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:01:34,662 INFO 3318922:repeater task 3dayplots_os150nb starting 14.7 seconds late 2024-11-21 23:01:40,744 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-21 23:01:40,744 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:01:40,745 INFO 3318922:repeater task lastensq_os38nb starting 19.7 seconds late 2024-11-21 23:01:43,379 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-21 23:01:43,379 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:01:16 2024-11-21 23:01:43,380 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:06:21 2024-11-21 23:01:43,380 INFO 3318922:repeater task quick_os38nb starting 22.4 seconds late 2024-11-21 23:01:47,483 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-21 23:01:47,483 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:01:16 2024-11-21 23:01:47,483 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:06:21 2024-11-21 23:01:47,483 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.5 seconds late 2024-11-21 23:01:48,520 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:01:48,521 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:01:16 2024-11-21 23:01:48,521 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:06:21 2024-11-21 23:01:48,521 INFO 3318922:repeater task 3dayplots_os38nb starting 28.5 seconds late 2024-11-21 23:01:54,541 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 23:01:54,542 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:02:19,604 INFO 3318922:repeater task lastensq_wh300 starting 0.6 seconds late 2024-11-21 23:02:22,939 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:02:22,939 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:02:14 2024-11-21 23:02:22,940 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:04:19 2024-11-21 23:02:22,940 INFO 3318922:repeater task quick_wh300 starting 3.9 seconds late 2024-11-21 23:02:28,317 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:02:28,317 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:02:14 2024-11-21 23:02:28,317 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:04:19 2024-11-21 23:02:28,317 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2024-11-21 23:02:29,357 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:02:29,358 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:02:14 2024-11-21 23:02:29,358 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:04:19 2024-11-21 23:04:19,673 INFO 3318922:repeater task lastensq_wh300 starting 0.7 seconds late 2024-11-21 23:04:22,759 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 23:04:22,759 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:04:14 2024-11-21 23:04:22,759 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:04:22,759 INFO 3318922:repeater task quick_wh300 starting 3.8 seconds late 2024-11-21 23:04:28,189 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:04:28,190 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:04:14 2024-11-21 23:04:28,190 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:04:28,190 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2024-11-21 23:04:29,240 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:04:29,240 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:04:14 2024-11-21 23:04:29,240 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:06:19 2024-11-21 23:06:19,564 INFO 3318922:repeater task lastensq_wh300 starting 0.6 seconds late 2024-11-21 23:06:22,786 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:06:22,787 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:06:14 2024-11-21 23:06:22,787 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:08:19 2024-11-21 23:06:22,787 INFO 3318922:repeater task quick_wh300 starting 3.8 seconds late 2024-11-21 23:06:28,149 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:06:28,149 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:06:14 2024-11-21 23:06:28,149 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:08:19 2024-11-21 23:06:28,150 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2024-11-21 23:06:29,200 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:06:29,200 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:06:14 2024-11-21 23:06:29,201 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:08:19 2024-11-21 23:06:29,201 INFO 3318922:repeater task 3dayplots_wh300 starting 9.2 seconds late 2024-11-21 23:06:35,548 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:06:35,548 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:11:19 2024-11-21 23:06:35,549 INFO 3318922:repeater task lastensq_os150nb starting 16.5 seconds late 2024-11-21 23:06:38,140 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-21 23:06:38,140 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:06:14 2024-11-21 23:06:38,140 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:11:19 2024-11-21 23:06:38,141 INFO 3318922:repeater task quick_os150nb starting 19.1 seconds late 2024-11-21 23:06:41,585 INFO 3318922:repeater task quick_os150nb ended after 3.4 seconds 2024-11-21 23:06:41,585 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:06:14 2024-11-21 23:06:41,585 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:11:19 2024-11-21 23:06:41,585 INFO 3318922:repeater task lastfew_vec_os150nb starting 22.6 seconds late 2024-11-21 23:06:42,594 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:06:42,594 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:06:14 2024-11-21 23:06:42,594 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:11:19 2024-11-21 23:06:42,595 INFO 3318922:repeater task 3dayplots_os150nb starting 22.6 seconds late 2024-11-21 23:06:48,729 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-21 23:06:48,729 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:11:19 2024-11-21 23:06:48,731 INFO 3318922:repeater task lastensq_os38nb starting 27.7 seconds late 2024-11-21 23:06:51,367 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-21 23:06:51,368 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:06:16 2024-11-21 23:06:51,368 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:11:21 2024-11-21 23:06:51,368 INFO 3318922:repeater task quick_os38nb starting 30.4 seconds late 2024-11-21 23:06:55,325 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-21 23:06:55,325 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:06:16 2024-11-21 23:06:55,325 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:11:21 2024-11-21 23:06:55,325 INFO 3318922:repeater task lastfew_vec_os38nb starting 34.3 seconds late 2024-11-21 23:06:56,332 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:06:56,333 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:06:16 2024-11-21 23:06:56,333 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:11:21 2024-11-21 23:06:56,333 INFO 3318922:repeater task 3dayplots_os38nb starting 36.4 seconds late 2024-11-21 23:07:02,395 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-21 23:07:02,395 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:11:19 2024-11-21 23:07:02,396 INFO 3318922:repeater task daily_netcdf starting 42.4 seconds late 2024-11-21 23:07:03,182 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 23:07:03,182 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:07:08,186 INFO 3318922:repeater task noisy_rawtemps_os150 starting 48.2 seconds late 2024-11-21 23:07:11,243 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-21 23:07:11,243 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:07:11,243 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 51.3 seconds late 2024-11-21 23:07:14,364 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-21 23:07:14,365 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:07:14,365 INFO 3318922:repeater task noisy_rawtemps_os38 starting 54.4 seconds late 2024-11-21 23:07:17,844 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-21 23:07:17,844 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:08:23,035 INFO 3318922:repeater task lastensq_wh300 starting 4.0 seconds late 2024-11-21 23:08:26,274 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:08:26,275 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:08:14 2024-11-21 23:08:26,275 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:10:19 2024-11-21 23:08:26,275 INFO 3318922:repeater task quick_wh300 starting 7.3 seconds late 2024-11-21 23:08:31,687 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:08:31,687 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:08:14 2024-11-21 23:08:31,687 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:10:19 2024-11-21 23:08:31,688 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2024-11-21 23:08:32,742 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:08:32,743 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:08:14 2024-11-21 23:08:32,743 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:10:19 2024-11-21 23:10:23,064 INFO 3318922:repeater task lastensq_wh300 starting 4.1 seconds late 2024-11-21 23:10:26,321 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:10:26,321 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:10:14 2024-11-21 23:10:26,321 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:12:19 2024-11-21 23:10:26,321 INFO 3318922:repeater task quick_wh300 starting 7.3 seconds late 2024-11-21 23:10:31,881 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-21 23:10:31,882 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:10:14 2024-11-21 23:10:31,882 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:12:19 2024-11-21 23:10:31,882 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2024-11-21 23:10:32,905 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:10:32,905 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:10:14 2024-11-21 23:10:32,905 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:12:19 2024-11-21 23:11:18,038 INFO 3318922:repeater task plot_tsstats_wh300 starting 3.1 seconds late 2024-11-21 23:11:19,457 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:11:19,457 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-21 23:11:19,458 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:19,458 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.5 seconds late 2024-11-21 23:11:19,624 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-21 23:11:19,624 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:19,625 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.6 seconds late 2024-11-21 23:11:21,383 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-21 23:11:21,383 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:21,384 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.4 seconds late 2024-11-21 23:11:23,063 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 23:11:23,065 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:23,066 INFO 3318922:repeater task lastensq_os150nb starting 4.1 seconds late 2024-11-21 23:11:25,740 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-21 23:11:25,740 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:11:14 2024-11-21 23:11:25,741 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:11:25,741 INFO 3318922:repeater task quick_os150nb starting 6.7 seconds late 2024-11-21 23:11:29,475 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-21 23:11:29,476 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:11:14 2024-11-21 23:11:29,476 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:11:29,476 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.5 seconds late 2024-11-21 23:11:30,479 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:11:30,479 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:11:14 2024-11-21 23:11:30,479 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:11:30,479 INFO 3318922:repeater task 3dayplots_os150nb starting 10.5 seconds late 2024-11-21 23:11:36,510 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 23:11:36,510 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:11:36,510 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.5 seconds late 2024-11-21 23:11:37,760 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:11:37,761 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-21 23:11:37,761 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:37,761 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.8 seconds late 2024-11-21 23:11:37,921 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-21 23:11:37,921 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:37,921 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.9 seconds late 2024-11-21 23:11:39,617 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:11:39,617 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:39,617 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.6 seconds late 2024-11-21 23:11:41,316 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:11:41,316 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:41,316 INFO 3318922:repeater task lastensq_os38nb starting 20.3 seconds late 2024-11-21 23:11:44,040 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-21 23:11:44,040 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:11:16 2024-11-21 23:11:44,040 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:16:21 2024-11-21 23:11:44,041 INFO 3318922:repeater task quick_os38nb starting 23.0 seconds late 2024-11-21 23:11:47,885 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-21 23:11:47,885 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:11:16 2024-11-21 23:11:47,885 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:16:21 2024-11-21 23:11:47,885 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.9 seconds late 2024-11-21 23:11:48,894 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:11:48,894 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:11:16 2024-11-21 23:11:48,894 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:16:21 2024-11-21 23:11:48,895 INFO 3318922:repeater task 3dayplots_os38nb starting 28.9 seconds late 2024-11-21 23:11:55,069 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-21 23:11:55,070 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:11:55,070 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.1 seconds late 2024-11-21 23:11:56,342 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:11:56,343 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-21 23:11:56,343 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:56,343 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.4 seconds late 2024-11-21 23:11:56,504 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-21 23:11:56,504 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:56,504 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.5 seconds late 2024-11-21 23:11:58,202 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-21 23:11:58,203 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:58,203 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.2 seconds late 2024-11-21 23:11:59,982 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-21 23:11:59,982 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:26:14 2024-11-21 23:11:59,984 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 44.9 seconds late 2024-11-21 23:12:01,810 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-21 23:12:01,811 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-21 23:12:01,811 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/21 23:26:15 2024-11-21 23:12:01,811 INFO 3318922:repeater task plot_hbins starting 46.7 seconds late 2024-11-21 23:12:05,995 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-21 23:12:05,995 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-21 23:12:05,996 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/21 23:26:15 2024-11-21 23:12:11,001 INFO 3318922:repeater task 3dayplots_wh300 starting 51.0 seconds late 2024-11-21 23:12:17,330 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:12:17,331 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:12:22,339 INFO 3318922:repeater task lastensq_wh300 starting 3.3 seconds late 2024-11-21 23:12:25,471 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 23:12:25,471 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:12:14 2024-11-21 23:12:25,471 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:14:19 2024-11-21 23:12:25,472 INFO 3318922:repeater task quick_wh300 starting 6.5 seconds late 2024-11-21 23:12:30,842 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:12:30,843 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:12:14 2024-11-21 23:12:30,843 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:14:19 2024-11-21 23:12:30,843 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2024-11-21 23:12:31,939 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:12:31,939 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:12:14 2024-11-21 23:12:31,939 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:14:19 2024-11-21 23:14:22,246 INFO 3318922:repeater task lastensq_wh300 starting 3.2 seconds late 2024-11-21 23:14:25,480 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:14:25,480 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:14:14 2024-11-21 23:14:25,480 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:14:25,481 INFO 3318922:repeater task quick_wh300 starting 6.5 seconds late 2024-11-21 23:14:31,272 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-21 23:14:31,272 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:14:14 2024-11-21 23:14:31,272 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:14:31,272 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-21 23:14:32,365 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:14:32,365 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:14:14 2024-11-21 23:14:32,365 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:16:19 2024-11-21 23:16:22,693 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.7 seconds late 2024-11-21 23:16:25,702 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2024-11-21 23:16:25,703 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:16:25,703 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.7 seconds late 2024-11-21 23:16:28,789 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-21 23:16:28,789 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:16:28,789 INFO 3318922:repeater task noisy_rawtemps_os38 starting 8.8 seconds late 2024-11-21 23:16:32,343 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-21 23:16:32,343 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:16:32,344 INFO 3318922:repeater task lastensq_wh300 starting 13.3 seconds late 2024-11-21 23:16:35,517 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:16:35,518 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:16:14 2024-11-21 23:16:35,518 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:18:19 2024-11-21 23:16:35,518 INFO 3318922:repeater task quick_wh300 starting 16.5 seconds late 2024-11-21 23:16:40,897 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:16:40,898 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:16:14 2024-11-21 23:16:40,898 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:18:19 2024-11-21 23:16:40,898 INFO 3318922:repeater task lastfew_vec_wh300 starting 21.9 seconds late 2024-11-21 23:16:41,933 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:16:41,934 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:16:14 2024-11-21 23:16:41,934 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:18:19 2024-11-21 23:16:41,934 INFO 3318922:repeater task 3dayplots_wh300 starting 22.0 seconds late 2024-11-21 23:16:48,275 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:16:48,276 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:21:19 2024-11-21 23:16:48,277 INFO 3318922:repeater task lastensq_os150nb starting 29.3 seconds late 2024-11-21 23:16:50,899 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-21 23:16:50,899 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:16:14 2024-11-21 23:16:50,899 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:21:19 2024-11-21 23:16:50,899 INFO 3318922:repeater task quick_os150nb starting 31.9 seconds late 2024-11-21 23:16:54,649 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-21 23:16:54,649 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:16:14 2024-11-21 23:16:54,649 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:21:19 2024-11-21 23:16:54,650 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.7 seconds late 2024-11-21 23:16:55,666 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:16:55,667 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:16:14 2024-11-21 23:16:55,667 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:21:19 2024-11-21 23:16:55,667 INFO 3318922:repeater task 3dayplots_os150nb starting 35.7 seconds late 2024-11-21 23:17:01,629 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 23:17:01,629 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:21:19 2024-11-21 23:17:01,631 INFO 3318922:repeater task lastensq_os38nb starting 40.6 seconds late 2024-11-21 23:17:04,291 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-21 23:17:04,291 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:16:16 2024-11-21 23:17:04,291 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:21:21 2024-11-21 23:17:04,292 INFO 3318922:repeater task quick_os38nb starting 43.3 seconds late 2024-11-21 23:17:08,079 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-21 23:17:08,080 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:16:16 2024-11-21 23:17:08,080 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:21:21 2024-11-21 23:17:08,080 INFO 3318922:repeater task lastfew_vec_os38nb starting 47.1 seconds late 2024-11-21 23:17:09,134 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.1 seconds 2024-11-21 23:17:09,135 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:16:16 2024-11-21 23:17:09,135 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:21:21 2024-11-21 23:17:09,135 INFO 3318922:repeater task 3dayplots_os38nb starting 49.2 seconds late 2024-11-21 23:17:15,177 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 23:17:15,178 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:21:19 2024-11-21 23:17:15,179 INFO 3318922:repeater task daily_netcdf starting 55.2 seconds late 2024-11-21 23:17:16,010 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 23:17:16,011 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:18:21,277 INFO 3318922:repeater task lastensq_wh300 starting 2.3 seconds late 2024-11-21 23:18:24,613 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:18:24,613 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:18:14 2024-11-21 23:18:24,613 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:20:19 2024-11-21 23:18:24,614 INFO 3318922:repeater task quick_wh300 starting 5.6 seconds late 2024-11-21 23:18:30,150 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 23:18:30,151 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:18:14 2024-11-21 23:18:30,151 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:20:19 2024-11-21 23:18:30,151 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.2 seconds late 2024-11-21 23:18:31,199 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:18:31,199 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:18:14 2024-11-21 23:18:31,199 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:20:19 2024-11-21 23:20:21,533 INFO 3318922:repeater task lastensq_wh300 starting 2.5 seconds late 2024-11-21 23:20:24,650 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 23:20:24,651 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:20:14 2024-11-21 23:20:24,651 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:22:19 2024-11-21 23:20:24,651 INFO 3318922:repeater task quick_wh300 starting 5.7 seconds late 2024-11-21 23:20:30,090 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:20:30,091 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:20:14 2024-11-21 23:20:30,091 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:22:19 2024-11-21 23:20:30,091 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2024-11-21 23:20:31,146 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:20:31,203 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:20:14 2024-11-21 23:20:31,224 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:22:19 2024-11-21 23:21:21,601 INFO 3318922:repeater task 3dayplots_wh300 starting 1.6 seconds late 2024-11-21 23:21:27,940 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:21:27,940 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:21:27,941 INFO 3318922:repeater task lastensq_os150nb starting 8.9 seconds late 2024-11-21 23:21:30,732 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-21 23:21:30,732 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:21:14 2024-11-21 23:21:30,732 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:21:30,732 INFO 3318922:repeater task quick_os150nb starting 11.7 seconds late 2024-11-21 23:21:34,736 INFO 3318922:repeater task quick_os150nb ended after 4.0 seconds 2024-11-21 23:21:34,736 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:21:14 2024-11-21 23:21:34,737 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:21:34,737 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.7 seconds late 2024-11-21 23:21:35,757 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:21:35,757 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:21:14 2024-11-21 23:21:35,757 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:21:35,758 INFO 3318922:repeater task 3dayplots_os150nb starting 15.8 seconds late 2024-11-21 23:21:41,808 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-21 23:21:41,808 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:21:41,810 INFO 3318922:repeater task lastensq_os38nb starting 20.8 seconds late 2024-11-21 23:21:44,489 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-21 23:21:44,489 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:21:16 2024-11-21 23:21:44,489 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:26:21 2024-11-21 23:21:44,490 INFO 3318922:repeater task quick_os38nb starting 23.5 seconds late 2024-11-21 23:21:48,298 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-21 23:21:48,298 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:21:16 2024-11-21 23:21:48,298 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:26:21 2024-11-21 23:21:48,299 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.3 seconds late 2024-11-21 23:21:49,325 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:21:49,325 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:21:16 2024-11-21 23:21:49,325 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:26:21 2024-11-21 23:21:49,326 INFO 3318922:repeater task 3dayplots_os38nb starting 29.3 seconds late 2024-11-21 23:21:55,416 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-21 23:21:55,416 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:22:20,485 INFO 3318922:repeater task lastensq_wh300 starting 1.5 seconds late 2024-11-21 23:22:23,595 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-21 23:22:23,595 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:22:14 2024-11-21 23:22:23,595 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:24:19 2024-11-21 23:22:23,595 INFO 3318922:repeater task quick_wh300 starting 4.6 seconds late 2024-11-21 23:22:29,036 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:22:29,036 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:22:14 2024-11-21 23:22:29,036 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:24:19 2024-11-21 23:22:29,037 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-21 23:22:30,088 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:22:30,088 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:22:14 2024-11-21 23:22:30,088 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:24:19 2024-11-21 23:24:20,411 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-21 23:24:23,638 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:24:23,638 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:24:14 2024-11-21 23:24:23,638 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:24:23,639 INFO 3318922:repeater task quick_wh300 starting 4.6 seconds late 2024-11-21 23:24:28,958 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-21 23:24:28,958 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:24:14 2024-11-21 23:24:28,958 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:24:28,958 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-21 23:24:29,982 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:24:29,983 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:24:14 2024-11-21 23:24:29,983 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:26:19 2024-11-21 23:26:15,286 INFO 3318922:repeater task plot_tsstats_wh300 starting 0.3 seconds late 2024-11-21 23:26:16,735 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:26:16,735 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-21 23:26:16,735 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:16,735 INFO 3318922:repeater task grab_lastpings_wh300 starting 1.8 seconds late 2024-11-21 23:26:16,892 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-21 23:26:16,892 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:16,892 INFO 3318922:repeater task beamdiagnostics_wh300 starting 1.9 seconds late 2024-11-21 23:26:18,597 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 23:26:18,597 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:18,597 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.6 seconds late 2024-11-21 23:26:20,321 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 23:26:20,321 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:20,322 INFO 3318922:repeater task lastensq_os150nb starting 1.3 seconds late 2024-11-21 23:26:22,918 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-21 23:26:22,918 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:26:14 2024-11-21 23:26:22,918 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:31:19 2024-11-21 23:26:22,919 INFO 3318922:repeater task quick_os150nb starting 3.9 seconds late 2024-11-21 23:26:26,719 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-21 23:26:26,719 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:26:14 2024-11-21 23:26:26,719 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:31:19 2024-11-21 23:26:26,719 INFO 3318922:repeater task lastfew_vec_os150nb starting 7.7 seconds late 2024-11-21 23:26:27,733 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:26:27,734 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:26:14 2024-11-21 23:26:27,734 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:31:19 2024-11-21 23:26:27,734 INFO 3318922:repeater task 3dayplots_os150nb starting 7.8 seconds late 2024-11-21 23:26:33,765 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 23:26:33,765 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:31:19 2024-11-21 23:26:33,765 INFO 3318922:repeater task plot_tsstats_os150nb starting 18.8 seconds late 2024-11-21 23:26:35,019 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:26:35,019 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-21 23:26:35,019 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:35,019 INFO 3318922:repeater task grab_lastpings_os150nb starting 20.0 seconds late 2024-11-21 23:26:35,181 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-21 23:26:35,181 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:35,182 INFO 3318922:repeater task beamdiagnostics_os150nb starting 20.2 seconds late 2024-11-21 23:26:36,930 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:26:36,930 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:36,931 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.0 seconds late 2024-11-21 23:26:38,635 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:26:38,635 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:38,635 INFO 3318922:repeater task lastensq_os38nb starting 17.6 seconds late 2024-11-21 23:26:41,472 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-21 23:26:41,472 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:26:16 2024-11-21 23:26:41,472 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:31:21 2024-11-21 23:26:41,472 INFO 3318922:repeater task quick_os38nb starting 20.5 seconds late 2024-11-21 23:26:45,465 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-21 23:26:45,465 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:26:16 2024-11-21 23:26:45,465 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:31:21 2024-11-21 23:26:45,466 INFO 3318922:repeater task lastfew_vec_os38nb starting 24.5 seconds late 2024-11-21 23:26:46,518 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.1 seconds 2024-11-21 23:26:46,518 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:26:16 2024-11-21 23:26:46,518 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:31:21 2024-11-21 23:26:46,518 INFO 3318922:repeater task 3dayplots_os38nb starting 26.5 seconds late 2024-11-21 23:26:52,537 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 23:26:52,537 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:31:19 2024-11-21 23:26:52,537 INFO 3318922:repeater task plot_tsstats_os38nb starting 37.6 seconds late 2024-11-21 23:26:53,804 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:26:53,804 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-21 23:26:53,804 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:53,805 INFO 3318922:repeater task grab_lastpings_os38nb starting 38.8 seconds late 2024-11-21 23:26:53,967 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-21 23:26:53,968 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:53,968 INFO 3318922:repeater task beamdiagnostics_os38nb starting 39.0 seconds late 2024-11-21 23:26:55,710 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-21 23:26:55,711 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:55,711 INFO 3318922:repeater task beamdiagnostics_os38nb starting 40.7 seconds late 2024-11-21 23:26:57,523 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-21 23:26:57,523 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:41:14 2024-11-21 23:26:57,524 INFO 3318922:repeater task daily_netcdf starting 37.5 seconds late 2024-11-21 23:26:58,370 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 23:26:58,370 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:26:58,370 INFO 3318922:repeater task ggatime_diagnostics starting 43.4 seconds late 2024-11-21 23:27:06,150 INFO 3318922:repeater task ggatime_diagnostics ended after 7.8 seconds 2024-11-21 23:27:06,150 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:27:06,150 INFO 3318922:repeater task ggatime_diagnostics starting 51.2 seconds late 2024-11-21 23:27:13,620 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-21 23:27:13,620 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:27:13,621 INFO 3318922:repeater task ggatime_diagnostics starting 58.6 seconds late 2024-11-21 23:27:21,140 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-21 23:27:21,141 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:27:21,141 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 66.0 seconds late 2024-11-21 23:27:22,916 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-21 23:27:22,916 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-21 23:27:22,916 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/21 23:41:15 2024-11-21 23:27:22,916 INFO 3318922:repeater task plot_hbins starting 67.8 seconds late 2024-11-21 23:27:27,062 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-21 23:27:27,062 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-21 23:27:27,062 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/21 23:41:15 2024-11-21 23:27:27,062 INFO 3318922:repeater task report_generator starting 71.9 seconds late 2024-11-21 23:27:31,673 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 4.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-21 23:27:31,673 INFO 3318922:repeater task report_generator ended after 4.6 seconds 2024-11-21 23:27:31,673 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/22 00:26:15 2024-11-21 23:27:36,679 INFO 3318922:repeater task noisy_rawtemps_os150 starting 76.7 seconds late 2024-11-21 23:27:39,712 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2024-11-21 23:27:39,712 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:27:39,713 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 79.7 seconds late 2024-11-21 23:27:42,891 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-21 23:27:42,891 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:27:42,892 INFO 3318922:repeater task noisy_rawtemps_os38 starting 82.9 seconds late 2024-11-21 23:27:46,407 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-21 23:27:46,408 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:27:46,408 INFO 3318922:repeater task lastensq_wh300 starting 87.4 seconds late 2024-11-21 23:27:49,701 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:27:49,701 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:26:14 2024-11-21 23:27:49,701 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:28:19 2024-11-21 23:27:49,702 INFO 3318922:repeater task quick_wh300 starting 90.7 seconds late 2024-11-21 23:27:54,954 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-21 23:27:54,954 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:26:14 2024-11-21 23:27:54,954 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:28:19 2024-11-21 23:27:54,955 INFO 3318922:repeater task lastfew_vec_wh300 starting 96.0 seconds late 2024-11-21 23:27:55,988 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:27:55,988 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:26:14 2024-11-21 23:27:55,988 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:28:19 2024-11-21 23:27:55,989 INFO 3318922:repeater task 3dayplots_wh300 starting 96.0 seconds late 2024-11-21 23:28:02,281 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:28:02,282 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:31:19 2024-11-21 23:28:22,327 INFO 3318922:repeater task lastensq_wh300 starting 3.3 seconds late 2024-11-21 23:28:25,590 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:28:25,591 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:28:14 2024-11-21 23:28:25,591 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:30:19 2024-11-21 23:28:25,591 INFO 3318922:repeater task quick_wh300 starting 6.6 seconds late 2024-11-21 23:28:30,915 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-21 23:28:30,915 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:28:14 2024-11-21 23:28:30,915 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:30:19 2024-11-21 23:28:30,915 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2024-11-21 23:28:31,990 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:28:31,991 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:28:14 2024-11-21 23:28:31,991 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:30:19 2024-11-21 23:30:22,313 INFO 3318922:repeater task lastensq_wh300 starting 3.3 seconds late 2024-11-21 23:30:25,650 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:30:25,650 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:30:14 2024-11-21 23:30:25,650 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:32:19 2024-11-21 23:30:25,650 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-21 23:30:31,093 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:30:31,093 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:30:14 2024-11-21 23:30:31,094 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:32:19 2024-11-21 23:30:31,094 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-21 23:30:32,138 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:30:32,138 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:30:14 2024-11-21 23:30:32,139 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:32:19 2024-11-21 23:31:22,261 INFO 3318922:repeater task 3dayplots_wh300 starting 2.3 seconds late 2024-11-21 23:31:28,705 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-21 23:31:28,705 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:31:28,706 INFO 3318922:repeater task lastensq_os150nb starting 9.7 seconds late 2024-11-21 23:31:31,313 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-21 23:31:31,313 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:31:14 2024-11-21 23:31:31,313 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:31:31,313 INFO 3318922:repeater task quick_os150nb starting 12.3 seconds late 2024-11-21 23:31:34,956 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-21 23:31:34,956 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:31:14 2024-11-21 23:31:34,956 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:31:34,957 INFO 3318922:repeater task lastfew_vec_os150nb starting 16.0 seconds late 2024-11-21 23:31:35,986 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:31:35,986 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:31:14 2024-11-21 23:31:35,986 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:31:35,987 INFO 3318922:repeater task 3dayplots_os150nb starting 16.0 seconds late 2024-11-21 23:31:42,043 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-21 23:31:42,044 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:31:42,045 INFO 3318922:repeater task lastensq_os38nb starting 21.0 seconds late 2024-11-21 23:31:44,603 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-21 23:31:44,603 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:31:16 2024-11-21 23:31:44,604 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:36:21 2024-11-21 23:31:44,604 INFO 3318922:repeater task quick_os38nb starting 23.6 seconds late 2024-11-21 23:31:48,633 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-21 23:31:48,633 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:31:16 2024-11-21 23:31:48,633 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:36:21 2024-11-21 23:31:48,633 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.6 seconds late 2024-11-21 23:31:49,641 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:31:49,642 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:31:16 2024-11-21 23:31:49,642 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:36:21 2024-11-21 23:31:49,642 INFO 3318922:repeater task 3dayplots_os38nb starting 29.7 seconds late 2024-11-21 23:31:55,673 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 23:31:55,673 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:32:20,731 INFO 3318922:repeater task lastensq_wh300 starting 1.7 seconds late 2024-11-21 23:32:24,007 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:32:24,007 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:32:14 2024-11-21 23:32:24,007 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:34:19 2024-11-21 23:32:24,008 INFO 3318922:repeater task quick_wh300 starting 5.0 seconds late 2024-11-21 23:32:29,476 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 23:32:29,476 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:32:14 2024-11-21 23:32:29,477 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:34:19 2024-11-21 23:32:29,477 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-21 23:32:30,523 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:32:30,523 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:32:14 2024-11-21 23:32:30,524 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:34:19 2024-11-21 23:34:20,844 INFO 3318922:repeater task lastensq_wh300 starting 1.8 seconds late 2024-11-21 23:34:24,055 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:34:24,056 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:34:14 2024-11-21 23:34:24,056 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:34:24,056 INFO 3318922:repeater task quick_wh300 starting 5.1 seconds late 2024-11-21 23:34:29,246 INFO 3318922:repeater task quick_wh300 ended after 5.2 seconds 2024-11-21 23:34:29,247 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:34:14 2024-11-21 23:34:29,247 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:34:29,247 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2024-11-21 23:34:30,301 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:34:30,302 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:34:14 2024-11-21 23:34:30,302 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:36:19 2024-11-21 23:36:20,614 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.6 seconds late 2024-11-21 23:36:23,677 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-21 23:36:23,677 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:36:23,677 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.7 seconds late 2024-11-21 23:36:26,984 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.3 seconds 2024-11-21 23:36:26,984 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:36:26,984 INFO 3318922:repeater task noisy_rawtemps_os38 starting 7.0 seconds late 2024-11-21 23:36:30,611 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-21 23:36:30,611 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:36:30,611 INFO 3318922:repeater task lastensq_wh300 starting 11.6 seconds late 2024-11-21 23:36:33,927 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:36:33,928 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:36:14 2024-11-21 23:36:33,928 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:38:19 2024-11-21 23:36:33,928 INFO 3318922:repeater task quick_wh300 starting 14.9 seconds late 2024-11-21 23:36:39,256 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-21 23:36:39,256 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:36:14 2024-11-21 23:36:39,257 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:38:19 2024-11-21 23:36:39,257 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.3 seconds late 2024-11-21 23:36:40,293 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:36:40,293 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:36:14 2024-11-21 23:36:40,294 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:38:19 2024-11-21 23:36:40,294 INFO 3318922:repeater task 3dayplots_wh300 starting 20.3 seconds late 2024-11-21 23:36:46,697 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-21 23:36:46,697 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:41:19 2024-11-21 23:36:46,698 INFO 3318922:repeater task lastensq_os150nb starting 27.7 seconds late 2024-11-21 23:36:49,176 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-21 23:36:49,176 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:36:14 2024-11-21 23:36:49,176 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:41:19 2024-11-21 23:36:49,177 INFO 3318922:repeater task quick_os150nb starting 30.2 seconds late 2024-11-21 23:36:52,778 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-21 23:36:52,779 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:36:14 2024-11-21 23:36:52,779 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:41:19 2024-11-21 23:36:52,779 INFO 3318922:repeater task lastfew_vec_os150nb starting 33.8 seconds late 2024-11-21 23:36:53,789 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:36:53,790 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:36:14 2024-11-21 23:36:53,790 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:41:19 2024-11-21 23:36:53,790 INFO 3318922:repeater task 3dayplots_os150nb starting 33.8 seconds late 2024-11-21 23:36:59,767 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 23:36:59,767 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:41:19 2024-11-21 23:36:59,768 INFO 3318922:repeater task lastensq_os38nb starting 38.8 seconds late 2024-11-21 23:37:02,270 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-21 23:37:02,270 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:36:16 2024-11-21 23:37:02,270 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:41:21 2024-11-21 23:37:02,271 INFO 3318922:repeater task quick_os38nb starting 41.3 seconds late 2024-11-21 23:37:06,216 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-21 23:37:06,216 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:36:16 2024-11-21 23:37:06,216 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:41:21 2024-11-21 23:37:06,216 INFO 3318922:repeater task lastfew_vec_os38nb starting 45.2 seconds late 2024-11-21 23:37:07,245 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:37:07,245 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:36:16 2024-11-21 23:37:07,245 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:41:21 2024-11-21 23:37:07,246 INFO 3318922:repeater task 3dayplots_os38nb starting 47.3 seconds late 2024-11-21 23:37:13,204 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 23:37:13,205 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:41:19 2024-11-21 23:37:13,206 INFO 3318922:repeater task daily_netcdf starting 53.2 seconds late 2024-11-21 23:37:13,990 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 23:37:13,990 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:38:19,183 INFO 3318922:repeater task lastensq_wh300 starting 0.2 seconds late 2024-11-21 23:38:22,452 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:38:22,452 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:38:14 2024-11-21 23:38:22,452 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:40:19 2024-11-21 23:38:22,453 INFO 3318922:repeater task quick_wh300 starting 3.5 seconds late 2024-11-21 23:38:27,986 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 23:38:27,986 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:38:14 2024-11-21 23:38:27,986 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:40:19 2024-11-21 23:38:27,986 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.0 seconds late 2024-11-21 23:38:29,021 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:38:29,021 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:38:14 2024-11-21 23:38:29,021 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:40:19 2024-11-21 23:40:19,337 INFO 3318922:repeater task lastensq_wh300 starting 0.3 seconds late 2024-11-21 23:40:22,566 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:40:22,567 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:40:14 2024-11-21 23:40:22,567 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:42:19 2024-11-21 23:40:22,567 INFO 3318922:repeater task quick_wh300 starting 3.6 seconds late 2024-11-21 23:40:28,011 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:40:28,012 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:40:14 2024-11-21 23:40:28,012 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:42:19 2024-11-21 23:40:28,012 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.0 seconds late 2024-11-21 23:40:29,068 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:40:29,068 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:40:14 2024-11-21 23:40:29,068 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:42:19 2024-11-21 23:41:19,183 INFO 3318922:repeater task plot_tsstats_wh300 starting 4.2 seconds late 2024-11-21 23:41:20,614 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:41:20,614 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-21 23:41:20,614 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:20,614 INFO 3318922:repeater task grab_lastpings_wh300 starting 5.6 seconds late 2024-11-21 23:41:20,774 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-21 23:41:20,775 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:20,775 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.8 seconds late 2024-11-21 23:41:22,524 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 23:41:22,524 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:22,524 INFO 3318922:repeater task beamdiagnostics_wh300 starting 7.5 seconds late 2024-11-21 23:41:24,270 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 23:41:24,272 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:24,273 INFO 3318922:repeater task lastensq_os150nb starting 5.3 seconds late 2024-11-21 23:41:26,995 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-21 23:41:26,996 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:41:14 2024-11-21 23:41:26,996 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:41:26,996 INFO 3318922:repeater task quick_os150nb starting 8.0 seconds late 2024-11-21 23:41:30,534 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-21 23:41:30,534 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:41:14 2024-11-21 23:41:30,534 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:41:30,534 INFO 3318922:repeater task lastfew_vec_os150nb starting 11.5 seconds late 2024-11-21 23:41:31,632 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-21 23:41:31,634 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:41:14 2024-11-21 23:41:31,634 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:41:31,634 INFO 3318922:repeater task 3dayplots_os150nb starting 11.7 seconds late 2024-11-21 23:41:37,742 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-21 23:41:37,742 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:41:37,743 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.8 seconds late 2024-11-21 23:41:38,996 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:41:38,996 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-21 23:41:38,996 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:38,996 INFO 3318922:repeater task grab_lastpings_os150nb starting 24.0 seconds late 2024-11-21 23:41:39,158 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-21 23:41:39,158 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:39,159 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.2 seconds late 2024-11-21 23:41:40,851 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:41:40,851 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:40,852 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.9 seconds late 2024-11-21 23:41:42,543 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:41:42,543 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:42,543 INFO 3318922:repeater task lastensq_os38nb starting 21.5 seconds late 2024-11-21 23:41:45,070 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-21 23:41:45,070 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:41:16 2024-11-21 23:41:45,070 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:46:21 2024-11-21 23:41:45,070 INFO 3318922:repeater task quick_os38nb starting 24.1 seconds late 2024-11-21 23:41:48,958 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-21 23:41:48,959 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:41:16 2024-11-21 23:41:48,959 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:46:21 2024-11-21 23:41:48,959 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.0 seconds late 2024-11-21 23:41:49,987 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:41:49,988 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:41:16 2024-11-21 23:41:49,988 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:46:21 2024-11-21 23:41:49,988 INFO 3318922:repeater task 3dayplots_os38nb starting 30.0 seconds late 2024-11-21 23:41:56,162 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-21 23:41:56,162 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:41:56,162 INFO 3318922:repeater task plot_tsstats_os38nb starting 41.2 seconds late 2024-11-21 23:41:57,438 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:41:57,438 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-21 23:41:57,438 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:57,438 INFO 3318922:repeater task grab_lastpings_os38nb starting 42.5 seconds late 2024-11-21 23:41:57,599 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-21 23:41:57,599 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:57,600 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.6 seconds late 2024-11-21 23:41:59,295 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-21 23:41:59,295 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:41:59,296 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.3 seconds late 2024-11-21 23:42:01,033 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-21 23:42:01,035 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/21 23:56:14 2024-11-21 23:42:01,037 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 45.9 seconds late 2024-11-21 23:42:02,941 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-21 23:42:02,941 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-21 23:42:02,941 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/21 23:56:15 2024-11-21 23:42:02,941 INFO 3318922:repeater task plot_hbins starting 47.8 seconds late 2024-11-21 23:42:07,253 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-21 23:42:07,253 INFO 3318922:repeater task plot_hbins ended after 4.3 seconds 2024-11-21 23:42:07,253 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/21 23:56:15 2024-11-21 23:42:12,259 INFO 3318922:repeater task 3dayplots_wh300 starting 52.3 seconds late 2024-11-21 23:42:18,648 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-21 23:42:18,648 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:42:23,655 INFO 3318922:repeater task lastensq_wh300 starting 4.7 seconds late 2024-11-21 23:42:26,998 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:42:26,998 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:42:14 2024-11-21 23:42:26,998 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:44:19 2024-11-21 23:42:26,999 INFO 3318922:repeater task quick_wh300 starting 8.0 seconds late 2024-11-21 23:42:32,421 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:42:32,421 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:42:14 2024-11-21 23:42:32,421 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:44:19 2024-11-21 23:42:32,422 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2024-11-21 23:42:33,466 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:42:33,466 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:42:14 2024-11-21 23:42:33,466 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:44:19 2024-11-21 23:44:23,736 INFO 3318922:repeater task lastensq_wh300 starting 4.7 seconds late 2024-11-21 23:44:27,105 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-21 23:44:27,105 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:44:14 2024-11-21 23:44:27,105 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:44:27,106 INFO 3318922:repeater task quick_wh300 starting 8.1 seconds late 2024-11-21 23:44:32,552 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:44:32,552 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:44:14 2024-11-21 23:44:32,552 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:44:32,552 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.6 seconds late 2024-11-21 23:44:33,619 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:44:33,619 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:44:14 2024-11-21 23:44:33,619 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:46:19 2024-11-21 23:46:23,931 INFO 3318922:repeater task noisy_rawtemps_os150 starting 4.0 seconds late 2024-11-21 23:46:27,102 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-21 23:46:27,123 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:46:27,180 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 7.2 seconds late 2024-11-21 23:46:30,369 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-21 23:46:30,369 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:46:30,370 INFO 3318922:repeater task noisy_rawtemps_os38 starting 10.4 seconds late 2024-11-21 23:46:33,972 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-21 23:46:33,972 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:46:33,973 INFO 3318922:repeater task lastensq_wh300 starting 15.0 seconds late 2024-11-21 23:46:37,245 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:46:37,246 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:46:14 2024-11-21 23:46:37,246 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:48:19 2024-11-21 23:46:37,246 INFO 3318922:repeater task quick_wh300 starting 18.2 seconds late 2024-11-21 23:46:42,499 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-21 23:46:42,499 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:46:14 2024-11-21 23:46:42,499 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:48:19 2024-11-21 23:46:42,500 INFO 3318922:repeater task lastfew_vec_wh300 starting 23.5 seconds late 2024-11-21 23:46:43,528 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:46:43,529 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:46:14 2024-11-21 23:46:43,529 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:48:19 2024-11-21 23:46:43,529 INFO 3318922:repeater task 3dayplots_wh300 starting 23.6 seconds late 2024-11-21 23:46:49,851 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:46:49,851 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:51:19 2024-11-21 23:46:49,853 INFO 3318922:repeater task lastensq_os150nb starting 30.9 seconds late 2024-11-21 23:46:52,601 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-21 23:46:52,602 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:46:14 2024-11-21 23:46:52,602 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:51:19 2024-11-21 23:46:52,602 INFO 3318922:repeater task quick_os150nb starting 33.6 seconds late 2024-11-21 23:46:56,379 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-21 23:46:56,379 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:46:14 2024-11-21 23:46:56,379 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:51:19 2024-11-21 23:46:56,380 INFO 3318922:repeater task lastfew_vec_os150nb starting 37.4 seconds late 2024-11-21 23:46:57,398 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:46:57,399 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:46:14 2024-11-21 23:46:57,399 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:51:19 2024-11-21 23:46:57,399 INFO 3318922:repeater task 3dayplots_os150nb starting 37.4 seconds late 2024-11-21 23:47:03,487 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-21 23:47:03,487 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:51:19 2024-11-21 23:47:03,489 INFO 3318922:repeater task lastensq_os38nb starting 42.5 seconds late 2024-11-21 23:47:06,139 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-21 23:47:06,139 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:46:16 2024-11-21 23:47:06,139 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:51:21 2024-11-21 23:47:06,140 INFO 3318922:repeater task quick_os38nb starting 45.1 seconds late 2024-11-21 23:47:10,076 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-21 23:47:10,077 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:46:16 2024-11-21 23:47:10,077 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:51:21 2024-11-21 23:47:10,077 INFO 3318922:repeater task lastfew_vec_os38nb starting 49.1 seconds late 2024-11-21 23:47:11,109 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:47:11,109 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:46:16 2024-11-21 23:47:11,109 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:51:21 2024-11-21 23:47:11,109 INFO 3318922:repeater task 3dayplots_os38nb starting 51.1 seconds late 2024-11-21 23:47:17,164 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-21 23:47:17,164 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:51:19 2024-11-21 23:47:17,166 INFO 3318922:repeater task daily_netcdf starting 57.2 seconds late 2024-11-21 23:47:17,971 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 23:47:17,971 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:48:23,155 INFO 3318922:repeater task lastensq_wh300 starting 4.2 seconds late 2024-11-21 23:48:26,437 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:48:26,438 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:48:14 2024-11-21 23:48:26,438 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:50:19 2024-11-21 23:48:26,438 INFO 3318922:repeater task quick_wh300 starting 7.4 seconds late 2024-11-21 23:48:31,803 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:48:31,804 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:48:14 2024-11-21 23:48:31,804 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:50:19 2024-11-21 23:48:31,804 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2024-11-21 23:48:32,892 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:48:32,892 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:48:14 2024-11-21 23:48:32,892 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:50:19 2024-11-21 23:50:23,212 INFO 3318922:repeater task lastensq_wh300 starting 4.2 seconds late 2024-11-21 23:50:26,484 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:50:26,484 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:50:14 2024-11-21 23:50:26,484 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:52:19 2024-11-21 23:50:26,484 INFO 3318922:repeater task quick_wh300 starting 7.5 seconds late 2024-11-21 23:50:32,280 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-21 23:50:32,281 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:50:14 2024-11-21 23:50:32,281 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:52:19 2024-11-21 23:50:32,281 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.3 seconds late 2024-11-21 23:50:33,379 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-21 23:50:33,379 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:50:14 2024-11-21 23:50:33,379 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:52:19 2024-11-21 23:51:23,533 INFO 3318922:repeater task 3dayplots_wh300 starting 3.6 seconds late 2024-11-21 23:51:29,881 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.3 seconds 2024-11-21 23:51:29,881 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:51:29,883 INFO 3318922:repeater task lastensq_os150nb starting 10.9 seconds late 2024-11-21 23:51:32,516 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-21 23:51:32,516 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:51:14 2024-11-21 23:51:32,516 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:51:32,517 INFO 3318922:repeater task quick_os150nb starting 13.5 seconds late 2024-11-21 23:51:36,050 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-21 23:51:36,051 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:51:14 2024-11-21 23:51:36,051 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:51:36,051 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.1 seconds late 2024-11-21 23:51:37,077 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:51:37,077 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:51:14 2024-11-21 23:51:37,077 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:51:37,077 INFO 3318922:repeater task 3dayplots_os150nb starting 17.1 seconds late 2024-11-21 23:51:43,054 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 23:51:43,055 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:51:43,056 INFO 3318922:repeater task lastensq_os38nb starting 22.1 seconds late 2024-11-21 23:51:45,581 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-21 23:51:45,581 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:51:16 2024-11-21 23:51:45,581 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/21 23:56:21 2024-11-21 23:51:45,582 INFO 3318922:repeater task quick_os38nb starting 24.6 seconds late 2024-11-21 23:51:49,537 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-21 23:51:49,537 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:51:16 2024-11-21 23:51:49,537 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/21 23:56:21 2024-11-21 23:51:49,537 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.5 seconds late 2024-11-21 23:51:50,544 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-21 23:51:50,544 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:51:16 2024-11-21 23:51:50,544 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/21 23:56:21 2024-11-21 23:51:50,544 INFO 3318922:repeater task 3dayplots_os38nb starting 30.6 seconds late 2024-11-21 23:51:56,517 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 23:51:56,517 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:52:21,559 INFO 3318922:repeater task lastensq_wh300 starting 2.6 seconds late 2024-11-21 23:52:24,962 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-21 23:52:24,962 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:52:14 2024-11-21 23:52:24,962 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:54:19 2024-11-21 23:52:24,962 INFO 3318922:repeater task quick_wh300 starting 6.0 seconds late 2024-11-21 23:52:30,415 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-21 23:52:30,415 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:52:14 2024-11-21 23:52:30,415 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:54:19 2024-11-21 23:52:30,416 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2024-11-21 23:52:31,460 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:52:31,461 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:52:14 2024-11-21 23:52:31,461 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:54:19 2024-11-21 23:54:21,759 INFO 3318922:repeater task lastensq_wh300 starting 2.8 seconds late 2024-11-21 23:54:25,086 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:54:25,086 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:54:14 2024-11-21 23:54:25,086 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:54:25,087 INFO 3318922:repeater task quick_wh300 starting 6.1 seconds late 2024-11-21 23:54:30,400 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-21 23:54:30,401 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:54:14 2024-11-21 23:54:30,401 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:54:30,401 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2024-11-21 23:54:31,433 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:54:31,434 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:54:14 2024-11-21 23:54:31,434 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:56:19 2024-11-21 23:56:16,694 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.7 seconds late 2024-11-21 23:56:18,141 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:56:18,141 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-21 23:56:18,141 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:18,141 INFO 3318922:repeater task grab_lastpings_wh300 starting 3.2 seconds late 2024-11-21 23:56:18,302 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-21 23:56:18,302 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:18,302 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.3 seconds late 2024-11-21 23:56:20,036 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 23:56:20,036 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:20,036 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.1 seconds late 2024-11-21 23:56:21,766 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-21 23:56:21,768 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:21,769 INFO 3318922:repeater task lastensq_os150nb starting 2.8 seconds late 2024-11-21 23:56:24,608 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-21 23:56:24,609 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:56:14 2024-11-21 23:56:24,609 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:01:19 2024-11-21 23:56:24,609 INFO 3318922:repeater task quick_os150nb starting 5.6 seconds late 2024-11-21 23:56:28,319 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-21 23:56:28,320 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:56:14 2024-11-21 23:56:28,320 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:01:19 2024-11-21 23:56:28,320 INFO 3318922:repeater task lastfew_vec_os150nb starting 9.3 seconds late 2024-11-21 23:56:29,320 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-21 23:56:29,320 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/21 23:56:14 2024-11-21 23:56:29,320 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:01:19 2024-11-21 23:56:29,321 INFO 3318922:repeater task 3dayplots_os150nb starting 9.3 seconds late 2024-11-21 23:56:35,347 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-21 23:56:35,348 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:01:19 2024-11-21 23:56:35,348 INFO 3318922:repeater task plot_tsstats_os150nb starting 20.4 seconds late 2024-11-21 23:56:36,610 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:56:36,610 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-21 23:56:36,610 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:36,611 INFO 3318922:repeater task grab_lastpings_os150nb starting 21.6 seconds late 2024-11-21 23:56:36,770 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-21 23:56:36,770 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:36,771 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.8 seconds late 2024-11-21 23:56:38,504 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:56:38,504 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:38,504 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.5 seconds late 2024-11-21 23:56:40,229 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-21 23:56:40,229 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:40,230 INFO 3318922:repeater task lastensq_os38nb starting 19.2 seconds late 2024-11-21 23:56:42,820 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-21 23:56:42,821 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:56:16 2024-11-21 23:56:42,821 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:01:21 2024-11-21 23:56:42,821 INFO 3318922:repeater task quick_os38nb starting 21.8 seconds late 2024-11-21 23:56:46,765 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-21 23:56:46,765 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:56:16 2024-11-21 23:56:46,765 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:01:21 2024-11-21 23:56:46,766 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.8 seconds late 2024-11-21 23:56:47,824 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.1 seconds 2024-11-21 23:56:47,824 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/21 23:56:16 2024-11-21 23:56:47,825 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:01:21 2024-11-21 23:56:47,825 INFO 3318922:repeater task 3dayplots_os38nb starting 27.8 seconds late 2024-11-21 23:56:53,820 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.0 seconds 2024-11-21 23:56:53,820 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:01:19 2024-11-21 23:56:53,821 INFO 3318922:repeater task plot_tsstats_os38nb starting 38.8 seconds late 2024-11-21 23:56:55,094 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-21 23:56:55,095 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-21 23:56:55,095 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:55,095 INFO 3318922:repeater task grab_lastpings_os38nb starting 40.1 seconds late 2024-11-21 23:56:55,256 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-21 23:56:55,256 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:55,257 INFO 3318922:repeater task beamdiagnostics_os38nb starting 40.3 seconds late 2024-11-21 23:56:57,020 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-21 23:56:57,020 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:57,020 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.0 seconds late 2024-11-21 23:56:58,730 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-21 23:56:58,730 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:11:14 2024-11-21 23:56:58,731 INFO 3318922:repeater task daily_netcdf starting 38.8 seconds late 2024-11-21 23:56:59,533 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-21 23:56:59,533 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 00:06:19 2024-11-21 23:56:59,534 INFO 3318922:repeater task ggatime_diagnostics starting 44.6 seconds late 2024-11-21 23:57:07,631 INFO 3318922:repeater task ggatime_diagnostics ended after 8.1 seconds 2024-11-21 23:57:07,632 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 00:26:14 2024-11-21 23:57:07,633 INFO 3318922:repeater task ggatime_diagnostics starting 52.7 seconds late 2024-11-21 23:57:15,268 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-21 23:57:15,268 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 00:26:14 2024-11-21 23:57:15,268 INFO 3318922:repeater task ggatime_diagnostics starting 60.3 seconds late 2024-11-21 23:57:22,942 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-21 23:57:22,943 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 00:26:14 2024-11-21 23:57:22,943 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 67.8 seconds late 2024-11-21 23:57:24,797 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-21 23:57:24,797 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-21 23:57:24,797 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 00:11:15 2024-11-21 23:57:24,797 INFO 3318922:repeater task plot_hbins starting 69.7 seconds late 2024-11-21 23:57:28,930 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-21 23:57:28,930 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-21 23:57:28,930 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 00:11:15 2024-11-21 23:57:33,934 INFO 3318922:repeater task noisy_rawtemps_os150 starting 74.0 seconds late 2024-11-21 23:57:36,979 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2024-11-21 23:57:36,979 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 00:06:19 2024-11-21 23:57:36,979 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 77.0 seconds late 2024-11-21 23:57:40,113 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-21 23:57:40,114 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 00:06:19 2024-11-21 23:57:40,114 INFO 3318922:repeater task noisy_rawtemps_os38 starting 80.1 seconds late 2024-11-21 23:57:43,712 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-21 23:57:43,712 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 00:06:19 2024-11-21 23:57:43,713 INFO 3318922:repeater task lastensq_wh300 starting 84.7 seconds late 2024-11-21 23:57:46,944 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-21 23:57:46,944 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:56:14 2024-11-21 23:57:46,944 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/21 23:58:19 2024-11-21 23:57:46,944 INFO 3318922:repeater task quick_wh300 starting 87.9 seconds late 2024-11-21 23:57:52,298 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-21 23:57:52,299 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:56:14 2024-11-21 23:57:52,299 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/21 23:58:19 2024-11-21 23:57:52,299 INFO 3318922:repeater task lastfew_vec_wh300 starting 93.3 seconds late 2024-11-21 23:57:53,347 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:57:53,347 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:56:14 2024-11-21 23:57:53,347 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/21 23:58:19 2024-11-21 23:57:53,348 INFO 3318922:repeater task 3dayplots_wh300 starting 93.4 seconds late 2024-11-21 23:57:59,715 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-21 23:57:59,715 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:01:19 2024-11-21 23:58:19,764 INFO 3318922:repeater task lastensq_wh300 starting 0.8 seconds late 2024-11-21 23:58:23,054 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-21 23:58:23,055 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:58:14 2024-11-21 23:58:23,055 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:00:19 2024-11-21 23:58:23,055 INFO 3318922:repeater task quick_wh300 starting 4.1 seconds late 2024-11-21 23:58:28,336 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-21 23:58:28,336 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:58:14 2024-11-21 23:58:28,336 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:00:19 2024-11-21 23:58:28,337 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2024-11-21 23:58:29,380 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-21 23:58:29,381 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/21 23:58:14 2024-11-21 23:58:29,381 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:00:19 2024-11-22 00:00:19,704 INFO 3318922:repeater task lastensq_wh300 starting 0.7 seconds late 2024-11-22 00:00:23,082 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:00:23,082 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:00:14 2024-11-22 00:00:23,082 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:02:19 2024-11-22 00:00:23,083 INFO 3318922:repeater task quick_wh300 starting 4.1 seconds late 2024-11-22 00:00:28,355 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:00:28,355 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:00:14 2024-11-22 00:00:28,355 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:02:19 2024-11-22 00:00:28,355 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.4 seconds late 2024-11-22 00:00:29,397 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:00:29,398 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:00:14 2024-11-22 00:00:29,398 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:02:19 2024-11-22 00:01:19,546 INFO 3318922:repeater task lastensq_os150nb starting 0.5 seconds late 2024-11-22 00:01:22,227 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:01:22,228 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:01:14 2024-11-22 00:01:22,228 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:01:22,228 INFO 3318922:repeater task quick_os150nb starting 3.2 seconds late 2024-11-22 00:01:25,956 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 00:01:25,957 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:01:14 2024-11-22 00:01:25,957 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:01:25,957 INFO 3318922:repeater task lastfew_vec_os150nb starting 7.0 seconds late 2024-11-22 00:01:26,956 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:01:26,956 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:01:14 2024-11-22 00:01:26,956 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:01:26,956 INFO 3318922:repeater task 3dayplots_os150nb starting 7.0 seconds late 2024-11-22 00:01:32,906 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-22 00:01:32,907 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:01:32,908 INFO 3318922:repeater task lastensq_os38nb starting 11.9 seconds late 2024-11-22 00:01:35,383 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:01:35,383 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:01:16 2024-11-22 00:01:35,383 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:06:21 2024-11-22 00:01:35,383 INFO 3318922:repeater task quick_os38nb starting 14.4 seconds late 2024-11-22 00:01:39,247 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 00:01:39,247 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:01:16 2024-11-22 00:01:39,247 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:06:21 2024-11-22 00:01:39,248 INFO 3318922:repeater task lastfew_vec_os38nb starting 18.2 seconds late 2024-11-22 00:01:40,260 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:01:40,260 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:01:16 2024-11-22 00:01:40,261 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:06:21 2024-11-22 00:01:40,261 INFO 3318922:repeater task 3dayplots_os38nb starting 20.3 seconds late 2024-11-22 00:01:46,197 INFO 3318922:repeater task 3dayplots_os38nb ended after 5.9 seconds 2024-11-22 00:01:46,197 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:01:51,204 INFO 3318922:repeater task 3dayplots_wh300 starting 31.2 seconds late 2024-11-22 00:01:57,611 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-22 00:01:57,611 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:02:22,677 INFO 3318922:repeater task lastensq_wh300 starting 3.7 seconds late 2024-11-22 00:02:26,018 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:02:26,018 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:02:14 2024-11-22 00:02:26,018 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:04:19 2024-11-22 00:02:26,019 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-22 00:02:31,481 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 00:02:31,481 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:02:14 2024-11-22 00:02:31,481 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:04:19 2024-11-22 00:02:31,481 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2024-11-22 00:02:32,526 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:02:32,526 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:02:14 2024-11-22 00:02:32,526 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:04:19 2024-11-22 00:04:22,833 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 00:04:25,984 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 00:04:25,985 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:04:14 2024-11-22 00:04:25,985 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:04:25,985 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-22 00:04:31,315 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:04:31,316 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:04:14 2024-11-22 00:04:31,316 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:04:31,316 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-22 00:04:32,350 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:04:32,350 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:04:14 2024-11-22 00:04:32,350 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:06:19 2024-11-22 00:06:22,670 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.7 seconds late 2024-11-22 00:06:25,729 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 00:06:25,729 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:06:25,730 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.8 seconds late 2024-11-22 00:06:28,948 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 00:06:28,948 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:06:28,949 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.0 seconds late 2024-11-22 00:06:32,488 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 00:06:32,488 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:06:32,489 INFO 3318922:repeater task lastensq_wh300 starting 13.5 seconds late 2024-11-22 00:06:35,769 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:06:35,769 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:06:14 2024-11-22 00:06:35,769 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:08:19 2024-11-22 00:06:35,770 INFO 3318922:repeater task quick_wh300 starting 16.8 seconds late 2024-11-22 00:06:41,288 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 00:06:41,288 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:06:14 2024-11-22 00:06:41,288 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:08:19 2024-11-22 00:06:41,289 INFO 3318922:repeater task lastfew_vec_wh300 starting 22.3 seconds late 2024-11-22 00:06:42,330 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:06:42,330 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:06:14 2024-11-22 00:06:42,330 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:08:19 2024-11-22 00:06:42,331 INFO 3318922:repeater task 3dayplots_wh300 starting 22.4 seconds late 2024-11-22 00:06:48,681 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-22 00:06:48,681 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:11:19 2024-11-22 00:06:48,683 INFO 3318922:repeater task lastensq_os150nb starting 29.7 seconds late 2024-11-22 00:06:51,326 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 00:06:51,327 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:06:14 2024-11-22 00:06:51,327 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:11:19 2024-11-22 00:06:51,327 INFO 3318922:repeater task quick_os150nb starting 32.3 seconds late 2024-11-22 00:06:54,836 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 00:06:54,836 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:06:14 2024-11-22 00:06:54,836 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:11:19 2024-11-22 00:06:54,836 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.8 seconds late 2024-11-22 00:06:55,848 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:06:55,849 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:06:14 2024-11-22 00:06:55,849 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:11:19 2024-11-22 00:06:55,849 INFO 3318922:repeater task 3dayplots_os150nb starting 35.9 seconds late 2024-11-22 00:07:01,877 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-22 00:07:01,877 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:11:19 2024-11-22 00:07:01,878 INFO 3318922:repeater task lastensq_os38nb starting 40.9 seconds late 2024-11-22 00:07:04,394 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:07:04,394 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:06:16 2024-11-22 00:07:04,394 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:11:21 2024-11-22 00:07:04,395 INFO 3318922:repeater task quick_os38nb starting 43.4 seconds late 2024-11-22 00:07:08,202 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 00:07:08,202 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:06:16 2024-11-22 00:07:08,202 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:11:21 2024-11-22 00:07:08,203 INFO 3318922:repeater task lastfew_vec_os38nb starting 47.2 seconds late 2024-11-22 00:07:09,234 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:07:09,234 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:06:16 2024-11-22 00:07:09,234 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:11:21 2024-11-22 00:07:09,235 INFO 3318922:repeater task 3dayplots_os38nb starting 49.3 seconds late 2024-11-22 00:07:15,353 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 00:07:15,354 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:11:19 2024-11-22 00:07:15,355 INFO 3318922:repeater task daily_netcdf starting 55.4 seconds late 2024-11-22 00:07:16,023 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 00:07:16,023 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:08:21,184 INFO 3318922:repeater task lastensq_wh300 starting 2.2 seconds late 2024-11-22 00:08:24,549 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:08:24,549 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:08:14 2024-11-22 00:08:24,549 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:10:19 2024-11-22 00:08:24,550 INFO 3318922:repeater task quick_wh300 starting 5.6 seconds late 2024-11-22 00:08:30,146 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 00:08:30,147 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:08:14 2024-11-22 00:08:30,147 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:10:19 2024-11-22 00:08:30,147 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2024-11-22 00:08:31,209 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 00:08:31,210 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:08:14 2024-11-22 00:08:31,210 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:10:19 2024-11-22 00:10:21,541 INFO 3318922:repeater task lastensq_wh300 starting 2.5 seconds late 2024-11-22 00:10:24,859 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:10:24,859 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:10:14 2024-11-22 00:10:24,859 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:12:19 2024-11-22 00:10:24,859 INFO 3318922:repeater task quick_wh300 starting 5.9 seconds late 2024-11-22 00:10:30,276 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 00:10:30,276 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:10:14 2024-11-22 00:10:30,276 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:12:19 2024-11-22 00:10:30,276 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2024-11-22 00:10:31,318 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:10:31,318 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:10:14 2024-11-22 00:10:31,318 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:12:19 2024-11-22 00:11:16,443 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.5 seconds late 2024-11-22 00:11:17,876 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:11:17,876 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 00:11:17,876 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:17,876 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.9 seconds late 2024-11-22 00:11:18,034 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 00:11:18,034 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:18,035 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.1 seconds late 2024-11-22 00:11:19,718 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 00:11:19,718 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:19,719 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.7 seconds late 2024-11-22 00:11:21,425 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 00:11:21,425 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:21,426 INFO 3318922:repeater task lastensq_os150nb starting 2.4 seconds late 2024-11-22 00:11:24,226 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 00:11:24,226 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:11:14 2024-11-22 00:11:24,226 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:11:24,227 INFO 3318922:repeater task quick_os150nb starting 5.2 seconds late 2024-11-22 00:11:27,800 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 00:11:27,800 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:11:14 2024-11-22 00:11:27,801 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:11:27,801 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.8 seconds late 2024-11-22 00:11:28,815 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:11:28,816 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:11:14 2024-11-22 00:11:28,816 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:11:28,816 INFO 3318922:repeater task 3dayplots_os150nb starting 8.8 seconds late 2024-11-22 00:11:34,841 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.0 seconds 2024-11-22 00:11:34,842 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:11:34,842 INFO 3318922:repeater task plot_tsstats_os150nb starting 19.9 seconds late 2024-11-22 00:11:36,101 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:11:36,101 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 00:11:36,101 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:36,101 INFO 3318922:repeater task grab_lastpings_os150nb starting 21.1 seconds late 2024-11-22 00:11:36,262 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 00:11:36,263 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:36,263 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.3 seconds late 2024-11-22 00:11:37,999 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:11:37,999 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:37,999 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.0 seconds late 2024-11-22 00:11:39,662 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:11:39,662 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:39,663 INFO 3318922:repeater task lastensq_os38nb starting 18.7 seconds late 2024-11-22 00:11:42,180 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:11:42,180 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:11:16 2024-11-22 00:11:42,181 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:16:21 2024-11-22 00:11:42,181 INFO 3318922:repeater task quick_os38nb starting 21.2 seconds late 2024-11-22 00:11:46,007 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 00:11:46,008 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:11:16 2024-11-22 00:11:46,008 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:16:21 2024-11-22 00:11:46,008 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.0 seconds late 2024-11-22 00:11:47,024 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:11:47,024 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:11:16 2024-11-22 00:11:47,024 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:16:21 2024-11-22 00:11:47,025 INFO 3318922:repeater task 3dayplots_os38nb starting 27.0 seconds late 2024-11-22 00:11:53,213 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 00:11:53,213 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:11:53,214 INFO 3318922:repeater task plot_tsstats_os38nb starting 38.2 seconds late 2024-11-22 00:11:54,494 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:11:54,494 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 00:11:54,494 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:54,494 INFO 3318922:repeater task grab_lastpings_os38nb starting 39.5 seconds late 2024-11-22 00:11:54,656 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 00:11:54,657 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:54,657 INFO 3318922:repeater task beamdiagnostics_os38nb starting 39.7 seconds late 2024-11-22 00:11:56,351 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 00:11:56,351 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:56,351 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.4 seconds late 2024-11-22 00:11:58,100 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 00:11:58,102 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:26:14 2024-11-22 00:11:58,104 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 43.0 seconds late 2024-11-22 00:11:59,944 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 00:11:59,944 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 00:11:59,944 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 00:26:15 2024-11-22 00:11:59,944 INFO 3318922:repeater task plot_hbins starting 44.8 seconds late 2024-11-22 00:12:04,034 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 00:12:04,034 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 00:12:04,034 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 00:26:15 2024-11-22 00:12:09,042 INFO 3318922:repeater task 3dayplots_wh300 starting 49.1 seconds late 2024-11-22 00:12:15,398 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.4 seconds 2024-11-22 00:12:15,398 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:12:20,407 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 00:12:23,611 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 00:12:23,611 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:12:14 2024-11-22 00:12:23,611 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:14:19 2024-11-22 00:12:23,611 INFO 3318922:repeater task quick_wh300 starting 4.6 seconds late 2024-11-22 00:12:29,026 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 00:12:29,026 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:12:14 2024-11-22 00:12:29,027 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:14:19 2024-11-22 00:12:29,027 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-22 00:12:30,087 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 00:12:30,087 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:12:14 2024-11-22 00:12:30,088 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:14:19 2024-11-22 00:14:20,372 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 00:14:23,701 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:14:23,701 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:14:14 2024-11-22 00:14:23,701 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:14:23,702 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 00:14:28,923 INFO 3318922:repeater task quick_wh300 ended after 5.2 seconds 2024-11-22 00:14:28,923 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:14:14 2024-11-22 00:14:28,923 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:14:28,924 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2024-11-22 00:14:29,956 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:14:29,956 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:14:14 2024-11-22 00:14:29,956 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:16:19 2024-11-22 00:16:20,290 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.3 seconds late 2024-11-22 00:16:23,358 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 00:16:23,359 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:16:23,359 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.4 seconds late 2024-11-22 00:16:26,562 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 00:16:26,562 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:16:26,562 INFO 3318922:repeater task noisy_rawtemps_os38 starting 6.6 seconds late 2024-11-22 00:16:30,126 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 00:16:30,126 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:16:30,127 INFO 3318922:repeater task lastensq_wh300 starting 11.1 seconds late 2024-11-22 00:16:33,480 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:16:33,480 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:16:14 2024-11-22 00:16:33,480 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:18:19 2024-11-22 00:16:33,480 INFO 3318922:repeater task quick_wh300 starting 14.5 seconds late 2024-11-22 00:16:38,915 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 00:16:38,915 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:16:14 2024-11-22 00:16:38,915 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:18:19 2024-11-22 00:16:38,915 INFO 3318922:repeater task lastfew_vec_wh300 starting 19.9 seconds late 2024-11-22 00:16:39,952 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:16:39,953 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:16:14 2024-11-22 00:16:39,953 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:18:19 2024-11-22 00:16:39,953 INFO 3318922:repeater task 3dayplots_wh300 starting 20.0 seconds late 2024-11-22 00:16:46,484 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 00:16:46,484 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:21:19 2024-11-22 00:16:46,486 INFO 3318922:repeater task lastensq_os150nb starting 27.5 seconds late 2024-11-22 00:16:49,174 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:16:49,174 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:16:14 2024-11-22 00:16:49,174 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:21:19 2024-11-22 00:16:49,175 INFO 3318922:repeater task quick_os150nb starting 30.2 seconds late 2024-11-22 00:16:52,706 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 00:16:52,706 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:16:14 2024-11-22 00:16:52,706 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:21:19 2024-11-22 00:16:52,707 INFO 3318922:repeater task lastfew_vec_os150nb starting 33.7 seconds late 2024-11-22 00:16:53,717 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:16:53,717 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:16:14 2024-11-22 00:16:53,717 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:21:19 2024-11-22 00:16:53,717 INFO 3318922:repeater task 3dayplots_os150nb starting 33.7 seconds late 2024-11-22 00:16:59,817 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-22 00:16:59,817 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:21:19 2024-11-22 00:16:59,818 INFO 3318922:repeater task lastensq_os38nb starting 38.8 seconds late 2024-11-22 00:17:02,349 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:17:02,349 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:16:16 2024-11-22 00:17:02,349 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:21:21 2024-11-22 00:17:02,350 INFO 3318922:repeater task quick_os38nb starting 41.4 seconds late 2024-11-22 00:17:06,472 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 00:17:06,472 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:16:16 2024-11-22 00:17:06,473 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:21:21 2024-11-22 00:17:06,473 INFO 3318922:repeater task lastfew_vec_os38nb starting 45.5 seconds late 2024-11-22 00:17:07,510 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:17:07,510 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:16:16 2024-11-22 00:17:07,510 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:21:21 2024-11-22 00:17:07,511 INFO 3318922:repeater task 3dayplots_os38nb starting 47.5 seconds late 2024-11-22 00:17:13,676 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 00:17:13,676 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:21:19 2024-11-22 00:17:13,677 INFO 3318922:repeater task daily_netcdf starting 53.7 seconds late 2024-11-22 00:17:14,400 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 00:17:14,400 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:18:19,651 INFO 3318922:repeater task lastensq_wh300 starting 0.7 seconds late 2024-11-22 00:18:22,947 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:18:22,947 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:18:14 2024-11-22 00:18:22,947 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:20:19 2024-11-22 00:18:22,947 INFO 3318922:repeater task quick_wh300 starting 3.9 seconds late 2024-11-22 00:18:28,243 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:18:28,243 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:18:14 2024-11-22 00:18:28,243 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:20:19 2024-11-22 00:18:28,243 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2024-11-22 00:18:29,281 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:18:29,282 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:18:14 2024-11-22 00:18:29,282 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:20:19 2024-11-22 00:20:19,632 INFO 3318922:repeater task lastensq_wh300 starting 0.6 seconds late 2024-11-22 00:20:22,995 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:20:22,995 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:20:14 2024-11-22 00:20:22,996 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:22:19 2024-11-22 00:20:22,996 INFO 3318922:repeater task quick_wh300 starting 4.0 seconds late 2024-11-22 00:20:28,345 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:20:28,346 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:20:14 2024-11-22 00:20:28,346 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:22:19 2024-11-22 00:20:28,346 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2024-11-22 00:20:29,385 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:20:29,385 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:20:14 2024-11-22 00:20:29,385 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:22:19 2024-11-22 00:21:19,539 INFO 3318922:repeater task lastensq_os150nb starting 0.5 seconds late 2024-11-22 00:21:22,170 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 00:21:22,171 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:21:14 2024-11-22 00:21:22,171 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:21:22,171 INFO 3318922:repeater task quick_os150nb starting 3.2 seconds late 2024-11-22 00:21:25,980 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 00:21:25,981 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:21:14 2024-11-22 00:21:25,981 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:21:25,981 INFO 3318922:repeater task lastfew_vec_os150nb starting 7.0 seconds late 2024-11-22 00:21:26,984 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:21:26,984 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:21:14 2024-11-22 00:21:26,984 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:21:26,984 INFO 3318922:repeater task 3dayplots_os150nb starting 7.0 seconds late 2024-11-22 00:21:33,219 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 00:21:33,219 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:21:33,220 INFO 3318922:repeater task lastensq_os38nb starting 12.2 seconds late 2024-11-22 00:21:35,727 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:21:35,727 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:21:16 2024-11-22 00:21:35,727 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:26:21 2024-11-22 00:21:35,727 INFO 3318922:repeater task quick_os38nb starting 14.7 seconds late 2024-11-22 00:21:39,528 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 00:21:39,529 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:21:16 2024-11-22 00:21:39,529 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:26:21 2024-11-22 00:21:39,529 INFO 3318922:repeater task lastfew_vec_os38nb starting 18.5 seconds late 2024-11-22 00:21:40,556 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:21:40,557 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:21:16 2024-11-22 00:21:40,557 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:26:21 2024-11-22 00:21:40,557 INFO 3318922:repeater task 3dayplots_os38nb starting 20.6 seconds late 2024-11-22 00:21:46,693 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 00:21:46,693 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:21:51,703 INFO 3318922:repeater task 3dayplots_wh300 starting 31.7 seconds late 2024-11-22 00:21:58,287 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 00:21:58,288 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:22:23,352 INFO 3318922:repeater task lastensq_wh300 starting 4.4 seconds late 2024-11-22 00:22:26,769 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:22:26,770 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:22:14 2024-11-22 00:22:26,770 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:24:19 2024-11-22 00:22:26,770 INFO 3318922:repeater task quick_wh300 starting 7.8 seconds late 2024-11-22 00:22:32,050 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:22:32,050 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:22:14 2024-11-22 00:22:32,050 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:24:19 2024-11-22 00:22:32,050 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2024-11-22 00:22:33,084 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:22:33,084 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:22:14 2024-11-22 00:22:33,084 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:24:19 2024-11-22 00:24:23,408 INFO 3318922:repeater task lastensq_wh300 starting 4.4 seconds late 2024-11-22 00:24:26,672 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:24:26,672 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:24:14 2024-11-22 00:24:26,672 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:24:26,673 INFO 3318922:repeater task quick_wh300 starting 7.7 seconds late 2024-11-22 00:24:32,260 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 00:24:32,261 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:24:14 2024-11-22 00:24:32,261 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:24:32,261 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.3 seconds late 2024-11-22 00:24:33,328 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 00:24:33,329 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:24:14 2024-11-22 00:24:33,329 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:26:19 2024-11-22 00:26:18,657 INFO 3318922:repeater task plot_tsstats_wh300 starting 3.7 seconds late 2024-11-22 00:26:20,095 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:26:20,095 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 00:26:20,095 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:20,096 INFO 3318922:repeater task grab_lastpings_wh300 starting 5.1 seconds late 2024-11-22 00:26:20,258 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 00:26:20,258 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:20,258 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.3 seconds late 2024-11-22 00:26:21,995 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 00:26:21,995 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:21,995 INFO 3318922:repeater task beamdiagnostics_wh300 starting 7.0 seconds late 2024-11-22 00:26:23,698 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 00:26:23,699 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:23,699 INFO 3318922:repeater task lastensq_os150nb starting 4.7 seconds late 2024-11-22 00:26:26,370 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:26:26,370 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:26:14 2024-11-22 00:26:26,370 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:31:19 2024-11-22 00:26:26,370 INFO 3318922:repeater task quick_os150nb starting 7.4 seconds late 2024-11-22 00:26:30,040 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 00:26:30,040 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:26:14 2024-11-22 00:26:30,040 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:31:19 2024-11-22 00:26:30,041 INFO 3318922:repeater task lastfew_vec_os150nb starting 11.0 seconds late 2024-11-22 00:26:31,069 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:26:31,070 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:26:14 2024-11-22 00:26:31,070 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:31:19 2024-11-22 00:26:31,070 INFO 3318922:repeater task 3dayplots_os150nb starting 11.1 seconds late 2024-11-22 00:26:37,455 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 00:26:37,455 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:31:19 2024-11-22 00:26:37,455 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.5 seconds late 2024-11-22 00:26:38,710 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:26:38,710 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 00:26:38,710 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:38,710 INFO 3318922:repeater task grab_lastpings_os150nb starting 23.7 seconds late 2024-11-22 00:26:38,870 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 00:26:38,870 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:38,870 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.9 seconds late 2024-11-22 00:26:40,582 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:26:40,582 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:40,582 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.6 seconds late 2024-11-22 00:26:42,319 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:26:42,319 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:42,320 INFO 3318922:repeater task lastensq_os38nb starting 21.3 seconds late 2024-11-22 00:26:44,860 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:26:44,860 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:26:16 2024-11-22 00:26:44,860 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:31:21 2024-11-22 00:26:44,861 INFO 3318922:repeater task quick_os38nb starting 23.9 seconds late 2024-11-22 00:26:48,493 INFO 3318922:repeater task quick_os38nb ended after 3.6 seconds 2024-11-22 00:26:48,494 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:26:16 2024-11-22 00:26:48,494 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:31:21 2024-11-22 00:26:48,494 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.5 seconds late 2024-11-22 00:26:49,499 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:26:49,500 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:26:16 2024-11-22 00:26:49,500 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:31:21 2024-11-22 00:26:49,500 INFO 3318922:repeater task 3dayplots_os38nb starting 29.5 seconds late 2024-11-22 00:26:55,857 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.4 seconds 2024-11-22 00:26:55,857 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:31:19 2024-11-22 00:26:55,857 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.9 seconds late 2024-11-22 00:26:57,142 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:26:57,143 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 00:26:57,143 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:57,143 INFO 3318922:repeater task grab_lastpings_os38nb starting 42.2 seconds late 2024-11-22 00:26:57,303 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 00:26:57,303 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:57,303 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.3 seconds late 2024-11-22 00:26:58,983 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 00:26:58,984 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:26:58,984 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.0 seconds late 2024-11-22 00:27:00,756 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 00:27:00,756 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:41:14 2024-11-22 00:27:00,756 INFO 3318922:repeater task daily_netcdf starting 40.8 seconds late 2024-11-22 00:27:01,495 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 00:27:01,495 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:27:01,495 INFO 3318922:repeater task ggatime_diagnostics starting 46.5 seconds late 2024-11-22 00:27:09,188 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 00:27:09,188 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:27:09,188 INFO 3318922:repeater task ggatime_diagnostics starting 54.2 seconds late 2024-11-22 00:27:16,681 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 00:27:16,681 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:27:16,681 INFO 3318922:repeater task ggatime_diagnostics starting 61.7 seconds late 2024-11-22 00:27:24,237 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 00:27:24,237 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:27:24,237 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 69.1 seconds late 2024-11-22 00:27:26,087 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 00:27:26,087 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 00:27:26,087 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 00:41:15 2024-11-22 00:27:26,088 INFO 3318922:repeater task plot_hbins starting 71.0 seconds late 2024-11-22 00:27:30,190 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 00:27:30,190 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 00:27:30,190 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 00:41:15 2024-11-22 00:27:30,190 INFO 3318922:repeater task report_generator starting 75.1 seconds late 2024-11-22 00:27:34,889 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 4.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-22 00:27:34,891 INFO 3318922:repeater task report_generator ended after 4.7 seconds 2024-11-22 00:27:34,891 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/22 01:26:15 2024-11-22 00:27:39,897 INFO 3318922:repeater task noisy_rawtemps_os150 starting 79.9 seconds late 2024-11-22 00:27:42,978 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 00:27:42,978 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:27:42,979 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 83.0 seconds late 2024-11-22 00:27:46,183 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 00:27:46,183 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:27:46,183 INFO 3318922:repeater task noisy_rawtemps_os38 starting 86.2 seconds late 2024-11-22 00:27:49,738 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 00:27:49,738 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:27:49,738 INFO 3318922:repeater task lastensq_wh300 starting 90.7 seconds late 2024-11-22 00:27:53,277 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-22 00:27:53,278 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:26:14 2024-11-22 00:27:53,278 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:28:19 2024-11-22 00:27:53,278 INFO 3318922:repeater task quick_wh300 starting 94.3 seconds late 2024-11-22 00:27:58,578 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:27:58,578 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:26:14 2024-11-22 00:27:58,578 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:28:19 2024-11-22 00:27:58,578 INFO 3318922:repeater task lastfew_vec_wh300 starting 99.6 seconds late 2024-11-22 00:27:59,610 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:27:59,611 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:26:14 2024-11-22 00:27:59,611 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:28:19 2024-11-22 00:27:59,611 INFO 3318922:repeater task 3dayplots_wh300 starting 99.6 seconds late 2024-11-22 00:28:06,107 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 00:28:06,107 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:31:19 2024-11-22 00:28:21,143 INFO 3318922:repeater task lastensq_wh300 starting 2.1 seconds late 2024-11-22 00:28:24,510 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:28:24,510 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:28:14 2024-11-22 00:28:24,511 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:30:19 2024-11-22 00:28:24,511 INFO 3318922:repeater task quick_wh300 starting 5.5 seconds late 2024-11-22 00:28:29,783 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:28:29,783 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:28:14 2024-11-22 00:28:29,783 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:30:19 2024-11-22 00:28:29,784 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-22 00:28:30,821 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:28:30,821 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:28:14 2024-11-22 00:28:30,822 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:30:19 2024-11-22 00:30:21,099 INFO 3318922:repeater task lastensq_wh300 starting 2.1 seconds late 2024-11-22 00:30:24,362 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:30:24,363 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:30:14 2024-11-22 00:30:24,363 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:32:19 2024-11-22 00:30:24,363 INFO 3318922:repeater task quick_wh300 starting 5.4 seconds late 2024-11-22 00:30:29,817 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 00:30:29,817 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:30:14 2024-11-22 00:30:29,817 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:32:19 2024-11-22 00:30:29,817 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-22 00:30:30,851 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:30:30,852 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:30:14 2024-11-22 00:30:30,852 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:32:19 2024-11-22 00:31:20,978 INFO 3318922:repeater task 3dayplots_wh300 starting 1.0 seconds late 2024-11-22 00:31:27,555 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 00:31:27,555 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:31:27,557 INFO 3318922:repeater task lastensq_os150nb starting 8.6 seconds late 2024-11-22 00:31:30,289 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:31:30,289 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:31:14 2024-11-22 00:31:30,289 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:31:30,289 INFO 3318922:repeater task quick_os150nb starting 11.3 seconds late 2024-11-22 00:31:33,784 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 00:31:33,785 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:31:14 2024-11-22 00:31:33,785 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:31:33,785 INFO 3318922:repeater task lastfew_vec_os150nb starting 14.8 seconds late 2024-11-22 00:31:34,812 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:31:34,813 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:31:14 2024-11-22 00:31:34,813 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:31:34,813 INFO 3318922:repeater task 3dayplots_os150nb starting 14.8 seconds late 2024-11-22 00:31:40,993 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 00:31:40,993 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:31:40,995 INFO 3318922:repeater task lastensq_os38nb starting 20.0 seconds late 2024-11-22 00:31:43,501 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:31:43,502 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:31:16 2024-11-22 00:31:43,502 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:36:21 2024-11-22 00:31:43,502 INFO 3318922:repeater task quick_os38nb starting 22.5 seconds late 2024-11-22 00:31:47,407 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 00:31:47,407 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:31:16 2024-11-22 00:31:47,407 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:36:21 2024-11-22 00:31:47,407 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.4 seconds late 2024-11-22 00:31:48,426 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:31:48,426 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:31:16 2024-11-22 00:31:48,427 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:36:21 2024-11-22 00:31:48,427 INFO 3318922:repeater task 3dayplots_os38nb starting 28.4 seconds late 2024-11-22 00:31:54,690 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 00:31:54,690 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:32:19,761 INFO 3318922:repeater task lastensq_wh300 starting 0.8 seconds late 2024-11-22 00:32:23,069 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:32:23,070 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:32:14 2024-11-22 00:32:23,070 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:34:19 2024-11-22 00:32:23,070 INFO 3318922:repeater task quick_wh300 starting 4.1 seconds late 2024-11-22 00:32:28,426 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 00:32:28,426 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:32:14 2024-11-22 00:32:28,427 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:34:19 2024-11-22 00:32:28,427 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.4 seconds late 2024-11-22 00:32:29,446 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:32:29,447 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:32:14 2024-11-22 00:32:29,447 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:34:19 2024-11-22 00:34:19,770 INFO 3318922:repeater task lastensq_wh300 starting 0.8 seconds late 2024-11-22 00:34:23,177 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:34:23,177 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:34:14 2024-11-22 00:34:23,177 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:34:23,178 INFO 3318922:repeater task quick_wh300 starting 4.2 seconds late 2024-11-22 00:34:28,706 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 00:34:28,707 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:34:14 2024-11-22 00:34:28,707 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:34:28,707 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.7 seconds late 2024-11-22 00:34:29,746 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:34:29,747 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:34:14 2024-11-22 00:34:29,747 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:36:19 2024-11-22 00:36:20,082 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.1 seconds late 2024-11-22 00:36:23,126 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2024-11-22 00:36:23,126 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:36:23,126 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.2 seconds late 2024-11-22 00:36:26,366 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 00:36:26,366 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:36:26,367 INFO 3318922:repeater task noisy_rawtemps_os38 starting 6.4 seconds late 2024-11-22 00:36:29,934 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 00:36:29,934 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:36:29,934 INFO 3318922:repeater task lastensq_wh300 starting 10.9 seconds late 2024-11-22 00:36:33,097 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 00:36:33,097 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:36:14 2024-11-22 00:36:33,098 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:38:19 2024-11-22 00:36:33,098 INFO 3318922:repeater task quick_wh300 starting 14.1 seconds late 2024-11-22 00:36:38,645 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 00:36:38,645 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:36:14 2024-11-22 00:36:38,645 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:38:19 2024-11-22 00:36:38,645 INFO 3318922:repeater task lastfew_vec_wh300 starting 19.6 seconds late 2024-11-22 00:36:39,673 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:36:39,673 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:36:14 2024-11-22 00:36:39,673 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:38:19 2024-11-22 00:36:39,674 INFO 3318922:repeater task 3dayplots_wh300 starting 19.7 seconds late 2024-11-22 00:36:46,391 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 00:36:46,391 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:41:19 2024-11-22 00:36:46,393 INFO 3318922:repeater task lastensq_os150nb starting 27.4 seconds late 2024-11-22 00:36:49,066 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:36:49,066 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:36:14 2024-11-22 00:36:49,066 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:41:19 2024-11-22 00:36:49,066 INFO 3318922:repeater task quick_os150nb starting 30.1 seconds late 2024-11-22 00:36:52,762 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 00:36:52,762 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:36:14 2024-11-22 00:36:52,762 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:41:19 2024-11-22 00:36:52,762 INFO 3318922:repeater task lastfew_vec_os150nb starting 33.8 seconds late 2024-11-22 00:36:53,773 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:36:53,773 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:36:14 2024-11-22 00:36:53,774 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:41:19 2024-11-22 00:36:53,774 INFO 3318922:repeater task 3dayplots_os150nb starting 33.8 seconds late 2024-11-22 00:36:59,948 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 00:36:59,948 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:41:19 2024-11-22 00:36:59,949 INFO 3318922:repeater task lastensq_os38nb starting 38.9 seconds late 2024-11-22 00:37:02,420 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:37:02,420 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:36:16 2024-11-22 00:37:02,420 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:41:21 2024-11-22 00:37:02,421 INFO 3318922:repeater task quick_os38nb starting 41.4 seconds late 2024-11-22 00:37:06,434 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 00:37:06,435 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:36:16 2024-11-22 00:37:06,435 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:41:21 2024-11-22 00:37:06,435 INFO 3318922:repeater task lastfew_vec_os38nb starting 45.4 seconds late 2024-11-22 00:37:07,452 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:37:07,453 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:36:16 2024-11-22 00:37:07,453 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:41:21 2024-11-22 00:37:07,453 INFO 3318922:repeater task 3dayplots_os38nb starting 47.5 seconds late 2024-11-22 00:37:13,584 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 00:37:13,584 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:41:19 2024-11-22 00:37:13,586 INFO 3318922:repeater task daily_netcdf starting 53.6 seconds late 2024-11-22 00:37:14,329 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 00:37:14,329 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:38:19,521 INFO 3318922:repeater task lastensq_wh300 starting 0.5 seconds late 2024-11-22 00:38:22,758 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 00:38:22,759 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:38:14 2024-11-22 00:38:22,759 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:40:19 2024-11-22 00:38:22,759 INFO 3318922:repeater task quick_wh300 starting 3.8 seconds late 2024-11-22 00:38:27,898 INFO 3318922:repeater task quick_wh300 ended after 5.1 seconds 2024-11-22 00:38:27,898 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:38:14 2024-11-22 00:38:27,898 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:40:19 2024-11-22 00:38:27,899 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.9 seconds late 2024-11-22 00:38:28,937 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:38:28,937 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:38:14 2024-11-22 00:38:28,937 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:40:19 2024-11-22 00:40:19,240 INFO 3318922:repeater task lastensq_wh300 starting 0.2 seconds late 2024-11-22 00:40:22,464 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 00:40:22,464 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:40:14 2024-11-22 00:40:22,464 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:42:19 2024-11-22 00:40:22,464 INFO 3318922:repeater task quick_wh300 starting 3.5 seconds late 2024-11-22 00:40:27,756 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:40:27,757 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:40:14 2024-11-22 00:40:27,757 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:42:19 2024-11-22 00:40:27,757 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.8 seconds late 2024-11-22 00:40:28,783 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:40:28,784 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:40:14 2024-11-22 00:40:28,784 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:42:19 2024-11-22 00:41:18,911 INFO 3318922:repeater task plot_tsstats_wh300 starting 3.9 seconds late 2024-11-22 00:41:20,357 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:41:20,357 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 00:41:20,357 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:20,358 INFO 3318922:repeater task grab_lastpings_wh300 starting 5.4 seconds late 2024-11-22 00:41:20,518 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 00:41:20,518 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:20,518 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.5 seconds late 2024-11-22 00:41:22,248 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 00:41:22,248 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:22,249 INFO 3318922:repeater task beamdiagnostics_wh300 starting 7.3 seconds late 2024-11-22 00:41:23,958 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 00:41:23,959 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:23,959 INFO 3318922:repeater task lastensq_os150nb starting 5.0 seconds late 2024-11-22 00:41:26,605 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 00:41:26,605 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:41:14 2024-11-22 00:41:26,605 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:41:26,605 INFO 3318922:repeater task quick_os150nb starting 7.6 seconds late 2024-11-22 00:41:30,397 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 00:41:30,397 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:41:14 2024-11-22 00:41:30,398 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:41:30,398 INFO 3318922:repeater task lastfew_vec_os150nb starting 11.4 seconds late 2024-11-22 00:41:31,405 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:41:31,405 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:41:14 2024-11-22 00:41:31,405 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:41:31,406 INFO 3318922:repeater task 3dayplots_os150nb starting 11.4 seconds late 2024-11-22 00:41:37,744 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 00:41:37,744 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:41:37,745 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.8 seconds late 2024-11-22 00:41:39,008 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:41:39,008 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 00:41:39,008 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:39,008 INFO 3318922:repeater task grab_lastpings_os150nb starting 24.0 seconds late 2024-11-22 00:41:39,169 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 00:41:39,169 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:39,170 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.2 seconds late 2024-11-22 00:41:40,861 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:41:40,861 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:40,862 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.9 seconds late 2024-11-22 00:41:42,572 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:41:42,573 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:42,573 INFO 3318922:repeater task lastensq_os38nb starting 21.6 seconds late 2024-11-22 00:41:45,030 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 00:41:45,031 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:41:16 2024-11-22 00:41:45,031 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:46:21 2024-11-22 00:41:45,031 INFO 3318922:repeater task quick_os38nb starting 24.0 seconds late 2024-11-22 00:41:48,760 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-22 00:41:48,760 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:41:16 2024-11-22 00:41:48,760 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:46:21 2024-11-22 00:41:48,761 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.8 seconds late 2024-11-22 00:41:49,789 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:41:49,790 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:41:16 2024-11-22 00:41:49,790 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:46:21 2024-11-22 00:41:49,791 INFO 3318922:repeater task 3dayplots_os38nb starting 29.8 seconds late 2024-11-22 00:41:55,961 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 00:41:55,961 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:41:55,961 INFO 3318922:repeater task plot_tsstats_os38nb starting 41.0 seconds late 2024-11-22 00:41:57,243 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:41:57,243 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 00:41:57,243 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:57,243 INFO 3318922:repeater task grab_lastpings_os38nb starting 42.3 seconds late 2024-11-22 00:41:57,403 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 00:41:57,403 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:57,403 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.4 seconds late 2024-11-22 00:41:59,103 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 00:41:59,103 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:41:59,104 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.1 seconds late 2024-11-22 00:42:00,871 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 00:42:00,871 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 00:56:14 2024-11-22 00:42:00,872 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 45.7 seconds late 2024-11-22 00:42:02,810 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 00:42:02,811 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 00:42:02,811 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 00:56:15 2024-11-22 00:42:02,811 INFO 3318922:repeater task plot_hbins starting 47.7 seconds late 2024-11-22 00:42:06,947 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 00:42:06,947 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 00:42:06,947 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 00:56:15 2024-11-22 00:42:11,956 INFO 3318922:repeater task 3dayplots_wh300 starting 52.0 seconds late 2024-11-22 00:42:18,552 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 00:42:18,552 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:42:23,562 INFO 3318922:repeater task lastensq_wh300 starting 4.6 seconds late 2024-11-22 00:42:26,704 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-22 00:42:26,704 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:42:14 2024-11-22 00:42:26,704 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:44:19 2024-11-22 00:42:26,705 INFO 3318922:repeater task quick_wh300 starting 7.7 seconds late 2024-11-22 00:42:32,024 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:42:32,024 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:42:14 2024-11-22 00:42:32,025 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:44:19 2024-11-22 00:42:32,025 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.0 seconds late 2024-11-22 00:42:33,058 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:42:33,059 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:42:14 2024-11-22 00:42:33,059 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:44:19 2024-11-22 00:44:23,380 INFO 3318922:repeater task lastensq_wh300 starting 4.4 seconds late 2024-11-22 00:44:26,643 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:44:26,643 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:44:14 2024-11-22 00:44:26,643 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:44:26,643 INFO 3318922:repeater task quick_wh300 starting 7.6 seconds late 2024-11-22 00:44:31,920 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 00:44:31,920 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:44:14 2024-11-22 00:44:31,921 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:44:31,921 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2024-11-22 00:44:32,962 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:44:32,962 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:44:14 2024-11-22 00:44:32,962 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:46:19 2024-11-22 00:46:23,269 INFO 3318922:repeater task noisy_rawtemps_os150 starting 3.3 seconds late 2024-11-22 00:46:26,369 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 00:46:26,369 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:46:26,370 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 6.4 seconds late 2024-11-22 00:46:29,590 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 00:46:29,590 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:46:29,590 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.6 seconds late 2024-11-22 00:46:33,118 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 00:46:33,118 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:46:33,119 INFO 3318922:repeater task lastensq_wh300 starting 14.1 seconds late 2024-11-22 00:46:36,462 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:46:36,463 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:46:14 2024-11-22 00:46:36,463 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:48:19 2024-11-22 00:46:36,463 INFO 3318922:repeater task quick_wh300 starting 17.5 seconds late 2024-11-22 00:46:42,025 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 00:46:42,025 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:46:14 2024-11-22 00:46:42,025 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:48:19 2024-11-22 00:46:42,025 INFO 3318922:repeater task lastfew_vec_wh300 starting 23.0 seconds late 2024-11-22 00:46:43,050 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:46:43,050 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:46:14 2024-11-22 00:46:43,051 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:48:19 2024-11-22 00:46:43,051 INFO 3318922:repeater task 3dayplots_wh300 starting 23.1 seconds late 2024-11-22 00:46:49,577 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 00:46:49,577 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:51:19 2024-11-22 00:46:49,578 INFO 3318922:repeater task lastensq_os150nb starting 30.6 seconds late 2024-11-22 00:46:52,239 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:46:52,240 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:46:14 2024-11-22 00:46:52,240 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:51:19 2024-11-22 00:46:52,240 INFO 3318922:repeater task quick_os150nb starting 33.2 seconds late 2024-11-22 00:46:56,079 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 00:46:56,079 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:46:14 2024-11-22 00:46:56,079 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:51:19 2024-11-22 00:46:56,080 INFO 3318922:repeater task lastfew_vec_os150nb starting 37.1 seconds late 2024-11-22 00:46:57,098 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:46:57,098 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:46:14 2024-11-22 00:46:57,098 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:51:19 2024-11-22 00:46:57,099 INFO 3318922:repeater task 3dayplots_os150nb starting 37.1 seconds late 2024-11-22 00:47:03,347 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 00:47:03,347 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:51:19 2024-11-22 00:47:03,349 INFO 3318922:repeater task lastensq_os38nb starting 42.3 seconds late 2024-11-22 00:47:05,980 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 00:47:05,981 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:46:16 2024-11-22 00:47:05,981 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:51:21 2024-11-22 00:47:05,981 INFO 3318922:repeater task quick_os38nb starting 45.0 seconds late 2024-11-22 00:47:10,052 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 00:47:10,052 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:46:16 2024-11-22 00:47:10,053 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:51:21 2024-11-22 00:47:10,053 INFO 3318922:repeater task lastfew_vec_os38nb starting 49.1 seconds late 2024-11-22 00:47:11,064 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:47:11,064 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:46:16 2024-11-22 00:47:11,064 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:51:21 2024-11-22 00:47:11,093 INFO 3318922:repeater task 3dayplots_os38nb starting 51.1 seconds late 2024-11-22 00:47:17,172 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 00:47:17,172 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:51:19 2024-11-22 00:47:17,174 INFO 3318922:repeater task daily_netcdf starting 57.2 seconds late 2024-11-22 00:47:18,044 INFO 3318922:repeater task daily_netcdf ended after 0.9 seconds 2024-11-22 00:47:18,044 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:48:23,229 INFO 3318922:repeater task lastensq_wh300 starting 4.2 seconds late 2024-11-22 00:48:26,629 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 00:48:26,630 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:48:14 2024-11-22 00:48:26,630 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:50:19 2024-11-22 00:48:26,630 INFO 3318922:repeater task quick_wh300 starting 7.6 seconds late 2024-11-22 00:48:32,014 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 00:48:32,014 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:48:14 2024-11-22 00:48:32,014 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:50:19 2024-11-22 00:48:32,015 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.0 seconds late 2024-11-22 00:48:33,035 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:48:33,035 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:48:14 2024-11-22 00:48:33,036 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:50:19 2024-11-22 00:50:23,368 INFO 3318922:repeater task lastensq_wh300 starting 4.4 seconds late 2024-11-22 00:50:26,877 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-22 00:50:26,877 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:50:14 2024-11-22 00:50:26,877 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:52:19 2024-11-22 00:50:26,877 INFO 3318922:repeater task quick_wh300 starting 7.9 seconds late 2024-11-22 00:50:32,355 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 00:50:32,355 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:50:14 2024-11-22 00:50:32,355 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:52:19 2024-11-22 00:50:32,355 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2024-11-22 00:50:33,407 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 00:50:33,408 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:50:14 2024-11-22 00:50:33,408 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:52:19 2024-11-22 00:51:23,553 INFO 3318922:repeater task 3dayplots_wh300 starting 3.6 seconds late 2024-11-22 00:51:30,177 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 00:51:30,177 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:51:30,178 INFO 3318922:repeater task lastensq_os150nb starting 11.2 seconds late 2024-11-22 00:51:32,850 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:51:32,850 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:51:14 2024-11-22 00:51:32,850 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:51:32,850 INFO 3318922:repeater task quick_os150nb starting 13.9 seconds late 2024-11-22 00:51:36,421 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 00:51:36,421 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:51:14 2024-11-22 00:51:36,421 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:51:36,422 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.4 seconds late 2024-11-22 00:51:37,440 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:51:37,440 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:51:14 2024-11-22 00:51:37,440 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:51:37,441 INFO 3318922:repeater task 3dayplots_os150nb starting 17.5 seconds late 2024-11-22 00:51:43,612 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 00:51:43,612 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:51:43,613 INFO 3318922:repeater task lastensq_os38nb starting 22.6 seconds late 2024-11-22 00:51:46,297 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 00:51:46,297 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:51:16 2024-11-22 00:51:46,298 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 00:56:21 2024-11-22 00:51:46,298 INFO 3318922:repeater task quick_os38nb starting 25.3 seconds late 2024-11-22 00:51:49,964 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-22 00:51:49,964 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:51:16 2024-11-22 00:51:49,964 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 00:56:21 2024-11-22 00:51:49,965 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.0 seconds late 2024-11-22 00:51:50,993 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:51:50,993 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:51:16 2024-11-22 00:51:50,993 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 00:56:21 2024-11-22 00:51:50,994 INFO 3318922:repeater task 3dayplots_os38nb starting 31.0 seconds late 2024-11-22 00:51:57,401 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.4 seconds 2024-11-22 00:51:57,402 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:52:22,468 INFO 3318922:repeater task lastensq_wh300 starting 3.5 seconds late 2024-11-22 00:52:25,690 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 00:52:25,690 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:52:14 2024-11-22 00:52:25,690 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:54:19 2024-11-22 00:52:25,690 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-22 00:52:31,403 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 00:52:31,404 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:52:14 2024-11-22 00:52:31,404 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:54:19 2024-11-22 00:52:31,404 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2024-11-22 00:52:32,442 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:52:32,442 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:52:14 2024-11-22 00:52:32,442 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:54:19 2024-11-22 00:54:22,761 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 00:54:26,077 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:54:26,077 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:54:14 2024-11-22 00:54:26,077 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:54:26,078 INFO 3318922:repeater task quick_wh300 starting 7.1 seconds late 2024-11-22 00:54:31,507 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 00:54:31,507 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:54:14 2024-11-22 00:54:31,507 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:54:31,507 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2024-11-22 00:54:32,546 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:54:32,547 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:54:14 2024-11-22 00:54:32,547 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:56:19 2024-11-22 00:56:17,845 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.9 seconds late 2024-11-22 00:56:19,280 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:56:19,281 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 00:56:19,281 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:19,281 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.3 seconds late 2024-11-22 00:56:19,444 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 00:56:19,444 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:19,445 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.5 seconds late 2024-11-22 00:56:21,137 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 00:56:21,137 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:21,138 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.2 seconds late 2024-11-22 00:56:22,902 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-22 00:56:22,904 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:22,905 INFO 3318922:repeater task lastensq_os150nb starting 3.9 seconds late 2024-11-22 00:56:25,572 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 00:56:25,572 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:56:14 2024-11-22 00:56:25,572 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:01:19 2024-11-22 00:56:25,572 INFO 3318922:repeater task quick_os150nb starting 6.6 seconds late 2024-11-22 00:56:29,394 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 00:56:29,394 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:56:14 2024-11-22 00:56:29,394 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:01:19 2024-11-22 00:56:29,395 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.4 seconds late 2024-11-22 00:56:30,420 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 00:56:30,420 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 00:56:14 2024-11-22 00:56:30,421 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:01:19 2024-11-22 00:56:30,421 INFO 3318922:repeater task 3dayplots_os150nb starting 10.4 seconds late 2024-11-22 00:56:36,628 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 00:56:36,628 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:01:19 2024-11-22 00:56:36,629 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.7 seconds late 2024-11-22 00:56:37,887 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:56:37,887 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 00:56:37,887 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:37,887 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.9 seconds late 2024-11-22 00:56:38,049 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 00:56:38,049 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:38,050 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.1 seconds late 2024-11-22 00:56:39,767 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:56:39,767 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:39,767 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.8 seconds late 2024-11-22 00:56:41,473 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 00:56:41,473 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:41,474 INFO 3318922:repeater task lastensq_os38nb starting 20.5 seconds late 2024-11-22 00:56:44,196 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 00:56:44,196 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:56:16 2024-11-22 00:56:44,196 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:01:21 2024-11-22 00:56:44,196 INFO 3318922:repeater task quick_os38nb starting 23.2 seconds late 2024-11-22 00:56:48,006 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 00:56:48,007 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:56:16 2024-11-22 00:56:48,007 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:01:21 2024-11-22 00:56:48,007 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.0 seconds late 2024-11-22 00:56:49,027 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 00:56:49,027 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 00:56:16 2024-11-22 00:56:49,027 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:01:21 2024-11-22 00:56:49,028 INFO 3318922:repeater task 3dayplots_os38nb starting 29.0 seconds late 2024-11-22 00:56:55,330 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 00:56:55,330 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:01:19 2024-11-22 00:56:55,331 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.4 seconds late 2024-11-22 00:56:56,619 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 00:56:56,619 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 00:56:56,619 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:56,620 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.6 seconds late 2024-11-22 00:56:56,781 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 00:56:56,781 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:56,781 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.8 seconds late 2024-11-22 00:56:58,475 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 00:56:58,475 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:56:58,475 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.5 seconds late 2024-11-22 00:57:00,261 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 00:57:00,261 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:11:14 2024-11-22 00:57:00,261 INFO 3318922:repeater task daily_netcdf starting 40.3 seconds late 2024-11-22 00:57:01,030 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 00:57:01,030 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 01:06:19 2024-11-22 00:57:01,031 INFO 3318922:repeater task ggatime_diagnostics starting 46.1 seconds late 2024-11-22 00:57:08,793 INFO 3318922:repeater task ggatime_diagnostics ended after 7.8 seconds 2024-11-22 00:57:08,793 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 01:26:14 2024-11-22 00:57:08,793 INFO 3318922:repeater task ggatime_diagnostics starting 53.8 seconds late 2024-11-22 00:57:16,385 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 00:57:16,385 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 01:26:14 2024-11-22 00:57:16,386 INFO 3318922:repeater task ggatime_diagnostics starting 61.4 seconds late 2024-11-22 00:57:23,920 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 00:57:23,920 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 01:26:14 2024-11-22 00:57:23,920 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 68.8 seconds late 2024-11-22 00:57:25,731 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 00:57:25,731 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 00:57:25,731 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 01:11:15 2024-11-22 00:57:25,731 INFO 3318922:repeater task plot_hbins starting 70.6 seconds late 2024-11-22 00:57:29,869 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 00:57:29,869 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 00:57:29,869 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 01:11:15 2024-11-22 00:57:34,874 INFO 3318922:repeater task noisy_rawtemps_os150 starting 74.9 seconds late 2024-11-22 00:57:37,961 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 00:57:37,961 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 01:06:19 2024-11-22 00:57:37,961 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 78.0 seconds late 2024-11-22 00:57:41,158 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 00:57:41,158 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 01:06:19 2024-11-22 00:57:41,158 INFO 3318922:repeater task noisy_rawtemps_os38 starting 81.2 seconds late 2024-11-22 00:57:44,591 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-22 00:57:44,592 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 01:06:19 2024-11-22 00:57:44,592 INFO 3318922:repeater task lastensq_wh300 starting 85.6 seconds late 2024-11-22 00:57:47,848 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:57:47,848 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:56:14 2024-11-22 00:57:47,848 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 00:58:19 2024-11-22 00:57:47,849 INFO 3318922:repeater task quick_wh300 starting 88.8 seconds late 2024-11-22 00:57:53,261 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 00:57:53,261 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:56:14 2024-11-22 00:57:53,261 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 00:58:19 2024-11-22 00:57:53,262 INFO 3318922:repeater task lastfew_vec_wh300 starting 94.3 seconds late 2024-11-22 00:57:54,322 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 00:57:54,322 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:56:14 2024-11-22 00:57:54,322 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 00:58:19 2024-11-22 00:57:54,322 INFO 3318922:repeater task 3dayplots_wh300 starting 94.3 seconds late 2024-11-22 00:58:00,832 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 00:58:00,832 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:01:19 2024-11-22 00:58:20,880 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 00:58:24,229 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 00:58:24,229 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:58:14 2024-11-22 00:58:24,229 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:00:19 2024-11-22 00:58:24,230 INFO 3318922:repeater task quick_wh300 starting 5.2 seconds late 2024-11-22 00:58:29,404 INFO 3318922:repeater task quick_wh300 ended after 5.2 seconds 2024-11-22 00:58:29,404 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:58:14 2024-11-22 00:58:29,404 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:00:19 2024-11-22 00:58:29,405 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2024-11-22 00:58:30,442 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 00:58:30,443 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 00:58:14 2024-11-22 00:58:30,443 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:00:19 2024-11-22 01:00:20,740 INFO 3318922:repeater task lastensq_wh300 starting 1.7 seconds late 2024-11-22 01:00:24,103 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:00:24,103 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:00:14 2024-11-22 01:00:24,104 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:02:19 2024-11-22 01:00:24,104 INFO 3318922:repeater task quick_wh300 starting 5.1 seconds late 2024-11-22 01:00:29,472 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 01:00:29,472 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:00:14 2024-11-22 01:00:29,472 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:02:19 2024-11-22 01:00:29,472 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-22 01:00:30,517 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:00:30,517 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:00:14 2024-11-22 01:00:30,517 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:02:19 2024-11-22 01:01:20,653 INFO 3318922:repeater task 3dayplots_wh300 starting 0.7 seconds late 2024-11-22 01:01:27,391 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 01:01:27,391 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:01:27,393 INFO 3318922:repeater task lastensq_os150nb starting 8.4 seconds late 2024-11-22 01:01:30,071 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 01:01:30,071 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:01:14 2024-11-22 01:01:30,071 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:01:30,072 INFO 3318922:repeater task quick_os150nb starting 11.1 seconds late 2024-11-22 01:01:33,786 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 01:01:33,786 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:01:14 2024-11-22 01:01:33,786 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:01:33,786 INFO 3318922:repeater task lastfew_vec_os150nb starting 14.8 seconds late 2024-11-22 01:01:34,804 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:01:34,804 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:01:14 2024-11-22 01:01:34,804 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:01:34,805 INFO 3318922:repeater task 3dayplots_os150nb starting 14.8 seconds late 2024-11-22 01:01:41,089 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 01:01:41,089 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:01:41,090 INFO 3318922:repeater task lastensq_os38nb starting 20.1 seconds late 2024-11-22 01:01:43,653 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 01:01:43,653 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:01:16 2024-11-22 01:01:43,653 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:06:21 2024-11-22 01:01:43,653 INFO 3318922:repeater task quick_os38nb starting 22.7 seconds late 2024-11-22 01:01:47,662 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 01:01:47,662 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:01:16 2024-11-22 01:01:47,662 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:06:21 2024-11-22 01:01:47,662 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.7 seconds late 2024-11-22 01:01:48,688 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:01:48,689 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:01:16 2024-11-22 01:01:48,689 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:06:21 2024-11-22 01:01:48,689 INFO 3318922:repeater task 3dayplots_os38nb starting 28.7 seconds late 2024-11-22 01:01:54,902 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 01:01:54,902 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:02:19,971 INFO 3318922:repeater task lastensq_wh300 starting 1.0 seconds late 2024-11-22 01:02:23,352 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:02:23,352 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:02:14 2024-11-22 01:02:23,352 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:04:19 2024-11-22 01:02:23,353 INFO 3318922:repeater task quick_wh300 starting 4.4 seconds late 2024-11-22 01:02:28,754 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 01:02:28,754 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:02:14 2024-11-22 01:02:28,754 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:04:19 2024-11-22 01:02:28,754 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.8 seconds late 2024-11-22 01:02:29,809 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:02:29,810 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:02:14 2024-11-22 01:02:29,810 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:04:19 2024-11-22 01:04:20,084 INFO 3318922:repeater task lastensq_wh300 starting 1.1 seconds late 2024-11-22 01:04:23,301 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 01:04:23,302 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:04:14 2024-11-22 01:04:23,302 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:04:23,302 INFO 3318922:repeater task quick_wh300 starting 4.3 seconds late 2024-11-22 01:04:28,779 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:04:28,779 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:04:14 2024-11-22 01:04:28,779 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:04:28,779 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.8 seconds late 2024-11-22 01:04:29,837 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:04:29,838 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:04:14 2024-11-22 01:04:29,838 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:06:19 2024-11-22 01:06:20,146 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.2 seconds late 2024-11-22 01:06:23,332 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 01:06:23,332 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:06:23,332 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.4 seconds late 2024-11-22 01:06:26,592 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.3 seconds 2024-11-22 01:06:26,593 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:06:26,593 INFO 3318922:repeater task noisy_rawtemps_os38 starting 6.6 seconds late 2024-11-22 01:06:30,041 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-22 01:06:30,041 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:06:30,041 INFO 3318922:repeater task lastensq_wh300 starting 11.0 seconds late 2024-11-22 01:06:33,288 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 01:06:33,288 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:06:14 2024-11-22 01:06:33,288 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:08:19 2024-11-22 01:06:33,289 INFO 3318922:repeater task quick_wh300 starting 14.3 seconds late 2024-11-22 01:06:38,961 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 01:06:38,962 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:06:14 2024-11-22 01:06:38,962 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:08:19 2024-11-22 01:06:38,962 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.0 seconds late 2024-11-22 01:06:40,030 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:06:40,031 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:06:14 2024-11-22 01:06:40,031 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:08:19 2024-11-22 01:06:40,031 INFO 3318922:repeater task 3dayplots_wh300 starting 20.1 seconds late 2024-11-22 01:06:46,627 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:06:46,627 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:11:19 2024-11-22 01:06:46,629 INFO 3318922:repeater task lastensq_os150nb starting 27.6 seconds late 2024-11-22 01:06:49,431 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 01:06:49,431 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:06:14 2024-11-22 01:06:49,431 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:11:19 2024-11-22 01:06:49,431 INFO 3318922:repeater task quick_os150nb starting 30.4 seconds late 2024-11-22 01:06:53,033 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 01:06:53,033 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:06:14 2024-11-22 01:06:53,033 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:11:19 2024-11-22 01:06:53,033 INFO 3318922:repeater task lastfew_vec_os150nb starting 34.0 seconds late 2024-11-22 01:06:54,045 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:06:54,045 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:06:14 2024-11-22 01:06:54,045 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:11:19 2024-11-22 01:06:54,045 INFO 3318922:repeater task 3dayplots_os150nb starting 34.1 seconds late 2024-11-22 01:07:00,445 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 01:07:00,445 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:11:19 2024-11-22 01:07:00,446 INFO 3318922:repeater task lastensq_os38nb starting 39.4 seconds late 2024-11-22 01:07:03,082 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 01:07:03,082 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:06:16 2024-11-22 01:07:03,082 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:11:21 2024-11-22 01:07:03,083 INFO 3318922:repeater task quick_os38nb starting 42.1 seconds late 2024-11-22 01:07:07,145 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 01:07:07,146 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:06:16 2024-11-22 01:07:07,146 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:11:21 2024-11-22 01:07:07,146 INFO 3318922:repeater task lastfew_vec_os38nb starting 46.1 seconds late 2024-11-22 01:07:08,153 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:07:08,153 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:06:16 2024-11-22 01:07:08,153 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:11:21 2024-11-22 01:07:08,153 INFO 3318922:repeater task 3dayplots_os38nb starting 48.2 seconds late 2024-11-22 01:07:14,318 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 01:07:14,318 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:11:19 2024-11-22 01:07:14,319 INFO 3318922:repeater task daily_netcdf starting 54.3 seconds late 2024-11-22 01:07:15,055 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 01:07:15,055 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:08:20,236 INFO 3318922:repeater task lastensq_wh300 starting 1.2 seconds late 2024-11-22 01:08:23,541 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:08:23,541 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:08:14 2024-11-22 01:08:23,541 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:10:19 2024-11-22 01:08:23,542 INFO 3318922:repeater task quick_wh300 starting 4.5 seconds late 2024-11-22 01:08:29,005 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:08:29,005 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:08:14 2024-11-22 01:08:29,005 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:10:19 2024-11-22 01:08:29,006 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-22 01:08:30,063 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:08:30,063 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:08:14 2024-11-22 01:08:30,063 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:10:19 2024-11-22 01:10:20,356 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 01:10:23,664 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:10:23,665 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:10:14 2024-11-22 01:10:23,665 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:12:19 2024-11-22 01:10:23,665 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 01:10:28,964 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 01:10:28,964 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:10:14 2024-11-22 01:10:28,964 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:12:19 2024-11-22 01:10:28,964 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-22 01:10:30,001 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:10:30,001 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:10:14 2024-11-22 01:10:30,001 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:12:19 2024-11-22 01:11:15,118 INFO 3318922:repeater task plot_tsstats_wh300 starting 0.1 seconds late 2024-11-22 01:11:16,577 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:11:16,578 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 01:11:16,578 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:16,578 INFO 3318922:repeater task grab_lastpings_wh300 starting 1.6 seconds late 2024-11-22 01:11:16,739 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 01:11:16,740 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:16,740 INFO 3318922:repeater task beamdiagnostics_wh300 starting 1.8 seconds late 2024-11-22 01:11:18,479 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:11:18,479 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:18,480 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.5 seconds late 2024-11-22 01:11:20,201 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:11:20,202 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:20,202 INFO 3318922:repeater task lastensq_os150nb starting 1.2 seconds late 2024-11-22 01:11:23,031 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 01:11:23,031 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:11:14 2024-11-22 01:11:23,031 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:11:23,032 INFO 3318922:repeater task quick_os150nb starting 4.0 seconds late 2024-11-22 01:11:26,865 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 01:11:26,865 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:11:14 2024-11-22 01:11:26,865 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:11:26,865 INFO 3318922:repeater task lastfew_vec_os150nb starting 7.9 seconds late 2024-11-22 01:11:27,925 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-22 01:11:27,926 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:11:14 2024-11-22 01:11:27,926 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:11:27,926 INFO 3318922:repeater task 3dayplots_os150nb starting 7.9 seconds late 2024-11-22 01:11:34,278 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 01:11:34,278 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:11:34,279 INFO 3318922:repeater task plot_tsstats_os150nb starting 19.3 seconds late 2024-11-22 01:11:35,536 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:11:35,536 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 01:11:35,536 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:35,536 INFO 3318922:repeater task grab_lastpings_os150nb starting 20.6 seconds late 2024-11-22 01:11:35,701 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 01:11:35,701 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:35,701 INFO 3318922:repeater task beamdiagnostics_os150nb starting 20.7 seconds late 2024-11-22 01:11:37,445 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:11:37,445 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:37,445 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.5 seconds late 2024-11-22 01:11:39,154 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:11:39,154 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:39,155 INFO 3318922:repeater task lastensq_os38nb starting 18.2 seconds late 2024-11-22 01:11:41,861 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 01:11:41,861 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:11:16 2024-11-22 01:11:41,861 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:16:21 2024-11-22 01:11:41,862 INFO 3318922:repeater task quick_os38nb starting 20.9 seconds late 2024-11-22 01:11:45,658 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 01:11:45,658 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:11:16 2024-11-22 01:11:45,658 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:16:21 2024-11-22 01:11:45,659 INFO 3318922:repeater task lastfew_vec_os38nb starting 24.7 seconds late 2024-11-22 01:11:46,666 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:11:46,667 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:11:16 2024-11-22 01:11:46,667 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:16:21 2024-11-22 01:11:46,667 INFO 3318922:repeater task 3dayplots_os38nb starting 26.7 seconds late 2024-11-22 01:11:52,718 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 01:11:52,718 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:11:52,719 INFO 3318922:repeater task plot_tsstats_os38nb starting 37.7 seconds late 2024-11-22 01:11:54,085 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:11:54,085 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.4 seconds 2024-11-22 01:11:54,085 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:54,085 INFO 3318922:repeater task grab_lastpings_os38nb starting 39.1 seconds late 2024-11-22 01:11:54,248 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 01:11:54,248 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:54,248 INFO 3318922:repeater task beamdiagnostics_os38nb starting 39.3 seconds late 2024-11-22 01:11:55,989 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 01:11:55,989 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:55,990 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.0 seconds late 2024-11-22 01:11:57,697 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 01:11:57,697 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:26:14 2024-11-22 01:11:57,699 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 42.6 seconds late 2024-11-22 01:11:59,616 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 01:11:59,616 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 01:11:59,616 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 01:26:15 2024-11-22 01:11:59,617 INFO 3318922:repeater task plot_hbins starting 44.5 seconds late 2024-11-22 01:12:03,740 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 01:12:03,740 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 01:12:03,740 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 01:26:15 2024-11-22 01:12:08,748 INFO 3318922:repeater task 3dayplots_wh300 starting 48.8 seconds late 2024-11-22 01:12:15,311 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:12:15,311 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:12:20,319 INFO 3318922:repeater task lastensq_wh300 starting 1.3 seconds late 2024-11-22 01:12:23,607 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:12:23,608 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:12:14 2024-11-22 01:12:23,608 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:14:19 2024-11-22 01:12:23,608 INFO 3318922:repeater task quick_wh300 starting 4.6 seconds late 2024-11-22 01:12:29,123 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:12:29,123 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:12:14 2024-11-22 01:12:29,123 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:14:19 2024-11-22 01:12:29,124 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2024-11-22 01:12:30,153 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:12:30,153 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:12:14 2024-11-22 01:12:30,153 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:14:19 2024-11-22 01:14:20,469 INFO 3318922:repeater task lastensq_wh300 starting 1.5 seconds late 2024-11-22 01:14:23,878 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:14:23,878 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:14:14 2024-11-22 01:14:23,878 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:14:23,879 INFO 3318922:repeater task quick_wh300 starting 4.9 seconds late 2024-11-22 01:14:29,321 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 01:14:29,321 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:14:14 2024-11-22 01:14:29,321 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:14:29,322 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2024-11-22 01:14:30,365 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:14:30,365 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:14:14 2024-11-22 01:14:30,366 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:16:19 2024-11-22 01:16:20,694 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.7 seconds late 2024-11-22 01:16:23,879 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 01:16:23,879 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:16:23,880 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.9 seconds late 2024-11-22 01:16:27,120 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 01:16:27,121 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:16:27,121 INFO 3318922:repeater task noisy_rawtemps_os38 starting 7.1 seconds late 2024-11-22 01:16:30,575 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 01:16:30,575 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:16:30,576 INFO 3318922:repeater task lastensq_wh300 starting 11.6 seconds late 2024-11-22 01:16:33,854 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:16:33,854 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:16:14 2024-11-22 01:16:33,854 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:18:19 2024-11-22 01:16:33,855 INFO 3318922:repeater task quick_wh300 starting 14.9 seconds late 2024-11-22 01:16:39,418 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 01:16:39,418 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:16:14 2024-11-22 01:16:39,418 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:18:19 2024-11-22 01:16:39,419 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.4 seconds late 2024-11-22 01:16:40,463 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:16:40,464 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:16:14 2024-11-22 01:16:40,464 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:18:19 2024-11-22 01:16:40,464 INFO 3318922:repeater task 3dayplots_wh300 starting 20.5 seconds late 2024-11-22 01:16:47,029 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:16:47,029 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:21:19 2024-11-22 01:16:47,030 INFO 3318922:repeater task lastensq_os150nb starting 28.0 seconds late 2024-11-22 01:16:49,689 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 01:16:49,690 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:16:14 2024-11-22 01:16:49,690 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:21:19 2024-11-22 01:16:49,690 INFO 3318922:repeater task quick_os150nb starting 30.7 seconds late 2024-11-22 01:16:53,472 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 01:16:53,472 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:16:14 2024-11-22 01:16:53,472 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:21:19 2024-11-22 01:16:53,472 INFO 3318922:repeater task lastfew_vec_os150nb starting 34.5 seconds late 2024-11-22 01:16:54,484 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:16:54,484 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:16:14 2024-11-22 01:16:54,484 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:21:19 2024-11-22 01:16:54,485 INFO 3318922:repeater task 3dayplots_os150nb starting 34.5 seconds late 2024-11-22 01:17:00,992 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.5 seconds 2024-11-22 01:17:00,992 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:21:19 2024-11-22 01:17:00,993 INFO 3318922:repeater task lastensq_os38nb starting 40.0 seconds late 2024-11-22 01:17:03,672 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 01:17:03,672 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:16:16 2024-11-22 01:17:03,672 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:21:21 2024-11-22 01:17:03,673 INFO 3318922:repeater task quick_os38nb starting 42.7 seconds late 2024-11-22 01:17:07,554 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 01:17:07,554 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:16:16 2024-11-22 01:17:07,554 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:21:21 2024-11-22 01:17:07,555 INFO 3318922:repeater task lastfew_vec_os38nb starting 46.6 seconds late 2024-11-22 01:17:08,561 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:17:08,562 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:16:16 2024-11-22 01:17:08,562 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:21:21 2024-11-22 01:17:08,562 INFO 3318922:repeater task 3dayplots_os38nb starting 48.6 seconds late 2024-11-22 01:17:14,654 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 01:17:14,654 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:21:19 2024-11-22 01:17:14,655 INFO 3318922:repeater task daily_netcdf starting 54.7 seconds late 2024-11-22 01:17:15,382 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 01:17:15,382 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:18:20,571 INFO 3318922:repeater task lastensq_wh300 starting 1.6 seconds late 2024-11-22 01:18:23,883 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:18:23,883 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:18:14 2024-11-22 01:18:23,883 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:20:19 2024-11-22 01:18:23,884 INFO 3318922:repeater task quick_wh300 starting 4.9 seconds late 2024-11-22 01:18:29,345 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:18:29,345 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:18:14 2024-11-22 01:18:29,345 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:20:19 2024-11-22 01:18:29,345 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2024-11-22 01:18:30,381 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:18:30,381 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:18:14 2024-11-22 01:18:30,381 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:20:19 2024-11-22 01:20:20,723 INFO 3318922:repeater task lastensq_wh300 starting 1.7 seconds late 2024-11-22 01:20:24,618 INFO 3318922:repeater task lastensq_wh300 ended after 3.9 seconds 2024-11-22 01:20:24,619 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:20:14 2024-11-22 01:20:24,619 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:22:19 2024-11-22 01:20:24,619 INFO 3318922:repeater task quick_wh300 starting 5.6 seconds late 2024-11-22 01:20:30,112 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:20:30,113 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:20:14 2024-11-22 01:20:30,113 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:22:19 2024-11-22 01:20:30,113 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2024-11-22 01:20:31,162 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:20:31,162 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:20:14 2024-11-22 01:20:31,162 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:22:19 2024-11-22 01:21:21,311 INFO 3318922:repeater task 3dayplots_wh300 starting 1.3 seconds late 2024-11-22 01:21:27,872 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:21:27,872 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:21:27,873 INFO 3318922:repeater task lastensq_os150nb starting 8.9 seconds late 2024-11-22 01:21:30,560 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 01:21:30,560 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:21:14 2024-11-22 01:21:30,560 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:21:30,560 INFO 3318922:repeater task quick_os150nb starting 11.6 seconds late 2024-11-22 01:21:34,273 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 01:21:34,274 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:21:14 2024-11-22 01:21:34,274 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:21:34,274 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.3 seconds late 2024-11-22 01:21:35,296 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:21:35,297 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:21:14 2024-11-22 01:21:35,297 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:21:35,297 INFO 3318922:repeater task 3dayplots_os150nb starting 15.3 seconds late 2024-11-22 01:21:41,576 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 01:21:41,576 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:21:41,577 INFO 3318922:repeater task lastensq_os38nb starting 20.6 seconds late 2024-11-22 01:21:44,209 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 01:21:44,209 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:21:16 2024-11-22 01:21:44,209 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:26:21 2024-11-22 01:21:44,210 INFO 3318922:repeater task quick_os38nb starting 23.2 seconds late 2024-11-22 01:21:47,990 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 01:21:47,991 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:21:16 2024-11-22 01:21:47,991 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:26:21 2024-11-22 01:21:47,991 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.0 seconds late 2024-11-22 01:21:49,042 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.1 seconds 2024-11-22 01:21:49,042 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:21:16 2024-11-22 01:21:49,042 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:26:21 2024-11-22 01:21:49,043 INFO 3318922:repeater task 3dayplots_os38nb starting 29.1 seconds late 2024-11-22 01:21:55,299 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 01:21:55,299 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:22:20,357 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 01:22:23,664 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:22:23,664 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:22:14 2024-11-22 01:22:23,664 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:24:19 2024-11-22 01:22:23,664 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 01:22:29,080 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 01:22:29,080 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:22:14 2024-11-22 01:22:29,081 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:24:19 2024-11-22 01:22:29,081 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2024-11-22 01:22:30,121 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:22:30,122 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:22:14 2024-11-22 01:22:30,122 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:24:19 2024-11-22 01:24:20,442 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 01:24:23,805 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:24:23,806 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:24:14 2024-11-22 01:24:23,806 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:24:23,806 INFO 3318922:repeater task quick_wh300 starting 4.8 seconds late 2024-11-22 01:24:29,575 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 01:24:29,575 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:24:14 2024-11-22 01:24:29,575 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:24:29,576 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2024-11-22 01:24:30,641 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:24:30,641 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:24:14 2024-11-22 01:24:30,642 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:26:19 2024-11-22 01:26:15,970 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.0 seconds late 2024-11-22 01:26:17,430 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:26:17,430 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 01:26:17,430 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:17,430 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.5 seconds late 2024-11-22 01:26:17,662 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 01:26:17,663 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:17,663 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.7 seconds late 2024-11-22 01:26:19,410 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:26:19,410 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:19,410 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.4 seconds late 2024-11-22 01:26:21,124 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:26:21,124 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:21,124 INFO 3318922:repeater task lastensq_os150nb starting 2.1 seconds late 2024-11-22 01:26:23,809 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 01:26:23,810 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:26:14 2024-11-22 01:26:23,810 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:31:19 2024-11-22 01:26:23,810 INFO 3318922:repeater task quick_os150nb starting 4.8 seconds late 2024-11-22 01:26:27,653 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 01:26:27,653 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:26:14 2024-11-22 01:26:27,653 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:31:19 2024-11-22 01:26:27,653 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.7 seconds late 2024-11-22 01:26:28,684 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:26:28,685 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:26:14 2024-11-22 01:26:28,685 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:31:19 2024-11-22 01:26:28,685 INFO 3318922:repeater task 3dayplots_os150nb starting 8.7 seconds late 2024-11-22 01:26:34,914 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 01:26:34,914 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:31:19 2024-11-22 01:26:34,914 INFO 3318922:repeater task plot_tsstats_os150nb starting 19.9 seconds late 2024-11-22 01:26:36,220 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:26:36,220 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 01:26:36,220 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:36,220 INFO 3318922:repeater task grab_lastpings_os150nb starting 21.2 seconds late 2024-11-22 01:26:36,378 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 01:26:36,378 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:36,378 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.4 seconds late 2024-11-22 01:26:38,122 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:26:38,122 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:38,122 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.1 seconds late 2024-11-22 01:26:39,837 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:26:39,837 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:39,837 INFO 3318922:repeater task lastensq_os38nb starting 18.8 seconds late 2024-11-22 01:26:42,426 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 01:26:42,427 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:26:16 2024-11-22 01:26:42,427 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:31:21 2024-11-22 01:26:42,427 INFO 3318922:repeater task quick_os38nb starting 21.4 seconds late 2024-11-22 01:26:46,455 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 01:26:46,455 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:26:16 2024-11-22 01:26:46,456 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:31:21 2024-11-22 01:26:46,456 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.5 seconds late 2024-11-22 01:26:47,481 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:26:47,481 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:26:16 2024-11-22 01:26:47,482 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:31:21 2024-11-22 01:26:47,482 INFO 3318922:repeater task 3dayplots_os38nb starting 27.5 seconds late 2024-11-22 01:26:53,628 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 01:26:53,628 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:31:19 2024-11-22 01:26:53,628 INFO 3318922:repeater task plot_tsstats_os38nb starting 38.6 seconds late 2024-11-22 01:26:54,920 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:26:54,920 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 01:26:54,920 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:54,920 INFO 3318922:repeater task grab_lastpings_os38nb starting 39.9 seconds late 2024-11-22 01:26:55,081 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 01:26:55,081 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:55,082 INFO 3318922:repeater task beamdiagnostics_os38nb starting 40.1 seconds late 2024-11-22 01:26:56,772 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 01:26:56,772 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:56,773 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.8 seconds late 2024-11-22 01:26:58,568 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 01:26:58,568 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:41:14 2024-11-22 01:26:58,568 INFO 3318922:repeater task daily_netcdf starting 38.6 seconds late 2024-11-22 01:26:59,353 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 01:26:59,354 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:26:59,354 INFO 3318922:repeater task ggatime_diagnostics starting 44.4 seconds late 2024-11-22 01:27:07,380 INFO 3318922:repeater task ggatime_diagnostics ended after 8.0 seconds 2024-11-22 01:27:07,380 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:27:07,380 INFO 3318922:repeater task ggatime_diagnostics starting 52.4 seconds late 2024-11-22 01:27:14,941 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 01:27:14,941 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:27:14,941 INFO 3318922:repeater task ggatime_diagnostics starting 60.0 seconds late 2024-11-22 01:27:22,629 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 01:27:22,630 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:27:22,630 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 67.5 seconds late 2024-11-22 01:27:24,508 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 01:27:24,508 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 01:27:24,508 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 01:41:15 2024-11-22 01:27:24,509 INFO 3318922:repeater task plot_hbins starting 69.4 seconds late 2024-11-22 01:27:28,697 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 01:27:28,697 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-22 01:27:28,697 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 01:41:15 2024-11-22 01:27:28,698 INFO 3318922:repeater task report_generator starting 73.6 seconds late 2024-11-22 01:27:33,228 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-22 01:27:33,228 INFO 3318922:repeater task report_generator ended after 4.5 seconds 2024-11-22 01:27:33,229 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/22 02:26:15 2024-11-22 01:27:38,234 INFO 3318922:repeater task noisy_rawtemps_os150 starting 78.3 seconds late 2024-11-22 01:27:41,321 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 01:27:41,321 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:27:41,322 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 81.3 seconds late 2024-11-22 01:27:44,534 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 01:27:44,535 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:27:44,535 INFO 3318922:repeater task noisy_rawtemps_os38 starting 84.6 seconds late 2024-11-22 01:27:48,068 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 01:27:48,069 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:27:48,069 INFO 3318922:repeater task lastensq_wh300 starting 89.1 seconds late 2024-11-22 01:27:51,328 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:27:51,329 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:26:14 2024-11-22 01:27:51,329 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:28:19 2024-11-22 01:27:51,329 INFO 3318922:repeater task quick_wh300 starting 92.3 seconds late 2024-11-22 01:27:57,132 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 01:27:57,132 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:26:14 2024-11-22 01:27:57,132 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:28:19 2024-11-22 01:27:57,133 INFO 3318922:repeater task lastfew_vec_wh300 starting 98.1 seconds late 2024-11-22 01:27:58,182 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:27:58,182 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:26:14 2024-11-22 01:27:58,182 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:28:19 2024-11-22 01:27:58,183 INFO 3318922:repeater task 3dayplots_wh300 starting 98.2 seconds late 2024-11-22 01:28:04,763 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:28:04,763 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:31:19 2024-11-22 01:28:19,795 INFO 3318922:repeater task lastensq_wh300 starting 0.8 seconds late 2024-11-22 01:28:23,157 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:28:23,157 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:28:14 2024-11-22 01:28:23,157 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:30:19 2024-11-22 01:28:23,158 INFO 3318922:repeater task quick_wh300 starting 4.2 seconds late 2024-11-22 01:28:28,826 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 01:28:28,826 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:28:14 2024-11-22 01:28:28,826 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:30:19 2024-11-22 01:28:28,827 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.8 seconds late 2024-11-22 01:28:29,890 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:28:29,891 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:28:14 2024-11-22 01:28:29,891 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:30:19 2024-11-22 01:30:20,182 INFO 3318922:repeater task lastensq_wh300 starting 1.2 seconds late 2024-11-22 01:30:23,548 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:30:23,549 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:30:14 2024-11-22 01:30:23,549 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:32:19 2024-11-22 01:30:23,549 INFO 3318922:repeater task quick_wh300 starting 4.5 seconds late 2024-11-22 01:30:29,067 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:30:29,067 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:30:14 2024-11-22 01:30:29,067 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:32:19 2024-11-22 01:30:29,067 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2024-11-22 01:30:30,125 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:30:30,126 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:30:14 2024-11-22 01:30:30,126 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:32:19 2024-11-22 01:31:20,269 INFO 3318922:repeater task 3dayplots_wh300 starting 0.3 seconds late 2024-11-22 01:31:26,889 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:31:26,889 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:31:26,890 INFO 3318922:repeater task lastensq_os150nb starting 7.9 seconds late 2024-11-22 01:31:29,460 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 01:31:29,460 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:31:14 2024-11-22 01:31:29,460 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:31:29,461 INFO 3318922:repeater task quick_os150nb starting 10.5 seconds late 2024-11-22 01:31:33,185 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 01:31:33,185 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:31:14 2024-11-22 01:31:33,185 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:31:33,185 INFO 3318922:repeater task lastfew_vec_os150nb starting 14.2 seconds late 2024-11-22 01:31:34,183 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:31:34,184 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:31:14 2024-11-22 01:31:34,184 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:31:34,184 INFO 3318922:repeater task 3dayplots_os150nb starting 14.2 seconds late 2024-11-22 01:31:40,557 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 01:31:40,557 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:31:40,558 INFO 3318922:repeater task lastensq_os38nb starting 19.6 seconds late 2024-11-22 01:31:43,043 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 01:31:43,044 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:31:16 2024-11-22 01:31:43,044 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:36:21 2024-11-22 01:31:43,044 INFO 3318922:repeater task quick_os38nb starting 22.0 seconds late 2024-11-22 01:31:47,126 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 01:31:47,127 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:31:16 2024-11-22 01:31:47,127 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:36:21 2024-11-22 01:31:47,127 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.1 seconds late 2024-11-22 01:31:48,154 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:31:48,154 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:31:16 2024-11-22 01:31:48,155 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:36:21 2024-11-22 01:31:48,155 INFO 3318922:repeater task 3dayplots_os38nb starting 28.2 seconds late 2024-11-22 01:31:54,347 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 01:31:54,348 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:32:19,408 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-22 01:32:22,723 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:32:22,723 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:32:14 2024-11-22 01:32:22,723 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:34:19 2024-11-22 01:32:22,724 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-22 01:32:28,013 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 01:32:28,013 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:32:14 2024-11-22 01:32:28,014 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:34:19 2024-11-22 01:32:28,014 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.0 seconds late 2024-11-22 01:32:29,078 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:32:29,079 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:32:14 2024-11-22 01:32:29,079 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:34:19 2024-11-22 01:34:19,412 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-22 01:34:22,758 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:34:22,758 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:34:14 2024-11-22 01:34:22,758 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:34:22,759 INFO 3318922:repeater task quick_wh300 starting 3.8 seconds late 2024-11-22 01:34:28,245 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:34:28,245 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:34:14 2024-11-22 01:34:28,245 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:34:28,245 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2024-11-22 01:34:29,299 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:34:29,299 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:34:14 2024-11-22 01:34:29,299 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:36:19 2024-11-22 01:36:19,616 INFO 3318922:repeater task lastensq_wh300 starting 0.6 seconds late 2024-11-22 01:36:22,912 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:36:22,912 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:36:14 2024-11-22 01:36:22,912 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:38:19 2024-11-22 01:36:22,912 INFO 3318922:repeater task quick_wh300 starting 3.9 seconds late 2024-11-22 01:36:28,452 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:36:28,453 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:36:14 2024-11-22 01:36:28,453 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:38:19 2024-11-22 01:36:28,453 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.5 seconds late 2024-11-22 01:36:29,490 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:36:29,490 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:36:14 2024-11-22 01:36:29,490 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:38:19 2024-11-22 01:36:29,490 INFO 3318922:repeater task 3dayplots_wh300 starting 9.5 seconds late 2024-11-22 01:36:36,050 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:36:36,050 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:41:19 2024-11-22 01:36:36,052 INFO 3318922:repeater task lastensq_os150nb starting 17.1 seconds late 2024-11-22 01:36:38,695 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 01:36:38,695 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:36:14 2024-11-22 01:36:38,695 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:41:19 2024-11-22 01:36:38,695 INFO 3318922:repeater task quick_os150nb starting 19.7 seconds late 2024-11-22 01:36:42,351 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 01:36:42,351 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:36:14 2024-11-22 01:36:42,351 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:41:19 2024-11-22 01:36:42,352 INFO 3318922:repeater task lastfew_vec_os150nb starting 23.4 seconds late 2024-11-22 01:36:43,369 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:36:43,370 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:36:14 2024-11-22 01:36:43,370 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:41:19 2024-11-22 01:36:43,370 INFO 3318922:repeater task 3dayplots_os150nb starting 23.4 seconds late 2024-11-22 01:36:49,576 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 01:36:49,576 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:41:19 2024-11-22 01:36:49,577 INFO 3318922:repeater task lastensq_os38nb starting 28.6 seconds late 2024-11-22 01:36:52,253 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 01:36:52,254 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:36:16 2024-11-22 01:36:52,254 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:41:21 2024-11-22 01:36:52,254 INFO 3318922:repeater task quick_os38nb starting 31.3 seconds late 2024-11-22 01:36:56,212 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 01:36:56,212 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:36:16 2024-11-22 01:36:56,212 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:41:21 2024-11-22 01:36:56,213 INFO 3318922:repeater task lastfew_vec_os38nb starting 35.2 seconds late 2024-11-22 01:36:57,244 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:36:57,245 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:36:16 2024-11-22 01:36:57,245 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:41:21 2024-11-22 01:36:57,245 INFO 3318922:repeater task 3dayplots_os38nb starting 37.3 seconds late 2024-11-22 01:37:03,615 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.4 seconds 2024-11-22 01:37:03,616 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:41:19 2024-11-22 01:37:03,617 INFO 3318922:repeater task daily_netcdf starting 43.6 seconds late 2024-11-22 01:37:04,393 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 01:37:04,393 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:37:09,398 INFO 3318922:repeater task noisy_rawtemps_os150 starting 49.4 seconds late 2024-11-22 01:37:12,484 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 01:37:12,484 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:37:12,485 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 52.5 seconds late 2024-11-22 01:37:15,617 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-22 01:37:15,617 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:37:15,618 INFO 3318922:repeater task noisy_rawtemps_os38 starting 55.6 seconds late 2024-11-22 01:37:19,205 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 01:37:19,205 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:38:19,387 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-22 01:38:22,666 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:38:22,666 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:38:14 2024-11-22 01:38:22,666 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:40:19 2024-11-22 01:38:22,666 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-22 01:38:28,221 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 01:38:28,221 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:38:14 2024-11-22 01:38:28,221 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:40:19 2024-11-22 01:38:28,221 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2024-11-22 01:38:29,263 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:38:29,263 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:38:14 2024-11-22 01:38:29,263 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:40:19 2024-11-22 01:40:19,595 INFO 3318922:repeater task lastensq_wh300 starting 0.6 seconds late 2024-11-22 01:40:22,845 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:40:22,845 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:40:14 2024-11-22 01:40:22,845 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:42:19 2024-11-22 01:40:22,846 INFO 3318922:repeater task quick_wh300 starting 3.8 seconds late 2024-11-22 01:40:28,310 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:40:28,310 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:40:14 2024-11-22 01:40:28,310 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:42:19 2024-11-22 01:40:28,310 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2024-11-22 01:40:29,359 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:40:29,360 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:40:14 2024-11-22 01:40:29,360 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:42:19 2024-11-22 01:41:19,495 INFO 3318922:repeater task plot_tsstats_wh300 starting 4.5 seconds late 2024-11-22 01:41:20,929 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:41:20,929 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 01:41:20,929 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:20,929 INFO 3318922:repeater task grab_lastpings_wh300 starting 6.0 seconds late 2024-11-22 01:41:21,091 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 01:41:21,091 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:21,091 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.1 seconds late 2024-11-22 01:41:22,829 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:41:22,829 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:22,830 INFO 3318922:repeater task beamdiagnostics_wh300 starting 7.9 seconds late 2024-11-22 01:41:24,514 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:41:24,516 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:24,516 INFO 3318922:repeater task lastensq_os150nb starting 5.5 seconds late 2024-11-22 01:41:27,180 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 01:41:27,180 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:41:14 2024-11-22 01:41:27,180 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:41:27,180 INFO 3318922:repeater task quick_os150nb starting 8.2 seconds late 2024-11-22 01:41:30,951 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 01:41:30,952 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:41:14 2024-11-22 01:41:30,952 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:41:30,952 INFO 3318922:repeater task lastfew_vec_os150nb starting 12.0 seconds late 2024-11-22 01:41:31,961 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:41:31,961 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:41:14 2024-11-22 01:41:31,961 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:41:31,961 INFO 3318922:repeater task 3dayplots_os150nb starting 12.0 seconds late 2024-11-22 01:41:38,171 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 01:41:38,171 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:41:38,171 INFO 3318922:repeater task plot_tsstats_os150nb starting 23.2 seconds late 2024-11-22 01:41:39,427 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:41:39,427 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 01:41:39,428 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:39,428 INFO 3318922:repeater task grab_lastpings_os150nb starting 24.5 seconds late 2024-11-22 01:41:39,588 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 01:41:39,588 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:39,588 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.6 seconds late 2024-11-22 01:41:41,311 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:41:41,311 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:41,312 INFO 3318922:repeater task beamdiagnostics_os150nb starting 26.3 seconds late 2024-11-22 01:41:43,027 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:41:43,027 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:43,027 INFO 3318922:repeater task lastensq_os38nb starting 22.0 seconds late 2024-11-22 01:41:45,473 INFO 3318922:repeater task lastensq_os38nb ended after 2.4 seconds 2024-11-22 01:41:45,473 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:41:16 2024-11-22 01:41:45,473 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:46:21 2024-11-22 01:41:45,473 INFO 3318922:repeater task quick_os38nb starting 24.5 seconds late 2024-11-22 01:41:49,418 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 01:41:49,418 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:41:16 2024-11-22 01:41:49,418 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:46:21 2024-11-22 01:41:49,418 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.4 seconds late 2024-11-22 01:41:50,432 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:41:50,432 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:41:16 2024-11-22 01:41:50,432 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:46:21 2024-11-22 01:41:50,432 INFO 3318922:repeater task 3dayplots_os38nb starting 30.5 seconds late 2024-11-22 01:41:56,762 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 01:41:56,762 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:41:56,762 INFO 3318922:repeater task plot_tsstats_os38nb starting 41.8 seconds late 2024-11-22 01:41:58,043 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:41:58,043 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 01:41:58,043 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:58,043 INFO 3318922:repeater task grab_lastpings_os38nb starting 43.1 seconds late 2024-11-22 01:41:58,215 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 01:41:58,216 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:58,216 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.2 seconds late 2024-11-22 01:41:59,850 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.6 seconds 2024-11-22 01:41:59,850 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:41:59,851 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.9 seconds late 2024-11-22 01:42:01,643 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 01:42:01,644 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 01:56:14 2024-11-22 01:42:01,645 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 46.5 seconds late 2024-11-22 01:42:03,423 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 01:42:03,423 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 01:42:03,423 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 01:56:15 2024-11-22 01:42:03,423 INFO 3318922:repeater task plot_hbins starting 48.3 seconds late 2024-11-22 01:42:07,536 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 01:42:07,536 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 01:42:07,537 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 01:56:15 2024-11-22 01:42:12,545 INFO 3318922:repeater task 3dayplots_wh300 starting 52.6 seconds late 2024-11-22 01:42:19,151 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:42:19,152 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:42:24,158 INFO 3318922:repeater task lastensq_wh300 starting 5.2 seconds late 2024-11-22 01:42:27,364 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 01:42:27,364 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:42:14 2024-11-22 01:42:27,365 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:44:19 2024-11-22 01:42:27,365 INFO 3318922:repeater task quick_wh300 starting 8.4 seconds late 2024-11-22 01:42:32,850 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 01:42:32,851 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:42:14 2024-11-22 01:42:32,851 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:44:19 2024-11-22 01:42:32,851 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2024-11-22 01:42:33,897 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:42:33,897 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:42:14 2024-11-22 01:42:33,898 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:44:19 2024-11-22 01:44:19,180 INFO 3318922:repeater task lastensq_wh300 starting 0.2 seconds late 2024-11-22 01:44:22,376 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 01:44:22,377 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:44:14 2024-11-22 01:44:22,377 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:44:22,377 INFO 3318922:repeater task quick_wh300 starting 3.4 seconds late 2024-11-22 01:44:28,026 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 01:44:28,027 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:44:14 2024-11-22 01:44:28,027 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:44:28,027 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.0 seconds late 2024-11-22 01:44:29,066 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:44:29,066 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:44:14 2024-11-22 01:44:29,066 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:46:19 2024-11-22 01:46:19,364 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-22 01:46:22,668 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:46:22,668 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:46:14 2024-11-22 01:46:22,669 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:48:19 2024-11-22 01:46:22,669 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-22 01:46:28,324 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 01:46:28,324 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:46:14 2024-11-22 01:46:28,324 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:48:19 2024-11-22 01:46:28,325 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2024-11-22 01:46:29,390 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:46:29,390 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:46:14 2024-11-22 01:46:29,391 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:48:19 2024-11-22 01:46:29,391 INFO 3318922:repeater task 3dayplots_wh300 starting 9.4 seconds late 2024-11-22 01:46:36,040 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:46:36,040 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:51:19 2024-11-22 01:46:36,041 INFO 3318922:repeater task lastensq_os150nb starting 17.0 seconds late 2024-11-22 01:46:38,715 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 01:46:38,715 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:46:14 2024-11-22 01:46:38,715 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:51:19 2024-11-22 01:46:38,716 INFO 3318922:repeater task quick_os150nb starting 19.7 seconds late 2024-11-22 01:46:42,391 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 01:46:42,391 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:46:14 2024-11-22 01:46:42,391 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:51:19 2024-11-22 01:46:42,392 INFO 3318922:repeater task lastfew_vec_os150nb starting 23.4 seconds late 2024-11-22 01:46:43,397 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:46:43,397 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:46:14 2024-11-22 01:46:43,397 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:51:19 2024-11-22 01:46:43,398 INFO 3318922:repeater task 3dayplots_os150nb starting 23.4 seconds late 2024-11-22 01:46:49,578 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 01:46:49,578 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:51:19 2024-11-22 01:46:49,579 INFO 3318922:repeater task lastensq_os38nb starting 28.6 seconds late 2024-11-22 01:46:52,125 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 01:46:52,126 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:46:16 2024-11-22 01:46:52,126 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:51:21 2024-11-22 01:46:52,126 INFO 3318922:repeater task quick_os38nb starting 31.1 seconds late 2024-11-22 01:46:56,007 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 01:46:56,007 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:46:16 2024-11-22 01:46:56,007 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:51:21 2024-11-22 01:46:56,007 INFO 3318922:repeater task lastfew_vec_os38nb starting 35.0 seconds late 2024-11-22 01:46:57,036 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:46:57,036 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:46:16 2024-11-22 01:46:57,036 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:51:21 2024-11-22 01:46:57,036 INFO 3318922:repeater task 3dayplots_os38nb starting 37.1 seconds late 2024-11-22 01:47:03,312 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 01:47:03,313 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:51:19 2024-11-22 01:47:03,314 INFO 3318922:repeater task daily_netcdf starting 43.3 seconds late 2024-11-22 01:47:04,033 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 01:47:04,033 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:47:09,038 INFO 3318922:repeater task noisy_rawtemps_os150 starting 49.1 seconds late 2024-11-22 01:47:12,141 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 01:47:12,141 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:47:12,142 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 52.2 seconds late 2024-11-22 01:47:15,381 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 01:47:15,381 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:47:15,381 INFO 3318922:repeater task noisy_rawtemps_os38 starting 55.4 seconds late 2024-11-22 01:47:18,955 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 01:47:18,955 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:48:19,120 INFO 3318922:repeater task lastensq_wh300 starting 0.1 seconds late 2024-11-22 01:48:22,415 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:48:22,416 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:48:14 2024-11-22 01:48:22,416 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:50:19 2024-11-22 01:48:22,416 INFO 3318922:repeater task quick_wh300 starting 3.4 seconds late 2024-11-22 01:48:28,001 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 01:48:28,001 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:48:14 2024-11-22 01:48:28,002 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:50:19 2024-11-22 01:48:28,002 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.0 seconds late 2024-11-22 01:48:29,045 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:48:29,045 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:48:14 2024-11-22 01:48:29,046 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:50:19 2024-11-22 01:50:19,360 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-22 01:50:22,524 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 01:50:22,524 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:50:14 2024-11-22 01:50:22,524 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:52:19 2024-11-22 01:50:22,525 INFO 3318922:repeater task quick_wh300 starting 3.5 seconds late 2024-11-22 01:50:27,853 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 01:50:27,853 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:50:14 2024-11-22 01:50:27,853 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:52:19 2024-11-22 01:50:27,853 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.9 seconds late 2024-11-22 01:50:28,887 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:50:28,887 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:50:14 2024-11-22 01:50:28,887 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:52:19 2024-11-22 01:51:19,032 INFO 3318922:repeater task lastensq_os150nb starting 0.0 seconds late 2024-11-22 01:51:21,803 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 01:51:21,803 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:51:14 2024-11-22 01:51:21,803 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:51:21,804 INFO 3318922:repeater task quick_os150nb starting 2.8 seconds late 2024-11-22 01:51:25,502 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 01:51:25,502 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:51:14 2024-11-22 01:51:25,502 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:51:25,503 INFO 3318922:repeater task lastfew_vec_os150nb starting 6.5 seconds late 2024-11-22 01:51:26,532 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:51:26,533 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:51:14 2024-11-22 01:51:26,533 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:51:26,533 INFO 3318922:repeater task 3dayplots_os150nb starting 6.6 seconds late 2024-11-22 01:51:32,700 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 01:51:32,700 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:51:32,702 INFO 3318922:repeater task lastensq_os38nb starting 11.7 seconds late 2024-11-22 01:51:35,282 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 01:51:35,283 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:51:16 2024-11-22 01:51:35,283 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 01:56:21 2024-11-22 01:51:35,283 INFO 3318922:repeater task quick_os38nb starting 14.3 seconds late 2024-11-22 01:51:39,119 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 01:51:39,120 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:51:16 2024-11-22 01:51:39,120 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 01:56:21 2024-11-22 01:51:39,120 INFO 3318922:repeater task lastfew_vec_os38nb starting 18.1 seconds late 2024-11-22 01:51:40,144 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:51:40,144 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:51:16 2024-11-22 01:51:40,144 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 01:56:21 2024-11-22 01:51:40,144 INFO 3318922:repeater task 3dayplots_os38nb starting 20.2 seconds late 2024-11-22 01:51:46,295 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 01:51:46,296 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:51:51,305 INFO 3318922:repeater task 3dayplots_wh300 starting 31.3 seconds late 2024-11-22 01:51:57,881 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 01:51:57,882 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:52:22,945 INFO 3318922:repeater task lastensq_wh300 starting 3.9 seconds late 2024-11-22 01:52:26,470 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-22 01:52:26,470 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:52:14 2024-11-22 01:52:26,471 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:54:19 2024-11-22 01:52:26,471 INFO 3318922:repeater task quick_wh300 starting 7.5 seconds late 2024-11-22 01:52:31,829 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 01:52:31,829 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:52:14 2024-11-22 01:52:31,829 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:54:19 2024-11-22 01:52:31,829 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2024-11-22 01:52:32,888 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:52:32,889 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:52:14 2024-11-22 01:52:32,889 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:54:19 2024-11-22 01:54:23,205 INFO 3318922:repeater task lastensq_wh300 starting 4.2 seconds late 2024-11-22 01:54:26,638 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:54:26,639 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:54:14 2024-11-22 01:54:26,639 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:54:26,639 INFO 3318922:repeater task quick_wh300 starting 7.6 seconds late 2024-11-22 01:54:32,036 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 01:54:32,036 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:54:14 2024-11-22 01:54:32,036 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:54:32,036 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.0 seconds late 2024-11-22 01:54:33,082 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:54:33,082 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:54:14 2024-11-22 01:54:33,082 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:56:19 2024-11-22 01:56:18,398 INFO 3318922:repeater task plot_tsstats_wh300 starting 3.4 seconds late 2024-11-22 01:56:19,835 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:56:19,835 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 01:56:19,835 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:19,835 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.9 seconds late 2024-11-22 01:56:19,997 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 01:56:19,997 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:19,997 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.0 seconds late 2024-11-22 01:56:21,733 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:56:21,733 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:21,733 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.8 seconds late 2024-11-22 01:56:23,481 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 01:56:23,483 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:23,484 INFO 3318922:repeater task lastensq_os150nb starting 4.5 seconds late 2024-11-22 01:56:26,093 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 01:56:26,093 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:56:14 2024-11-22 01:56:26,093 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:01:19 2024-11-22 01:56:26,093 INFO 3318922:repeater task quick_os150nb starting 7.1 seconds late 2024-11-22 01:56:29,665 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 01:56:29,665 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:56:14 2024-11-22 01:56:29,665 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:01:19 2024-11-22 01:56:29,666 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.7 seconds late 2024-11-22 01:56:30,695 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 01:56:30,695 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 01:56:14 2024-11-22 01:56:30,695 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:01:19 2024-11-22 01:56:30,696 INFO 3318922:repeater task 3dayplots_os150nb starting 10.7 seconds late 2024-11-22 01:56:37,104 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 01:56:37,104 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:01:19 2024-11-22 01:56:37,104 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.1 seconds late 2024-11-22 01:56:38,372 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:56:38,372 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 01:56:38,372 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:38,372 INFO 3318922:repeater task grab_lastpings_os150nb starting 23.4 seconds late 2024-11-22 01:56:38,532 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 01:56:38,532 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:38,532 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.6 seconds late 2024-11-22 01:56:40,266 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:56:40,266 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:40,267 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.3 seconds late 2024-11-22 01:56:41,989 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 01:56:41,990 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:41,990 INFO 3318922:repeater task lastensq_os38nb starting 21.0 seconds late 2024-11-22 01:56:44,608 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 01:56:44,608 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:56:16 2024-11-22 01:56:44,609 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:01:21 2024-11-22 01:56:44,609 INFO 3318922:repeater task quick_os38nb starting 23.6 seconds late 2024-11-22 01:56:48,386 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 01:56:48,386 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:56:16 2024-11-22 01:56:48,386 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:01:21 2024-11-22 01:56:48,386 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.4 seconds late 2024-11-22 01:56:49,400 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 01:56:49,401 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 01:56:16 2024-11-22 01:56:49,401 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:01:21 2024-11-22 01:56:49,401 INFO 3318922:repeater task 3dayplots_os38nb starting 29.4 seconds late 2024-11-22 01:56:55,542 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 01:56:55,542 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:01:19 2024-11-22 01:56:55,542 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.6 seconds late 2024-11-22 01:56:56,822 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 01:56:56,822 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 01:56:56,822 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:56,822 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.8 seconds late 2024-11-22 01:56:57,016 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 01:56:57,016 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:57,017 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.0 seconds late 2024-11-22 01:56:58,723 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 01:56:58,723 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:56:58,723 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.7 seconds late 2024-11-22 01:57:00,494 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 01:57:00,494 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:11:14 2024-11-22 01:57:00,495 INFO 3318922:repeater task daily_netcdf starting 40.5 seconds late 2024-11-22 01:57:01,168 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 01:57:01,168 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 02:06:19 2024-11-22 01:57:01,169 INFO 3318922:repeater task ggatime_diagnostics starting 46.2 seconds late 2024-11-22 01:57:08,955 INFO 3318922:repeater task ggatime_diagnostics ended after 7.8 seconds 2024-11-22 01:57:08,956 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 02:26:14 2024-11-22 01:57:08,956 INFO 3318922:repeater task ggatime_diagnostics starting 54.0 seconds late 2024-11-22 01:57:16,423 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 01:57:16,423 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 02:26:14 2024-11-22 01:57:16,424 INFO 3318922:repeater task ggatime_diagnostics starting 61.4 seconds late 2024-11-22 01:57:23,930 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 01:57:23,930 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 02:26:14 2024-11-22 01:57:23,930 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 68.8 seconds late 2024-11-22 01:57:25,758 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 01:57:25,758 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 01:57:25,758 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 02:11:15 2024-11-22 01:57:25,758 INFO 3318922:repeater task plot_hbins starting 70.6 seconds late 2024-11-22 01:57:29,903 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 01:57:29,903 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 01:57:29,903 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 02:11:15 2024-11-22 01:57:34,909 INFO 3318922:repeater task noisy_rawtemps_os150 starting 74.9 seconds late 2024-11-22 01:57:38,074 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 01:57:38,074 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 02:06:19 2024-11-22 01:57:38,074 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 78.1 seconds late 2024-11-22 01:57:41,236 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 01:57:41,237 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 02:06:19 2024-11-22 01:57:41,237 INFO 3318922:repeater task noisy_rawtemps_os38 starting 81.3 seconds late 2024-11-22 01:57:44,756 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 01:57:44,757 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 02:06:19 2024-11-22 01:57:44,757 INFO 3318922:repeater task lastensq_wh300 starting 85.8 seconds late 2024-11-22 01:57:48,032 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 01:57:48,032 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:56:14 2024-11-22 01:57:48,032 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 01:58:19 2024-11-22 01:57:48,033 INFO 3318922:repeater task quick_wh300 starting 89.0 seconds late 2024-11-22 01:57:53,880 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 01:57:53,880 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:56:14 2024-11-22 01:57:53,880 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 01:58:19 2024-11-22 01:57:53,881 INFO 3318922:repeater task lastfew_vec_wh300 starting 94.9 seconds late 2024-11-22 01:57:54,986 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 01:57:54,986 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:56:14 2024-11-22 01:57:54,986 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 01:58:19 2024-11-22 01:57:54,987 INFO 3318922:repeater task 3dayplots_wh300 starting 95.0 seconds late 2024-11-22 01:58:01,518 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 01:58:01,519 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:01:19 2024-11-22 01:58:21,565 INFO 3318922:repeater task lastensq_wh300 starting 2.6 seconds late 2024-11-22 01:58:24,919 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 01:58:24,919 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:58:14 2024-11-22 01:58:24,919 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:00:19 2024-11-22 01:58:24,920 INFO 3318922:repeater task quick_wh300 starting 5.9 seconds late 2024-11-22 01:58:30,553 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 01:58:30,553 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:58:14 2024-11-22 01:58:30,553 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:00:19 2024-11-22 01:58:30,553 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2024-11-22 01:58:31,596 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 01:58:31,596 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 01:58:14 2024-11-22 01:58:31,596 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:00:19 2024-11-22 02:00:21,932 INFO 3318922:repeater task lastensq_wh300 starting 2.9 seconds late 2024-11-22 02:00:25,246 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:00:25,246 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:00:14 2024-11-22 02:00:25,247 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:02:19 2024-11-22 02:00:25,247 INFO 3318922:repeater task quick_wh300 starting 6.2 seconds late 2024-11-22 02:00:30,862 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:00:30,862 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:00:14 2024-11-22 02:00:30,862 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:02:19 2024-11-22 02:00:30,863 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2024-11-22 02:00:31,918 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:00:31,918 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:00:14 2024-11-22 02:00:31,918 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:02:19 2024-11-22 02:01:22,061 INFO 3318922:repeater task 3dayplots_wh300 starting 2.1 seconds late 2024-11-22 02:01:28,645 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 02:01:28,645 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:01:28,646 INFO 3318922:repeater task lastensq_os150nb starting 9.6 seconds late 2024-11-22 02:01:31,198 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 02:01:31,198 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:01:14 2024-11-22 02:01:31,198 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:01:31,198 INFO 3318922:repeater task quick_os150nb starting 12.2 seconds late 2024-11-22 02:01:34,896 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 02:01:34,896 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:01:14 2024-11-22 02:01:34,897 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:01:34,897 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.9 seconds late 2024-11-22 02:01:35,929 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:01:35,930 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:01:14 2024-11-22 02:01:35,930 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:01:35,930 INFO 3318922:repeater task 3dayplots_os150nb starting 16.0 seconds late 2024-11-22 02:01:42,196 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 02:01:42,196 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:01:42,197 INFO 3318922:repeater task lastensq_os38nb starting 21.2 seconds late 2024-11-22 02:01:44,788 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 02:01:44,788 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:01:16 2024-11-22 02:01:44,788 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:06:21 2024-11-22 02:01:44,789 INFO 3318922:repeater task quick_os38nb starting 23.8 seconds late 2024-11-22 02:01:48,722 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 02:01:48,723 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:01:16 2024-11-22 02:01:48,723 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:06:21 2024-11-22 02:01:48,723 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.7 seconds late 2024-11-22 02:01:49,752 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:01:49,752 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:01:16 2024-11-22 02:01:49,753 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:06:21 2024-11-22 02:01:49,753 INFO 3318922:repeater task 3dayplots_os38nb starting 29.8 seconds late 2024-11-22 02:01:55,927 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 02:01:55,927 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:02:20,996 INFO 3318922:repeater task lastensq_wh300 starting 2.0 seconds late 2024-11-22 02:02:24,201 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:02:24,202 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:02:14 2024-11-22 02:02:24,202 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:04:19 2024-11-22 02:02:24,202 INFO 3318922:repeater task quick_wh300 starting 5.2 seconds late 2024-11-22 02:02:29,534 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 02:02:29,534 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:02:14 2024-11-22 02:02:29,534 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:04:19 2024-11-22 02:02:29,535 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-22 02:02:30,583 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:02:30,583 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:02:14 2024-11-22 02:02:30,583 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:04:19 2024-11-22 02:04:20,908 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 02:04:24,276 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 02:04:24,276 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:04:14 2024-11-22 02:04:24,276 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:04:24,277 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-22 02:04:29,803 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 02:04:29,803 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:04:14 2024-11-22 02:04:29,803 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:04:29,803 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-22 02:04:30,847 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:04:30,847 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:04:14 2024-11-22 02:04:30,847 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:06:19 2024-11-22 02:06:21,158 INFO 3318922:repeater task noisy_rawtemps_os150 starting 1.2 seconds late 2024-11-22 02:06:24,252 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 02:06:24,252 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:06:24,252 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 4.3 seconds late 2024-11-22 02:06:27,434 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 02:06:27,435 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:06:27,435 INFO 3318922:repeater task noisy_rawtemps_os38 starting 7.5 seconds late 2024-11-22 02:06:30,892 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 02:06:30,892 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:06:30,893 INFO 3318922:repeater task lastensq_wh300 starting 11.9 seconds late 2024-11-22 02:06:34,128 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:06:34,128 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:06:14 2024-11-22 02:06:34,128 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:08:19 2024-11-22 02:06:34,129 INFO 3318922:repeater task quick_wh300 starting 15.1 seconds late 2024-11-22 02:06:39,781 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 02:06:39,782 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:06:14 2024-11-22 02:06:39,782 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:08:19 2024-11-22 02:06:39,782 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.8 seconds late 2024-11-22 02:06:40,839 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:06:40,840 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:06:14 2024-11-22 02:06:40,840 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:08:19 2024-11-22 02:06:40,840 INFO 3318922:repeater task 3dayplots_wh300 starting 20.9 seconds late 2024-11-22 02:06:47,372 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 02:06:47,372 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:11:19 2024-11-22 02:06:47,374 INFO 3318922:repeater task lastensq_os150nb starting 28.4 seconds late 2024-11-22 02:06:50,084 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 02:06:50,085 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:06:14 2024-11-22 02:06:50,085 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:11:19 2024-11-22 02:06:50,085 INFO 3318922:repeater task quick_os150nb starting 31.1 seconds late 2024-11-22 02:06:53,662 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 02:06:53,662 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:06:14 2024-11-22 02:06:53,662 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:11:19 2024-11-22 02:06:53,663 INFO 3318922:repeater task lastfew_vec_os150nb starting 34.7 seconds late 2024-11-22 02:06:54,692 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:06:54,692 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:06:14 2024-11-22 02:06:54,692 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:11:19 2024-11-22 02:06:54,693 INFO 3318922:repeater task 3dayplots_os150nb starting 34.7 seconds late 2024-11-22 02:07:00,956 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 02:07:00,956 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:11:19 2024-11-22 02:07:00,957 INFO 3318922:repeater task lastensq_os38nb starting 40.0 seconds late 2024-11-22 02:07:03,572 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 02:07:03,572 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:06:16 2024-11-22 02:07:03,573 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:11:21 2024-11-22 02:07:03,573 INFO 3318922:repeater task quick_os38nb starting 42.6 seconds late 2024-11-22 02:07:07,225 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-22 02:07:07,225 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:06:16 2024-11-22 02:07:07,225 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:11:21 2024-11-22 02:07:07,226 INFO 3318922:repeater task lastfew_vec_os38nb starting 46.2 seconds late 2024-11-22 02:07:08,242 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:07:08,242 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:06:16 2024-11-22 02:07:08,242 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:11:21 2024-11-22 02:07:08,242 INFO 3318922:repeater task 3dayplots_os38nb starting 48.3 seconds late 2024-11-22 02:07:14,335 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 02:07:14,335 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:11:19 2024-11-22 02:07:14,336 INFO 3318922:repeater task daily_netcdf starting 54.4 seconds late 2024-11-22 02:07:15,108 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 02:07:15,108 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:08:20,288 INFO 3318922:repeater task lastensq_wh300 starting 1.3 seconds late 2024-11-22 02:08:23,714 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 02:08:23,715 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:08:14 2024-11-22 02:08:23,715 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:10:19 2024-11-22 02:08:23,715 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 02:08:29,087 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:08:29,087 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:08:14 2024-11-22 02:08:29,088 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:10:19 2024-11-22 02:08:29,088 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2024-11-22 02:08:30,130 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:08:30,130 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:08:14 2024-11-22 02:08:30,130 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:10:19 2024-11-22 02:10:20,396 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 02:10:23,691 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:10:23,691 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:10:14 2024-11-22 02:10:23,691 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:12:19 2024-11-22 02:10:23,691 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 02:10:29,168 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 02:10:29,168 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:10:14 2024-11-22 02:10:29,168 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:12:19 2024-11-22 02:10:29,168 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2024-11-22 02:10:30,229 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:10:30,229 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:10:14 2024-11-22 02:10:30,229 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:12:19 2024-11-22 02:11:15,365 INFO 3318922:repeater task plot_tsstats_wh300 starting 0.4 seconds late 2024-11-22 02:11:16,818 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:11:16,818 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 02:11:16,819 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:16,819 INFO 3318922:repeater task grab_lastpings_wh300 starting 1.8 seconds late 2024-11-22 02:11:16,983 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 02:11:16,983 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:16,983 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.0 seconds late 2024-11-22 02:11:18,719 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 02:11:18,719 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:18,720 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.7 seconds late 2024-11-22 02:11:20,380 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 02:11:20,382 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:20,383 INFO 3318922:repeater task lastensq_os150nb starting 1.4 seconds late 2024-11-22 02:11:23,030 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 02:11:23,031 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:11:14 2024-11-22 02:11:23,031 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:11:23,031 INFO 3318922:repeater task quick_os150nb starting 4.0 seconds late 2024-11-22 02:11:26,685 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 02:11:26,685 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:11:14 2024-11-22 02:11:26,685 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:11:26,686 INFO 3318922:repeater task lastfew_vec_os150nb starting 7.7 seconds late 2024-11-22 02:11:27,713 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:11:27,713 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:11:14 2024-11-22 02:11:27,713 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:11:27,714 INFO 3318922:repeater task 3dayplots_os150nb starting 7.7 seconds late 2024-11-22 02:11:33,861 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.1 seconds 2024-11-22 02:11:33,861 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:11:33,861 INFO 3318922:repeater task plot_tsstats_os150nb starting 18.9 seconds late 2024-11-22 02:11:35,116 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:11:35,116 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 02:11:35,116 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:35,116 INFO 3318922:repeater task grab_lastpings_os150nb starting 20.1 seconds late 2024-11-22 02:11:35,277 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 02:11:35,277 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:35,278 INFO 3318922:repeater task beamdiagnostics_os150nb starting 20.3 seconds late 2024-11-22 02:11:37,006 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:11:37,006 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:37,006 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.0 seconds late 2024-11-22 02:11:38,711 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:11:38,712 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:38,712 INFO 3318922:repeater task lastensq_os38nb starting 17.7 seconds late 2024-11-22 02:11:41,410 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 02:11:41,411 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:11:16 2024-11-22 02:11:41,411 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:16:21 2024-11-22 02:11:41,411 INFO 3318922:repeater task quick_os38nb starting 20.4 seconds late 2024-11-22 02:11:45,152 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-22 02:11:45,153 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:11:16 2024-11-22 02:11:45,153 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:16:21 2024-11-22 02:11:45,153 INFO 3318922:repeater task lastfew_vec_os38nb starting 24.2 seconds late 2024-11-22 02:11:46,183 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:11:46,183 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:11:16 2024-11-22 02:11:46,183 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:16:21 2024-11-22 02:11:46,183 INFO 3318922:repeater task 3dayplots_os38nb starting 26.2 seconds late 2024-11-22 02:11:52,418 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 02:11:52,418 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:11:52,419 INFO 3318922:repeater task plot_tsstats_os38nb starting 37.4 seconds late 2024-11-22 02:11:53,696 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:11:53,696 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 02:11:53,696 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:53,696 INFO 3318922:repeater task grab_lastpings_os38nb starting 38.7 seconds late 2024-11-22 02:11:53,856 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 02:11:53,856 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:53,857 INFO 3318922:repeater task beamdiagnostics_os38nb starting 38.9 seconds late 2024-11-22 02:11:55,649 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 02:11:55,649 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:55,649 INFO 3318922:repeater task beamdiagnostics_os38nb starting 40.7 seconds late 2024-11-22 02:11:57,432 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 02:11:57,432 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:26:14 2024-11-22 02:11:57,433 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 42.3 seconds late 2024-11-22 02:11:59,242 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 02:11:59,242 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 02:11:59,242 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 02:26:15 2024-11-22 02:11:59,242 INFO 3318922:repeater task plot_hbins starting 44.1 seconds late 2024-11-22 02:12:03,295 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 02:12:03,295 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 02:12:03,296 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 02:26:15 2024-11-22 02:12:08,303 INFO 3318922:repeater task 3dayplots_wh300 starting 48.3 seconds late 2024-11-22 02:12:14,837 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 02:12:14,837 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:12:19,847 INFO 3318922:repeater task lastensq_wh300 starting 0.8 seconds late 2024-11-22 02:12:23,104 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:12:23,104 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:12:14 2024-11-22 02:12:23,104 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:14:19 2024-11-22 02:12:23,105 INFO 3318922:repeater task quick_wh300 starting 4.1 seconds late 2024-11-22 02:12:28,511 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:12:28,512 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:12:14 2024-11-22 02:12:28,512 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:14:19 2024-11-22 02:12:28,512 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.5 seconds late 2024-11-22 02:12:29,554 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:12:29,554 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:12:14 2024-11-22 02:12:29,554 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:14:19 2024-11-22 02:14:19,872 INFO 3318922:repeater task lastensq_wh300 starting 0.9 seconds late 2024-11-22 02:14:23,274 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 02:14:23,275 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:14:14 2024-11-22 02:14:23,275 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:14:23,275 INFO 3318922:repeater task quick_wh300 starting 4.3 seconds late 2024-11-22 02:14:28,885 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:14:28,885 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:14:14 2024-11-22 02:14:28,886 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:14:28,886 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2024-11-22 02:14:29,936 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:14:29,936 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:14:14 2024-11-22 02:14:29,936 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:16:19 2024-11-22 02:16:20,290 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.3 seconds late 2024-11-22 02:16:23,421 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 02:16:23,421 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:16:23,422 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.4 seconds late 2024-11-22 02:16:26,630 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 02:16:26,630 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:16:26,631 INFO 3318922:repeater task noisy_rawtemps_os38 starting 6.7 seconds late 2024-11-22 02:16:30,133 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 02:16:30,133 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:16:30,134 INFO 3318922:repeater task lastensq_wh300 starting 11.1 seconds late 2024-11-22 02:16:33,389 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:16:33,389 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:16:14 2024-11-22 02:16:33,389 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:18:19 2024-11-22 02:16:33,389 INFO 3318922:repeater task quick_wh300 starting 14.4 seconds late 2024-11-22 02:16:38,842 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 02:16:38,842 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:16:14 2024-11-22 02:16:38,842 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:18:19 2024-11-22 02:16:38,843 INFO 3318922:repeater task lastfew_vec_wh300 starting 19.8 seconds late 2024-11-22 02:16:39,892 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:16:39,893 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:16:14 2024-11-22 02:16:39,893 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:18:19 2024-11-22 02:16:39,893 INFO 3318922:repeater task 3dayplots_wh300 starting 19.9 seconds late 2024-11-22 02:16:46,398 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 02:16:46,398 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:21:19 2024-11-22 02:16:46,400 INFO 3318922:repeater task lastensq_os150nb starting 27.4 seconds late 2024-11-22 02:16:49,113 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 02:16:49,113 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:16:14 2024-11-22 02:16:49,113 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:21:19 2024-11-22 02:16:49,113 INFO 3318922:repeater task quick_os150nb starting 30.1 seconds late 2024-11-22 02:16:52,705 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 02:16:52,705 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:16:14 2024-11-22 02:16:52,705 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:21:19 2024-11-22 02:16:52,705 INFO 3318922:repeater task lastfew_vec_os150nb starting 33.7 seconds late 2024-11-22 02:16:53,719 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:16:53,720 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:16:14 2024-11-22 02:16:53,720 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:21:19 2024-11-22 02:16:53,720 INFO 3318922:repeater task 3dayplots_os150nb starting 33.7 seconds late 2024-11-22 02:17:00,058 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 02:17:00,059 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:21:19 2024-11-22 02:17:00,060 INFO 3318922:repeater task lastensq_os38nb starting 39.1 seconds late 2024-11-22 02:17:02,703 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 02:17:02,703 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:16:16 2024-11-22 02:17:02,704 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:21:21 2024-11-22 02:17:02,704 INFO 3318922:repeater task quick_os38nb starting 41.7 seconds late 2024-11-22 02:17:06,542 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 02:17:06,542 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:16:16 2024-11-22 02:17:06,542 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:21:21 2024-11-22 02:17:06,542 INFO 3318922:repeater task lastfew_vec_os38nb starting 45.5 seconds late 2024-11-22 02:17:07,578 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:17:07,579 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:16:16 2024-11-22 02:17:07,579 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:21:21 2024-11-22 02:17:07,579 INFO 3318922:repeater task 3dayplots_os38nb starting 47.6 seconds late 2024-11-22 02:17:13,678 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 02:17:13,679 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:21:19 2024-11-22 02:17:13,680 INFO 3318922:repeater task daily_netcdf starting 53.7 seconds late 2024-11-22 02:17:14,413 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 02:17:14,413 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:18:19,593 INFO 3318922:repeater task lastensq_wh300 starting 0.6 seconds late 2024-11-22 02:18:22,934 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:18:22,934 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:18:14 2024-11-22 02:18:22,934 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:20:19 2024-11-22 02:18:22,935 INFO 3318922:repeater task quick_wh300 starting 3.9 seconds late 2024-11-22 02:18:28,358 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:18:28,358 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:18:14 2024-11-22 02:18:28,358 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:20:19 2024-11-22 02:18:28,358 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.4 seconds late 2024-11-22 02:18:29,395 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:18:29,395 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:18:14 2024-11-22 02:18:29,395 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:20:19 2024-11-22 02:20:19,704 INFO 3318922:repeater task lastensq_wh300 starting 0.7 seconds late 2024-11-22 02:20:23,029 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:20:23,029 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:20:14 2024-11-22 02:20:23,029 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:22:19 2024-11-22 02:20:23,029 INFO 3318922:repeater task quick_wh300 starting 4.0 seconds late 2024-11-22 02:20:28,594 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:20:28,594 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:20:14 2024-11-22 02:20:28,594 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:22:19 2024-11-22 02:20:28,594 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.6 seconds late 2024-11-22 02:20:29,638 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:20:29,638 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:20:14 2024-11-22 02:20:29,638 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:22:19 2024-11-22 02:21:19,775 INFO 3318922:repeater task lastensq_os150nb starting 0.8 seconds late 2024-11-22 02:21:22,609 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 02:21:22,609 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:21:14 2024-11-22 02:21:22,610 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:21:22,610 INFO 3318922:repeater task quick_os150nb starting 3.6 seconds late 2024-11-22 02:21:26,374 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 02:21:26,375 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:21:14 2024-11-22 02:21:26,375 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:21:26,375 INFO 3318922:repeater task lastfew_vec_os150nb starting 7.4 seconds late 2024-11-22 02:21:27,398 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:21:27,398 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:21:14 2024-11-22 02:21:27,398 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:21:27,398 INFO 3318922:repeater task 3dayplots_os150nb starting 7.4 seconds late 2024-11-22 02:21:33,688 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 02:21:33,688 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:21:33,689 INFO 3318922:repeater task lastensq_os38nb starting 12.7 seconds late 2024-11-22 02:21:36,447 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-22 02:21:36,447 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:21:16 2024-11-22 02:21:36,447 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:26:21 2024-11-22 02:21:36,448 INFO 3318922:repeater task quick_os38nb starting 15.4 seconds late 2024-11-22 02:21:40,236 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 02:21:40,236 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:21:16 2024-11-22 02:21:40,236 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:26:21 2024-11-22 02:21:40,236 INFO 3318922:repeater task lastfew_vec_os38nb starting 19.2 seconds late 2024-11-22 02:21:41,245 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:21:41,245 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:21:16 2024-11-22 02:21:41,245 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:26:21 2024-11-22 02:21:41,246 INFO 3318922:repeater task 3dayplots_os38nb starting 21.3 seconds late 2024-11-22 02:21:47,381 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 02:21:47,381 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:21:52,388 INFO 3318922:repeater task 3dayplots_wh300 starting 32.4 seconds late 2024-11-22 02:21:58,997 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 02:21:58,997 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:22:19,051 INFO 3318922:repeater task lastensq_wh300 starting 0.1 seconds late 2024-11-22 02:22:22,281 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:22:22,281 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:22:14 2024-11-22 02:22:22,281 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:24:19 2024-11-22 02:22:22,282 INFO 3318922:repeater task quick_wh300 starting 3.3 seconds late 2024-11-22 02:22:27,727 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:22:27,728 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:22:14 2024-11-22 02:22:27,728 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:24:19 2024-11-22 02:22:27,728 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.7 seconds late 2024-11-22 02:22:28,757 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:22:28,757 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:22:14 2024-11-22 02:22:28,757 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:24:19 2024-11-22 02:24:19,096 INFO 3318922:repeater task lastensq_wh300 starting 0.1 seconds late 2024-11-22 02:24:22,406 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:24:22,406 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:24:14 2024-11-22 02:24:22,406 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:24:22,406 INFO 3318922:repeater task quick_wh300 starting 3.4 seconds late 2024-11-22 02:24:27,901 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 02:24:27,901 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:24:14 2024-11-22 02:24:27,901 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:24:27,901 INFO 3318922:repeater task lastfew_vec_wh300 starting 8.9 seconds late 2024-11-22 02:24:28,955 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:24:28,955 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:24:14 2024-11-22 02:24:28,955 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:26:19 2024-11-22 02:26:19,288 INFO 3318922:repeater task lastensq_wh300 starting 0.3 seconds late 2024-11-22 02:26:22,616 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:26:22,616 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:26:14 2024-11-22 02:26:22,616 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:28:19 2024-11-22 02:26:22,617 INFO 3318922:repeater task quick_wh300 starting 3.6 seconds late 2024-11-22 02:26:28,174 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:26:28,175 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:26:14 2024-11-22 02:26:28,175 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:28:19 2024-11-22 02:26:28,175 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2024-11-22 02:26:29,229 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:26:29,229 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:26:14 2024-11-22 02:26:29,229 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:28:19 2024-11-22 02:26:29,230 INFO 3318922:repeater task 3dayplots_wh300 starting 9.3 seconds late 2024-11-22 02:26:35,775 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 02:26:35,775 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:31:19 2024-11-22 02:26:35,776 INFO 3318922:repeater task plot_tsstats_wh300 starting 20.8 seconds late 2024-11-22 02:26:37,281 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:26:37,281 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 02:26:37,281 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:37,282 INFO 3318922:repeater task grab_lastpings_wh300 starting 22.3 seconds late 2024-11-22 02:26:37,443 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 02:26:37,443 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:37,443 INFO 3318922:repeater task beamdiagnostics_wh300 starting 22.5 seconds late 2024-11-22 02:26:39,146 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 02:26:39,146 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:39,147 INFO 3318922:repeater task beamdiagnostics_wh300 starting 24.2 seconds late 2024-11-22 02:26:40,871 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 02:26:40,871 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:40,871 INFO 3318922:repeater task lastensq_os150nb starting 21.9 seconds late 2024-11-22 02:26:43,595 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 02:26:43,595 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:26:14 2024-11-22 02:26:43,595 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:31:19 2024-11-22 02:26:43,596 INFO 3318922:repeater task quick_os150nb starting 24.6 seconds late 2024-11-22 02:26:47,297 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 02:26:47,298 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:26:14 2024-11-22 02:26:47,298 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:31:19 2024-11-22 02:26:47,298 INFO 3318922:repeater task lastfew_vec_os150nb starting 28.3 seconds late 2024-11-22 02:26:48,321 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:26:48,321 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:26:14 2024-11-22 02:26:48,321 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:31:19 2024-11-22 02:26:48,322 INFO 3318922:repeater task 3dayplots_os150nb starting 28.3 seconds late 2024-11-22 02:26:54,567 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 02:26:54,567 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:31:19 2024-11-22 02:26:54,567 INFO 3318922:repeater task plot_tsstats_os150nb starting 39.6 seconds late 2024-11-22 02:26:55,823 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:26:55,823 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 02:26:55,825 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:55,825 INFO 3318922:repeater task grab_lastpings_os150nb starting 40.8 seconds late 2024-11-22 02:26:55,985 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 02:26:55,985 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:55,985 INFO 3318922:repeater task beamdiagnostics_os150nb starting 41.0 seconds late 2024-11-22 02:26:57,708 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:26:57,708 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:57,709 INFO 3318922:repeater task beamdiagnostics_os150nb starting 42.7 seconds late 2024-11-22 02:26:59,445 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:26:59,445 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:26:59,446 INFO 3318922:repeater task lastensq_os38nb starting 38.4 seconds late 2024-11-22 02:27:02,082 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 02:27:02,082 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:26:16 2024-11-22 02:27:02,082 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:31:21 2024-11-22 02:27:02,083 INFO 3318922:repeater task quick_os38nb starting 41.1 seconds late 2024-11-22 02:27:05,970 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 02:27:05,971 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:26:16 2024-11-22 02:27:05,971 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:31:21 2024-11-22 02:27:05,971 INFO 3318922:repeater task lastfew_vec_os38nb starting 45.0 seconds late 2024-11-22 02:27:06,994 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:27:06,994 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:26:16 2024-11-22 02:27:06,994 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:31:21 2024-11-22 02:27:06,994 INFO 3318922:repeater task 3dayplots_os38nb starting 47.0 seconds late 2024-11-22 02:27:13,259 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 02:27:13,259 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:31:19 2024-11-22 02:27:13,260 INFO 3318922:repeater task plot_tsstats_os38nb starting 58.3 seconds late 2024-11-22 02:27:14,537 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:27:14,537 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 02:27:14,537 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:27:14,538 INFO 3318922:repeater task grab_lastpings_os38nb starting 59.6 seconds late 2024-11-22 02:27:14,699 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 02:27:14,699 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:27:14,700 INFO 3318922:repeater task beamdiagnostics_os38nb starting 59.7 seconds late 2024-11-22 02:27:16,459 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 02:27:16,459 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:27:16,459 INFO 3318922:repeater task beamdiagnostics_os38nb starting 61.5 seconds late 2024-11-22 02:27:18,190 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 02:27:18,190 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:41:14 2024-11-22 02:27:18,191 INFO 3318922:repeater task daily_netcdf starting 58.2 seconds late 2024-11-22 02:27:18,916 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 02:27:18,916 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:27:18,917 INFO 3318922:repeater task ggatime_diagnostics starting 63.9 seconds late 2024-11-22 02:27:26,611 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 02:27:26,611 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:27:26,611 INFO 3318922:repeater task ggatime_diagnostics starting 71.6 seconds late 2024-11-22 02:27:34,151 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 02:27:34,152 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:27:34,152 INFO 3318922:repeater task ggatime_diagnostics starting 79.2 seconds late 2024-11-22 02:27:41,689 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 02:27:41,689 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:27:41,690 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 86.6 seconds late 2024-11-22 02:27:43,513 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 02:27:43,514 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 02:27:43,514 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 02:41:15 2024-11-22 02:27:43,514 INFO 3318922:repeater task plot_hbins starting 88.4 seconds late 2024-11-22 02:27:47,627 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 02:27:47,627 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 02:27:47,627 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 02:41:15 2024-11-22 02:27:47,627 INFO 3318922:repeater task report_generator starting 92.5 seconds late 2024-11-22 02:27:52,171 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-22 02:27:52,171 INFO 3318922:repeater task report_generator ended after 4.5 seconds 2024-11-22 02:27:52,171 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/22 03:26:15 2024-11-22 02:27:57,174 INFO 3318922:repeater task noisy_rawtemps_os150 starting 97.2 seconds late 2024-11-22 02:28:00,315 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 02:28:00,315 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:28:00,316 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 100.3 seconds late 2024-11-22 02:28:03,545 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 02:28:03,545 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:28:03,545 INFO 3318922:repeater task noisy_rawtemps_os38 starting 103.6 seconds late 2024-11-22 02:28:07,092 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 02:28:07,092 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:28:22,129 INFO 3318922:repeater task lastensq_wh300 starting 3.1 seconds late 2024-11-22 02:28:25,371 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:28:25,372 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:28:14 2024-11-22 02:28:25,372 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:30:19 2024-11-22 02:28:25,372 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-22 02:28:31,020 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:28:31,020 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:28:14 2024-11-22 02:28:31,020 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:30:19 2024-11-22 02:28:31,020 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2024-11-22 02:28:32,054 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:28:32,054 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:28:14 2024-11-22 02:28:32,055 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:30:19 2024-11-22 02:30:22,379 INFO 3318922:repeater task lastensq_wh300 starting 3.4 seconds late 2024-11-22 02:30:25,762 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 02:30:25,762 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:30:14 2024-11-22 02:30:25,763 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:32:19 2024-11-22 02:30:25,763 INFO 3318922:repeater task quick_wh300 starting 6.8 seconds late 2024-11-22 02:30:31,137 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:30:31,138 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:30:14 2024-11-22 02:30:31,138 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:32:19 2024-11-22 02:30:31,138 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-22 02:30:32,192 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:30:32,192 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:30:14 2024-11-22 02:30:32,192 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:32:19 2024-11-22 02:31:22,342 INFO 3318922:repeater task 3dayplots_wh300 starting 2.4 seconds late 2024-11-22 02:31:28,836 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 02:31:28,836 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:31:28,837 INFO 3318922:repeater task lastensq_os150nb starting 9.8 seconds late 2024-11-22 02:31:31,545 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 02:31:31,545 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:31:14 2024-11-22 02:31:31,545 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:31:31,546 INFO 3318922:repeater task quick_os150nb starting 12.5 seconds late 2024-11-22 02:31:35,282 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 02:31:35,282 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:31:14 2024-11-22 02:31:35,282 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:31:35,283 INFO 3318922:repeater task lastfew_vec_os150nb starting 16.3 seconds late 2024-11-22 02:31:36,320 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:31:36,321 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:31:14 2024-11-22 02:31:36,321 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:31:36,321 INFO 3318922:repeater task 3dayplots_os150nb starting 16.3 seconds late 2024-11-22 02:31:42,704 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 02:31:42,704 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:31:42,705 INFO 3318922:repeater task lastensq_os38nb starting 21.7 seconds late 2024-11-22 02:31:45,433 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 02:31:45,433 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:31:16 2024-11-22 02:31:45,433 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:36:21 2024-11-22 02:31:45,434 INFO 3318922:repeater task quick_os38nb starting 24.4 seconds late 2024-11-22 02:31:49,379 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 02:31:49,380 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:31:16 2024-11-22 02:31:49,380 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:36:21 2024-11-22 02:31:49,380 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.4 seconds late 2024-11-22 02:31:50,398 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:31:50,398 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:31:16 2024-11-22 02:31:50,398 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:36:21 2024-11-22 02:31:50,399 INFO 3318922:repeater task 3dayplots_os38nb starting 30.4 seconds late 2024-11-22 02:31:56,685 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 02:31:56,685 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:32:21,748 INFO 3318922:repeater task lastensq_wh300 starting 2.7 seconds late 2024-11-22 02:32:24,952 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:32:24,952 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:32:14 2024-11-22 02:32:24,952 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:34:19 2024-11-22 02:32:24,953 INFO 3318922:repeater task quick_wh300 starting 6.0 seconds late 2024-11-22 02:32:30,321 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:32:30,321 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:32:14 2024-11-22 02:32:30,321 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:34:19 2024-11-22 02:32:30,322 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2024-11-22 02:32:31,387 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:32:31,387 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:32:14 2024-11-22 02:32:31,388 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:34:19 2024-11-22 02:34:21,720 INFO 3318922:repeater task lastensq_wh300 starting 2.7 seconds late 2024-11-22 02:34:25,011 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:34:25,012 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:34:14 2024-11-22 02:34:25,012 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:34:25,012 INFO 3318922:repeater task quick_wh300 starting 6.0 seconds late 2024-11-22 02:34:30,620 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:34:30,620 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:34:14 2024-11-22 02:34:30,621 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:34:30,621 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2024-11-22 02:34:31,671 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:34:31,671 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:34:14 2024-11-22 02:34:31,671 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:36:19 2024-11-22 02:36:21,926 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.0 seconds late 2024-11-22 02:36:25,029 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 02:36:25,029 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:36:25,030 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.1 seconds late 2024-11-22 02:36:28,190 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 02:36:28,190 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:36:28,190 INFO 3318922:repeater task noisy_rawtemps_os38 starting 8.2 seconds late 2024-11-22 02:36:31,730 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 02:36:31,730 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:36:31,731 INFO 3318922:repeater task lastensq_wh300 starting 12.7 seconds late 2024-11-22 02:36:34,933 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:36:34,933 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:36:14 2024-11-22 02:36:34,933 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:38:19 2024-11-22 02:36:34,933 INFO 3318922:repeater task quick_wh300 starting 15.9 seconds late 2024-11-22 02:36:40,563 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:36:40,563 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:36:14 2024-11-22 02:36:40,563 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:38:19 2024-11-22 02:36:40,564 INFO 3318922:repeater task lastfew_vec_wh300 starting 21.6 seconds late 2024-11-22 02:36:41,607 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:36:41,607 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:36:14 2024-11-22 02:36:41,607 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:38:19 2024-11-22 02:36:41,608 INFO 3318922:repeater task 3dayplots_wh300 starting 21.6 seconds late 2024-11-22 02:36:48,253 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 02:36:48,254 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:41:19 2024-11-22 02:36:48,255 INFO 3318922:repeater task lastensq_os150nb starting 29.3 seconds late 2024-11-22 02:36:50,941 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 02:36:50,941 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:36:14 2024-11-22 02:36:50,941 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:41:19 2024-11-22 02:36:50,942 INFO 3318922:repeater task quick_os150nb starting 31.9 seconds late 2024-11-22 02:36:54,745 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 02:36:54,745 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:36:14 2024-11-22 02:36:54,745 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:41:19 2024-11-22 02:36:54,745 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.7 seconds late 2024-11-22 02:36:55,764 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:36:55,764 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:36:14 2024-11-22 02:36:55,764 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:41:19 2024-11-22 02:36:55,764 INFO 3318922:repeater task 3dayplots_os150nb starting 35.8 seconds late 2024-11-22 02:37:01,976 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 02:37:01,976 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:41:19 2024-11-22 02:37:01,977 INFO 3318922:repeater task lastensq_os38nb starting 41.0 seconds late 2024-11-22 02:37:04,599 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 02:37:04,599 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:36:16 2024-11-22 02:37:04,599 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:41:21 2024-11-22 02:37:04,599 INFO 3318922:repeater task quick_os38nb starting 43.6 seconds late 2024-11-22 02:37:08,500 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 02:37:08,500 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:36:16 2024-11-22 02:37:08,500 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:41:21 2024-11-22 02:37:08,501 INFO 3318922:repeater task lastfew_vec_os38nb starting 47.5 seconds late 2024-11-22 02:37:09,540 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:37:09,541 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:36:16 2024-11-22 02:37:09,541 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:41:21 2024-11-22 02:37:09,541 INFO 3318922:repeater task 3dayplots_os38nb starting 49.6 seconds late 2024-11-22 02:37:15,670 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 02:37:15,671 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:41:19 2024-11-22 02:37:15,672 INFO 3318922:repeater task daily_netcdf starting 55.7 seconds late 2024-11-22 02:37:16,448 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 02:37:16,448 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:38:21,716 INFO 3318922:repeater task lastensq_wh300 starting 2.7 seconds late 2024-11-22 02:38:25,000 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:38:25,000 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:38:14 2024-11-22 02:38:25,000 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:40:19 2024-11-22 02:38:25,001 INFO 3318922:repeater task quick_wh300 starting 6.0 seconds late 2024-11-22 02:38:30,655 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 02:38:30,656 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:38:14 2024-11-22 02:38:30,656 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:40:19 2024-11-22 02:38:30,656 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2024-11-22 02:38:31,710 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:38:31,711 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:38:14 2024-11-22 02:38:31,711 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:40:19 2024-11-22 02:40:22,033 INFO 3318922:repeater task lastensq_wh300 starting 3.0 seconds late 2024-11-22 02:40:25,375 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:40:25,375 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:40:14 2024-11-22 02:40:25,375 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:42:19 2024-11-22 02:40:25,375 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-22 02:40:30,970 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:40:30,970 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:40:14 2024-11-22 02:40:30,970 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:42:19 2024-11-22 02:40:30,970 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2024-11-22 02:40:32,025 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:40:32,025 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:40:14 2024-11-22 02:40:32,025 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:42:19 2024-11-22 02:41:17,142 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.2 seconds late 2024-11-22 02:41:18,587 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:41:18,587 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 02:41:18,587 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:18,587 INFO 3318922:repeater task grab_lastpings_wh300 starting 3.6 seconds late 2024-11-22 02:41:18,746 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 02:41:18,746 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:18,747 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.8 seconds late 2024-11-22 02:41:20,540 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-22 02:41:20,540 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:20,541 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.6 seconds late 2024-11-22 02:41:22,308 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-22 02:41:22,309 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:22,309 INFO 3318922:repeater task lastensq_os150nb starting 3.3 seconds late 2024-11-22 02:41:25,139 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 02:41:25,140 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:41:14 2024-11-22 02:41:25,140 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:41:25,140 INFO 3318922:repeater task quick_os150nb starting 6.1 seconds late 2024-11-22 02:41:28,922 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 02:41:28,922 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:41:14 2024-11-22 02:41:28,922 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:41:28,922 INFO 3318922:repeater task lastfew_vec_os150nb starting 9.9 seconds late 2024-11-22 02:41:29,941 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:41:29,941 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:41:14 2024-11-22 02:41:29,941 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:41:29,941 INFO 3318922:repeater task 3dayplots_os150nb starting 10.0 seconds late 2024-11-22 02:41:36,233 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 02:41:36,233 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:41:36,233 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.3 seconds late 2024-11-22 02:41:37,479 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:41:37,480 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.2 seconds 2024-11-22 02:41:37,480 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:37,481 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.5 seconds late 2024-11-22 02:41:37,640 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 02:41:37,640 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:37,641 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.7 seconds late 2024-11-22 02:41:39,363 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:41:39,363 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:39,363 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.4 seconds late 2024-11-22 02:41:41,084 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:41:41,084 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:41,084 INFO 3318922:repeater task lastensq_os38nb starting 20.1 seconds late 2024-11-22 02:41:43,739 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 02:41:43,739 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:41:16 2024-11-22 02:41:43,739 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:46:21 2024-11-22 02:41:43,739 INFO 3318922:repeater task quick_os38nb starting 22.7 seconds late 2024-11-22 02:41:47,917 INFO 3318922:repeater task quick_os38nb ended after 4.2 seconds 2024-11-22 02:41:47,917 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:41:16 2024-11-22 02:41:47,917 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:46:21 2024-11-22 02:41:47,918 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.9 seconds late 2024-11-22 02:41:48,966 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:41:48,966 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:41:16 2024-11-22 02:41:48,966 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:46:21 2024-11-22 02:41:48,966 INFO 3318922:repeater task 3dayplots_os38nb starting 29.0 seconds late 2024-11-22 02:41:55,231 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 02:41:55,231 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:41:55,232 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.3 seconds late 2024-11-22 02:41:56,586 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:41:56,587 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.4 seconds 2024-11-22 02:41:56,587 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:56,587 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.6 seconds late 2024-11-22 02:41:56,748 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 02:41:56,748 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:56,748 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.8 seconds late 2024-11-22 02:41:58,468 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 02:41:58,468 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:41:58,468 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.5 seconds late 2024-11-22 02:42:00,204 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 02:42:00,204 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 02:56:14 2024-11-22 02:42:00,206 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 45.1 seconds late 2024-11-22 02:42:02,040 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 02:42:02,041 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 02:42:02,041 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 02:56:15 2024-11-22 02:42:02,041 INFO 3318922:repeater task plot_hbins starting 46.9 seconds late 2024-11-22 02:42:06,148 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 02:42:06,148 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 02:42:06,148 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 02:56:15 2024-11-22 02:42:11,157 INFO 3318922:repeater task 3dayplots_wh300 starting 51.2 seconds late 2024-11-22 02:42:17,754 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 02:42:17,754 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:42:22,765 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 02:42:26,018 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:42:26,018 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:42:14 2024-11-22 02:42:26,018 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:44:19 2024-11-22 02:42:26,019 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-22 02:42:31,416 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:42:31,416 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:42:14 2024-11-22 02:42:31,416 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:44:19 2024-11-22 02:42:31,416 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2024-11-22 02:42:32,468 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:42:32,468 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:42:14 2024-11-22 02:42:32,468 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:44:19 2024-11-22 02:44:22,792 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 02:44:25,993 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:44:25,993 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:44:14 2024-11-22 02:44:25,993 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:44:25,994 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-22 02:44:31,339 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 02:44:31,339 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:44:14 2024-11-22 02:44:31,339 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:44:31,339 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-22 02:44:32,387 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:44:32,387 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:44:14 2024-11-22 02:44:32,387 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:46:19 2024-11-22 02:46:22,694 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.7 seconds late 2024-11-22 02:46:25,926 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 02:46:25,926 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:46:25,927 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 6.0 seconds late 2024-11-22 02:46:29,033 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-22 02:46:29,033 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:46:29,034 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.1 seconds late 2024-11-22 02:46:32,576 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 02:46:32,576 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:46:32,576 INFO 3318922:repeater task lastensq_wh300 starting 13.6 seconds late 2024-11-22 02:46:35,833 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:46:35,833 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:46:14 2024-11-22 02:46:35,833 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:48:19 2024-11-22 02:46:35,833 INFO 3318922:repeater task quick_wh300 starting 16.8 seconds late 2024-11-22 02:46:41,405 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:46:41,405 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:46:14 2024-11-22 02:46:41,405 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:48:19 2024-11-22 02:46:41,406 INFO 3318922:repeater task lastfew_vec_wh300 starting 22.4 seconds late 2024-11-22 02:46:42,454 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:46:42,454 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:46:14 2024-11-22 02:46:42,454 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:48:19 2024-11-22 02:46:42,455 INFO 3318922:repeater task 3dayplots_wh300 starting 22.5 seconds late 2024-11-22 02:46:49,095 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 02:46:49,095 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:51:19 2024-11-22 02:46:49,096 INFO 3318922:repeater task lastensq_os150nb starting 30.1 seconds late 2024-11-22 02:46:51,831 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 02:46:51,831 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:46:14 2024-11-22 02:46:51,831 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:51:19 2024-11-22 02:46:51,832 INFO 3318922:repeater task quick_os150nb starting 32.8 seconds late 2024-11-22 02:46:55,515 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 02:46:55,515 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:46:14 2024-11-22 02:46:55,515 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:51:19 2024-11-22 02:46:55,515 INFO 3318922:repeater task lastfew_vec_os150nb starting 36.5 seconds late 2024-11-22 02:46:56,553 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:46:56,553 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:46:14 2024-11-22 02:46:56,553 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:51:19 2024-11-22 02:46:56,553 INFO 3318922:repeater task 3dayplots_os150nb starting 36.6 seconds late 2024-11-22 02:47:02,749 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 02:47:02,749 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:51:19 2024-11-22 02:47:02,750 INFO 3318922:repeater task lastensq_os38nb starting 41.8 seconds late 2024-11-22 02:47:05,423 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 02:47:05,424 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:46:16 2024-11-22 02:47:05,424 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:51:21 2024-11-22 02:47:05,424 INFO 3318922:repeater task quick_os38nb starting 44.4 seconds late 2024-11-22 02:47:09,409 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 02:47:09,409 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:46:16 2024-11-22 02:47:09,409 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:51:21 2024-11-22 02:47:09,410 INFO 3318922:repeater task lastfew_vec_os38nb starting 48.4 seconds late 2024-11-22 02:47:10,438 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:47:10,438 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:46:16 2024-11-22 02:47:10,438 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:51:21 2024-11-22 02:47:10,438 INFO 3318922:repeater task 3dayplots_os38nb starting 50.5 seconds late 2024-11-22 02:47:16,576 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 02:47:16,576 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:51:19 2024-11-22 02:47:16,577 INFO 3318922:repeater task daily_netcdf starting 56.6 seconds late 2024-11-22 02:47:17,364 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 02:47:17,365 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:48:22,560 INFO 3318922:repeater task lastensq_wh300 starting 3.6 seconds late 2024-11-22 02:48:25,962 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 02:48:25,963 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:48:14 2024-11-22 02:48:25,963 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:50:19 2024-11-22 02:48:25,963 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-22 02:48:31,475 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 02:48:31,475 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:48:14 2024-11-22 02:48:31,475 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:50:19 2024-11-22 02:48:31,475 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2024-11-22 02:48:32,517 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:48:32,518 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:48:14 2024-11-22 02:48:32,518 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:50:19 2024-11-22 02:50:22,865 INFO 3318922:repeater task lastensq_wh300 starting 3.9 seconds late 2024-11-22 02:50:26,165 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:50:26,165 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:50:14 2024-11-22 02:50:26,165 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:52:19 2024-11-22 02:50:26,166 INFO 3318922:repeater task quick_wh300 starting 7.2 seconds late 2024-11-22 02:50:31,675 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 02:50:31,676 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:50:14 2024-11-22 02:50:31,676 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:52:19 2024-11-22 02:50:31,676 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2024-11-22 02:50:32,708 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:50:32,709 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:50:14 2024-11-22 02:50:32,709 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:52:19 2024-11-22 02:51:22,857 INFO 3318922:repeater task 3dayplots_wh300 starting 2.9 seconds late 2024-11-22 02:51:29,693 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.8 seconds 2024-11-22 02:51:29,694 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:51:29,695 INFO 3318922:repeater task lastensq_os150nb starting 10.7 seconds late 2024-11-22 02:51:32,352 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 02:51:32,353 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:51:14 2024-11-22 02:51:32,353 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:51:32,353 INFO 3318922:repeater task quick_os150nb starting 13.4 seconds late 2024-11-22 02:51:35,875 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 02:51:35,875 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:51:14 2024-11-22 02:51:35,875 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:51:35,875 INFO 3318922:repeater task lastfew_vec_os150nb starting 16.9 seconds late 2024-11-22 02:51:36,882 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:51:36,883 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:51:14 2024-11-22 02:51:36,883 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:51:36,883 INFO 3318922:repeater task 3dayplots_os150nb starting 16.9 seconds late 2024-11-22 02:51:43,181 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 02:51:43,181 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:51:43,183 INFO 3318922:repeater task lastensq_os38nb starting 22.2 seconds late 2024-11-22 02:51:45,840 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 02:51:45,840 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:51:16 2024-11-22 02:51:45,840 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 02:56:21 2024-11-22 02:51:45,841 INFO 3318922:repeater task quick_os38nb starting 24.8 seconds late 2024-11-22 02:51:49,772 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 02:51:49,773 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:51:16 2024-11-22 02:51:49,773 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 02:56:21 2024-11-22 02:51:49,773 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.8 seconds late 2024-11-22 02:51:50,805 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:51:50,805 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:51:16 2024-11-22 02:51:50,805 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 02:56:21 2024-11-22 02:51:50,806 INFO 3318922:repeater task 3dayplots_os38nb starting 30.8 seconds late 2024-11-22 02:51:56,989 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 02:51:56,989 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:52:22,060 INFO 3318922:repeater task lastensq_wh300 starting 3.1 seconds late 2024-11-22 02:52:25,368 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:52:25,368 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:52:14 2024-11-22 02:52:25,368 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:54:19 2024-11-22 02:52:25,368 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-22 02:52:30,820 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 02:52:30,821 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:52:14 2024-11-22 02:52:30,821 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:54:19 2024-11-22 02:52:30,821 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2024-11-22 02:52:31,849 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:52:31,850 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:52:14 2024-11-22 02:52:31,850 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:54:19 2024-11-22 02:54:22,164 INFO 3318922:repeater task lastensq_wh300 starting 3.2 seconds late 2024-11-22 02:54:25,507 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:54:25,507 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:54:14 2024-11-22 02:54:25,507 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:54:25,508 INFO 3318922:repeater task quick_wh300 starting 6.5 seconds late 2024-11-22 02:54:31,137 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:54:31,138 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:54:14 2024-11-22 02:54:31,138 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:54:31,138 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-22 02:54:32,191 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:54:32,191 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:54:14 2024-11-22 02:54:32,191 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:56:19 2024-11-22 02:56:17,484 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.5 seconds late 2024-11-22 02:56:18,924 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:56:18,925 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 02:56:18,925 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:18,925 INFO 3318922:repeater task grab_lastpings_wh300 starting 3.9 seconds late 2024-11-22 02:56:19,087 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 02:56:19,087 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:19,088 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.1 seconds late 2024-11-22 02:56:20,790 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 02:56:20,790 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:20,790 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.8 seconds late 2024-11-22 02:56:22,558 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-22 02:56:22,560 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:22,560 INFO 3318922:repeater task lastensq_os150nb starting 3.6 seconds late 2024-11-22 02:56:25,320 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 02:56:25,320 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:56:14 2024-11-22 02:56:25,320 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:01:19 2024-11-22 02:56:25,321 INFO 3318922:repeater task quick_os150nb starting 6.3 seconds late 2024-11-22 02:56:28,985 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 02:56:28,985 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:56:14 2024-11-22 02:56:28,985 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:01:19 2024-11-22 02:56:28,986 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.0 seconds late 2024-11-22 02:56:30,000 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 02:56:30,001 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 02:56:14 2024-11-22 02:56:30,001 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:01:19 2024-11-22 02:56:30,001 INFO 3318922:repeater task 3dayplots_os150nb starting 10.0 seconds late 2024-11-22 02:56:36,227 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 02:56:36,227 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:01:19 2024-11-22 02:56:36,227 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.2 seconds late 2024-11-22 02:56:37,555 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:56:37,555 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 02:56:37,555 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:37,556 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.6 seconds late 2024-11-22 02:56:37,731 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 02:56:37,731 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:37,732 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.8 seconds late 2024-11-22 02:56:39,433 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:56:39,434 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:39,434 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.5 seconds late 2024-11-22 02:56:41,147 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 02:56:41,147 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:41,147 INFO 3318922:repeater task lastensq_os38nb starting 20.1 seconds late 2024-11-22 02:56:43,811 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 02:56:43,811 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:56:16 2024-11-22 02:56:43,811 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:01:21 2024-11-22 02:56:43,811 INFO 3318922:repeater task quick_os38nb starting 22.8 seconds late 2024-11-22 02:56:47,753 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 02:56:47,754 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:56:16 2024-11-22 02:56:47,754 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:01:21 2024-11-22 02:56:47,754 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.8 seconds late 2024-11-22 02:56:48,779 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 02:56:48,780 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 02:56:16 2024-11-22 02:56:48,780 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:01:21 2024-11-22 02:56:48,780 INFO 3318922:repeater task 3dayplots_os38nb starting 28.8 seconds late 2024-11-22 02:56:55,063 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 02:56:55,063 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:01:19 2024-11-22 02:56:55,064 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.1 seconds late 2024-11-22 02:56:56,354 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 02:56:56,354 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 02:56:56,354 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:56,354 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.4 seconds late 2024-11-22 02:56:56,514 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 02:56:56,514 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:56,514 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.5 seconds late 2024-11-22 02:56:58,214 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 02:56:58,214 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:58,215 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.2 seconds late 2024-11-22 02:56:59,972 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 02:56:59,973 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:11:14 2024-11-22 02:56:59,973 INFO 3318922:repeater task daily_netcdf starting 40.0 seconds late 2024-11-22 02:57:00,748 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 02:57:00,748 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 03:06:19 2024-11-22 02:57:00,749 INFO 3318922:repeater task ggatime_diagnostics starting 45.8 seconds late 2024-11-22 02:57:08,409 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 02:57:08,409 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 03:26:14 2024-11-22 02:57:08,410 INFO 3318922:repeater task ggatime_diagnostics starting 53.4 seconds late 2024-11-22 02:57:16,019 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 02:57:16,019 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 03:26:14 2024-11-22 02:57:16,021 INFO 3318922:repeater task ggatime_diagnostics starting 61.0 seconds late 2024-11-22 02:57:23,696 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 02:57:23,696 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 03:26:14 2024-11-22 02:57:23,697 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 68.6 seconds late 2024-11-22 02:57:25,536 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 02:57:25,536 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 02:57:25,537 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 03:11:15 2024-11-22 02:57:25,537 INFO 3318922:repeater task plot_hbins starting 70.4 seconds late 2024-11-22 02:57:29,681 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 02:57:29,682 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 02:57:29,682 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 03:11:15 2024-11-22 02:57:34,688 INFO 3318922:repeater task noisy_rawtemps_os150 starting 74.7 seconds late 2024-11-22 02:57:37,822 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 02:57:37,822 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 03:06:19 2024-11-22 02:57:37,822 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 77.8 seconds late 2024-11-22 02:57:40,991 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 02:57:40,992 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 03:06:19 2024-11-22 02:57:40,992 INFO 3318922:repeater task noisy_rawtemps_os38 starting 81.0 seconds late 2024-11-22 02:57:44,489 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-22 02:57:44,489 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 03:06:19 2024-11-22 02:57:44,490 INFO 3318922:repeater task lastensq_wh300 starting 85.5 seconds late 2024-11-22 02:57:47,759 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 02:57:47,760 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:56:14 2024-11-22 02:57:47,760 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 02:58:19 2024-11-22 02:57:47,760 INFO 3318922:repeater task quick_wh300 starting 88.8 seconds late 2024-11-22 02:57:53,185 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 02:57:53,185 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:56:14 2024-11-22 02:57:53,185 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 02:58:19 2024-11-22 02:57:53,186 INFO 3318922:repeater task lastfew_vec_wh300 starting 94.2 seconds late 2024-11-22 02:57:54,220 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 02:57:54,220 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:56:14 2024-11-22 02:57:54,221 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 02:58:19 2024-11-22 02:57:54,221 INFO 3318922:repeater task 3dayplots_wh300 starting 94.2 seconds late 2024-11-22 02:58:00,830 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 02:58:00,830 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:01:19 2024-11-22 02:58:20,880 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 02:58:24,124 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 02:58:24,124 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:58:14 2024-11-22 02:58:24,124 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:00:19 2024-11-22 02:58:24,125 INFO 3318922:repeater task quick_wh300 starting 5.1 seconds late 2024-11-22 02:58:29,746 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 02:58:29,747 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:58:14 2024-11-22 02:58:29,747 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:00:19 2024-11-22 02:58:29,747 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2024-11-22 02:58:30,797 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 02:58:30,797 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 02:58:14 2024-11-22 02:58:30,797 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:00:19 2024-11-22 03:00:21,106 INFO 3318922:repeater task lastensq_wh300 starting 2.1 seconds late 2024-11-22 03:00:24,399 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:00:24,399 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:00:14 2024-11-22 03:00:24,399 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:02:19 2024-11-22 03:00:24,399 INFO 3318922:repeater task quick_wh300 starting 5.4 seconds late 2024-11-22 03:00:29,780 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 03:00:29,780 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:00:14 2024-11-22 03:00:29,780 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:02:19 2024-11-22 03:00:29,781 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-22 03:00:30,823 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:00:30,823 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:00:14 2024-11-22 03:00:30,823 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:02:19 2024-11-22 03:01:20,944 INFO 3318922:repeater task 3dayplots_wh300 starting 1.0 seconds late 2024-11-22 03:01:27,829 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.9 seconds 2024-11-22 03:01:27,829 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:01:27,830 INFO 3318922:repeater task lastensq_os150nb starting 8.8 seconds late 2024-11-22 03:01:30,563 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:01:30,563 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:01:14 2024-11-22 03:01:30,563 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:01:30,563 INFO 3318922:repeater task quick_os150nb starting 11.6 seconds late 2024-11-22 03:01:34,370 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 03:01:34,370 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:01:14 2024-11-22 03:01:34,370 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:01:34,371 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.4 seconds late 2024-11-22 03:01:35,394 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:01:35,394 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:01:14 2024-11-22 03:01:35,395 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:01:35,395 INFO 3318922:repeater task 3dayplots_os150nb starting 15.4 seconds late 2024-11-22 03:01:41,592 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 03:01:41,592 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:01:41,593 INFO 3318922:repeater task lastensq_os38nb starting 20.6 seconds late 2024-11-22 03:01:44,344 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-22 03:01:44,345 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:01:16 2024-11-22 03:01:44,345 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:06:21 2024-11-22 03:01:44,345 INFO 3318922:repeater task quick_os38nb starting 23.3 seconds late 2024-11-22 03:01:48,075 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-22 03:01:48,075 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:01:16 2024-11-22 03:01:48,075 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:06:21 2024-11-22 03:01:48,075 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.1 seconds late 2024-11-22 03:01:49,093 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:01:49,094 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:01:16 2024-11-22 03:01:49,094 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:06:21 2024-11-22 03:01:49,094 INFO 3318922:repeater task 3dayplots_os38nb starting 29.1 seconds late 2024-11-22 03:01:55,380 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 03:01:55,381 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:02:20,447 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 03:02:23,594 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-22 03:02:23,594 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:02:14 2024-11-22 03:02:23,594 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:04:19 2024-11-22 03:02:23,595 INFO 3318922:repeater task quick_wh300 starting 4.6 seconds late 2024-11-22 03:02:29,072 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 03:02:29,073 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:02:14 2024-11-22 03:02:29,073 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:04:19 2024-11-22 03:02:29,073 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2024-11-22 03:02:30,104 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:02:30,104 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:02:14 2024-11-22 03:02:30,104 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:04:19 2024-11-22 03:04:20,424 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 03:04:23,727 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:04:23,728 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:04:14 2024-11-22 03:04:23,728 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:04:23,728 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 03:04:29,193 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 03:04:29,193 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:04:14 2024-11-22 03:04:29,193 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:04:29,194 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2024-11-22 03:04:30,224 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:04:30,225 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:04:14 2024-11-22 03:04:30,225 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:06:19 2024-11-22 03:06:20,538 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.6 seconds late 2024-11-22 03:06:23,668 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 03:06:23,668 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:06:23,669 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.7 seconds late 2024-11-22 03:06:26,885 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 03:06:26,885 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:06:26,885 INFO 3318922:repeater task noisy_rawtemps_os38 starting 6.9 seconds late 2024-11-22 03:06:30,481 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 03:06:30,481 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:06:30,482 INFO 3318922:repeater task lastensq_wh300 starting 11.5 seconds late 2024-11-22 03:06:33,725 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:06:33,726 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:06:14 2024-11-22 03:06:33,726 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:08:19 2024-11-22 03:06:33,726 INFO 3318922:repeater task quick_wh300 starting 14.7 seconds late 2024-11-22 03:06:39,360 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:06:39,360 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:06:14 2024-11-22 03:06:39,360 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:08:19 2024-11-22 03:06:39,360 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.4 seconds late 2024-11-22 03:06:40,409 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:06:40,410 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:06:14 2024-11-22 03:06:40,410 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:08:19 2024-11-22 03:06:40,410 INFO 3318922:repeater task 3dayplots_wh300 starting 20.4 seconds late 2024-11-22 03:06:46,927 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 03:06:46,927 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:11:19 2024-11-22 03:06:46,928 INFO 3318922:repeater task lastensq_os150nb starting 27.9 seconds late 2024-11-22 03:06:49,658 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:06:49,659 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:06:14 2024-11-22 03:06:49,659 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:11:19 2024-11-22 03:06:49,659 INFO 3318922:repeater task quick_os150nb starting 30.7 seconds late 2024-11-22 03:06:53,233 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 03:06:53,233 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:06:14 2024-11-22 03:06:53,233 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:11:19 2024-11-22 03:06:53,234 INFO 3318922:repeater task lastfew_vec_os150nb starting 34.2 seconds late 2024-11-22 03:06:54,248 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:06:54,248 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:06:14 2024-11-22 03:06:54,248 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:11:19 2024-11-22 03:06:54,248 INFO 3318922:repeater task 3dayplots_os150nb starting 34.3 seconds late 2024-11-22 03:07:00,556 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 03:07:00,556 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:11:19 2024-11-22 03:07:00,557 INFO 3318922:repeater task lastensq_os38nb starting 39.6 seconds late 2024-11-22 03:07:03,349 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-22 03:07:03,349 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:06:16 2024-11-22 03:07:03,349 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:11:21 2024-11-22 03:07:03,350 INFO 3318922:repeater task quick_os38nb starting 42.4 seconds late 2024-11-22 03:07:07,682 INFO 3318922:repeater task quick_os38nb ended after 4.3 seconds 2024-11-22 03:07:07,682 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:06:16 2024-11-22 03:07:07,682 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:11:21 2024-11-22 03:07:07,682 INFO 3318922:repeater task lastfew_vec_os38nb starting 46.7 seconds late 2024-11-22 03:07:08,732 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:07:08,733 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:06:16 2024-11-22 03:07:08,733 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:11:21 2024-11-22 03:07:08,733 INFO 3318922:repeater task 3dayplots_os38nb starting 48.8 seconds late 2024-11-22 03:07:15,007 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 03:07:15,007 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:11:19 2024-11-22 03:07:15,009 INFO 3318922:repeater task daily_netcdf starting 55.0 seconds late 2024-11-22 03:07:15,751 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 03:07:15,752 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:08:20,929 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 03:08:24,181 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:08:24,181 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:08:14 2024-11-22 03:08:24,181 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:10:19 2024-11-22 03:08:24,181 INFO 3318922:repeater task quick_wh300 starting 5.2 seconds late 2024-11-22 03:08:29,545 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 03:08:29,546 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:08:14 2024-11-22 03:08:29,546 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:10:19 2024-11-22 03:08:29,546 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-22 03:08:30,605 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 03:08:30,606 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:08:14 2024-11-22 03:08:30,606 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:10:19 2024-11-22 03:10:20,912 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 03:10:24,272 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 03:10:24,272 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:10:14 2024-11-22 03:10:24,273 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:12:19 2024-11-22 03:10:24,273 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-22 03:10:29,854 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:10:29,854 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:10:14 2024-11-22 03:10:29,854 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:12:19 2024-11-22 03:10:29,855 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2024-11-22 03:10:30,905 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 03:10:30,905 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:10:14 2024-11-22 03:10:30,906 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:12:19 2024-11-22 03:11:16,030 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.1 seconds late 2024-11-22 03:11:17,467 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:11:17,467 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 03:11:17,467 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:17,467 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.5 seconds late 2024-11-22 03:11:17,631 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 03:11:17,631 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:17,631 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.7 seconds late 2024-11-22 03:11:19,353 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:11:19,353 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:19,353 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.4 seconds late 2024-11-22 03:11:21,083 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:11:21,085 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:21,086 INFO 3318922:repeater task lastensq_os150nb starting 2.1 seconds late 2024-11-22 03:11:23,731 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 03:11:23,731 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:11:14 2024-11-22 03:11:23,732 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:11:23,732 INFO 3318922:repeater task quick_os150nb starting 4.7 seconds late 2024-11-22 03:11:27,517 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 03:11:27,518 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:11:14 2024-11-22 03:11:27,518 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:11:27,518 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.5 seconds late 2024-11-22 03:11:28,538 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:11:28,539 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:11:14 2024-11-22 03:11:28,539 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:11:28,539 INFO 3318922:repeater task 3dayplots_os150nb starting 8.6 seconds late 2024-11-22 03:11:34,693 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 03:11:34,693 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:11:34,694 INFO 3318922:repeater task plot_tsstats_os150nb starting 19.7 seconds late 2024-11-22 03:11:35,961 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:11:35,961 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 03:11:35,961 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:35,962 INFO 3318922:repeater task grab_lastpings_os150nb starting 21.0 seconds late 2024-11-22 03:11:36,122 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 03:11:36,122 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:36,123 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.1 seconds late 2024-11-22 03:11:37,806 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 03:11:37,806 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:37,807 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.8 seconds late 2024-11-22 03:11:39,518 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 03:11:39,518 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:39,518 INFO 3318922:repeater task lastensq_os38nb starting 18.5 seconds late 2024-11-22 03:11:42,119 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 03:11:42,120 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:11:16 2024-11-22 03:11:42,120 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:16:21 2024-11-22 03:11:42,120 INFO 3318922:repeater task quick_os38nb starting 21.1 seconds late 2024-11-22 03:11:46,175 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 03:11:46,176 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:11:16 2024-11-22 03:11:46,176 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:16:21 2024-11-22 03:11:46,176 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.2 seconds late 2024-11-22 03:11:47,188 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:11:47,188 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:11:16 2024-11-22 03:11:47,188 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:16:21 2024-11-22 03:11:47,188 INFO 3318922:repeater task 3dayplots_os38nb starting 27.2 seconds late 2024-11-22 03:11:53,423 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 03:11:53,423 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:11:53,424 INFO 3318922:repeater task plot_tsstats_os38nb starting 38.4 seconds late 2024-11-22 03:11:54,706 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:11:54,706 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 03:11:54,706 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:54,706 INFO 3318922:repeater task grab_lastpings_os38nb starting 39.7 seconds late 2024-11-22 03:11:54,866 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 03:11:54,866 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:54,867 INFO 3318922:repeater task beamdiagnostics_os38nb starting 39.9 seconds late 2024-11-22 03:11:56,588 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 03:11:56,588 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:56,588 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.6 seconds late 2024-11-22 03:11:58,352 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 03:11:58,353 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:26:14 2024-11-22 03:11:58,354 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 43.2 seconds late 2024-11-22 03:12:00,226 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 03:12:00,226 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 03:12:00,226 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 03:26:15 2024-11-22 03:12:00,227 INFO 3318922:repeater task plot_hbins starting 45.1 seconds late 2024-11-22 03:12:04,479 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 03:12:04,479 INFO 3318922:repeater task plot_hbins ended after 4.3 seconds 2024-11-22 03:12:04,479 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 03:26:15 2024-11-22 03:12:09,486 INFO 3318922:repeater task 3dayplots_wh300 starting 49.5 seconds late 2024-11-22 03:12:16,288 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.8 seconds 2024-11-22 03:12:16,288 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:12:21,298 INFO 3318922:repeater task lastensq_wh300 starting 2.3 seconds late 2024-11-22 03:12:24,648 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:12:24,648 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:12:14 2024-11-22 03:12:24,649 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:14:19 2024-11-22 03:12:24,649 INFO 3318922:repeater task quick_wh300 starting 5.6 seconds late 2024-11-22 03:12:30,340 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 03:12:30,341 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:12:14 2024-11-22 03:12:30,341 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:14:19 2024-11-22 03:12:30,341 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2024-11-22 03:12:31,384 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:12:31,385 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:12:14 2024-11-22 03:12:31,385 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:14:19 2024-11-22 03:14:21,792 INFO 3318922:repeater task lastensq_wh300 starting 2.8 seconds late 2024-11-22 03:14:25,124 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:14:25,124 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:14:14 2024-11-22 03:14:25,124 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:14:25,124 INFO 3318922:repeater task quick_wh300 starting 6.1 seconds late 2024-11-22 03:14:30,761 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:14:30,761 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:14:14 2024-11-22 03:14:30,761 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:14:30,762 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2024-11-22 03:14:31,813 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 03:14:31,813 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:14:14 2024-11-22 03:14:31,813 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:16:19 2024-11-22 03:16:22,134 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.2 seconds late 2024-11-22 03:16:25,266 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 03:16:25,266 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:16:25,266 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.3 seconds late 2024-11-22 03:16:28,447 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 03:16:28,447 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:16:28,447 INFO 3318922:repeater task noisy_rawtemps_os38 starting 8.5 seconds late 2024-11-22 03:16:32,012 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 03:16:32,012 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:16:32,012 INFO 3318922:repeater task lastensq_wh300 starting 13.0 seconds late 2024-11-22 03:16:35,373 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 03:16:35,373 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:16:14 2024-11-22 03:16:35,373 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:18:19 2024-11-22 03:16:35,374 INFO 3318922:repeater task quick_wh300 starting 16.4 seconds late 2024-11-22 03:16:40,946 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:16:40,946 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:16:14 2024-11-22 03:16:40,946 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:18:19 2024-11-22 03:16:40,946 INFO 3318922:repeater task lastfew_vec_wh300 starting 21.9 seconds late 2024-11-22 03:16:41,990 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:16:41,990 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:16:14 2024-11-22 03:16:41,990 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:18:19 2024-11-22 03:16:41,991 INFO 3318922:repeater task 3dayplots_wh300 starting 22.0 seconds late 2024-11-22 03:16:48,663 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 03:16:48,663 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:21:19 2024-11-22 03:16:48,664 INFO 3318922:repeater task lastensq_os150nb starting 29.7 seconds late 2024-11-22 03:16:51,326 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:16:51,326 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:16:14 2024-11-22 03:16:51,327 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:21:19 2024-11-22 03:16:51,327 INFO 3318922:repeater task quick_os150nb starting 32.3 seconds late 2024-11-22 03:16:55,069 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 03:16:55,069 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:16:14 2024-11-22 03:16:55,069 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:21:19 2024-11-22 03:16:55,070 INFO 3318922:repeater task lastfew_vec_os150nb starting 36.1 seconds late 2024-11-22 03:16:56,081 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:16:56,082 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:16:14 2024-11-22 03:16:56,082 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:21:19 2024-11-22 03:16:56,082 INFO 3318922:repeater task 3dayplots_os150nb starting 36.1 seconds late 2024-11-22 03:17:02,385 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 03:17:02,385 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:21:19 2024-11-22 03:17:02,387 INFO 3318922:repeater task lastensq_os38nb starting 41.4 seconds late 2024-11-22 03:17:05,180 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-22 03:17:05,180 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:16:16 2024-11-22 03:17:05,180 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:21:21 2024-11-22 03:17:05,180 INFO 3318922:repeater task quick_os38nb starting 44.2 seconds late 2024-11-22 03:17:09,370 INFO 3318922:repeater task quick_os38nb ended after 4.2 seconds 2024-11-22 03:17:09,370 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:16:16 2024-11-22 03:17:09,370 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:21:21 2024-11-22 03:17:09,371 INFO 3318922:repeater task lastfew_vec_os38nb starting 48.4 seconds late 2024-11-22 03:17:10,406 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:17:10,406 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:16:16 2024-11-22 03:17:10,406 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:21:21 2024-11-22 03:17:10,406 INFO 3318922:repeater task 3dayplots_os38nb starting 50.4 seconds late 2024-11-22 03:17:16,825 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.4 seconds 2024-11-22 03:17:16,825 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:21:19 2024-11-22 03:17:16,826 INFO 3318922:repeater task daily_netcdf starting 56.8 seconds late 2024-11-22 03:17:17,659 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 03:17:17,659 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:18:22,840 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 03:18:26,073 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:18:26,073 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:18:14 2024-11-22 03:18:26,073 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:20:19 2024-11-22 03:18:26,073 INFO 3318922:repeater task quick_wh300 starting 7.1 seconds late 2024-11-22 03:18:31,678 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:18:31,679 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:18:14 2024-11-22 03:18:31,679 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:20:19 2024-11-22 03:18:31,679 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2024-11-22 03:18:32,725 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:18:32,725 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:18:14 2024-11-22 03:18:32,725 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:20:19 2024-11-22 03:20:23,000 INFO 3318922:repeater task lastensq_wh300 starting 4.0 seconds late 2024-11-22 03:20:26,265 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:20:26,265 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:20:14 2024-11-22 03:20:26,265 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:22:19 2024-11-22 03:20:26,266 INFO 3318922:repeater task quick_wh300 starting 7.3 seconds late 2024-11-22 03:20:31,837 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:20:31,837 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:20:14 2024-11-22 03:20:31,837 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:22:19 2024-11-22 03:20:31,838 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2024-11-22 03:20:32,890 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 03:20:32,890 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:20:14 2024-11-22 03:20:32,890 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:22:19 2024-11-22 03:21:23,029 INFO 3318922:repeater task 3dayplots_wh300 starting 3.1 seconds late 2024-11-22 03:21:29,635 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 03:21:29,635 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:21:29,636 INFO 3318922:repeater task lastensq_os150nb starting 10.6 seconds late 2024-11-22 03:21:32,436 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 03:21:32,437 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:21:14 2024-11-22 03:21:32,437 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:21:32,437 INFO 3318922:repeater task quick_os150nb starting 13.4 seconds late 2024-11-22 03:21:35,937 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 03:21:35,937 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:21:14 2024-11-22 03:21:35,937 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:21:35,938 INFO 3318922:repeater task lastfew_vec_os150nb starting 16.9 seconds late 2024-11-22 03:21:36,944 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:21:36,944 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:21:14 2024-11-22 03:21:36,944 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:21:36,945 INFO 3318922:repeater task 3dayplots_os150nb starting 17.0 seconds late 2024-11-22 03:21:43,214 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 03:21:43,214 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:21:43,215 INFO 3318922:repeater task lastensq_os38nb starting 22.2 seconds late 2024-11-22 03:21:45,946 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 03:21:45,946 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:21:16 2024-11-22 03:21:45,946 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:26:21 2024-11-22 03:21:45,947 INFO 3318922:repeater task quick_os38nb starting 24.9 seconds late 2024-11-22 03:21:49,847 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 03:21:49,848 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:21:16 2024-11-22 03:21:49,848 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:26:21 2024-11-22 03:21:49,848 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.8 seconds late 2024-11-22 03:21:50,852 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:21:50,852 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:21:16 2024-11-22 03:21:50,852 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:26:21 2024-11-22 03:21:50,852 INFO 3318922:repeater task 3dayplots_os38nb starting 30.9 seconds late 2024-11-22 03:21:57,021 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 03:21:57,021 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:22:22,124 INFO 3318922:repeater task lastensq_wh300 starting 3.1 seconds late 2024-11-22 03:22:25,416 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:22:25,416 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:22:14 2024-11-22 03:22:25,416 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:24:19 2024-11-22 03:22:25,416 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-22 03:22:30,923 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 03:22:30,924 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:22:14 2024-11-22 03:22:30,924 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:24:19 2024-11-22 03:22:30,924 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2024-11-22 03:22:31,955 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:22:31,956 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:22:14 2024-11-22 03:22:31,956 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:24:19 2024-11-22 03:24:22,272 INFO 3318922:repeater task lastensq_wh300 starting 3.3 seconds late 2024-11-22 03:24:25,481 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:24:25,482 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:24:14 2024-11-22 03:24:25,482 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:24:25,482 INFO 3318922:repeater task quick_wh300 starting 6.5 seconds late 2024-11-22 03:24:31,030 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 03:24:31,031 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:24:14 2024-11-22 03:24:31,031 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:24:31,031 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2024-11-22 03:24:32,055 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:24:32,056 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:24:14 2024-11-22 03:24:32,056 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:26:19 2024-11-22 03:26:17,350 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.4 seconds late 2024-11-22 03:26:18,785 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:26:18,785 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 03:26:18,785 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:18,785 INFO 3318922:repeater task grab_lastpings_wh300 starting 3.8 seconds late 2024-11-22 03:26:18,946 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 03:26:18,946 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:18,946 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.0 seconds late 2024-11-22 03:26:20,654 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:26:20,654 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:20,654 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.7 seconds late 2024-11-22 03:26:22,358 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:26:22,360 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:22,360 INFO 3318922:repeater task lastensq_os150nb starting 3.4 seconds late 2024-11-22 03:26:25,126 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 03:26:25,126 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:26:14 2024-11-22 03:26:25,126 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:31:19 2024-11-22 03:26:25,127 INFO 3318922:repeater task quick_os150nb starting 6.1 seconds late 2024-11-22 03:26:28,865 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 03:26:28,865 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:26:14 2024-11-22 03:26:28,865 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:31:19 2024-11-22 03:26:28,866 INFO 3318922:repeater task lastfew_vec_os150nb starting 9.9 seconds late 2024-11-22 03:26:29,880 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:26:29,880 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:26:14 2024-11-22 03:26:29,880 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:31:19 2024-11-22 03:26:29,881 INFO 3318922:repeater task 3dayplots_os150nb starting 9.9 seconds late 2024-11-22 03:26:36,256 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 03:26:36,257 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:31:19 2024-11-22 03:26:36,257 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.3 seconds late 2024-11-22 03:26:37,514 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:26:37,514 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 03:26:37,514 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:37,514 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.5 seconds late 2024-11-22 03:26:37,678 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 03:26:37,678 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:37,678 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.7 seconds late 2024-11-22 03:26:39,430 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-22 03:26:39,430 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:39,431 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.5 seconds late 2024-11-22 03:26:41,186 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-22 03:26:41,186 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:41,186 INFO 3318922:repeater task lastensq_os38nb starting 20.2 seconds late 2024-11-22 03:26:43,851 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 03:26:43,851 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:26:16 2024-11-22 03:26:43,851 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:31:21 2024-11-22 03:26:43,851 INFO 3318922:repeater task quick_os38nb starting 22.9 seconds late 2024-11-22 03:26:47,570 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-22 03:26:47,570 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:26:16 2024-11-22 03:26:47,570 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:31:21 2024-11-22 03:26:47,571 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.6 seconds late 2024-11-22 03:26:48,567 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:26:48,567 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:26:16 2024-11-22 03:26:48,568 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:31:21 2024-11-22 03:26:48,568 INFO 3318922:repeater task 3dayplots_os38nb starting 28.6 seconds late 2024-11-22 03:26:54,734 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 03:26:54,735 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:31:19 2024-11-22 03:26:54,735 INFO 3318922:repeater task plot_tsstats_os38nb starting 39.8 seconds late 2024-11-22 03:26:56,029 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:26:56,029 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 03:26:56,029 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:56,029 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.1 seconds late 2024-11-22 03:26:56,189 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 03:26:56,189 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:56,189 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.2 seconds late 2024-11-22 03:26:57,871 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 03:26:57,871 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:57,871 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.9 seconds late 2024-11-22 03:26:59,608 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 03:26:59,608 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:41:14 2024-11-22 03:26:59,609 INFO 3318922:repeater task daily_netcdf starting 39.6 seconds late 2024-11-22 03:27:00,389 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 03:27:00,391 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:27:00,391 INFO 3318922:repeater task ggatime_diagnostics starting 45.4 seconds late 2024-11-22 03:27:08,065 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 03:27:08,065 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:27:08,065 INFO 3318922:repeater task ggatime_diagnostics starting 53.1 seconds late 2024-11-22 03:27:15,507 INFO 3318922:repeater task ggatime_diagnostics ended after 7.4 seconds 2024-11-22 03:27:15,507 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:27:15,507 INFO 3318922:repeater task ggatime_diagnostics starting 60.5 seconds late 2024-11-22 03:27:23,063 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 03:27:23,063 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:27:23,063 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 67.9 seconds late 2024-11-22 03:27:24,896 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 03:27:24,896 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 03:27:24,896 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 03:41:15 2024-11-22 03:27:24,897 INFO 3318922:repeater task plot_hbins starting 69.8 seconds late 2024-11-22 03:27:29,068 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 03:27:29,068 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-22 03:27:29,068 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 03:41:15 2024-11-22 03:27:29,068 INFO 3318922:repeater task report_generator starting 73.9 seconds late 2024-11-22 03:27:33,647 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-22 03:27:33,649 INFO 3318922:repeater task report_generator ended after 4.6 seconds 2024-11-22 03:27:33,649 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/22 04:26:15 2024-11-22 03:27:38,654 INFO 3318922:repeater task noisy_rawtemps_os150 starting 78.7 seconds late 2024-11-22 03:27:41,838 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 03:27:41,838 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:27:41,838 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 81.9 seconds late 2024-11-22 03:27:44,945 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-22 03:27:44,945 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:27:44,946 INFO 3318922:repeater task noisy_rawtemps_os38 starting 85.0 seconds late 2024-11-22 03:27:48,579 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 03:27:48,580 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:27:48,580 INFO 3318922:repeater task lastensq_wh300 starting 89.6 seconds late 2024-11-22 03:27:51,798 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:27:51,798 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:26:14 2024-11-22 03:27:51,798 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:28:19 2024-11-22 03:27:51,799 INFO 3318922:repeater task quick_wh300 starting 92.8 seconds late 2024-11-22 03:27:57,290 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 03:27:57,290 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:26:14 2024-11-22 03:27:57,290 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:28:19 2024-11-22 03:27:57,291 INFO 3318922:repeater task lastfew_vec_wh300 starting 98.3 seconds late 2024-11-22 03:27:58,324 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:27:58,324 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:26:14 2024-11-22 03:27:58,324 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:28:19 2024-11-22 03:27:58,325 INFO 3318922:repeater task 3dayplots_wh300 starting 98.3 seconds late 2024-11-22 03:28:05,182 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.9 seconds 2024-11-22 03:28:05,182 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:31:19 2024-11-22 03:28:20,224 INFO 3318922:repeater task lastensq_wh300 starting 1.2 seconds late 2024-11-22 03:28:23,701 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-22 03:28:23,702 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:28:14 2024-11-22 03:28:23,702 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:30:19 2024-11-22 03:28:23,702 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 03:28:29,519 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 03:28:29,519 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:28:14 2024-11-22 03:28:29,519 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:30:19 2024-11-22 03:28:29,520 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-22 03:28:30,606 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 03:28:30,606 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:28:14 2024-11-22 03:28:30,606 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:30:19 2024-11-22 03:30:20,935 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 03:30:24,277 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:30:24,278 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:30:14 2024-11-22 03:30:24,278 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:32:19 2024-11-22 03:30:24,278 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-22 03:30:29,675 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 03:30:29,675 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:30:14 2024-11-22 03:30:29,675 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:32:19 2024-11-22 03:30:29,676 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2024-11-22 03:30:30,723 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:30:30,723 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:30:14 2024-11-22 03:30:30,723 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:32:19 2024-11-22 03:31:20,869 INFO 3318922:repeater task 3dayplots_wh300 starting 0.9 seconds late 2024-11-22 03:31:27,438 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 03:31:27,438 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:31:27,439 INFO 3318922:repeater task lastensq_os150nb starting 8.4 seconds late 2024-11-22 03:31:30,134 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:31:30,135 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:31:14 2024-11-22 03:31:30,135 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:31:30,135 INFO 3318922:repeater task quick_os150nb starting 11.1 seconds late 2024-11-22 03:31:33,647 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 03:31:33,647 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:31:14 2024-11-22 03:31:33,647 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:31:33,648 INFO 3318922:repeater task lastfew_vec_os150nb starting 14.6 seconds late 2024-11-22 03:31:34,639 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:31:34,640 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:31:14 2024-11-22 03:31:34,640 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:31:34,640 INFO 3318922:repeater task 3dayplots_os150nb starting 14.7 seconds late 2024-11-22 03:31:41,259 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.6 seconds 2024-11-22 03:31:41,259 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:31:41,260 INFO 3318922:repeater task lastensq_os38nb starting 20.3 seconds late 2024-11-22 03:31:43,861 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 03:31:43,862 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:31:16 2024-11-22 03:31:43,862 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:36:21 2024-11-22 03:31:43,862 INFO 3318922:repeater task quick_os38nb starting 22.9 seconds late 2024-11-22 03:31:47,727 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 03:31:47,727 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:31:16 2024-11-22 03:31:47,727 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:36:21 2024-11-22 03:31:47,728 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.7 seconds late 2024-11-22 03:31:48,753 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:31:48,753 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:31:16 2024-11-22 03:31:48,753 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:36:21 2024-11-22 03:31:48,753 INFO 3318922:repeater task 3dayplots_os38nb starting 28.8 seconds late 2024-11-22 03:31:54,863 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 03:31:54,864 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:32:19,929 INFO 3318922:repeater task lastensq_wh300 starting 0.9 seconds late 2024-11-22 03:32:23,156 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:32:23,156 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:32:14 2024-11-22 03:32:23,156 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:34:19 2024-11-22 03:32:23,156 INFO 3318922:repeater task quick_wh300 starting 4.2 seconds late 2024-11-22 03:32:28,462 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 03:32:28,462 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:32:14 2024-11-22 03:32:28,462 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:34:19 2024-11-22 03:32:28,463 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.5 seconds late 2024-11-22 03:32:29,510 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:32:29,511 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:32:14 2024-11-22 03:32:29,511 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:34:19 2024-11-22 03:34:19,823 INFO 3318922:repeater task lastensq_wh300 starting 0.8 seconds late 2024-11-22 03:34:23,087 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:34:23,087 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:34:14 2024-11-22 03:34:23,088 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:34:23,088 INFO 3318922:repeater task quick_wh300 starting 4.1 seconds late 2024-11-22 03:34:28,733 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:34:28,733 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:34:14 2024-11-22 03:34:28,733 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:34:28,734 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.7 seconds late 2024-11-22 03:34:29,767 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:34:29,767 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:34:14 2024-11-22 03:34:29,767 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:36:19 2024-11-22 03:36:20,106 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.1 seconds late 2024-11-22 03:36:23,289 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 03:36:23,290 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:36:23,290 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.3 seconds late 2024-11-22 03:36:26,447 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 03:36:26,447 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:36:26,447 INFO 3318922:repeater task noisy_rawtemps_os38 starting 6.5 seconds late 2024-11-22 03:36:30,045 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 03:36:30,045 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:36:30,045 INFO 3318922:repeater task lastensq_wh300 starting 11.0 seconds late 2024-11-22 03:36:33,259 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:36:33,259 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:36:14 2024-11-22 03:36:33,260 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:38:19 2024-11-22 03:36:33,260 INFO 3318922:repeater task quick_wh300 starting 14.3 seconds late 2024-11-22 03:36:38,938 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 03:36:38,938 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:36:14 2024-11-22 03:36:38,938 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:38:19 2024-11-22 03:36:38,939 INFO 3318922:repeater task lastfew_vec_wh300 starting 19.9 seconds late 2024-11-22 03:36:39,972 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:36:39,972 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:36:14 2024-11-22 03:36:39,972 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:38:19 2024-11-22 03:36:39,973 INFO 3318922:repeater task 3dayplots_wh300 starting 20.0 seconds late 2024-11-22 03:36:46,591 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 03:36:46,591 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:41:19 2024-11-22 03:36:46,592 INFO 3318922:repeater task lastensq_os150nb starting 27.6 seconds late 2024-11-22 03:36:49,264 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:36:49,264 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:36:14 2024-11-22 03:36:49,264 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:41:19 2024-11-22 03:36:49,265 INFO 3318922:repeater task quick_os150nb starting 30.3 seconds late 2024-11-22 03:36:52,842 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 03:36:52,842 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:36:14 2024-11-22 03:36:52,842 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:41:19 2024-11-22 03:36:52,843 INFO 3318922:repeater task lastfew_vec_os150nb starting 33.8 seconds late 2024-11-22 03:36:53,879 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:36:53,879 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:36:14 2024-11-22 03:36:53,879 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:41:19 2024-11-22 03:36:53,880 INFO 3318922:repeater task 3dayplots_os150nb starting 33.9 seconds late 2024-11-22 03:37:00,261 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 03:37:00,261 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:41:19 2024-11-22 03:37:00,262 INFO 3318922:repeater task lastensq_os38nb starting 39.3 seconds late 2024-11-22 03:37:02,888 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 03:37:02,888 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:36:16 2024-11-22 03:37:02,888 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:41:21 2024-11-22 03:37:02,888 INFO 3318922:repeater task quick_os38nb starting 41.9 seconds late 2024-11-22 03:37:06,790 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 03:37:06,791 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:36:16 2024-11-22 03:37:06,791 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:41:21 2024-11-22 03:37:06,791 INFO 3318922:repeater task lastfew_vec_os38nb starting 45.8 seconds late 2024-11-22 03:37:07,799 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:37:07,799 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:36:16 2024-11-22 03:37:07,800 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:41:21 2024-11-22 03:37:07,800 INFO 3318922:repeater task 3dayplots_os38nb starting 47.8 seconds late 2024-11-22 03:37:14,042 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 03:37:14,042 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:41:19 2024-11-22 03:37:14,043 INFO 3318922:repeater task daily_netcdf starting 54.1 seconds late 2024-11-22 03:37:14,817 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 03:37:14,817 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:38:20,008 INFO 3318922:repeater task lastensq_wh300 starting 1.0 seconds late 2024-11-22 03:38:23,320 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:38:23,321 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:38:14 2024-11-22 03:38:23,321 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:40:19 2024-11-22 03:38:23,321 INFO 3318922:repeater task quick_wh300 starting 4.3 seconds late 2024-11-22 03:38:29,039 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 03:38:29,039 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:38:14 2024-11-22 03:38:29,040 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:40:19 2024-11-22 03:38:29,040 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-22 03:38:30,093 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 03:38:30,094 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:38:14 2024-11-22 03:38:30,094 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:40:19 2024-11-22 03:40:20,433 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 03:40:23,675 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:40:23,675 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:40:14 2024-11-22 03:40:23,675 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:42:19 2024-11-22 03:40:23,676 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 03:40:29,222 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 03:40:29,222 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:40:14 2024-11-22 03:40:29,222 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:42:19 2024-11-22 03:40:29,222 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2024-11-22 03:40:30,264 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:40:30,265 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:40:14 2024-11-22 03:40:30,265 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:42:19 2024-11-22 03:41:15,402 INFO 3318922:repeater task plot_tsstats_wh300 starting 0.4 seconds late 2024-11-22 03:41:16,848 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:41:16,848 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 03:41:16,849 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:16,849 INFO 3318922:repeater task grab_lastpings_wh300 starting 1.9 seconds late 2024-11-22 03:41:17,008 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 03:41:17,008 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:17,009 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.0 seconds late 2024-11-22 03:41:18,755 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:41:18,755 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:18,756 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.8 seconds late 2024-11-22 03:41:20,477 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:41:20,477 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:20,478 INFO 3318922:repeater task lastensq_os150nb starting 1.5 seconds late 2024-11-22 03:41:23,197 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:41:23,197 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:41:14 2024-11-22 03:41:23,197 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:41:23,198 INFO 3318922:repeater task quick_os150nb starting 4.2 seconds late 2024-11-22 03:41:27,033 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 03:41:27,034 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:41:14 2024-11-22 03:41:27,034 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:41:27,034 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.0 seconds late 2024-11-22 03:41:28,046 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:41:28,046 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:41:14 2024-11-22 03:41:28,046 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:41:28,046 INFO 3318922:repeater task 3dayplots_os150nb starting 8.1 seconds late 2024-11-22 03:41:34,365 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 03:41:34,365 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:41:34,366 INFO 3318922:repeater task plot_tsstats_os150nb starting 19.4 seconds late 2024-11-22 03:41:35,618 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:41:35,619 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 03:41:35,621 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:35,621 INFO 3318922:repeater task grab_lastpings_os150nb starting 20.6 seconds late 2024-11-22 03:41:35,788 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 03:41:35,789 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:35,789 INFO 3318922:repeater task beamdiagnostics_os150nb starting 20.8 seconds late 2024-11-22 03:41:37,472 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 03:41:37,472 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:37,472 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.5 seconds late 2024-11-22 03:41:39,212 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 03:41:39,212 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:39,213 INFO 3318922:repeater task lastensq_os38nb starting 18.2 seconds late 2024-11-22 03:41:41,899 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 03:41:41,899 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:41:16 2024-11-22 03:41:41,899 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:46:21 2024-11-22 03:41:41,899 INFO 3318922:repeater task quick_os38nb starting 20.9 seconds late 2024-11-22 03:41:46,067 INFO 3318922:repeater task quick_os38nb ended after 4.2 seconds 2024-11-22 03:41:46,067 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:41:16 2024-11-22 03:41:46,067 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:46:21 2024-11-22 03:41:46,068 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.1 seconds late 2024-11-22 03:41:47,088 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:41:47,088 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:41:16 2024-11-22 03:41:47,088 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:46:21 2024-11-22 03:41:47,089 INFO 3318922:repeater task 3dayplots_os38nb starting 27.1 seconds late 2024-11-22 03:41:53,292 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 03:41:53,292 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:41:53,292 INFO 3318922:repeater task plot_tsstats_os38nb starting 38.3 seconds late 2024-11-22 03:41:54,591 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:41:54,591 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 03:41:54,591 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:54,591 INFO 3318922:repeater task grab_lastpings_os38nb starting 39.6 seconds late 2024-11-22 03:41:54,751 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 03:41:54,751 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:54,751 INFO 3318922:repeater task beamdiagnostics_os38nb starting 39.8 seconds late 2024-11-22 03:41:56,436 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 03:41:56,436 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:56,436 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.5 seconds late 2024-11-22 03:41:58,233 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 03:41:58,233 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 03:56:14 2024-11-22 03:41:58,234 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 43.1 seconds late 2024-11-22 03:42:00,139 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 03:42:00,140 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 03:42:00,140 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 03:56:15 2024-11-22 03:42:00,140 INFO 3318922:repeater task plot_hbins starting 45.0 seconds late 2024-11-22 03:42:04,334 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 03:42:04,334 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-22 03:42:04,334 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 03:56:15 2024-11-22 03:42:09,343 INFO 3318922:repeater task 3dayplots_wh300 starting 49.4 seconds late 2024-11-22 03:42:15,875 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 03:42:15,875 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:42:20,886 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 03:42:24,103 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:42:24,104 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:42:14 2024-11-22 03:42:24,104 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:44:19 2024-11-22 03:42:24,104 INFO 3318922:repeater task quick_wh300 starting 5.1 seconds late 2024-11-22 03:42:29,583 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 03:42:29,583 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:42:14 2024-11-22 03:42:29,583 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:44:19 2024-11-22 03:42:29,584 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2024-11-22 03:42:30,621 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:42:30,621 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:42:14 2024-11-22 03:42:30,622 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:44:19 2024-11-22 03:44:20,936 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 03:44:24,261 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:44:24,261 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:44:14 2024-11-22 03:44:24,261 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:44:24,262 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-22 03:44:29,944 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 03:44:29,944 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:44:14 2024-11-22 03:44:29,944 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:44:29,945 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2024-11-22 03:44:30,975 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:44:30,976 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:44:14 2024-11-22 03:44:30,976 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:46:19 2024-11-22 03:46:21,294 INFO 3318922:repeater task noisy_rawtemps_os150 starting 1.3 seconds late 2024-11-22 03:46:24,433 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 03:46:24,433 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:46:24,434 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 4.5 seconds late 2024-11-22 03:46:27,584 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 03:46:27,585 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:46:27,585 INFO 3318922:repeater task noisy_rawtemps_os38 starting 7.6 seconds late 2024-11-22 03:46:31,237 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 03:46:31,237 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:46:31,237 INFO 3318922:repeater task lastensq_wh300 starting 12.2 seconds late 2024-11-22 03:46:34,466 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:46:34,467 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:46:14 2024-11-22 03:46:34,467 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:48:19 2024-11-22 03:46:34,467 INFO 3318922:repeater task quick_wh300 starting 15.5 seconds late 2024-11-22 03:46:40,083 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:46:40,083 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:46:14 2024-11-22 03:46:40,083 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:48:19 2024-11-22 03:46:40,083 INFO 3318922:repeater task lastfew_vec_wh300 starting 21.1 seconds late 2024-11-22 03:46:41,124 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:46:41,124 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:46:14 2024-11-22 03:46:41,124 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:48:19 2024-11-22 03:46:41,125 INFO 3318922:repeater task 3dayplots_wh300 starting 21.1 seconds late 2024-11-22 03:46:47,865 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 03:46:47,865 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:51:19 2024-11-22 03:46:47,866 INFO 3318922:repeater task lastensq_os150nb starting 28.9 seconds late 2024-11-22 03:46:50,661 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 03:46:50,661 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:46:14 2024-11-22 03:46:50,662 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:51:19 2024-11-22 03:46:50,662 INFO 3318922:repeater task quick_os150nb starting 31.7 seconds late 2024-11-22 03:46:54,207 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 03:46:54,207 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:46:14 2024-11-22 03:46:54,207 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:51:19 2024-11-22 03:46:54,207 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.2 seconds late 2024-11-22 03:46:55,219 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:46:55,219 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:46:14 2024-11-22 03:46:55,219 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:51:19 2024-11-22 03:46:55,219 INFO 3318922:repeater task 3dayplots_os150nb starting 35.2 seconds late 2024-11-22 03:47:01,459 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 03:47:01,459 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:51:19 2024-11-22 03:47:01,460 INFO 3318922:repeater task lastensq_os38nb starting 40.5 seconds late 2024-11-22 03:47:04,239 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-22 03:47:04,240 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:46:16 2024-11-22 03:47:04,240 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:51:21 2024-11-22 03:47:04,240 INFO 3318922:repeater task quick_os38nb starting 43.2 seconds late 2024-11-22 03:47:08,234 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 03:47:08,234 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:46:16 2024-11-22 03:47:08,235 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:51:21 2024-11-22 03:47:08,235 INFO 3318922:repeater task lastfew_vec_os38nb starting 47.2 seconds late 2024-11-22 03:47:09,271 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:47:09,271 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:46:16 2024-11-22 03:47:09,271 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:51:21 2024-11-22 03:47:09,272 INFO 3318922:repeater task 3dayplots_os38nb starting 49.3 seconds late 2024-11-22 03:47:15,629 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.4 seconds 2024-11-22 03:47:15,629 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:51:19 2024-11-22 03:47:15,630 INFO 3318922:repeater task daily_netcdf starting 55.7 seconds late 2024-11-22 03:47:16,364 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 03:47:16,364 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:48:21,543 INFO 3318922:repeater task lastensq_wh300 starting 2.5 seconds late 2024-11-22 03:48:24,871 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:48:24,871 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:48:14 2024-11-22 03:48:24,871 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:50:19 2024-11-22 03:48:24,872 INFO 3318922:repeater task quick_wh300 starting 5.9 seconds late 2024-11-22 03:48:30,244 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 03:48:30,244 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:48:14 2024-11-22 03:48:30,244 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:50:19 2024-11-22 03:48:30,244 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.2 seconds late 2024-11-22 03:48:31,279 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:48:31,280 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:48:14 2024-11-22 03:48:31,280 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:50:19 2024-11-22 03:50:21,596 INFO 3318922:repeater task lastensq_wh300 starting 2.6 seconds late 2024-11-22 03:50:24,815 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:50:24,816 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:50:14 2024-11-22 03:50:24,816 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:52:19 2024-11-22 03:50:24,816 INFO 3318922:repeater task quick_wh300 starting 5.8 seconds late 2024-11-22 03:50:30,208 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 03:50:30,209 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:50:14 2024-11-22 03:50:30,209 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:52:19 2024-11-22 03:50:30,209 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.2 seconds late 2024-11-22 03:50:31,266 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 03:50:31,267 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:50:14 2024-11-22 03:50:31,267 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:52:19 2024-11-22 03:51:21,392 INFO 3318922:repeater task 3dayplots_wh300 starting 1.4 seconds late 2024-11-22 03:51:28,053 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 03:51:28,053 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:51:28,055 INFO 3318922:repeater task lastensq_os150nb starting 9.1 seconds late 2024-11-22 03:51:30,756 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:51:30,756 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:51:14 2024-11-22 03:51:30,756 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:51:30,756 INFO 3318922:repeater task quick_os150nb starting 11.8 seconds late 2024-11-22 03:51:34,515 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 03:51:34,515 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:51:14 2024-11-22 03:51:34,515 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:51:34,515 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.5 seconds late 2024-11-22 03:51:35,544 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:51:35,544 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:51:14 2024-11-22 03:51:35,544 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:51:35,545 INFO 3318922:repeater task 3dayplots_os150nb starting 15.6 seconds late 2024-11-22 03:51:41,845 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 03:51:41,845 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:51:41,846 INFO 3318922:repeater task lastensq_os38nb starting 20.8 seconds late 2024-11-22 03:51:44,482 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 03:51:44,482 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:51:16 2024-11-22 03:51:44,482 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 03:56:21 2024-11-22 03:51:44,482 INFO 3318922:repeater task quick_os38nb starting 23.5 seconds late 2024-11-22 03:51:48,253 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 03:51:48,254 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:51:16 2024-11-22 03:51:48,254 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 03:56:21 2024-11-22 03:51:48,254 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.3 seconds late 2024-11-22 03:51:49,286 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:51:49,286 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:51:16 2024-11-22 03:51:49,286 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 03:56:21 2024-11-22 03:51:49,287 INFO 3318922:repeater task 3dayplots_os38nb starting 29.3 seconds late 2024-11-22 03:51:55,438 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 03:51:55,438 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:52:20,501 INFO 3318922:repeater task lastensq_wh300 starting 1.5 seconds late 2024-11-22 03:52:23,693 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 03:52:23,693 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:52:14 2024-11-22 03:52:23,693 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:54:19 2024-11-22 03:52:23,694 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 03:52:29,342 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:52:29,343 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:52:14 2024-11-22 03:52:29,343 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:54:19 2024-11-22 03:52:29,343 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2024-11-22 03:52:30,376 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:52:30,376 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:52:14 2024-11-22 03:52:30,376 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:54:19 2024-11-22 03:54:20,681 INFO 3318922:repeater task lastensq_wh300 starting 1.7 seconds late 2024-11-22 03:54:24,149 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-22 03:54:24,149 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:54:14 2024-11-22 03:54:24,149 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:54:24,150 INFO 3318922:repeater task quick_wh300 starting 5.2 seconds late 2024-11-22 03:54:29,812 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 03:54:29,812 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:54:14 2024-11-22 03:54:29,812 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:54:29,813 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-22 03:54:30,844 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:54:30,844 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:54:14 2024-11-22 03:54:30,845 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:56:19 2024-11-22 03:56:16,149 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.2 seconds late 2024-11-22 03:56:17,601 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:56:17,601 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 03:56:17,601 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:17,601 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.6 seconds late 2024-11-22 03:56:17,761 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 03:56:17,762 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:17,762 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.8 seconds late 2024-11-22 03:56:19,467 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:56:19,467 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:19,468 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.5 seconds late 2024-11-22 03:56:21,130 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 03:56:21,130 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:21,131 INFO 3318922:repeater task lastensq_os150nb starting 2.1 seconds late 2024-11-22 03:56:23,858 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 03:56:23,859 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:56:14 2024-11-22 03:56:23,859 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:01:19 2024-11-22 03:56:23,859 INFO 3318922:repeater task quick_os150nb starting 4.9 seconds late 2024-11-22 03:56:27,663 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 03:56:27,664 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:56:14 2024-11-22 03:56:27,664 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:01:19 2024-11-22 03:56:27,664 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.7 seconds late 2024-11-22 03:56:28,679 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 03:56:28,679 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 03:56:14 2024-11-22 03:56:28,680 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:01:19 2024-11-22 03:56:28,680 INFO 3318922:repeater task 3dayplots_os150nb starting 8.7 seconds late 2024-11-22 03:56:35,136 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.5 seconds 2024-11-22 03:56:35,136 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:01:19 2024-11-22 03:56:35,136 INFO 3318922:repeater task plot_tsstats_os150nb starting 20.2 seconds late 2024-11-22 03:56:36,384 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:56:36,384 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.2 seconds 2024-11-22 03:56:36,384 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:36,384 INFO 3318922:repeater task grab_lastpings_os150nb starting 21.4 seconds late 2024-11-22 03:56:36,547 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 03:56:36,547 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:36,548 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.6 seconds late 2024-11-22 03:56:38,266 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 03:56:38,266 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:38,267 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.3 seconds late 2024-11-22 03:56:40,029 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-22 03:56:40,029 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:40,029 INFO 3318922:repeater task lastensq_os38nb starting 19.0 seconds late 2024-11-22 03:56:42,716 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 03:56:42,717 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:56:16 2024-11-22 03:56:42,717 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:01:21 2024-11-22 03:56:42,717 INFO 3318922:repeater task quick_os38nb starting 21.7 seconds late 2024-11-22 03:56:46,762 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 03:56:46,762 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:56:16 2024-11-22 03:56:46,762 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:01:21 2024-11-22 03:56:46,762 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.8 seconds late 2024-11-22 03:56:47,807 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 03:56:47,807 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 03:56:16 2024-11-22 03:56:47,807 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:01:21 2024-11-22 03:56:47,807 INFO 3318922:repeater task 3dayplots_os38nb starting 27.8 seconds late 2024-11-22 03:56:54,049 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 03:56:54,050 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:01:19 2024-11-22 03:56:54,050 INFO 3318922:repeater task plot_tsstats_os38nb starting 39.1 seconds late 2024-11-22 03:56:55,344 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 03:56:55,344 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 03:56:55,344 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:55,344 INFO 3318922:repeater task grab_lastpings_os38nb starting 40.4 seconds late 2024-11-22 03:56:55,505 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 03:56:55,505 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:55,506 INFO 3318922:repeater task beamdiagnostics_os38nb starting 40.5 seconds late 2024-11-22 03:56:57,213 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 03:56:57,213 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:57,213 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.2 seconds late 2024-11-22 03:56:58,970 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 03:56:58,970 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:11:14 2024-11-22 03:56:58,970 INFO 3318922:repeater task daily_netcdf starting 39.0 seconds late 2024-11-22 03:56:59,764 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 03:56:59,765 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 04:06:19 2024-11-22 03:56:59,765 INFO 3318922:repeater task ggatime_diagnostics starting 44.8 seconds late 2024-11-22 03:57:07,658 INFO 3318922:repeater task ggatime_diagnostics ended after 7.9 seconds 2024-11-22 03:57:07,658 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 04:26:14 2024-11-22 03:57:07,658 INFO 3318922:repeater task ggatime_diagnostics starting 52.7 seconds late 2024-11-22 03:57:15,197 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 03:57:15,197 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 04:26:14 2024-11-22 03:57:15,197 INFO 3318922:repeater task ggatime_diagnostics starting 60.2 seconds late 2024-11-22 03:57:22,798 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 03:57:22,798 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 04:26:14 2024-11-22 03:57:22,798 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 67.7 seconds late 2024-11-22 03:57:24,641 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 03:57:24,641 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 03:57:24,641 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 04:11:15 2024-11-22 03:57:24,642 INFO 3318922:repeater task plot_hbins starting 69.5 seconds late 2024-11-22 03:57:28,823 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 03:57:28,823 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-22 03:57:28,823 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 04:11:15 2024-11-22 03:57:33,829 INFO 3318922:repeater task noisy_rawtemps_os150 starting 73.9 seconds late 2024-11-22 03:57:36,986 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 03:57:36,986 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 04:06:19 2024-11-22 03:57:36,986 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 77.0 seconds late 2024-11-22 03:57:40,208 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 03:57:40,208 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 04:06:19 2024-11-22 03:57:40,240 INFO 3318922:repeater task noisy_rawtemps_os38 starting 80.3 seconds late 2024-11-22 03:57:43,898 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 03:57:43,898 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 04:06:19 2024-11-22 03:57:43,899 INFO 3318922:repeater task lastensq_wh300 starting 84.9 seconds late 2024-11-22 03:57:47,235 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:57:47,235 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:56:14 2024-11-22 03:57:47,235 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 03:58:19 2024-11-22 03:57:47,235 INFO 3318922:repeater task quick_wh300 starting 88.2 seconds late 2024-11-22 03:57:52,829 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 03:57:52,829 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:56:14 2024-11-22 03:57:52,830 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 03:58:19 2024-11-22 03:57:52,830 INFO 3318922:repeater task lastfew_vec_wh300 starting 93.8 seconds late 2024-11-22 03:57:53,850 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:57:53,850 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:56:14 2024-11-22 03:57:53,850 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 03:58:19 2024-11-22 03:57:53,851 INFO 3318922:repeater task 3dayplots_wh300 starting 93.9 seconds late 2024-11-22 03:58:00,474 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 03:58:00,474 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:01:19 2024-11-22 03:58:20,529 INFO 3318922:repeater task lastensq_wh300 starting 1.5 seconds late 2024-11-22 03:58:23,814 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 03:58:23,815 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:58:14 2024-11-22 03:58:23,815 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:00:19 2024-11-22 03:58:23,815 INFO 3318922:repeater task quick_wh300 starting 4.8 seconds late 2024-11-22 03:58:29,238 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 03:58:29,239 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:58:14 2024-11-22 03:58:29,239 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:00:19 2024-11-22 03:58:29,239 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2024-11-22 03:58:30,266 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 03:58:30,267 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 03:58:14 2024-11-22 03:58:30,267 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:00:19 2024-11-22 04:00:20,564 INFO 3318922:repeater task lastensq_wh300 starting 1.6 seconds late 2024-11-22 04:00:23,901 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 04:00:23,901 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:00:14 2024-11-22 04:00:23,902 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:02:19 2024-11-22 04:00:23,902 INFO 3318922:repeater task quick_wh300 starting 4.9 seconds late 2024-11-22 04:00:29,552 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:00:29,553 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:00:14 2024-11-22 04:00:29,553 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:02:19 2024-11-22 04:00:29,553 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2024-11-22 04:00:30,585 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:00:30,585 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:00:14 2024-11-22 04:00:30,585 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:02:19 2024-11-22 04:01:20,725 INFO 3318922:repeater task 3dayplots_wh300 starting 0.7 seconds late 2024-11-22 04:01:27,393 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 04:01:27,393 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:01:27,394 INFO 3318922:repeater task lastensq_os150nb starting 8.4 seconds late 2024-11-22 04:01:30,082 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 04:01:30,082 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:01:14 2024-11-22 04:01:30,083 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:01:30,083 INFO 3318922:repeater task quick_os150nb starting 11.1 seconds late 2024-11-22 04:01:33,960 INFO 3318922:repeater task quick_os150nb ended after 3.9 seconds 2024-11-22 04:01:33,960 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:01:14 2024-11-22 04:01:33,960 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:01:33,961 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.0 seconds late 2024-11-22 04:01:34,982 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:01:34,982 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:01:14 2024-11-22 04:01:34,982 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:01:34,983 INFO 3318922:repeater task 3dayplots_os150nb starting 15.0 seconds late 2024-11-22 04:01:41,244 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 04:01:41,244 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:01:41,245 INFO 3318922:repeater task lastensq_os38nb starting 20.2 seconds late 2024-11-22 04:01:43,875 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 04:01:43,875 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:01:16 2024-11-22 04:01:43,875 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:06:21 2024-11-22 04:01:43,876 INFO 3318922:repeater task quick_os38nb starting 22.9 seconds late 2024-11-22 04:01:47,872 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 04:01:47,873 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:01:16 2024-11-22 04:01:47,873 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:06:21 2024-11-22 04:01:47,873 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.9 seconds late 2024-11-22 04:01:48,914 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:01:48,914 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:01:16 2024-11-22 04:01:48,914 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:06:21 2024-11-22 04:01:48,914 INFO 3318922:repeater task 3dayplots_os38nb starting 28.9 seconds late 2024-11-22 04:01:55,075 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:01:55,075 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:02:20,191 INFO 3318922:repeater task lastensq_wh300 starting 1.2 seconds late 2024-11-22 04:02:23,391 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:02:23,392 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:02:14 2024-11-22 04:02:23,392 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:04:19 2024-11-22 04:02:23,392 INFO 3318922:repeater task quick_wh300 starting 4.4 seconds late 2024-11-22 04:02:28,951 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:02:28,951 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:02:14 2024-11-22 04:02:28,951 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:04:19 2024-11-22 04:02:28,952 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-22 04:02:29,988 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:02:29,988 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:02:14 2024-11-22 04:02:29,988 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:04:19 2024-11-22 04:04:20,356 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-22 04:04:23,547 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:04:23,548 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:04:14 2024-11-22 04:04:23,548 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:04:23,548 INFO 3318922:repeater task quick_wh300 starting 4.5 seconds late 2024-11-22 04:04:29,117 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:04:29,117 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:04:14 2024-11-22 04:04:29,117 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:04:29,117 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2024-11-22 04:04:30,153 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:04:30,153 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:04:14 2024-11-22 04:04:30,154 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:06:19 2024-11-22 04:06:20,462 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.5 seconds late 2024-11-22 04:06:23,585 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 04:06:23,585 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:06:23,585 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.6 seconds late 2024-11-22 04:06:26,772 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 04:06:26,772 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:06:26,772 INFO 3318922:repeater task noisy_rawtemps_os38 starting 6.8 seconds late 2024-11-22 04:06:30,391 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 04:06:30,391 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:06:30,392 INFO 3318922:repeater task lastensq_wh300 starting 11.4 seconds late 2024-11-22 04:06:33,526 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-22 04:06:33,527 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:06:14 2024-11-22 04:06:33,527 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:08:19 2024-11-22 04:06:33,527 INFO 3318922:repeater task quick_wh300 starting 14.5 seconds late 2024-11-22 04:06:39,198 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:06:39,198 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:06:14 2024-11-22 04:06:39,198 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:08:19 2024-11-22 04:06:39,199 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.2 seconds late 2024-11-22 04:06:40,261 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 04:06:40,261 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:06:14 2024-11-22 04:06:40,261 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:08:19 2024-11-22 04:06:40,261 INFO 3318922:repeater task 3dayplots_wh300 starting 20.3 seconds late 2024-11-22 04:06:46,857 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 04:06:46,857 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:11:19 2024-11-22 04:06:46,858 INFO 3318922:repeater task lastensq_os150nb starting 27.9 seconds late 2024-11-22 04:06:49,517 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 04:06:49,517 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:06:14 2024-11-22 04:06:49,518 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:11:19 2024-11-22 04:06:49,518 INFO 3318922:repeater task quick_os150nb starting 30.5 seconds late 2024-11-22 04:06:53,409 INFO 3318922:repeater task quick_os150nb ended after 3.9 seconds 2024-11-22 04:06:53,409 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:06:14 2024-11-22 04:06:53,409 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:11:19 2024-11-22 04:06:53,410 INFO 3318922:repeater task lastfew_vec_os150nb starting 34.4 seconds late 2024-11-22 04:06:54,425 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:06:54,425 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:06:14 2024-11-22 04:06:54,426 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:11:19 2024-11-22 04:06:54,426 INFO 3318922:repeater task 3dayplots_os150nb starting 34.4 seconds late 2024-11-22 04:07:00,608 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:07:00,608 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:11:19 2024-11-22 04:07:00,609 INFO 3318922:repeater task lastensq_os38nb starting 39.6 seconds late 2024-11-22 04:07:03,295 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 04:07:03,295 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:06:16 2024-11-22 04:07:03,295 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:11:21 2024-11-22 04:07:03,296 INFO 3318922:repeater task quick_os38nb starting 42.3 seconds late 2024-11-22 04:07:07,312 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 04:07:07,314 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:06:16 2024-11-22 04:07:07,315 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:11:21 2024-11-22 04:07:07,315 INFO 3318922:repeater task lastfew_vec_os38nb starting 46.3 seconds late 2024-11-22 04:07:08,346 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:07:08,346 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:06:16 2024-11-22 04:07:08,347 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:11:21 2024-11-22 04:07:08,347 INFO 3318922:repeater task 3dayplots_os38nb starting 48.4 seconds late 2024-11-22 04:07:14,533 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:07:14,533 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:11:19 2024-11-22 04:07:14,534 INFO 3318922:repeater task daily_netcdf starting 54.6 seconds late 2024-11-22 04:07:15,316 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 04:07:15,316 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:08:20,503 INFO 3318922:repeater task lastensq_wh300 starting 1.5 seconds late 2024-11-22 04:08:23,709 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:08:23,709 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:08:14 2024-11-22 04:08:23,709 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:10:19 2024-11-22 04:08:23,710 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-22 04:08:29,329 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:08:29,329 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:08:14 2024-11-22 04:08:29,330 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:10:19 2024-11-22 04:08:29,330 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2024-11-22 04:08:30,368 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:08:30,368 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:08:14 2024-11-22 04:08:30,368 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:10:19 2024-11-22 04:10:20,660 INFO 3318922:repeater task lastensq_wh300 starting 1.7 seconds late 2024-11-22 04:10:23,928 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 04:10:23,929 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:10:14 2024-11-22 04:10:23,929 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:12:19 2024-11-22 04:10:23,929 INFO 3318922:repeater task quick_wh300 starting 4.9 seconds late 2024-11-22 04:10:29,621 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:10:29,622 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:10:14 2024-11-22 04:10:29,622 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:12:19 2024-11-22 04:10:29,622 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2024-11-22 04:10:30,695 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 04:10:30,695 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:10:14 2024-11-22 04:10:30,696 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:12:19 2024-11-22 04:11:15,818 INFO 3318922:repeater task plot_tsstats_wh300 starting 0.8 seconds late 2024-11-22 04:11:17,269 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:11:17,269 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 04:11:17,269 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:17,270 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.3 seconds late 2024-11-22 04:11:17,430 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 04:11:17,430 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:17,430 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.5 seconds late 2024-11-22 04:11:19,164 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 04:11:19,165 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:19,165 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.2 seconds late 2024-11-22 04:11:20,889 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 04:11:20,891 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:20,892 INFO 3318922:repeater task lastensq_os150nb starting 1.9 seconds late 2024-11-22 04:11:23,555 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 04:11:23,555 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:11:14 2024-11-22 04:11:23,555 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:11:23,556 INFO 3318922:repeater task quick_os150nb starting 4.6 seconds late 2024-11-22 04:11:27,363 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 04:11:27,363 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:11:14 2024-11-22 04:11:27,364 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:11:27,364 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.4 seconds late 2024-11-22 04:11:28,378 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:11:28,378 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:11:14 2024-11-22 04:11:28,378 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:11:28,378 INFO 3318922:repeater task 3dayplots_os150nb starting 8.4 seconds late 2024-11-22 04:11:34,589 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:11:34,589 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:11:34,589 INFO 3318922:repeater task plot_tsstats_os150nb starting 19.6 seconds late 2024-11-22 04:11:35,848 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:11:35,848 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 04:11:35,848 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:35,849 INFO 3318922:repeater task grab_lastpings_os150nb starting 20.9 seconds late 2024-11-22 04:11:36,011 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 04:11:36,012 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:36,012 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.0 seconds late 2024-11-22 04:11:37,728 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 04:11:37,728 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:37,729 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.8 seconds late 2024-11-22 04:11:39,443 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 04:11:39,443 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:39,444 INFO 3318922:repeater task lastensq_os38nb starting 18.4 seconds late 2024-11-22 04:11:42,001 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 04:11:42,002 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:11:16 2024-11-22 04:11:42,002 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:16:21 2024-11-22 04:11:42,002 INFO 3318922:repeater task quick_os38nb starting 21.0 seconds late 2024-11-22 04:11:46,158 INFO 3318922:repeater task quick_os38nb ended after 4.2 seconds 2024-11-22 04:11:46,159 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:11:16 2024-11-22 04:11:46,159 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:16:21 2024-11-22 04:11:46,159 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.2 seconds late 2024-11-22 04:11:47,198 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:11:47,199 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:11:16 2024-11-22 04:11:47,199 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:16:21 2024-11-22 04:11:47,199 INFO 3318922:repeater task 3dayplots_os38nb starting 27.2 seconds late 2024-11-22 04:11:53,350 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:11:53,350 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:11:53,351 INFO 3318922:repeater task plot_tsstats_os38nb starting 38.4 seconds late 2024-11-22 04:11:54,635 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:11:54,635 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 04:11:54,635 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:54,635 INFO 3318922:repeater task grab_lastpings_os38nb starting 39.7 seconds late 2024-11-22 04:11:54,797 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 04:11:54,797 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:54,797 INFO 3318922:repeater task beamdiagnostics_os38nb starting 39.8 seconds late 2024-11-22 04:11:56,493 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 04:11:56,493 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:56,494 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.5 seconds late 2024-11-22 04:11:58,261 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 04:11:58,261 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:26:14 2024-11-22 04:11:58,265 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 43.1 seconds late 2024-11-22 04:12:00,117 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 04:12:00,117 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 04:12:00,117 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 04:26:15 2024-11-22 04:12:00,117 INFO 3318922:repeater task plot_hbins starting 45.0 seconds late 2024-11-22 04:12:04,207 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 04:12:04,208 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 04:12:04,208 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 04:26:15 2024-11-22 04:12:09,216 INFO 3318922:repeater task 3dayplots_wh300 starting 49.2 seconds late 2024-11-22 04:12:15,843 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 04:12:15,843 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:12:20,854 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 04:12:24,022 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:12:24,023 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:12:14 2024-11-22 04:12:24,023 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:14:19 2024-11-22 04:12:24,023 INFO 3318922:repeater task quick_wh300 starting 5.0 seconds late 2024-11-22 04:12:29,432 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 04:12:29,432 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:12:14 2024-11-22 04:12:29,432 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:14:19 2024-11-22 04:12:29,433 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2024-11-22 04:12:30,481 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:12:30,481 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:12:14 2024-11-22 04:12:30,481 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:14:19 2024-11-22 04:14:20,800 INFO 3318922:repeater task lastensq_wh300 starting 1.8 seconds late 2024-11-22 04:14:24,039 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:14:24,039 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:14:14 2024-11-22 04:14:24,039 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:14:24,040 INFO 3318922:repeater task quick_wh300 starting 5.0 seconds late 2024-11-22 04:14:29,575 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 04:14:29,576 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:14:14 2024-11-22 04:14:29,576 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:14:29,576 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2024-11-22 04:14:30,610 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:14:30,610 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:14:14 2024-11-22 04:14:30,610 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:16:19 2024-11-22 04:16:20,910 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.9 seconds late 2024-11-22 04:16:24,111 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 04:16:24,111 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:16:24,112 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 4.1 seconds late 2024-11-22 04:16:27,202 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-22 04:16:27,202 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:16:27,202 INFO 3318922:repeater task noisy_rawtemps_os38 starting 7.2 seconds late 2024-11-22 04:16:30,826 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 04:16:30,826 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:16:30,826 INFO 3318922:repeater task lastensq_wh300 starting 11.8 seconds late 2024-11-22 04:16:34,034 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:16:34,034 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:16:14 2024-11-22 04:16:34,035 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:18:19 2024-11-22 04:16:34,035 INFO 3318922:repeater task quick_wh300 starting 15.0 seconds late 2024-11-22 04:16:39,760 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:16:39,760 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:16:14 2024-11-22 04:16:39,761 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:18:19 2024-11-22 04:16:39,761 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.8 seconds late 2024-11-22 04:16:40,809 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:16:40,810 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:16:14 2024-11-22 04:16:40,810 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:18:19 2024-11-22 04:16:40,810 INFO 3318922:repeater task 3dayplots_wh300 starting 20.8 seconds late 2024-11-22 04:16:47,313 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 04:16:47,313 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:21:19 2024-11-22 04:16:47,314 INFO 3318922:repeater task lastensq_os150nb starting 28.3 seconds late 2024-11-22 04:16:49,982 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 04:16:49,982 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:16:14 2024-11-22 04:16:49,982 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:21:19 2024-11-22 04:16:49,983 INFO 3318922:repeater task quick_os150nb starting 31.0 seconds late 2024-11-22 04:16:53,746 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 04:16:53,746 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:16:14 2024-11-22 04:16:53,746 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:21:19 2024-11-22 04:16:53,747 INFO 3318922:repeater task lastfew_vec_os150nb starting 34.7 seconds late 2024-11-22 04:16:54,779 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:16:54,779 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:16:14 2024-11-22 04:16:54,779 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:21:19 2024-11-22 04:16:54,780 INFO 3318922:repeater task 3dayplots_os150nb starting 34.8 seconds late 2024-11-22 04:17:00,961 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:17:00,962 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:21:19 2024-11-22 04:17:00,963 INFO 3318922:repeater task lastensq_os38nb starting 40.0 seconds late 2024-11-22 04:17:03,528 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 04:17:03,529 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:16:16 2024-11-22 04:17:03,529 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:21:21 2024-11-22 04:17:03,529 INFO 3318922:repeater task quick_os38nb starting 42.5 seconds late 2024-11-22 04:17:07,661 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 04:17:07,661 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:16:16 2024-11-22 04:17:07,662 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:21:21 2024-11-22 04:17:07,662 INFO 3318922:repeater task lastfew_vec_os38nb starting 46.7 seconds late 2024-11-22 04:17:08,676 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:17:08,677 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:16:16 2024-11-22 04:17:08,677 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:21:21 2024-11-22 04:17:08,677 INFO 3318922:repeater task 3dayplots_os38nb starting 48.7 seconds late 2024-11-22 04:17:15,004 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 04:17:15,004 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:21:19 2024-11-22 04:17:15,005 INFO 3318922:repeater task daily_netcdf starting 55.0 seconds late 2024-11-22 04:17:15,832 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 04:17:15,832 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:18:21,017 INFO 3318922:repeater task lastensq_wh300 starting 2.0 seconds late 2024-11-22 04:18:24,472 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-22 04:18:24,473 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:18:14 2024-11-22 04:18:24,473 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:20:19 2024-11-22 04:18:24,473 INFO 3318922:repeater task quick_wh300 starting 5.5 seconds late 2024-11-22 04:18:30,074 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:18:30,075 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:18:14 2024-11-22 04:18:30,075 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:20:19 2024-11-22 04:18:30,075 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2024-11-22 04:18:31,135 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 04:18:31,135 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:18:14 2024-11-22 04:18:31,135 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:20:19 2024-11-22 04:20:21,453 INFO 3318922:repeater task lastensq_wh300 starting 2.5 seconds late 2024-11-22 04:20:24,703 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 04:20:24,703 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:20:14 2024-11-22 04:20:24,704 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:22:19 2024-11-22 04:20:24,704 INFO 3318922:repeater task quick_wh300 starting 5.7 seconds late 2024-11-22 04:20:30,276 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:20:30,277 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:20:14 2024-11-22 04:20:30,277 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:22:19 2024-11-22 04:20:30,277 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2024-11-22 04:20:31,337 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 04:20:31,337 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:20:14 2024-11-22 04:20:31,337 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:22:19 2024-11-22 04:21:21,474 INFO 3318922:repeater task 3dayplots_wh300 starting 1.5 seconds late 2024-11-22 04:21:28,302 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.8 seconds 2024-11-22 04:21:28,302 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:21:28,303 INFO 3318922:repeater task lastensq_os150nb starting 9.3 seconds late 2024-11-22 04:21:30,944 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 04:21:30,945 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:21:14 2024-11-22 04:21:30,945 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:21:30,945 INFO 3318922:repeater task quick_os150nb starting 11.9 seconds late 2024-11-22 04:21:34,630 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 04:21:34,630 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:21:14 2024-11-22 04:21:34,630 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:21:34,630 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.6 seconds late 2024-11-22 04:21:35,644 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:21:35,645 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:21:14 2024-11-22 04:21:35,645 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:21:35,645 INFO 3318922:repeater task 3dayplots_os150nb starting 15.7 seconds late 2024-11-22 04:21:41,852 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:21:41,852 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:21:41,854 INFO 3318922:repeater task lastensq_os38nb starting 20.9 seconds late 2024-11-22 04:21:44,407 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 04:21:44,408 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:21:16 2024-11-22 04:21:44,408 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:26:21 2024-11-22 04:21:44,408 INFO 3318922:repeater task quick_os38nb starting 23.4 seconds late 2024-11-22 04:21:48,406 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 04:21:48,406 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:21:16 2024-11-22 04:21:48,407 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:26:21 2024-11-22 04:21:48,407 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.4 seconds late 2024-11-22 04:21:49,446 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:21:49,446 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:21:16 2024-11-22 04:21:49,446 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:26:21 2024-11-22 04:21:49,447 INFO 3318922:repeater task 3dayplots_os38nb starting 29.5 seconds late 2024-11-22 04:21:55,585 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 04:21:55,585 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:22:20,647 INFO 3318922:repeater task lastensq_wh300 starting 1.6 seconds late 2024-11-22 04:22:23,939 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 04:22:23,939 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:22:14 2024-11-22 04:22:23,939 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:24:19 2024-11-22 04:22:23,939 INFO 3318922:repeater task quick_wh300 starting 4.9 seconds late 2024-11-22 04:22:29,438 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 04:22:29,439 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:22:14 2024-11-22 04:22:29,439 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:24:19 2024-11-22 04:22:29,439 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2024-11-22 04:22:30,482 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:22:30,482 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:22:14 2024-11-22 04:22:30,482 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:24:19 2024-11-22 04:24:20,809 INFO 3318922:repeater task lastensq_wh300 starting 1.8 seconds late 2024-11-22 04:24:24,007 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:24:24,007 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:24:14 2024-11-22 04:24:24,007 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:24:24,008 INFO 3318922:repeater task quick_wh300 starting 5.0 seconds late 2024-11-22 04:24:29,669 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:24:29,669 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:24:14 2024-11-22 04:24:29,669 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:24:29,669 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2024-11-22 04:24:30,701 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:24:30,701 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:24:14 2024-11-22 04:24:30,702 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:26:19 2024-11-22 04:26:16,022 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.0 seconds late 2024-11-22 04:26:17,472 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:26:17,472 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 04:26:17,472 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:17,473 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.5 seconds late 2024-11-22 04:26:17,636 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 04:26:17,636 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:17,637 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.7 seconds late 2024-11-22 04:26:19,339 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 04:26:19,339 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:19,339 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.4 seconds late 2024-11-22 04:26:21,068 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 04:26:21,068 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:21,069 INFO 3318922:repeater task lastensq_os150nb starting 2.1 seconds late 2024-11-22 04:26:23,602 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-22 04:26:23,602 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:26:14 2024-11-22 04:26:23,602 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:31:19 2024-11-22 04:26:23,603 INFO 3318922:repeater task quick_os150nb starting 4.6 seconds late 2024-11-22 04:26:27,507 INFO 3318922:repeater task quick_os150nb ended after 3.9 seconds 2024-11-22 04:26:27,507 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:26:14 2024-11-22 04:26:27,507 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:31:19 2024-11-22 04:26:27,507 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.5 seconds late 2024-11-22 04:26:28,541 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:26:28,541 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:26:14 2024-11-22 04:26:28,541 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:31:19 2024-11-22 04:26:28,541 INFO 3318922:repeater task 3dayplots_os150nb starting 8.6 seconds late 2024-11-22 04:26:34,806 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 04:26:34,806 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:31:19 2024-11-22 04:26:34,806 INFO 3318922:repeater task plot_tsstats_os150nb starting 19.8 seconds late 2024-11-22 04:26:36,066 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:26:36,067 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 04:26:36,067 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:36,067 INFO 3318922:repeater task grab_lastpings_os150nb starting 21.1 seconds late 2024-11-22 04:26:36,226 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 04:26:36,226 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:36,227 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.2 seconds late 2024-11-22 04:26:37,914 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 04:26:37,914 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:37,914 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.9 seconds late 2024-11-22 04:26:39,750 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-22 04:26:39,751 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:39,751 INFO 3318922:repeater task lastensq_os38nb starting 18.8 seconds late 2024-11-22 04:26:42,308 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 04:26:42,308 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:26:16 2024-11-22 04:26:42,308 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:31:21 2024-11-22 04:26:42,309 INFO 3318922:repeater task quick_os38nb starting 21.3 seconds late 2024-11-22 04:26:46,137 INFO 3318922:repeater task quick_os38nb ended after 3.8 seconds 2024-11-22 04:26:46,137 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:26:16 2024-11-22 04:26:46,137 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:31:21 2024-11-22 04:26:46,138 INFO 3318922:repeater task lastfew_vec_os38nb starting 25.1 seconds late 2024-11-22 04:26:47,170 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:26:47,170 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:26:16 2024-11-22 04:26:47,170 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:31:21 2024-11-22 04:26:47,171 INFO 3318922:repeater task 3dayplots_os38nb starting 27.2 seconds late 2024-11-22 04:26:53,366 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:26:53,367 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:31:19 2024-11-22 04:26:53,367 INFO 3318922:repeater task plot_tsstats_os38nb starting 38.4 seconds late 2024-11-22 04:26:54,665 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:26:54,665 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 04:26:54,666 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:54,666 INFO 3318922:repeater task grab_lastpings_os38nb starting 39.7 seconds late 2024-11-22 04:26:54,827 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 04:26:54,828 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:54,828 INFO 3318922:repeater task beamdiagnostics_os38nb starting 39.8 seconds late 2024-11-22 04:26:56,502 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 04:26:56,502 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:56,503 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.5 seconds late 2024-11-22 04:26:58,232 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 04:26:58,232 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:41:14 2024-11-22 04:26:58,232 INFO 3318922:repeater task daily_netcdf starting 38.3 seconds late 2024-11-22 04:26:58,970 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 04:26:58,971 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:26:58,971 INFO 3318922:repeater task ggatime_diagnostics starting 44.0 seconds late 2024-11-22 04:27:06,706 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 04:27:06,706 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:27:06,706 INFO 3318922:repeater task ggatime_diagnostics starting 51.7 seconds late 2024-11-22 04:27:14,240 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 04:27:14,240 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:27:14,241 INFO 3318922:repeater task ggatime_diagnostics starting 59.3 seconds late 2024-11-22 04:27:21,866 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 04:27:21,867 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:27:21,867 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 66.7 seconds late 2024-11-22 04:27:23,765 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 04:27:23,765 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 04:27:23,765 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 04:41:15 2024-11-22 04:27:23,766 INFO 3318922:repeater task plot_hbins starting 68.6 seconds late 2024-11-22 04:27:27,849 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 04:27:27,849 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 04:27:27,849 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 04:41:15 2024-11-22 04:27:27,850 INFO 3318922:repeater task report_generator starting 72.7 seconds late 2024-11-22 04:27:32,278 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-22 04:27:32,278 INFO 3318922:repeater task report_generator ended after 4.4 seconds 2024-11-22 04:27:32,278 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/22 05:26:15 2024-11-22 04:27:37,282 INFO 3318922:repeater task noisy_rawtemps_os150 starting 77.3 seconds late 2024-11-22 04:27:40,456 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 04:27:40,456 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:27:40,456 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 80.5 seconds late 2024-11-22 04:27:43,629 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 04:27:43,629 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:27:43,630 INFO 3318922:repeater task noisy_rawtemps_os38 starting 83.7 seconds late 2024-11-22 04:27:47,243 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 04:27:47,243 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:27:47,243 INFO 3318922:repeater task lastensq_wh300 starting 88.2 seconds late 2024-11-22 04:27:50,433 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:27:50,434 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:26:14 2024-11-22 04:27:50,434 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:28:19 2024-11-22 04:27:50,434 INFO 3318922:repeater task quick_wh300 starting 91.4 seconds late 2024-11-22 04:27:55,871 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 04:27:55,871 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:26:14 2024-11-22 04:27:55,871 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:28:19 2024-11-22 04:27:55,871 INFO 3318922:repeater task lastfew_vec_wh300 starting 96.9 seconds late 2024-11-22 04:27:56,916 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:27:56,917 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:26:14 2024-11-22 04:27:56,917 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:28:19 2024-11-22 04:27:56,917 INFO 3318922:repeater task 3dayplots_wh300 starting 96.9 seconds late 2024-11-22 04:28:03,511 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 04:28:03,511 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:31:19 2024-11-22 04:28:23,555 INFO 3318922:repeater task lastensq_wh300 starting 4.6 seconds late 2024-11-22 04:28:26,725 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:28:26,725 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:28:14 2024-11-22 04:28:26,725 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:30:19 2024-11-22 04:28:26,726 INFO 3318922:repeater task quick_wh300 starting 7.7 seconds late 2024-11-22 04:28:32,154 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 04:28:32,154 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:28:14 2024-11-22 04:28:32,154 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:30:19 2024-11-22 04:28:32,155 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2024-11-22 04:28:33,186 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:28:33,186 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:28:14 2024-11-22 04:28:33,186 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:30:19 2024-11-22 04:30:23,536 INFO 3318922:repeater task lastensq_wh300 starting 4.5 seconds late 2024-11-22 04:30:26,703 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:30:26,703 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:30:14 2024-11-22 04:30:26,703 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:32:19 2024-11-22 04:30:26,703 INFO 3318922:repeater task quick_wh300 starting 7.7 seconds late 2024-11-22 04:30:32,077 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 04:30:32,077 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:30:14 2024-11-22 04:30:32,077 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:32:19 2024-11-22 04:30:32,078 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2024-11-22 04:30:33,114 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:30:33,115 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:30:14 2024-11-22 04:30:33,115 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:32:19 2024-11-22 04:31:23,243 INFO 3318922:repeater task 3dayplots_wh300 starting 3.3 seconds late 2024-11-22 04:31:29,969 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 04:31:29,969 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:31:29,970 INFO 3318922:repeater task lastensq_os150nb starting 11.0 seconds late 2024-11-22 04:31:32,543 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 04:31:32,543 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:31:14 2024-11-22 04:31:32,543 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:31:32,544 INFO 3318922:repeater task quick_os150nb starting 13.5 seconds late 2024-11-22 04:31:36,158 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 04:31:36,158 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:31:14 2024-11-22 04:31:36,158 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:31:36,158 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.2 seconds late 2024-11-22 04:31:37,174 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:31:37,175 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:31:14 2024-11-22 04:31:37,175 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:31:37,175 INFO 3318922:repeater task 3dayplots_os150nb starting 17.2 seconds late 2024-11-22 04:31:43,366 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:31:43,366 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:31:43,368 INFO 3318922:repeater task lastensq_os38nb starting 22.4 seconds late 2024-11-22 04:31:46,051 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 04:31:46,051 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:31:16 2024-11-22 04:31:46,051 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:36:21 2024-11-22 04:31:46,052 INFO 3318922:repeater task quick_os38nb starting 25.1 seconds late 2024-11-22 04:31:50,195 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 04:31:50,195 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:31:16 2024-11-22 04:31:50,196 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:36:21 2024-11-22 04:31:50,196 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.2 seconds late 2024-11-22 04:31:51,226 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:31:51,226 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:31:16 2024-11-22 04:31:51,226 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:36:21 2024-11-22 04:31:51,226 INFO 3318922:repeater task 3dayplots_os38nb starting 31.2 seconds late 2024-11-22 04:31:57,371 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 04:31:57,371 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:32:22,446 INFO 3318922:repeater task lastensq_wh300 starting 3.4 seconds late 2024-11-22 04:32:25,605 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:32:25,605 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:32:14 2024-11-22 04:32:25,605 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:34:19 2024-11-22 04:32:25,606 INFO 3318922:repeater task quick_wh300 starting 6.6 seconds late 2024-11-22 04:32:31,084 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 04:32:31,084 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:32:14 2024-11-22 04:32:31,085 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:34:19 2024-11-22 04:32:31,085 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-22 04:32:32,123 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:32:32,123 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:32:14 2024-11-22 04:32:32,123 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:34:19 2024-11-22 04:34:22,429 INFO 3318922:repeater task lastensq_wh300 starting 3.4 seconds late 2024-11-22 04:34:25,662 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:34:25,662 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:34:14 2024-11-22 04:34:25,662 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:34:25,663 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-22 04:34:31,295 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:34:31,296 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:34:14 2024-11-22 04:34:31,296 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:34:31,296 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-22 04:34:32,341 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:34:32,342 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:34:14 2024-11-22 04:34:32,342 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:36:19 2024-11-22 04:36:22,742 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.8 seconds late 2024-11-22 04:36:25,987 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 04:36:25,987 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:36:25,987 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 6.0 seconds late 2024-11-22 04:36:29,260 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.3 seconds 2024-11-22 04:36:29,260 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:36:29,261 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.3 seconds late 2024-11-22 04:36:32,908 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 04:36:32,908 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:36:32,909 INFO 3318922:repeater task lastensq_wh300 starting 13.9 seconds late 2024-11-22 04:36:36,061 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:36:36,061 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:36:14 2024-11-22 04:36:36,061 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:38:19 2024-11-22 04:36:36,062 INFO 3318922:repeater task quick_wh300 starting 17.1 seconds late 2024-11-22 04:36:41,930 INFO 3318922:repeater task quick_wh300 ended after 5.9 seconds 2024-11-22 04:36:41,930 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:36:14 2024-11-22 04:36:41,930 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:38:19 2024-11-22 04:36:41,931 INFO 3318922:repeater task lastfew_vec_wh300 starting 22.9 seconds late 2024-11-22 04:36:42,969 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:36:42,969 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:36:14 2024-11-22 04:36:42,969 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:38:19 2024-11-22 04:36:42,970 INFO 3318922:repeater task 3dayplots_wh300 starting 23.0 seconds late 2024-11-22 04:36:49,664 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 04:36:49,664 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:41:19 2024-11-22 04:36:49,665 INFO 3318922:repeater task lastensq_os150nb starting 30.7 seconds late 2024-11-22 04:36:52,195 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-22 04:36:52,195 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:36:14 2024-11-22 04:36:52,195 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:41:19 2024-11-22 04:36:52,196 INFO 3318922:repeater task quick_os150nb starting 33.2 seconds late 2024-11-22 04:36:55,886 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 04:36:55,887 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:36:14 2024-11-22 04:36:55,887 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:41:19 2024-11-22 04:36:55,887 INFO 3318922:repeater task lastfew_vec_os150nb starting 36.9 seconds late 2024-11-22 04:36:56,895 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:36:56,895 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:36:14 2024-11-22 04:36:56,895 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:41:19 2024-11-22 04:36:56,896 INFO 3318922:repeater task 3dayplots_os150nb starting 36.9 seconds late 2024-11-22 04:37:03,089 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:37:03,089 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:41:19 2024-11-22 04:37:03,091 INFO 3318922:repeater task lastensq_os38nb starting 42.1 seconds late 2024-11-22 04:37:05,701 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 04:37:05,702 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:36:16 2024-11-22 04:37:05,702 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:41:21 2024-11-22 04:37:05,702 INFO 3318922:repeater task quick_os38nb starting 44.7 seconds late 2024-11-22 04:37:09,676 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 04:37:09,676 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:36:16 2024-11-22 04:37:09,676 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:41:21 2024-11-22 04:37:09,676 INFO 3318922:repeater task lastfew_vec_os38nb starting 48.7 seconds late 2024-11-22 04:37:10,691 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:37:10,692 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:36:16 2024-11-22 04:37:10,692 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:41:21 2024-11-22 04:37:10,692 INFO 3318922:repeater task 3dayplots_os38nb starting 50.7 seconds late 2024-11-22 04:37:16,845 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:37:16,845 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:41:19 2024-11-22 04:37:16,846 INFO 3318922:repeater task daily_netcdf starting 56.9 seconds late 2024-11-22 04:37:17,627 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 04:37:17,627 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:38:22,777 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 04:38:25,927 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:38:25,927 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:38:14 2024-11-22 04:38:25,928 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:40:19 2024-11-22 04:38:25,928 INFO 3318922:repeater task quick_wh300 starting 6.9 seconds late 2024-11-22 04:38:31,393 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 04:38:31,393 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:38:14 2024-11-22 04:38:31,393 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:40:19 2024-11-22 04:38:31,393 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2024-11-22 04:38:32,447 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 04:38:32,447 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:38:14 2024-11-22 04:38:32,447 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:40:19 2024-11-22 04:40:22,722 INFO 3318922:repeater task lastensq_wh300 starting 3.7 seconds late 2024-11-22 04:40:25,924 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:40:25,924 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:40:14 2024-11-22 04:40:25,924 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:42:19 2024-11-22 04:40:25,925 INFO 3318922:repeater task quick_wh300 starting 6.9 seconds late 2024-11-22 04:40:31,718 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 04:40:31,719 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:40:14 2024-11-22 04:40:31,719 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:42:19 2024-11-22 04:40:31,719 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2024-11-22 04:40:32,758 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:40:32,758 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:40:14 2024-11-22 04:40:32,758 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:42:19 2024-11-22 04:41:17,882 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.9 seconds late 2024-11-22 04:41:19,332 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:41:19,332 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 04:41:19,332 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:19,333 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.4 seconds late 2024-11-22 04:41:19,492 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 04:41:19,492 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:19,492 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.5 seconds late 2024-11-22 04:41:21,262 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-22 04:41:21,262 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:21,262 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.3 seconds late 2024-11-22 04:41:22,996 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 04:41:22,996 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:22,997 INFO 3318922:repeater task lastensq_os150nb starting 4.0 seconds late 2024-11-22 04:41:25,770 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-22 04:41:25,770 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:41:14 2024-11-22 04:41:25,770 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:41:25,770 INFO 3318922:repeater task quick_os150nb starting 6.8 seconds late 2024-11-22 04:41:29,714 INFO 3318922:repeater task quick_os150nb ended after 3.9 seconds 2024-11-22 04:41:29,714 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:41:14 2024-11-22 04:41:29,714 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:41:29,715 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.7 seconds late 2024-11-22 04:41:30,752 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:41:30,753 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:41:14 2024-11-22 04:41:30,753 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:41:30,753 INFO 3318922:repeater task 3dayplots_os150nb starting 10.8 seconds late 2024-11-22 04:41:36,930 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:41:36,930 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:41:36,931 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.0 seconds late 2024-11-22 04:41:38,199 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:41:38,201 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 04:41:38,201 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:38,201 INFO 3318922:repeater task grab_lastpings_os150nb starting 23.2 seconds late 2024-11-22 04:41:38,364 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 04:41:38,364 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:38,364 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.4 seconds late 2024-11-22 04:41:40,090 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 04:41:40,090 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:40,090 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.1 seconds late 2024-11-22 04:41:41,805 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 04:41:41,805 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:41,805 INFO 3318922:repeater task lastensq_os38nb starting 20.8 seconds late 2024-11-22 04:41:44,425 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 04:41:44,425 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:41:16 2024-11-22 04:41:44,425 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:46:21 2024-11-22 04:41:44,426 INFO 3318922:repeater task quick_os38nb starting 23.4 seconds late 2024-11-22 04:41:48,365 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 04:41:48,366 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:41:16 2024-11-22 04:41:48,366 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:46:21 2024-11-22 04:41:48,366 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.4 seconds late 2024-11-22 04:41:49,393 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:41:49,393 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:41:16 2024-11-22 04:41:49,393 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:46:21 2024-11-22 04:41:49,394 INFO 3318922:repeater task 3dayplots_os38nb starting 29.4 seconds late 2024-11-22 04:41:55,571 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:41:55,572 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:41:55,572 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.6 seconds late 2024-11-22 04:41:56,865 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:41:56,866 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 04:41:56,866 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:56,866 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.9 seconds late 2024-11-22 04:41:57,028 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 04:41:57,028 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:57,028 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.0 seconds late 2024-11-22 04:41:58,836 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 04:41:58,836 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:41:58,837 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.9 seconds late 2024-11-22 04:42:00,591 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 04:42:00,591 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 04:56:14 2024-11-22 04:42:00,593 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 45.5 seconds late 2024-11-22 04:42:02,471 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 04:42:02,471 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 04:42:02,471 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 04:56:15 2024-11-22 04:42:02,472 INFO 3318922:repeater task plot_hbins starting 47.3 seconds late 2024-11-22 04:42:06,597 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 04:42:06,597 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 04:42:06,597 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 04:56:15 2024-11-22 04:42:11,604 INFO 3318922:repeater task 3dayplots_wh300 starting 51.6 seconds late 2024-11-22 04:42:18,318 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 04:42:18,319 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:42:23,327 INFO 3318922:repeater task lastensq_wh300 starting 4.3 seconds late 2024-11-22 04:42:26,472 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-22 04:42:26,472 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:42:14 2024-11-22 04:42:26,472 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:44:19 2024-11-22 04:42:26,472 INFO 3318922:repeater task quick_wh300 starting 7.5 seconds late 2024-11-22 04:42:31,947 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 04:42:31,947 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:42:14 2024-11-22 04:42:31,947 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:44:19 2024-11-22 04:42:31,948 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2024-11-22 04:42:32,988 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:42:32,988 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:42:14 2024-11-22 04:42:32,988 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:44:19 2024-11-22 04:44:23,313 INFO 3318922:repeater task lastensq_wh300 starting 4.3 seconds late 2024-11-22 04:44:26,588 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 04:44:26,588 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:44:14 2024-11-22 04:44:26,588 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:44:26,588 INFO 3318922:repeater task quick_wh300 starting 7.6 seconds late 2024-11-22 04:44:32,337 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:44:32,337 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:44:14 2024-11-22 04:44:32,337 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:44:32,337 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.3 seconds late 2024-11-22 04:44:33,379 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:44:33,380 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:44:14 2024-11-22 04:44:33,380 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:46:19 2024-11-22 04:46:23,799 INFO 3318922:repeater task noisy_rawtemps_os150 starting 3.8 seconds late 2024-11-22 04:46:26,970 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 04:46:26,970 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:46:26,971 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 7.0 seconds late 2024-11-22 04:46:30,238 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.3 seconds 2024-11-22 04:46:30,238 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:46:30,239 INFO 3318922:repeater task noisy_rawtemps_os38 starting 10.3 seconds late 2024-11-22 04:46:33,873 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 04:46:33,873 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:46:33,873 INFO 3318922:repeater task lastensq_wh300 starting 14.9 seconds late 2024-11-22 04:46:37,043 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:46:37,043 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:46:14 2024-11-22 04:46:37,043 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:48:19 2024-11-22 04:46:37,043 INFO 3318922:repeater task quick_wh300 starting 18.0 seconds late 2024-11-22 04:46:42,710 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:46:42,711 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:46:14 2024-11-22 04:46:42,711 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:48:19 2024-11-22 04:46:42,711 INFO 3318922:repeater task lastfew_vec_wh300 starting 23.7 seconds late 2024-11-22 04:46:43,743 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:46:43,743 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:46:14 2024-11-22 04:46:43,743 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:48:19 2024-11-22 04:46:43,744 INFO 3318922:repeater task 3dayplots_wh300 starting 23.8 seconds late 2024-11-22 04:46:50,400 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 04:46:50,400 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:51:19 2024-11-22 04:46:50,401 INFO 3318922:repeater task lastensq_os150nb starting 31.4 seconds late 2024-11-22 04:46:52,933 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-22 04:46:52,934 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:46:14 2024-11-22 04:46:52,934 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:51:19 2024-11-22 04:46:52,934 INFO 3318922:repeater task quick_os150nb starting 33.9 seconds late 2024-11-22 04:46:56,690 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 04:46:56,690 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:46:14 2024-11-22 04:46:56,691 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:51:19 2024-11-22 04:46:56,691 INFO 3318922:repeater task lastfew_vec_os150nb starting 37.7 seconds late 2024-11-22 04:46:57,700 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:46:57,700 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:46:14 2024-11-22 04:46:57,700 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:51:19 2024-11-22 04:46:57,701 INFO 3318922:repeater task 3dayplots_os150nb starting 37.7 seconds late 2024-11-22 04:47:03,987 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 04:47:03,988 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:51:19 2024-11-22 04:47:03,989 INFO 3318922:repeater task lastensq_os38nb starting 43.0 seconds late 2024-11-22 04:47:06,694 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 04:47:06,694 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:46:16 2024-11-22 04:47:06,694 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:51:21 2024-11-22 04:47:06,695 INFO 3318922:repeater task quick_os38nb starting 45.7 seconds late 2024-11-22 04:47:10,836 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 04:47:10,836 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:46:16 2024-11-22 04:47:10,836 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:51:21 2024-11-22 04:47:10,836 INFO 3318922:repeater task lastfew_vec_os38nb starting 49.8 seconds late 2024-11-22 04:47:11,874 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:47:11,875 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:46:16 2024-11-22 04:47:11,875 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:51:21 2024-11-22 04:47:11,875 INFO 3318922:repeater task 3dayplots_os38nb starting 51.9 seconds late 2024-11-22 04:47:18,164 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 04:47:18,164 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:51:19 2024-11-22 04:47:18,165 INFO 3318922:repeater task daily_netcdf starting 58.2 seconds late 2024-11-22 04:47:19,017 INFO 3318922:repeater task daily_netcdf ended after 0.9 seconds 2024-11-22 04:47:19,017 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:48:19,189 INFO 3318922:repeater task lastensq_wh300 starting 0.2 seconds late 2024-11-22 04:48:22,515 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 04:48:22,515 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:48:14 2024-11-22 04:48:22,515 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:50:19 2024-11-22 04:48:22,515 INFO 3318922:repeater task quick_wh300 starting 3.5 seconds late 2024-11-22 04:48:28,072 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:48:28,072 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:48:14 2024-11-22 04:48:28,073 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:50:19 2024-11-22 04:48:28,073 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.1 seconds late 2024-11-22 04:48:29,108 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:48:29,108 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:48:14 2024-11-22 04:48:29,108 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:50:19 2024-11-22 04:50:19,437 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-22 04:50:22,603 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:50:22,604 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:50:14 2024-11-22 04:50:22,604 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:52:19 2024-11-22 04:50:22,604 INFO 3318922:repeater task quick_wh300 starting 3.6 seconds late 2024-11-22 04:50:28,308 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 04:50:28,308 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:50:14 2024-11-22 04:50:28,308 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:52:19 2024-11-22 04:50:28,308 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2024-11-22 04:50:29,334 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:50:29,334 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:50:14 2024-11-22 04:50:29,334 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:52:19 2024-11-22 04:51:19,485 INFO 3318922:repeater task lastensq_os150nb starting 0.5 seconds late 2024-11-22 04:51:22,089 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 04:51:22,090 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:51:14 2024-11-22 04:51:22,090 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:51:22,090 INFO 3318922:repeater task quick_os150nb starting 3.1 seconds late 2024-11-22 04:51:26,014 INFO 3318922:repeater task quick_os150nb ended after 3.9 seconds 2024-11-22 04:51:26,015 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:51:14 2024-11-22 04:51:26,015 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:51:26,015 INFO 3318922:repeater task lastfew_vec_os150nb starting 7.0 seconds late 2024-11-22 04:51:27,041 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:51:27,042 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:51:14 2024-11-22 04:51:27,042 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:51:27,042 INFO 3318922:repeater task 3dayplots_os150nb starting 7.1 seconds late 2024-11-22 04:51:33,255 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:51:33,256 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:51:33,257 INFO 3318922:repeater task lastensq_os38nb starting 12.3 seconds late 2024-11-22 04:51:35,991 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 04:51:35,991 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:51:16 2024-11-22 04:51:35,991 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 04:56:21 2024-11-22 04:51:35,991 INFO 3318922:repeater task quick_os38nb starting 15.0 seconds late 2024-11-22 04:51:39,908 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 04:51:39,908 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:51:16 2024-11-22 04:51:39,909 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 04:56:21 2024-11-22 04:51:39,909 INFO 3318922:repeater task lastfew_vec_os38nb starting 18.9 seconds late 2024-11-22 04:51:40,932 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:51:40,933 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:51:16 2024-11-22 04:51:40,933 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 04:56:21 2024-11-22 04:51:40,933 INFO 3318922:repeater task 3dayplots_os38nb starting 21.0 seconds late 2024-11-22 04:51:47,099 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:51:47,100 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:51:52,108 INFO 3318922:repeater task 3dayplots_wh300 starting 32.1 seconds late 2024-11-22 04:51:58,718 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 04:51:58,718 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:52:23,776 INFO 3318922:repeater task lastensq_wh300 starting 4.8 seconds late 2024-11-22 04:52:27,078 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 04:52:27,078 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:52:14 2024-11-22 04:52:27,078 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:54:19 2024-11-22 04:52:27,078 INFO 3318922:repeater task quick_wh300 starting 8.1 seconds late 2024-11-22 04:52:32,844 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 04:52:32,844 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:52:14 2024-11-22 04:52:32,845 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:54:19 2024-11-22 04:52:32,845 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.8 seconds late 2024-11-22 04:52:33,872 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:52:33,873 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:52:14 2024-11-22 04:52:33,873 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:54:19 2024-11-22 04:54:19,276 INFO 3318922:repeater task lastensq_wh300 starting 0.3 seconds late 2024-11-22 04:54:22,515 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:54:22,516 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:54:14 2024-11-22 04:54:22,516 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:54:22,516 INFO 3318922:repeater task quick_wh300 starting 3.5 seconds late 2024-11-22 04:54:28,100 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:54:28,100 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:54:14 2024-11-22 04:54:28,100 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:54:28,101 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.1 seconds late 2024-11-22 04:54:29,124 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:54:29,125 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:54:14 2024-11-22 04:54:29,125 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:56:19 2024-11-22 04:56:19,445 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-22 04:56:22,626 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:56:22,627 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:56:14 2024-11-22 04:56:22,627 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 04:58:19 2024-11-22 04:56:22,627 INFO 3318922:repeater task quick_wh300 starting 3.6 seconds late 2024-11-22 04:56:28,183 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:56:28,184 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:56:14 2024-11-22 04:56:28,184 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 04:58:19 2024-11-22 04:56:28,184 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2024-11-22 04:56:29,225 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 04:56:29,225 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:56:14 2024-11-22 04:56:29,225 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 04:58:19 2024-11-22 04:56:29,226 INFO 3318922:repeater task 3dayplots_wh300 starting 9.2 seconds late 2024-11-22 04:56:35,821 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 04:56:35,821 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:01:19 2024-11-22 04:56:35,821 INFO 3318922:repeater task plot_tsstats_wh300 starting 20.8 seconds late 2024-11-22 04:56:37,262 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:56:37,262 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 04:56:37,262 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:37,263 INFO 3318922:repeater task grab_lastpings_wh300 starting 22.3 seconds late 2024-11-22 04:56:37,422 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 04:56:37,422 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:37,423 INFO 3318922:repeater task beamdiagnostics_wh300 starting 22.4 seconds late 2024-11-22 04:56:39,147 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 04:56:39,147 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:39,147 INFO 3318922:repeater task beamdiagnostics_wh300 starting 24.2 seconds late 2024-11-22 04:56:40,928 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-22 04:56:40,928 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:40,929 INFO 3318922:repeater task lastensq_os150nb starting 21.9 seconds late 2024-11-22 04:56:43,424 INFO 3318922:repeater task lastensq_os150nb ended after 2.5 seconds 2024-11-22 04:56:43,424 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:56:14 2024-11-22 04:56:43,424 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:01:19 2024-11-22 04:56:43,425 INFO 3318922:repeater task quick_os150nb starting 24.4 seconds late 2024-11-22 04:56:47,070 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 04:56:47,070 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:56:14 2024-11-22 04:56:47,070 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:01:19 2024-11-22 04:56:47,071 INFO 3318922:repeater task lastfew_vec_os150nb starting 28.1 seconds late 2024-11-22 04:56:48,094 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 04:56:48,095 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 04:56:14 2024-11-22 04:56:48,095 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:01:19 2024-11-22 04:56:48,095 INFO 3318922:repeater task 3dayplots_os150nb starting 28.1 seconds late 2024-11-22 04:56:54,319 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 04:56:54,319 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:01:19 2024-11-22 04:56:54,319 INFO 3318922:repeater task plot_tsstats_os150nb starting 39.3 seconds late 2024-11-22 04:56:55,578 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:56:55,578 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 04:56:55,578 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:55,578 INFO 3318922:repeater task grab_lastpings_os150nb starting 40.6 seconds late 2024-11-22 04:56:55,736 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 04:56:55,736 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:55,737 INFO 3318922:repeater task beamdiagnostics_os150nb starting 40.8 seconds late 2024-11-22 04:56:57,455 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 04:56:57,455 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:57,456 INFO 3318922:repeater task beamdiagnostics_os150nb starting 42.5 seconds late 2024-11-22 04:56:59,157 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 04:56:59,157 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:56:59,158 INFO 3318922:repeater task lastensq_os38nb starting 38.2 seconds late 2024-11-22 04:57:01,690 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 04:57:01,690 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:56:16 2024-11-22 04:57:01,690 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:01:21 2024-11-22 04:57:01,690 INFO 3318922:repeater task quick_os38nb starting 40.7 seconds late 2024-11-22 04:57:05,799 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 04:57:05,799 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:56:16 2024-11-22 04:57:05,799 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:01:21 2024-11-22 04:57:05,800 INFO 3318922:repeater task lastfew_vec_os38nb starting 44.8 seconds late 2024-11-22 04:57:06,840 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 04:57:06,840 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 04:56:16 2024-11-22 04:57:06,840 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:01:21 2024-11-22 04:57:06,841 INFO 3318922:repeater task 3dayplots_os38nb starting 46.9 seconds late 2024-11-22 04:57:12,995 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 04:57:12,996 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:01:19 2024-11-22 04:57:12,996 INFO 3318922:repeater task plot_tsstats_os38nb starting 58.0 seconds late 2024-11-22 04:57:14,285 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 04:57:14,285 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 04:57:14,285 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:57:14,285 INFO 3318922:repeater task grab_lastpings_os38nb starting 59.3 seconds late 2024-11-22 04:57:14,446 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 04:57:14,446 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:57:14,447 INFO 3318922:repeater task beamdiagnostics_os38nb starting 59.5 seconds late 2024-11-22 04:57:16,139 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 04:57:16,139 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:57:16,139 INFO 3318922:repeater task beamdiagnostics_os38nb starting 61.2 seconds late 2024-11-22 04:57:17,902 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 04:57:17,904 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:11:14 2024-11-22 04:57:17,904 INFO 3318922:repeater task daily_netcdf starting 57.9 seconds late 2024-11-22 04:57:18,726 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 04:57:18,727 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 05:06:19 2024-11-22 04:57:18,727 INFO 3318922:repeater task ggatime_diagnostics starting 63.7 seconds late 2024-11-22 04:57:26,647 INFO 3318922:repeater task ggatime_diagnostics ended after 7.9 seconds 2024-11-22 04:57:26,647 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 05:26:14 2024-11-22 04:57:26,647 INFO 3318922:repeater task ggatime_diagnostics starting 71.7 seconds late 2024-11-22 04:57:34,163 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 04:57:34,163 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 05:26:14 2024-11-22 04:57:34,164 INFO 3318922:repeater task ggatime_diagnostics starting 79.2 seconds late 2024-11-22 04:57:41,798 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 04:57:41,798 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 05:26:14 2024-11-22 04:57:41,799 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 86.7 seconds late 2024-11-22 04:57:43,662 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 04:57:43,662 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 04:57:43,662 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 05:11:15 2024-11-22 04:57:43,662 INFO 3318922:repeater task plot_hbins starting 88.5 seconds late 2024-11-22 04:57:47,821 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 04:57:47,822 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-22 04:57:47,822 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 05:11:15 2024-11-22 04:57:52,826 INFO 3318922:repeater task noisy_rawtemps_os150 starting 92.9 seconds late 2024-11-22 04:57:55,999 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 04:57:55,999 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 05:06:19 2024-11-22 04:57:56,000 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 96.0 seconds late 2024-11-22 04:57:59,255 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.3 seconds 2024-11-22 04:57:59,255 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 05:06:19 2024-11-22 04:57:59,256 INFO 3318922:repeater task noisy_rawtemps_os38 starting 99.3 seconds late 2024-11-22 04:58:02,985 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 04:58:02,985 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 05:06:19 2024-11-22 04:58:23,036 INFO 3318922:repeater task lastensq_wh300 starting 4.0 seconds late 2024-11-22 04:58:26,202 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 04:58:26,202 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:58:14 2024-11-22 04:58:26,202 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:00:19 2024-11-22 04:58:26,203 INFO 3318922:repeater task quick_wh300 starting 7.2 seconds late 2024-11-22 04:58:31,791 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 04:58:31,792 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:58:14 2024-11-22 04:58:31,792 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:00:19 2024-11-22 04:58:31,792 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2024-11-22 04:58:32,847 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 04:58:32,847 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 04:58:14 2024-11-22 04:58:32,847 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:00:19 2024-11-22 05:00:23,164 INFO 3318922:repeater task lastensq_wh300 starting 4.2 seconds late 2024-11-22 05:00:26,399 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:00:26,399 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:00:14 2024-11-22 05:00:26,399 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:02:19 2024-11-22 05:00:26,399 INFO 3318922:repeater task quick_wh300 starting 7.4 seconds late 2024-11-22 05:00:31,710 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 05:00:31,711 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:00:14 2024-11-22 05:00:31,711 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:02:19 2024-11-22 05:00:31,711 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2024-11-22 05:00:32,739 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:00:32,739 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:00:14 2024-11-22 05:00:32,739 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:02:19 2024-11-22 05:01:22,878 INFO 3318922:repeater task 3dayplots_wh300 starting 2.9 seconds late 2024-11-22 05:01:29,522 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:01:29,523 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:01:29,524 INFO 3318922:repeater task lastensq_os150nb starting 10.5 seconds late 2024-11-22 05:01:32,230 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 05:01:32,230 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:01:14 2024-11-22 05:01:32,230 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:01:32,231 INFO 3318922:repeater task quick_os150nb starting 13.2 seconds late 2024-11-22 05:01:35,961 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 05:01:35,962 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:01:14 2024-11-22 05:01:35,962 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:01:35,962 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.0 seconds late 2024-11-22 05:01:36,993 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:01:36,994 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:01:14 2024-11-22 05:01:36,994 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:01:36,994 INFO 3318922:repeater task 3dayplots_os150nb starting 17.0 seconds late 2024-11-22 05:01:43,297 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 05:01:43,298 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:01:43,299 INFO 3318922:repeater task lastensq_os38nb starting 22.3 seconds late 2024-11-22 05:01:46,087 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-22 05:01:46,087 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:01:16 2024-11-22 05:01:46,087 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:06:21 2024-11-22 05:01:46,088 INFO 3318922:repeater task quick_os38nb starting 25.1 seconds late 2024-11-22 05:01:50,072 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:01:50,072 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:01:16 2024-11-22 05:01:50,073 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:06:21 2024-11-22 05:01:50,073 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.1 seconds late 2024-11-22 05:01:51,074 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:01:51,074 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:01:16 2024-11-22 05:01:51,074 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:06:21 2024-11-22 05:01:51,075 INFO 3318922:repeater task 3dayplots_os38nb starting 31.1 seconds late 2024-11-22 05:01:57,229 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 05:01:57,229 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:02:22,283 INFO 3318922:repeater task lastensq_wh300 starting 3.3 seconds late 2024-11-22 05:02:25,442 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:02:25,442 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:02:14 2024-11-22 05:02:25,443 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:04:19 2024-11-22 05:02:25,443 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-22 05:02:31,037 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 05:02:31,038 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:02:14 2024-11-22 05:02:31,038 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:04:19 2024-11-22 05:02:31,038 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2024-11-22 05:02:32,082 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:02:32,082 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:02:14 2024-11-22 05:02:32,082 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:04:19 2024-11-22 05:04:22,388 INFO 3318922:repeater task lastensq_wh300 starting 3.4 seconds late 2024-11-22 05:04:25,668 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:04:25,669 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:04:14 2024-11-22 05:04:25,669 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:04:25,669 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-22 05:04:31,257 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 05:04:31,258 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:04:14 2024-11-22 05:04:31,258 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:04:31,258 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-22 05:04:32,285 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:04:32,285 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:04:14 2024-11-22 05:04:32,285 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:06:19 2024-11-22 05:06:22,603 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.6 seconds late 2024-11-22 05:06:25,746 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2024-11-22 05:06:25,746 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:06:25,747 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.8 seconds late 2024-11-22 05:06:28,947 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 05:06:28,947 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:06:28,947 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.0 seconds late 2024-11-22 05:06:32,592 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 05:06:32,592 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:06:32,593 INFO 3318922:repeater task lastensq_wh300 starting 13.6 seconds late 2024-11-22 05:06:35,731 INFO 3318922:repeater task lastensq_wh300 ended after 3.1 seconds 2024-11-22 05:06:35,731 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:06:14 2024-11-22 05:06:35,732 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:08:19 2024-11-22 05:06:35,732 INFO 3318922:repeater task quick_wh300 starting 16.7 seconds late 2024-11-22 05:06:41,486 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 05:06:41,486 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:06:14 2024-11-22 05:06:41,486 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:08:19 2024-11-22 05:06:41,486 INFO 3318922:repeater task lastfew_vec_wh300 starting 22.5 seconds late 2024-11-22 05:06:42,508 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:06:42,508 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:06:14 2024-11-22 05:06:42,509 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:08:19 2024-11-22 05:06:42,509 INFO 3318922:repeater task 3dayplots_wh300 starting 22.5 seconds late 2024-11-22 05:06:49,059 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 05:06:49,059 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:11:19 2024-11-22 05:06:49,060 INFO 3318922:repeater task lastensq_os150nb starting 30.1 seconds late 2024-11-22 05:06:51,685 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 05:06:51,685 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:06:14 2024-11-22 05:06:51,686 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:11:19 2024-11-22 05:06:51,686 INFO 3318922:repeater task quick_os150nb starting 32.7 seconds late 2024-11-22 05:06:55,392 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 05:06:55,392 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:06:14 2024-11-22 05:06:55,392 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:11:19 2024-11-22 05:06:55,393 INFO 3318922:repeater task lastfew_vec_os150nb starting 36.4 seconds late 2024-11-22 05:06:56,413 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:06:56,413 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:06:14 2024-11-22 05:06:56,413 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:11:19 2024-11-22 05:06:56,414 INFO 3318922:repeater task 3dayplots_os150nb starting 36.4 seconds late 2024-11-22 05:07:02,743 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 05:07:02,743 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:11:19 2024-11-22 05:07:02,744 INFO 3318922:repeater task lastensq_os38nb starting 41.7 seconds late 2024-11-22 05:07:05,342 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 05:07:05,342 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:06:16 2024-11-22 05:07:05,342 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:11:21 2024-11-22 05:07:05,342 INFO 3318922:repeater task quick_os38nb starting 44.3 seconds late 2024-11-22 05:07:09,306 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:07:09,307 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:06:16 2024-11-22 05:07:09,307 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:11:21 2024-11-22 05:07:09,307 INFO 3318922:repeater task lastfew_vec_os38nb starting 48.3 seconds late 2024-11-22 05:07:10,337 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:07:10,337 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:06:16 2024-11-22 05:07:10,337 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:11:21 2024-11-22 05:07:10,337 INFO 3318922:repeater task 3dayplots_os38nb starting 50.4 seconds late 2024-11-22 05:07:16,452 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 05:07:16,452 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:11:19 2024-11-22 05:07:16,454 INFO 3318922:repeater task daily_netcdf starting 56.5 seconds late 2024-11-22 05:07:17,222 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 05:07:17,222 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:08:22,376 INFO 3318922:repeater task lastensq_wh300 starting 3.4 seconds late 2024-11-22 05:08:25,676 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:08:25,676 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:08:14 2024-11-22 05:08:25,676 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:10:19 2024-11-22 05:08:25,677 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-22 05:08:31,300 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 05:08:31,301 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:08:14 2024-11-22 05:08:31,301 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:10:19 2024-11-22 05:08:31,301 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-22 05:08:32,350 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:08:32,350 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:08:14 2024-11-22 05:08:32,351 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:10:19 2024-11-22 05:10:22,680 INFO 3318922:repeater task lastensq_wh300 starting 3.7 seconds late 2024-11-22 05:10:25,915 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:10:25,916 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:10:14 2024-11-22 05:10:25,916 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:12:19 2024-11-22 05:10:25,916 INFO 3318922:repeater task quick_wh300 starting 6.9 seconds late 2024-11-22 05:10:31,604 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:10:31,605 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:10:14 2024-11-22 05:10:31,605 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:12:19 2024-11-22 05:10:31,605 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2024-11-22 05:10:32,656 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:10:32,656 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:10:14 2024-11-22 05:10:32,656 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:12:19 2024-11-22 05:11:17,795 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.8 seconds late 2024-11-22 05:11:19,244 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:11:19,244 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 05:11:19,244 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:19,244 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.3 seconds late 2024-11-22 05:11:19,403 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 05:11:19,403 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:19,403 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.4 seconds late 2024-11-22 05:11:21,142 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 05:11:21,142 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:21,142 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.2 seconds late 2024-11-22 05:11:22,907 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-22 05:11:22,907 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:22,907 INFO 3318922:repeater task lastensq_os150nb starting 3.9 seconds late 2024-11-22 05:11:25,601 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 05:11:25,601 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:11:14 2024-11-22 05:11:25,602 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:11:25,602 INFO 3318922:repeater task quick_os150nb starting 6.6 seconds late 2024-11-22 05:11:29,250 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 05:11:29,251 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:11:14 2024-11-22 05:11:29,251 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:11:29,251 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.3 seconds late 2024-11-22 05:11:30,279 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:11:30,279 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:11:14 2024-11-22 05:11:30,279 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:11:30,280 INFO 3318922:repeater task 3dayplots_os150nb starting 10.3 seconds late 2024-11-22 05:11:36,497 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 05:11:36,497 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:11:36,497 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.5 seconds late 2024-11-22 05:11:37,769 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:11:37,769 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 05:11:37,769 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:37,769 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.8 seconds late 2024-11-22 05:11:37,929 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 05:11:37,929 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:37,929 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.0 seconds late 2024-11-22 05:11:39,655 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 05:11:39,655 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:39,656 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.7 seconds late 2024-11-22 05:11:41,380 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 05:11:41,380 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:41,380 INFO 3318922:repeater task lastensq_os38nb starting 20.4 seconds late 2024-11-22 05:11:44,103 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-22 05:11:44,103 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:11:16 2024-11-22 05:11:44,104 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:16:21 2024-11-22 05:11:44,104 INFO 3318922:repeater task quick_os38nb starting 23.1 seconds late 2024-11-22 05:11:48,209 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 05:11:48,209 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:11:16 2024-11-22 05:11:48,210 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:16:21 2024-11-22 05:11:48,210 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.2 seconds late 2024-11-22 05:11:49,220 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:11:49,220 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:11:16 2024-11-22 05:11:49,220 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:16:21 2024-11-22 05:11:49,220 INFO 3318922:repeater task 3dayplots_os38nb starting 29.2 seconds late 2024-11-22 05:11:55,506 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 05:11:55,506 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:11:55,506 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.5 seconds late 2024-11-22 05:11:56,799 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:11:56,799 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 05:11:56,799 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:56,799 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.8 seconds late 2024-11-22 05:11:56,958 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 05:11:56,958 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:56,958 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.0 seconds late 2024-11-22 05:11:58,643 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 05:11:58,643 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:11:58,644 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.7 seconds late 2024-11-22 05:12:00,492 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 05:12:00,492 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:26:14 2024-11-22 05:12:00,493 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 45.4 seconds late 2024-11-22 05:12:02,335 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 05:12:02,335 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 05:12:02,335 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 05:26:15 2024-11-22 05:12:02,336 INFO 3318922:repeater task plot_hbins starting 47.2 seconds late 2024-11-22 05:12:06,571 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 05:12:06,571 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-22 05:12:06,571 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 05:26:15 2024-11-22 05:12:11,580 INFO 3318922:repeater task 3dayplots_wh300 starting 51.6 seconds late 2024-11-22 05:12:18,121 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.5 seconds 2024-11-22 05:12:18,121 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:12:23,131 INFO 3318922:repeater task lastensq_wh300 starting 4.1 seconds late 2024-11-22 05:12:26,375 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:12:26,376 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:12:14 2024-11-22 05:12:26,376 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:14:19 2024-11-22 05:12:26,376 INFO 3318922:repeater task quick_wh300 starting 7.4 seconds late 2024-11-22 05:12:31,912 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 05:12:31,913 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:12:14 2024-11-22 05:12:31,913 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:14:19 2024-11-22 05:12:31,913 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2024-11-22 05:12:32,936 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:12:32,936 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:12:14 2024-11-22 05:12:32,936 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:14:19 2024-11-22 05:14:23,248 INFO 3318922:repeater task lastensq_wh300 starting 4.2 seconds late 2024-11-22 05:14:26,622 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 05:14:26,622 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:14:14 2024-11-22 05:14:26,622 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:14:26,622 INFO 3318922:repeater task quick_wh300 starting 7.6 seconds late 2024-11-22 05:14:31,959 INFO 3318922:repeater task quick_wh300 ended after 5.3 seconds 2024-11-22 05:14:31,959 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:14:14 2024-11-22 05:14:31,960 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:14:31,960 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.0 seconds late 2024-11-22 05:14:33,015 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:14:33,016 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:14:14 2024-11-22 05:14:33,016 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:16:19 2024-11-22 05:16:23,338 INFO 3318922:repeater task noisy_rawtemps_os150 starting 3.4 seconds late 2024-11-22 05:16:26,548 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 05:16:26,548 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:16:26,549 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 6.6 seconds late 2024-11-22 05:16:29,719 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 05:16:29,719 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:16:29,719 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.7 seconds late 2024-11-22 05:16:33,334 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 05:16:33,335 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:16:33,335 INFO 3318922:repeater task lastensq_wh300 starting 14.3 seconds late 2024-11-22 05:16:36,571 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:16:36,571 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:16:14 2024-11-22 05:16:36,571 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:18:19 2024-11-22 05:16:36,572 INFO 3318922:repeater task quick_wh300 starting 17.6 seconds late 2024-11-22 05:16:42,231 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:16:42,232 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:16:14 2024-11-22 05:16:42,232 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:18:19 2024-11-22 05:16:42,232 INFO 3318922:repeater task lastfew_vec_wh300 starting 23.2 seconds late 2024-11-22 05:16:43,286 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:16:43,286 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:16:14 2024-11-22 05:16:43,286 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:18:19 2024-11-22 05:16:43,286 INFO 3318922:repeater task 3dayplots_wh300 starting 23.3 seconds late 2024-11-22 05:16:49,938 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 05:16:49,938 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:21:19 2024-11-22 05:16:49,940 INFO 3318922:repeater task lastensq_os150nb starting 30.9 seconds late 2024-11-22 05:16:52,589 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 05:16:52,589 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:16:14 2024-11-22 05:16:52,589 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:21:19 2024-11-22 05:16:52,589 INFO 3318922:repeater task quick_os150nb starting 33.6 seconds late 2024-11-22 05:16:56,131 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 05:16:56,132 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:16:14 2024-11-22 05:16:56,132 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:21:19 2024-11-22 05:16:56,132 INFO 3318922:repeater task lastfew_vec_os150nb starting 37.1 seconds late 2024-11-22 05:16:57,157 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:16:57,157 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:16:14 2024-11-22 05:16:57,157 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:21:19 2024-11-22 05:16:57,157 INFO 3318922:repeater task 3dayplots_os150nb starting 37.2 seconds late 2024-11-22 05:17:03,480 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 05:17:03,480 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:21:19 2024-11-22 05:17:03,482 INFO 3318922:repeater task lastensq_os38nb starting 42.5 seconds late 2024-11-22 05:17:05,978 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 05:17:05,979 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:16:16 2024-11-22 05:17:05,979 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:21:21 2024-11-22 05:17:05,979 INFO 3318922:repeater task quick_os38nb starting 45.0 seconds late 2024-11-22 05:17:09,902 INFO 3318922:repeater task quick_os38nb ended after 3.9 seconds 2024-11-22 05:17:09,902 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:16:16 2024-11-22 05:17:09,903 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:21:21 2024-11-22 05:17:09,903 INFO 3318922:repeater task lastfew_vec_os38nb starting 48.9 seconds late 2024-11-22 05:17:10,918 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:17:10,919 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:16:16 2024-11-22 05:17:10,919 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:21:21 2024-11-22 05:17:10,919 INFO 3318922:repeater task 3dayplots_os38nb starting 50.9 seconds late 2024-11-22 05:17:17,077 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 05:17:17,077 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:21:19 2024-11-22 05:17:17,078 INFO 3318922:repeater task daily_netcdf starting 57.1 seconds late 2024-11-22 05:17:17,819 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 05:17:17,819 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:18:22,996 INFO 3318922:repeater task lastensq_wh300 starting 4.0 seconds late 2024-11-22 05:18:26,228 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:18:26,228 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:18:14 2024-11-22 05:18:26,229 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:20:19 2024-11-22 05:18:26,229 INFO 3318922:repeater task quick_wh300 starting 7.2 seconds late 2024-11-22 05:18:31,834 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 05:18:31,835 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:18:14 2024-11-22 05:18:31,835 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:20:19 2024-11-22 05:18:31,835 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2024-11-22 05:18:32,872 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:18:32,872 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:18:14 2024-11-22 05:18:32,872 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:20:19 2024-11-22 05:20:23,260 INFO 3318922:repeater task lastensq_wh300 starting 4.3 seconds late 2024-11-22 05:20:26,612 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 05:20:26,612 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:20:14 2024-11-22 05:20:26,613 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:22:19 2024-11-22 05:20:26,613 INFO 3318922:repeater task quick_wh300 starting 7.6 seconds late 2024-11-22 05:20:32,132 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 05:20:32,133 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:20:14 2024-11-22 05:20:32,133 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:22:19 2024-11-22 05:20:32,133 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2024-11-22 05:20:33,189 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:20:33,189 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:20:14 2024-11-22 05:20:33,189 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:22:19 2024-11-22 05:21:23,341 INFO 3318922:repeater task 3dayplots_wh300 starting 3.4 seconds late 2024-11-22 05:21:29,959 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:21:29,959 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:21:29,960 INFO 3318922:repeater task lastensq_os150nb starting 11.0 seconds late 2024-11-22 05:21:32,595 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 05:21:32,595 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:21:14 2024-11-22 05:21:32,595 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:21:32,596 INFO 3318922:repeater task quick_os150nb starting 13.6 seconds late 2024-11-22 05:21:36,280 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 05:21:36,280 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:21:14 2024-11-22 05:21:36,280 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:21:36,281 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.3 seconds late 2024-11-22 05:21:37,296 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:21:37,296 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:21:14 2024-11-22 05:21:37,296 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:21:37,297 INFO 3318922:repeater task 3dayplots_os150nb starting 17.3 seconds late 2024-11-22 05:21:43,538 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 05:21:43,538 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:21:43,539 INFO 3318922:repeater task lastensq_os38nb starting 22.5 seconds late 2024-11-22 05:21:46,004 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 05:21:46,004 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:21:16 2024-11-22 05:21:46,004 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:26:21 2024-11-22 05:21:46,005 INFO 3318922:repeater task quick_os38nb starting 25.0 seconds late 2024-11-22 05:21:50,194 INFO 3318922:repeater task quick_os38nb ended after 4.2 seconds 2024-11-22 05:21:50,194 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:21:16 2024-11-22 05:21:50,194 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:26:21 2024-11-22 05:21:50,195 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.2 seconds late 2024-11-22 05:21:51,220 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:21:51,220 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:21:16 2024-11-22 05:21:51,220 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:26:21 2024-11-22 05:21:51,220 INFO 3318922:repeater task 3dayplots_os38nb starting 31.2 seconds late 2024-11-22 05:21:57,400 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 05:21:57,400 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:22:22,468 INFO 3318922:repeater task lastensq_wh300 starting 3.5 seconds late 2024-11-22 05:22:25,735 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:22:25,735 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:22:14 2024-11-22 05:22:25,735 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:24:19 2024-11-22 05:22:25,736 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-22 05:22:31,276 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 05:22:31,277 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:22:14 2024-11-22 05:22:31,277 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:24:19 2024-11-22 05:22:31,277 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-22 05:22:32,327 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:22:32,327 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:22:14 2024-11-22 05:22:32,327 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:24:19 2024-11-22 05:24:22,648 INFO 3318922:repeater task lastensq_wh300 starting 3.6 seconds late 2024-11-22 05:24:26,065 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 05:24:26,066 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:24:14 2024-11-22 05:24:26,066 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:24:26,066 INFO 3318922:repeater task quick_wh300 starting 7.1 seconds late 2024-11-22 05:24:31,641 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 05:24:31,641 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:24:14 2024-11-22 05:24:31,641 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:24:31,642 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2024-11-22 05:24:32,692 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:24:32,693 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:24:14 2024-11-22 05:24:32,693 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:26:19 2024-11-22 05:26:18,006 INFO 3318922:repeater task plot_tsstats_wh300 starting 3.0 seconds late 2024-11-22 05:26:19,463 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:26:19,463 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 05:26:19,463 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:19,464 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.5 seconds late 2024-11-22 05:26:19,624 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 05:26:19,624 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:19,624 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.6 seconds late 2024-11-22 05:26:21,324 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 05:26:21,324 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:21,324 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.3 seconds late 2024-11-22 05:26:23,038 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 05:26:23,040 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:23,041 INFO 3318922:repeater task lastensq_os150nb starting 4.0 seconds late 2024-11-22 05:26:25,674 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 05:26:25,674 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:26:14 2024-11-22 05:26:25,674 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:31:19 2024-11-22 05:26:25,675 INFO 3318922:repeater task quick_os150nb starting 6.7 seconds late 2024-11-22 05:26:29,484 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 05:26:29,484 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:26:14 2024-11-22 05:26:29,484 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:31:19 2024-11-22 05:26:29,485 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.5 seconds late 2024-11-22 05:26:30,508 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:26:30,509 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:26:14 2024-11-22 05:26:30,509 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:31:19 2024-11-22 05:26:30,509 INFO 3318922:repeater task 3dayplots_os150nb starting 10.5 seconds late 2024-11-22 05:26:36,800 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 05:26:36,800 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:31:19 2024-11-22 05:26:36,801 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.8 seconds late 2024-11-22 05:26:38,052 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:26:38,053 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 05:26:38,053 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:38,053 INFO 3318922:repeater task grab_lastpings_os150nb starting 23.1 seconds late 2024-11-22 05:26:38,213 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 05:26:38,213 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:38,213 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.2 seconds late 2024-11-22 05:26:39,905 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 05:26:39,905 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:39,905 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.9 seconds late 2024-11-22 05:26:41,726 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-22 05:26:41,726 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:41,726 INFO 3318922:repeater task lastensq_os38nb starting 20.7 seconds late 2024-11-22 05:26:44,314 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 05:26:44,315 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:26:16 2024-11-22 05:26:44,315 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:31:21 2024-11-22 05:26:44,315 INFO 3318922:repeater task quick_os38nb starting 23.3 seconds late 2024-11-22 05:26:48,457 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 05:26:48,457 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:26:16 2024-11-22 05:26:48,457 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:31:21 2024-11-22 05:26:48,458 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.5 seconds late 2024-11-22 05:26:49,455 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:26:49,455 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:26:16 2024-11-22 05:26:49,455 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:31:21 2024-11-22 05:26:49,455 INFO 3318922:repeater task 3dayplots_os38nb starting 29.5 seconds late 2024-11-22 05:26:55,568 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 05:26:55,569 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:31:19 2024-11-22 05:26:55,569 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.6 seconds late 2024-11-22 05:26:56,858 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:26:56,858 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 05:26:56,858 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:56,858 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.9 seconds late 2024-11-22 05:26:57,019 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 05:26:57,019 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:57,020 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.0 seconds late 2024-11-22 05:26:58,713 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 05:26:58,713 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:26:58,714 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.7 seconds late 2024-11-22 05:27:00,469 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 05:27:00,469 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:41:14 2024-11-22 05:27:00,470 INFO 3318922:repeater task daily_netcdf starting 40.5 seconds late 2024-11-22 05:27:01,216 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 05:27:01,216 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:27:01,216 INFO 3318922:repeater task ggatime_diagnostics starting 46.2 seconds late 2024-11-22 05:27:08,851 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 05:27:08,851 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:27:08,851 INFO 3318922:repeater task ggatime_diagnostics starting 53.9 seconds late 2024-11-22 05:27:16,346 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-22 05:27:16,346 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:27:16,346 INFO 3318922:repeater task ggatime_diagnostics starting 61.4 seconds late 2024-11-22 05:27:23,922 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 05:27:23,922 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:27:23,923 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 68.8 seconds late 2024-11-22 05:27:25,754 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 05:27:25,754 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 05:27:25,754 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 05:41:15 2024-11-22 05:27:25,754 INFO 3318922:repeater task plot_hbins starting 70.6 seconds late 2024-11-22 05:27:29,871 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 05:27:29,871 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 05:27:29,871 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 05:41:15 2024-11-22 05:27:29,872 INFO 3318922:repeater task report_generator starting 74.7 seconds late 2024-11-22 05:27:34,435 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-22 05:27:34,435 INFO 3318922:repeater task report_generator ended after 4.6 seconds 2024-11-22 05:27:34,435 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/22 06:26:15 2024-11-22 05:27:39,441 INFO 3318922:repeater task noisy_rawtemps_os150 starting 79.5 seconds late 2024-11-22 05:27:42,623 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 05:27:42,623 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:27:42,624 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 82.6 seconds late 2024-11-22 05:27:45,733 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2024-11-22 05:27:45,733 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:27:45,733 INFO 3318922:repeater task noisy_rawtemps_os38 starting 85.8 seconds late 2024-11-22 05:27:49,386 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 05:27:49,386 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:27:49,386 INFO 3318922:repeater task lastensq_wh300 starting 90.4 seconds late 2024-11-22 05:27:52,600 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:27:52,601 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:26:14 2024-11-22 05:27:52,601 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:28:19 2024-11-22 05:27:52,601 INFO 3318922:repeater task quick_wh300 starting 93.6 seconds late 2024-11-22 05:27:58,122 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 05:27:58,122 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:26:14 2024-11-22 05:27:58,122 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:28:19 2024-11-22 05:27:58,123 INFO 3318922:repeater task lastfew_vec_wh300 starting 99.1 seconds late 2024-11-22 05:27:59,158 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:27:59,159 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:26:14 2024-11-22 05:27:59,159 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:28:19 2024-11-22 05:27:59,159 INFO 3318922:repeater task 3dayplots_wh300 starting 99.2 seconds late 2024-11-22 05:28:05,766 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:28:05,766 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:31:19 2024-11-22 05:28:20,809 INFO 3318922:repeater task lastensq_wh300 starting 1.8 seconds late 2024-11-22 05:28:24,014 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:28:24,014 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:28:14 2024-11-22 05:28:24,014 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:30:19 2024-11-22 05:28:24,015 INFO 3318922:repeater task quick_wh300 starting 5.0 seconds late 2024-11-22 05:28:29,564 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 05:28:29,565 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:28:14 2024-11-22 05:28:29,565 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:30:19 2024-11-22 05:28:29,565 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2024-11-22 05:28:30,623 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:28:30,623 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:28:14 2024-11-22 05:28:30,623 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:30:19 2024-11-22 05:30:20,916 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-22 05:30:24,289 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-22 05:30:24,290 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:30:14 2024-11-22 05:30:24,290 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:32:19 2024-11-22 05:30:24,290 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-22 05:30:29,758 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 05:30:29,758 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:30:14 2024-11-22 05:30:29,758 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:32:19 2024-11-22 05:30:29,758 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-22 05:30:30,805 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:30:30,805 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:30:14 2024-11-22 05:30:30,805 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:32:19 2024-11-22 05:31:20,953 INFO 3318922:repeater task 3dayplots_wh300 starting 1.0 seconds late 2024-11-22 05:31:27,574 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:31:27,574 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:31:27,575 INFO 3318922:repeater task lastensq_os150nb starting 8.6 seconds late 2024-11-22 05:31:30,219 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 05:31:30,219 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:31:14 2024-11-22 05:31:30,219 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:31:30,220 INFO 3318922:repeater task quick_os150nb starting 11.2 seconds late 2024-11-22 05:31:33,855 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 05:31:33,855 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:31:14 2024-11-22 05:31:33,855 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:31:33,856 INFO 3318922:repeater task lastfew_vec_os150nb starting 14.9 seconds late 2024-11-22 05:31:34,868 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:31:34,868 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:31:14 2024-11-22 05:31:34,868 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:31:34,869 INFO 3318922:repeater task 3dayplots_os150nb starting 14.9 seconds late 2024-11-22 05:31:41,224 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.4 seconds 2024-11-22 05:31:41,224 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:31:41,225 INFO 3318922:repeater task lastensq_os38nb starting 20.2 seconds late 2024-11-22 05:31:43,758 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 05:31:43,758 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:31:16 2024-11-22 05:31:43,758 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:36:21 2024-11-22 05:31:43,759 INFO 3318922:repeater task quick_os38nb starting 22.8 seconds late 2024-11-22 05:31:47,722 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:31:47,722 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:31:16 2024-11-22 05:31:47,722 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:36:21 2024-11-22 05:31:47,723 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.7 seconds late 2024-11-22 05:31:48,740 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:31:48,741 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:31:16 2024-11-22 05:31:48,741 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:36:21 2024-11-22 05:31:48,741 INFO 3318922:repeater task 3dayplots_os38nb starting 28.8 seconds late 2024-11-22 05:31:54,863 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 05:31:54,863 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:32:19,919 INFO 3318922:repeater task lastensq_wh300 starting 0.9 seconds late 2024-11-22 05:32:23,145 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:32:23,145 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:32:14 2024-11-22 05:32:23,145 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:34:19 2024-11-22 05:32:23,146 INFO 3318922:repeater task quick_wh300 starting 4.1 seconds late 2024-11-22 05:32:28,595 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 05:32:28,595 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:32:14 2024-11-22 05:32:28,595 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:34:19 2024-11-22 05:32:28,596 INFO 3318922:repeater task lastfew_vec_wh300 starting 9.6 seconds late 2024-11-22 05:32:29,654 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:32:29,654 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:32:14 2024-11-22 05:32:29,654 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:34:19 2024-11-22 05:34:19,980 INFO 3318922:repeater task lastensq_wh300 starting 1.0 seconds late 2024-11-22 05:34:23,293 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:34:23,293 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:34:14 2024-11-22 05:34:23,293 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:34:23,293 INFO 3318922:repeater task quick_wh300 starting 4.3 seconds late 2024-11-22 05:34:29,422 INFO 3318922:repeater task quick_wh300 ended after 6.1 seconds 2024-11-22 05:34:29,422 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:34:14 2024-11-22 05:34:29,422 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:34:29,423 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2024-11-22 05:34:30,470 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:34:30,470 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:34:14 2024-11-22 05:34:30,471 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:36:19 2024-11-22 05:36:20,798 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.8 seconds late 2024-11-22 05:36:24,132 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.3 seconds 2024-11-22 05:36:24,132 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:36:24,132 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 4.2 seconds late 2024-11-22 05:36:27,297 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 05:36:27,298 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:36:27,298 INFO 3318922:repeater task noisy_rawtemps_os38 starting 7.3 seconds late 2024-11-22 05:36:30,971 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 05:36:30,972 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:36:30,972 INFO 3318922:repeater task lastensq_wh300 starting 12.0 seconds late 2024-11-22 05:36:34,313 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:36:34,313 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:36:14 2024-11-22 05:36:34,314 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:38:19 2024-11-22 05:36:34,314 INFO 3318922:repeater task quick_wh300 starting 15.3 seconds late 2024-11-22 05:36:40,038 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:36:40,038 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:36:14 2024-11-22 05:36:40,038 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:38:19 2024-11-22 05:36:40,039 INFO 3318922:repeater task lastfew_vec_wh300 starting 21.0 seconds late 2024-11-22 05:36:41,108 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:36:41,108 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:36:14 2024-11-22 05:36:41,108 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:38:19 2024-11-22 05:36:41,109 INFO 3318922:repeater task 3dayplots_wh300 starting 21.1 seconds late 2024-11-22 05:36:47,732 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:36:47,733 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:41:19 2024-11-22 05:36:47,734 INFO 3318922:repeater task lastensq_os150nb starting 28.7 seconds late 2024-11-22 05:36:50,375 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-22 05:36:50,375 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:36:14 2024-11-22 05:36:50,375 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:41:19 2024-11-22 05:36:50,375 INFO 3318922:repeater task quick_os150nb starting 31.4 seconds late 2024-11-22 05:36:54,196 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 05:36:54,196 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:36:14 2024-11-22 05:36:54,196 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:41:19 2024-11-22 05:36:54,196 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.2 seconds late 2024-11-22 05:36:55,212 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:36:55,213 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:36:14 2024-11-22 05:36:55,213 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:41:19 2024-11-22 05:36:55,213 INFO 3318922:repeater task 3dayplots_os150nb starting 35.2 seconds late 2024-11-22 05:37:01,482 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 05:37:01,482 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:41:19 2024-11-22 05:37:01,483 INFO 3318922:repeater task lastensq_os38nb starting 40.5 seconds late 2024-11-22 05:37:04,076 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 05:37:04,076 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:36:16 2024-11-22 05:37:04,076 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:41:21 2024-11-22 05:37:04,077 INFO 3318922:repeater task quick_os38nb starting 43.1 seconds late 2024-11-22 05:37:08,042 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:37:08,042 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:36:16 2024-11-22 05:37:08,042 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:41:21 2024-11-22 05:37:08,042 INFO 3318922:repeater task lastfew_vec_os38nb starting 47.0 seconds late 2024-11-22 05:37:09,069 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:37:09,069 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:36:16 2024-11-22 05:37:09,069 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:41:21 2024-11-22 05:37:09,070 INFO 3318922:repeater task 3dayplots_os38nb starting 49.1 seconds late 2024-11-22 05:37:15,141 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 05:37:15,141 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:41:19 2024-11-22 05:37:15,142 INFO 3318922:repeater task daily_netcdf starting 55.2 seconds late 2024-11-22 05:37:15,971 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 05:37:15,971 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:38:21,165 INFO 3318922:repeater task lastensq_wh300 starting 2.2 seconds late 2024-11-22 05:38:24,447 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:38:24,447 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:38:14 2024-11-22 05:38:24,447 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:40:19 2024-11-22 05:38:24,447 INFO 3318922:repeater task quick_wh300 starting 5.4 seconds late 2024-11-22 05:38:30,001 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 05:38:30,001 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:38:14 2024-11-22 05:38:30,001 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:40:19 2024-11-22 05:38:30,002 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2024-11-22 05:38:31,044 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:38:31,044 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:38:14 2024-11-22 05:38:31,044 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:40:19 2024-11-22 05:40:21,341 INFO 3318922:repeater task lastensq_wh300 starting 2.3 seconds late 2024-11-22 05:40:24,674 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:40:24,674 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:40:14 2024-11-22 05:40:24,675 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:42:19 2024-11-22 05:40:24,675 INFO 3318922:repeater task quick_wh300 starting 5.7 seconds late 2024-11-22 05:40:30,444 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 05:40:30,444 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:40:14 2024-11-22 05:40:30,444 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:42:19 2024-11-22 05:40:30,445 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2024-11-22 05:40:31,485 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:40:31,486 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:40:14 2024-11-22 05:40:31,486 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:42:19 2024-11-22 05:41:16,607 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.6 seconds late 2024-11-22 05:41:18,056 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:41:18,056 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 05:41:18,056 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:18,056 INFO 3318922:repeater task grab_lastpings_wh300 starting 3.1 seconds late 2024-11-22 05:41:18,224 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 05:41:18,224 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:18,224 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.2 seconds late 2024-11-22 05:41:19,973 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 05:41:19,973 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:19,973 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.0 seconds late 2024-11-22 05:41:21,688 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 05:41:21,690 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:21,690 INFO 3318922:repeater task lastensq_os150nb starting 2.7 seconds late 2024-11-22 05:41:24,380 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 05:41:24,380 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:41:14 2024-11-22 05:41:24,380 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:41:24,381 INFO 3318922:repeater task quick_os150nb starting 5.4 seconds late 2024-11-22 05:41:28,080 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 05:41:28,081 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:41:14 2024-11-22 05:41:28,081 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:41:28,081 INFO 3318922:repeater task lastfew_vec_os150nb starting 9.1 seconds late 2024-11-22 05:41:29,114 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:41:29,114 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:41:14 2024-11-22 05:41:29,114 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:41:29,115 INFO 3318922:repeater task 3dayplots_os150nb starting 9.1 seconds late 2024-11-22 05:41:35,449 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 05:41:35,449 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:41:35,450 INFO 3318922:repeater task plot_tsstats_os150nb starting 20.5 seconds late 2024-11-22 05:41:36,711 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:41:36,712 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 05:41:36,712 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:36,712 INFO 3318922:repeater task grab_lastpings_os150nb starting 21.7 seconds late 2024-11-22 05:41:36,873 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 05:41:36,873 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:36,874 INFO 3318922:repeater task beamdiagnostics_os150nb starting 21.9 seconds late 2024-11-22 05:41:38,563 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 05:41:38,563 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:38,563 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.6 seconds late 2024-11-22 05:41:40,300 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 05:41:40,300 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:40,301 INFO 3318922:repeater task lastensq_os38nb starting 19.3 seconds late 2024-11-22 05:41:42,921 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 05:41:42,921 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:41:16 2024-11-22 05:41:42,921 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:46:21 2024-11-22 05:41:42,921 INFO 3318922:repeater task quick_os38nb starting 21.9 seconds late 2024-11-22 05:41:46,951 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:41:46,951 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:41:16 2024-11-22 05:41:46,951 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:46:21 2024-11-22 05:41:46,952 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.0 seconds late 2024-11-22 05:41:47,976 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:41:47,977 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:41:16 2024-11-22 05:41:47,977 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:46:21 2024-11-22 05:41:47,977 INFO 3318922:repeater task 3dayplots_os38nb starting 28.0 seconds late 2024-11-22 05:41:54,071 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 05:41:54,071 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:41:54,072 INFO 3318922:repeater task plot_tsstats_os38nb starting 39.1 seconds late 2024-11-22 05:41:55,365 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:41:55,366 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 05:41:55,366 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:55,366 INFO 3318922:repeater task grab_lastpings_os38nb starting 40.4 seconds late 2024-11-22 05:41:55,528 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 05:41:55,528 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:55,529 INFO 3318922:repeater task beamdiagnostics_os38nb starting 40.5 seconds late 2024-11-22 05:41:57,294 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 05:41:57,294 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:57,295 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.3 seconds late 2024-11-22 05:41:59,072 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 05:41:59,073 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 05:56:14 2024-11-22 05:41:59,074 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 44.0 seconds late 2024-11-22 05:42:01,038 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 05:42:01,039 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 2.0 seconds 2024-11-22 05:42:01,039 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 05:56:15 2024-11-22 05:42:01,039 INFO 3318922:repeater task plot_hbins starting 45.9 seconds late 2024-11-22 05:42:05,178 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 05:42:05,178 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 05:42:05,179 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 05:56:15 2024-11-22 05:42:10,187 INFO 3318922:repeater task 3dayplots_wh300 starting 50.2 seconds late 2024-11-22 05:42:16,839 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 05:42:16,840 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:42:21,848 INFO 3318922:repeater task lastensq_wh300 starting 2.8 seconds late 2024-11-22 05:42:25,093 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:42:25,093 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:42:14 2024-11-22 05:42:25,093 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:44:19 2024-11-22 05:42:25,094 INFO 3318922:repeater task quick_wh300 starting 6.1 seconds late 2024-11-22 05:42:30,759 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:42:30,760 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:42:14 2024-11-22 05:42:30,760 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:44:19 2024-11-22 05:42:30,760 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2024-11-22 05:42:31,805 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:42:31,805 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:42:14 2024-11-22 05:42:31,805 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:44:19 2024-11-22 05:44:22,150 INFO 3318922:repeater task lastensq_wh300 starting 3.2 seconds late 2024-11-22 05:44:25,475 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:44:25,475 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:44:14 2024-11-22 05:44:25,476 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:44:25,476 INFO 3318922:repeater task quick_wh300 starting 6.5 seconds late 2024-11-22 05:44:31,096 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 05:44:31,096 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:44:14 2024-11-22 05:44:31,096 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:44:31,097 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-22 05:44:32,120 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:44:32,120 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:44:14 2024-11-22 05:44:32,120 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:46:19 2024-11-22 05:46:22,438 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.5 seconds late 2024-11-22 05:46:25,608 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 05:46:25,608 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:46:25,609 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.6 seconds late 2024-11-22 05:46:28,808 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 05:46:28,808 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:46:28,809 INFO 3318922:repeater task noisy_rawtemps_os38 starting 8.8 seconds late 2024-11-22 05:46:32,461 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 05:46:32,461 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:46:32,461 INFO 3318922:repeater task lastensq_wh300 starting 13.5 seconds late 2024-11-22 05:46:35,689 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:46:35,689 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:46:14 2024-11-22 05:46:35,689 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:48:19 2024-11-22 05:46:35,689 INFO 3318922:repeater task quick_wh300 starting 16.7 seconds late 2024-11-22 05:46:41,417 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:46:41,418 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:46:14 2024-11-22 05:46:41,418 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:48:19 2024-11-22 05:46:41,418 INFO 3318922:repeater task lastfew_vec_wh300 starting 22.4 seconds late 2024-11-22 05:46:42,473 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:46:42,473 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:46:14 2024-11-22 05:46:42,473 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:48:19 2024-11-22 05:46:42,474 INFO 3318922:repeater task 3dayplots_wh300 starting 22.5 seconds late 2024-11-22 05:46:49,066 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:46:49,067 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:51:19 2024-11-22 05:46:49,068 INFO 3318922:repeater task lastensq_os150nb starting 30.1 seconds late 2024-11-22 05:46:51,764 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 05:46:51,764 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:46:14 2024-11-22 05:46:51,764 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:51:19 2024-11-22 05:46:51,765 INFO 3318922:repeater task quick_os150nb starting 32.8 seconds late 2024-11-22 05:46:55,553 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 05:46:55,554 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:46:14 2024-11-22 05:46:55,554 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:51:19 2024-11-22 05:46:55,554 INFO 3318922:repeater task lastfew_vec_os150nb starting 36.6 seconds late 2024-11-22 05:46:56,568 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:46:56,568 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:46:14 2024-11-22 05:46:56,568 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:51:19 2024-11-22 05:46:56,568 INFO 3318922:repeater task 3dayplots_os150nb starting 36.6 seconds late 2024-11-22 05:47:02,815 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 05:47:02,815 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:51:19 2024-11-22 05:47:02,817 INFO 3318922:repeater task lastensq_os38nb starting 41.8 seconds late 2024-11-22 05:47:05,252 INFO 3318922:repeater task lastensq_os38nb ended after 2.4 seconds 2024-11-22 05:47:05,252 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:46:16 2024-11-22 05:47:05,252 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:51:21 2024-11-22 05:47:05,253 INFO 3318922:repeater task quick_os38nb starting 44.3 seconds late 2024-11-22 05:47:09,249 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:47:09,249 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:46:16 2024-11-22 05:47:09,249 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:51:21 2024-11-22 05:47:09,250 INFO 3318922:repeater task lastfew_vec_os38nb starting 48.3 seconds late 2024-11-22 05:47:10,267 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:47:10,267 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:46:16 2024-11-22 05:47:10,267 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:51:21 2024-11-22 05:47:10,268 INFO 3318922:repeater task 3dayplots_os38nb starting 50.3 seconds late 2024-11-22 05:47:16,454 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 05:47:16,454 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:51:19 2024-11-22 05:47:16,455 INFO 3318922:repeater task daily_netcdf starting 56.5 seconds late 2024-11-22 05:47:17,198 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 05:47:17,198 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:48:22,389 INFO 3318922:repeater task lastensq_wh300 starting 3.4 seconds late 2024-11-22 05:48:25,658 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:48:25,658 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:48:14 2024-11-22 05:48:25,658 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:50:19 2024-11-22 05:48:25,659 INFO 3318922:repeater task quick_wh300 starting 6.7 seconds late 2024-11-22 05:48:31,445 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 05:48:31,445 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:48:14 2024-11-22 05:48:31,445 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:50:19 2024-11-22 05:48:31,446 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2024-11-22 05:48:32,473 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:48:32,473 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:48:14 2024-11-22 05:48:32,473 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:50:19 2024-11-22 05:50:22,793 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 05:50:26,028 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:50:26,028 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:50:14 2024-11-22 05:50:26,028 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:52:19 2024-11-22 05:50:26,029 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-22 05:50:31,750 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:50:31,750 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:50:14 2024-11-22 05:50:31,750 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:52:19 2024-11-22 05:50:31,751 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2024-11-22 05:50:32,779 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:50:32,779 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:50:14 2024-11-22 05:50:32,780 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:52:19 2024-11-22 05:51:22,918 INFO 3318922:repeater task 3dayplots_wh300 starting 2.9 seconds late 2024-11-22 05:51:29,531 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:51:29,531 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:51:29,532 INFO 3318922:repeater task lastensq_os150nb starting 10.5 seconds late 2024-11-22 05:51:32,191 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 05:51:32,191 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:51:14 2024-11-22 05:51:32,191 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:51:32,192 INFO 3318922:repeater task quick_os150nb starting 13.2 seconds late 2024-11-22 05:51:35,936 INFO 3318922:repeater task quick_os150nb ended after 3.7 seconds 2024-11-22 05:51:35,936 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:51:14 2024-11-22 05:51:35,937 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:51:35,937 INFO 3318922:repeater task lastfew_vec_os150nb starting 16.9 seconds late 2024-11-22 05:51:36,954 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:51:36,955 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:51:14 2024-11-22 05:51:36,955 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:51:36,955 INFO 3318922:repeater task 3dayplots_os150nb starting 17.0 seconds late 2024-11-22 05:51:43,263 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 05:51:43,263 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:51:43,265 INFO 3318922:repeater task lastensq_os38nb starting 22.3 seconds late 2024-11-22 05:51:46,091 INFO 3318922:repeater task lastensq_os38nb ended after 2.8 seconds 2024-11-22 05:51:46,092 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:51:16 2024-11-22 05:51:46,092 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 05:56:21 2024-11-22 05:51:46,092 INFO 3318922:repeater task quick_os38nb starting 25.1 seconds late 2024-11-22 05:51:50,095 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:51:50,096 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:51:16 2024-11-22 05:51:50,096 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 05:56:21 2024-11-22 05:51:50,096 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.1 seconds late 2024-11-22 05:51:51,115 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:51:51,115 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:51:16 2024-11-22 05:51:51,116 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 05:56:21 2024-11-22 05:51:51,116 INFO 3318922:repeater task 3dayplots_os38nb starting 31.1 seconds late 2024-11-22 05:51:57,268 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 05:51:57,268 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:52:22,334 INFO 3318922:repeater task lastensq_wh300 starting 3.3 seconds late 2024-11-22 05:52:25,569 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:52:25,569 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:52:14 2024-11-22 05:52:25,569 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:54:19 2024-11-22 05:52:25,570 INFO 3318922:repeater task quick_wh300 starting 6.6 seconds late 2024-11-22 05:52:31,296 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:52:31,296 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:52:14 2024-11-22 05:52:31,296 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:54:19 2024-11-22 05:52:31,296 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-22 05:52:32,321 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:52:32,321 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:52:14 2024-11-22 05:52:32,321 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:54:19 2024-11-22 05:54:22,633 INFO 3318922:repeater task lastensq_wh300 starting 3.6 seconds late 2024-11-22 05:54:25,965 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:54:25,966 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:54:14 2024-11-22 05:54:25,966 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:54:25,966 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-22 05:54:31,361 INFO 3318922:repeater task quick_wh300 ended after 5.4 seconds 2024-11-22 05:54:31,361 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:54:14 2024-11-22 05:54:31,361 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:54:31,362 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2024-11-22 05:54:32,412 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:54:32,413 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:54:14 2024-11-22 05:54:32,413 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:56:19 2024-11-22 05:56:17,706 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.7 seconds late 2024-11-22 05:56:19,152 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:56:19,153 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.4 seconds 2024-11-22 05:56:19,153 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:19,153 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.2 seconds late 2024-11-22 05:56:19,310 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 05:56:19,310 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:19,310 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.3 seconds late 2024-11-22 05:56:21,045 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 05:56:21,045 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:21,045 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.1 seconds late 2024-11-22 05:56:22,751 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 05:56:22,753 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:22,753 INFO 3318922:repeater task lastensq_os150nb starting 3.8 seconds late 2024-11-22 05:56:25,479 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 05:56:25,479 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:56:14 2024-11-22 05:56:25,479 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 06:01:19 2024-11-22 05:56:25,480 INFO 3318922:repeater task quick_os150nb starting 6.5 seconds late 2024-11-22 05:56:29,032 INFO 3318922:repeater task quick_os150nb ended after 3.6 seconds 2024-11-22 05:56:29,032 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:56:14 2024-11-22 05:56:29,032 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 06:01:19 2024-11-22 05:56:29,033 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.0 seconds late 2024-11-22 05:56:30,044 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 05:56:30,044 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 05:56:14 2024-11-22 05:56:30,044 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 06:01:19 2024-11-22 05:56:30,044 INFO 3318922:repeater task 3dayplots_os150nb starting 10.1 seconds late 2024-11-22 05:56:36,273 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 05:56:36,273 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 06:01:19 2024-11-22 05:56:36,273 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.3 seconds late 2024-11-22 05:56:37,539 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:56:37,539 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 05:56:37,539 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:37,539 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.6 seconds late 2024-11-22 05:56:37,702 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 05:56:37,702 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:37,703 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.7 seconds late 2024-11-22 05:56:39,397 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 05:56:39,398 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:39,398 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.4 seconds late 2024-11-22 05:56:41,133 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 05:56:41,133 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:41,134 INFO 3318922:repeater task lastensq_os38nb starting 20.1 seconds late 2024-11-22 05:56:43,708 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 05:56:43,709 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:56:16 2024-11-22 05:56:43,709 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 06:01:21 2024-11-22 05:56:43,709 INFO 3318922:repeater task quick_os38nb starting 22.7 seconds late 2024-11-22 05:56:47,713 INFO 3318922:repeater task quick_os38nb ended after 4.0 seconds 2024-11-22 05:56:47,713 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:56:16 2024-11-22 05:56:47,713 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 06:01:21 2024-11-22 05:56:47,714 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.7 seconds late 2024-11-22 05:56:48,732 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 05:56:48,732 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 05:56:16 2024-11-22 05:56:48,732 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 06:01:21 2024-11-22 05:56:48,732 INFO 3318922:repeater task 3dayplots_os38nb starting 28.8 seconds late 2024-11-22 05:56:54,793 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.1 seconds 2024-11-22 05:56:54,793 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 06:01:19 2024-11-22 05:56:54,794 INFO 3318922:repeater task plot_tsstats_os38nb starting 39.8 seconds late 2024-11-22 05:56:56,081 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 05:56:56,082 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 05:56:56,082 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:56,082 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.1 seconds late 2024-11-22 05:56:56,244 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 05:56:56,244 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:56,244 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.3 seconds late 2024-11-22 05:56:57,968 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 05:56:57,968 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:57,969 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.0 seconds late 2024-11-22 05:56:59,683 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 05:56:59,683 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 06:11:14 2024-11-22 05:56:59,683 INFO 3318922:repeater task daily_netcdf starting 39.7 seconds late 2024-11-22 05:57:00,461 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 05:57:00,461 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 06:06:19 2024-11-22 05:57:00,462 INFO 3318922:repeater task ggatime_diagnostics starting 45.5 seconds late 2024-11-22 05:57:08,178 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 05:57:08,178 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 06:26:14 2024-11-22 05:57:08,178 INFO 3318922:repeater task ggatime_diagnostics starting 53.2 seconds late 2024-11-22 05:57:15,804 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-22 05:57:15,804 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 06:26:14 2024-11-22 05:57:15,806 INFO 3318922:repeater task ggatime_diagnostics starting 60.8 seconds late 2024-11-22 05:57:23,498 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-22 05:57:23,498 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/22 06:26:14 2024-11-22 05:57:23,498 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 68.4 seconds late 2024-11-22 05:57:25,311 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 05:57:25,311 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-22 05:57:25,311 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 06:11:15 2024-11-22 05:57:25,311 INFO 3318922:repeater task plot_hbins starting 70.2 seconds late 2024-11-22 05:57:29,482 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 05:57:29,482 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-22 05:57:29,482 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 06:11:15 2024-11-22 05:57:34,486 INFO 3318922:repeater task noisy_rawtemps_os150 starting 74.5 seconds late 2024-11-22 05:57:37,685 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.2 seconds 2024-11-22 05:57:37,685 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 06:06:19 2024-11-22 05:57:37,686 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 77.7 seconds late 2024-11-22 05:57:40,837 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 05:57:40,837 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 06:06:19 2024-11-22 05:57:40,837 INFO 3318922:repeater task noisy_rawtemps_os38 starting 80.9 seconds late 2024-11-22 05:57:44,533 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 05:57:44,533 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 06:06:19 2024-11-22 05:57:44,534 INFO 3318922:repeater task lastensq_wh300 starting 85.5 seconds late 2024-11-22 05:57:47,732 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 05:57:47,732 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:56:14 2024-11-22 05:57:47,732 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 05:58:19 2024-11-22 05:57:47,732 INFO 3318922:repeater task quick_wh300 starting 88.7 seconds late 2024-11-22 05:57:53,457 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 05:57:53,458 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:56:14 2024-11-22 05:57:53,458 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 05:58:19 2024-11-22 05:57:53,458 INFO 3318922:repeater task lastfew_vec_wh300 starting 94.5 seconds late 2024-11-22 05:57:54,509 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 05:57:54,509 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:56:14 2024-11-22 05:57:54,510 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 05:58:19 2024-11-22 05:57:54,510 INFO 3318922:repeater task 3dayplots_wh300 starting 94.5 seconds late 2024-11-22 05:58:01,142 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 05:58:01,142 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 06:01:19 2024-11-22 05:58:21,198 INFO 3318922:repeater task lastensq_wh300 starting 2.2 seconds late 2024-11-22 05:58:24,477 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 05:58:24,477 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:58:14 2024-11-22 05:58:24,477 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:00:19 2024-11-22 05:58:24,478 INFO 3318922:repeater task quick_wh300 starting 5.5 seconds late 2024-11-22 05:58:30,282 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 05:58:30,282 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:58:14 2024-11-22 05:58:30,282 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:00:19 2024-11-22 05:58:30,283 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2024-11-22 05:58:31,322 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 05:58:31,322 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 05:58:14 2024-11-22 05:58:31,322 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:00:19 2024-11-22 06:00:21,647 INFO 3318922:repeater task lastensq_wh300 starting 2.6 seconds late 2024-11-22 06:00:25,098 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-22 06:00:25,098 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:00:14 2024-11-22 06:00:25,098 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:02:19 2024-11-22 06:00:25,098 INFO 3318922:repeater task quick_wh300 starting 6.1 seconds late 2024-11-22 06:00:30,952 INFO 3318922:repeater task quick_wh300 ended after 5.9 seconds 2024-11-22 06:00:30,952 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:00:14 2024-11-22 06:00:30,952 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:02:19 2024-11-22 06:00:30,953 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2024-11-22 06:00:32,002 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:00:32,002 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:00:14 2024-11-22 06:00:32,002 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:02:19 2024-11-22 06:01:22,153 INFO 3318922:repeater task 3dayplots_wh300 starting 2.2 seconds late 2024-11-22 06:01:28,796 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 06:01:28,797 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:01:28,798 INFO 3318922:repeater task lastensq_os150nb starting 9.8 seconds late 2024-11-22 06:01:31,482 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 06:01:31,483 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:01:14 2024-11-22 06:01:31,483 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:01:31,483 INFO 3318922:repeater task quick_os150nb starting 12.5 seconds late 2024-11-22 06:01:35,273 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 06:01:35,274 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:01:14 2024-11-22 06:01:35,274 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:01:35,274 INFO 3318922:repeater task lastfew_vec_os150nb starting 16.3 seconds late 2024-11-22 06:01:36,288 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 06:01:36,288 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:01:14 2024-11-22 06:01:36,289 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:01:36,289 INFO 3318922:repeater task 3dayplots_os150nb starting 16.3 seconds late 2024-11-22 06:01:42,580 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.3 seconds 2024-11-22 06:01:42,580 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:01:42,581 INFO 3318922:repeater task lastensq_os38nb starting 21.6 seconds late 2024-11-22 06:01:45,099 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 06:01:45,099 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:01:16 2024-11-22 06:01:45,099 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 06:06:21 2024-11-22 06:01:45,100 INFO 3318922:repeater task quick_os38nb starting 24.1 seconds late 2024-11-22 06:01:48,833 INFO 3318922:repeater task quick_os38nb ended after 3.7 seconds 2024-11-22 06:01:48,833 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:01:16 2024-11-22 06:01:48,833 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 06:06:21 2024-11-22 06:01:48,834 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.8 seconds late 2024-11-22 06:01:49,868 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 06:01:49,869 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:01:16 2024-11-22 06:01:49,869 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 06:06:21 2024-11-22 06:01:49,869 INFO 3318922:repeater task 3dayplots_os38nb starting 29.9 seconds late 2024-11-22 06:01:56,043 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 06:01:56,043 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:02:21,113 INFO 3318922:repeater task lastensq_wh300 starting 2.1 seconds late 2024-11-22 06:02:24,291 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 06:02:24,291 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:02:14 2024-11-22 06:02:24,291 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:04:19 2024-11-22 06:02:24,292 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-22 06:02:29,893 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 06:02:29,894 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:02:14 2024-11-22 06:02:29,894 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:04:19 2024-11-22 06:02:29,894 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2024-11-22 06:02:30,926 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:02:30,927 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:02:14 2024-11-22 06:02:30,927 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:04:19 2024-11-22 06:04:21,236 INFO 3318922:repeater task lastensq_wh300 starting 2.2 seconds late 2024-11-22 06:04:24,472 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 06:04:24,473 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:04:14 2024-11-22 06:04:24,473 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:04:24,473 INFO 3318922:repeater task quick_wh300 starting 5.5 seconds late 2024-11-22 06:04:30,051 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 06:04:30,051 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:04:14 2024-11-22 06:04:30,051 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:04:30,052 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2024-11-22 06:04:31,078 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:04:31,078 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:04:14 2024-11-22 06:04:31,078 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:06:19 2024-11-22 06:06:21,394 INFO 3318922:repeater task noisy_rawtemps_os150 starting 1.4 seconds late 2024-11-22 06:06:24,688 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.3 seconds 2024-11-22 06:06:24,688 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:06:24,688 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 4.7 seconds late 2024-11-22 06:06:27,899 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 06:06:27,899 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:06:27,899 INFO 3318922:repeater task noisy_rawtemps_os38 starting 7.9 seconds late 2024-11-22 06:06:31,544 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.6 seconds 2024-11-22 06:06:31,544 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:06:31,544 INFO 3318922:repeater task lastensq_wh300 starting 12.5 seconds late 2024-11-22 06:06:34,826 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 06:06:34,827 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:06:14 2024-11-22 06:06:34,827 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:08:19 2024-11-22 06:06:34,827 INFO 3318922:repeater task quick_wh300 starting 15.8 seconds late 2024-11-22 06:06:40,563 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 06:06:40,563 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:06:14 2024-11-22 06:06:40,564 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:08:19 2024-11-22 06:06:40,564 INFO 3318922:repeater task lastfew_vec_wh300 starting 21.6 seconds late 2024-11-22 06:06:41,617 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 06:06:41,617 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:06:14 2024-11-22 06:06:41,617 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:08:19 2024-11-22 06:06:41,618 INFO 3318922:repeater task 3dayplots_wh300 starting 21.6 seconds late 2024-11-22 06:06:48,372 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.8 seconds 2024-11-22 06:06:48,372 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 06:11:19 2024-11-22 06:06:48,373 INFO 3318922:repeater task lastensq_os150nb starting 29.4 seconds late 2024-11-22 06:06:51,048 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 06:06:51,048 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:06:14 2024-11-22 06:06:51,049 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 06:11:19 2024-11-22 06:06:51,049 INFO 3318922:repeater task quick_os150nb starting 32.0 seconds late 2024-11-22 06:06:54,590 INFO 3318922:repeater task quick_os150nb ended after 3.5 seconds 2024-11-22 06:06:54,590 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:06:14 2024-11-22 06:06:54,590 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 06:11:19 2024-11-22 06:06:54,591 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.6 seconds late 2024-11-22 06:06:55,610 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 06:06:55,610 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:06:14 2024-11-22 06:06:55,610 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 06:11:19 2024-11-22 06:06:55,610 INFO 3318922:repeater task 3dayplots_os150nb starting 35.6 seconds late 2024-11-22 06:07:01,838 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 06:07:01,838 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 06:11:19 2024-11-22 06:07:01,839 INFO 3318922:repeater task lastensq_os38nb starting 40.8 seconds late 2024-11-22 06:07:04,354 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 06:07:04,354 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:06:16 2024-11-22 06:07:04,354 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 06:11:21 2024-11-22 06:07:04,354 INFO 3318922:repeater task quick_os38nb starting 43.4 seconds late 2024-11-22 06:07:08,494 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 06:07:08,494 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:06:16 2024-11-22 06:07:08,494 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 06:11:21 2024-11-22 06:07:08,495 INFO 3318922:repeater task lastfew_vec_os38nb starting 47.5 seconds late 2024-11-22 06:07:09,528 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 06:07:09,528 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:06:16 2024-11-22 06:07:09,528 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 06:11:21 2024-11-22 06:07:09,529 INFO 3318922:repeater task 3dayplots_os38nb starting 49.6 seconds late 2024-11-22 06:07:15,861 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 06:07:15,861 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 06:11:19 2024-11-22 06:07:15,862 INFO 3318922:repeater task daily_netcdf starting 55.9 seconds late 2024-11-22 06:07:16,675 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-22 06:07:16,675 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:08:21,867 INFO 3318922:repeater task lastensq_wh300 starting 2.9 seconds late 2024-11-22 06:08:25,130 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 06:08:25,130 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:08:14 2024-11-22 06:08:25,130 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:10:19 2024-11-22 06:08:25,131 INFO 3318922:repeater task quick_wh300 starting 6.1 seconds late 2024-11-22 06:08:30,862 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 06:08:30,862 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:08:14 2024-11-22 06:08:30,862 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:10:19 2024-11-22 06:08:30,863 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2024-11-22 06:08:31,919 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 06:08:31,920 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:08:14 2024-11-22 06:08:31,920 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:10:19 2024-11-22 06:10:22,244 INFO 3318922:repeater task lastensq_wh300 starting 3.2 seconds late 2024-11-22 06:10:25,416 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 06:10:25,417 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:10:14 2024-11-22 06:10:25,417 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:12:19 2024-11-22 06:10:25,417 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-22 06:10:31,091 INFO 3318922:repeater task quick_wh300 ended after 5.7 seconds 2024-11-22 06:10:31,091 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:10:14 2024-11-22 06:10:31,091 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:12:19 2024-11-22 06:10:31,092 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-22 06:10:32,119 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:10:32,120 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:10:14 2024-11-22 06:10:32,120 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:12:19 2024-11-22 06:11:17,230 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.3 seconds late 2024-11-22 06:11:18,685 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 06:11:18,686 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-22 06:11:18,686 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:18,686 INFO 3318922:repeater task grab_lastpings_wh300 starting 3.7 seconds late 2024-11-22 06:11:18,846 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-22 06:11:18,847 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:18,847 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.9 seconds late 2024-11-22 06:11:20,586 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 06:11:20,587 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:20,587 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.6 seconds late 2024-11-22 06:11:22,311 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-22 06:11:22,312 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:22,312 INFO 3318922:repeater task lastensq_os150nb starting 3.3 seconds late 2024-11-22 06:11:25,021 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 06:11:25,021 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:11:14 2024-11-22 06:11:25,022 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:11:25,022 INFO 3318922:repeater task quick_os150nb starting 6.0 seconds late 2024-11-22 06:11:28,836 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 06:11:28,836 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:11:14 2024-11-22 06:11:28,836 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:11:28,836 INFO 3318922:repeater task lastfew_vec_os150nb starting 9.8 seconds late 2024-11-22 06:11:29,891 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-22 06:11:29,891 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:11:14 2024-11-22 06:11:29,892 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:11:29,892 INFO 3318922:repeater task 3dayplots_os150nb starting 9.9 seconds late 2024-11-22 06:11:36,084 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 06:11:36,084 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:11:36,084 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.1 seconds late 2024-11-22 06:11:37,343 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 06:11:37,344 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-22 06:11:37,344 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:37,344 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.4 seconds late 2024-11-22 06:11:37,505 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-22 06:11:37,505 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:37,506 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.5 seconds late 2024-11-22 06:11:39,210 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 06:11:39,210 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:39,210 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.2 seconds late 2024-11-22 06:11:40,924 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-22 06:11:40,924 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:40,924 INFO 3318922:repeater task lastensq_os38nb starting 19.9 seconds late 2024-11-22 06:11:43,393 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-22 06:11:43,393 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:11:16 2024-11-22 06:11:43,393 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 06:16:21 2024-11-22 06:11:43,394 INFO 3318922:repeater task quick_os38nb starting 22.4 seconds late 2024-11-22 06:11:47,580 INFO 3318922:repeater task quick_os38nb ended after 4.2 seconds 2024-11-22 06:11:47,580 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:11:16 2024-11-22 06:11:47,580 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 06:16:21 2024-11-22 06:11:47,580 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.6 seconds late 2024-11-22 06:11:48,596 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 06:11:48,596 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:11:16 2024-11-22 06:11:48,596 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 06:16:21 2024-11-22 06:11:48,597 INFO 3318922:repeater task 3dayplots_os38nb starting 28.6 seconds late 2024-11-22 06:11:54,941 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.3 seconds 2024-11-22 06:11:54,942 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:11:54,942 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.0 seconds late 2024-11-22 06:11:56,241 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-22 06:11:56,241 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-22 06:11:56,241 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:56,242 INFO 3318922:repeater task grab_lastpings_os38nb starting 41.3 seconds late 2024-11-22 06:11:56,411 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-22 06:11:56,411 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:56,411 INFO 3318922:repeater task beamdiagnostics_os38nb starting 41.4 seconds late 2024-11-22 06:11:58,117 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-22 06:11:58,117 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:58,117 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.1 seconds late 2024-11-22 06:11:59,907 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-22 06:11:59,907 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/22 06:26:14 2024-11-22 06:11:59,909 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 44.8 seconds late 2024-11-22 06:12:01,850 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-22 06:12:01,850 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-22 06:12:01,850 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/22 06:26:15 2024-11-22 06:12:01,850 INFO 3318922:repeater task plot_hbins starting 46.7 seconds late 2024-11-22 06:12:05,954 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-22 06:12:05,954 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-22 06:12:05,954 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/22 06:26:15 2024-11-22 06:12:10,960 INFO 3318922:repeater task 3dayplots_wh300 starting 51.0 seconds late 2024-11-22 06:12:17,616 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 06:12:17,616 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:12:22,624 INFO 3318922:repeater task lastensq_wh300 starting 3.6 seconds late 2024-11-22 06:12:25,799 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 06:12:25,799 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:12:14 2024-11-22 06:12:25,799 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:14:19 2024-11-22 06:12:25,800 INFO 3318922:repeater task quick_wh300 starting 6.8 seconds late 2024-11-22 06:12:31,445 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 06:12:31,445 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:12:14 2024-11-22 06:12:31,446 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:14:19 2024-11-22 06:12:31,446 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2024-11-22 06:12:32,490 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:12:32,491 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:12:14 2024-11-22 06:12:32,491 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:14:19 2024-11-22 06:14:22,813 INFO 3318922:repeater task lastensq_wh300 starting 3.8 seconds late 2024-11-22 06:14:26,057 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 06:14:26,057 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:14:14 2024-11-22 06:14:26,058 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:14:26,058 INFO 3318922:repeater task quick_wh300 starting 7.1 seconds late 2024-11-22 06:14:31,677 INFO 3318922:repeater task quick_wh300 ended after 5.6 seconds 2024-11-22 06:14:31,677 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:14:14 2024-11-22 06:14:31,677 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:14:31,678 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2024-11-22 06:14:32,727 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:14:32,727 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:14:14 2024-11-22 06:14:32,727 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:16:19 2024-11-22 06:16:23,034 INFO 3318922:repeater task noisy_rawtemps_os150 starting 3.1 seconds late 2024-11-22 06:16:26,297 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.3 seconds 2024-11-22 06:16:26,297 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/22 06:26:19 2024-11-22 06:16:26,298 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 6.3 seconds late 2024-11-22 06:16:29,460 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.2 seconds 2024-11-22 06:16:29,460 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/22 06:26:19 2024-11-22 06:16:29,461 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.5 seconds late 2024-11-22 06:16:33,112 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.7 seconds 2024-11-22 06:16:33,113 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/22 06:26:19 2024-11-22 06:16:33,113 INFO 3318922:repeater task lastensq_wh300 starting 14.1 seconds late 2024-11-22 06:16:36,340 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 06:16:36,340 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:16:14 2024-11-22 06:16:36,340 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:18:19 2024-11-22 06:16:36,341 INFO 3318922:repeater task quick_wh300 starting 17.3 seconds late 2024-11-22 06:16:42,211 INFO 3318922:repeater task quick_wh300 ended after 5.9 seconds 2024-11-22 06:16:42,212 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:16:14 2024-11-22 06:16:42,212 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:18:19 2024-11-22 06:16:42,213 INFO 3318922:repeater task lastfew_vec_wh300 starting 23.2 seconds late 2024-11-22 06:16:43,311 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-22 06:16:43,311 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:16:14 2024-11-22 06:16:43,311 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:18:19 2024-11-22 06:16:43,311 INFO 3318922:repeater task 3dayplots_wh300 starting 23.3 seconds late 2024-11-22 06:16:49,901 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.6 seconds 2024-11-22 06:16:49,901 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 06:21:19 2024-11-22 06:16:49,902 INFO 3318922:repeater task lastensq_os150nb starting 30.9 seconds late 2024-11-22 06:16:52,564 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-22 06:16:52,579 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:16:14 2024-11-22 06:16:52,580 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/22 06:21:19 2024-11-22 06:16:52,580 INFO 3318922:repeater task quick_os150nb starting 33.6 seconds late 2024-11-22 06:16:56,352 INFO 3318922:repeater task quick_os150nb ended after 3.8 seconds 2024-11-22 06:16:56,352 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:16:14 2024-11-22 06:16:56,352 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/22 06:21:19 2024-11-22 06:16:56,353 INFO 3318922:repeater task lastfew_vec_os150nb starting 37.4 seconds late 2024-11-22 06:16:57,385 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-22 06:16:57,385 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/22 06:16:14 2024-11-22 06:16:57,385 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/22 06:21:19 2024-11-22 06:16:57,386 INFO 3318922:repeater task 3dayplots_os150nb starting 37.4 seconds late 2024-11-22 06:17:03,614 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.2 seconds 2024-11-22 06:17:03,614 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/22 06:21:19 2024-11-22 06:17:03,616 INFO 3318922:repeater task lastensq_os38nb starting 42.6 seconds late 2024-11-22 06:17:06,225 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-22 06:17:06,226 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:16:16 2024-11-22 06:17:06,226 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/22 06:21:21 2024-11-22 06:17:06,226 INFO 3318922:repeater task quick_os38nb starting 45.2 seconds late 2024-11-22 06:17:10,316 INFO 3318922:repeater task quick_os38nb ended after 4.1 seconds 2024-11-22 06:17:10,316 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:16:16 2024-11-22 06:17:10,316 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/22 06:21:21 2024-11-22 06:17:10,316 INFO 3318922:repeater task lastfew_vec_os38nb starting 49.3 seconds late 2024-11-22 06:17:11,342 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-22 06:17:11,342 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/22 06:16:16 2024-11-22 06:17:11,342 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/22 06:21:21 2024-11-22 06:17:11,343 INFO 3318922:repeater task 3dayplots_os38nb starting 51.4 seconds late 2024-11-22 06:17:17,548 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.2 seconds 2024-11-22 06:17:17,548 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/22 06:21:19 2024-11-22 06:17:17,549 INFO 3318922:repeater task daily_netcdf starting 57.6 seconds late 2024-11-22 06:17:18,235 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-22 06:17:18,235 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/22 06:26:19 2024-11-22 06:18:23,412 INFO 3318922:repeater task lastensq_wh300 starting 4.4 seconds late 2024-11-22 06:18:26,669 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-22 06:18:26,670 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:18:14 2024-11-22 06:18:26,670 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:20:19 2024-11-22 06:18:26,670 INFO 3318922:repeater task quick_wh300 starting 7.7 seconds late 2024-11-22 06:18:32,454 INFO 3318922:repeater task quick_wh300 ended after 5.8 seconds 2024-11-22 06:18:32,454 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:18:14 2024-11-22 06:18:32,454 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:20:19 2024-11-22 06:18:32,455 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.5 seconds late 2024-11-22 06:18:33,497 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:18:33,497 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:18:14 2024-11-22 06:18:33,497 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:20:19 2024-11-22 06:20:23,812 INFO 3318922:repeater task lastensq_wh300 starting 4.8 seconds late 2024-11-22 06:20:26,980 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-22 06:20:26,980 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:20:14 2024-11-22 06:20:26,980 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/22 06:22:19 2024-11-22 06:20:26,980 INFO 3318922:repeater task quick_wh300 starting 8.0 seconds late 2024-11-22 06:20:32,479 INFO 3318922:repeater task quick_wh300 ended after 5.5 seconds 2024-11-22 06:20:32,479 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:20:14 2024-11-22 06:20:32,479 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/22 06:22:19 2024-11-22 06:20:32,479 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.5 seconds late 2024-11-22 06:20:33,518 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-22 06:20:33,518 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/22 06:20:14 2024-11-22 06:20:33,518 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/22 06:22:19 2024-11-22 06:21:23,647 INFO 3318922:repeater task 3dayplots_wh300 starting 3.7 seconds late 2024-11-22 06:21:30,340 INFO 3318922:repeater task 3dayplots_wh300 ended after 6.7 seconds 2024-11-22 06:21:30,341 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/22 06:26:19 2024-11-22 06:21:30,342 INFO 3318922:repeater task lastensq_os150nb starting 11.3 seconds late