2024-11-24 20:16:32,679 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 20:16:32,680 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:16:14 2024-11-24 20:16:32,680 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:18:19 2024-11-24 20:16:32,681 INFO 3318922:repeater task quick_wh300 starting 13.7 seconds late 2024-11-24 20:16:39,661 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 20:16:39,661 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:16:14 2024-11-24 20:16:39,661 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:18:19 2024-11-24 20:16:39,662 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.7 seconds late 2024-11-24 20:16:40,720 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:16:40,721 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:16:14 2024-11-24 20:16:40,721 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:18:19 2024-11-24 20:16:40,721 INFO 3318922:repeater task 3dayplots_wh300 starting 20.7 seconds late 2024-11-24 20:16:47,991 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.3 seconds 2024-11-24 20:16:47,991 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:21:19 2024-11-24 20:16:47,993 INFO 3318922:repeater task lastensq_os150nb starting 29.0 seconds late 2024-11-24 20:16:50,752 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-24 20:16:50,752 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:16:14 2024-11-24 20:16:50,752 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:21:19 2024-11-24 20:16:50,753 INFO 3318922:repeater task quick_os150nb starting 31.8 seconds late 2024-11-24 20:16:55,133 INFO 3318922:repeater task quick_os150nb ended after 4.4 seconds 2024-11-24 20:16:55,134 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:16:14 2024-11-24 20:16:55,134 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:21:19 2024-11-24 20:16:55,134 INFO 3318922:repeater task lastfew_vec_os150nb starting 36.1 seconds late 2024-11-24 20:16:56,166 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:16:56,166 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:16:14 2024-11-24 20:16:56,166 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:21:19 2024-11-24 20:16:56,166 INFO 3318922:repeater task 3dayplots_os150nb starting 36.2 seconds late 2024-11-24 20:17:03,197 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.0 seconds 2024-11-24 20:17:03,197 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:21:19 2024-11-24 20:17:03,198 INFO 3318922:repeater task lastensq_os38nb starting 42.2 seconds late 2024-11-24 20:17:05,846 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-24 20:17:05,846 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:16:16 2024-11-24 20:17:05,846 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:21:21 2024-11-24 20:17:05,847 INFO 3318922:repeater task quick_os38nb starting 44.8 seconds late 2024-11-24 20:17:10,234 INFO 3318922:repeater task quick_os38nb ended after 4.4 seconds 2024-11-24 20:17:10,234 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:16:16 2024-11-24 20:17:10,234 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:21:21 2024-11-24 20:17:10,234 INFO 3318922:repeater task lastfew_vec_os38nb starting 49.2 seconds late 2024-11-24 20:17:11,269 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:17:11,269 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:16:16 2024-11-24 20:17:11,269 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:21:21 2024-11-24 20:17:11,269 INFO 3318922:repeater task 3dayplots_os38nb starting 51.3 seconds late 2024-11-24 20:17:18,013 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 20:17:18,013 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:21:19 2024-11-24 20:17:18,014 INFO 3318922:repeater task daily_netcdf starting 58.0 seconds late 2024-11-24 20:17:18,758 INFO 3318922:repeater task daily_netcdf ended after 0.7 seconds 2024-11-24 20:17:18,758 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:18:23,944 INFO 3318922:repeater task lastensq_wh300 starting 4.9 seconds late 2024-11-24 20:18:27,324 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 20:18:27,325 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:18:14 2024-11-24 20:18:27,325 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:20:19 2024-11-24 20:18:27,325 INFO 3318922:repeater task quick_wh300 starting 8.3 seconds late 2024-11-24 20:18:34,107 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 20:18:34,107 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:18:14 2024-11-24 20:18:34,107 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:20:19 2024-11-24 20:18:34,108 INFO 3318922:repeater task lastfew_vec_wh300 starting 15.1 seconds late 2024-11-24 20:18:35,153 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:18:35,154 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:18:14 2024-11-24 20:18:35,154 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:20:19 2024-11-24 20:20:20,468 INFO 3318922:repeater task lastensq_wh300 starting 1.5 seconds late 2024-11-24 20:20:23,715 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 20:20:23,715 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:20:14 2024-11-24 20:20:23,715 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:22:19 2024-11-24 20:20:23,716 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-24 20:20:30,825 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 20:20:30,825 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:20:14 2024-11-24 20:20:30,825 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:22:19 2024-11-24 20:20:30,826 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2024-11-24 20:20:31,872 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:20:31,872 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:20:14 2024-11-24 20:20:31,872 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:22:19 2024-11-24 20:21:22,017 INFO 3318922:repeater task 3dayplots_wh300 starting 2.0 seconds late 2024-11-24 20:21:29,234 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.2 seconds 2024-11-24 20:21:29,235 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:21:29,236 INFO 3318922:repeater task lastensq_os150nb starting 10.2 seconds late 2024-11-24 20:21:31,852 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-24 20:21:31,853 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:21:14 2024-11-24 20:21:31,853 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:21:31,853 INFO 3318922:repeater task quick_os150nb starting 12.9 seconds late 2024-11-24 20:21:36,297 INFO 3318922:repeater task quick_os150nb ended after 4.4 seconds 2024-11-24 20:21:36,297 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:21:14 2024-11-24 20:21:36,298 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:21:36,298 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.3 seconds late 2024-11-24 20:21:37,337 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:21:37,337 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:21:14 2024-11-24 20:21:37,337 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:21:37,337 INFO 3318922:repeater task 3dayplots_os150nb starting 17.4 seconds late 2024-11-24 20:21:44,191 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 20:21:44,191 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:21:44,192 INFO 3318922:repeater task lastensq_os38nb starting 23.2 seconds late 2024-11-24 20:21:46,792 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-24 20:21:46,793 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:21:16 2024-11-24 20:21:46,793 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:26:21 2024-11-24 20:21:46,793 INFO 3318922:repeater task quick_os38nb starting 25.8 seconds late 2024-11-24 20:21:51,238 INFO 3318922:repeater task quick_os38nb ended after 4.4 seconds 2024-11-24 20:21:51,239 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:21:16 2024-11-24 20:21:51,239 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:26:21 2024-11-24 20:21:51,239 INFO 3318922:repeater task lastfew_vec_os38nb starting 30.2 seconds late 2024-11-24 20:21:52,260 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:21:52,261 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:21:16 2024-11-24 20:21:52,261 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:26:21 2024-11-24 20:21:52,261 INFO 3318922:repeater task 3dayplots_os38nb starting 32.3 seconds late 2024-11-24 20:21:58,992 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 20:21:58,992 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:22:19,039 INFO 3318922:repeater task lastensq_wh300 starting 0.0 seconds late 2024-11-24 20:22:22,403 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 20:22:22,403 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:22:14 2024-11-24 20:22:22,404 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:24:19 2024-11-24 20:22:22,404 INFO 3318922:repeater task quick_wh300 starting 3.4 seconds late 2024-11-24 20:22:29,328 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 20:22:29,329 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:22:14 2024-11-24 20:22:29,329 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:24:19 2024-11-24 20:22:29,329 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2024-11-24 20:22:30,372 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:22:30,372 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:22:14 2024-11-24 20:22:30,372 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:24:19 2024-11-24 20:24:20,697 INFO 3318922:repeater task lastensq_wh300 starting 1.7 seconds late 2024-11-24 20:24:23,987 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:24:23,987 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:24:14 2024-11-24 20:24:23,987 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:24:23,988 INFO 3318922:repeater task quick_wh300 starting 5.0 seconds late 2024-11-24 20:24:31,236 INFO 3318922:repeater task quick_wh300 ended after 7.2 seconds 2024-11-24 20:24:31,237 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:24:14 2024-11-24 20:24:31,237 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:24:31,237 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2024-11-24 20:24:32,284 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:24:32,285 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:24:14 2024-11-24 20:24:32,285 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:26:19 2024-11-24 20:26:17,588 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.6 seconds late 2024-11-24 20:26:19,166 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:26:19,166 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.6 seconds 2024-11-24 20:26:19,166 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:19,167 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.2 seconds late 2024-11-24 20:26:19,331 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-24 20:26:19,331 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:19,331 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.4 seconds late 2024-11-24 20:26:21,111 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-24 20:26:21,111 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:21,112 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.1 seconds late 2024-11-24 20:26:22,892 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.8 seconds 2024-11-24 20:26:22,892 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:22,892 INFO 3318922:repeater task lastensq_os150nb starting 3.9 seconds late 2024-11-24 20:26:25,694 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-24 20:26:25,694 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:26:14 2024-11-24 20:26:25,694 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:31:19 2024-11-24 20:26:25,695 INFO 3318922:repeater task quick_os150nb starting 6.7 seconds late 2024-11-24 20:26:29,849 INFO 3318922:repeater task quick_os150nb ended after 4.2 seconds 2024-11-24 20:26:29,849 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:26:14 2024-11-24 20:26:29,849 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:31:19 2024-11-24 20:26:29,850 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.9 seconds late 2024-11-24 20:26:30,895 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:26:30,895 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:26:14 2024-11-24 20:26:30,896 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:31:19 2024-11-24 20:26:30,896 INFO 3318922:repeater task 3dayplots_os150nb starting 10.9 seconds late 2024-11-24 20:26:37,745 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.8 seconds 2024-11-24 20:26:37,745 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:31:19 2024-11-24 20:26:37,745 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.8 seconds late 2024-11-24 20:26:39,055 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:26:39,055 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-24 20:26:39,055 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:39,055 INFO 3318922:repeater task grab_lastpings_os150nb starting 24.1 seconds late 2024-11-24 20:26:39,218 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-24 20:26:39,219 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:39,219 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.2 seconds late 2024-11-24 20:26:40,970 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-24 20:26:40,970 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:40,971 INFO 3318922:repeater task beamdiagnostics_os150nb starting 26.0 seconds late 2024-11-24 20:26:42,677 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 20:26:42,678 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:42,678 INFO 3318922:repeater task lastensq_os38nb starting 21.7 seconds late 2024-11-24 20:26:45,428 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-24 20:26:45,428 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:26:16 2024-11-24 20:26:45,428 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:31:21 2024-11-24 20:26:45,428 INFO 3318922:repeater task quick_os38nb starting 24.4 seconds late 2024-11-24 20:26:49,972 INFO 3318922:repeater task quick_os38nb ended after 4.5 seconds 2024-11-24 20:26:49,972 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:26:16 2024-11-24 20:26:49,973 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:31:21 2024-11-24 20:26:49,973 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.0 seconds late 2024-11-24 20:26:50,997 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:26:50,998 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:26:16 2024-11-24 20:26:50,998 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:31:21 2024-11-24 20:26:50,998 INFO 3318922:repeater task 3dayplots_os38nb starting 31.0 seconds late 2024-11-24 20:26:57,844 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.8 seconds 2024-11-24 20:26:57,844 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:31:19 2024-11-24 20:26:57,845 INFO 3318922:repeater task plot_tsstats_os38nb starting 42.9 seconds late 2024-11-24 20:26:59,176 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:26:59,177 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-24 20:26:59,177 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:59,177 INFO 3318922:repeater task grab_lastpings_os38nb starting 44.2 seconds late 2024-11-24 20:26:59,335 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-24 20:26:59,336 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:26:59,336 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.4 seconds late 2024-11-24 20:27:01,029 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 20:27:01,030 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:27:01,030 INFO 3318922:repeater task beamdiagnostics_os38nb starting 46.1 seconds late 2024-11-24 20:27:02,778 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 20:27:02,778 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 20:41:14 2024-11-24 20:27:02,779 INFO 3318922:repeater task daily_netcdf starting 42.8 seconds late 2024-11-24 20:27:03,606 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 20:27:03,607 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:27:03,607 INFO 3318922:repeater task ggatime_diagnostics starting 48.6 seconds late 2024-11-24 20:27:11,236 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-24 20:27:11,236 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:27:11,237 INFO 3318922:repeater task ggatime_diagnostics starting 56.3 seconds late 2024-11-24 20:27:18,763 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-24 20:27:18,763 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:27:18,764 INFO 3318922:repeater task ggatime_diagnostics starting 63.8 seconds late 2024-11-24 20:27:26,289 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-24 20:27:26,289 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:27:26,290 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 71.2 seconds late 2024-11-24 20:27:28,159 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-24 20:27:28,159 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-24 20:27:28,159 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/24 20:41:15 2024-11-24 20:27:28,159 INFO 3318922:repeater task plot_hbins starting 73.0 seconds late 2024-11-24 20:27:32,260 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-24 20:27:32,260 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-24 20:27:32,260 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/24 20:41:15 2024-11-24 20:27:32,261 INFO 3318922:repeater task report_generator starting 77.1 seconds late 2024-11-24 20:27:37,308 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 4.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-24 20:27:37,308 INFO 3318922:repeater task report_generator ended after 5.0 seconds 2024-11-24 20:27:37,308 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/24 21:26:15 2024-11-24 20:27:42,315 INFO 3318922:repeater task noisy_rawtemps_os150 starting 82.3 seconds late 2024-11-24 20:27:45,232 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2024-11-24 20:27:45,232 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:27:45,232 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 85.3 seconds late 2024-11-24 20:27:48,170 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 20:27:48,170 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:27:48,170 INFO 3318922:repeater task noisy_rawtemps_os38 starting 88.2 seconds late 2024-11-24 20:27:51,634 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-24 20:27:51,634 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:27:51,635 INFO 3318922:repeater task lastensq_wh300 starting 92.6 seconds late 2024-11-24 20:27:54,930 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:27:54,931 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:26:14 2024-11-24 20:27:54,931 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:28:19 2024-11-24 20:27:54,931 INFO 3318922:repeater task quick_wh300 starting 95.9 seconds late 2024-11-24 20:28:02,184 INFO 3318922:repeater task quick_wh300 ended after 7.3 seconds 2024-11-24 20:28:02,185 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:26:14 2024-11-24 20:28:02,185 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:28:19 2024-11-24 20:28:02,185 INFO 3318922:repeater task lastfew_vec_wh300 starting 103.2 seconds late 2024-11-24 20:28:03,234 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:28:03,234 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:26:14 2024-11-24 20:28:03,234 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:28:19 2024-11-24 20:28:03,235 INFO 3318922:repeater task 3dayplots_wh300 starting 103.3 seconds late 2024-11-24 20:28:10,371 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.1 seconds 2024-11-24 20:28:10,371 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:31:19 2024-11-24 20:28:20,395 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-24 20:28:23,728 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:28:23,729 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:28:14 2024-11-24 20:28:23,729 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:30:19 2024-11-24 20:28:23,729 INFO 3318922:repeater task quick_wh300 starting 4.7 seconds late 2024-11-24 20:28:30,796 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 20:28:30,797 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:28:14 2024-11-24 20:28:30,797 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:30:19 2024-11-24 20:28:30,797 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2024-11-24 20:28:31,850 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:28:31,850 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:28:14 2024-11-24 20:28:31,850 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:30:19 2024-11-24 20:30:22,176 INFO 3318922:repeater task lastensq_wh300 starting 3.2 seconds late 2024-11-24 20:30:25,385 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 20:30:25,385 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:30:14 2024-11-24 20:30:25,385 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:32:19 2024-11-24 20:30:25,385 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-24 20:30:32,176 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 20:30:32,176 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:30:14 2024-11-24 20:30:32,176 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:32:19 2024-11-24 20:30:32,176 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2024-11-24 20:30:33,222 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:30:33,223 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:30:14 2024-11-24 20:30:33,223 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:32:19 2024-11-24 20:31:23,378 INFO 3318922:repeater task 3dayplots_wh300 starting 3.4 seconds late 2024-11-24 20:31:30,615 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.2 seconds 2024-11-24 20:31:30,615 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:31:30,616 INFO 3318922:repeater task lastensq_os150nb starting 11.6 seconds late 2024-11-24 20:31:33,411 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-24 20:31:33,412 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:31:14 2024-11-24 20:31:33,412 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:31:33,412 INFO 3318922:repeater task quick_os150nb starting 14.4 seconds late 2024-11-24 20:31:37,841 INFO 3318922:repeater task quick_os150nb ended after 4.4 seconds 2024-11-24 20:31:37,841 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:31:14 2024-11-24 20:31:37,841 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:31:37,842 INFO 3318922:repeater task lastfew_vec_os150nb starting 18.8 seconds late 2024-11-24 20:31:38,871 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:31:38,872 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:31:14 2024-11-24 20:31:38,872 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:31:38,872 INFO 3318922:repeater task 3dayplots_os150nb starting 18.9 seconds late 2024-11-24 20:31:45,767 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 20:31:45,767 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:31:45,769 INFO 3318922:repeater task lastensq_os38nb starting 24.8 seconds late 2024-11-24 20:31:48,448 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-24 20:31:48,448 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:31:16 2024-11-24 20:31:48,448 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:36:21 2024-11-24 20:31:48,449 INFO 3318922:repeater task quick_os38nb starting 27.4 seconds late 2024-11-24 20:31:52,982 INFO 3318922:repeater task quick_os38nb ended after 4.5 seconds 2024-11-24 20:31:52,982 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:31:16 2024-11-24 20:31:52,982 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:36:21 2024-11-24 20:31:52,982 INFO 3318922:repeater task lastfew_vec_os38nb starting 32.0 seconds late 2024-11-24 20:31:54,007 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:31:54,008 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:31:16 2024-11-24 20:31:54,008 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:36:21 2024-11-24 20:31:54,008 INFO 3318922:repeater task 3dayplots_os38nb starting 34.0 seconds late 2024-11-24 20:32:00,856 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.8 seconds 2024-11-24 20:32:00,856 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:32:20,900 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-24 20:32:24,269 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 20:32:24,269 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:32:14 2024-11-24 20:32:24,269 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:34:19 2024-11-24 20:32:24,270 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-24 20:32:31,130 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 20:32:31,131 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:32:14 2024-11-24 20:32:31,131 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:34:19 2024-11-24 20:32:31,131 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-24 20:32:32,176 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:32:32,176 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:32:14 2024-11-24 20:32:32,176 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:34:19 2024-11-24 20:34:22,511 INFO 3318922:repeater task lastensq_wh300 starting 3.5 seconds late 2024-11-24 20:34:25,983 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-24 20:34:25,983 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:34:14 2024-11-24 20:34:25,983 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:34:25,983 INFO 3318922:repeater task quick_wh300 starting 7.0 seconds late 2024-11-24 20:34:33,082 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 20:34:33,082 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:34:14 2024-11-24 20:34:33,083 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:34:33,083 INFO 3318922:repeater task lastfew_vec_wh300 starting 14.1 seconds late 2024-11-24 20:34:34,142 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:34:34,142 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:34:14 2024-11-24 20:34:34,142 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:36:19 2024-11-24 20:36:19,463 INFO 3318922:repeater task lastensq_wh300 starting 0.5 seconds late 2024-11-24 20:36:22,691 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 20:36:22,691 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:36:14 2024-11-24 20:36:22,691 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:38:19 2024-11-24 20:36:22,691 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-24 20:36:29,731 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 20:36:29,731 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:36:14 2024-11-24 20:36:29,731 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:38:19 2024-11-24 20:36:29,731 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2024-11-24 20:36:30,801 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:36:30,801 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:36:14 2024-11-24 20:36:30,801 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:38:19 2024-11-24 20:36:30,802 INFO 3318922:repeater task 3dayplots_wh300 starting 10.8 seconds late 2024-11-24 20:36:38,073 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.3 seconds 2024-11-24 20:36:38,073 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:41:19 2024-11-24 20:36:38,074 INFO 3318922:repeater task lastensq_os150nb starting 19.1 seconds late 2024-11-24 20:36:40,807 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 20:36:40,808 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:36:14 2024-11-24 20:36:40,808 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:41:19 2024-11-24 20:36:40,808 INFO 3318922:repeater task quick_os150nb starting 21.8 seconds late 2024-11-24 20:36:44,827 INFO 3318922:repeater task quick_os150nb ended after 4.0 seconds 2024-11-24 20:36:44,828 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:36:14 2024-11-24 20:36:44,828 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:41:19 2024-11-24 20:36:44,828 INFO 3318922:repeater task lastfew_vec_os150nb starting 25.8 seconds late 2024-11-24 20:36:45,877 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:36:45,877 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:36:14 2024-11-24 20:36:45,877 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:41:19 2024-11-24 20:36:45,877 INFO 3318922:repeater task 3dayplots_os150nb starting 25.9 seconds late 2024-11-24 20:36:52,747 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 20:36:52,747 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:41:19 2024-11-24 20:36:52,749 INFO 3318922:repeater task lastensq_os38nb starting 31.7 seconds late 2024-11-24 20:36:55,375 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-24 20:36:55,375 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:36:16 2024-11-24 20:36:55,376 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:41:21 2024-11-24 20:36:55,376 INFO 3318922:repeater task quick_os38nb starting 34.4 seconds late 2024-11-24 20:36:59,911 INFO 3318922:repeater task quick_os38nb ended after 4.5 seconds 2024-11-24 20:36:59,913 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:36:16 2024-11-24 20:36:59,913 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:41:21 2024-11-24 20:36:59,913 INFO 3318922:repeater task lastfew_vec_os38nb starting 38.9 seconds late 2024-11-24 20:37:00,949 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:37:00,949 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:36:16 2024-11-24 20:37:00,949 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:41:21 2024-11-24 20:37:00,949 INFO 3318922:repeater task 3dayplots_os38nb starting 41.0 seconds late 2024-11-24 20:37:07,679 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 20:37:07,679 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:41:19 2024-11-24 20:37:07,680 INFO 3318922:repeater task daily_netcdf starting 47.7 seconds late 2024-11-24 20:37:08,495 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 20:37:08,495 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:37:13,502 INFO 3318922:repeater task noisy_rawtemps_os150 starting 53.5 seconds late 2024-11-24 20:37:16,416 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2024-11-24 20:37:16,416 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:37:16,416 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 56.4 seconds late 2024-11-24 20:37:19,326 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 20:37:19,327 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:37:19,327 INFO 3318922:repeater task noisy_rawtemps_os38 starting 59.4 seconds late 2024-11-24 20:37:22,754 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-24 20:37:22,755 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:38:22,936 INFO 3318922:repeater task lastensq_wh300 starting 3.9 seconds late 2024-11-24 20:38:26,164 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 20:38:26,164 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:38:14 2024-11-24 20:38:26,164 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:40:19 2024-11-24 20:38:26,165 INFO 3318922:repeater task quick_wh300 starting 7.2 seconds late 2024-11-24 20:38:32,837 INFO 3318922:repeater task quick_wh300 ended after 6.7 seconds 2024-11-24 20:38:32,837 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:38:14 2024-11-24 20:38:32,838 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:40:19 2024-11-24 20:38:32,838 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.8 seconds late 2024-11-24 20:38:33,885 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:38:33,885 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:38:14 2024-11-24 20:38:33,885 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:40:19 2024-11-24 20:40:19,207 INFO 3318922:repeater task lastensq_wh300 starting 0.2 seconds late 2024-11-24 20:40:22,530 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:40:22,531 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:40:14 2024-11-24 20:40:22,531 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:42:19 2024-11-24 20:40:22,531 INFO 3318922:repeater task quick_wh300 starting 3.5 seconds late 2024-11-24 20:40:29,427 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 20:40:29,428 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:40:14 2024-11-24 20:40:29,428 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:42:19 2024-11-24 20:40:29,428 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2024-11-24 20:40:30,472 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:40:30,473 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:40:14 2024-11-24 20:40:30,473 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:42:19 2024-11-24 20:41:15,589 INFO 3318922:repeater task plot_tsstats_wh300 starting 0.6 seconds late 2024-11-24 20:41:17,133 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:41:17,133 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-24 20:41:17,134 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:17,134 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.2 seconds late 2024-11-24 20:41:17,294 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-24 20:41:17,294 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:17,294 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.3 seconds late 2024-11-24 20:41:18,951 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 20:41:18,952 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:18,952 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.0 seconds late 2024-11-24 20:41:20,682 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 20:41:20,682 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:20,682 INFO 3318922:repeater task lastensq_os150nb starting 1.7 seconds late 2024-11-24 20:41:23,391 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 20:41:23,391 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:41:14 2024-11-24 20:41:23,392 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:41:23,392 INFO 3318922:repeater task quick_os150nb starting 4.4 seconds late 2024-11-24 20:41:27,642 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 20:41:27,642 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:41:14 2024-11-24 20:41:27,643 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:41:27,643 INFO 3318922:repeater task lastfew_vec_os150nb starting 8.6 seconds late 2024-11-24 20:41:28,697 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-24 20:41:28,698 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:41:14 2024-11-24 20:41:28,698 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:41:28,698 INFO 3318922:repeater task 3dayplots_os150nb starting 8.7 seconds late 2024-11-24 20:41:35,655 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.0 seconds 2024-11-24 20:41:35,655 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:41:35,656 INFO 3318922:repeater task plot_tsstats_os150nb starting 20.7 seconds late 2024-11-24 20:41:37,016 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:41:37,017 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.4 seconds 2024-11-24 20:41:37,017 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:37,017 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.0 seconds late 2024-11-24 20:41:37,195 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-24 20:41:37,195 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:37,196 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.2 seconds late 2024-11-24 20:41:38,901 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 20:41:38,901 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:38,902 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.9 seconds late 2024-11-24 20:41:40,612 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 20:41:40,613 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:40,613 INFO 3318922:repeater task lastensq_os38nb starting 19.6 seconds late 2024-11-24 20:41:43,298 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-24 20:41:43,298 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:41:16 2024-11-24 20:41:43,298 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:46:21 2024-11-24 20:41:43,299 INFO 3318922:repeater task quick_os38nb starting 22.3 seconds late 2024-11-24 20:41:47,922 INFO 3318922:repeater task quick_os38nb ended after 4.6 seconds 2024-11-24 20:41:47,922 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:41:16 2024-11-24 20:41:47,922 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:46:21 2024-11-24 20:41:47,923 INFO 3318922:repeater task lastfew_vec_os38nb starting 26.9 seconds late 2024-11-24 20:41:48,948 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:41:48,948 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:41:16 2024-11-24 20:41:48,948 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:46:21 2024-11-24 20:41:48,949 INFO 3318922:repeater task 3dayplots_os38nb starting 29.0 seconds late 2024-11-24 20:41:55,693 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 20:41:55,693 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:41:55,693 INFO 3318922:repeater task plot_tsstats_os38nb starting 40.7 seconds late 2024-11-24 20:41:57,026 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:41:57,027 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-24 20:41:57,027 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:57,027 INFO 3318922:repeater task grab_lastpings_os38nb starting 42.0 seconds late 2024-11-24 20:41:57,192 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-24 20:41:57,192 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:57,193 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.2 seconds late 2024-11-24 20:41:58,949 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-24 20:41:58,950 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:41:58,950 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.0 seconds late 2024-11-24 20:42:00,681 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 20:42:00,681 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 20:56:14 2024-11-24 20:42:00,683 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 45.6 seconds late 2024-11-24 20:42:02,558 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-24 20:42:02,558 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-24 20:42:02,558 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/24 20:56:15 2024-11-24 20:42:02,558 INFO 3318922:repeater task plot_hbins starting 47.4 seconds late 2024-11-24 20:42:06,694 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-24 20:42:06,694 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-24 20:42:06,694 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/24 20:56:15 2024-11-24 20:42:11,703 INFO 3318922:repeater task 3dayplots_wh300 starting 51.7 seconds late 2024-11-24 20:42:18,944 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.2 seconds 2024-11-24 20:42:18,944 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:42:23,953 INFO 3318922:repeater task lastensq_wh300 starting 5.0 seconds late 2024-11-24 20:42:27,256 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:42:27,257 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:42:14 2024-11-24 20:42:27,257 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:44:19 2024-11-24 20:42:27,257 INFO 3318922:repeater task quick_wh300 starting 8.3 seconds late 2024-11-24 20:42:34,074 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 20:42:34,075 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:42:14 2024-11-24 20:42:34,075 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:44:19 2024-11-24 20:42:34,075 INFO 3318922:repeater task lastfew_vec_wh300 starting 15.1 seconds late 2024-11-24 20:42:35,133 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:42:35,133 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:42:14 2024-11-24 20:42:35,133 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:44:19 2024-11-24 20:44:20,440 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-24 20:44:23,789 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:44:23,789 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:44:14 2024-11-24 20:44:23,789 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:44:23,790 INFO 3318922:repeater task quick_wh300 starting 4.8 seconds late 2024-11-24 20:44:31,027 INFO 3318922:repeater task quick_wh300 ended after 7.2 seconds 2024-11-24 20:44:31,027 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:44:14 2024-11-24 20:44:31,027 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:44:31,028 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2024-11-24 20:44:32,061 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:44:32,061 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:44:14 2024-11-24 20:44:32,061 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:46:19 2024-11-24 20:46:22,402 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.4 seconds late 2024-11-24 20:46:25,334 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2024-11-24 20:46:25,334 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:46:25,335 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.4 seconds late 2024-11-24 20:46:28,263 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 20:46:28,264 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:46:28,264 INFO 3318922:repeater task noisy_rawtemps_os38 starting 8.3 seconds late 2024-11-24 20:46:31,747 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.5 seconds 2024-11-24 20:46:31,747 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:46:31,748 INFO 3318922:repeater task lastensq_wh300 starting 12.7 seconds late 2024-11-24 20:46:34,973 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 20:46:34,973 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:46:14 2024-11-24 20:46:34,973 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:48:19 2024-11-24 20:46:34,973 INFO 3318922:repeater task quick_wh300 starting 16.0 seconds late 2024-11-24 20:46:41,854 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 20:46:41,854 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:46:14 2024-11-24 20:46:41,854 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:48:19 2024-11-24 20:46:41,855 INFO 3318922:repeater task lastfew_vec_wh300 starting 22.9 seconds late 2024-11-24 20:46:42,898 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:46:42,898 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:46:14 2024-11-24 20:46:42,898 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:48:19 2024-11-24 20:46:42,898 INFO 3318922:repeater task 3dayplots_wh300 starting 22.9 seconds late 2024-11-24 20:46:50,070 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.2 seconds 2024-11-24 20:46:50,070 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:51:19 2024-11-24 20:46:50,071 INFO 3318922:repeater task lastensq_os150nb starting 31.1 seconds late 2024-11-24 20:46:52,860 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-24 20:46:52,860 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:46:14 2024-11-24 20:46:52,860 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:51:19 2024-11-24 20:46:52,861 INFO 3318922:repeater task quick_os150nb starting 33.9 seconds late 2024-11-24 20:46:57,230 INFO 3318922:repeater task quick_os150nb ended after 4.4 seconds 2024-11-24 20:46:57,230 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:46:14 2024-11-24 20:46:57,230 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:51:19 2024-11-24 20:46:57,231 INFO 3318922:repeater task lastfew_vec_os150nb starting 38.2 seconds late 2024-11-24 20:46:58,258 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:46:58,258 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:46:14 2024-11-24 20:46:58,258 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:51:19 2024-11-24 20:46:58,259 INFO 3318922:repeater task 3dayplots_os150nb starting 38.3 seconds late 2024-11-24 20:47:05,479 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.2 seconds 2024-11-24 20:47:05,480 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:51:19 2024-11-24 20:47:05,481 INFO 3318922:repeater task lastensq_os38nb starting 44.5 seconds late 2024-11-24 20:47:08,166 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-24 20:47:08,166 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:46:16 2024-11-24 20:47:08,166 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:51:21 2024-11-24 20:47:08,166 INFO 3318922:repeater task quick_os38nb starting 47.2 seconds late 2024-11-24 20:47:12,801 INFO 3318922:repeater task quick_os38nb ended after 4.6 seconds 2024-11-24 20:47:12,801 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:46:16 2024-11-24 20:47:12,801 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:51:21 2024-11-24 20:47:12,802 INFO 3318922:repeater task lastfew_vec_os38nb starting 51.8 seconds late 2024-11-24 20:47:13,830 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:47:13,830 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:46:16 2024-11-24 20:47:13,830 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:51:21 2024-11-24 20:47:13,830 INFO 3318922:repeater task 3dayplots_os38nb starting 53.9 seconds late 2024-11-24 20:47:20,668 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.8 seconds 2024-11-24 20:47:20,668 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:51:19 2024-11-24 20:47:20,669 INFO 3318922:repeater task daily_netcdf starting 60.7 seconds late 2024-11-24 20:47:21,509 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 20:47:21,509 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:48:21,664 INFO 3318922:repeater task lastensq_wh300 starting 2.7 seconds late 2024-11-24 20:48:24,998 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:48:24,998 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:48:14 2024-11-24 20:48:24,999 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:50:19 2024-11-24 20:48:24,999 INFO 3318922:repeater task quick_wh300 starting 6.0 seconds late 2024-11-24 20:48:32,026 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 20:48:32,027 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:48:14 2024-11-24 20:48:32,027 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:50:19 2024-11-24 20:48:32,027 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.0 seconds late 2024-11-24 20:48:33,066 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:48:33,066 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:48:14 2024-11-24 20:48:33,066 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:50:19 2024-11-24 20:50:23,392 INFO 3318922:repeater task lastensq_wh300 starting 4.4 seconds late 2024-11-24 20:50:26,811 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 20:50:26,811 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:50:14 2024-11-24 20:50:26,811 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:52:19 2024-11-24 20:50:26,812 INFO 3318922:repeater task quick_wh300 starting 7.8 seconds late 2024-11-24 20:50:34,152 INFO 3318922:repeater task quick_wh300 ended after 7.3 seconds 2024-11-24 20:50:34,152 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:50:14 2024-11-24 20:50:34,152 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:52:19 2024-11-24 20:50:34,153 INFO 3318922:repeater task lastfew_vec_wh300 starting 15.2 seconds late 2024-11-24 20:50:35,206 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:50:35,206 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:50:14 2024-11-24 20:50:35,206 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:52:19 2024-11-24 20:51:20,341 INFO 3318922:repeater task 3dayplots_wh300 starting 0.4 seconds late 2024-11-24 20:51:27,722 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.4 seconds 2024-11-24 20:51:27,722 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:51:27,723 INFO 3318922:repeater task lastensq_os150nb starting 8.7 seconds late 2024-11-24 20:51:30,425 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 20:51:30,425 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:51:14 2024-11-24 20:51:30,426 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:51:30,426 INFO 3318922:repeater task quick_os150nb starting 11.4 seconds late 2024-11-24 20:51:34,677 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 20:51:34,677 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:51:14 2024-11-24 20:51:34,677 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:51:34,678 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.7 seconds late 2024-11-24 20:51:35,720 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:51:35,720 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:51:14 2024-11-24 20:51:35,720 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:51:35,721 INFO 3318922:repeater task 3dayplots_os150nb starting 15.7 seconds late 2024-11-24 20:51:42,790 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.1 seconds 2024-11-24 20:51:42,790 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:51:42,792 INFO 3318922:repeater task lastensq_os38nb starting 21.8 seconds late 2024-11-24 20:51:45,530 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-24 20:51:45,532 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:51:16 2024-11-24 20:51:45,532 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 20:56:21 2024-11-24 20:51:45,533 INFO 3318922:repeater task quick_os38nb starting 24.5 seconds late 2024-11-24 20:51:49,877 INFO 3318922:repeater task quick_os38nb ended after 4.3 seconds 2024-11-24 20:51:49,877 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:51:16 2024-11-24 20:51:49,877 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 20:56:21 2024-11-24 20:51:49,878 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.9 seconds late 2024-11-24 20:51:50,907 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:51:50,907 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:51:16 2024-11-24 20:51:50,907 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 20:56:21 2024-11-24 20:51:50,908 INFO 3318922:repeater task 3dayplots_os38nb starting 30.9 seconds late 2024-11-24 20:51:57,793 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.9 seconds 2024-11-24 20:51:57,793 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:52:22,852 INFO 3318922:repeater task lastensq_wh300 starting 3.9 seconds late 2024-11-24 20:52:26,225 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 20:52:26,226 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:52:14 2024-11-24 20:52:26,226 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:54:19 2024-11-24 20:52:26,226 INFO 3318922:repeater task quick_wh300 starting 7.2 seconds late 2024-11-24 20:52:33,061 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 20:52:33,061 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:52:14 2024-11-24 20:52:33,062 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:54:19 2024-11-24 20:52:33,062 INFO 3318922:repeater task lastfew_vec_wh300 starting 14.1 seconds late 2024-11-24 20:52:34,100 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 20:52:34,100 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:52:14 2024-11-24 20:52:34,100 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:54:19 2024-11-24 20:54:19,426 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-24 20:54:22,678 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:54:22,678 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:54:14 2024-11-24 20:54:22,678 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:54:22,679 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-24 20:54:29,657 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 20:54:29,657 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:54:14 2024-11-24 20:54:29,657 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:54:29,658 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2024-11-24 20:54:30,720 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:54:30,721 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:54:14 2024-11-24 20:54:30,721 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:56:19 2024-11-24 20:56:16,041 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.1 seconds late 2024-11-24 20:56:17,613 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:56:17,613 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.6 seconds 2024-11-24 20:56:17,613 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:17,613 INFO 3318922:repeater task grab_lastpings_wh300 starting 2.6 seconds late 2024-11-24 20:56:17,773 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-24 20:56:17,773 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:17,773 INFO 3318922:repeater task beamdiagnostics_wh300 starting 2.8 seconds late 2024-11-24 20:56:19,509 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 20:56:19,509 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:19,510 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.5 seconds late 2024-11-24 20:56:21,195 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 20:56:21,195 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:21,196 INFO 3318922:repeater task lastensq_os150nb starting 2.2 seconds late 2024-11-24 20:56:23,942 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 20:56:23,942 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:56:14 2024-11-24 20:56:23,943 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:01:19 2024-11-24 20:56:23,943 INFO 3318922:repeater task quick_os150nb starting 4.9 seconds late 2024-11-24 20:56:28,050 INFO 3318922:repeater task quick_os150nb ended after 4.1 seconds 2024-11-24 20:56:28,050 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:56:14 2024-11-24 20:56:28,051 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:01:19 2024-11-24 20:56:28,051 INFO 3318922:repeater task lastfew_vec_os150nb starting 9.1 seconds late 2024-11-24 20:56:29,070 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 20:56:29,070 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 20:56:14 2024-11-24 20:56:29,070 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:01:19 2024-11-24 20:56:29,071 INFO 3318922:repeater task 3dayplots_os150nb starting 9.1 seconds late 2024-11-24 20:56:35,948 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 20:56:35,948 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:01:19 2024-11-24 20:56:35,949 INFO 3318922:repeater task plot_tsstats_os150nb starting 21.0 seconds late 2024-11-24 20:56:37,254 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:56:37,254 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-24 20:56:37,254 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:37,255 INFO 3318922:repeater task grab_lastpings_os150nb starting 22.3 seconds late 2024-11-24 20:56:37,415 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-24 20:56:37,415 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:37,415 INFO 3318922:repeater task beamdiagnostics_os150nb starting 22.4 seconds late 2024-11-24 20:56:39,107 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 20:56:39,107 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:39,107 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.1 seconds late 2024-11-24 20:56:40,814 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 20:56:40,814 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:40,814 INFO 3318922:repeater task lastensq_os38nb starting 19.8 seconds late 2024-11-24 20:56:43,462 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-24 20:56:43,462 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:56:16 2024-11-24 20:56:43,462 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:01:21 2024-11-24 20:56:43,463 INFO 3318922:repeater task quick_os38nb starting 22.5 seconds late 2024-11-24 20:56:48,026 INFO 3318922:repeater task quick_os38nb ended after 4.6 seconds 2024-11-24 20:56:48,026 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:56:16 2024-11-24 20:56:48,027 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:01:21 2024-11-24 20:56:48,027 INFO 3318922:repeater task lastfew_vec_os38nb starting 27.0 seconds late 2024-11-24 20:56:49,069 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 20:56:49,069 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 20:56:16 2024-11-24 20:56:49,069 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:01:21 2024-11-24 20:56:49,070 INFO 3318922:repeater task 3dayplots_os38nb starting 29.1 seconds late 2024-11-24 20:56:55,952 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.9 seconds 2024-11-24 20:56:55,953 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:01:19 2024-11-24 20:56:55,953 INFO 3318922:repeater task plot_tsstats_os38nb starting 41.0 seconds late 2024-11-24 20:56:57,292 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 20:56:57,292 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-24 20:56:57,292 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:57,293 INFO 3318922:repeater task grab_lastpings_os38nb starting 42.3 seconds late 2024-11-24 20:56:57,455 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-24 20:56:57,455 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:57,455 INFO 3318922:repeater task beamdiagnostics_os38nb starting 42.5 seconds late 2024-11-24 20:56:59,224 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-24 20:56:59,225 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:56:59,225 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.2 seconds late 2024-11-24 20:57:00,963 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 20:57:00,963 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:11:14 2024-11-24 20:57:00,964 INFO 3318922:repeater task daily_netcdf starting 41.0 seconds late 2024-11-24 20:57:01,781 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 20:57:01,781 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 21:06:19 2024-11-24 20:57:01,782 INFO 3318922:repeater task ggatime_diagnostics starting 46.8 seconds late 2024-11-24 20:57:09,788 INFO 3318922:repeater task ggatime_diagnostics ended after 8.0 seconds 2024-11-24 20:57:09,788 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 21:26:14 2024-11-24 20:57:09,789 INFO 3318922:repeater task ggatime_diagnostics starting 54.8 seconds late 2024-11-24 20:57:17,263 INFO 3318922:repeater task ggatime_diagnostics ended after 7.5 seconds 2024-11-24 20:57:17,264 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 21:26:14 2024-11-24 20:57:17,264 INFO 3318922:repeater task ggatime_diagnostics starting 62.3 seconds late 2024-11-24 20:57:24,965 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-24 20:57:24,965 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 21:26:14 2024-11-24 20:57:24,965 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 69.8 seconds late 2024-11-24 20:57:26,888 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-24 20:57:26,888 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-24 20:57:26,888 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/24 21:11:15 2024-11-24 20:57:26,889 INFO 3318922:repeater task plot_hbins starting 71.8 seconds late 2024-11-24 20:57:31,064 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-24 20:57:31,064 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-24 20:57:31,064 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/24 21:11:15 2024-11-24 20:57:36,069 INFO 3318922:repeater task noisy_rawtemps_os150 starting 76.1 seconds late 2024-11-24 20:57:39,074 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2024-11-24 20:57:39,074 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 21:06:19 2024-11-24 20:57:39,075 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 79.1 seconds late 2024-11-24 20:57:41,971 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 20:57:41,971 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 21:06:19 2024-11-24 20:57:41,971 INFO 3318922:repeater task noisy_rawtemps_os38 starting 82.0 seconds late 2024-11-24 20:57:45,343 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-24 20:57:45,344 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 21:06:19 2024-11-24 20:57:45,344 INFO 3318922:repeater task lastensq_wh300 starting 86.3 seconds late 2024-11-24 20:57:48,658 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 20:57:48,659 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:56:14 2024-11-24 20:57:48,659 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 20:58:19 2024-11-24 20:57:48,659 INFO 3318922:repeater task quick_wh300 starting 89.7 seconds late 2024-11-24 20:57:55,545 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 20:57:55,545 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:56:14 2024-11-24 20:57:55,545 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 20:58:19 2024-11-24 20:57:55,546 INFO 3318922:repeater task lastfew_vec_wh300 starting 96.5 seconds late 2024-11-24 20:57:56,603 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:57:56,604 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:56:14 2024-11-24 20:57:56,604 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 20:58:19 2024-11-24 20:57:56,604 INFO 3318922:repeater task 3dayplots_wh300 starting 96.6 seconds late 2024-11-24 20:58:03,981 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.4 seconds 2024-11-24 20:58:03,981 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:01:19 2024-11-24 20:58:19,019 INFO 3318922:repeater task lastensq_wh300 starting 0.0 seconds late 2024-11-24 20:58:22,650 INFO 3318922:repeater task lastensq_wh300 ended after 3.6 seconds 2024-11-24 20:58:22,650 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:58:14 2024-11-24 20:58:22,650 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:00:19 2024-11-24 20:58:22,651 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-24 20:58:29,477 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 20:58:29,478 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:58:14 2024-11-24 20:58:29,478 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:00:19 2024-11-24 20:58:29,478 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-24 20:58:30,539 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 20:58:30,539 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 20:58:14 2024-11-24 20:58:30,539 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:00:19 2024-11-24 21:00:20,852 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-24 21:00:24,271 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 21:00:24,272 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:00:14 2024-11-24 21:00:24,272 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:02:19 2024-11-24 21:00:24,272 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-24 21:00:31,122 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 21:00:31,122 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:00:14 2024-11-24 21:00:31,122 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:02:19 2024-11-24 21:00:31,122 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2024-11-24 21:00:32,173 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:00:32,173 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:00:14 2024-11-24 21:00:32,174 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:02:19 2024-11-24 21:01:22,318 INFO 3318922:repeater task 3dayplots_wh300 starting 2.3 seconds late 2024-11-24 21:01:29,541 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.2 seconds 2024-11-24 21:01:29,541 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:01:29,543 INFO 3318922:repeater task lastensq_os150nb starting 10.5 seconds late 2024-11-24 21:01:32,184 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-24 21:01:32,185 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:01:14 2024-11-24 21:01:32,185 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:01:32,185 INFO 3318922:repeater task quick_os150nb starting 13.2 seconds late 2024-11-24 21:01:36,519 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 21:01:36,519 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:01:14 2024-11-24 21:01:36,520 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:01:36,520 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.5 seconds late 2024-11-24 21:01:37,607 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-24 21:01:37,608 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:01:14 2024-11-24 21:01:37,608 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:01:37,608 INFO 3318922:repeater task 3dayplots_os150nb starting 17.6 seconds late 2024-11-24 21:01:44,491 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 21:01:44,492 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:01:44,493 INFO 3318922:repeater task lastensq_os38nb starting 23.5 seconds late 2024-11-24 21:01:46,959 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-24 21:01:46,960 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:01:16 2024-11-24 21:01:46,960 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:06:21 2024-11-24 21:01:46,960 INFO 3318922:repeater task quick_os38nb starting 26.0 seconds late 2024-11-24 21:01:51,442 INFO 3318922:repeater task quick_os38nb ended after 4.5 seconds 2024-11-24 21:01:51,442 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:01:16 2024-11-24 21:01:51,442 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:06:21 2024-11-24 21:01:51,443 INFO 3318922:repeater task lastfew_vec_os38nb starting 30.4 seconds late 2024-11-24 21:01:52,474 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:01:52,474 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:01:16 2024-11-24 21:01:52,474 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:06:21 2024-11-24 21:01:52,474 INFO 3318922:repeater task 3dayplots_os38nb starting 32.5 seconds late 2024-11-24 21:01:59,209 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 21:01:59,209 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:02:19,258 INFO 3318922:repeater task lastensq_wh300 starting 0.3 seconds late 2024-11-24 21:02:22,719 INFO 3318922:repeater task lastensq_wh300 ended after 3.5 seconds 2024-11-24 21:02:22,719 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:02:14 2024-11-24 21:02:22,720 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:04:19 2024-11-24 21:02:22,720 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-24 21:02:29,721 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 21:02:29,722 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:02:14 2024-11-24 21:02:29,722 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:04:19 2024-11-24 21:02:29,722 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2024-11-24 21:02:30,780 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:02:30,780 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:02:14 2024-11-24 21:02:30,780 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:04:19 2024-11-24 21:04:21,099 INFO 3318922:repeater task lastensq_wh300 starting 2.1 seconds late 2024-11-24 21:04:24,427 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:04:24,427 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:04:14 2024-11-24 21:04:24,427 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:04:24,428 INFO 3318922:repeater task quick_wh300 starting 5.4 seconds late 2024-11-24 21:04:31,189 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 21:04:31,191 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:04:14 2024-11-24 21:04:31,191 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:04:31,191 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2024-11-24 21:04:32,249 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:04:32,250 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:04:14 2024-11-24 21:04:32,250 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:06:19 2024-11-24 21:06:22,589 INFO 3318922:repeater task noisy_rawtemps_os150 starting 2.6 seconds late 2024-11-24 21:06:25,475 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2024-11-24 21:06:25,476 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:06:25,476 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 5.5 seconds late 2024-11-24 21:06:28,346 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 21:06:28,346 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:06:28,346 INFO 3318922:repeater task noisy_rawtemps_os38 starting 8.4 seconds late 2024-11-24 21:06:31,744 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-24 21:06:31,744 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:06:31,745 INFO 3318922:repeater task lastensq_wh300 starting 12.7 seconds late 2024-11-24 21:06:34,993 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 21:06:34,993 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:06:14 2024-11-24 21:06:34,993 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:08:19 2024-11-24 21:06:34,994 INFO 3318922:repeater task quick_wh300 starting 16.0 seconds late 2024-11-24 21:06:41,884 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 21:06:41,885 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:06:14 2024-11-24 21:06:41,885 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:08:19 2024-11-24 21:06:41,885 INFO 3318922:repeater task lastfew_vec_wh300 starting 22.9 seconds late 2024-11-24 21:06:42,946 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:06:42,946 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:06:14 2024-11-24 21:06:42,946 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:08:19 2024-11-24 21:06:42,946 INFO 3318922:repeater task 3dayplots_wh300 starting 23.0 seconds late 2024-11-24 21:06:50,148 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.2 seconds 2024-11-24 21:06:50,148 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:11:19 2024-11-24 21:06:50,149 INFO 3318922:repeater task lastensq_os150nb starting 31.1 seconds late 2024-11-24 21:06:52,982 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-24 21:06:52,982 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:06:14 2024-11-24 21:06:52,982 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:11:19 2024-11-24 21:06:52,983 INFO 3318922:repeater task quick_os150nb starting 34.0 seconds late 2024-11-24 21:06:57,147 INFO 3318922:repeater task quick_os150nb ended after 4.2 seconds 2024-11-24 21:06:57,147 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:06:14 2024-11-24 21:06:57,147 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:11:19 2024-11-24 21:06:57,148 INFO 3318922:repeater task lastfew_vec_os150nb starting 38.1 seconds late 2024-11-24 21:06:58,166 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:06:58,166 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:06:14 2024-11-24 21:06:58,167 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:11:19 2024-11-24 21:06:58,167 INFO 3318922:repeater task 3dayplots_os150nb starting 38.2 seconds late 2024-11-24 21:07:05,184 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.0 seconds 2024-11-24 21:07:05,185 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:11:19 2024-11-24 21:07:05,186 INFO 3318922:repeater task lastensq_os38nb starting 44.2 seconds late 2024-11-24 21:07:07,775 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-24 21:07:07,775 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:06:16 2024-11-24 21:07:07,775 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:11:21 2024-11-24 21:07:07,776 INFO 3318922:repeater task quick_os38nb starting 46.8 seconds late 2024-11-24 21:07:12,179 INFO 3318922:repeater task quick_os38nb ended after 4.4 seconds 2024-11-24 21:07:12,179 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:06:16 2024-11-24 21:07:12,179 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:11:21 2024-11-24 21:07:12,179 INFO 3318922:repeater task lastfew_vec_os38nb starting 51.2 seconds late 2024-11-24 21:07:13,223 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:07:13,223 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:06:16 2024-11-24 21:07:13,223 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:11:21 2024-11-24 21:07:13,224 INFO 3318922:repeater task 3dayplots_os38nb starting 53.2 seconds late 2024-11-24 21:07:19,968 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 21:07:19,968 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:11:19 2024-11-24 21:07:19,969 INFO 3318922:repeater task daily_netcdf starting 60.0 seconds late 2024-11-24 21:07:20,784 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 21:07:20,784 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:08:20,948 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-24 21:08:24,363 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 21:08:24,364 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:08:14 2024-11-24 21:08:24,364 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:10:19 2024-11-24 21:08:24,364 INFO 3318922:repeater task quick_wh300 starting 5.4 seconds late 2024-11-24 21:08:31,203 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 21:08:31,203 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:08:14 2024-11-24 21:08:31,204 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:10:19 2024-11-24 21:08:31,204 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2024-11-24 21:08:32,249 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 21:08:32,249 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:08:14 2024-11-24 21:08:32,249 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:10:19 2024-11-24 21:10:22,564 INFO 3318922:repeater task lastensq_wh300 starting 3.6 seconds late 2024-11-24 21:10:25,830 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:10:25,831 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:10:14 2024-11-24 21:10:25,831 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:12:19 2024-11-24 21:10:25,831 INFO 3318922:repeater task quick_wh300 starting 6.8 seconds late 2024-11-24 21:10:32,629 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 21:10:32,630 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:10:14 2024-11-24 21:10:32,630 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:12:19 2024-11-24 21:10:32,630 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.6 seconds late 2024-11-24 21:10:33,685 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:10:33,685 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:10:14 2024-11-24 21:10:33,685 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:12:19 2024-11-24 21:11:18,821 INFO 3318922:repeater task plot_tsstats_wh300 starting 3.8 seconds late 2024-11-24 21:11:20,363 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:11:20,363 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.5 seconds 2024-11-24 21:11:20,363 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:20,363 INFO 3318922:repeater task grab_lastpings_wh300 starting 5.4 seconds late 2024-11-24 21:11:20,523 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-24 21:11:20,523 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:20,524 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.5 seconds late 2024-11-24 21:11:22,194 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 21:11:22,194 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:22,194 INFO 3318922:repeater task beamdiagnostics_wh300 starting 7.2 seconds late 2024-11-24 21:11:23,924 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 21:11:23,925 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:23,925 INFO 3318922:repeater task lastensq_os150nb starting 4.9 seconds late 2024-11-24 21:11:26,651 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 21:11:26,651 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:11:14 2024-11-24 21:11:26,651 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:11:26,652 INFO 3318922:repeater task quick_os150nb starting 7.7 seconds late 2024-11-24 21:11:30,890 INFO 3318922:repeater task quick_os150nb ended after 4.2 seconds 2024-11-24 21:11:30,891 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:11:14 2024-11-24 21:11:30,891 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:11:30,891 INFO 3318922:repeater task lastfew_vec_os150nb starting 11.9 seconds late 2024-11-24 21:11:31,909 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:11:31,909 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:11:14 2024-11-24 21:11:31,909 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:11:31,910 INFO 3318922:repeater task 3dayplots_os150nb starting 11.9 seconds late 2024-11-24 21:11:38,825 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 21:11:38,825 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:11:38,826 INFO 3318922:repeater task plot_tsstats_os150nb starting 23.8 seconds late 2024-11-24 21:11:40,127 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:11:40,127 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-24 21:11:40,127 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:40,128 INFO 3318922:repeater task grab_lastpings_os150nb starting 25.1 seconds late 2024-11-24 21:11:40,289 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-24 21:11:40,289 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:40,289 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.3 seconds late 2024-11-24 21:11:41,983 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 21:11:41,983 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:41,983 INFO 3318922:repeater task beamdiagnostics_os150nb starting 27.0 seconds late 2024-11-24 21:11:43,699 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 21:11:43,700 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:43,700 INFO 3318922:repeater task lastensq_os38nb starting 22.7 seconds late 2024-11-24 21:11:46,221 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-24 21:11:46,221 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:11:16 2024-11-24 21:11:46,221 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:16:21 2024-11-24 21:11:46,222 INFO 3318922:repeater task quick_os38nb starting 25.2 seconds late 2024-11-24 21:11:50,802 INFO 3318922:repeater task quick_os38nb ended after 4.6 seconds 2024-11-24 21:11:50,802 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:11:16 2024-11-24 21:11:50,802 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:16:21 2024-11-24 21:11:50,803 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.8 seconds late 2024-11-24 21:11:51,817 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:11:51,817 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:11:16 2024-11-24 21:11:51,817 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:16:21 2024-11-24 21:11:51,818 INFO 3318922:repeater task 3dayplots_os38nb starting 31.8 seconds late 2024-11-24 21:11:58,450 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.6 seconds 2024-11-24 21:11:58,451 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:11:58,451 INFO 3318922:repeater task plot_tsstats_os38nb starting 43.5 seconds late 2024-11-24 21:11:59,802 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:11:59,802 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.4 seconds 2024-11-24 21:11:59,803 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:59,803 INFO 3318922:repeater task grab_lastpings_os38nb starting 44.8 seconds late 2024-11-24 21:11:59,961 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-24 21:11:59,961 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:11:59,961 INFO 3318922:repeater task beamdiagnostics_os38nb starting 45.0 seconds late 2024-11-24 21:12:01,611 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.6 seconds 2024-11-24 21:12:01,611 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:12:01,611 INFO 3318922:repeater task beamdiagnostics_os38nb starting 46.6 seconds late 2024-11-24 21:12:03,322 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 21:12:03,322 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:26:14 2024-11-24 21:12:03,323 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 48.2 seconds late 2024-11-24 21:12:05,212 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-24 21:12:05,213 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-24 21:12:05,213 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/24 21:26:15 2024-11-24 21:12:05,213 INFO 3318922:repeater task plot_hbins starting 50.1 seconds late 2024-11-24 21:12:09,317 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-24 21:12:09,317 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-24 21:12:09,317 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/24 21:26:15 2024-11-24 21:12:14,325 INFO 3318922:repeater task 3dayplots_wh300 starting 54.3 seconds late 2024-11-24 21:12:21,657 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.3 seconds 2024-11-24 21:12:21,657 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:12:26,667 INFO 3318922:repeater task lastensq_wh300 starting 7.7 seconds late 2024-11-24 21:12:29,986 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:12:29,987 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:12:14 2024-11-24 21:12:29,987 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:14:19 2024-11-24 21:12:29,987 INFO 3318922:repeater task quick_wh300 starting 11.0 seconds late 2024-11-24 21:12:37,032 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 21:12:37,032 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:12:14 2024-11-24 21:12:37,033 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:14:19 2024-11-24 21:12:37,033 INFO 3318922:repeater task lastfew_vec_wh300 starting 18.0 seconds late 2024-11-24 21:12:38,090 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:12:38,090 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:12:14 2024-11-24 21:12:38,090 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:14:19 2024-11-24 21:14:23,397 INFO 3318922:repeater task lastensq_wh300 starting 4.4 seconds late 2024-11-24 21:14:26,715 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:14:26,715 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:14:14 2024-11-24 21:14:26,715 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:14:26,716 INFO 3318922:repeater task quick_wh300 starting 7.7 seconds late 2024-11-24 21:14:33,796 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 21:14:33,797 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:14:14 2024-11-24 21:14:33,797 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:14:33,797 INFO 3318922:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2024-11-24 21:14:34,836 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 21:14:34,836 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:14:14 2024-11-24 21:14:34,836 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:16:19 2024-11-24 21:16:20,146 INFO 3318922:repeater task noisy_rawtemps_os150 starting 0.2 seconds late 2024-11-24 21:16:22,961 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2024-11-24 21:16:22,961 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:16:22,961 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 3.0 seconds late 2024-11-24 21:16:25,889 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 21:16:25,899 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:16:25,920 INFO 3318922:repeater task noisy_rawtemps_os38 starting 5.9 seconds late 2024-11-24 21:16:29,260 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.3 seconds 2024-11-24 21:16:29,260 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:16:29,261 INFO 3318922:repeater task lastensq_wh300 starting 10.3 seconds late 2024-11-24 21:16:32,570 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:16:32,570 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:16:14 2024-11-24 21:16:32,571 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:18:19 2024-11-24 21:16:32,571 INFO 3318922:repeater task quick_wh300 starting 13.6 seconds late 2024-11-24 21:16:39,760 INFO 3318922:repeater task quick_wh300 ended after 7.2 seconds 2024-11-24 21:16:39,761 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:16:14 2024-11-24 21:16:39,761 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:18:19 2024-11-24 21:16:39,761 INFO 3318922:repeater task lastfew_vec_wh300 starting 20.8 seconds late 2024-11-24 21:16:40,812 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:16:40,812 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:16:14 2024-11-24 21:16:40,813 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:18:19 2024-11-24 21:16:40,813 INFO 3318922:repeater task 3dayplots_wh300 starting 20.8 seconds late 2024-11-24 21:16:47,871 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.1 seconds 2024-11-24 21:16:47,871 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:21:19 2024-11-24 21:16:47,872 INFO 3318922:repeater task lastensq_os150nb starting 28.9 seconds late 2024-11-24 21:16:50,526 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 21:16:50,526 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:16:14 2024-11-24 21:16:50,526 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:21:19 2024-11-24 21:16:50,526 INFO 3318922:repeater task quick_os150nb starting 31.5 seconds late 2024-11-24 21:16:54,927 INFO 3318922:repeater task quick_os150nb ended after 4.4 seconds 2024-11-24 21:16:54,927 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:16:14 2024-11-24 21:16:54,927 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:21:19 2024-11-24 21:16:54,928 INFO 3318922:repeater task lastfew_vec_os150nb starting 35.9 seconds late 2024-11-24 21:16:55,979 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-24 21:16:55,981 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:16:14 2024-11-24 21:16:55,981 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:21:19 2024-11-24 21:16:55,982 INFO 3318922:repeater task 3dayplots_os150nb starting 36.0 seconds late 2024-11-24 21:17:02,815 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.8 seconds 2024-11-24 21:17:02,815 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:21:19 2024-11-24 21:17:02,816 INFO 3318922:repeater task lastensq_os38nb starting 41.8 seconds late 2024-11-24 21:17:05,447 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-24 21:17:05,448 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:16:16 2024-11-24 21:17:05,448 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:21:21 2024-11-24 21:17:05,448 INFO 3318922:repeater task quick_os38nb starting 44.4 seconds late 2024-11-24 21:17:10,258 INFO 3318922:repeater task quick_os38nb ended after 4.8 seconds 2024-11-24 21:17:10,258 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:16:16 2024-11-24 21:17:10,259 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:21:21 2024-11-24 21:17:10,259 INFO 3318922:repeater task lastfew_vec_os38nb starting 49.3 seconds late 2024-11-24 21:17:11,282 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:17:11,282 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:16:16 2024-11-24 21:17:11,283 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:21:21 2024-11-24 21:17:11,283 INFO 3318922:repeater task 3dayplots_os38nb starting 51.3 seconds late 2024-11-24 21:17:18,096 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.8 seconds 2024-11-24 21:17:18,096 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:21:19 2024-11-24 21:17:18,097 INFO 3318922:repeater task daily_netcdf starting 58.1 seconds late 2024-11-24 21:17:18,946 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 21:17:18,946 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:18:19,097 INFO 3318922:repeater task lastensq_wh300 starting 0.1 seconds late 2024-11-24 21:18:22,499 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 21:18:22,499 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:18:14 2024-11-24 21:18:22,499 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:20:19 2024-11-24 21:18:22,499 INFO 3318922:repeater task quick_wh300 starting 3.5 seconds late 2024-11-24 21:18:29,501 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 21:18:29,501 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:18:14 2024-11-24 21:18:29,502 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:20:19 2024-11-24 21:18:29,502 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-24 21:18:30,655 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.2 seconds 2024-11-24 21:18:30,656 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:18:14 2024-11-24 21:18:30,656 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:20:19 2024-11-24 21:20:20,972 INFO 3318922:repeater task lastensq_wh300 starting 2.0 seconds late 2024-11-24 21:20:24,293 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:20:24,293 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:20:14 2024-11-24 21:20:24,293 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:22:19 2024-11-24 21:20:24,294 INFO 3318922:repeater task quick_wh300 starting 5.3 seconds late 2024-11-24 21:20:31,277 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 21:20:31,277 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:20:14 2024-11-24 21:20:31,277 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:22:19 2024-11-24 21:20:31,278 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2024-11-24 21:20:32,345 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:20:32,346 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:20:14 2024-11-24 21:20:32,346 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:22:19 2024-11-24 21:21:22,506 INFO 3318922:repeater task 3dayplots_wh300 starting 2.5 seconds late 2024-11-24 21:21:29,882 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.4 seconds 2024-11-24 21:21:29,882 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:21:29,883 INFO 3318922:repeater task lastensq_os150nb starting 10.9 seconds late 2024-11-24 21:21:32,561 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 21:21:32,562 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:21:14 2024-11-24 21:21:32,562 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:21:32,563 INFO 3318922:repeater task quick_os150nb starting 13.6 seconds late 2024-11-24 21:21:36,843 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 21:21:36,843 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:21:14 2024-11-24 21:21:36,843 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:21:36,843 INFO 3318922:repeater task lastfew_vec_os150nb starting 17.8 seconds late 2024-11-24 21:21:37,932 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.1 seconds 2024-11-24 21:21:37,932 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:21:14 2024-11-24 21:21:37,932 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:21:37,932 INFO 3318922:repeater task 3dayplots_os150nb starting 18.0 seconds late 2024-11-24 21:21:44,860 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 21:21:44,860 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:21:44,861 INFO 3318922:repeater task lastensq_os38nb starting 23.9 seconds late 2024-11-24 21:21:47,382 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-24 21:21:47,382 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:21:16 2024-11-24 21:21:47,382 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:26:21 2024-11-24 21:21:47,382 INFO 3318922:repeater task quick_os38nb starting 26.4 seconds late 2024-11-24 21:21:51,906 INFO 3318922:repeater task quick_os38nb ended after 4.5 seconds 2024-11-24 21:21:51,906 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:21:16 2024-11-24 21:21:51,906 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:26:21 2024-11-24 21:21:51,907 INFO 3318922:repeater task lastfew_vec_os38nb starting 30.9 seconds late 2024-11-24 21:21:52,948 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:21:52,948 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:21:16 2024-11-24 21:21:52,948 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:26:21 2024-11-24 21:21:52,949 INFO 3318922:repeater task 3dayplots_os38nb starting 33.0 seconds late 2024-11-24 21:21:59,673 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 21:21:59,673 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:22:19,728 INFO 3318922:repeater task lastensq_wh300 starting 0.7 seconds late 2024-11-24 21:22:22,941 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 21:22:22,941 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:22:14 2024-11-24 21:22:22,941 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:24:19 2024-11-24 21:22:22,942 INFO 3318922:repeater task quick_wh300 starting 3.9 seconds late 2024-11-24 21:22:29,772 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 21:22:29,772 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:22:14 2024-11-24 21:22:29,772 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:24:19 2024-11-24 21:22:29,772 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2024-11-24 21:22:30,827 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:22:30,827 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:22:14 2024-11-24 21:22:30,827 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:24:19 2024-11-24 21:24:21,148 INFO 3318922:repeater task lastensq_wh300 starting 2.1 seconds late 2024-11-24 21:24:24,430 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:24:24,430 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:24:14 2024-11-24 21:24:24,430 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:24:24,431 INFO 3318922:repeater task quick_wh300 starting 5.4 seconds late 2024-11-24 21:24:31,557 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 21:24:31,558 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:24:14 2024-11-24 21:24:31,558 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:24:31,558 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2024-11-24 21:24:32,621 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:24:32,621 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:24:14 2024-11-24 21:24:32,621 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:26:19 2024-11-24 21:26:17,924 INFO 3318922:repeater task plot_tsstats_wh300 starting 2.9 seconds late 2024-11-24 21:26:19,474 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:26:19,474 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.6 seconds 2024-11-24 21:26:19,474 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:19,474 INFO 3318922:repeater task grab_lastpings_wh300 starting 4.5 seconds late 2024-11-24 21:26:19,637 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-24 21:26:19,637 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:19,637 INFO 3318922:repeater task beamdiagnostics_wh300 starting 4.7 seconds late 2024-11-24 21:26:21,355 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 21:26:21,355 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:21,356 INFO 3318922:repeater task beamdiagnostics_wh300 starting 6.4 seconds late 2024-11-24 21:26:23,086 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 21:26:23,086 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:23,086 INFO 3318922:repeater task lastensq_os150nb starting 4.1 seconds late 2024-11-24 21:26:25,920 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-24 21:26:25,920 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:26:14 2024-11-24 21:26:25,920 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:31:19 2024-11-24 21:26:25,920 INFO 3318922:repeater task quick_os150nb starting 6.9 seconds late 2024-11-24 21:26:30,173 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 21:26:30,173 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:26:14 2024-11-24 21:26:30,173 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:31:19 2024-11-24 21:26:30,174 INFO 3318922:repeater task lastfew_vec_os150nb starting 11.2 seconds late 2024-11-24 21:26:31,207 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:26:31,207 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:26:14 2024-11-24 21:26:31,208 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:31:19 2024-11-24 21:26:31,208 INFO 3318922:repeater task 3dayplots_os150nb starting 11.2 seconds late 2024-11-24 21:26:38,168 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.0 seconds 2024-11-24 21:26:38,169 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:31:19 2024-11-24 21:26:38,169 INFO 3318922:repeater task plot_tsstats_os150nb starting 23.2 seconds late 2024-11-24 21:26:39,472 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:26:39,472 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.3 seconds 2024-11-24 21:26:39,472 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:39,473 INFO 3318922:repeater task grab_lastpings_os150nb starting 24.5 seconds late 2024-11-24 21:26:39,637 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-24 21:26:39,637 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:39,637 INFO 3318922:repeater task beamdiagnostics_os150nb starting 24.7 seconds late 2024-11-24 21:26:41,391 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.8 seconds 2024-11-24 21:26:41,392 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:41,392 INFO 3318922:repeater task beamdiagnostics_os150nb starting 26.4 seconds late 2024-11-24 21:26:43,118 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 21:26:43,118 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:43,118 INFO 3318922:repeater task lastensq_os38nb starting 22.1 seconds late 2024-11-24 21:26:45,777 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-24 21:26:45,778 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:26:16 2024-11-24 21:26:45,778 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:31:21 2024-11-24 21:26:45,778 INFO 3318922:repeater task quick_os38nb starting 24.8 seconds late 2024-11-24 21:26:50,245 INFO 3318922:repeater task quick_os38nb ended after 4.5 seconds 2024-11-24 21:26:50,246 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:26:16 2024-11-24 21:26:50,246 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:31:21 2024-11-24 21:26:50,246 INFO 3318922:repeater task lastfew_vec_os38nb starting 29.2 seconds late 2024-11-24 21:26:51,253 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:26:51,253 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:26:16 2024-11-24 21:26:51,253 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:31:21 2024-11-24 21:26:51,254 INFO 3318922:repeater task 3dayplots_os38nb starting 31.3 seconds late 2024-11-24 21:26:57,977 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.7 seconds 2024-11-24 21:26:57,977 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:31:19 2024-11-24 21:26:57,977 INFO 3318922:repeater task plot_tsstats_os38nb starting 43.0 seconds late 2024-11-24 21:26:59,309 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:26:59,309 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-24 21:26:59,309 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:59,310 INFO 3318922:repeater task grab_lastpings_os38nb starting 44.3 seconds late 2024-11-24 21:26:59,470 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-24 21:26:59,470 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:26:59,470 INFO 3318922:repeater task beamdiagnostics_os38nb starting 44.5 seconds late 2024-11-24 21:27:01,186 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 21:27:01,186 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:27:01,187 INFO 3318922:repeater task beamdiagnostics_os38nb starting 46.2 seconds late 2024-11-24 21:27:02,880 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 21:27:02,880 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:41:14 2024-11-24 21:27:02,881 INFO 3318922:repeater task daily_netcdf starting 42.9 seconds late 2024-11-24 21:27:03,712 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 21:27:03,712 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:27:03,713 INFO 3318922:repeater task ggatime_diagnostics starting 48.7 seconds late 2024-11-24 21:27:11,416 INFO 3318922:repeater task ggatime_diagnostics ended after 7.7 seconds 2024-11-24 21:27:11,416 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:27:11,417 INFO 3318922:repeater task ggatime_diagnostics starting 56.4 seconds late 2024-11-24 21:27:18,847 INFO 3318922:repeater task ggatime_diagnostics ended after 7.4 seconds 2024-11-24 21:27:18,847 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:27:18,848 INFO 3318922:repeater task ggatime_diagnostics starting 63.9 seconds late 2024-11-24 21:27:26,443 INFO 3318922:repeater task ggatime_diagnostics ended after 7.6 seconds 2024-11-24 21:27:26,444 INFO 3318922:repeater task ggatime_diagnostics next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:27:26,444 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 71.3 seconds late 2024-11-24 21:27:28,222 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-24 21:27:28,222 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.8 seconds 2024-11-24 21:27:28,222 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/24 21:41:15 2024-11-24 21:27:28,223 INFO 3318922:repeater task plot_hbins starting 73.1 seconds late 2024-11-24 21:27:32,375 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-24 21:27:32,375 INFO 3318922:repeater task plot_hbins ended after 4.2 seconds 2024-11-24 21:27:32,375 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/24 21:41:15 2024-11-24 21:27:32,375 INFO 3318922:repeater task report_generator starting 77.3 seconds late 2024-11-24 21:27:37,388 INFO 3318922:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2421/reports/db_timerange --noshow /home/data/SR2421 1.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2421/reports /home/data/SR2421/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os150nb/png_archive/ /home/data/SR2421/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/os38nb/png_archive/ /home/data/SR2421/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2421/proc/wh300/png_archive/ /home/data/SR2421/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2421/reports/uhdas_overview.txt /home/data/SR2421 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2421/reports/ADCP_settings.txt /home/data/SR2421 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2421/reports/ADCP_cals.txt /home/data/SR2421 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_proc.toml /home/data/SR2421/reports/SR2421_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SR2421/raw/config/SR2421_sensor.toml /home/data/SR2421/reports/SR2421_sensor.toml.txt 4.0 seconds elapsed html page starts here: /home/data/SR2421/reports/index.html 2024-11-24 21:27:37,388 INFO 3318922:repeater task report_generator ended after 5.0 seconds 2024-11-24 21:27:37,388 INFO 3318922:repeater task report_generator next time_to_run is 2024/11/24 22:26:15 2024-11-24 21:27:42,394 INFO 3318922:repeater task noisy_rawtemps_os150 starting 82.4 seconds late 2024-11-24 21:27:45,298 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2024-11-24 21:27:45,298 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:27:45,298 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 85.3 seconds late 2024-11-24 21:27:48,208 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 21:27:48,208 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:27:48,208 INFO 3318922:repeater task noisy_rawtemps_os38 starting 88.2 seconds late 2024-11-24 21:27:51,568 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-24 21:27:51,569 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:27:51,569 INFO 3318922:repeater task lastensq_wh300 starting 92.6 seconds late 2024-11-24 21:27:54,891 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:27:54,892 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:26:14 2024-11-24 21:27:54,892 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:28:19 2024-11-24 21:27:54,892 INFO 3318922:repeater task quick_wh300 starting 95.9 seconds late 2024-11-24 21:28:01,752 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 21:28:01,752 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:26:14 2024-11-24 21:28:01,752 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:28:19 2024-11-24 21:28:01,753 INFO 3318922:repeater task lastfew_vec_wh300 starting 102.8 seconds late 2024-11-24 21:28:02,805 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:28:02,806 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:26:14 2024-11-24 21:28:02,806 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:28:19 2024-11-24 21:28:02,806 INFO 3318922:repeater task 3dayplots_wh300 starting 102.8 seconds late 2024-11-24 21:28:09,946 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.1 seconds 2024-11-24 21:28:09,946 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:31:19 2024-11-24 21:28:19,968 INFO 3318922:repeater task lastensq_wh300 starting 1.0 seconds late 2024-11-24 21:28:23,251 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:28:23,251 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:28:14 2024-11-24 21:28:23,252 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:30:19 2024-11-24 21:28:23,252 INFO 3318922:repeater task quick_wh300 starting 4.3 seconds late 2024-11-24 21:28:30,670 INFO 3318922:repeater task quick_wh300 ended after 7.4 seconds 2024-11-24 21:28:30,670 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:28:14 2024-11-24 21:28:30,671 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:30:19 2024-11-24 21:28:30,671 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2024-11-24 21:28:31,736 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:28:31,737 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:28:14 2024-11-24 21:28:31,737 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:30:19 2024-11-24 21:30:22,048 INFO 3318922:repeater task lastensq_wh300 starting 3.0 seconds late 2024-11-24 21:30:25,387 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:30:25,387 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:30:14 2024-11-24 21:30:25,387 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:32:19 2024-11-24 21:30:25,388 INFO 3318922:repeater task quick_wh300 starting 6.4 seconds late 2024-11-24 21:30:32,393 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 21:30:32,393 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:30:14 2024-11-24 21:30:32,393 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:32:19 2024-11-24 21:30:32,393 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2024-11-24 21:30:33,466 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:30:33,466 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:30:14 2024-11-24 21:30:33,466 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:32:19 2024-11-24 21:31:23,597 INFO 3318922:repeater task 3dayplots_wh300 starting 3.6 seconds late 2024-11-24 21:31:30,899 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.3 seconds 2024-11-24 21:31:30,899 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:31:30,900 INFO 3318922:repeater task lastensq_os150nb starting 11.9 seconds late 2024-11-24 21:31:33,723 INFO 3318922:repeater task lastensq_os150nb ended after 2.8 seconds 2024-11-24 21:31:33,723 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:31:14 2024-11-24 21:31:33,723 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:31:33,723 INFO 3318922:repeater task quick_os150nb starting 14.7 seconds late 2024-11-24 21:31:37,991 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 21:31:37,992 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:31:14 2024-11-24 21:31:37,992 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:31:37,992 INFO 3318922:repeater task lastfew_vec_os150nb starting 19.0 seconds late 2024-11-24 21:31:39,035 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:31:39,035 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:31:14 2024-11-24 21:31:39,035 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:31:39,036 INFO 3318922:repeater task 3dayplots_os150nb starting 19.1 seconds late 2024-11-24 21:31:45,978 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 21:31:45,978 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:31:45,980 INFO 3318922:repeater task lastensq_os38nb starting 25.0 seconds late 2024-11-24 21:31:48,650 INFO 3318922:repeater task lastensq_os38nb ended after 2.7 seconds 2024-11-24 21:31:48,650 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:31:16 2024-11-24 21:31:48,650 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:36:21 2024-11-24 21:31:48,651 INFO 3318922:repeater task quick_os38nb starting 27.7 seconds late 2024-11-24 21:31:52,983 INFO 3318922:repeater task quick_os38nb ended after 4.3 seconds 2024-11-24 21:31:52,983 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:31:16 2024-11-24 21:31:52,983 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:36:21 2024-11-24 21:31:52,983 INFO 3318922:repeater task lastfew_vec_os38nb starting 32.0 seconds late 2024-11-24 21:31:54,017 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:31:54,017 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:31:16 2024-11-24 21:31:54,017 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:36:21 2024-11-24 21:31:54,017 INFO 3318922:repeater task 3dayplots_os38nb starting 34.0 seconds late 2024-11-24 21:32:00,886 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.9 seconds 2024-11-24 21:32:00,886 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:32:20,941 INFO 3318922:repeater task lastensq_wh300 starting 1.9 seconds late 2024-11-24 21:32:24,538 INFO 3318922:repeater task lastensq_wh300 ended after 3.6 seconds 2024-11-24 21:32:24,539 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:32:14 2024-11-24 21:32:24,539 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:34:19 2024-11-24 21:32:24,539 INFO 3318922:repeater task quick_wh300 starting 5.5 seconds late 2024-11-24 21:32:31,614 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 21:32:31,614 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:32:14 2024-11-24 21:32:31,615 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:34:19 2024-11-24 21:32:31,615 INFO 3318922:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2024-11-24 21:32:32,681 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:32:32,682 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:32:14 2024-11-24 21:32:32,682 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:34:19 2024-11-24 21:34:22,993 INFO 3318922:repeater task lastensq_wh300 starting 4.0 seconds late 2024-11-24 21:34:26,319 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:34:26,319 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:34:14 2024-11-24 21:34:26,319 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:34:26,319 INFO 3318922:repeater task quick_wh300 starting 7.3 seconds late 2024-11-24 21:34:33,427 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 21:34:33,428 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:34:14 2024-11-24 21:34:33,428 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:34:33,428 INFO 3318922:repeater task lastfew_vec_wh300 starting 14.4 seconds late 2024-11-24 21:34:34,486 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:34:34,486 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:34:14 2024-11-24 21:34:34,486 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:36:19 2024-11-24 21:36:19,784 INFO 3318922:repeater task lastensq_wh300 starting 0.8 seconds late 2024-11-24 21:36:23,136 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 21:36:23,136 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:36:14 2024-11-24 21:36:23,137 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:38:19 2024-11-24 21:36:23,137 INFO 3318922:repeater task quick_wh300 starting 4.1 seconds late 2024-11-24 21:36:30,066 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 21:36:30,066 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:36:14 2024-11-24 21:36:30,067 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:38:19 2024-11-24 21:36:30,067 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2024-11-24 21:36:31,137 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:36:31,138 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:36:14 2024-11-24 21:36:31,138 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:38:19 2024-11-24 21:36:31,138 INFO 3318922:repeater task 3dayplots_wh300 starting 11.2 seconds late 2024-11-24 21:36:38,305 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.2 seconds 2024-11-24 21:36:38,305 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:41:19 2024-11-24 21:36:38,306 INFO 3318922:repeater task lastensq_os150nb starting 19.3 seconds late 2024-11-24 21:36:40,945 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-24 21:36:40,946 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:36:14 2024-11-24 21:36:40,946 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:41:19 2024-11-24 21:36:40,946 INFO 3318922:repeater task quick_os150nb starting 21.9 seconds late 2024-11-24 21:36:45,397 INFO 3318922:repeater task quick_os150nb ended after 4.5 seconds 2024-11-24 21:36:45,397 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:36:14 2024-11-24 21:36:45,398 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:41:19 2024-11-24 21:36:45,398 INFO 3318922:repeater task lastfew_vec_os150nb starting 26.4 seconds late 2024-11-24 21:36:46,413 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:36:46,413 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:36:14 2024-11-24 21:36:46,413 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:41:19 2024-11-24 21:36:46,414 INFO 3318922:repeater task 3dayplots_os150nb starting 26.4 seconds late 2024-11-24 21:36:53,307 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 21:36:53,307 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:41:19 2024-11-24 21:36:53,309 INFO 3318922:repeater task lastensq_os38nb starting 32.3 seconds late 2024-11-24 21:36:55,806 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-24 21:36:55,807 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:36:16 2024-11-24 21:36:55,807 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:41:21 2024-11-24 21:36:55,807 INFO 3318922:repeater task quick_os38nb starting 34.8 seconds late 2024-11-24 21:37:00,407 INFO 3318922:repeater task quick_os38nb ended after 4.6 seconds 2024-11-24 21:37:00,407 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:36:16 2024-11-24 21:37:00,407 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:41:21 2024-11-24 21:37:00,408 INFO 3318922:repeater task lastfew_vec_os38nb starting 39.4 seconds late 2024-11-24 21:37:01,418 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:37:01,419 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:36:16 2024-11-24 21:37:01,419 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:41:21 2024-11-24 21:37:01,419 INFO 3318922:repeater task 3dayplots_os38nb starting 41.4 seconds late 2024-11-24 21:37:08,193 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.8 seconds 2024-11-24 21:37:08,193 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:41:19 2024-11-24 21:37:08,194 INFO 3318922:repeater task daily_netcdf starting 48.2 seconds late 2024-11-24 21:37:09,086 INFO 3318922:repeater task daily_netcdf ended after 0.9 seconds 2024-11-24 21:37:09,086 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:37:14,093 INFO 3318922:repeater task noisy_rawtemps_os150 starting 54.1 seconds late 2024-11-24 21:37:16,977 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2024-11-24 21:37:16,977 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:37:16,977 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 57.0 seconds late 2024-11-24 21:37:19,902 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2024-11-24 21:37:19,902 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:37:19,902 INFO 3318922:repeater task noisy_rawtemps_os38 starting 59.9 seconds late 2024-11-24 21:37:23,276 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-24 21:37:23,277 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:38:19,079 INFO 3318922:repeater task lastensq_wh300 starting 0.1 seconds late 2024-11-24 21:38:22,333 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:38:22,333 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:38:14 2024-11-24 21:38:22,333 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:40:19 2024-11-24 21:38:22,334 INFO 3318922:repeater task quick_wh300 starting 3.3 seconds late 2024-11-24 21:38:29,041 INFO 3318922:repeater task quick_wh300 ended after 6.7 seconds 2024-11-24 21:38:29,041 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:38:14 2024-11-24 21:38:29,041 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:40:19 2024-11-24 21:38:29,042 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2024-11-24 21:38:30,094 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:38:30,094 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:38:14 2024-11-24 21:38:30,094 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:40:19 2024-11-24 21:40:20,411 INFO 3318922:repeater task lastensq_wh300 starting 1.4 seconds late 2024-11-24 21:40:23,834 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 21:40:23,834 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:40:14 2024-11-24 21:40:23,834 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:42:19 2024-11-24 21:40:23,835 INFO 3318922:repeater task quick_wh300 starting 4.8 seconds late 2024-11-24 21:40:30,718 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 21:40:30,719 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:40:14 2024-11-24 21:40:30,719 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:42:19 2024-11-24 21:40:30,719 INFO 3318922:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2024-11-24 21:40:31,764 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 21:40:31,764 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:40:14 2024-11-24 21:40:31,764 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:42:19 2024-11-24 21:41:16,877 INFO 3318922:repeater task plot_tsstats_wh300 starting 1.9 seconds late 2024-11-24 21:41:18,430 INFO 3318922:repeater task plot_tsstats_wh300 output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:41:18,431 INFO 3318922:repeater task plot_tsstats_wh300 ended after 1.6 seconds 2024-11-24 21:41:18,431 INFO 3318922:repeater task plot_tsstats_wh300 next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:18,431 INFO 3318922:repeater task grab_lastpings_wh300 starting 3.5 seconds late 2024-11-24 21:41:18,590 INFO 3318922:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2024-11-24 21:41:18,591 INFO 3318922:repeater task grab_lastpings_wh300 next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:18,591 INFO 3318922:repeater task beamdiagnostics_wh300 starting 3.6 seconds late 2024-11-24 21:41:20,336 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 21:41:20,336 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:20,336 INFO 3318922:repeater task beamdiagnostics_wh300 starting 5.4 seconds late 2024-11-24 21:41:22,067 INFO 3318922:repeater task beamdiagnostics_wh300 ended after 1.7 seconds 2024-11-24 21:41:22,068 INFO 3318922:repeater task beamdiagnostics_wh300 next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:22,068 INFO 3318922:repeater task lastensq_os150nb starting 3.1 seconds late 2024-11-24 21:41:24,754 INFO 3318922:repeater task lastensq_os150nb ended after 2.7 seconds 2024-11-24 21:41:24,754 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:41:14 2024-11-24 21:41:24,755 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:41:24,755 INFO 3318922:repeater task quick_os150nb starting 5.8 seconds late 2024-11-24 21:41:29,078 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 21:41:29,078 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:41:14 2024-11-24 21:41:29,078 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:41:29,078 INFO 3318922:repeater task lastfew_vec_os150nb starting 10.1 seconds late 2024-11-24 21:41:30,107 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:41:30,107 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:41:14 2024-11-24 21:41:30,107 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:41:30,108 INFO 3318922:repeater task 3dayplots_os150nb starting 10.1 seconds late 2024-11-24 21:41:37,149 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.0 seconds 2024-11-24 21:41:37,149 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:41:37,150 INFO 3318922:repeater task plot_tsstats_os150nb starting 22.2 seconds late 2024-11-24 21:41:38,503 INFO 3318922:repeater task plot_tsstats_os150nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:41:38,503 INFO 3318922:repeater task plot_tsstats_os150nb ended after 1.4 seconds 2024-11-24 21:41:38,503 INFO 3318922:repeater task plot_tsstats_os150nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:38,503 INFO 3318922:repeater task grab_lastpings_os150nb starting 23.5 seconds late 2024-11-24 21:41:38,663 INFO 3318922:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2024-11-24 21:41:38,663 INFO 3318922:repeater task grab_lastpings_os150nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:38,663 INFO 3318922:repeater task beamdiagnostics_os150nb starting 23.7 seconds late 2024-11-24 21:41:40,355 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 21:41:40,355 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:40,356 INFO 3318922:repeater task beamdiagnostics_os150nb starting 25.4 seconds late 2024-11-24 21:41:42,068 INFO 3318922:repeater task beamdiagnostics_os150nb ended after 1.7 seconds 2024-11-24 21:41:42,068 INFO 3318922:repeater task beamdiagnostics_os150nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:42,068 INFO 3318922:repeater task lastensq_os38nb starting 21.1 seconds late 2024-11-24 21:41:44,646 INFO 3318922:repeater task lastensq_os38nb ended after 2.6 seconds 2024-11-24 21:41:44,647 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:41:16 2024-11-24 21:41:44,647 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:46:21 2024-11-24 21:41:44,647 INFO 3318922:repeater task quick_os38nb starting 23.6 seconds late 2024-11-24 21:41:49,148 INFO 3318922:repeater task quick_os38nb ended after 4.5 seconds 2024-11-24 21:41:49,148 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:41:16 2024-11-24 21:41:49,148 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:46:21 2024-11-24 21:41:49,149 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.1 seconds late 2024-11-24 21:41:50,166 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:41:50,166 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:41:16 2024-11-24 21:41:50,166 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:46:21 2024-11-24 21:41:50,166 INFO 3318922:repeater task 3dayplots_os38nb starting 30.2 seconds late 2024-11-24 21:41:56,988 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.8 seconds 2024-11-24 21:41:56,989 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:41:56,989 INFO 3318922:repeater task plot_tsstats_os38nb starting 42.0 seconds late 2024-11-24 21:41:58,321 INFO 3318922:repeater task plot_tsstats_os38nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2024-11-24 21:41:58,321 INFO 3318922:repeater task plot_tsstats_os38nb ended after 1.3 seconds 2024-11-24 21:41:58,321 INFO 3318922:repeater task plot_tsstats_os38nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:58,322 INFO 3318922:repeater task grab_lastpings_os38nb starting 43.3 seconds late 2024-11-24 21:41:58,486 INFO 3318922:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2024-11-24 21:41:58,487 INFO 3318922:repeater task grab_lastpings_os38nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:41:58,487 INFO 3318922:repeater task beamdiagnostics_os38nb starting 43.5 seconds late 2024-11-24 21:42:00,194 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.7 seconds 2024-11-24 21:42:00,194 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:42:00,195 INFO 3318922:repeater task beamdiagnostics_os38nb starting 45.2 seconds late 2024-11-24 21:42:01,990 INFO 3318922:repeater task beamdiagnostics_os38nb ended after 1.8 seconds 2024-11-24 21:42:01,991 INFO 3318922:repeater task beamdiagnostics_os38nb next time_to_run is 2024/11/24 21:56:14 2024-11-24 21:42:01,992 INFO 3318922:repeater task hbinhcorrstats_hemisphere starting 46.9 seconds late 2024-11-24 21:42:03,847 INFO 3318922:repeater task hbinhcorrstats_hemisphere output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2024-11-24 21:42:03,847 INFO 3318922:repeater task hbinhcorrstats_hemisphere ended after 1.9 seconds 2024-11-24 21:42:03,847 INFO 3318922:repeater task hbinhcorrstats_hemisphere next time_to_run is 2024/11/24 21:56:15 2024-11-24 21:42:03,848 INFO 3318922:repeater task plot_hbins starting 48.7 seconds late 2024-11-24 21:42:07,979 INFO 3318922:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2024-11-24 21:42:07,979 INFO 3318922:repeater task plot_hbins ended after 4.1 seconds 2024-11-24 21:42:07,979 INFO 3318922:repeater task plot_hbins next time_to_run is 2024/11/24 21:56:15 2024-11-24 21:42:12,988 INFO 3318922:repeater task 3dayplots_wh300 starting 53.0 seconds late 2024-11-24 21:42:20,362 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.4 seconds 2024-11-24 21:42:20,362 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:42:25,374 INFO 3318922:repeater task lastensq_wh300 starting 6.4 seconds late 2024-11-24 21:42:28,603 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 21:42:28,603 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:42:14 2024-11-24 21:42:28,603 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:44:19 2024-11-24 21:42:28,604 INFO 3318922:repeater task quick_wh300 starting 9.6 seconds late 2024-11-24 21:42:35,566 INFO 3318922:repeater task quick_wh300 ended after 7.0 seconds 2024-11-24 21:42:35,566 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:42:14 2024-11-24 21:42:35,566 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:44:19 2024-11-24 21:42:35,567 INFO 3318922:repeater task lastfew_vec_wh300 starting 16.6 seconds late 2024-11-24 21:42:36,612 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 21:42:36,612 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:42:14 2024-11-24 21:42:36,612 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:44:19 2024-11-24 21:44:21,916 INFO 3318922:repeater task lastensq_wh300 starting 2.9 seconds late 2024-11-24 21:44:25,299 INFO 3318922:repeater task lastensq_wh300 ended after 3.4 seconds 2024-11-24 21:44:25,300 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:44:14 2024-11-24 21:44:25,300 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:44:25,300 INFO 3318922:repeater task quick_wh300 starting 6.3 seconds late 2024-11-24 21:44:32,212 INFO 3318922:repeater task quick_wh300 ended after 6.9 seconds 2024-11-24 21:44:32,213 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:44:14 2024-11-24 21:44:32,213 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:44:32,213 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2024-11-24 21:44:33,289 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:44:33,290 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:44:14 2024-11-24 21:44:33,290 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:46:19 2024-11-24 21:46:23,695 INFO 3318922:repeater task noisy_rawtemps_os150 starting 3.7 seconds late 2024-11-24 21:46:26,587 INFO 3318922:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2024-11-24 21:46:26,587 INFO 3318922:repeater task noisy_rawtemps_os150 next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:46:26,587 INFO 3318922:repeater task noisy_rawtemps_wh300 starting 6.6 seconds late 2024-11-24 21:46:29,541 INFO 3318922:repeater task noisy_rawtemps_wh300 ended after 3.0 seconds 2024-11-24 21:46:29,541 INFO 3318922:repeater task noisy_rawtemps_wh300 next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:46:29,541 INFO 3318922:repeater task noisy_rawtemps_os38 starting 9.6 seconds late 2024-11-24 21:46:32,904 INFO 3318922:repeater task noisy_rawtemps_os38 ended after 3.4 seconds 2024-11-24 21:46:32,904 INFO 3318922:repeater task noisy_rawtemps_os38 next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:46:32,904 INFO 3318922:repeater task lastensq_wh300 starting 13.9 seconds late 2024-11-24 21:46:36,082 INFO 3318922:repeater task lastensq_wh300 ended after 3.2 seconds 2024-11-24 21:46:36,082 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:46:14 2024-11-24 21:46:36,082 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:48:19 2024-11-24 21:46:36,083 INFO 3318922:repeater task quick_wh300 starting 17.1 seconds late 2024-11-24 21:46:43,188 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 21:46:43,189 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:46:14 2024-11-24 21:46:43,189 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:48:19 2024-11-24 21:46:43,189 INFO 3318922:repeater task lastfew_vec_wh300 starting 24.2 seconds late 2024-11-24 21:46:44,240 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:46:44,241 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:46:14 2024-11-24 21:46:44,241 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:48:19 2024-11-24 21:46:44,242 INFO 3318922:repeater task 3dayplots_wh300 starting 24.3 seconds late 2024-11-24 21:46:51,526 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.3 seconds 2024-11-24 21:46:51,526 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:51:19 2024-11-24 21:46:51,527 INFO 3318922:repeater task lastensq_os150nb starting 32.5 seconds late 2024-11-24 21:46:54,165 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-24 21:46:54,165 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:46:14 2024-11-24 21:46:54,165 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:51:19 2024-11-24 21:46:54,165 INFO 3318922:repeater task quick_os150nb starting 35.2 seconds late 2024-11-24 21:46:58,454 INFO 3318922:repeater task quick_os150nb ended after 4.3 seconds 2024-11-24 21:46:58,454 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:46:14 2024-11-24 21:46:58,454 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:51:19 2024-11-24 21:46:58,454 INFO 3318922:repeater task lastfew_vec_os150nb starting 39.5 seconds late 2024-11-24 21:46:59,469 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:46:59,469 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:46:14 2024-11-24 21:46:59,469 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:51:19 2024-11-24 21:46:59,470 INFO 3318922:repeater task 3dayplots_os150nb starting 39.5 seconds late 2024-11-24 21:47:06,495 INFO 3318922:repeater task 3dayplots_os150nb ended after 7.0 seconds 2024-11-24 21:47:06,495 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:51:19 2024-11-24 21:47:06,496 INFO 3318922:repeater task lastensq_os38nb starting 45.5 seconds late 2024-11-24 21:47:08,969 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-24 21:47:08,969 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:46:16 2024-11-24 21:47:08,969 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:51:21 2024-11-24 21:47:08,970 INFO 3318922:repeater task quick_os38nb starting 48.0 seconds late 2024-11-24 21:47:13,287 INFO 3318922:repeater task quick_os38nb ended after 4.3 seconds 2024-11-24 21:47:13,287 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:46:16 2024-11-24 21:47:13,287 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:51:21 2024-11-24 21:47:13,287 INFO 3318922:repeater task lastfew_vec_os38nb starting 52.3 seconds late 2024-11-24 21:47:14,337 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.1 seconds 2024-11-24 21:47:14,338 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:46:16 2024-11-24 21:47:14,338 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:51:21 2024-11-24 21:47:14,338 INFO 3318922:repeater task 3dayplots_os38nb starting 54.4 seconds late 2024-11-24 21:47:21,189 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.9 seconds 2024-11-24 21:47:21,190 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:51:19 2024-11-24 21:47:21,191 INFO 3318922:repeater task daily_netcdf starting 61.2 seconds late 2024-11-24 21:47:22,009 INFO 3318922:repeater task daily_netcdf ended after 0.8 seconds 2024-11-24 21:47:22,009 INFO 3318922:repeater task daily_netcdf next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:48:22,176 INFO 3318922:repeater task lastensq_wh300 starting 3.2 seconds late 2024-11-24 21:48:25,480 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:48:25,481 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:48:14 2024-11-24 21:48:25,481 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:50:19 2024-11-24 21:48:25,481 INFO 3318922:repeater task quick_wh300 starting 6.5 seconds late 2024-11-24 21:48:32,600 INFO 3318922:repeater task quick_wh300 ended after 7.1 seconds 2024-11-24 21:48:32,601 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:48:14 2024-11-24 21:48:32,601 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:50:19 2024-11-24 21:48:32,601 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.6 seconds late 2024-11-24 21:48:33,651 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 21:48:33,682 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:48:14 2024-11-24 21:48:33,699 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:50:19 2024-11-24 21:50:19,369 INFO 3318922:repeater task lastensq_wh300 starting 0.4 seconds late 2024-11-24 21:50:22,703 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:50:22,704 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:50:14 2024-11-24 21:50:22,704 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:52:19 2024-11-24 21:50:22,704 INFO 3318922:repeater task quick_wh300 starting 3.7 seconds late 2024-11-24 21:50:29,514 INFO 3318922:repeater task quick_wh300 ended after 6.8 seconds 2024-11-24 21:50:29,515 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:50:14 2024-11-24 21:50:29,515 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:52:19 2024-11-24 21:50:29,515 INFO 3318922:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2024-11-24 21:50:30,584 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.1 seconds 2024-11-24 21:50:30,585 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:50:14 2024-11-24 21:50:30,585 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:52:19 2024-11-24 21:51:20,721 INFO 3318922:repeater task 3dayplots_wh300 starting 0.7 seconds late 2024-11-24 21:51:27,995 INFO 3318922:repeater task 3dayplots_wh300 ended after 7.3 seconds 2024-11-24 21:51:27,995 INFO 3318922:repeater task 3dayplots_wh300 next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:51:27,997 INFO 3318922:repeater task lastensq_os150nb starting 9.0 seconds late 2024-11-24 21:51:30,615 INFO 3318922:repeater task lastensq_os150nb ended after 2.6 seconds 2024-11-24 21:51:30,615 INFO 3318922:repeater task lastensq_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:51:14 2024-11-24 21:51:30,615 INFO 3318922:repeater task lastensq_os150nb next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:51:30,616 INFO 3318922:repeater task quick_os150nb starting 11.6 seconds late 2024-11-24 21:51:34,853 INFO 3318922:repeater task quick_os150nb ended after 4.2 seconds 2024-11-24 21:51:34,854 INFO 3318922:repeater task quick_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:51:14 2024-11-24 21:51:34,854 INFO 3318922:repeater task quick_os150nb next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:51:34,854 INFO 3318922:repeater task lastfew_vec_os150nb starting 15.9 seconds late 2024-11-24 21:51:35,897 INFO 3318922:repeater task lastfew_vec_os150nb ended after 1.0 seconds 2024-11-24 21:51:35,897 INFO 3318922:repeater task lastfew_vec_os150nb: Time from /home/data/SR2421/proc/os150nb/load/wrote_ens_time: 2024/11/24 21:51:14 2024-11-24 21:51:35,897 INFO 3318922:repeater task lastfew_vec_os150nb next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:51:35,898 INFO 3318922:repeater task 3dayplots_os150nb starting 15.9 seconds late 2024-11-24 21:51:42,804 INFO 3318922:repeater task 3dayplots_os150nb ended after 6.9 seconds 2024-11-24 21:51:42,804 INFO 3318922:repeater task 3dayplots_os150nb next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:51:42,805 INFO 3318922:repeater task lastensq_os38nb starting 21.8 seconds late 2024-11-24 21:51:45,270 INFO 3318922:repeater task lastensq_os38nb ended after 2.5 seconds 2024-11-24 21:51:45,271 INFO 3318922:repeater task lastensq_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:51:16 2024-11-24 21:51:45,271 INFO 3318922:repeater task lastensq_os38nb next time_to_run is 2024/11/24 21:56:21 2024-11-24 21:51:45,271 INFO 3318922:repeater task quick_os38nb starting 24.3 seconds late 2024-11-24 21:51:49,912 INFO 3318922:repeater task quick_os38nb ended after 4.6 seconds 2024-11-24 21:51:49,912 INFO 3318922:repeater task quick_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:51:16 2024-11-24 21:51:49,912 INFO 3318922:repeater task quick_os38nb next time_to_run is 2024/11/24 21:56:21 2024-11-24 21:51:49,913 INFO 3318922:repeater task lastfew_vec_os38nb starting 28.9 seconds late 2024-11-24 21:51:50,947 INFO 3318922:repeater task lastfew_vec_os38nb ended after 1.0 seconds 2024-11-24 21:51:50,947 INFO 3318922:repeater task lastfew_vec_os38nb: Time from /home/data/SR2421/proc/os38nb/load/wrote_ens_time: 2024/11/24 21:51:16 2024-11-24 21:51:50,948 INFO 3318922:repeater task lastfew_vec_os38nb next time_to_run is 2024/11/24 21:56:21 2024-11-24 21:51:50,948 INFO 3318922:repeater task 3dayplots_os38nb starting 31.0 seconds late 2024-11-24 21:51:57,860 INFO 3318922:repeater task 3dayplots_os38nb ended after 6.9 seconds 2024-11-24 21:51:57,860 INFO 3318922:repeater task 3dayplots_os38nb next time_to_run is 2024/11/24 21:56:19 2024-11-24 21:52:22,924 INFO 3318922:repeater task lastensq_wh300 starting 3.9 seconds late 2024-11-24 21:52:26,183 INFO 3318922:repeater task lastensq_wh300 ended after 3.3 seconds 2024-11-24 21:52:26,183 INFO 3318922:repeater task lastensq_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:52:14 2024-11-24 21:52:26,183 INFO 3318922:repeater task lastensq_wh300 next time_to_run is 2024/11/24 21:54:19 2024-11-24 21:52:26,184 INFO 3318922:repeater task quick_wh300 starting 7.2 seconds late 2024-11-24 21:52:32,917 INFO 3318922:repeater task quick_wh300 ended after 6.7 seconds 2024-11-24 21:52:32,917 INFO 3318922:repeater task quick_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:52:14 2024-11-24 21:52:32,917 INFO 3318922:repeater task quick_wh300 next time_to_run is 2024/11/24 21:54:19 2024-11-24 21:52:32,917 INFO 3318922:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2024-11-24 21:52:33,962 INFO 3318922:repeater task lastfew_vec_wh300 ended after 1.0 seconds 2024-11-24 21:52:33,962 INFO 3318922:repeater task lastfew_vec_wh300: Time from /home/data/SR2421/proc/wh300/load/wrote_ens_time: 2024/11/24 21:52:14 2024-11-24 21:52:33,962 INFO 3318922:repeater task lastfew_vec_wh300 next time_to_run is 2024/11/24 21:54:19 2024-11-24 21:53:20,733 INFO 3318922:incr_class translate exiting, gps /home/data/SR2421/rbin/hemisphere 2024-11-24 21:53:20,733 INFO 3318922:incr_class translate exiting, sea /home/data/SR2421/rbin/seapath_mgc 2024-11-24 21:53:20,734 INFO 3318922:incr_class translate exiting, gph /home/data/SR2421/rbin/hemisphere 2024-11-24 21:53:20,734 INFO 3318922:incr_class translate exiting, gps /home/data/SR2421/rbin/seapath_mgc 2024-11-24 21:53:20,734 INFO 3318922:incr_class translate exiting, hdg /home/data/SR2421/rbin/seapath_mgc 2024-11-24 21:53:24,113 INFO 3318922:DAS_while_logging cleanup_figs not_logging 2024-11-24 21:53:24,153 INFO 3318922:DAS_while_logging Exiting DAS_while_logging.py