2023-11-16 09:31:45,476 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.5 seconds late 2023-11-16 09:31:46,080 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 09:31:46,081 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:31:20 2023-11-16 09:31:46,081 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 09:36:25 2023-11-16 09:31:46,081 INFO 1947952:repeater task 3dayplots_os38nb starting 22.0 seconds late 2023-11-16 09:31:49,562 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 09:31:49,562 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 09:36:24 2023-11-16 09:32:24,660 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-16 09:32:27,346 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 09:32:27,346 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:32:18 2023-11-16 09:32:27,346 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:34:23 2023-11-16 09:32:27,347 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-16 09:32:33,590 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 09:32:33,590 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:32:18 2023-11-16 09:32:33,591 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:34:23 2023-11-16 09:32:33,591 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2023-11-16 09:32:34,223 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:32:34,224 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:32:18 2023-11-16 09:32:34,224 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:34:23 2023-11-16 09:34:24,540 INFO 1947952:repeater task lastensq_wh300 starting 1.5 seconds late 2023-11-16 09:34:27,360 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 09:34:27,360 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:34:18 2023-11-16 09:34:27,360 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:36:23 2023-11-16 09:34:27,360 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-16 09:34:33,463 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 09:34:33,464 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:34:18 2023-11-16 09:34:33,464 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:36:23 2023-11-16 09:34:33,464 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-16 09:34:34,102 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:34:34,103 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:34:18 2023-11-16 09:34:34,103 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:36:23 2023-11-16 09:36:19,360 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.3 seconds late 2023-11-16 09:36:20,527 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 09:36:20,527 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:20,528 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.5 seconds late 2023-11-16 09:36:20,694 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 09:36:20,694 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:20,695 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.6 seconds late 2023-11-16 09:36:21,979 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 09:36:21,980 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:21,980 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.9 seconds late 2023-11-16 09:36:23,254 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 09:36:23,254 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:23,255 INFO 1947952:repeater task lastensq_os150nb starting 0.3 seconds late 2023-11-16 09:36:25,459 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 09:36:25,460 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:36:18 2023-11-16 09:36:25,460 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 09:41:23 2023-11-16 09:36:25,460 INFO 1947952:repeater task quick_os150nb starting 2.5 seconds late 2023-11-16 09:36:29,090 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 09:36:29,090 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:36:18 2023-11-16 09:36:29,091 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 09:41:23 2023-11-16 09:36:29,091 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.1 seconds late 2023-11-16 09:36:29,707 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 09:36:29,707 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:36:18 2023-11-16 09:36:29,707 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 09:41:23 2023-11-16 09:36:29,708 INFO 1947952:repeater task 3dayplots_os150nb starting 5.7 seconds late 2023-11-16 09:36:33,090 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 09:36:33,091 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 09:41:24 2023-11-16 09:36:33,091 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.0 seconds late 2023-11-16 09:36:34,058 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 09:36:34,058 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:34,059 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.0 seconds late 2023-11-16 09:36:34,227 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 09:36:34,227 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:34,228 INFO 1947952:repeater task beamdiagnostics_os150nb starting 15.2 seconds late 2023-11-16 09:36:35,468 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 09:36:35,468 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:35,468 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.4 seconds late 2023-11-16 09:36:36,724 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 09:36:36,724 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:36,724 INFO 1947952:repeater task lastensq_os38nb starting 11.7 seconds late 2023-11-16 09:36:38,823 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 09:36:38,823 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:36:20 2023-11-16 09:36:38,823 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 09:41:25 2023-11-16 09:36:38,824 INFO 1947952:repeater task quick_os38nb starting 13.8 seconds late 2023-11-16 09:36:42,778 INFO 1947952:repeater task quick_os38nb ended after 4.0 seconds 2023-11-16 09:36:42,779 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:36:20 2023-11-16 09:36:42,779 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 09:41:25 2023-11-16 09:36:42,779 INFO 1947952:repeater task lastfew_vec_os38nb starting 17.8 seconds late 2023-11-16 09:36:43,391 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 09:36:43,392 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:36:20 2023-11-16 09:36:43,392 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 09:41:25 2023-11-16 09:36:43,392 INFO 1947952:repeater task 3dayplots_os38nb starting 19.3 seconds late 2023-11-16 09:36:46,834 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 09:36:46,834 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 09:41:24 2023-11-16 09:36:46,834 INFO 1947952:repeater task plot_tsstats_os38nb starting 27.8 seconds late 2023-11-16 09:36:47,831 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 09:36:47,831 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:47,832 INFO 1947952:repeater task grab_lastpings_os38nb starting 28.8 seconds late 2023-11-16 09:36:47,997 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 09:36:47,997 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:47,998 INFO 1947952:repeater task beamdiagnostics_os38nb starting 28.9 seconds late 2023-11-16 09:36:49,234 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 09:36:49,234 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:49,234 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.2 seconds late 2023-11-16 09:36:50,465 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 09:36:50,465 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:36:50,466 INFO 1947952:repeater task daily_netcdf starting 26.4 seconds late 2023-11-16 09:36:51,267 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 09:36:51,267 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 09:46:24 2023-11-16 09:36:51,267 INFO 1947952:repeater task ggatime_diagnostics starting 32.2 seconds late 2023-11-16 09:37:07,430 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-16 09:37:07,430 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:37:07,430 INFO 1947952:repeater task ggatime_diagnostics starting 48.4 seconds late 2023-11-16 09:37:23,428 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 09:37:23,428 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:37:23,429 INFO 1947952:repeater task ggatime_diagnostics starting 64.4 seconds late 2023-11-16 09:37:39,603 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-16 09:37:39,604 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:37:39,604 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 80.3 seconds late 2023-11-16 09:37:40,940 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 09:37:40,940 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:37:40,941 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 81.7 seconds late 2023-11-16 09:37:42,208 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 09:37:42,208 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:37:42,209 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 82.9 seconds late 2023-11-16 09:37:43,509 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 09:37:43,509 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:37:43,509 INFO 1947952:repeater task plot_hbins starting 84.2 seconds late 2023-11-16 09:37:47,817 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 09:37:47,817 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 09:37:47,817 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 09:51:19 2023-11-16 09:37:47,818 INFO 1947952:repeater task report_generator starting 88.5 seconds late 2023-11-16 09:37:53,438 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 09:37:53,439 INFO 1947952:repeater task report_generator ended after 5.6 seconds 2023-11-16 09:37:53,439 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 10:36:19 2023-11-16 09:37:58,441 INFO 1947952:repeater task noisy_rawtemps_os150 starting 94.4 seconds late 2023-11-16 09:38:01,342 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 09:38:01,342 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 09:46:24 2023-11-16 09:38:01,342 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 97.3 seconds late 2023-11-16 09:38:04,249 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 09:38:04,249 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 09:46:24 2023-11-16 09:38:04,249 INFO 1947952:repeater task noisy_rawtemps_os38 starting 100.2 seconds late 2023-11-16 09:38:07,302 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 09:38:07,303 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 09:46:24 2023-11-16 09:38:07,303 INFO 1947952:repeater task lastensq_wh300 starting 104.3 seconds late 2023-11-16 09:38:09,876 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 09:38:09,877 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:36:18 2023-11-16 09:38:09,877 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:38:23 2023-11-16 09:38:09,877 INFO 1947952:repeater task quick_wh300 starting 106.9 seconds late 2023-11-16 09:38:16,193 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 09:38:16,194 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:36:18 2023-11-16 09:38:16,194 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:38:23 2023-11-16 09:38:16,194 INFO 1947952:repeater task lastfew_vec_wh300 starting 113.2 seconds late 2023-11-16 09:38:16,821 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:38:16,822 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:36:18 2023-11-16 09:38:16,822 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:38:23 2023-11-16 09:38:16,822 INFO 1947952:repeater task 3dayplots_wh300 starting 112.8 seconds late 2023-11-16 09:38:20,658 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 09:38:20,658 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 09:41:24 2023-11-16 09:38:25,669 INFO 1947952:repeater task lastensq_wh300 starting 2.7 seconds late 2023-11-16 09:38:28,460 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 09:38:28,460 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:38:18 2023-11-16 09:38:28,460 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:40:23 2023-11-16 09:38:28,461 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-16 09:38:34,120 INFO 1947952:repeater task quick_wh300 ended after 5.7 seconds 2023-11-16 09:38:34,120 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:38:18 2023-11-16 09:38:34,121 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:40:23 2023-11-16 09:38:34,121 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-16 09:38:34,752 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:38:34,752 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:38:18 2023-11-16 09:38:34,752 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:40:23 2023-11-16 09:40:25,037 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-16 09:40:27,793 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 09:40:27,793 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:40:18 2023-11-16 09:40:27,793 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:42:23 2023-11-16 09:40:27,794 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 09:40:33,543 INFO 1947952:repeater task quick_wh300 ended after 5.7 seconds 2023-11-16 09:40:33,543 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:40:18 2023-11-16 09:40:33,543 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:42:23 2023-11-16 09:40:33,544 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-16 09:40:34,175 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:40:34,175 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:40:18 2023-11-16 09:40:34,175 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:42:23 2023-11-16 09:41:24,296 INFO 1947952:repeater task 3dayplots_wh300 starting 0.2 seconds late 2023-11-16 09:41:28,085 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 09:41:28,085 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 09:46:24 2023-11-16 09:41:28,086 INFO 1947952:repeater task lastensq_os150nb starting 5.1 seconds late 2023-11-16 09:41:30,324 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 09:41:30,325 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:41:18 2023-11-16 09:41:30,325 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 09:46:23 2023-11-16 09:41:30,325 INFO 1947952:repeater task quick_os150nb starting 7.3 seconds late 2023-11-16 09:41:33,781 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 09:41:33,781 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:41:18 2023-11-16 09:41:33,781 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 09:46:23 2023-11-16 09:41:33,781 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.8 seconds late 2023-11-16 09:41:34,395 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 09:41:34,396 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:41:18 2023-11-16 09:41:34,396 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 09:46:23 2023-11-16 09:41:34,396 INFO 1947952:repeater task 3dayplots_os150nb starting 10.3 seconds late 2023-11-16 09:41:37,821 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 09:41:37,821 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 09:46:24 2023-11-16 09:41:37,822 INFO 1947952:repeater task lastensq_os38nb starting 12.8 seconds late 2023-11-16 09:41:39,872 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 09:41:39,872 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:41:20 2023-11-16 09:41:39,872 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 09:46:25 2023-11-16 09:41:39,873 INFO 1947952:repeater task quick_os38nb starting 14.9 seconds late 2023-11-16 09:41:44,081 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 09:41:44,082 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:41:20 2023-11-16 09:41:44,082 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 09:46:25 2023-11-16 09:41:44,082 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.1 seconds late 2023-11-16 09:41:44,696 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 09:41:44,697 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:41:20 2023-11-16 09:41:44,697 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 09:46:25 2023-11-16 09:41:44,697 INFO 1947952:repeater task 3dayplots_os38nb starting 20.6 seconds late 2023-11-16 09:41:48,121 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 09:41:48,121 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 09:46:24 2023-11-16 09:42:23,199 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-16 09:42:25,952 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 09:42:25,952 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:42:18 2023-11-16 09:42:25,952 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:44:23 2023-11-16 09:42:25,953 INFO 1947952:repeater task quick_wh300 starting 3.0 seconds late 2023-11-16 09:42:32,357 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 09:42:32,358 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:42:18 2023-11-16 09:42:32,358 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:44:23 2023-11-16 09:42:32,358 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.4 seconds late 2023-11-16 09:42:33,002 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:42:33,002 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:42:18 2023-11-16 09:42:33,002 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:44:23 2023-11-16 09:44:23,288 INFO 1947952:repeater task lastensq_wh300 starting 0.3 seconds late 2023-11-16 09:44:26,167 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 09:44:26,168 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:44:18 2023-11-16 09:44:26,168 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:46:23 2023-11-16 09:44:26,168 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-16 09:44:32,483 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 09:44:32,484 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:44:18 2023-11-16 09:44:32,484 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:46:23 2023-11-16 09:44:32,484 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.5 seconds late 2023-11-16 09:44:33,114 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:44:33,115 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:44:18 2023-11-16 09:44:33,115 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:46:23 2023-11-16 09:46:23,475 INFO 1947952:repeater task lastensq_wh300 starting 0.5 seconds late 2023-11-16 09:46:26,381 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 09:46:26,382 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:46:18 2023-11-16 09:46:26,382 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:48:23 2023-11-16 09:46:26,382 INFO 1947952:repeater task quick_wh300 starting 3.4 seconds late 2023-11-16 09:46:32,439 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 09:46:32,439 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:46:18 2023-11-16 09:46:32,439 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:48:23 2023-11-16 09:46:32,440 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.4 seconds late 2023-11-16 09:46:33,068 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:46:33,068 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:46:18 2023-11-16 09:46:33,068 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:48:23 2023-11-16 09:46:33,069 INFO 1947952:repeater task 3dayplots_wh300 starting 9.0 seconds late 2023-11-16 09:46:36,872 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 09:46:36,873 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 09:51:24 2023-11-16 09:46:36,874 INFO 1947952:repeater task lastensq_os150nb starting 13.9 seconds late 2023-11-16 09:46:39,228 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 09:46:39,228 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:46:18 2023-11-16 09:46:39,228 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 09:51:23 2023-11-16 09:46:39,229 INFO 1947952:repeater task quick_os150nb starting 16.2 seconds late 2023-11-16 09:46:43,042 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 09:46:43,043 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:46:18 2023-11-16 09:46:43,043 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 09:51:23 2023-11-16 09:46:43,043 INFO 1947952:repeater task lastfew_vec_os150nb starting 20.0 seconds late 2023-11-16 09:46:43,656 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 09:46:43,656 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:46:18 2023-11-16 09:46:43,657 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 09:51:23 2023-11-16 09:46:43,657 INFO 1947952:repeater task 3dayplots_os150nb starting 19.6 seconds late 2023-11-16 09:46:47,054 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 09:46:47,054 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 09:51:24 2023-11-16 09:46:47,056 INFO 1947952:repeater task lastensq_os38nb starting 22.1 seconds late 2023-11-16 09:46:49,164 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 09:46:49,164 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:46:20 2023-11-16 09:46:49,164 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 09:51:25 2023-11-16 09:46:49,164 INFO 1947952:repeater task quick_os38nb starting 24.2 seconds late 2023-11-16 09:46:54,000 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-16 09:46:54,001 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:46:20 2023-11-16 09:46:54,001 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 09:51:25 2023-11-16 09:46:54,001 INFO 1947952:repeater task lastfew_vec_os38nb starting 29.0 seconds late 2023-11-16 09:46:54,621 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 09:46:54,621 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:46:20 2023-11-16 09:46:54,621 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 09:51:25 2023-11-16 09:46:54,622 INFO 1947952:repeater task 3dayplots_os38nb starting 30.6 seconds late 2023-11-16 09:46:58,056 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 09:46:58,056 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 09:51:24 2023-11-16 09:46:58,057 INFO 1947952:repeater task daily_netcdf starting 34.0 seconds late 2023-11-16 09:46:58,886 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 09:46:58,886 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 09:56:24 2023-11-16 09:47:03,893 INFO 1947952:repeater task noisy_rawtemps_os150 starting 39.8 seconds late 2023-11-16 09:47:06,865 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 09:47:06,865 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 09:56:24 2023-11-16 09:47:06,866 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 42.8 seconds late 2023-11-16 09:47:09,741 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 09:47:09,741 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 09:56:24 2023-11-16 09:47:09,741 INFO 1947952:repeater task noisy_rawtemps_os38 starting 45.7 seconds late 2023-11-16 09:47:12,814 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 09:47:12,814 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 09:56:24 2023-11-16 09:48:27,989 INFO 1947952:repeater task lastensq_wh300 starting 5.0 seconds late 2023-11-16 09:48:31,052 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 09:48:31,052 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:48:18 2023-11-16 09:48:31,052 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:50:23 2023-11-16 09:48:31,052 INFO 1947952:repeater task quick_wh300 starting 8.1 seconds late 2023-11-16 09:48:37,759 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 09:48:37,759 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:48:18 2023-11-16 09:48:37,759 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:50:23 2023-11-16 09:48:37,760 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-16 09:48:38,387 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:48:38,387 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:48:18 2023-11-16 09:48:38,388 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:50:23 2023-11-16 09:50:23,640 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-16 09:50:26,639 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 09:50:26,639 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:50:18 2023-11-16 09:50:26,639 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:52:23 2023-11-16 09:50:26,640 INFO 1947952:repeater task quick_wh300 starting 3.6 seconds late 2023-11-16 09:50:32,879 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 09:50:32,880 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:50:18 2023-11-16 09:50:32,880 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:52:23 2023-11-16 09:50:32,880 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 09:50:33,509 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:50:33,509 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:50:18 2023-11-16 09:50:33,509 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:52:23 2023-11-16 09:51:23,641 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.6 seconds late 2023-11-16 09:51:24,818 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 09:51:24,819 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:24,819 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.8 seconds late 2023-11-16 09:51:24,986 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 09:51:24,987 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:24,987 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.9 seconds late 2023-11-16 09:51:26,264 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 09:51:26,265 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:26,265 INFO 1947952:repeater task beamdiagnostics_wh300 starting 7.2 seconds late 2023-11-16 09:51:27,529 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 09:51:27,530 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:27,530 INFO 1947952:repeater task lastensq_os150nb starting 4.5 seconds late 2023-11-16 09:51:30,294 INFO 1947952:repeater task lastensq_os150nb ended after 2.8 seconds 2023-11-16 09:51:30,295 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:51:18 2023-11-16 09:51:30,295 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 09:56:23 2023-11-16 09:51:30,295 INFO 1947952:repeater task quick_os150nb starting 7.3 seconds late 2023-11-16 09:51:33,772 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 09:51:33,772 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:51:18 2023-11-16 09:51:33,772 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 09:56:23 2023-11-16 09:51:33,773 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.8 seconds late 2023-11-16 09:51:34,385 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 09:51:34,385 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:51:18 2023-11-16 09:51:34,385 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 09:56:23 2023-11-16 09:51:34,386 INFO 1947952:repeater task 3dayplots_os150nb starting 10.3 seconds late 2023-11-16 09:51:37,791 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 09:51:37,791 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 09:56:24 2023-11-16 09:51:37,791 INFO 1947952:repeater task plot_tsstats_os150nb starting 18.7 seconds late 2023-11-16 09:51:38,767 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 09:51:38,767 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:38,767 INFO 1947952:repeater task grab_lastpings_os150nb starting 19.7 seconds late 2023-11-16 09:51:38,935 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 09:51:38,935 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:38,936 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.9 seconds late 2023-11-16 09:51:40,164 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 09:51:40,164 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:40,164 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.1 seconds late 2023-11-16 09:51:41,423 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 09:51:41,423 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:41,424 INFO 1947952:repeater task lastensq_os38nb starting 16.4 seconds late 2023-11-16 09:51:43,536 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 09:51:43,537 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:51:20 2023-11-16 09:51:43,537 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 09:56:25 2023-11-16 09:51:43,537 INFO 1947952:repeater task quick_os38nb starting 18.5 seconds late 2023-11-16 09:51:47,707 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 09:51:47,707 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:51:20 2023-11-16 09:51:47,707 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 09:56:25 2023-11-16 09:51:47,708 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.7 seconds late 2023-11-16 09:51:48,322 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 09:51:48,322 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:51:20 2023-11-16 09:51:48,322 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 09:56:25 2023-11-16 09:51:48,323 INFO 1947952:repeater task 3dayplots_os38nb starting 24.3 seconds late 2023-11-16 09:51:51,772 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 09:51:51,772 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 09:56:24 2023-11-16 09:51:51,773 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.7 seconds late 2023-11-16 09:51:52,767 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 09:51:52,768 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:52,768 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.7 seconds late 2023-11-16 09:51:52,936 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 09:51:52,936 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:52,936 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.9 seconds late 2023-11-16 09:51:54,242 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 09:51:54,242 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:54,243 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.2 seconds late 2023-11-16 09:51:55,474 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 09:51:55,475 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:55,476 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 36.2 seconds late 2023-11-16 09:51:56,781 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 09:51:56,781 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:56,781 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 37.5 seconds late 2023-11-16 09:51:58,043 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 09:51:58,044 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:58,044 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 38.8 seconds late 2023-11-16 09:51:59,320 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 09:51:59,320 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:51:59,320 INFO 1947952:repeater task plot_hbins starting 40.0 seconds late 2023-11-16 09:52:03,656 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 09:52:03,656 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 09:52:03,656 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 10:06:19 2023-11-16 09:52:08,663 INFO 1947952:repeater task 3dayplots_wh300 starting 44.6 seconds late 2023-11-16 09:52:12,678 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-16 09:52:12,678 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 09:56:24 2023-11-16 09:52:27,718 INFO 1947952:repeater task lastensq_wh300 starting 4.7 seconds late 2023-11-16 09:52:30,690 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 09:52:30,691 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:52:18 2023-11-16 09:52:30,691 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:54:23 2023-11-16 09:52:30,691 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-16 09:52:36,967 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 09:52:36,967 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:52:18 2023-11-16 09:52:36,967 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:54:23 2023-11-16 09:52:36,968 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.0 seconds late 2023-11-16 09:52:37,595 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:52:37,596 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:52:18 2023-11-16 09:52:37,596 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:54:23 2023-11-16 09:54:27,888 INFO 1947952:repeater task lastensq_wh300 starting 4.9 seconds late 2023-11-16 09:54:30,621 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 09:54:30,622 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:54:18 2023-11-16 09:54:30,622 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:56:23 2023-11-16 09:54:30,622 INFO 1947952:repeater task quick_wh300 starting 7.6 seconds late 2023-11-16 09:54:36,661 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 09:54:36,661 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:54:18 2023-11-16 09:54:36,662 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:56:23 2023-11-16 09:54:36,662 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-16 09:54:37,291 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:54:37,292 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:54:18 2023-11-16 09:54:37,292 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:56:23 2023-11-16 09:56:27,589 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.5 seconds late 2023-11-16 09:56:30,584 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 09:56:30,584 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 10:06:24 2023-11-16 09:56:30,585 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.5 seconds late 2023-11-16 09:56:33,484 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 09:56:33,484 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 10:06:24 2023-11-16 09:56:33,485 INFO 1947952:repeater task noisy_rawtemps_os38 starting 9.4 seconds late 2023-11-16 09:56:36,572 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 09:56:36,573 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 10:06:24 2023-11-16 09:56:36,573 INFO 1947952:repeater task lastensq_wh300 starting 13.6 seconds late 2023-11-16 09:56:39,237 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 09:56:39,237 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:56:18 2023-11-16 09:56:39,238 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 09:58:23 2023-11-16 09:56:39,238 INFO 1947952:repeater task quick_wh300 starting 16.2 seconds late 2023-11-16 09:56:45,232 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 09:56:45,233 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:56:18 2023-11-16 09:56:45,233 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 09:58:23 2023-11-16 09:56:45,233 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.2 seconds late 2023-11-16 09:56:45,879 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:56:45,880 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:56:18 2023-11-16 09:56:45,880 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 09:58:23 2023-11-16 09:56:45,880 INFO 1947952:repeater task 3dayplots_wh300 starting 21.8 seconds late 2023-11-16 09:56:49,624 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 09:56:49,625 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:01:24 2023-11-16 09:56:49,626 INFO 1947952:repeater task lastensq_os150nb starting 26.6 seconds late 2023-11-16 09:56:51,677 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 09:56:51,678 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:56:18 2023-11-16 09:56:51,678 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:01:23 2023-11-16 09:56:51,678 INFO 1947952:repeater task quick_os150nb starting 28.7 seconds late 2023-11-16 09:56:55,359 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 09:56:55,359 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:56:18 2023-11-16 09:56:55,359 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:01:23 2023-11-16 09:56:55,360 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.4 seconds late 2023-11-16 09:56:55,977 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 09:56:55,978 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 09:56:18 2023-11-16 09:56:55,978 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:01:23 2023-11-16 09:56:55,978 INFO 1947952:repeater task 3dayplots_os150nb starting 31.9 seconds late 2023-11-16 09:56:59,407 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 09:56:59,408 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:01:24 2023-11-16 09:56:59,409 INFO 1947952:repeater task lastensq_os38nb starting 34.4 seconds late 2023-11-16 09:57:01,420 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 09:57:01,420 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:56:20 2023-11-16 09:57:01,420 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:01:25 2023-11-16 09:57:01,420 INFO 1947952:repeater task quick_os38nb starting 36.4 seconds late 2023-11-16 09:57:05,084 INFO 1947952:repeater task quick_os38nb ended after 3.7 seconds 2023-11-16 09:57:05,084 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:56:20 2023-11-16 09:57:05,085 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:01:25 2023-11-16 09:57:05,085 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.1 seconds late 2023-11-16 09:57:05,703 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 09:57:05,703 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 09:56:20 2023-11-16 09:57:05,704 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:01:25 2023-11-16 09:57:05,704 INFO 1947952:repeater task 3dayplots_os38nb starting 41.7 seconds late 2023-11-16 09:57:09,234 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 09:57:09,234 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:01:24 2023-11-16 09:57:09,236 INFO 1947952:repeater task daily_netcdf starting 45.2 seconds late 2023-11-16 09:57:10,046 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 09:57:10,046 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 10:06:24 2023-11-16 09:58:25,237 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-16 09:58:28,030 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 09:58:28,030 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:58:18 2023-11-16 09:58:28,030 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:00:23 2023-11-16 09:58:28,030 INFO 1947952:repeater task quick_wh300 starting 5.0 seconds late 2023-11-16 09:58:34,121 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 09:58:34,121 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:58:18 2023-11-16 09:58:34,122 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:00:23 2023-11-16 09:58:34,122 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-16 09:58:34,742 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 09:58:34,743 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 09:58:18 2023-11-16 09:58:34,743 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:00:23 2023-11-16 10:00:25,029 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-16 10:00:28,121 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 10:00:28,121 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:00:18 2023-11-16 10:00:28,122 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:02:23 2023-11-16 10:00:28,122 INFO 1947952:repeater task quick_wh300 starting 5.1 seconds late 2023-11-16 10:00:34,359 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 10:00:34,359 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:00:18 2023-11-16 10:00:34,359 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:02:23 2023-11-16 10:00:34,360 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-16 10:00:34,991 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:00:34,991 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:00:18 2023-11-16 10:00:34,991 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:02:23 2023-11-16 10:01:25,108 INFO 1947952:repeater task 3dayplots_wh300 starting 1.1 seconds late 2023-11-16 10:01:28,920 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:01:28,920 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:06:24 2023-11-16 10:01:28,921 INFO 1947952:repeater task lastensq_os150nb starting 5.9 seconds late 2023-11-16 10:01:31,537 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 10:01:31,538 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:01:18 2023-11-16 10:01:31,538 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:06:23 2023-11-16 10:01:31,539 INFO 1947952:repeater task quick_os150nb starting 8.5 seconds late 2023-11-16 10:01:34,993 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 10:01:34,993 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:01:18 2023-11-16 10:01:34,993 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:06:23 2023-11-16 10:01:34,994 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.0 seconds late 2023-11-16 10:01:35,612 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:01:35,613 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:01:18 2023-11-16 10:01:35,613 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:06:23 2023-11-16 10:01:35,613 INFO 1947952:repeater task 3dayplots_os150nb starting 11.6 seconds late 2023-11-16 10:01:39,014 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:01:39,015 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:06:24 2023-11-16 10:01:39,016 INFO 1947952:repeater task lastensq_os38nb starting 14.0 seconds late 2023-11-16 10:01:41,220 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 10:01:41,220 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:01:20 2023-11-16 10:01:41,220 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:06:25 2023-11-16 10:01:41,221 INFO 1947952:repeater task quick_os38nb starting 16.2 seconds late 2023-11-16 10:01:45,848 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 10:01:45,848 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:01:20 2023-11-16 10:01:45,848 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:06:25 2023-11-16 10:01:45,849 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.8 seconds late 2023-11-16 10:01:46,468 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:01:46,469 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:01:20 2023-11-16 10:01:46,469 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:06:25 2023-11-16 10:01:46,469 INFO 1947952:repeater task 3dayplots_os38nb starting 22.4 seconds late 2023-11-16 10:01:49,942 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 10:01:49,943 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:06:24 2023-11-16 10:02:25,039 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-16 10:02:28,261 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-16 10:02:28,261 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:02:18 2023-11-16 10:02:28,261 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:04:23 2023-11-16 10:02:28,262 INFO 1947952:repeater task quick_wh300 starting 5.3 seconds late 2023-11-16 10:02:34,608 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 10:02:34,608 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:02:18 2023-11-16 10:02:34,608 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:04:23 2023-11-16 10:02:34,609 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2023-11-16 10:02:35,227 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:02:35,227 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:02:18 2023-11-16 10:02:35,227 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:04:23 2023-11-16 10:04:25,519 INFO 1947952:repeater task lastensq_wh300 starting 2.5 seconds late 2023-11-16 10:04:28,444 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 10:04:28,444 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:04:18 2023-11-16 10:04:28,444 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:06:23 2023-11-16 10:04:28,444 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-16 10:04:34,725 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 10:04:34,726 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:04:18 2023-11-16 10:04:34,726 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:06:23 2023-11-16 10:04:34,726 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-16 10:04:35,352 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:04:35,352 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:04:18 2023-11-16 10:04:35,353 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:06:23 2023-11-16 10:06:20,617 INFO 1947952:repeater task plot_tsstats_wh300 starting 1.6 seconds late 2023-11-16 10:06:21,788 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 10:06:21,789 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:21,789 INFO 1947952:repeater task grab_lastpings_wh300 starting 2.7 seconds late 2023-11-16 10:06:21,958 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 10:06:21,958 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:21,958 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.9 seconds late 2023-11-16 10:06:23,168 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 10:06:23,169 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:23,169 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.1 seconds late 2023-11-16 10:06:24,422 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 10:06:24,422 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:24,423 INFO 1947952:repeater task lastensq_os150nb starting 1.4 seconds late 2023-11-16 10:06:26,998 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 10:06:26,998 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:06:18 2023-11-16 10:06:26,999 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:11:23 2023-11-16 10:06:26,999 INFO 1947952:repeater task quick_os150nb starting 4.0 seconds late 2023-11-16 10:06:30,818 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 10:06:30,819 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:06:18 2023-11-16 10:06:30,819 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:11:23 2023-11-16 10:06:30,819 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.8 seconds late 2023-11-16 10:06:31,433 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:06:31,433 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:06:18 2023-11-16 10:06:31,433 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:11:23 2023-11-16 10:06:31,434 INFO 1947952:repeater task 3dayplots_os150nb starting 7.4 seconds late 2023-11-16 10:06:34,848 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:06:34,848 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:11:24 2023-11-16 10:06:34,849 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.8 seconds late 2023-11-16 10:06:35,823 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 10:06:35,824 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:35,824 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.8 seconds late 2023-11-16 10:06:35,991 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 10:06:35,991 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:35,991 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.9 seconds late 2023-11-16 10:06:37,235 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 10:06:37,235 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:37,235 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.2 seconds late 2023-11-16 10:06:38,418 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 10:06:38,419 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:38,419 INFO 1947952:repeater task lastensq_os38nb starting 13.4 seconds late 2023-11-16 10:06:40,450 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 10:06:40,450 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:06:20 2023-11-16 10:06:40,450 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:11:25 2023-11-16 10:06:40,451 INFO 1947952:repeater task quick_os38nb starting 15.5 seconds late 2023-11-16 10:06:45,071 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 10:06:45,072 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:06:20 2023-11-16 10:06:45,072 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:11:25 2023-11-16 10:06:45,072 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.1 seconds late 2023-11-16 10:06:45,690 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:06:45,691 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:06:20 2023-11-16 10:06:45,691 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:11:25 2023-11-16 10:06:45,691 INFO 1947952:repeater task 3dayplots_os38nb starting 21.6 seconds late 2023-11-16 10:06:49,114 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 10:06:49,115 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:11:24 2023-11-16 10:06:49,115 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.1 seconds late 2023-11-16 10:06:50,104 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 10:06:50,104 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:50,104 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.1 seconds late 2023-11-16 10:06:50,267 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 10:06:50,267 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:50,267 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.2 seconds late 2023-11-16 10:06:51,495 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 10:06:51,495 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:51,496 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.4 seconds late 2023-11-16 10:06:52,723 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 10:06:52,723 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:06:52,723 INFO 1947952:repeater task daily_netcdf starting 28.7 seconds late 2023-11-16 10:06:53,568 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 10:06:53,568 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 10:16:24 2023-11-16 10:06:53,568 INFO 1947952:repeater task ggatime_diagnostics starting 34.5 seconds late 2023-11-16 10:07:09,653 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-16 10:07:09,653 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:07:09,655 INFO 1947952:repeater task ggatime_diagnostics starting 50.6 seconds late 2023-11-16 10:07:25,689 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 10:07:25,690 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:07:25,690 INFO 1947952:repeater task ggatime_diagnostics starting 66.6 seconds late 2023-11-16 10:07:41,769 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-16 10:07:41,769 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:07:41,770 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 82.5 seconds late 2023-11-16 10:07:43,109 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 10:07:43,109 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:07:43,109 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 83.8 seconds late 2023-11-16 10:07:44,391 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 10:07:44,392 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:07:44,392 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 85.1 seconds late 2023-11-16 10:07:45,638 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 10:07:45,638 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:07:45,639 INFO 1947952:repeater task plot_hbins starting 86.4 seconds late 2023-11-16 10:07:49,898 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 10:07:49,898 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 10:07:49,898 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 10:21:19 2023-11-16 10:07:54,901 INFO 1947952:repeater task noisy_rawtemps_os150 starting 90.9 seconds late 2023-11-16 10:07:57,901 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 10:07:57,901 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 10:16:24 2023-11-16 10:07:57,902 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 93.9 seconds late 2023-11-16 10:08:00,796 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 10:08:00,796 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 10:16:24 2023-11-16 10:08:00,796 INFO 1947952:repeater task noisy_rawtemps_os38 starting 96.7 seconds late 2023-11-16 10:08:03,895 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 10:08:03,895 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 10:16:24 2023-11-16 10:08:03,896 INFO 1947952:repeater task lastensq_wh300 starting 100.9 seconds late 2023-11-16 10:08:06,533 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 10:08:06,534 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:06:18 2023-11-16 10:08:06,534 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:08:23 2023-11-16 10:08:06,534 INFO 1947952:repeater task quick_wh300 starting 103.5 seconds late 2023-11-16 10:08:13,391 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-16 10:08:13,392 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:06:18 2023-11-16 10:08:13,392 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:08:23 2023-11-16 10:08:13,392 INFO 1947952:repeater task lastfew_vec_wh300 starting 110.4 seconds late 2023-11-16 10:08:14,062 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 10:08:14,063 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:06:18 2023-11-16 10:08:14,063 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:08:23 2023-11-16 10:08:14,063 INFO 1947952:repeater task 3dayplots_wh300 starting 110.0 seconds late 2023-11-16 10:08:18,063 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-16 10:08:18,063 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:11:24 2023-11-16 10:08:23,070 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 10:08:25,760 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:08:25,760 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:08:18 2023-11-16 10:08:25,760 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:10:23 2023-11-16 10:08:25,761 INFO 1947952:repeater task quick_wh300 starting 2.8 seconds late 2023-11-16 10:08:32,445 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 10:08:32,445 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:08:18 2023-11-16 10:08:32,445 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:10:23 2023-11-16 10:08:32,445 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.4 seconds late 2023-11-16 10:08:33,091 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:08:33,091 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:08:18 2023-11-16 10:08:33,091 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:10:23 2023-11-16 10:10:23,359 INFO 1947952:repeater task lastensq_wh300 starting 0.4 seconds late 2023-11-16 10:10:26,555 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-16 10:10:26,555 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:10:18 2023-11-16 10:10:26,555 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:12:23 2023-11-16 10:10:26,556 INFO 1947952:repeater task quick_wh300 starting 3.6 seconds late 2023-11-16 10:10:32,756 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 10:10:32,756 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:10:18 2023-11-16 10:10:32,756 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:12:23 2023-11-16 10:10:32,757 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.8 seconds late 2023-11-16 10:10:33,403 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:10:33,404 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:10:18 2023-11-16 10:10:33,404 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:12:23 2023-11-16 10:11:23,522 INFO 1947952:repeater task lastensq_os150nb starting 0.5 seconds late 2023-11-16 10:11:26,087 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 10:11:26,087 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:11:18 2023-11-16 10:11:26,087 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:16:23 2023-11-16 10:11:26,088 INFO 1947952:repeater task quick_os150nb starting 3.1 seconds late 2023-11-16 10:11:29,752 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 10:11:29,752 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:11:18 2023-11-16 10:11:29,753 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:16:23 2023-11-16 10:11:29,753 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.8 seconds late 2023-11-16 10:11:30,361 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:11:30,362 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:11:18 2023-11-16 10:11:30,362 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:16:23 2023-11-16 10:11:30,362 INFO 1947952:repeater task 3dayplots_os150nb starting 6.3 seconds late 2023-11-16 10:11:33,737 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:11:33,737 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:16:24 2023-11-16 10:11:33,739 INFO 1947952:repeater task lastensq_os38nb starting 8.7 seconds late 2023-11-16 10:11:36,076 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-16 10:11:36,076 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:11:20 2023-11-16 10:11:36,076 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:16:25 2023-11-16 10:11:36,076 INFO 1947952:repeater task quick_os38nb starting 11.1 seconds late 2023-11-16 10:11:40,439 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 10:11:40,439 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:11:20 2023-11-16 10:11:40,439 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:16:25 2023-11-16 10:11:40,440 INFO 1947952:repeater task lastfew_vec_os38nb starting 15.4 seconds late 2023-11-16 10:11:41,042 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:11:41,042 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:11:20 2023-11-16 10:11:41,042 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:16:25 2023-11-16 10:11:41,043 INFO 1947952:repeater task 3dayplots_os38nb starting 17.0 seconds late 2023-11-16 10:11:44,566 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 10:11:44,567 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:16:24 2023-11-16 10:11:49,576 INFO 1947952:repeater task 3dayplots_wh300 starting 25.5 seconds late 2023-11-16 10:11:53,369 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:11:53,369 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:16:24 2023-11-16 10:12:23,434 INFO 1947952:repeater task lastensq_wh300 starting 0.4 seconds late 2023-11-16 10:12:26,176 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:12:26,177 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:12:18 2023-11-16 10:12:26,177 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:14:23 2023-11-16 10:12:26,177 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-16 10:12:32,104 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 10:12:32,104 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:12:18 2023-11-16 10:12:32,104 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:14:23 2023-11-16 10:12:32,105 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.1 seconds late 2023-11-16 10:12:32,744 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:12:32,744 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:12:18 2023-11-16 10:12:32,744 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:14:23 2023-11-16 10:14:23,047 INFO 1947952:repeater task lastensq_wh300 starting 0.0 seconds late 2023-11-16 10:14:25,942 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 10:14:25,943 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:14:18 2023-11-16 10:14:25,943 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:16:23 2023-11-16 10:14:25,943 INFO 1947952:repeater task quick_wh300 starting 2.9 seconds late 2023-11-16 10:14:32,069 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 10:14:32,069 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:14:18 2023-11-16 10:14:32,069 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:16:23 2023-11-16 10:14:32,070 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.1 seconds late 2023-11-16 10:14:32,711 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:14:32,711 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:14:18 2023-11-16 10:14:32,711 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:16:23 2023-11-16 10:16:28,001 INFO 1947952:repeater task noisy_rawtemps_os150 starting 4.0 seconds late 2023-11-16 10:16:31,012 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 10:16:31,013 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 10:26:24 2023-11-16 10:16:31,013 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 7.0 seconds late 2023-11-16 10:16:33,852 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 10:16:33,852 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 10:26:24 2023-11-16 10:16:33,853 INFO 1947952:repeater task noisy_rawtemps_os38 starting 9.8 seconds late 2023-11-16 10:16:36,902 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 10:16:36,902 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 10:26:24 2023-11-16 10:16:36,903 INFO 1947952:repeater task lastensq_wh300 starting 13.9 seconds late 2023-11-16 10:16:39,571 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:16:39,571 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:16:18 2023-11-16 10:16:39,571 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:18:23 2023-11-16 10:16:39,572 INFO 1947952:repeater task quick_wh300 starting 16.6 seconds late 2023-11-16 10:16:45,607 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 10:16:45,608 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:16:18 2023-11-16 10:16:45,608 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:18:23 2023-11-16 10:16:45,608 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.6 seconds late 2023-11-16 10:16:46,250 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:16:46,251 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:16:18 2023-11-16 10:16:46,251 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:18:23 2023-11-16 10:16:46,251 INFO 1947952:repeater task 3dayplots_wh300 starting 22.2 seconds late 2023-11-16 10:16:50,016 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:16:50,016 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:21:24 2023-11-16 10:16:50,017 INFO 1947952:repeater task lastensq_os150nb starting 27.0 seconds late 2023-11-16 10:16:52,201 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 10:16:52,201 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:16:18 2023-11-16 10:16:52,201 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:21:23 2023-11-16 10:16:52,202 INFO 1947952:repeater task quick_os150nb starting 29.2 seconds late 2023-11-16 10:16:55,810 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 10:16:55,811 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:16:18 2023-11-16 10:16:55,811 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:21:23 2023-11-16 10:16:55,811 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.8 seconds late 2023-11-16 10:16:56,422 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:16:56,423 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:16:18 2023-11-16 10:16:56,423 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:21:23 2023-11-16 10:16:56,423 INFO 1947952:repeater task 3dayplots_os150nb starting 32.4 seconds late 2023-11-16 10:16:59,828 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:16:59,828 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:21:24 2023-11-16 10:16:59,829 INFO 1947952:repeater task lastensq_os38nb starting 34.8 seconds late 2023-11-16 10:17:02,257 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-16 10:17:02,257 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:16:20 2023-11-16 10:17:02,257 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:21:25 2023-11-16 10:17:02,258 INFO 1947952:repeater task quick_os38nb starting 37.3 seconds late 2023-11-16 10:17:06,743 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 10:17:06,744 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:16:20 2023-11-16 10:17:06,744 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:21:25 2023-11-16 10:17:06,744 INFO 1947952:repeater task lastfew_vec_os38nb starting 41.7 seconds late 2023-11-16 10:17:07,362 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:17:07,362 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:16:20 2023-11-16 10:17:07,362 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:21:25 2023-11-16 10:17:07,363 INFO 1947952:repeater task 3dayplots_os38nb starting 43.3 seconds late 2023-11-16 10:17:10,816 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 10:17:10,816 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:21:24 2023-11-16 10:17:10,817 INFO 1947952:repeater task daily_netcdf starting 46.8 seconds late 2023-11-16 10:17:12,022 INFO 1947952:repeater task daily_netcdf ended after 1.2 seconds 2023-11-16 10:17:12,023 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 10:26:24 2023-11-16 10:18:27,214 INFO 1947952:repeater task lastensq_wh300 starting 4.2 seconds late 2023-11-16 10:18:30,330 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 10:18:30,331 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:18:18 2023-11-16 10:18:30,331 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:20:23 2023-11-16 10:18:30,331 INFO 1947952:repeater task quick_wh300 starting 7.3 seconds late 2023-11-16 10:18:36,470 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 10:18:36,470 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:18:18 2023-11-16 10:18:36,470 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:20:23 2023-11-16 10:18:36,471 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.5 seconds late 2023-11-16 10:18:37,112 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:18:37,113 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:18:18 2023-11-16 10:18:37,113 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:20:23 2023-11-16 10:20:27,408 INFO 1947952:repeater task lastensq_wh300 starting 4.4 seconds late 2023-11-16 10:20:30,110 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:20:30,110 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:20:18 2023-11-16 10:20:30,111 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:22:23 2023-11-16 10:20:30,111 INFO 1947952:repeater task quick_wh300 starting 7.1 seconds late 2023-11-16 10:20:36,458 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 10:20:36,458 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:20:18 2023-11-16 10:20:36,458 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:22:23 2023-11-16 10:20:36,459 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.5 seconds late 2023-11-16 10:20:37,101 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:20:37,101 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:20:18 2023-11-16 10:20:37,101 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:22:23 2023-11-16 10:21:22,217 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.2 seconds late 2023-11-16 10:21:23,400 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 10:21:23,400 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:23,401 INFO 1947952:repeater task grab_lastpings_wh300 starting 4.4 seconds late 2023-11-16 10:21:23,569 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 10:21:23,569 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:23,569 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.5 seconds late 2023-11-16 10:21:24,845 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 10:21:24,845 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:24,845 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.8 seconds late 2023-11-16 10:21:26,102 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 10:21:26,102 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:26,103 INFO 1947952:repeater task lastensq_os150nb starting 3.1 seconds late 2023-11-16 10:21:28,473 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 10:21:28,473 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:21:18 2023-11-16 10:21:28,473 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:26:23 2023-11-16 10:21:28,474 INFO 1947952:repeater task quick_os150nb starting 5.5 seconds late 2023-11-16 10:21:31,756 INFO 1947952:repeater task quick_os150nb ended after 3.3 seconds 2023-11-16 10:21:31,756 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:21:18 2023-11-16 10:21:31,757 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:26:23 2023-11-16 10:21:31,757 INFO 1947952:repeater task lastfew_vec_os150nb starting 8.8 seconds late 2023-11-16 10:21:32,364 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:21:32,365 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:21:18 2023-11-16 10:21:32,365 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:26:23 2023-11-16 10:21:32,365 INFO 1947952:repeater task 3dayplots_os150nb starting 8.3 seconds late 2023-11-16 10:21:35,651 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-16 10:21:35,651 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:26:24 2023-11-16 10:21:35,652 INFO 1947952:repeater task plot_tsstats_os150nb starting 16.6 seconds late 2023-11-16 10:21:36,632 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 10:21:36,632 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:36,632 INFO 1947952:repeater task grab_lastpings_os150nb starting 17.6 seconds late 2023-11-16 10:21:36,800 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 10:21:36,800 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:36,800 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.7 seconds late 2023-11-16 10:21:38,069 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 10:21:38,070 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:38,070 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.0 seconds late 2023-11-16 10:21:39,308 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 10:21:39,309 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:39,309 INFO 1947952:repeater task lastensq_os38nb starting 14.3 seconds late 2023-11-16 10:21:41,405 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 10:21:41,405 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:21:20 2023-11-16 10:21:41,405 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:26:25 2023-11-16 10:21:41,406 INFO 1947952:repeater task quick_os38nb starting 16.4 seconds late 2023-11-16 10:21:45,511 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 10:21:45,512 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:21:20 2023-11-16 10:21:45,512 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:26:25 2023-11-16 10:21:45,512 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.5 seconds late 2023-11-16 10:21:46,135 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:21:46,135 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:21:20 2023-11-16 10:21:46,135 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:26:25 2023-11-16 10:21:46,136 INFO 1947952:repeater task 3dayplots_os38nb starting 22.1 seconds late 2023-11-16 10:21:49,546 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 10:21:49,546 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:26:24 2023-11-16 10:21:49,546 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.5 seconds late 2023-11-16 10:21:50,552 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 10:21:50,552 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:50,553 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.5 seconds late 2023-11-16 10:21:50,723 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 10:21:50,723 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:50,724 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.7 seconds late 2023-11-16 10:21:51,997 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 10:21:51,997 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:51,998 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.9 seconds late 2023-11-16 10:21:53,221 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 10:21:53,222 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:53,223 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 34.0 seconds late 2023-11-16 10:21:54,513 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 10:21:54,513 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:54,513 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 35.2 seconds late 2023-11-16 10:21:55,701 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.2 seconds 2023-11-16 10:21:55,702 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:55,702 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 36.4 seconds late 2023-11-16 10:21:57,027 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 10:21:57,027 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:21:57,028 INFO 1947952:repeater task plot_hbins starting 37.8 seconds late 2023-11-16 10:22:01,398 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 10:22:01,399 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 10:22:01,399 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 10:36:19 2023-11-16 10:22:06,403 INFO 1947952:repeater task 3dayplots_wh300 starting 42.4 seconds late 2023-11-16 10:22:10,317 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 10:22:10,317 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:26:24 2023-11-16 10:22:25,351 INFO 1947952:repeater task lastensq_wh300 starting 2.4 seconds late 2023-11-16 10:22:28,103 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 10:22:28,104 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:22:18 2023-11-16 10:22:28,104 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:24:23 2023-11-16 10:22:28,104 INFO 1947952:repeater task quick_wh300 starting 5.1 seconds late 2023-11-16 10:22:33,986 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 10:22:33,986 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:22:18 2023-11-16 10:22:33,986 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:24:23 2023-11-16 10:22:33,987 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 10:22:34,631 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:22:34,632 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:22:18 2023-11-16 10:22:34,632 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:24:23 2023-11-16 10:24:24,932 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 10:24:27,580 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 10:24:27,581 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:24:18 2023-11-16 10:24:27,581 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:26:23 2023-11-16 10:24:27,581 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-16 10:24:33,429 INFO 1947952:repeater task quick_wh300 ended after 5.8 seconds 2023-11-16 10:24:33,430 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:24:18 2023-11-16 10:24:33,430 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:26:23 2023-11-16 10:24:33,430 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-16 10:24:34,066 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:24:34,067 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:24:18 2023-11-16 10:24:34,067 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:26:23 2023-11-16 10:26:24,352 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.3 seconds late 2023-11-16 10:26:27,329 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 10:26:27,330 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 10:36:24 2023-11-16 10:26:27,330 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.3 seconds late 2023-11-16 10:26:30,200 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 10:26:30,200 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 10:36:24 2023-11-16 10:26:30,200 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.2 seconds late 2023-11-16 10:26:33,236 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 10:26:33,236 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 10:36:24 2023-11-16 10:26:33,236 INFO 1947952:repeater task lastensq_wh300 starting 10.2 seconds late 2023-11-16 10:26:35,902 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:26:35,902 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:26:18 2023-11-16 10:26:35,902 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:28:23 2023-11-16 10:26:35,902 INFO 1947952:repeater task quick_wh300 starting 12.9 seconds late 2023-11-16 10:26:41,605 INFO 1947952:repeater task quick_wh300 ended after 5.7 seconds 2023-11-16 10:26:41,605 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:26:18 2023-11-16 10:26:41,605 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:28:23 2023-11-16 10:26:41,606 INFO 1947952:repeater task lastfew_vec_wh300 starting 18.6 seconds late 2023-11-16 10:26:42,245 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:26:42,245 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:26:18 2023-11-16 10:26:42,245 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:28:23 2023-11-16 10:26:42,246 INFO 1947952:repeater task 3dayplots_wh300 starting 18.2 seconds late 2023-11-16 10:26:46,067 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:26:46,067 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:31:24 2023-11-16 10:26:46,069 INFO 1947952:repeater task lastensq_os150nb starting 23.1 seconds late 2023-11-16 10:26:48,276 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 10:26:48,277 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:26:18 2023-11-16 10:26:48,277 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:31:23 2023-11-16 10:26:48,277 INFO 1947952:repeater task quick_os150nb starting 25.3 seconds late 2023-11-16 10:26:51,802 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 10:26:51,803 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:26:18 2023-11-16 10:26:51,803 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:31:23 2023-11-16 10:26:51,803 INFO 1947952:repeater task lastfew_vec_os150nb starting 28.8 seconds late 2023-11-16 10:26:52,418 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:26:52,419 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:26:18 2023-11-16 10:26:52,419 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:31:23 2023-11-16 10:26:52,419 INFO 1947952:repeater task 3dayplots_os150nb starting 28.4 seconds late 2023-11-16 10:26:55,820 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:26:55,820 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:31:24 2023-11-16 10:26:55,822 INFO 1947952:repeater task lastensq_os38nb starting 30.8 seconds late 2023-11-16 10:26:57,936 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 10:26:57,936 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:26:20 2023-11-16 10:26:57,936 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:31:25 2023-11-16 10:26:57,936 INFO 1947952:repeater task quick_os38nb starting 32.9 seconds late 2023-11-16 10:27:02,171 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 10:27:02,172 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:26:20 2023-11-16 10:27:02,172 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:31:25 2023-11-16 10:27:02,172 INFO 1947952:repeater task lastfew_vec_os38nb starting 37.2 seconds late 2023-11-16 10:27:02,842 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.7 seconds 2023-11-16 10:27:02,843 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:26:20 2023-11-16 10:27:02,843 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:31:25 2023-11-16 10:27:02,843 INFO 1947952:repeater task 3dayplots_os38nb starting 38.8 seconds late 2023-11-16 10:27:06,547 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.7 seconds 2023-11-16 10:27:06,547 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:31:24 2023-11-16 10:27:06,548 INFO 1947952:repeater task daily_netcdf starting 42.5 seconds late 2023-11-16 10:27:07,314 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 10:27:07,314 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 10:36:24 2023-11-16 10:28:27,516 INFO 1947952:repeater task lastensq_wh300 starting 4.5 seconds late 2023-11-16 10:28:30,274 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 10:28:30,274 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:28:18 2023-11-16 10:28:30,275 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:30:23 2023-11-16 10:28:30,275 INFO 1947952:repeater task quick_wh300 starting 7.3 seconds late 2023-11-16 10:28:36,406 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 10:28:36,406 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:28:18 2023-11-16 10:28:36,406 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:30:23 2023-11-16 10:28:36,407 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2023-11-16 10:28:37,042 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:28:37,042 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:28:18 2023-11-16 10:28:37,043 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:30:23 2023-11-16 10:30:27,342 INFO 1947952:repeater task lastensq_wh300 starting 4.3 seconds late 2023-11-16 10:30:29,870 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-16 10:30:29,870 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:30:18 2023-11-16 10:30:29,871 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:32:23 2023-11-16 10:30:29,871 INFO 1947952:repeater task quick_wh300 starting 6.9 seconds late 2023-11-16 10:30:35,766 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 10:30:35,766 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:30:18 2023-11-16 10:30:35,767 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:32:23 2023-11-16 10:30:35,767 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-16 10:30:36,401 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:30:36,402 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:30:18 2023-11-16 10:30:36,402 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:32:23 2023-11-16 10:31:26,532 INFO 1947952:repeater task 3dayplots_wh300 starting 2.5 seconds late 2023-11-16 10:31:30,414 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 10:31:30,414 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:36:24 2023-11-16 10:31:30,415 INFO 1947952:repeater task lastensq_os150nb starting 7.4 seconds late 2023-11-16 10:31:32,626 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 10:31:32,627 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:31:18 2023-11-16 10:31:32,627 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:36:23 2023-11-16 10:31:32,627 INFO 1947952:repeater task quick_os150nb starting 9.6 seconds late 2023-11-16 10:31:36,121 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 10:31:36,121 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:31:18 2023-11-16 10:31:36,121 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:36:23 2023-11-16 10:31:36,121 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.1 seconds late 2023-11-16 10:31:36,729 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:31:36,730 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:31:18 2023-11-16 10:31:36,730 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:36:23 2023-11-16 10:31:36,730 INFO 1947952:repeater task 3dayplots_os150nb starting 12.7 seconds late 2023-11-16 10:31:40,147 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:31:40,147 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:36:24 2023-11-16 10:31:40,148 INFO 1947952:repeater task lastensq_os38nb starting 15.1 seconds late 2023-11-16 10:31:42,197 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 10:31:42,197 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:31:20 2023-11-16 10:31:42,197 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:36:25 2023-11-16 10:31:42,198 INFO 1947952:repeater task quick_os38nb starting 17.2 seconds late 2023-11-16 10:31:45,926 INFO 1947952:repeater task quick_os38nb ended after 3.7 seconds 2023-11-16 10:31:45,926 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:31:20 2023-11-16 10:31:45,926 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:36:25 2023-11-16 10:31:45,926 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.9 seconds late 2023-11-16 10:31:46,538 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:31:46,539 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:31:20 2023-11-16 10:31:46,539 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:36:25 2023-11-16 10:31:46,539 INFO 1947952:repeater task 3dayplots_os38nb starting 22.5 seconds late 2023-11-16 10:31:49,975 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 10:31:49,975 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:36:24 2023-11-16 10:32:25,059 INFO 1947952:repeater task lastensq_wh300 starting 2.1 seconds late 2023-11-16 10:32:27,828 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 10:32:27,828 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:32:18 2023-11-16 10:32:27,828 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:34:23 2023-11-16 10:32:27,828 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 10:32:33,343 INFO 1947952:repeater task quick_wh300 ended after 5.5 seconds 2023-11-16 10:32:33,343 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:32:18 2023-11-16 10:32:33,343 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:34:23 2023-11-16 10:32:33,344 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-16 10:32:33,991 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:32:33,992 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:32:18 2023-11-16 10:32:33,992 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:34:23 2023-11-16 10:34:24,275 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 10:34:27,040 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 10:34:27,040 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:34:18 2023-11-16 10:34:27,040 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:36:23 2023-11-16 10:34:27,041 INFO 1947952:repeater task quick_wh300 starting 4.0 seconds late 2023-11-16 10:34:32,858 INFO 1947952:repeater task quick_wh300 ended after 5.8 seconds 2023-11-16 10:34:32,858 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:34:18 2023-11-16 10:34:32,858 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:36:23 2023-11-16 10:34:32,859 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 10:34:33,487 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:34:33,488 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:34:18 2023-11-16 10:34:33,488 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:36:23 2023-11-16 10:36:23,776 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-16 10:36:26,453 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:36:26,453 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:36:18 2023-11-16 10:36:26,454 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:38:23 2023-11-16 10:36:26,454 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 10:36:32,556 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 10:36:32,557 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:36:18 2023-11-16 10:36:32,557 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:38:23 2023-11-16 10:36:32,558 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.6 seconds late 2023-11-16 10:36:33,190 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:36:33,191 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:36:18 2023-11-16 10:36:33,191 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:38:23 2023-11-16 10:36:33,191 INFO 1947952:repeater task 3dayplots_wh300 starting 9.1 seconds late 2023-11-16 10:36:37,002 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:36:37,002 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:41:24 2023-11-16 10:36:37,002 INFO 1947952:repeater task plot_tsstats_wh300 starting 18.0 seconds late 2023-11-16 10:36:38,152 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.1 seconds 2023-11-16 10:36:38,152 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:38,152 INFO 1947952:repeater task grab_lastpings_wh300 starting 19.1 seconds late 2023-11-16 10:36:38,318 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 10:36:38,318 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:38,318 INFO 1947952:repeater task beamdiagnostics_wh300 starting 19.3 seconds late 2023-11-16 10:36:39,562 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 10:36:39,562 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:39,563 INFO 1947952:repeater task beamdiagnostics_wh300 starting 20.5 seconds late 2023-11-16 10:36:40,844 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 10:36:40,844 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:40,844 INFO 1947952:repeater task lastensq_os150nb starting 17.8 seconds late 2023-11-16 10:36:42,992 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 10:36:42,992 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:36:18 2023-11-16 10:36:42,992 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:41:23 2023-11-16 10:36:42,993 INFO 1947952:repeater task quick_os150nb starting 20.0 seconds late 2023-11-16 10:36:46,673 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 10:36:46,673 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:36:18 2023-11-16 10:36:46,673 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:41:23 2023-11-16 10:36:46,674 INFO 1947952:repeater task lastfew_vec_os150nb starting 23.7 seconds late 2023-11-16 10:36:47,283 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:36:47,284 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:36:18 2023-11-16 10:36:47,284 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:41:23 2023-11-16 10:36:47,284 INFO 1947952:repeater task 3dayplots_os150nb starting 23.2 seconds late 2023-11-16 10:36:50,644 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:36:50,644 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:41:24 2023-11-16 10:36:50,645 INFO 1947952:repeater task plot_tsstats_os150nb starting 31.6 seconds late 2023-11-16 10:36:51,618 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 10:36:51,618 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:51,619 INFO 1947952:repeater task grab_lastpings_os150nb starting 32.6 seconds late 2023-11-16 10:36:51,785 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 10:36:51,785 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:51,785 INFO 1947952:repeater task beamdiagnostics_os150nb starting 32.7 seconds late 2023-11-16 10:36:53,018 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 10:36:53,018 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:53,018 INFO 1947952:repeater task beamdiagnostics_os150nb starting 34.0 seconds late 2023-11-16 10:36:54,259 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 10:36:54,259 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:36:54,260 INFO 1947952:repeater task lastensq_os38nb starting 29.3 seconds late 2023-11-16 10:36:56,399 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 10:36:56,400 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:36:20 2023-11-16 10:36:56,400 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:41:25 2023-11-16 10:36:56,400 INFO 1947952:repeater task quick_os38nb starting 31.4 seconds late 2023-11-16 10:37:00,034 INFO 1947952:repeater task quick_os38nb ended after 3.6 seconds 2023-11-16 10:37:00,035 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:36:20 2023-11-16 10:37:00,035 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:41:25 2023-11-16 10:37:00,036 INFO 1947952:repeater task lastfew_vec_os38nb starting 35.0 seconds late 2023-11-16 10:37:00,655 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:37:00,656 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:36:20 2023-11-16 10:37:00,656 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:41:25 2023-11-16 10:37:00,656 INFO 1947952:repeater task 3dayplots_os38nb starting 36.6 seconds late 2023-11-16 10:37:04,275 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.6 seconds 2023-11-16 10:37:04,275 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:41:24 2023-11-16 10:37:04,276 INFO 1947952:repeater task plot_tsstats_os38nb starting 45.2 seconds late 2023-11-16 10:37:05,357 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.1 seconds 2023-11-16 10:37:05,358 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:37:05,358 INFO 1947952:repeater task grab_lastpings_os38nb starting 46.3 seconds late 2023-11-16 10:37:05,533 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 10:37:05,533 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:37:05,533 INFO 1947952:repeater task beamdiagnostics_os38nb starting 46.5 seconds late 2023-11-16 10:37:06,888 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.4 seconds 2023-11-16 10:37:06,888 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:37:06,889 INFO 1947952:repeater task beamdiagnostics_os38nb starting 47.8 seconds late 2023-11-16 10:37:08,198 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 10:37:08,198 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:37:08,199 INFO 1947952:repeater task daily_netcdf starting 44.1 seconds late 2023-11-16 10:37:09,022 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 10:37:09,022 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 10:46:24 2023-11-16 10:37:09,023 INFO 1947952:repeater task ggatime_diagnostics starting 50.0 seconds late 2023-11-16 10:37:25,355 INFO 1947952:repeater task ggatime_diagnostics ended after 16.3 seconds 2023-11-16 10:37:25,356 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:37:25,356 INFO 1947952:repeater task ggatime_diagnostics starting 66.3 seconds late 2023-11-16 10:37:41,236 INFO 1947952:repeater task ggatime_diagnostics ended after 15.9 seconds 2023-11-16 10:37:41,236 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:37:41,237 INFO 1947952:repeater task ggatime_diagnostics starting 82.2 seconds late 2023-11-16 10:37:57,011 INFO 1947952:repeater task ggatime_diagnostics ended after 15.8 seconds 2023-11-16 10:37:57,011 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:37:57,011 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 97.7 seconds late 2023-11-16 10:37:58,377 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-16 10:37:58,378 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:37:58,378 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 99.1 seconds late 2023-11-16 10:37:59,650 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 10:37:59,650 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:37:59,651 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 100.4 seconds late 2023-11-16 10:38:00,848 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 10:38:00,848 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:38:00,849 INFO 1947952:repeater task plot_hbins starting 101.6 seconds late 2023-11-16 10:38:05,110 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 10:38:05,111 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 10:38:05,111 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 10:51:19 2023-11-16 10:38:05,111 INFO 1947952:repeater task report_generator starting 105.8 seconds late 2023-11-16 10:38:10,964 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 10:38:10,965 INFO 1947952:repeater task report_generator ended after 5.9 seconds 2023-11-16 10:38:10,965 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 11:36:19 2023-11-16 10:38:15,969 INFO 1947952:repeater task noisy_rawtemps_os150 starting 111.9 seconds late 2023-11-16 10:38:18,956 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 10:38:18,956 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 10:46:24 2023-11-16 10:38:18,956 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 114.9 seconds late 2023-11-16 10:38:21,819 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 10:38:21,819 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 10:46:24 2023-11-16 10:38:21,820 INFO 1947952:repeater task noisy_rawtemps_os38 starting 117.8 seconds late 2023-11-16 10:38:24,887 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 10:38:24,887 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 10:46:24 2023-11-16 10:38:24,888 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 10:38:27,407 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-16 10:38:27,407 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:38:18 2023-11-16 10:38:27,407 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:40:23 2023-11-16 10:38:27,407 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-16 10:38:33,368 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 10:38:33,368 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:38:18 2023-11-16 10:38:33,368 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:40:23 2023-11-16 10:38:33,369 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-16 10:38:34,003 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:38:34,003 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:38:18 2023-11-16 10:38:34,004 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:40:23 2023-11-16 10:40:24,288 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 10:40:27,053 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 10:40:27,054 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:40:18 2023-11-16 10:40:27,054 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:42:23 2023-11-16 10:40:27,054 INFO 1947952:repeater task quick_wh300 starting 4.1 seconds late 2023-11-16 10:40:33,226 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 10:40:33,226 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:40:18 2023-11-16 10:40:33,226 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:42:23 2023-11-16 10:40:33,227 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2023-11-16 10:40:33,860 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:40:33,860 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:40:18 2023-11-16 10:40:33,860 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:42:23 2023-11-16 10:41:23,987 INFO 1947952:repeater task lastensq_os150nb starting 1.0 seconds late 2023-11-16 10:41:26,263 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 10:41:26,263 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:41:18 2023-11-16 10:41:26,264 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:46:23 2023-11-16 10:41:26,264 INFO 1947952:repeater task quick_os150nb starting 3.3 seconds late 2023-11-16 10:41:30,107 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 10:41:30,107 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:41:18 2023-11-16 10:41:30,108 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:46:23 2023-11-16 10:41:30,108 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.1 seconds late 2023-11-16 10:41:30,721 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:41:30,722 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:41:18 2023-11-16 10:41:30,722 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:46:23 2023-11-16 10:41:30,722 INFO 1947952:repeater task 3dayplots_os150nb starting 6.7 seconds late 2023-11-16 10:41:34,116 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:41:34,116 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:46:24 2023-11-16 10:41:34,118 INFO 1947952:repeater task lastensq_os38nb starting 9.1 seconds late 2023-11-16 10:41:36,392 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-16 10:41:36,393 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:41:20 2023-11-16 10:41:36,393 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:46:25 2023-11-16 10:41:36,393 INFO 1947952:repeater task quick_os38nb starting 11.4 seconds late 2023-11-16 10:41:40,581 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 10:41:40,582 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:41:20 2023-11-16 10:41:40,582 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:46:25 2023-11-16 10:41:40,582 INFO 1947952:repeater task lastfew_vec_os38nb starting 15.6 seconds late 2023-11-16 10:41:41,198 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:41:41,199 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:41:20 2023-11-16 10:41:41,199 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:46:25 2023-11-16 10:41:41,199 INFO 1947952:repeater task 3dayplots_os38nb starting 17.1 seconds late 2023-11-16 10:41:44,682 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 10:41:44,682 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:46:24 2023-11-16 10:41:49,691 INFO 1947952:repeater task 3dayplots_wh300 starting 25.6 seconds late 2023-11-16 10:41:53,473 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:41:53,474 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:46:24 2023-11-16 10:42:23,534 INFO 1947952:repeater task lastensq_wh300 starting 0.5 seconds late 2023-11-16 10:42:26,209 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:42:26,209 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:42:18 2023-11-16 10:42:26,209 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:44:23 2023-11-16 10:42:26,210 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-16 10:42:32,280 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 10:42:32,280 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:42:18 2023-11-16 10:42:32,280 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:44:23 2023-11-16 10:42:32,281 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2023-11-16 10:42:32,916 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:42:32,916 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:42:18 2023-11-16 10:42:32,916 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:44:23 2023-11-16 10:44:23,198 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-16 10:44:26,019 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 10:44:26,019 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:44:18 2023-11-16 10:44:26,019 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:46:23 2023-11-16 10:44:26,020 INFO 1947952:repeater task quick_wh300 starting 3.0 seconds late 2023-11-16 10:44:32,217 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 10:44:32,218 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:44:18 2023-11-16 10:44:32,218 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:46:23 2023-11-16 10:44:32,218 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2023-11-16 10:44:32,854 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:44:32,855 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:44:18 2023-11-16 10:44:32,855 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:46:23 2023-11-16 10:46:23,136 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 10:46:25,861 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:46:25,862 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:46:18 2023-11-16 10:46:25,862 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:48:23 2023-11-16 10:46:25,862 INFO 1947952:repeater task quick_wh300 starting 2.9 seconds late 2023-11-16 10:46:32,194 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 10:46:32,195 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:46:18 2023-11-16 10:46:32,195 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:48:23 2023-11-16 10:46:32,195 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2023-11-16 10:46:32,840 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:46:32,840 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:46:18 2023-11-16 10:46:32,840 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:48:23 2023-11-16 10:46:32,841 INFO 1947952:repeater task 3dayplots_wh300 starting 8.8 seconds late 2023-11-16 10:46:36,693 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 10:46:36,693 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:51:24 2023-11-16 10:46:36,695 INFO 1947952:repeater task lastensq_os150nb starting 13.7 seconds late 2023-11-16 10:46:39,150 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 10:46:39,150 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:46:18 2023-11-16 10:46:39,150 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:51:23 2023-11-16 10:46:39,151 INFO 1947952:repeater task quick_os150nb starting 16.2 seconds late 2023-11-16 10:46:43,031 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 10:46:43,032 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:46:18 2023-11-16 10:46:43,032 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:51:23 2023-11-16 10:46:43,032 INFO 1947952:repeater task lastfew_vec_os150nb starting 20.0 seconds late 2023-11-16 10:46:43,649 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:46:43,649 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:46:18 2023-11-16 10:46:43,649 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:51:23 2023-11-16 10:46:43,650 INFO 1947952:repeater task 3dayplots_os150nb starting 19.6 seconds late 2023-11-16 10:46:46,994 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-16 10:46:46,994 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:51:24 2023-11-16 10:46:46,995 INFO 1947952:repeater task lastensq_os38nb starting 22.0 seconds late 2023-11-16 10:46:49,219 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 10:46:49,220 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:46:20 2023-11-16 10:46:49,220 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:51:25 2023-11-16 10:46:49,220 INFO 1947952:repeater task quick_os38nb starting 24.2 seconds late 2023-11-16 10:46:53,489 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 10:46:53,489 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:46:20 2023-11-16 10:46:53,489 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:51:25 2023-11-16 10:46:53,490 INFO 1947952:repeater task lastfew_vec_os38nb starting 28.5 seconds late 2023-11-16 10:46:54,103 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:46:54,103 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:46:20 2023-11-16 10:46:54,103 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:51:25 2023-11-16 10:46:54,104 INFO 1947952:repeater task 3dayplots_os38nb starting 30.1 seconds late 2023-11-16 10:46:57,565 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 10:46:57,566 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:51:24 2023-11-16 10:46:57,567 INFO 1947952:repeater task daily_netcdf starting 33.5 seconds late 2023-11-16 10:46:58,408 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 10:46:58,408 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 10:56:24 2023-11-16 10:47:03,413 INFO 1947952:repeater task noisy_rawtemps_os150 starting 39.4 seconds late 2023-11-16 10:47:06,373 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 10:47:06,373 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 10:56:24 2023-11-16 10:47:06,374 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 42.3 seconds late 2023-11-16 10:47:09,121 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-16 10:47:09,121 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 10:56:24 2023-11-16 10:47:09,121 INFO 1947952:repeater task noisy_rawtemps_os38 starting 45.1 seconds late 2023-11-16 10:47:12,160 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 10:47:12,160 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 10:56:24 2023-11-16 10:48:27,350 INFO 1947952:repeater task lastensq_wh300 starting 4.4 seconds late 2023-11-16 10:48:30,285 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 10:48:30,286 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:48:18 2023-11-16 10:48:30,286 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:50:23 2023-11-16 10:48:30,286 INFO 1947952:repeater task quick_wh300 starting 7.3 seconds late 2023-11-16 10:48:36,088 INFO 1947952:repeater task quick_wh300 ended after 5.8 seconds 2023-11-16 10:48:36,088 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:48:18 2023-11-16 10:48:36,088 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:50:23 2023-11-16 10:48:36,088 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2023-11-16 10:48:36,728 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:48:36,729 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:48:18 2023-11-16 10:48:36,729 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:50:23 2023-11-16 10:50:27,016 INFO 1947952:repeater task lastensq_wh300 starting 4.0 seconds late 2023-11-16 10:50:29,733 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:50:29,733 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:50:18 2023-11-16 10:50:29,733 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:52:23 2023-11-16 10:50:29,734 INFO 1947952:repeater task quick_wh300 starting 6.7 seconds late 2023-11-16 10:50:35,376 INFO 1947952:repeater task quick_wh300 ended after 5.6 seconds 2023-11-16 10:50:35,376 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:50:18 2023-11-16 10:50:35,376 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:52:23 2023-11-16 10:50:35,377 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2023-11-16 10:50:36,012 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:50:36,012 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:50:18 2023-11-16 10:50:36,013 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:52:23 2023-11-16 10:51:21,137 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.1 seconds late 2023-11-16 10:51:22,299 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 10:51:22,299 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:22,299 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.2 seconds late 2023-11-16 10:51:22,469 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 10:51:22,469 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:22,469 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.4 seconds late 2023-11-16 10:51:23,760 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 10:51:23,760 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:23,760 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.7 seconds late 2023-11-16 10:51:25,057 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 10:51:25,057 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:25,058 INFO 1947952:repeater task lastensq_os150nb starting 2.1 seconds late 2023-11-16 10:51:27,268 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 10:51:27,268 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:51:18 2023-11-16 10:51:27,268 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 10:56:23 2023-11-16 10:51:27,268 INFO 1947952:repeater task quick_os150nb starting 4.3 seconds late 2023-11-16 10:51:30,973 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 10:51:30,973 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:51:18 2023-11-16 10:51:30,973 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 10:56:23 2023-11-16 10:51:30,974 INFO 1947952:repeater task lastfew_vec_os150nb starting 8.0 seconds late 2023-11-16 10:51:31,584 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:51:31,584 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:51:18 2023-11-16 10:51:31,584 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 10:56:23 2023-11-16 10:51:31,585 INFO 1947952:repeater task 3dayplots_os150nb starting 7.5 seconds late 2023-11-16 10:51:34,974 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:51:34,974 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 10:56:24 2023-11-16 10:51:34,975 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.9 seconds late 2023-11-16 10:51:35,945 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 10:51:35,945 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:35,946 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.9 seconds late 2023-11-16 10:51:36,113 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 10:51:36,114 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:36,114 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.1 seconds late 2023-11-16 10:51:37,328 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 10:51:37,328 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:37,328 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.3 seconds late 2023-11-16 10:51:38,563 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 10:51:38,563 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:38,564 INFO 1947952:repeater task lastensq_os38nb starting 13.6 seconds late 2023-11-16 10:51:40,672 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 10:51:40,673 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:51:20 2023-11-16 10:51:40,673 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 10:56:25 2023-11-16 10:51:40,673 INFO 1947952:repeater task quick_os38nb starting 15.7 seconds late 2023-11-16 10:51:45,150 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 10:51:45,150 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:51:20 2023-11-16 10:51:45,150 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 10:56:25 2023-11-16 10:51:45,151 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.2 seconds late 2023-11-16 10:51:45,761 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:51:45,762 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:51:20 2023-11-16 10:51:45,762 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 10:56:25 2023-11-16 10:51:45,762 INFO 1947952:repeater task 3dayplots_os38nb starting 21.7 seconds late 2023-11-16 10:51:49,229 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 10:51:49,229 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 10:56:24 2023-11-16 10:51:49,229 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.2 seconds late 2023-11-16 10:51:50,230 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 10:51:50,230 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:50,231 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.2 seconds late 2023-11-16 10:51:50,398 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 10:51:50,398 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:50,398 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.3 seconds late 2023-11-16 10:51:51,668 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 10:51:51,668 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:51,669 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.6 seconds late 2023-11-16 10:51:52,896 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 10:51:52,897 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:52,898 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 33.6 seconds late 2023-11-16 10:51:54,228 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 10:51:54,228 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:54,228 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 35.0 seconds late 2023-11-16 10:51:55,481 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 10:51:55,482 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:55,482 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 36.2 seconds late 2023-11-16 10:51:56,738 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 10:51:56,739 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:51:56,739 INFO 1947952:repeater task plot_hbins starting 37.5 seconds late 2023-11-16 10:52:01,032 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 10:52:01,032 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 10:52:01,032 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 11:06:19 2023-11-16 10:52:06,039 INFO 1947952:repeater task 3dayplots_wh300 starting 42.0 seconds late 2023-11-16 10:52:09,848 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:52:09,848 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 10:56:24 2023-11-16 10:52:24,880 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 10:52:28,502 INFO 1947952:repeater task lastensq_wh300 ended after 3.6 seconds 2023-11-16 10:52:28,502 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:52:18 2023-11-16 10:52:28,503 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:54:23 2023-11-16 10:52:28,503 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-16 10:52:34,997 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 10:52:34,998 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:52:18 2023-11-16 10:52:34,998 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:54:23 2023-11-16 10:52:34,998 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.0 seconds late 2023-11-16 10:52:35,633 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:52:35,634 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:52:18 2023-11-16 10:52:35,634 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:54:23 2023-11-16 10:54:25,908 INFO 1947952:repeater task lastensq_wh300 starting 2.9 seconds late 2023-11-16 10:54:28,840 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 10:54:28,841 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:54:18 2023-11-16 10:54:28,841 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:56:23 2023-11-16 10:54:28,841 INFO 1947952:repeater task quick_wh300 starting 5.8 seconds late 2023-11-16 10:54:35,237 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 10:54:35,237 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:54:18 2023-11-16 10:54:35,237 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:56:23 2023-11-16 10:54:35,238 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2023-11-16 10:54:35,869 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:54:35,869 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:54:18 2023-11-16 10:54:35,869 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:56:23 2023-11-16 10:56:26,157 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.1 seconds late 2023-11-16 10:56:29,134 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 10:56:29,134 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 11:06:24 2023-11-16 10:56:29,135 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.1 seconds late 2023-11-16 10:56:31,931 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 10:56:31,931 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 11:06:24 2023-11-16 10:56:31,931 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.9 seconds late 2023-11-16 10:56:34,987 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 10:56:34,987 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 11:06:24 2023-11-16 10:56:34,987 INFO 1947952:repeater task lastensq_wh300 starting 12.0 seconds late 2023-11-16 10:56:37,861 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 10:56:37,861 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:56:18 2023-11-16 10:56:37,861 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 10:58:23 2023-11-16 10:56:37,862 INFO 1947952:repeater task quick_wh300 starting 14.9 seconds late 2023-11-16 10:56:44,292 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 10:56:44,293 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:56:18 2023-11-16 10:56:44,293 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 10:58:23 2023-11-16 10:56:44,293 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.3 seconds late 2023-11-16 10:56:44,927 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:56:44,927 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:56:18 2023-11-16 10:56:44,927 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 10:58:23 2023-11-16 10:56:44,927 INFO 1947952:repeater task 3dayplots_wh300 starting 20.9 seconds late 2023-11-16 10:56:48,688 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 10:56:48,689 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:01:24 2023-11-16 10:56:48,690 INFO 1947952:repeater task lastensq_os150nb starting 25.7 seconds late 2023-11-16 10:56:50,770 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 10:56:50,770 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:56:18 2023-11-16 10:56:50,770 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:01:23 2023-11-16 10:56:50,770 INFO 1947952:repeater task quick_os150nb starting 27.8 seconds late 2023-11-16 10:56:54,156 INFO 1947952:repeater task quick_os150nb ended after 3.4 seconds 2023-11-16 10:56:54,157 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:56:18 2023-11-16 10:56:54,157 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:01:23 2023-11-16 10:56:54,157 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.2 seconds late 2023-11-16 10:56:54,768 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 10:56:54,769 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 10:56:18 2023-11-16 10:56:54,769 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:01:23 2023-11-16 10:56:54,769 INFO 1947952:repeater task 3dayplots_os150nb starting 30.7 seconds late 2023-11-16 10:56:58,207 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 10:56:58,207 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:01:24 2023-11-16 10:56:58,208 INFO 1947952:repeater task lastensq_os38nb starting 33.2 seconds late 2023-11-16 10:57:00,342 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 10:57:00,342 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:56:20 2023-11-16 10:57:00,342 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:01:25 2023-11-16 10:57:00,343 INFO 1947952:repeater task quick_os38nb starting 35.3 seconds late 2023-11-16 10:57:05,127 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-16 10:57:05,127 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:56:20 2023-11-16 10:57:05,128 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:01:25 2023-11-16 10:57:05,128 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.1 seconds late 2023-11-16 10:57:05,745 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 10:57:05,745 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 10:56:20 2023-11-16 10:57:05,745 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:01:25 2023-11-16 10:57:05,746 INFO 1947952:repeater task 3dayplots_os38nb starting 41.7 seconds late 2023-11-16 10:57:09,153 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 10:57:09,153 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:01:24 2023-11-16 10:57:09,154 INFO 1947952:repeater task daily_netcdf starting 45.1 seconds late 2023-11-16 10:57:09,898 INFO 1947952:repeater task daily_netcdf ended after 0.7 seconds 2023-11-16 10:57:09,898 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 11:06:24 2023-11-16 10:58:25,076 INFO 1947952:repeater task lastensq_wh300 starting 2.1 seconds late 2023-11-16 10:58:27,807 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 10:58:27,807 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:58:18 2023-11-16 10:58:27,807 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:00:23 2023-11-16 10:58:27,808 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 10:58:34,658 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-16 10:58:34,659 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:58:18 2023-11-16 10:58:34,659 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:00:23 2023-11-16 10:58:34,659 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-16 10:58:35,291 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 10:58:35,292 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 10:58:18 2023-11-16 10:58:35,292 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:00:23 2023-11-16 11:00:25,580 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-16 11:00:28,661 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 11:00:28,661 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:00:18 2023-11-16 11:00:28,661 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:02:23 2023-11-16 11:00:28,662 INFO 1947952:repeater task quick_wh300 starting 5.7 seconds late 2023-11-16 11:00:34,736 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 11:00:34,736 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:00:18 2023-11-16 11:00:34,736 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:02:23 2023-11-16 11:00:34,737 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-16 11:00:35,362 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:00:35,363 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:00:18 2023-11-16 11:00:35,363 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:02:23 2023-11-16 11:01:25,497 INFO 1947952:repeater task 3dayplots_wh300 starting 1.4 seconds late 2023-11-16 11:01:29,295 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 11:01:29,295 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:06:24 2023-11-16 11:01:29,297 INFO 1947952:repeater task lastensq_os150nb starting 6.3 seconds late 2023-11-16 11:01:31,632 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 11:01:31,633 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:01:18 2023-11-16 11:01:31,633 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:06:23 2023-11-16 11:01:31,633 INFO 1947952:repeater task quick_os150nb starting 8.6 seconds late 2023-11-16 11:01:35,338 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 11:01:35,339 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:01:18 2023-11-16 11:01:35,339 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:06:23 2023-11-16 11:01:35,339 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.3 seconds late 2023-11-16 11:01:35,949 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:01:35,950 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:01:18 2023-11-16 11:01:35,950 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:06:23 2023-11-16 11:01:35,950 INFO 1947952:repeater task 3dayplots_os150nb starting 11.9 seconds late 2023-11-16 11:01:39,283 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-16 11:01:39,283 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:06:24 2023-11-16 11:01:39,284 INFO 1947952:repeater task lastensq_os38nb starting 14.3 seconds late 2023-11-16 11:01:41,437 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 11:01:41,438 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:01:20 2023-11-16 11:01:41,438 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:06:25 2023-11-16 11:01:41,438 INFO 1947952:repeater task quick_os38nb starting 16.4 seconds late 2023-11-16 11:01:45,859 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 11:01:45,860 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:01:20 2023-11-16 11:01:45,860 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:06:25 2023-11-16 11:01:45,860 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.9 seconds late 2023-11-16 11:01:46,483 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:01:46,483 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:01:20 2023-11-16 11:01:46,483 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:06:25 2023-11-16 11:01:46,484 INFO 1947952:repeater task 3dayplots_os38nb starting 22.4 seconds late 2023-11-16 11:01:49,892 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 11:01:49,893 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:06:24 2023-11-16 11:02:24,984 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-16 11:02:27,947 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:02:27,948 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:02:18 2023-11-16 11:02:27,948 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:04:23 2023-11-16 11:02:27,948 INFO 1947952:repeater task quick_wh300 starting 4.9 seconds late 2023-11-16 11:02:34,355 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 11:02:34,355 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:02:18 2023-11-16 11:02:34,355 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:04:23 2023-11-16 11:02:34,356 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-16 11:02:34,989 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:02:34,990 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:02:18 2023-11-16 11:02:34,990 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:04:23 2023-11-16 11:04:25,280 INFO 1947952:repeater task lastensq_wh300 starting 2.3 seconds late 2023-11-16 11:04:27,969 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 11:04:27,970 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:04:18 2023-11-16 11:04:27,970 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:06:23 2023-11-16 11:04:27,970 INFO 1947952:repeater task quick_wh300 starting 5.0 seconds late 2023-11-16 11:04:34,081 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 11:04:34,082 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:04:18 2023-11-16 11:04:34,082 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:06:23 2023-11-16 11:04:34,082 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-16 11:04:34,716 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:04:34,717 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:04:18 2023-11-16 11:04:34,717 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:06:23 2023-11-16 11:06:19,999 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.9 seconds late 2023-11-16 11:06:21,169 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 11:06:21,170 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:21,170 INFO 1947952:repeater task grab_lastpings_wh300 starting 2.1 seconds late 2023-11-16 11:06:21,338 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 11:06:21,338 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:21,338 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.3 seconds late 2023-11-16 11:06:22,617 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 11:06:22,617 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:22,618 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.6 seconds late 2023-11-16 11:06:23,892 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 11:06:23,892 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:23,893 INFO 1947952:repeater task lastensq_os150nb starting 0.9 seconds late 2023-11-16 11:06:26,122 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 11:06:26,122 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:06:18 2023-11-16 11:06:26,122 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:11:23 2023-11-16 11:06:26,122 INFO 1947952:repeater task quick_os150nb starting 3.1 seconds late 2023-11-16 11:06:29,618 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 11:06:29,619 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:06:18 2023-11-16 11:06:29,619 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:11:23 2023-11-16 11:06:29,619 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.6 seconds late 2023-11-16 11:06:30,232 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:06:30,233 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:06:18 2023-11-16 11:06:30,233 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:11:23 2023-11-16 11:06:30,233 INFO 1947952:repeater task 3dayplots_os150nb starting 6.2 seconds late 2023-11-16 11:06:33,636 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:06:33,636 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:11:24 2023-11-16 11:06:33,636 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.6 seconds late 2023-11-16 11:06:34,615 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 11:06:34,615 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:34,615 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.6 seconds late 2023-11-16 11:06:34,784 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 11:06:34,784 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:34,784 INFO 1947952:repeater task beamdiagnostics_os150nb starting 15.7 seconds late 2023-11-16 11:06:36,078 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 11:06:36,078 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:36,079 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.0 seconds late 2023-11-16 11:06:37,342 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 11:06:37,342 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:37,342 INFO 1947952:repeater task lastensq_os38nb starting 12.3 seconds late 2023-11-16 11:06:39,451 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:06:39,460 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:06:20 2023-11-16 11:06:39,460 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:11:25 2023-11-16 11:06:39,460 INFO 1947952:repeater task quick_os38nb starting 14.5 seconds late 2023-11-16 11:06:43,454 INFO 1947952:repeater task quick_os38nb ended after 4.0 seconds 2023-11-16 11:06:43,454 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:06:20 2023-11-16 11:06:43,454 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:11:25 2023-11-16 11:06:43,455 INFO 1947952:repeater task lastfew_vec_os38nb starting 18.5 seconds late 2023-11-16 11:06:44,067 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:06:44,067 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:06:20 2023-11-16 11:06:44,067 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:11:25 2023-11-16 11:06:44,068 INFO 1947952:repeater task 3dayplots_os38nb starting 20.0 seconds late 2023-11-16 11:06:47,472 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 11:06:47,472 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:11:24 2023-11-16 11:06:47,473 INFO 1947952:repeater task plot_tsstats_os38nb starting 28.4 seconds late 2023-11-16 11:06:48,479 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 11:06:48,480 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:48,480 INFO 1947952:repeater task grab_lastpings_os38nb starting 29.4 seconds late 2023-11-16 11:06:48,651 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 11:06:48,651 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:48,652 INFO 1947952:repeater task beamdiagnostics_os38nb starting 29.6 seconds late 2023-11-16 11:06:49,867 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 11:06:49,867 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:49,868 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.8 seconds late 2023-11-16 11:06:51,088 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 11:06:51,089 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:06:51,089 INFO 1947952:repeater task daily_netcdf starting 27.0 seconds late 2023-11-16 11:06:51,925 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 11:06:51,925 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 11:16:24 2023-11-16 11:06:51,926 INFO 1947952:repeater task ggatime_diagnostics starting 32.9 seconds late 2023-11-16 11:07:08,015 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-16 11:07:08,015 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:07:08,015 INFO 1947952:repeater task ggatime_diagnostics starting 49.0 seconds late 2023-11-16 11:07:23,858 INFO 1947952:repeater task ggatime_diagnostics ended after 15.8 seconds 2023-11-16 11:07:23,858 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:07:23,858 INFO 1947952:repeater task ggatime_diagnostics starting 64.8 seconds late 2023-11-16 11:07:39,875 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 11:07:39,875 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:07:39,876 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 80.6 seconds late 2023-11-16 11:07:41,257 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-16 11:07:41,258 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:07:41,258 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 82.0 seconds late 2023-11-16 11:07:42,574 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 11:07:42,574 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:07:42,575 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 83.3 seconds late 2023-11-16 11:07:43,844 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 11:07:43,844 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:07:43,845 INFO 1947952:repeater task plot_hbins starting 84.6 seconds late 2023-11-16 11:07:48,202 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 11:07:48,203 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 11:07:48,203 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 11:21:19 2023-11-16 11:07:53,209 INFO 1947952:repeater task noisy_rawtemps_os150 starting 89.2 seconds late 2023-11-16 11:07:56,185 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 11:07:56,185 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 11:16:24 2023-11-16 11:07:56,186 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 92.1 seconds late 2023-11-16 11:07:59,022 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 11:07:59,022 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 11:16:24 2023-11-16 11:07:59,023 INFO 1947952:repeater task noisy_rawtemps_os38 starting 95.0 seconds late 2023-11-16 11:08:02,064 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 11:08:02,064 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 11:16:24 2023-11-16 11:08:02,065 INFO 1947952:repeater task lastensq_wh300 starting 99.1 seconds late 2023-11-16 11:08:05,060 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:08:05,060 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:06:18 2023-11-16 11:08:05,060 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:08:23 2023-11-16 11:08:05,060 INFO 1947952:repeater task quick_wh300 starting 102.1 seconds late 2023-11-16 11:08:11,237 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 11:08:11,238 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:06:18 2023-11-16 11:08:11,238 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:08:23 2023-11-16 11:08:11,238 INFO 1947952:repeater task lastfew_vec_wh300 starting 108.2 seconds late 2023-11-16 11:08:11,867 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:08:11,868 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:06:18 2023-11-16 11:08:11,868 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:08:23 2023-11-16 11:08:11,868 INFO 1947952:repeater task 3dayplots_wh300 starting 107.8 seconds late 2023-11-16 11:08:15,618 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 11:08:15,618 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:11:24 2023-11-16 11:08:25,638 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-16 11:08:28,450 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 11:08:28,451 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:08:18 2023-11-16 11:08:28,451 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:10:23 2023-11-16 11:08:28,451 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-16 11:08:34,493 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 11:08:34,494 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:08:18 2023-11-16 11:08:34,494 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:10:23 2023-11-16 11:08:34,494 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.5 seconds late 2023-11-16 11:08:35,131 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:08:35,131 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:08:18 2023-11-16 11:08:35,131 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:10:23 2023-11-16 11:10:25,400 INFO 1947952:repeater task lastensq_wh300 starting 2.4 seconds late 2023-11-16 11:10:28,216 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 11:10:28,216 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:10:18 2023-11-16 11:10:28,216 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:12:23 2023-11-16 11:10:28,216 INFO 1947952:repeater task quick_wh300 starting 5.2 seconds late 2023-11-16 11:10:34,419 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 11:10:34,420 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:10:18 2023-11-16 11:10:34,420 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:12:23 2023-11-16 11:10:34,420 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-16 11:10:35,050 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:10:35,051 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:10:18 2023-11-16 11:10:35,051 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:12:23 2023-11-16 11:11:25,173 INFO 1947952:repeater task 3dayplots_wh300 starting 1.1 seconds late 2023-11-16 11:11:29,007 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 11:11:29,007 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:16:24 2023-11-16 11:11:29,008 INFO 1947952:repeater task lastensq_os150nb starting 6.0 seconds late 2023-11-16 11:11:31,361 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 11:11:31,361 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:11:18 2023-11-16 11:11:31,361 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:16:23 2023-11-16 11:11:31,362 INFO 1947952:repeater task quick_os150nb starting 8.4 seconds late 2023-11-16 11:11:34,900 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 11:11:34,900 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:11:18 2023-11-16 11:11:34,900 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:16:23 2023-11-16 11:11:34,901 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.9 seconds late 2023-11-16 11:11:35,516 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:11:35,516 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:11:18 2023-11-16 11:11:35,516 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:16:23 2023-11-16 11:11:35,517 INFO 1947952:repeater task 3dayplots_os150nb starting 11.5 seconds late 2023-11-16 11:11:38,902 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:11:38,903 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:16:24 2023-11-16 11:11:38,904 INFO 1947952:repeater task lastensq_os38nb starting 13.9 seconds late 2023-11-16 11:11:41,111 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 11:11:41,111 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:11:20 2023-11-16 11:11:41,111 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:16:25 2023-11-16 11:11:41,112 INFO 1947952:repeater task quick_os38nb starting 16.1 seconds late 2023-11-16 11:11:45,699 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 11:11:45,699 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:11:20 2023-11-16 11:11:45,699 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:16:25 2023-11-16 11:11:45,700 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.7 seconds late 2023-11-16 11:11:46,311 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:11:46,312 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:11:20 2023-11-16 11:11:46,312 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:16:25 2023-11-16 11:11:46,312 INFO 1947952:repeater task 3dayplots_os38nb starting 22.3 seconds late 2023-11-16 11:11:49,813 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 11:11:49,813 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:16:24 2023-11-16 11:12:24,898 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 11:12:27,582 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 11:12:27,582 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:12:18 2023-11-16 11:12:27,582 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:14:23 2023-11-16 11:12:27,583 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-16 11:12:33,985 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 11:12:33,985 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:12:18 2023-11-16 11:12:33,985 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:14:23 2023-11-16 11:12:33,986 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 11:12:34,623 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:12:34,623 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:12:18 2023-11-16 11:12:34,623 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:14:23 2023-11-16 11:14:24,926 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 11:14:27,860 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 11:14:27,860 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:14:18 2023-11-16 11:14:27,860 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:16:23 2023-11-16 11:14:27,861 INFO 1947952:repeater task quick_wh300 starting 4.9 seconds late 2023-11-16 11:14:34,022 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 11:14:34,023 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:14:18 2023-11-16 11:14:34,023 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:16:23 2023-11-16 11:14:34,023 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 11:14:34,654 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:14:34,655 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:14:18 2023-11-16 11:14:34,655 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:16:23 2023-11-16 11:16:24,956 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.9 seconds late 2023-11-16 11:16:27,920 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 11:16:27,920 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 11:26:24 2023-11-16 11:16:27,920 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.9 seconds late 2023-11-16 11:16:30,746 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 11:16:30,746 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 11:26:24 2023-11-16 11:16:30,746 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.7 seconds late 2023-11-16 11:16:33,770 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 11:16:33,771 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 11:26:24 2023-11-16 11:16:33,771 INFO 1947952:repeater task lastensq_wh300 starting 10.8 seconds late 2023-11-16 11:16:36,760 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:16:36,761 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:16:18 2023-11-16 11:16:36,761 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:18:23 2023-11-16 11:16:36,762 INFO 1947952:repeater task quick_wh300 starting 13.8 seconds late 2023-11-16 11:16:42,983 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 11:16:42,983 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:16:18 2023-11-16 11:16:42,983 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:18:23 2023-11-16 11:16:42,984 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.0 seconds late 2023-11-16 11:16:43,617 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:16:43,617 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:16:18 2023-11-16 11:16:43,617 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:18:23 2023-11-16 11:16:43,618 INFO 1947952:repeater task 3dayplots_wh300 starting 19.6 seconds late 2023-11-16 11:16:47,350 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 11:16:47,351 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:21:24 2023-11-16 11:16:47,352 INFO 1947952:repeater task lastensq_os150nb starting 24.4 seconds late 2023-11-16 11:16:49,474 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 11:16:49,475 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:16:18 2023-11-16 11:16:49,475 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:21:23 2023-11-16 11:16:49,475 INFO 1947952:repeater task quick_os150nb starting 26.5 seconds late 2023-11-16 11:16:53,132 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 11:16:53,132 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:16:18 2023-11-16 11:16:53,132 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:21:23 2023-11-16 11:16:53,132 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.1 seconds late 2023-11-16 11:16:53,743 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:16:53,743 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:16:18 2023-11-16 11:16:53,744 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:21:23 2023-11-16 11:16:53,744 INFO 1947952:repeater task 3dayplots_os150nb starting 29.7 seconds late 2023-11-16 11:16:57,155 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:16:57,156 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:21:24 2023-11-16 11:16:57,157 INFO 1947952:repeater task lastensq_os38nb starting 32.2 seconds late 2023-11-16 11:16:59,218 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:16:59,219 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:16:20 2023-11-16 11:16:59,219 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:21:25 2023-11-16 11:16:59,219 INFO 1947952:repeater task quick_os38nb starting 34.2 seconds late 2023-11-16 11:17:03,804 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 11:17:03,805 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:16:20 2023-11-16 11:17:03,805 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:21:25 2023-11-16 11:17:03,805 INFO 1947952:repeater task lastfew_vec_os38nb starting 38.8 seconds late 2023-11-16 11:17:04,413 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:17:04,413 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:16:20 2023-11-16 11:17:04,413 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:21:25 2023-11-16 11:17:04,414 INFO 1947952:repeater task 3dayplots_os38nb starting 40.4 seconds late 2023-11-16 11:17:07,893 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 11:17:07,893 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:21:24 2023-11-16 11:17:07,894 INFO 1947952:repeater task daily_netcdf starting 43.8 seconds late 2023-11-16 11:17:08,735 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 11:17:08,735 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 11:26:24 2023-11-16 11:18:23,936 INFO 1947952:repeater task lastensq_wh300 starting 0.9 seconds late 2023-11-16 11:18:27,426 INFO 1947952:repeater task lastensq_wh300 ended after 3.5 seconds 2023-11-16 11:18:27,427 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:18:18 2023-11-16 11:18:27,427 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:20:23 2023-11-16 11:18:27,427 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-16 11:18:33,781 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 11:18:33,782 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:18:18 2023-11-16 11:18:33,782 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:20:23 2023-11-16 11:18:33,782 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2023-11-16 11:18:34,413 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:18:34,414 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:18:18 2023-11-16 11:18:34,414 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:20:23 2023-11-16 11:20:24,684 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-16 11:20:27,515 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 11:20:27,515 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:20:18 2023-11-16 11:20:27,515 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:22:23 2023-11-16 11:20:27,516 INFO 1947952:repeater task quick_wh300 starting 4.5 seconds late 2023-11-16 11:20:33,562 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 11:20:33,562 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:20:18 2023-11-16 11:20:33,563 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:22:23 2023-11-16 11:20:33,563 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2023-11-16 11:20:34,189 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:20:34,189 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:20:18 2023-11-16 11:20:34,189 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:22:23 2023-11-16 11:21:19,306 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.3 seconds late 2023-11-16 11:21:20,474 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 11:21:20,474 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:20,474 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.4 seconds late 2023-11-16 11:21:20,645 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 11:21:20,645 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:20,646 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.6 seconds late 2023-11-16 11:21:21,931 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 11:21:21,931 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:21,931 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.9 seconds late 2023-11-16 11:21:23,188 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 11:21:23,189 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:23,189 INFO 1947952:repeater task lastensq_os150nb starting 0.2 seconds late 2023-11-16 11:21:25,586 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 11:21:25,587 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:21:18 2023-11-16 11:21:25,587 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:26:23 2023-11-16 11:21:25,587 INFO 1947952:repeater task quick_os150nb starting 2.6 seconds late 2023-11-16 11:21:29,079 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 11:21:29,079 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:21:18 2023-11-16 11:21:29,079 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:26:23 2023-11-16 11:21:29,080 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.1 seconds late 2023-11-16 11:21:29,690 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:21:29,690 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:21:18 2023-11-16 11:21:29,690 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:26:23 2023-11-16 11:21:29,691 INFO 1947952:repeater task 3dayplots_os150nb starting 5.6 seconds late 2023-11-16 11:21:33,107 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:21:33,108 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:26:24 2023-11-16 11:21:33,108 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.1 seconds late 2023-11-16 11:21:34,081 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 11:21:34,081 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:34,082 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.0 seconds late 2023-11-16 11:21:34,249 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 11:21:34,249 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:34,250 INFO 1947952:repeater task beamdiagnostics_os150nb starting 15.2 seconds late 2023-11-16 11:21:35,489 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 11:21:35,489 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:35,490 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.4 seconds late 2023-11-16 11:21:36,752 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 11:21:36,753 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:36,753 INFO 1947952:repeater task lastensq_os38nb starting 11.8 seconds late 2023-11-16 11:21:38,838 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:21:38,839 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:21:20 2023-11-16 11:21:38,839 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:26:25 2023-11-16 11:21:38,839 INFO 1947952:repeater task quick_os38nb starting 13.8 seconds late 2023-11-16 11:21:43,500 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-16 11:21:43,501 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:21:20 2023-11-16 11:21:43,501 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:26:25 2023-11-16 11:21:43,501 INFO 1947952:repeater task lastfew_vec_os38nb starting 18.5 seconds late 2023-11-16 11:21:44,103 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:21:44,104 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:21:20 2023-11-16 11:21:44,104 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:26:25 2023-11-16 11:21:44,104 INFO 1947952:repeater task 3dayplots_os38nb starting 20.1 seconds late 2023-11-16 11:21:47,515 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 11:21:47,515 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:26:24 2023-11-16 11:21:47,515 INFO 1947952:repeater task plot_tsstats_os38nb starting 28.5 seconds late 2023-11-16 11:21:48,521 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 11:21:48,522 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:48,522 INFO 1947952:repeater task grab_lastpings_os38nb starting 29.5 seconds late 2023-11-16 11:21:48,692 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 11:21:48,692 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:48,693 INFO 1947952:repeater task beamdiagnostics_os38nb starting 29.6 seconds late 2023-11-16 11:21:49,921 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 11:21:49,921 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:49,921 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.9 seconds late 2023-11-16 11:21:51,161 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 11:21:51,162 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:51,163 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 31.9 seconds late 2023-11-16 11:21:52,475 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 11:21:52,475 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:52,476 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 33.2 seconds late 2023-11-16 11:21:53,787 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 11:21:53,787 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:53,787 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 34.5 seconds late 2023-11-16 11:21:55,081 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 11:21:55,081 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:21:55,082 INFO 1947952:repeater task plot_hbins starting 35.8 seconds late 2023-11-16 11:21:59,371 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 11:21:59,372 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 11:21:59,372 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 11:36:19 2023-11-16 11:22:04,380 INFO 1947952:repeater task 3dayplots_wh300 starting 40.3 seconds late 2023-11-16 11:22:08,256 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 11:22:08,257 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:26:24 2023-11-16 11:22:23,289 INFO 1947952:repeater task lastensq_wh300 starting 0.3 seconds late 2023-11-16 11:22:26,482 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-16 11:22:26,483 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:22:18 2023-11-16 11:22:26,483 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:24:23 2023-11-16 11:22:26,484 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 11:22:32,576 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 11:22:32,576 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:22:18 2023-11-16 11:22:32,576 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:24:23 2023-11-16 11:22:32,577 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.6 seconds late 2023-11-16 11:22:33,209 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:22:33,209 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:22:18 2023-11-16 11:22:33,209 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:24:23 2023-11-16 11:24:23,510 INFO 1947952:repeater task lastensq_wh300 starting 0.5 seconds late 2023-11-16 11:24:26,482 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:24:26,482 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:24:18 2023-11-16 11:24:26,482 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:26:23 2023-11-16 11:24:26,483 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 11:24:32,664 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 11:24:32,665 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:24:18 2023-11-16 11:24:32,665 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:26:23 2023-11-16 11:24:32,666 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.7 seconds late 2023-11-16 11:24:33,305 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:24:33,306 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:24:18 2023-11-16 11:24:33,306 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:26:23 2023-11-16 11:26:23,604 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-16 11:26:26,275 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 11:26:26,276 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:26:18 2023-11-16 11:26:26,276 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:28:23 2023-11-16 11:26:26,276 INFO 1947952:repeater task quick_wh300 starting 3.3 seconds late 2023-11-16 11:26:32,287 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 11:26:32,287 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:26:18 2023-11-16 11:26:32,287 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:28:23 2023-11-16 11:26:32,288 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2023-11-16 11:26:32,930 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:26:32,930 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:26:18 2023-11-16 11:26:32,930 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:28:23 2023-11-16 11:26:32,930 INFO 1947952:repeater task 3dayplots_wh300 starting 8.9 seconds late 2023-11-16 11:26:36,699 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 11:26:36,700 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:31:24 2023-11-16 11:26:36,701 INFO 1947952:repeater task lastensq_os150nb starting 13.7 seconds late 2023-11-16 11:26:38,829 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 11:26:38,829 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:26:18 2023-11-16 11:26:38,829 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:31:23 2023-11-16 11:26:38,830 INFO 1947952:repeater task quick_os150nb starting 15.8 seconds late 2023-11-16 11:26:42,365 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 11:26:42,365 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:26:18 2023-11-16 11:26:42,365 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:31:23 2023-11-16 11:26:42,366 INFO 1947952:repeater task lastfew_vec_os150nb starting 19.4 seconds late 2023-11-16 11:26:42,981 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:26:42,981 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:26:18 2023-11-16 11:26:42,981 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:31:23 2023-11-16 11:26:42,982 INFO 1947952:repeater task 3dayplots_os150nb starting 18.9 seconds late 2023-11-16 11:26:46,332 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:26:46,332 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:31:24 2023-11-16 11:26:46,334 INFO 1947952:repeater task lastensq_os38nb starting 21.3 seconds late 2023-11-16 11:26:48,361 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 11:26:48,362 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:26:20 2023-11-16 11:26:48,362 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:31:25 2023-11-16 11:26:48,362 INFO 1947952:repeater task quick_os38nb starting 23.4 seconds late 2023-11-16 11:26:52,582 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 11:26:52,582 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:26:20 2023-11-16 11:26:52,582 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:31:25 2023-11-16 11:26:52,583 INFO 1947952:repeater task lastfew_vec_os38nb starting 27.6 seconds late 2023-11-16 11:26:53,199 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:26:53,200 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:26:20 2023-11-16 11:26:53,200 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:31:25 2023-11-16 11:26:53,200 INFO 1947952:repeater task 3dayplots_os38nb starting 29.1 seconds late 2023-11-16 11:26:56,598 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 11:26:56,598 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:31:24 2023-11-16 11:26:56,599 INFO 1947952:repeater task daily_netcdf starting 32.5 seconds late 2023-11-16 11:26:57,418 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 11:26:57,418 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 11:36:24 2023-11-16 11:27:02,425 INFO 1947952:repeater task noisy_rawtemps_os150 starting 38.4 seconds late 2023-11-16 11:27:05,422 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 11:27:05,422 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 11:36:24 2023-11-16 11:27:05,423 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 41.4 seconds late 2023-11-16 11:27:08,301 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 11:27:08,301 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 11:36:24 2023-11-16 11:27:08,302 INFO 1947952:repeater task noisy_rawtemps_os38 starting 44.3 seconds late 2023-11-16 11:27:11,375 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 11:27:11,375 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 11:36:24 2023-11-16 11:28:26,584 INFO 1947952:repeater task lastensq_wh300 starting 3.6 seconds late 2023-11-16 11:28:29,311 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 11:28:29,311 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:28:18 2023-11-16 11:28:29,311 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:30:23 2023-11-16 11:28:29,312 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 11:28:35,824 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 11:28:35,825 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:28:18 2023-11-16 11:28:35,825 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:30:23 2023-11-16 11:28:35,825 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-16 11:28:36,454 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:28:36,455 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:28:18 2023-11-16 11:28:36,455 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:30:23 2023-11-16 11:30:26,716 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-16 11:30:29,481 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 11:30:29,482 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:30:18 2023-11-16 11:30:29,482 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:32:23 2023-11-16 11:30:29,482 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-16 11:30:35,643 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 11:30:35,643 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:30:18 2023-11-16 11:30:35,643 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:32:23 2023-11-16 11:30:35,644 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2023-11-16 11:30:36,278 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:30:36,278 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:30:18 2023-11-16 11:30:36,279 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:32:23 2023-11-16 11:31:26,401 INFO 1947952:repeater task 3dayplots_wh300 starting 2.4 seconds late 2023-11-16 11:31:30,189 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 11:31:30,189 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:36:24 2023-11-16 11:31:30,190 INFO 1947952:repeater task lastensq_os150nb starting 7.2 seconds late 2023-11-16 11:31:32,462 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 11:31:32,462 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:31:18 2023-11-16 11:31:32,463 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:36:23 2023-11-16 11:31:32,463 INFO 1947952:repeater task quick_os150nb starting 9.5 seconds late 2023-11-16 11:31:36,150 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 11:31:36,151 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:31:18 2023-11-16 11:31:36,151 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:36:23 2023-11-16 11:31:36,151 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.2 seconds late 2023-11-16 11:31:36,756 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:31:36,756 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:31:18 2023-11-16 11:31:36,756 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:36:23 2023-11-16 11:31:36,757 INFO 1947952:repeater task 3dayplots_os150nb starting 12.7 seconds late 2023-11-16 11:31:40,121 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:31:40,121 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:36:24 2023-11-16 11:31:40,122 INFO 1947952:repeater task lastensq_os38nb starting 15.1 seconds late 2023-11-16 11:31:42,180 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:31:42,181 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:31:20 2023-11-16 11:31:42,181 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:36:25 2023-11-16 11:31:42,181 INFO 1947952:repeater task quick_os38nb starting 17.2 seconds late 2023-11-16 11:31:46,932 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-16 11:31:46,933 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:31:20 2023-11-16 11:31:46,933 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:36:25 2023-11-16 11:31:46,933 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.9 seconds late 2023-11-16 11:31:47,549 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:31:47,550 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:31:20 2023-11-16 11:31:47,550 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:36:25 2023-11-16 11:31:47,550 INFO 1947952:repeater task 3dayplots_os38nb starting 23.5 seconds late 2023-11-16 11:31:51,011 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 11:31:51,011 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:36:24 2023-11-16 11:32:26,092 INFO 1947952:repeater task lastensq_wh300 starting 3.1 seconds late 2023-11-16 11:32:29,063 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:32:29,063 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:32:18 2023-11-16 11:32:29,063 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:34:23 2023-11-16 11:32:29,064 INFO 1947952:repeater task quick_wh300 starting 6.1 seconds late 2023-11-16 11:32:35,387 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 11:32:35,387 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:32:18 2023-11-16 11:32:35,387 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:34:23 2023-11-16 11:32:35,388 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2023-11-16 11:32:36,024 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:32:36,024 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:32:18 2023-11-16 11:32:36,025 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:34:23 2023-11-16 11:34:26,324 INFO 1947952:repeater task lastensq_wh300 starting 3.3 seconds late 2023-11-16 11:34:29,279 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:34:29,279 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:34:18 2023-11-16 11:34:29,280 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:36:23 2023-11-16 11:34:29,280 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 11:34:35,474 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 11:34:35,474 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:34:18 2023-11-16 11:34:35,474 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:36:23 2023-11-16 11:34:35,475 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2023-11-16 11:34:36,105 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:34:36,105 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:34:18 2023-11-16 11:34:36,105 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:36:23 2023-11-16 11:36:21,371 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.3 seconds late 2023-11-16 11:36:22,545 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 11:36:22,545 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:22,546 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.5 seconds late 2023-11-16 11:36:22,714 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 11:36:22,715 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:22,715 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.7 seconds late 2023-11-16 11:36:23,999 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 11:36:23,999 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:23,999 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.9 seconds late 2023-11-16 11:36:25,248 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 11:36:25,249 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:25,249 INFO 1947952:repeater task lastensq_os150nb starting 2.2 seconds late 2023-11-16 11:36:27,326 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 11:36:27,327 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:36:18 2023-11-16 11:36:27,327 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:41:23 2023-11-16 11:36:27,327 INFO 1947952:repeater task quick_os150nb starting 4.3 seconds late 2023-11-16 11:36:31,038 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 11:36:31,039 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:36:18 2023-11-16 11:36:31,039 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:41:23 2023-11-16 11:36:31,039 INFO 1947952:repeater task lastfew_vec_os150nb starting 8.0 seconds late 2023-11-16 11:36:31,650 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:36:31,650 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:36:18 2023-11-16 11:36:31,651 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:41:23 2023-11-16 11:36:31,651 INFO 1947952:repeater task 3dayplots_os150nb starting 7.6 seconds late 2023-11-16 11:36:35,077 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:36:35,077 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:41:24 2023-11-16 11:36:35,078 INFO 1947952:repeater task plot_tsstats_os150nb starting 16.0 seconds late 2023-11-16 11:36:36,042 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 11:36:36,042 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:36,043 INFO 1947952:repeater task grab_lastpings_os150nb starting 17.0 seconds late 2023-11-16 11:36:36,211 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 11:36:36,211 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:36,211 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.2 seconds late 2023-11-16 11:36:37,459 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 11:36:37,459 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:37,460 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.4 seconds late 2023-11-16 11:36:38,700 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 11:36:38,700 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:38,701 INFO 1947952:repeater task lastensq_os38nb starting 13.7 seconds late 2023-11-16 11:36:40,754 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:36:40,754 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:36:20 2023-11-16 11:36:40,754 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:41:25 2023-11-16 11:36:40,755 INFO 1947952:repeater task quick_os38nb starting 15.8 seconds late 2023-11-16 11:36:45,788 INFO 1947952:repeater task quick_os38nb ended after 5.0 seconds 2023-11-16 11:36:45,789 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:36:20 2023-11-16 11:36:45,789 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:41:25 2023-11-16 11:36:45,789 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.8 seconds late 2023-11-16 11:36:46,400 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:36:46,400 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:36:20 2023-11-16 11:36:46,400 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:41:25 2023-11-16 11:36:46,401 INFO 1947952:repeater task 3dayplots_os38nb starting 22.3 seconds late 2023-11-16 11:36:49,812 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 11:36:49,813 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:41:24 2023-11-16 11:36:49,813 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.8 seconds late 2023-11-16 11:36:50,818 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 11:36:50,819 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:50,819 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.8 seconds late 2023-11-16 11:36:50,988 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 11:36:50,989 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:50,989 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.9 seconds late 2023-11-16 11:36:52,223 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 11:36:52,223 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:52,224 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.2 seconds late 2023-11-16 11:36:53,464 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 11:36:53,464 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:36:53,464 INFO 1947952:repeater task daily_netcdf starting 29.4 seconds late 2023-11-16 11:36:54,363 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-16 11:36:54,364 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 11:46:24 2023-11-16 11:36:54,364 INFO 1947952:repeater task ggatime_diagnostics starting 35.3 seconds late 2023-11-16 11:37:10,781 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-16 11:37:10,781 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:37:10,782 INFO 1947952:repeater task ggatime_diagnostics starting 51.7 seconds late 2023-11-16 11:37:26,819 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 11:37:26,819 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:37:26,819 INFO 1947952:repeater task ggatime_diagnostics starting 67.8 seconds late 2023-11-16 11:37:42,939 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-16 11:37:42,940 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:37:42,940 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 83.7 seconds late 2023-11-16 11:37:44,283 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 11:37:44,283 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:37:44,284 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 85.0 seconds late 2023-11-16 11:37:45,579 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 11:37:45,579 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:37:45,580 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 86.3 seconds late 2023-11-16 11:37:46,895 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 11:37:46,895 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:37:46,896 INFO 1947952:repeater task plot_hbins starting 87.6 seconds late 2023-11-16 11:37:51,147 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 11:37:51,148 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 11:37:51,148 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 11:51:19 2023-11-16 11:37:51,148 INFO 1947952:repeater task report_generator starting 91.9 seconds late 2023-11-16 11:37:57,255 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 11:37:57,256 INFO 1947952:repeater task report_generator ended after 6.1 seconds 2023-11-16 11:37:57,256 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 12:36:19 2023-11-16 11:38:02,257 INFO 1947952:repeater task noisy_rawtemps_os150 starting 98.2 seconds late 2023-11-16 11:38:05,203 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 11:38:05,203 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 11:46:24 2023-11-16 11:38:05,204 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 101.2 seconds late 2023-11-16 11:38:08,043 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 11:38:08,043 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 11:46:24 2023-11-16 11:38:08,044 INFO 1947952:repeater task noisy_rawtemps_os38 starting 104.0 seconds late 2023-11-16 11:38:11,107 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 11:38:11,107 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 11:46:24 2023-11-16 11:38:11,108 INFO 1947952:repeater task lastensq_wh300 starting 108.1 seconds late 2023-11-16 11:38:14,092 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:38:14,093 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:36:18 2023-11-16 11:38:14,093 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:38:23 2023-11-16 11:38:14,094 INFO 1947952:repeater task quick_wh300 starting 111.1 seconds late 2023-11-16 11:38:20,995 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-16 11:38:20,996 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:36:18 2023-11-16 11:38:20,996 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:38:23 2023-11-16 11:38:20,996 INFO 1947952:repeater task lastfew_vec_wh300 starting 118.0 seconds late 2023-11-16 11:38:21,637 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:38:21,637 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:36:18 2023-11-16 11:38:21,637 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:38:23 2023-11-16 11:38:21,637 INFO 1947952:repeater task 3dayplots_wh300 starting 117.6 seconds late 2023-11-16 11:38:25,387 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 11:38:25,388 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:41:24 2023-11-16 11:38:30,395 INFO 1947952:repeater task lastensq_wh300 starting 7.4 seconds late 2023-11-16 11:38:33,119 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 11:38:33,119 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:38:18 2023-11-16 11:38:33,119 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:40:23 2023-11-16 11:38:33,120 INFO 1947952:repeater task quick_wh300 starting 10.1 seconds late 2023-11-16 11:38:39,412 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 11:38:39,413 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:38:18 2023-11-16 11:38:39,413 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:40:23 2023-11-16 11:38:39,413 INFO 1947952:repeater task lastfew_vec_wh300 starting 16.4 seconds late 2023-11-16 11:38:40,045 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:38:40,045 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:38:18 2023-11-16 11:38:40,045 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:40:23 2023-11-16 11:40:25,320 INFO 1947952:repeater task lastensq_wh300 starting 2.3 seconds late 2023-11-16 11:40:28,278 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:40:28,279 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:40:18 2023-11-16 11:40:28,279 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:42:23 2023-11-16 11:40:28,279 INFO 1947952:repeater task quick_wh300 starting 5.3 seconds late 2023-11-16 11:40:34,821 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 11:40:34,822 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:40:18 2023-11-16 11:40:34,822 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:42:23 2023-11-16 11:40:34,822 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-16 11:40:35,459 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:40:35,460 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:40:18 2023-11-16 11:40:35,460 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:42:23 2023-11-16 11:41:25,590 INFO 1947952:repeater task 3dayplots_wh300 starting 1.5 seconds late 2023-11-16 11:41:29,400 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 11:41:29,401 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:46:24 2023-11-16 11:41:29,402 INFO 1947952:repeater task lastensq_os150nb starting 6.4 seconds late 2023-11-16 11:41:31,672 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 11:41:31,673 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:41:18 2023-11-16 11:41:31,673 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:46:23 2023-11-16 11:41:31,674 INFO 1947952:repeater task quick_os150nb starting 8.7 seconds late 2023-11-16 11:41:35,659 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-16 11:41:35,659 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:41:18 2023-11-16 11:41:35,660 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:46:23 2023-11-16 11:41:35,660 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.7 seconds late 2023-11-16 11:41:36,262 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:41:36,262 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:41:18 2023-11-16 11:41:36,263 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:46:23 2023-11-16 11:41:36,263 INFO 1947952:repeater task 3dayplots_os150nb starting 12.2 seconds late 2023-11-16 11:41:39,684 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:41:39,684 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:46:24 2023-11-16 11:41:39,685 INFO 1947952:repeater task lastensq_os38nb starting 14.7 seconds late 2023-11-16 11:41:41,746 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:41:41,747 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:41:20 2023-11-16 11:41:41,747 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:46:25 2023-11-16 11:41:41,747 INFO 1947952:repeater task quick_os38nb starting 16.7 seconds late 2023-11-16 11:41:46,028 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 11:41:46,028 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:41:20 2023-11-16 11:41:46,028 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:46:25 2023-11-16 11:41:46,029 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.0 seconds late 2023-11-16 11:41:46,642 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:41:46,642 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:41:20 2023-11-16 11:41:46,642 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:46:25 2023-11-16 11:41:46,643 INFO 1947952:repeater task 3dayplots_os38nb starting 22.6 seconds late 2023-11-16 11:41:50,104 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 11:41:50,105 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:46:24 2023-11-16 11:42:25,190 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-16 11:42:27,827 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 11:42:27,828 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:42:18 2023-11-16 11:42:27,828 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:44:23 2023-11-16 11:42:27,828 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 11:42:34,262 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 11:42:34,262 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:42:18 2023-11-16 11:42:34,263 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:44:23 2023-11-16 11:42:34,263 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2023-11-16 11:42:34,896 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:42:34,896 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:42:18 2023-11-16 11:42:34,896 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:44:23 2023-11-16 11:44:25,184 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-16 11:44:28,097 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 11:44:28,098 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:44:18 2023-11-16 11:44:28,098 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:46:23 2023-11-16 11:44:28,098 INFO 1947952:repeater task quick_wh300 starting 5.1 seconds late 2023-11-16 11:44:34,813 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 11:44:34,814 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:44:18 2023-11-16 11:44:34,814 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:46:23 2023-11-16 11:44:34,814 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-16 11:44:35,442 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:44:35,442 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:44:18 2023-11-16 11:44:35,442 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:46:23 2023-11-16 11:46:25,713 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.7 seconds late 2023-11-16 11:46:28,675 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 11:46:28,675 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 11:56:24 2023-11-16 11:46:28,676 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 4.6 seconds late 2023-11-16 11:46:31,495 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 11:46:31,495 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 11:56:24 2023-11-16 11:46:31,496 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.4 seconds late 2023-11-16 11:46:34,556 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 11:46:34,556 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 11:56:24 2023-11-16 11:46:34,556 INFO 1947952:repeater task lastensq_wh300 starting 11.6 seconds late 2023-11-16 11:46:37,632 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 11:46:37,632 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:46:18 2023-11-16 11:46:37,633 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:48:23 2023-11-16 11:46:37,633 INFO 1947952:repeater task quick_wh300 starting 14.6 seconds late 2023-11-16 11:46:43,499 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 11:46:43,500 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:46:18 2023-11-16 11:46:43,500 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:48:23 2023-11-16 11:46:43,500 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.5 seconds late 2023-11-16 11:46:44,136 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:46:44,137 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:46:18 2023-11-16 11:46:44,137 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:48:23 2023-11-16 11:46:44,137 INFO 1947952:repeater task 3dayplots_wh300 starting 20.1 seconds late 2023-11-16 11:46:47,953 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 11:46:47,953 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:51:24 2023-11-16 11:46:47,954 INFO 1947952:repeater task lastensq_os150nb starting 25.0 seconds late 2023-11-16 11:46:50,441 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 11:46:50,441 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:46:18 2023-11-16 11:46:50,442 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:51:23 2023-11-16 11:46:50,442 INFO 1947952:repeater task quick_os150nb starting 27.4 seconds late 2023-11-16 11:46:54,182 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 11:46:54,182 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:46:18 2023-11-16 11:46:54,183 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:51:23 2023-11-16 11:46:54,183 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.2 seconds late 2023-11-16 11:46:54,796 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:46:54,797 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:46:18 2023-11-16 11:46:54,797 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:51:23 2023-11-16 11:46:54,797 INFO 1947952:repeater task 3dayplots_os150nb starting 30.7 seconds late 2023-11-16 11:46:58,268 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 11:46:58,268 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:51:24 2023-11-16 11:46:58,269 INFO 1947952:repeater task lastensq_os38nb starting 33.3 seconds late 2023-11-16 11:47:00,295 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 11:47:00,295 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:46:20 2023-11-16 11:47:00,295 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:51:25 2023-11-16 11:47:00,296 INFO 1947952:repeater task quick_os38nb starting 35.3 seconds late 2023-11-16 11:47:04,600 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 11:47:04,600 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:46:20 2023-11-16 11:47:04,600 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:51:25 2023-11-16 11:47:04,601 INFO 1947952:repeater task lastfew_vec_os38nb starting 39.6 seconds late 2023-11-16 11:47:05,214 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:47:05,215 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:46:20 2023-11-16 11:47:05,215 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:51:25 2023-11-16 11:47:05,215 INFO 1947952:repeater task 3dayplots_os38nb starting 41.2 seconds late 2023-11-16 11:47:08,646 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 11:47:08,646 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:51:24 2023-11-16 11:47:08,648 INFO 1947952:repeater task daily_netcdf starting 44.6 seconds late 2023-11-16 11:47:09,449 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 11:47:09,450 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 11:56:24 2023-11-16 11:48:24,644 INFO 1947952:repeater task lastensq_wh300 starting 1.6 seconds late 2023-11-16 11:48:27,532 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 11:48:27,532 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:48:18 2023-11-16 11:48:27,532 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:50:23 2023-11-16 11:48:27,533 INFO 1947952:repeater task quick_wh300 starting 4.5 seconds late 2023-11-16 11:48:33,510 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 11:48:33,510 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:48:18 2023-11-16 11:48:33,510 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:50:23 2023-11-16 11:48:33,511 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-16 11:48:34,144 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:48:34,145 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:48:18 2023-11-16 11:48:34,145 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:50:23 2023-11-16 11:50:24,431 INFO 1947952:repeater task lastensq_wh300 starting 1.4 seconds late 2023-11-16 11:50:27,242 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 11:50:27,243 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:50:18 2023-11-16 11:50:27,243 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:52:23 2023-11-16 11:50:27,243 INFO 1947952:repeater task quick_wh300 starting 4.2 seconds late 2023-11-16 11:50:33,302 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 11:50:33,302 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:50:18 2023-11-16 11:50:33,302 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:52:23 2023-11-16 11:50:33,303 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-16 11:50:33,947 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:50:33,947 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:50:18 2023-11-16 11:50:33,948 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:52:23 2023-11-16 11:51:19,061 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.0 seconds late 2023-11-16 11:51:20,231 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 11:51:20,231 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:20,232 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.2 seconds late 2023-11-16 11:51:20,400 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 11:51:20,400 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:20,400 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.4 seconds late 2023-11-16 11:51:21,656 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 11:51:21,656 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:21,656 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.6 seconds late 2023-11-16 11:51:22,930 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 11:51:22,930 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:22,932 INFO 1947952:repeater task plot_tsstats_os150nb starting 3.9 seconds late 2023-11-16 11:51:23,903 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 11:51:23,904 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:23,904 INFO 1947952:repeater task grab_lastpings_os150nb starting 4.9 seconds late 2023-11-16 11:51:24,072 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 11:51:24,073 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:24,074 INFO 1947952:repeater task beamdiagnostics_os150nb starting 5.0 seconds late 2023-11-16 11:51:25,247 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 11:51:25,247 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:25,248 INFO 1947952:repeater task beamdiagnostics_os150nb starting 6.2 seconds late 2023-11-16 11:51:26,491 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 11:51:26,491 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:26,492 INFO 1947952:repeater task lastensq_os38nb starting 1.5 seconds late 2023-11-16 11:51:28,629 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:51:28,629 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:51:20 2023-11-16 11:51:28,629 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 11:56:25 2023-11-16 11:51:28,629 INFO 1947952:repeater task quick_os38nb starting 3.6 seconds late 2023-11-16 11:51:32,781 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 11:51:32,781 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:51:20 2023-11-16 11:51:32,781 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 11:56:25 2023-11-16 11:51:32,782 INFO 1947952:repeater task lastfew_vec_os38nb starting 7.8 seconds late 2023-11-16 11:51:33,393 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:51:33,394 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:51:20 2023-11-16 11:51:33,394 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 11:56:25 2023-11-16 11:51:33,394 INFO 1947952:repeater task 3dayplots_os38nb starting 9.3 seconds late 2023-11-16 11:51:36,858 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 11:51:36,858 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 11:56:24 2023-11-16 11:51:36,859 INFO 1947952:repeater task plot_tsstats_os38nb starting 17.8 seconds late 2023-11-16 11:51:37,865 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 11:51:37,865 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:37,866 INFO 1947952:repeater task grab_lastpings_os38nb starting 18.8 seconds late 2023-11-16 11:51:38,034 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 11:51:38,034 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:38,034 INFO 1947952:repeater task beamdiagnostics_os38nb starting 19.0 seconds late 2023-11-16 11:51:39,295 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 11:51:39,295 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:39,296 INFO 1947952:repeater task beamdiagnostics_os38nb starting 20.2 seconds late 2023-11-16 11:51:40,610 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 11:51:40,610 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:40,612 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 21.3 seconds late 2023-11-16 11:51:41,959 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 11:51:41,959 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:41,960 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 22.7 seconds late 2023-11-16 11:51:43,225 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 11:51:43,225 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:43,226 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 24.0 seconds late 2023-11-16 11:51:44,457 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 11:51:44,457 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:44,457 INFO 1947952:repeater task plot_hbins starting 25.2 seconds late 2023-11-16 11:51:48,709 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 11:51:48,709 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 11:51:48,710 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 12:06:19 2023-11-16 11:51:53,718 INFO 1947952:repeater task 3dayplots_wh300 starting 29.7 seconds late 2023-11-16 11:51:57,577 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 11:51:57,578 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 11:56:24 2023-11-16 11:51:57,579 INFO 1947952:repeater task lastensq_os150nb starting 34.6 seconds late 2023-11-16 11:51:59,922 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 11:51:59,922 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:51:18 2023-11-16 11:51:59,922 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 11:56:23 2023-11-16 11:51:59,923 INFO 1947952:repeater task quick_os150nb starting 36.9 seconds late 2023-11-16 11:52:03,655 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 11:52:03,656 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:51:18 2023-11-16 11:52:03,656 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 11:56:23 2023-11-16 11:52:03,656 INFO 1947952:repeater task lastfew_vec_os150nb starting 40.7 seconds late 2023-11-16 11:52:04,263 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:52:04,263 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:51:18 2023-11-16 11:52:04,263 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 11:56:23 2023-11-16 11:52:04,264 INFO 1947952:repeater task 3dayplots_os150nb starting 40.2 seconds late 2023-11-16 11:52:07,715 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 11:52:07,716 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 11:56:24 2023-11-16 11:52:27,758 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-16 11:52:30,496 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 11:52:30,496 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:52:18 2023-11-16 11:52:30,496 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:54:23 2023-11-16 11:52:30,497 INFO 1947952:repeater task quick_wh300 starting 7.5 seconds late 2023-11-16 11:52:37,249 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-16 11:52:37,249 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:52:18 2023-11-16 11:52:37,249 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:54:23 2023-11-16 11:52:37,250 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-16 11:52:37,888 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:52:37,888 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:52:18 2023-11-16 11:52:37,888 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:54:23 2023-11-16 11:54:23,146 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 11:54:26,075 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 11:54:26,075 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:54:18 2023-11-16 11:54:26,076 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:56:23 2023-11-16 11:54:26,076 INFO 1947952:repeater task quick_wh300 starting 3.1 seconds late 2023-11-16 11:54:32,046 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 11:54:32,046 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:54:18 2023-11-16 11:54:32,046 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:56:23 2023-11-16 11:54:32,047 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.0 seconds late 2023-11-16 11:54:32,674 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:54:32,674 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:54:18 2023-11-16 11:54:32,674 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:56:23 2023-11-16 11:56:27,965 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.9 seconds late 2023-11-16 11:56:30,889 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 11:56:30,889 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 12:06:24 2023-11-16 11:56:30,890 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.8 seconds late 2023-11-16 11:56:33,637 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-16 11:56:33,637 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 12:06:24 2023-11-16 11:56:33,637 INFO 1947952:repeater task noisy_rawtemps_os38 starting 9.6 seconds late 2023-11-16 11:56:36,671 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 11:56:36,671 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 12:06:24 2023-11-16 11:56:36,672 INFO 1947952:repeater task lastensq_wh300 starting 13.7 seconds late 2023-11-16 11:56:39,467 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 11:56:39,467 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:56:18 2023-11-16 11:56:39,467 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 11:58:23 2023-11-16 11:56:39,468 INFO 1947952:repeater task quick_wh300 starting 16.5 seconds late 2023-11-16 11:56:45,104 INFO 1947952:repeater task quick_wh300 ended after 5.6 seconds 2023-11-16 11:56:45,105 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:56:18 2023-11-16 11:56:45,105 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 11:58:23 2023-11-16 11:56:45,105 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.1 seconds late 2023-11-16 11:56:45,735 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:56:45,736 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:56:18 2023-11-16 11:56:45,736 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 11:58:23 2023-11-16 11:56:45,736 INFO 1947952:repeater task 3dayplots_wh300 starting 21.7 seconds late 2023-11-16 11:56:49,575 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 11:56:49,575 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:01:24 2023-11-16 11:56:49,577 INFO 1947952:repeater task lastensq_os150nb starting 26.6 seconds late 2023-11-16 11:56:51,973 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 11:56:51,974 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:56:18 2023-11-16 11:56:51,974 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:01:23 2023-11-16 11:56:51,974 INFO 1947952:repeater task quick_os150nb starting 29.0 seconds late 2023-11-16 11:56:55,631 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 11:56:55,631 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:56:18 2023-11-16 11:56:55,631 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:01:23 2023-11-16 11:56:55,631 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.6 seconds late 2023-11-16 11:56:56,241 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 11:56:56,241 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 11:56:18 2023-11-16 11:56:56,241 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:01:23 2023-11-16 11:56:56,242 INFO 1947952:repeater task 3dayplots_os150nb starting 32.2 seconds late 2023-11-16 11:56:59,690 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 11:56:59,690 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:01:24 2023-11-16 11:56:59,691 INFO 1947952:repeater task lastensq_os38nb starting 34.7 seconds late 2023-11-16 11:57:01,745 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 11:57:01,745 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:56:20 2023-11-16 11:57:01,745 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:01:25 2023-11-16 11:57:01,746 INFO 1947952:repeater task quick_os38nb starting 36.7 seconds late 2023-11-16 11:57:05,807 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 11:57:05,807 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:56:20 2023-11-16 11:57:05,807 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:01:25 2023-11-16 11:57:05,808 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.8 seconds late 2023-11-16 11:57:06,426 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 11:57:06,426 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 11:56:20 2023-11-16 11:57:06,426 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:01:25 2023-11-16 11:57:06,427 INFO 1947952:repeater task 3dayplots_os38nb starting 42.4 seconds late 2023-11-16 11:57:09,831 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 11:57:09,831 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:01:24 2023-11-16 11:57:09,833 INFO 1947952:repeater task daily_netcdf starting 45.8 seconds late 2023-11-16 11:57:10,639 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 11:57:10,639 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 12:06:24 2023-11-16 11:58:25,843 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-16 11:58:28,824 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 11:58:28,824 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:58:18 2023-11-16 11:58:28,824 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:00:23 2023-11-16 11:58:28,824 INFO 1947952:repeater task quick_wh300 starting 5.8 seconds late 2023-11-16 11:58:34,899 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 11:58:34,899 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:58:18 2023-11-16 11:58:34,899 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:00:23 2023-11-16 11:58:34,900 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-16 11:58:35,537 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 11:58:35,537 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 11:58:18 2023-11-16 11:58:35,537 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:00:23 2023-11-16 12:00:25,812 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-16 12:00:28,576 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:00:28,577 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:00:18 2023-11-16 12:00:28,577 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:02:23 2023-11-16 12:00:28,577 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-16 12:00:34,543 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 12:00:34,544 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:00:18 2023-11-16 12:00:34,544 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:02:23 2023-11-16 12:00:34,544 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.5 seconds late 2023-11-16 12:00:35,173 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:00:35,173 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:00:18 2023-11-16 12:00:35,173 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:02:23 2023-11-16 12:01:25,278 INFO 1947952:repeater task 3dayplots_wh300 starting 1.2 seconds late 2023-11-16 12:01:29,123 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:01:29,123 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:06:24 2023-11-16 12:01:29,125 INFO 1947952:repeater task lastensq_os150nb starting 6.1 seconds late 2023-11-16 12:01:31,538 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 12:01:31,539 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:01:18 2023-11-16 12:01:31,539 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:06:23 2023-11-16 12:01:31,539 INFO 1947952:repeater task quick_os150nb starting 8.5 seconds late 2023-11-16 12:01:35,327 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 12:01:35,327 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:01:18 2023-11-16 12:01:35,327 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:06:23 2023-11-16 12:01:35,328 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.3 seconds late 2023-11-16 12:01:35,930 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:01:35,931 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:01:18 2023-11-16 12:01:35,931 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:06:23 2023-11-16 12:01:35,931 INFO 1947952:repeater task 3dayplots_os150nb starting 11.9 seconds late 2023-11-16 12:01:39,337 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:01:39,337 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:06:24 2023-11-16 12:01:39,338 INFO 1947952:repeater task lastensq_os38nb starting 14.3 seconds late 2023-11-16 12:01:41,598 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-16 12:01:41,599 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:01:20 2023-11-16 12:01:41,599 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:06:25 2023-11-16 12:01:41,599 INFO 1947952:repeater task quick_os38nb starting 16.6 seconds late 2023-11-16 12:01:46,224 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 12:01:46,225 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:01:20 2023-11-16 12:01:46,225 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:06:25 2023-11-16 12:01:46,225 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.2 seconds late 2023-11-16 12:01:46,839 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:01:46,839 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:01:20 2023-11-16 12:01:46,840 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:06:25 2023-11-16 12:01:46,840 INFO 1947952:repeater task 3dayplots_os38nb starting 22.8 seconds late 2023-11-16 12:01:50,346 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:01:50,346 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:06:24 2023-11-16 12:02:25,484 INFO 1947952:repeater task lastensq_wh300 starting 2.5 seconds late 2023-11-16 12:02:28,227 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 12:02:28,227 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:02:18 2023-11-16 12:02:28,227 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:04:23 2023-11-16 12:02:28,228 INFO 1947952:repeater task quick_wh300 starting 5.2 seconds late 2023-11-16 12:02:34,615 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:02:34,616 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:02:18 2023-11-16 12:02:34,616 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:04:23 2023-11-16 12:02:34,616 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2023-11-16 12:02:35,248 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:02:35,248 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:02:18 2023-11-16 12:02:35,248 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:04:23 2023-11-16 12:04:25,556 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-16 12:04:28,446 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:04:28,447 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:04:18 2023-11-16 12:04:28,447 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:06:23 2023-11-16 12:04:28,447 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-16 12:04:34,679 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 12:04:34,679 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:04:18 2023-11-16 12:04:34,679 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:06:23 2023-11-16 12:04:34,680 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-16 12:04:35,310 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:04:35,310 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:04:18 2023-11-16 12:04:35,310 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:06:23 2023-11-16 12:06:20,579 INFO 1947952:repeater task plot_tsstats_wh300 starting 1.5 seconds late 2023-11-16 12:06:21,782 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 12:06:21,783 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:21,783 INFO 1947952:repeater task grab_lastpings_wh300 starting 2.7 seconds late 2023-11-16 12:06:21,951 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 12:06:21,951 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:21,952 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.9 seconds late 2023-11-16 12:06:23,194 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 12:06:23,194 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:23,195 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.1 seconds late 2023-11-16 12:06:24,469 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 12:06:24,470 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:24,470 INFO 1947952:repeater task lastensq_os150nb starting 1.5 seconds late 2023-11-16 12:06:26,843 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 12:06:26,843 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:06:18 2023-11-16 12:06:26,844 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:11:23 2023-11-16 12:06:26,844 INFO 1947952:repeater task quick_os150nb starting 3.8 seconds late 2023-11-16 12:06:30,772 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 12:06:30,773 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:06:18 2023-11-16 12:06:30,773 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:11:23 2023-11-16 12:06:30,773 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.8 seconds late 2023-11-16 12:06:31,379 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:06:31,379 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:06:18 2023-11-16 12:06:31,379 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:11:23 2023-11-16 12:06:31,380 INFO 1947952:repeater task 3dayplots_os150nb starting 7.3 seconds late 2023-11-16 12:06:34,802 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:06:34,802 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:11:24 2023-11-16 12:06:34,802 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.8 seconds late 2023-11-16 12:06:35,766 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 12:06:35,766 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:35,767 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.7 seconds late 2023-11-16 12:06:35,934 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 12:06:35,934 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:35,935 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.9 seconds late 2023-11-16 12:06:37,229 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 12:06:37,229 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:37,229 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.2 seconds late 2023-11-16 12:06:38,461 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 12:06:38,461 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:38,461 INFO 1947952:repeater task lastensq_os38nb starting 13.5 seconds late 2023-11-16 12:06:40,664 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 12:06:40,665 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:06:20 2023-11-16 12:06:40,665 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:11:25 2023-11-16 12:06:40,665 INFO 1947952:repeater task quick_os38nb starting 15.7 seconds late 2023-11-16 12:06:45,078 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 12:06:45,079 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:06:20 2023-11-16 12:06:45,079 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:11:25 2023-11-16 12:06:45,079 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.1 seconds late 2023-11-16 12:06:45,697 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:06:45,697 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:06:20 2023-11-16 12:06:45,697 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:11:25 2023-11-16 12:06:45,698 INFO 1947952:repeater task 3dayplots_os38nb starting 21.6 seconds late 2023-11-16 12:06:49,152 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:06:49,152 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:11:24 2023-11-16 12:06:49,152 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.1 seconds late 2023-11-16 12:06:50,142 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 12:06:50,142 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:50,143 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.1 seconds late 2023-11-16 12:06:50,310 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 12:06:50,310 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:50,311 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.3 seconds late 2023-11-16 12:06:51,565 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 12:06:51,565 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:51,566 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.5 seconds late 2023-11-16 12:06:52,777 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 12:06:52,777 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:06:52,778 INFO 1947952:repeater task daily_netcdf starting 28.7 seconds late 2023-11-16 12:06:53,576 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 12:06:53,576 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 12:16:24 2023-11-16 12:06:53,576 INFO 1947952:repeater task ggatime_diagnostics starting 34.5 seconds late 2023-11-16 12:07:09,976 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-16 12:07:09,977 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:07:09,977 INFO 1947952:repeater task ggatime_diagnostics starting 50.9 seconds late 2023-11-16 12:07:26,284 INFO 1947952:repeater task ggatime_diagnostics ended after 16.3 seconds 2023-11-16 12:07:26,284 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:07:26,285 INFO 1947952:repeater task ggatime_diagnostics starting 67.2 seconds late 2023-11-16 12:07:42,480 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-16 12:07:42,480 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:07:42,481 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 83.2 seconds late 2023-11-16 12:07:43,770 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 12:07:43,771 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:07:43,771 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 84.5 seconds late 2023-11-16 12:07:45,074 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 12:07:45,074 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:07:45,074 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 85.8 seconds late 2023-11-16 12:07:46,317 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 12:07:46,317 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:07:46,318 INFO 1947952:repeater task plot_hbins starting 87.0 seconds late 2023-11-16 12:07:50,462 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 12:07:50,463 INFO 1947952:repeater task plot_hbins ended after 4.1 seconds 2023-11-16 12:07:50,463 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 12:21:19 2023-11-16 12:07:55,465 INFO 1947952:repeater task noisy_rawtemps_os150 starting 91.4 seconds late 2023-11-16 12:07:58,377 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 12:07:58,377 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 12:16:24 2023-11-16 12:07:58,378 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 94.3 seconds late 2023-11-16 12:08:01,192 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 12:08:01,192 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 12:16:24 2023-11-16 12:08:01,193 INFO 1947952:repeater task noisy_rawtemps_os38 starting 97.1 seconds late 2023-11-16 12:08:04,219 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 12:08:04,220 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 12:16:24 2023-11-16 12:08:04,220 INFO 1947952:repeater task lastensq_wh300 starting 101.2 seconds late 2023-11-16 12:08:07,232 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 12:08:07,232 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:06:18 2023-11-16 12:08:07,232 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:08:23 2023-11-16 12:08:07,233 INFO 1947952:repeater task quick_wh300 starting 104.2 seconds late 2023-11-16 12:08:13,353 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 12:08:13,354 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:06:18 2023-11-16 12:08:13,354 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:08:23 2023-11-16 12:08:13,354 INFO 1947952:repeater task lastfew_vec_wh300 starting 110.4 seconds late 2023-11-16 12:08:13,990 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:08:13,991 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:06:18 2023-11-16 12:08:13,991 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:08:23 2023-11-16 12:08:13,991 INFO 1947952:repeater task 3dayplots_wh300 starting 109.9 seconds late 2023-11-16 12:08:17,777 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:08:17,777 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:11:24 2023-11-16 12:08:27,800 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-16 12:08:30,700 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:08:30,701 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:08:18 2023-11-16 12:08:30,701 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:10:23 2023-11-16 12:08:30,701 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-16 12:08:36,974 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 12:08:36,975 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:08:18 2023-11-16 12:08:36,975 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:10:23 2023-11-16 12:08:36,975 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.0 seconds late 2023-11-16 12:08:37,604 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:08:37,604 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:08:18 2023-11-16 12:08:37,604 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:10:23 2023-11-16 12:10:27,885 INFO 1947952:repeater task lastensq_wh300 starting 4.9 seconds late 2023-11-16 12:10:30,589 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 12:10:30,590 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:10:18 2023-11-16 12:10:30,590 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:12:23 2023-11-16 12:10:30,590 INFO 1947952:repeater task quick_wh300 starting 7.6 seconds late 2023-11-16 12:10:36,802 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 12:10:36,803 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:10:18 2023-11-16 12:10:36,803 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:12:23 2023-11-16 12:10:36,803 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.8 seconds late 2023-11-16 12:10:37,429 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:10:37,429 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:10:18 2023-11-16 12:10:37,429 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:12:23 2023-11-16 12:11:27,560 INFO 1947952:repeater task 3dayplots_wh300 starting 3.5 seconds late 2023-11-16 12:11:31,342 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:11:31,342 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:16:24 2023-11-16 12:11:31,344 INFO 1947952:repeater task lastensq_os150nb starting 8.3 seconds late 2023-11-16 12:11:33,517 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 12:11:33,518 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:11:18 2023-11-16 12:11:33,518 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:16:23 2023-11-16 12:11:33,518 INFO 1947952:repeater task quick_os150nb starting 10.5 seconds late 2023-11-16 12:11:37,189 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 12:11:37,190 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:11:18 2023-11-16 12:11:37,190 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:16:23 2023-11-16 12:11:37,190 INFO 1947952:repeater task lastfew_vec_os150nb starting 14.2 seconds late 2023-11-16 12:11:37,795 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:11:37,796 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:11:18 2023-11-16 12:11:37,796 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:16:23 2023-11-16 12:11:37,796 INFO 1947952:repeater task 3dayplots_os150nb starting 13.7 seconds late 2023-11-16 12:11:41,204 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:11:41,204 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:16:24 2023-11-16 12:11:41,205 INFO 1947952:repeater task lastensq_os38nb starting 16.2 seconds late 2023-11-16 12:11:43,331 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 12:11:43,331 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:11:20 2023-11-16 12:11:43,331 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:16:25 2023-11-16 12:11:43,332 INFO 1947952:repeater task quick_os38nb starting 18.3 seconds late 2023-11-16 12:11:47,390 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 12:11:47,390 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:11:20 2023-11-16 12:11:47,391 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:16:25 2023-11-16 12:11:47,391 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.4 seconds late 2023-11-16 12:11:48,008 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:11:48,009 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:11:20 2023-11-16 12:11:48,009 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:16:25 2023-11-16 12:11:48,009 INFO 1947952:repeater task 3dayplots_os38nb starting 24.0 seconds late 2023-11-16 12:11:51,501 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:11:51,501 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:16:24 2023-11-16 12:12:26,583 INFO 1947952:repeater task lastensq_wh300 starting 3.6 seconds late 2023-11-16 12:12:29,333 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 12:12:29,333 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:12:18 2023-11-16 12:12:29,333 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:14:23 2023-11-16 12:12:29,334 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 12:12:35,565 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 12:12:35,566 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:12:18 2023-11-16 12:12:35,566 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:14:23 2023-11-16 12:12:35,566 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2023-11-16 12:12:36,202 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:12:36,203 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:12:18 2023-11-16 12:12:36,203 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:14:23 2023-11-16 12:14:26,499 INFO 1947952:repeater task lastensq_wh300 starting 3.5 seconds late 2023-11-16 12:14:29,338 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:14:29,339 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:14:18 2023-11-16 12:14:29,339 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:16:23 2023-11-16 12:14:29,339 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 12:14:35,934 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 12:14:35,934 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:14:18 2023-11-16 12:14:35,934 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:16:23 2023-11-16 12:14:35,935 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 12:14:36,574 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:14:36,575 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:14:18 2023-11-16 12:14:36,575 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:16:23 2023-11-16 12:16:26,859 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.8 seconds late 2023-11-16 12:16:29,801 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 12:16:29,801 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 12:26:24 2023-11-16 12:16:29,801 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.8 seconds late 2023-11-16 12:16:32,584 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 12:16:32,584 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 12:26:24 2023-11-16 12:16:32,585 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.5 seconds late 2023-11-16 12:16:35,625 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 12:16:35,625 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 12:26:24 2023-11-16 12:16:35,626 INFO 1947952:repeater task lastensq_wh300 starting 12.6 seconds late 2023-11-16 12:16:38,542 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:16:38,604 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:16:18 2023-11-16 12:16:38,604 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:18:23 2023-11-16 12:16:38,606 INFO 1947952:repeater task quick_wh300 starting 15.6 seconds late 2023-11-16 12:16:44,767 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 12:16:44,768 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:16:18 2023-11-16 12:16:44,768 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:18:23 2023-11-16 12:16:44,768 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.8 seconds late 2023-11-16 12:16:45,395 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:16:45,396 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:16:18 2023-11-16 12:16:45,396 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:18:23 2023-11-16 12:16:45,396 INFO 1947952:repeater task 3dayplots_wh300 starting 21.3 seconds late 2023-11-16 12:16:49,161 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:16:49,161 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:21:24 2023-11-16 12:16:49,162 INFO 1947952:repeater task lastensq_os150nb starting 26.2 seconds late 2023-11-16 12:16:51,297 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 12:16:51,298 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:16:18 2023-11-16 12:16:51,298 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:21:23 2023-11-16 12:16:51,298 INFO 1947952:repeater task quick_os150nb starting 28.3 seconds late 2023-11-16 12:16:54,677 INFO 1947952:repeater task quick_os150nb ended after 3.4 seconds 2023-11-16 12:16:54,677 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:16:18 2023-11-16 12:16:54,677 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:21:23 2023-11-16 12:16:54,678 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.7 seconds late 2023-11-16 12:16:55,289 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:16:55,290 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:16:18 2023-11-16 12:16:55,290 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:21:23 2023-11-16 12:16:55,290 INFO 1947952:repeater task 3dayplots_os150nb starting 31.2 seconds late 2023-11-16 12:16:58,687 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:16:58,687 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:21:24 2023-11-16 12:16:58,688 INFO 1947952:repeater task lastensq_os38nb starting 33.7 seconds late 2023-11-16 12:17:00,782 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 12:17:00,783 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:16:20 2023-11-16 12:17:00,783 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:21:25 2023-11-16 12:17:00,783 INFO 1947952:repeater task quick_os38nb starting 35.8 seconds late 2023-11-16 12:17:05,391 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 12:17:05,391 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:16:20 2023-11-16 12:17:05,391 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:21:25 2023-11-16 12:17:05,392 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.4 seconds late 2023-11-16 12:17:06,006 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:17:06,006 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:16:20 2023-11-16 12:17:06,006 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:21:25 2023-11-16 12:17:06,006 INFO 1947952:repeater task 3dayplots_os38nb starting 42.0 seconds late 2023-11-16 12:17:09,475 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:17:09,475 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:21:24 2023-11-16 12:17:09,476 INFO 1947952:repeater task daily_netcdf starting 45.4 seconds late 2023-11-16 12:17:10,343 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-16 12:17:10,343 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 12:26:24 2023-11-16 12:18:25,536 INFO 1947952:repeater task lastensq_wh300 starting 2.5 seconds late 2023-11-16 12:18:28,363 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:18:28,363 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:18:18 2023-11-16 12:18:28,363 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:20:23 2023-11-16 12:18:28,364 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-16 12:18:34,882 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 12:18:34,883 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:18:18 2023-11-16 12:18:34,883 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:20:23 2023-11-16 12:18:34,883 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-16 12:18:35,515 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:18:35,515 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:18:18 2023-11-16 12:18:35,515 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:20:23 2023-11-16 12:20:25,804 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-16 12:20:28,666 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:20:28,667 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:20:18 2023-11-16 12:20:28,667 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:22:23 2023-11-16 12:20:28,667 INFO 1947952:repeater task quick_wh300 starting 5.7 seconds late 2023-11-16 12:20:35,216 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 12:20:35,217 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:20:18 2023-11-16 12:20:35,217 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:22:23 2023-11-16 12:20:35,217 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2023-11-16 12:20:35,843 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:20:35,844 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:20:18 2023-11-16 12:20:35,844 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:22:23 2023-11-16 12:21:20,957 INFO 1947952:repeater task plot_tsstats_wh300 starting 1.9 seconds late 2023-11-16 12:21:22,122 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 12:21:22,123 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:22,123 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.1 seconds late 2023-11-16 12:21:22,290 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 12:21:22,290 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:22,290 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.2 seconds late 2023-11-16 12:21:23,549 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 12:21:23,550 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:23,550 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.5 seconds late 2023-11-16 12:21:24,920 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.4 seconds 2023-11-16 12:21:24,920 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:24,921 INFO 1947952:repeater task lastensq_os150nb starting 1.9 seconds late 2023-11-16 12:21:27,093 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 12:21:27,094 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:21:18 2023-11-16 12:21:27,094 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:26:23 2023-11-16 12:21:27,095 INFO 1947952:repeater task quick_os150nb starting 4.1 seconds late 2023-11-16 12:21:30,957 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 12:21:30,957 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:21:18 2023-11-16 12:21:30,958 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:26:23 2023-11-16 12:21:30,958 INFO 1947952:repeater task lastfew_vec_os150nb starting 8.0 seconds late 2023-11-16 12:21:31,573 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:21:31,573 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:21:18 2023-11-16 12:21:31,573 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:26:23 2023-11-16 12:21:31,574 INFO 1947952:repeater task 3dayplots_os150nb starting 7.5 seconds late 2023-11-16 12:21:34,950 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:21:34,950 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:26:24 2023-11-16 12:21:34,951 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.9 seconds late 2023-11-16 12:21:35,928 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 12:21:35,928 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:35,928 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.9 seconds late 2023-11-16 12:21:36,099 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 12:21:36,100 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:36,100 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.0 seconds late 2023-11-16 12:21:37,330 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 12:21:37,330 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:37,331 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.3 seconds late 2023-11-16 12:21:38,589 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 12:21:38,589 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:38,589 INFO 1947952:repeater task lastensq_os38nb starting 13.6 seconds late 2023-11-16 12:21:40,991 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-16 12:21:40,991 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:21:20 2023-11-16 12:21:40,991 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:26:25 2023-11-16 12:21:40,992 INFO 1947952:repeater task quick_os38nb starting 16.0 seconds late 2023-11-16 12:21:45,361 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 12:21:45,362 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:21:20 2023-11-16 12:21:45,362 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:26:25 2023-11-16 12:21:45,362 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.4 seconds late 2023-11-16 12:21:45,979 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:21:45,979 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:21:20 2023-11-16 12:21:45,979 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:26:25 2023-11-16 12:21:45,980 INFO 1947952:repeater task 3dayplots_os38nb starting 21.9 seconds late 2023-11-16 12:21:49,388 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 12:21:49,388 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:26:24 2023-11-16 12:21:49,389 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.3 seconds late 2023-11-16 12:21:50,387 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 12:21:50,388 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:50,388 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.3 seconds late 2023-11-16 12:21:50,557 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 12:21:50,557 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:50,557 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.5 seconds late 2023-11-16 12:21:51,829 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 12:21:51,829 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:51,830 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.8 seconds late 2023-11-16 12:21:53,060 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 12:21:53,060 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:53,061 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 33.8 seconds late 2023-11-16 12:21:54,344 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 12:21:54,344 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:54,345 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 35.1 seconds late 2023-11-16 12:21:55,677 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 12:21:55,677 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:55,677 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 36.4 seconds late 2023-11-16 12:21:56,930 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 12:21:56,930 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:21:56,931 INFO 1947952:repeater task plot_hbins starting 37.7 seconds late 2023-11-16 12:22:01,187 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 12:22:01,187 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 12:22:01,187 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 12:36:19 2023-11-16 12:22:06,195 INFO 1947952:repeater task 3dayplots_wh300 starting 42.1 seconds late 2023-11-16 12:22:09,990 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:22:09,990 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:26:24 2023-11-16 12:22:25,022 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-16 12:22:27,841 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:22:27,841 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:22:18 2023-11-16 12:22:27,842 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:24:23 2023-11-16 12:22:27,842 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 12:22:33,957 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 12:22:33,958 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:22:18 2023-11-16 12:22:33,958 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:24:23 2023-11-16 12:22:33,958 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 12:22:34,591 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:22:34,591 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:22:18 2023-11-16 12:22:34,591 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:24:23 2023-11-16 12:24:24,895 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 12:24:27,842 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:24:27,843 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:24:18 2023-11-16 12:24:27,843 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:26:23 2023-11-16 12:24:27,843 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 12:24:34,013 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 12:24:34,014 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:24:18 2023-11-16 12:24:34,014 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:26:23 2023-11-16 12:24:34,014 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 12:24:34,650 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:24:34,651 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:24:18 2023-11-16 12:24:34,651 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:26:23 2023-11-16 12:26:24,937 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.9 seconds late 2023-11-16 12:26:27,817 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 12:26:27,818 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 12:36:24 2023-11-16 12:26:27,818 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.8 seconds late 2023-11-16 12:26:30,559 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-16 12:26:30,559 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 12:36:24 2023-11-16 12:26:30,560 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.5 seconds late 2023-11-16 12:26:33,520 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 12:26:33,520 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 12:36:24 2023-11-16 12:26:33,521 INFO 1947952:repeater task lastensq_wh300 starting 10.5 seconds late 2023-11-16 12:26:36,402 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:26:36,402 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:26:18 2023-11-16 12:26:36,402 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:28:23 2023-11-16 12:26:36,403 INFO 1947952:repeater task quick_wh300 starting 13.4 seconds late 2023-11-16 12:26:42,871 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 12:26:42,872 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:26:18 2023-11-16 12:26:42,872 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:28:23 2023-11-16 12:26:42,872 INFO 1947952:repeater task lastfew_vec_wh300 starting 19.9 seconds late 2023-11-16 12:26:43,504 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:26:43,504 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:26:18 2023-11-16 12:26:43,504 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:28:23 2023-11-16 12:26:43,505 INFO 1947952:repeater task 3dayplots_wh300 starting 19.5 seconds late 2023-11-16 12:26:47,319 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:26:47,319 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:31:24 2023-11-16 12:26:47,321 INFO 1947952:repeater task lastensq_os150nb starting 24.3 seconds late 2023-11-16 12:26:49,829 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 12:26:49,829 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:26:18 2023-11-16 12:26:49,829 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:31:23 2023-11-16 12:26:49,830 INFO 1947952:repeater task quick_os150nb starting 26.8 seconds late 2023-11-16 12:26:53,292 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 12:26:53,292 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:26:18 2023-11-16 12:26:53,292 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:31:23 2023-11-16 12:26:53,293 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.3 seconds late 2023-11-16 12:26:53,905 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:26:53,905 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:26:18 2023-11-16 12:26:53,905 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:31:23 2023-11-16 12:26:53,906 INFO 1947952:repeater task 3dayplots_os150nb starting 29.9 seconds late 2023-11-16 12:26:57,356 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:26:57,356 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:31:24 2023-11-16 12:26:57,357 INFO 1947952:repeater task lastensq_os38nb starting 32.4 seconds late 2023-11-16 12:26:59,419 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 12:26:59,419 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:26:20 2023-11-16 12:26:59,420 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:31:25 2023-11-16 12:26:59,421 INFO 1947952:repeater task quick_os38nb starting 34.4 seconds late 2023-11-16 12:27:03,861 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 12:27:03,861 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:26:20 2023-11-16 12:27:03,861 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:31:25 2023-11-16 12:27:03,862 INFO 1947952:repeater task lastfew_vec_os38nb starting 38.9 seconds late 2023-11-16 12:27:04,524 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.7 seconds 2023-11-16 12:27:04,524 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:26:20 2023-11-16 12:27:04,524 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:31:25 2023-11-16 12:27:04,524 INFO 1947952:repeater task 3dayplots_os38nb starting 40.5 seconds late 2023-11-16 12:27:08,055 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:27:08,055 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:31:24 2023-11-16 12:27:08,057 INFO 1947952:repeater task daily_netcdf starting 44.0 seconds late 2023-11-16 12:27:08,856 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 12:27:08,856 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 12:36:24 2023-11-16 12:28:24,036 INFO 1947952:repeater task lastensq_wh300 starting 1.0 seconds late 2023-11-16 12:28:26,677 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 12:28:26,677 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:28:18 2023-11-16 12:28:26,677 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:30:23 2023-11-16 12:28:26,677 INFO 1947952:repeater task quick_wh300 starting 3.7 seconds late 2023-11-16 12:28:33,051 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:28:33,051 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:28:18 2023-11-16 12:28:33,051 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:30:23 2023-11-16 12:28:33,052 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2023-11-16 12:28:33,685 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:28:33,685 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:28:18 2023-11-16 12:28:33,685 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:30:23 2023-11-16 12:30:23,971 INFO 1947952:repeater task lastensq_wh300 starting 1.0 seconds late 2023-11-16 12:30:27,235 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-16 12:30:27,236 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:30:18 2023-11-16 12:30:27,236 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:32:23 2023-11-16 12:30:27,236 INFO 1947952:repeater task quick_wh300 starting 4.2 seconds late 2023-11-16 12:30:33,240 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 12:30:33,240 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:30:18 2023-11-16 12:30:33,240 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:32:23 2023-11-16 12:30:33,241 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2023-11-16 12:30:33,876 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:30:33,876 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:30:18 2023-11-16 12:30:33,876 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:32:23 2023-11-16 12:31:24,009 INFO 1947952:repeater task lastensq_os150nb starting 1.0 seconds late 2023-11-16 12:31:26,257 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 12:31:26,257 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:31:18 2023-11-16 12:31:26,257 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:36:23 2023-11-16 12:31:26,258 INFO 1947952:repeater task quick_os150nb starting 3.3 seconds late 2023-11-16 12:31:29,761 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 12:31:29,761 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:31:18 2023-11-16 12:31:29,762 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:36:23 2023-11-16 12:31:29,762 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.8 seconds late 2023-11-16 12:31:30,373 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:31:30,373 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:31:18 2023-11-16 12:31:30,373 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:36:23 2023-11-16 12:31:30,373 INFO 1947952:repeater task 3dayplots_os150nb starting 6.3 seconds late 2023-11-16 12:31:33,759 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:31:33,760 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:36:24 2023-11-16 12:31:33,761 INFO 1947952:repeater task lastensq_os38nb starting 8.8 seconds late 2023-11-16 12:31:35,802 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 12:31:35,803 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:31:20 2023-11-16 12:31:35,803 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:36:25 2023-11-16 12:31:35,803 INFO 1947952:repeater task quick_os38nb starting 10.8 seconds late 2023-11-16 12:31:39,918 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 12:31:39,918 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:31:20 2023-11-16 12:31:39,919 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:36:25 2023-11-16 12:31:39,919 INFO 1947952:repeater task lastfew_vec_os38nb starting 14.9 seconds late 2023-11-16 12:31:40,540 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:31:40,540 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:31:20 2023-11-16 12:31:40,540 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:36:25 2023-11-16 12:31:40,541 INFO 1947952:repeater task 3dayplots_os38nb starting 16.5 seconds late 2023-11-16 12:31:44,027 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:31:44,027 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:36:24 2023-11-16 12:31:49,035 INFO 1947952:repeater task 3dayplots_wh300 starting 25.0 seconds late 2023-11-16 12:31:52,890 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 12:31:52,890 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:36:24 2023-11-16 12:32:27,980 INFO 1947952:repeater task lastensq_wh300 starting 5.0 seconds late 2023-11-16 12:32:30,726 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 12:32:30,726 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:32:18 2023-11-16 12:32:30,726 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:34:23 2023-11-16 12:32:30,727 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-16 12:32:37,241 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 12:32:37,242 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:32:18 2023-11-16 12:32:37,242 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:34:23 2023-11-16 12:32:37,242 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.2 seconds late 2023-11-16 12:32:37,879 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:32:37,879 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:32:18 2023-11-16 12:32:37,880 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:34:23 2023-11-16 12:34:23,155 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-16 12:34:26,219 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 12:34:26,219 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:34:18 2023-11-16 12:34:26,220 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:36:23 2023-11-16 12:34:26,220 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-16 12:34:32,695 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 12:34:32,696 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:34:18 2023-11-16 12:34:32,696 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:36:23 2023-11-16 12:34:32,696 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.7 seconds late 2023-11-16 12:34:33,333 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:34:33,333 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:34:18 2023-11-16 12:34:33,333 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:36:23 2023-11-16 12:36:23,634 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-16 12:36:26,528 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:36:26,528 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:36:18 2023-11-16 12:36:26,528 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:38:23 2023-11-16 12:36:26,529 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 12:36:32,885 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:36:32,886 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:36:18 2023-11-16 12:36:32,886 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:38:23 2023-11-16 12:36:32,886 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 12:36:33,521 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:36:33,521 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:36:18 2023-11-16 12:36:33,521 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:38:23 2023-11-16 12:36:33,522 INFO 1947952:repeater task 3dayplots_wh300 starting 9.5 seconds late 2023-11-16 12:36:37,280 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:36:37,280 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:41:24 2023-11-16 12:36:37,280 INFO 1947952:repeater task plot_tsstats_wh300 starting 18.2 seconds late 2023-11-16 12:36:38,418 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.1 seconds 2023-11-16 12:36:38,418 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:38,419 INFO 1947952:repeater task grab_lastpings_wh300 starting 19.4 seconds late 2023-11-16 12:36:38,589 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 12:36:38,589 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:38,590 INFO 1947952:repeater task beamdiagnostics_wh300 starting 19.5 seconds late 2023-11-16 12:36:39,840 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 12:36:39,841 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:39,841 INFO 1947952:repeater task beamdiagnostics_wh300 starting 20.8 seconds late 2023-11-16 12:36:41,122 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 12:36:41,122 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:41,122 INFO 1947952:repeater task lastensq_os150nb starting 18.1 seconds late 2023-11-16 12:36:43,948 INFO 1947952:repeater task lastensq_os150nb ended after 2.8 seconds 2023-11-16 12:36:43,948 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:36:18 2023-11-16 12:36:43,948 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:41:23 2023-11-16 12:36:43,949 INFO 1947952:repeater task quick_os150nb starting 20.9 seconds late 2023-11-16 12:36:47,402 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 12:36:47,402 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:36:18 2023-11-16 12:36:47,402 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:41:23 2023-11-16 12:36:47,403 INFO 1947952:repeater task lastfew_vec_os150nb starting 24.4 seconds late 2023-11-16 12:36:48,011 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:36:48,011 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:36:18 2023-11-16 12:36:48,011 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:41:23 2023-11-16 12:36:48,011 INFO 1947952:repeater task 3dayplots_os150nb starting 24.0 seconds late 2023-11-16 12:36:51,425 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:36:51,426 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:41:24 2023-11-16 12:36:51,426 INFO 1947952:repeater task plot_tsstats_os150nb starting 32.4 seconds late 2023-11-16 12:36:52,401 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 12:36:52,401 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:52,402 INFO 1947952:repeater task grab_lastpings_os150nb starting 33.4 seconds late 2023-11-16 12:36:52,572 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 12:36:52,573 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:52,573 INFO 1947952:repeater task beamdiagnostics_os150nb starting 33.5 seconds late 2023-11-16 12:36:53,797 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 12:36:53,798 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:53,798 INFO 1947952:repeater task beamdiagnostics_os150nb starting 34.7 seconds late 2023-11-16 12:36:55,041 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 12:36:55,041 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:36:55,042 INFO 1947952:repeater task lastensq_os38nb starting 30.0 seconds late 2023-11-16 12:36:57,110 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 12:36:57,111 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:36:20 2023-11-16 12:36:57,111 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:41:25 2023-11-16 12:36:57,111 INFO 1947952:repeater task quick_os38nb starting 32.1 seconds late 2023-11-16 12:37:01,236 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 12:37:01,237 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:36:20 2023-11-16 12:37:01,237 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:41:25 2023-11-16 12:37:01,237 INFO 1947952:repeater task lastfew_vec_os38nb starting 36.2 seconds late 2023-11-16 12:37:01,879 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:37:01,880 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:36:20 2023-11-16 12:37:01,880 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:41:25 2023-11-16 12:37:01,880 INFO 1947952:repeater task 3dayplots_os38nb starting 37.8 seconds late 2023-11-16 12:37:05,536 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.7 seconds 2023-11-16 12:37:05,536 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:41:24 2023-11-16 12:37:05,536 INFO 1947952:repeater task plot_tsstats_os38nb starting 46.5 seconds late 2023-11-16 12:37:06,599 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.1 seconds 2023-11-16 12:37:06,600 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:37:06,600 INFO 1947952:repeater task grab_lastpings_os38nb starting 47.5 seconds late 2023-11-16 12:37:06,778 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 12:37:06,779 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:37:06,779 INFO 1947952:repeater task beamdiagnostics_os38nb starting 47.7 seconds late 2023-11-16 12:37:08,046 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 12:37:08,046 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:37:08,047 INFO 1947952:repeater task beamdiagnostics_os38nb starting 49.0 seconds late 2023-11-16 12:37:09,284 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 12:37:09,284 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:37:09,285 INFO 1947952:repeater task daily_netcdf starting 45.2 seconds late 2023-11-16 12:37:10,124 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 12:37:10,124 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 12:46:24 2023-11-16 12:37:10,124 INFO 1947952:repeater task ggatime_diagnostics starting 51.1 seconds late 2023-11-16 12:37:26,556 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-16 12:37:26,557 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:37:26,557 INFO 1947952:repeater task ggatime_diagnostics starting 67.5 seconds late 2023-11-16 12:37:42,619 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-16 12:37:42,619 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:37:42,620 INFO 1947952:repeater task ggatime_diagnostics starting 83.6 seconds late 2023-11-16 12:37:58,668 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 12:37:58,668 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:37:58,669 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 99.4 seconds late 2023-11-16 12:37:59,988 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 12:37:59,988 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:37:59,988 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 100.7 seconds late 2023-11-16 12:38:01,250 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 12:38:01,250 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:38:01,251 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 102.0 seconds late 2023-11-16 12:38:02,595 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 12:38:02,596 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:38:02,596 INFO 1947952:repeater task plot_hbins starting 103.3 seconds late 2023-11-16 12:38:06,878 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 12:38:06,878 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 12:38:06,878 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 12:51:19 2023-11-16 12:38:06,879 INFO 1947952:repeater task report_generator starting 107.6 seconds late 2023-11-16 12:38:12,764 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 12:38:12,765 INFO 1947952:repeater task report_generator ended after 5.9 seconds 2023-11-16 12:38:12,765 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 13:36:19 2023-11-16 12:38:17,769 INFO 1947952:repeater task noisy_rawtemps_os150 starting 113.7 seconds late 2023-11-16 12:38:20,653 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 12:38:20,653 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 12:46:24 2023-11-16 12:38:20,654 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 116.6 seconds late 2023-11-16 12:38:23,683 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 3.0 seconds 2023-11-16 12:38:23,683 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 12:46:24 2023-11-16 12:38:23,683 INFO 1947952:repeater task noisy_rawtemps_os38 starting 119.6 seconds late 2023-11-16 12:38:26,719 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 12:38:26,719 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 12:46:24 2023-11-16 12:38:26,720 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-16 12:38:29,489 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:38:29,490 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:38:18 2023-11-16 12:38:29,490 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:40:23 2023-11-16 12:38:29,490 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-16 12:38:35,911 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:38:35,912 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:38:18 2023-11-16 12:38:35,912 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:40:23 2023-11-16 12:38:35,912 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 12:38:36,554 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:38:36,554 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:38:18 2023-11-16 12:38:36,555 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:40:23 2023-11-16 12:40:26,851 INFO 1947952:repeater task lastensq_wh300 starting 3.9 seconds late 2023-11-16 12:40:29,769 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:40:29,770 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:40:18 2023-11-16 12:40:29,770 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:42:23 2023-11-16 12:40:29,770 INFO 1947952:repeater task quick_wh300 starting 6.8 seconds late 2023-11-16 12:40:36,046 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 12:40:36,046 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:40:18 2023-11-16 12:40:36,046 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:42:23 2023-11-16 12:40:36,047 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.0 seconds late 2023-11-16 12:40:36,680 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:40:36,681 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:40:18 2023-11-16 12:40:36,681 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:42:23 2023-11-16 12:41:26,801 INFO 1947952:repeater task 3dayplots_wh300 starting 2.8 seconds late 2023-11-16 12:41:30,584 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:41:30,584 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:46:24 2023-11-16 12:41:30,586 INFO 1947952:repeater task lastensq_os150nb starting 7.6 seconds late 2023-11-16 12:41:32,856 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 12:41:32,857 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:41:18 2023-11-16 12:41:32,857 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:46:23 2023-11-16 12:41:32,857 INFO 1947952:repeater task quick_os150nb starting 9.9 seconds late 2023-11-16 12:41:36,244 INFO 1947952:repeater task quick_os150nb ended after 3.4 seconds 2023-11-16 12:41:36,245 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:41:18 2023-11-16 12:41:36,245 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:46:23 2023-11-16 12:41:36,245 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.2 seconds late 2023-11-16 12:41:36,856 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:41:36,856 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:41:18 2023-11-16 12:41:36,857 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:46:23 2023-11-16 12:41:36,857 INFO 1947952:repeater task 3dayplots_os150nb starting 12.8 seconds late 2023-11-16 12:41:40,239 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:41:40,239 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:46:24 2023-11-16 12:41:40,240 INFO 1947952:repeater task lastensq_os38nb starting 15.2 seconds late 2023-11-16 12:41:42,308 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 12:41:42,308 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:41:20 2023-11-16 12:41:42,309 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:46:25 2023-11-16 12:41:42,309 INFO 1947952:repeater task quick_os38nb starting 17.3 seconds late 2023-11-16 12:41:46,726 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 12:41:46,726 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:41:20 2023-11-16 12:41:46,727 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:46:25 2023-11-16 12:41:46,727 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.7 seconds late 2023-11-16 12:41:47,337 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:41:47,338 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:41:20 2023-11-16 12:41:47,338 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:46:25 2023-11-16 12:41:47,338 INFO 1947952:repeater task 3dayplots_os38nb starting 23.3 seconds late 2023-11-16 12:41:50,850 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:41:50,850 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:46:24 2023-11-16 12:42:25,933 INFO 1947952:repeater task lastensq_wh300 starting 2.9 seconds late 2023-11-16 12:42:29,273 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-16 12:42:29,273 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:42:18 2023-11-16 12:42:29,273 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:44:23 2023-11-16 12:42:29,274 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 12:42:35,719 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:42:35,720 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:42:18 2023-11-16 12:42:35,720 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:44:23 2023-11-16 12:42:35,720 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2023-11-16 12:42:36,360 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:42:36,361 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:42:18 2023-11-16 12:42:36,361 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:44:23 2023-11-16 12:44:26,655 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-16 12:44:29,406 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:44:29,406 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:44:18 2023-11-16 12:44:29,406 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:46:23 2023-11-16 12:44:29,407 INFO 1947952:repeater task quick_wh300 starting 6.4 seconds late 2023-11-16 12:44:35,890 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 12:44:35,890 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:44:18 2023-11-16 12:44:35,890 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:46:23 2023-11-16 12:44:35,890 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 12:44:36,526 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:44:36,526 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:44:18 2023-11-16 12:44:36,526 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:46:23 2023-11-16 12:46:26,815 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.8 seconds late 2023-11-16 12:46:29,743 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 12:46:29,743 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 12:56:24 2023-11-16 12:46:29,744 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.7 seconds late 2023-11-16 12:46:32,509 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 12:46:32,509 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 12:56:24 2023-11-16 12:46:32,510 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.5 seconds late 2023-11-16 12:46:35,516 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 12:46:35,517 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 12:56:24 2023-11-16 12:46:35,517 INFO 1947952:repeater task lastensq_wh300 starting 12.5 seconds late 2023-11-16 12:46:38,141 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 12:46:38,141 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:46:18 2023-11-16 12:46:38,141 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:48:23 2023-11-16 12:46:38,141 INFO 1947952:repeater task quick_wh300 starting 15.1 seconds late 2023-11-16 12:46:44,506 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:46:44,507 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:46:18 2023-11-16 12:46:44,507 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:48:23 2023-11-16 12:46:44,507 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.5 seconds late 2023-11-16 12:46:45,147 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:46:45,148 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:46:18 2023-11-16 12:46:45,148 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:48:23 2023-11-16 12:46:45,148 INFO 1947952:repeater task 3dayplots_wh300 starting 21.1 seconds late 2023-11-16 12:46:48,946 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:46:48,947 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:51:24 2023-11-16 12:46:48,948 INFO 1947952:repeater task lastensq_os150nb starting 25.9 seconds late 2023-11-16 12:46:50,968 INFO 1947952:repeater task lastensq_os150nb ended after 2.0 seconds 2023-11-16 12:46:50,969 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:46:18 2023-11-16 12:46:50,969 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:51:23 2023-11-16 12:46:50,969 INFO 1947952:repeater task quick_os150nb starting 28.0 seconds late 2023-11-16 12:46:54,908 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 12:46:54,909 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:46:18 2023-11-16 12:46:54,909 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:51:23 2023-11-16 12:46:54,909 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.9 seconds late 2023-11-16 12:46:55,521 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:46:55,521 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:46:18 2023-11-16 12:46:55,521 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:51:23 2023-11-16 12:46:55,522 INFO 1947952:repeater task 3dayplots_os150nb starting 31.5 seconds late 2023-11-16 12:46:58,888 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:46:58,888 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:51:24 2023-11-16 12:46:58,889 INFO 1947952:repeater task lastensq_os38nb starting 33.9 seconds late 2023-11-16 12:47:00,902 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 12:47:00,902 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:46:20 2023-11-16 12:47:00,902 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:51:25 2023-11-16 12:47:00,902 INFO 1947952:repeater task quick_os38nb starting 35.9 seconds late 2023-11-16 12:47:05,711 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-16 12:47:05,711 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:46:20 2023-11-16 12:47:05,712 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:51:25 2023-11-16 12:47:05,712 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.7 seconds late 2023-11-16 12:47:06,352 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:47:06,352 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:46:20 2023-11-16 12:47:06,352 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:51:25 2023-11-16 12:47:06,353 INFO 1947952:repeater task 3dayplots_os38nb starting 42.3 seconds late 2023-11-16 12:47:09,832 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:47:09,833 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:51:24 2023-11-16 12:47:09,834 INFO 1947952:repeater task daily_netcdf starting 45.8 seconds late 2023-11-16 12:47:10,643 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 12:47:10,643 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 12:56:24 2023-11-16 12:48:25,850 INFO 1947952:repeater task lastensq_wh300 starting 2.9 seconds late 2023-11-16 12:48:28,761 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 12:48:28,762 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:48:18 2023-11-16 12:48:28,762 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:50:23 2023-11-16 12:48:28,762 INFO 1947952:repeater task quick_wh300 starting 5.8 seconds late 2023-11-16 12:48:35,116 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:48:35,116 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:48:18 2023-11-16 12:48:35,116 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:50:23 2023-11-16 12:48:35,117 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2023-11-16 12:48:35,751 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:48:35,751 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:48:18 2023-11-16 12:48:35,752 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:50:23 2023-11-16 12:50:26,036 INFO 1947952:repeater task lastensq_wh300 starting 3.0 seconds late 2023-11-16 12:50:29,284 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-16 12:50:29,284 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:50:18 2023-11-16 12:50:29,285 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:52:23 2023-11-16 12:50:29,285 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 12:50:35,765 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 12:50:35,766 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:50:18 2023-11-16 12:50:35,766 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:52:23 2023-11-16 12:50:35,766 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-16 12:50:36,404 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:50:36,404 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:50:18 2023-11-16 12:50:36,404 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:52:23 2023-11-16 12:51:21,529 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.5 seconds late 2023-11-16 12:51:22,694 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 12:51:22,694 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:22,694 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.6 seconds late 2023-11-16 12:51:22,860 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 12:51:22,860 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:22,861 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.8 seconds late 2023-11-16 12:51:24,147 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 12:51:24,147 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:24,148 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.1 seconds late 2023-11-16 12:51:25,360 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 12:51:25,360 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:25,361 INFO 1947952:repeater task lastensq_os150nb starting 2.4 seconds late 2023-11-16 12:51:27,475 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 12:51:27,476 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:51:18 2023-11-16 12:51:27,476 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 12:56:23 2023-11-16 12:51:27,476 INFO 1947952:repeater task quick_os150nb starting 4.5 seconds late 2023-11-16 12:51:31,078 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 12:51:31,078 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:51:18 2023-11-16 12:51:31,078 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 12:56:23 2023-11-16 12:51:31,079 INFO 1947952:repeater task lastfew_vec_os150nb starting 8.1 seconds late 2023-11-16 12:51:31,691 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:51:31,691 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:51:18 2023-11-16 12:51:31,691 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 12:56:23 2023-11-16 12:51:31,692 INFO 1947952:repeater task 3dayplots_os150nb starting 7.6 seconds late 2023-11-16 12:51:35,046 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 12:51:35,046 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 12:56:24 2023-11-16 12:51:35,046 INFO 1947952:repeater task plot_tsstats_os150nb starting 16.0 seconds late 2023-11-16 12:51:36,015 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 12:51:36,015 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:36,015 INFO 1947952:repeater task grab_lastpings_os150nb starting 17.0 seconds late 2023-11-16 12:51:36,184 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 12:51:36,184 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:36,185 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.1 seconds late 2023-11-16 12:51:37,486 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 12:51:37,486 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:37,486 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.4 seconds late 2023-11-16 12:51:38,738 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 12:51:38,738 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:38,738 INFO 1947952:repeater task lastensq_os38nb starting 13.7 seconds late 2023-11-16 12:51:40,842 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 12:51:40,842 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:51:20 2023-11-16 12:51:40,842 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 12:56:25 2023-11-16 12:51:40,843 INFO 1947952:repeater task quick_os38nb starting 15.8 seconds late 2023-11-16 12:51:44,513 INFO 1947952:repeater task quick_os38nb ended after 3.7 seconds 2023-11-16 12:51:44,513 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:51:20 2023-11-16 12:51:44,513 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 12:56:25 2023-11-16 12:51:44,514 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.5 seconds late 2023-11-16 12:51:45,128 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:51:45,128 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:51:20 2023-11-16 12:51:45,128 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 12:56:25 2023-11-16 12:51:45,129 INFO 1947952:repeater task 3dayplots_os38nb starting 21.1 seconds late 2023-11-16 12:51:48,566 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 12:51:48,566 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 12:56:24 2023-11-16 12:51:48,566 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.5 seconds late 2023-11-16 12:51:49,568 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 12:51:49,568 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:49,569 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.5 seconds late 2023-11-16 12:51:49,736 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 12:51:49,736 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:49,736 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.7 seconds late 2023-11-16 12:51:50,981 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 12:51:50,982 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:50,982 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.9 seconds late 2023-11-16 12:51:52,223 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 12:51:52,223 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:52,224 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 33.0 seconds late 2023-11-16 12:51:53,539 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 12:51:53,540 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:53,541 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 34.3 seconds late 2023-11-16 12:51:54,835 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 12:51:54,835 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:54,836 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 35.6 seconds late 2023-11-16 12:51:56,067 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 12:51:56,067 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:51:56,068 INFO 1947952:repeater task plot_hbins starting 36.8 seconds late 2023-11-16 12:52:00,384 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 12:52:00,384 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 12:52:00,384 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 13:06:19 2023-11-16 12:52:05,390 INFO 1947952:repeater task 3dayplots_wh300 starting 41.3 seconds late 2023-11-16 12:52:09,239 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:52:09,239 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 12:56:24 2023-11-16 12:52:24,276 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 12:52:27,121 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:52:27,121 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:52:18 2023-11-16 12:52:27,122 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:54:23 2023-11-16 12:52:27,122 INFO 1947952:repeater task quick_wh300 starting 4.1 seconds late 2023-11-16 12:52:33,020 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 12:52:33,020 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:52:18 2023-11-16 12:52:33,020 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:54:23 2023-11-16 12:52:33,021 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2023-11-16 12:52:33,657 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:52:33,657 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:52:18 2023-11-16 12:52:33,657 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:54:23 2023-11-16 12:54:23,943 INFO 1947952:repeater task lastensq_wh300 starting 0.9 seconds late 2023-11-16 12:54:26,733 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 12:54:26,734 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:54:18 2023-11-16 12:54:26,734 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:56:23 2023-11-16 12:54:26,734 INFO 1947952:repeater task quick_wh300 starting 3.7 seconds late 2023-11-16 12:54:32,747 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 12:54:32,747 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:54:18 2023-11-16 12:54:32,747 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:56:23 2023-11-16 12:54:32,748 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.7 seconds late 2023-11-16 12:54:33,381 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:54:33,381 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:54:18 2023-11-16 12:54:33,381 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:56:23 2023-11-16 12:56:23,666 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-16 12:56:26,323 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 12:56:26,323 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:56:18 2023-11-16 12:56:26,323 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 12:58:23 2023-11-16 12:56:26,324 INFO 1947952:repeater task quick_wh300 starting 3.3 seconds late 2023-11-16 12:56:32,943 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 12:56:32,944 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:56:18 2023-11-16 12:56:32,944 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 12:58:23 2023-11-16 12:56:32,944 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 12:56:33,575 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:56:33,576 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:56:18 2023-11-16 12:56:33,576 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 12:58:23 2023-11-16 12:56:33,576 INFO 1947952:repeater task 3dayplots_wh300 starting 9.5 seconds late 2023-11-16 12:56:37,413 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 12:56:37,414 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:01:24 2023-11-16 12:56:37,415 INFO 1947952:repeater task lastensq_os150nb starting 14.4 seconds late 2023-11-16 12:56:39,521 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 12:56:39,522 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:56:18 2023-11-16 12:56:39,522 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:01:23 2023-11-16 12:56:39,522 INFO 1947952:repeater task quick_os150nb starting 16.5 seconds late 2023-11-16 12:56:43,382 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 12:56:43,382 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:56:18 2023-11-16 12:56:43,382 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:01:23 2023-11-16 12:56:43,383 INFO 1947952:repeater task lastfew_vec_os150nb starting 20.4 seconds late 2023-11-16 12:56:43,988 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 12:56:43,988 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 12:56:18 2023-11-16 12:56:43,989 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:01:23 2023-11-16 12:56:43,989 INFO 1947952:repeater task 3dayplots_os150nb starting 19.9 seconds late 2023-11-16 12:56:47,335 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-16 12:56:47,335 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:01:24 2023-11-16 12:56:47,336 INFO 1947952:repeater task lastensq_os38nb starting 22.3 seconds late 2023-11-16 12:56:49,517 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 12:56:49,518 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:56:20 2023-11-16 12:56:49,518 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:01:25 2023-11-16 12:56:49,519 INFO 1947952:repeater task quick_os38nb starting 24.5 seconds late 2023-11-16 12:56:53,825 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 12:56:53,825 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:56:20 2023-11-16 12:56:53,825 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:01:25 2023-11-16 12:56:53,826 INFO 1947952:repeater task lastfew_vec_os38nb starting 28.8 seconds late 2023-11-16 12:56:54,438 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 12:56:54,439 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 12:56:20 2023-11-16 12:56:54,439 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:01:25 2023-11-16 12:56:54,439 INFO 1947952:repeater task 3dayplots_os38nb starting 30.4 seconds late 2023-11-16 12:56:57,945 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 12:56:57,945 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:01:24 2023-11-16 12:56:57,946 INFO 1947952:repeater task daily_netcdf starting 33.9 seconds late 2023-11-16 12:56:58,759 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 12:56:58,759 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 13:06:24 2023-11-16 12:57:03,766 INFO 1947952:repeater task noisy_rawtemps_os150 starting 39.7 seconds late 2023-11-16 12:57:06,736 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 12:57:06,736 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 13:06:24 2023-11-16 12:57:06,736 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 42.7 seconds late 2023-11-16 12:57:09,556 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 12:57:09,557 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 13:06:24 2023-11-16 12:57:09,558 INFO 1947952:repeater task noisy_rawtemps_os38 starting 45.5 seconds late 2023-11-16 12:57:12,608 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 12:57:12,608 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 13:06:24 2023-11-16 12:58:27,804 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-16 12:58:31,064 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-16 12:58:31,064 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:58:18 2023-11-16 12:58:31,065 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:00:23 2023-11-16 12:58:31,065 INFO 1947952:repeater task quick_wh300 starting 8.1 seconds late 2023-11-16 12:58:37,439 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 12:58:37,440 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:58:18 2023-11-16 12:58:37,440 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:00:23 2023-11-16 12:58:37,440 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.4 seconds late 2023-11-16 12:58:38,082 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 12:58:38,082 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 12:58:18 2023-11-16 12:58:38,082 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:00:23 2023-11-16 13:00:23,372 INFO 1947952:repeater task lastensq_wh300 starting 0.4 seconds late 2023-11-16 13:00:26,221 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:00:26,222 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:00:18 2023-11-16 13:00:26,222 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:02:23 2023-11-16 13:00:26,222 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-16 13:00:32,843 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 13:00:32,843 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:00:18 2023-11-16 13:00:32,843 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:02:23 2023-11-16 13:00:32,844 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.8 seconds late 2023-11-16 13:00:33,481 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:00:33,482 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:00:18 2023-11-16 13:00:33,482 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:02:23 2023-11-16 13:01:23,604 INFO 1947952:repeater task lastensq_os150nb starting 0.6 seconds late 2023-11-16 13:01:26,293 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-16 13:01:26,293 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:01:18 2023-11-16 13:01:26,293 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:06:23 2023-11-16 13:01:26,293 INFO 1947952:repeater task quick_os150nb starting 3.3 seconds late 2023-11-16 13:01:29,984 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 13:01:29,984 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:01:18 2023-11-16 13:01:29,984 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:06:23 2023-11-16 13:01:29,985 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.0 seconds late 2023-11-16 13:01:30,594 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:01:30,595 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:01:18 2023-11-16 13:01:30,595 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:06:23 2023-11-16 13:01:30,595 INFO 1947952:repeater task 3dayplots_os150nb starting 6.5 seconds late 2023-11-16 13:01:34,011 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:01:34,012 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:06:24 2023-11-16 13:01:34,013 INFO 1947952:repeater task lastensq_os38nb starting 9.0 seconds late 2023-11-16 13:01:36,054 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 13:01:36,054 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:01:20 2023-11-16 13:01:36,054 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:06:25 2023-11-16 13:01:36,055 INFO 1947952:repeater task quick_os38nb starting 11.1 seconds late 2023-11-16 13:01:40,092 INFO 1947952:repeater task quick_os38nb ended after 4.0 seconds 2023-11-16 13:01:40,093 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:01:20 2023-11-16 13:01:40,093 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:06:25 2023-11-16 13:01:40,094 INFO 1947952:repeater task lastfew_vec_os38nb starting 15.1 seconds late 2023-11-16 13:01:40,708 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:01:40,709 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:01:20 2023-11-16 13:01:40,709 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:06:25 2023-11-16 13:01:40,709 INFO 1947952:repeater task 3dayplots_os38nb starting 16.7 seconds late 2023-11-16 13:01:44,161 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 13:01:44,161 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:06:24 2023-11-16 13:01:49,172 INFO 1947952:repeater task 3dayplots_wh300 starting 25.1 seconds late 2023-11-16 13:01:52,983 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:01:52,984 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:06:24 2023-11-16 13:02:23,058 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 13:02:25,904 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:02:25,904 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:02:18 2023-11-16 13:02:25,904 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:04:23 2023-11-16 13:02:25,905 INFO 1947952:repeater task quick_wh300 starting 2.9 seconds late 2023-11-16 13:02:31,786 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 13:02:31,787 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:02:18 2023-11-16 13:02:31,787 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:04:23 2023-11-16 13:02:31,787 INFO 1947952:repeater task lastfew_vec_wh300 starting 8.8 seconds late 2023-11-16 13:02:32,417 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:02:32,417 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:02:18 2023-11-16 13:02:32,417 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:04:23 2023-11-16 13:04:27,719 INFO 1947952:repeater task lastensq_wh300 starting 4.7 seconds late 2023-11-16 13:04:30,878 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-16 13:04:30,879 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:04:18 2023-11-16 13:04:30,879 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:06:23 2023-11-16 13:04:30,879 INFO 1947952:repeater task quick_wh300 starting 7.9 seconds late 2023-11-16 13:04:37,613 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 13:04:37,614 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:04:18 2023-11-16 13:04:37,614 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:06:23 2023-11-16 13:04:37,614 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.6 seconds late 2023-11-16 13:04:38,251 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:04:38,251 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:04:18 2023-11-16 13:04:38,251 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:06:23 2023-11-16 13:06:23,543 INFO 1947952:repeater task lastensq_wh300 starting 0.5 seconds late 2023-11-16 13:06:26,802 INFO 1947952:repeater task lastensq_wh300 ended after 3.3 seconds 2023-11-16 13:06:26,802 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:06:18 2023-11-16 13:06:26,802 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:08:23 2023-11-16 13:06:26,803 INFO 1947952:repeater task quick_wh300 starting 3.8 seconds late 2023-11-16 13:06:32,935 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 13:06:32,935 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:06:18 2023-11-16 13:06:32,935 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:08:23 2023-11-16 13:06:32,936 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 13:06:33,564 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:06:33,565 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:06:18 2023-11-16 13:06:33,565 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:08:23 2023-11-16 13:06:33,565 INFO 1947952:repeater task 3dayplots_wh300 starting 9.5 seconds late 2023-11-16 13:06:37,396 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:06:37,396 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:11:24 2023-11-16 13:06:37,397 INFO 1947952:repeater task plot_tsstats_wh300 starting 18.3 seconds late 2023-11-16 13:06:38,551 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 13:06:38,552 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:38,552 INFO 1947952:repeater task grab_lastpings_wh300 starting 19.5 seconds late 2023-11-16 13:06:38,719 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 13:06:38,719 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:38,720 INFO 1947952:repeater task beamdiagnostics_wh300 starting 19.7 seconds late 2023-11-16 13:06:39,967 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 13:06:39,968 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:39,968 INFO 1947952:repeater task beamdiagnostics_wh300 starting 20.9 seconds late 2023-11-16 13:06:41,210 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 13:06:41,210 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:41,211 INFO 1947952:repeater task lastensq_os150nb starting 18.2 seconds late 2023-11-16 13:06:43,313 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 13:06:43,314 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:06:18 2023-11-16 13:06:43,314 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:11:23 2023-11-16 13:06:43,314 INFO 1947952:repeater task quick_os150nb starting 20.3 seconds late 2023-11-16 13:06:47,060 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 13:06:47,060 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:06:18 2023-11-16 13:06:47,060 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:11:23 2023-11-16 13:06:47,061 INFO 1947952:repeater task lastfew_vec_os150nb starting 24.1 seconds late 2023-11-16 13:06:47,672 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:06:47,672 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:06:18 2023-11-16 13:06:47,673 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:11:23 2023-11-16 13:06:47,673 INFO 1947952:repeater task 3dayplots_os150nb starting 23.6 seconds late 2023-11-16 13:06:51,077 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:06:51,077 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:11:24 2023-11-16 13:06:51,078 INFO 1947952:repeater task plot_tsstats_os150nb starting 32.0 seconds late 2023-11-16 13:06:52,044 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 13:06:52,044 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:52,045 INFO 1947952:repeater task grab_lastpings_os150nb starting 33.0 seconds late 2023-11-16 13:06:52,214 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 13:06:52,214 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:52,214 INFO 1947952:repeater task beamdiagnostics_os150nb starting 33.2 seconds late 2023-11-16 13:06:53,421 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 13:06:53,421 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:53,422 INFO 1947952:repeater task beamdiagnostics_os150nb starting 34.4 seconds late 2023-11-16 13:06:54,673 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 13:06:54,673 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:06:54,674 INFO 1947952:repeater task lastensq_os38nb starting 29.7 seconds late 2023-11-16 13:06:56,843 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 13:06:56,843 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:06:20 2023-11-16 13:06:56,843 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:11:25 2023-11-16 13:06:56,844 INFO 1947952:repeater task quick_os38nb starting 31.8 seconds late 2023-11-16 13:07:02,195 INFO 1947952:repeater task quick_os38nb ended after 5.4 seconds 2023-11-16 13:07:02,196 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:06:20 2023-11-16 13:07:02,196 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:11:25 2023-11-16 13:07:02,196 INFO 1947952:repeater task lastfew_vec_os38nb starting 37.2 seconds late 2023-11-16 13:07:02,809 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:07:02,810 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:06:20 2023-11-16 13:07:02,810 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:11:25 2023-11-16 13:07:02,810 INFO 1947952:repeater task 3dayplots_os38nb starting 38.8 seconds late 2023-11-16 13:07:06,155 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.3 seconds 2023-11-16 13:07:06,155 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:11:24 2023-11-16 13:07:06,156 INFO 1947952:repeater task plot_tsstats_os38nb starting 47.1 seconds late 2023-11-16 13:07:07,151 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 13:07:07,151 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:07:07,152 INFO 1947952:repeater task grab_lastpings_os38nb starting 48.1 seconds late 2023-11-16 13:07:07,319 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 13:07:07,319 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:07:07,320 INFO 1947952:repeater task beamdiagnostics_os38nb starting 48.3 seconds late 2023-11-16 13:07:08,613 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 13:07:08,614 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:07:08,614 INFO 1947952:repeater task beamdiagnostics_os38nb starting 49.6 seconds late 2023-11-16 13:07:09,846 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 13:07:09,846 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:07:09,846 INFO 1947952:repeater task daily_netcdf starting 45.8 seconds late 2023-11-16 13:07:10,716 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-16 13:07:10,717 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 13:16:24 2023-11-16 13:07:10,717 INFO 1947952:repeater task ggatime_diagnostics starting 51.7 seconds late 2023-11-16 13:07:26,766 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 13:07:26,767 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:07:26,767 INFO 1947952:repeater task ggatime_diagnostics starting 67.7 seconds late 2023-11-16 13:07:42,569 INFO 1947952:repeater task ggatime_diagnostics ended after 15.8 seconds 2023-11-16 13:07:42,570 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:07:42,570 INFO 1947952:repeater task ggatime_diagnostics starting 83.5 seconds late 2023-11-16 13:07:58,205 INFO 1947952:repeater task ggatime_diagnostics ended after 15.6 seconds 2023-11-16 13:07:58,206 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:07:58,206 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 98.9 seconds late 2023-11-16 13:07:59,575 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-16 13:07:59,575 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:07:59,575 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 100.3 seconds late 2023-11-16 13:08:00,822 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.2 seconds 2023-11-16 13:08:00,822 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:08:00,823 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 101.5 seconds late 2023-11-16 13:08:02,098 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 13:08:02,099 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:08:02,099 INFO 1947952:repeater task plot_hbins starting 102.8 seconds late 2023-11-16 13:08:06,595 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 13:08:06,595 INFO 1947952:repeater task plot_hbins ended after 4.5 seconds 2023-11-16 13:08:06,595 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 13:21:19 2023-11-16 13:08:11,597 INFO 1947952:repeater task noisy_rawtemps_os150 starting 107.5 seconds late 2023-11-16 13:08:14,521 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 13:08:14,521 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 13:16:24 2023-11-16 13:08:14,521 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 110.5 seconds late 2023-11-16 13:08:17,326 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 13:08:17,326 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 13:16:24 2023-11-16 13:08:17,327 INFO 1947952:repeater task noisy_rawtemps_os38 starting 113.3 seconds late 2023-11-16 13:08:20,381 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 13:08:20,381 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 13:16:24 2023-11-16 13:08:25,390 INFO 1947952:repeater task lastensq_wh300 starting 2.4 seconds late 2023-11-16 13:08:28,339 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 13:08:28,340 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:08:18 2023-11-16 13:08:28,340 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:10:23 2023-11-16 13:08:28,340 INFO 1947952:repeater task quick_wh300 starting 5.3 seconds late 2023-11-16 13:08:34,580 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 13:08:34,580 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:08:18 2023-11-16 13:08:34,581 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:10:23 2023-11-16 13:08:34,581 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2023-11-16 13:08:35,218 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:08:35,218 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:08:18 2023-11-16 13:08:35,218 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:10:23 2023-11-16 13:10:25,509 INFO 1947952:repeater task lastensq_wh300 starting 2.5 seconds late 2023-11-16 13:10:28,505 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 13:10:28,506 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:10:18 2023-11-16 13:10:28,506 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:12:23 2023-11-16 13:10:28,506 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-16 13:10:34,669 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 13:10:34,670 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:10:18 2023-11-16 13:10:34,670 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:12:23 2023-11-16 13:10:34,670 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-16 13:10:35,300 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:10:35,300 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:10:18 2023-11-16 13:10:35,301 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:12:23 2023-11-16 13:11:25,433 INFO 1947952:repeater task 3dayplots_wh300 starting 1.4 seconds late 2023-11-16 13:11:29,254 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:11:29,254 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:16:24 2023-11-16 13:11:29,255 INFO 1947952:repeater task lastensq_os150nb starting 6.3 seconds late 2023-11-16 13:11:31,825 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 13:11:31,826 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:11:18 2023-11-16 13:11:31,826 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:16:23 2023-11-16 13:11:31,826 INFO 1947952:repeater task quick_os150nb starting 8.8 seconds late 2023-11-16 13:11:35,361 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 13:11:35,362 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:11:18 2023-11-16 13:11:35,362 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:16:23 2023-11-16 13:11:35,362 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.4 seconds late 2023-11-16 13:11:35,969 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:11:35,970 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:11:18 2023-11-16 13:11:35,970 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:16:23 2023-11-16 13:11:35,970 INFO 1947952:repeater task 3dayplots_os150nb starting 11.9 seconds late 2023-11-16 13:11:39,362 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:11:39,363 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:16:24 2023-11-16 13:11:39,364 INFO 1947952:repeater task lastensq_os38nb starting 14.4 seconds late 2023-11-16 13:11:41,431 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 13:11:41,432 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:11:20 2023-11-16 13:11:41,432 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:16:25 2023-11-16 13:11:41,432 INFO 1947952:repeater task quick_os38nb starting 16.4 seconds late 2023-11-16 13:11:45,545 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 13:11:45,546 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:11:20 2023-11-16 13:11:45,546 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:16:25 2023-11-16 13:11:45,546 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.5 seconds late 2023-11-16 13:11:46,162 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:11:46,163 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:11:20 2023-11-16 13:11:46,163 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:16:25 2023-11-16 13:11:46,164 INFO 1947952:repeater task 3dayplots_os38nb starting 22.1 seconds late 2023-11-16 13:11:49,633 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 13:11:49,633 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:16:24 2023-11-16 13:12:24,710 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-16 13:12:27,389 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 13:12:27,390 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:12:18 2023-11-16 13:12:27,390 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:14:23 2023-11-16 13:12:27,390 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-16 13:12:33,479 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 13:12:33,480 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:12:18 2023-11-16 13:12:33,480 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:14:23 2023-11-16 13:12:33,480 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-16 13:12:34,115 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:12:34,115 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:12:18 2023-11-16 13:12:34,115 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:14:23 2023-11-16 13:14:24,406 INFO 1947952:repeater task lastensq_wh300 starting 1.4 seconds late 2023-11-16 13:14:27,197 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:14:27,198 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:14:18 2023-11-16 13:14:27,198 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:16:23 2023-11-16 13:14:27,198 INFO 1947952:repeater task quick_wh300 starting 4.2 seconds late 2023-11-16 13:14:33,070 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 13:14:33,070 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:14:18 2023-11-16 13:14:33,071 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:16:23 2023-11-16 13:14:33,071 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2023-11-16 13:14:33,706 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:14:33,707 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:14:18 2023-11-16 13:14:33,707 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:16:23 2023-11-16 13:16:23,996 INFO 1947952:repeater task lastensq_wh300 starting 1.0 seconds late 2023-11-16 13:16:26,887 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 13:16:26,888 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:16:18 2023-11-16 13:16:26,888 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:18:23 2023-11-16 13:16:26,888 INFO 1947952:repeater task quick_wh300 starting 3.9 seconds late 2023-11-16 13:16:32,866 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 13:16:32,866 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:16:18 2023-11-16 13:16:32,866 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:18:23 2023-11-16 13:16:32,867 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 13:16:33,500 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:16:33,500 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:16:18 2023-11-16 13:16:33,500 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:18:23 2023-11-16 13:16:33,500 INFO 1947952:repeater task 3dayplots_wh300 starting 9.5 seconds late 2023-11-16 13:16:37,325 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:16:37,326 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:21:24 2023-11-16 13:16:37,327 INFO 1947952:repeater task lastensq_os150nb starting 14.3 seconds late 2023-11-16 13:16:39,457 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 13:16:39,457 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:16:18 2023-11-16 13:16:39,457 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:21:23 2023-11-16 13:16:39,457 INFO 1947952:repeater task quick_os150nb starting 16.5 seconds late 2023-11-16 13:16:43,267 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 13:16:43,268 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:16:18 2023-11-16 13:16:43,268 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:21:23 2023-11-16 13:16:43,269 INFO 1947952:repeater task lastfew_vec_os150nb starting 20.3 seconds late 2023-11-16 13:16:43,867 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:16:43,867 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:16:18 2023-11-16 13:16:43,867 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:21:23 2023-11-16 13:16:43,867 INFO 1947952:repeater task 3dayplots_os150nb starting 19.8 seconds late 2023-11-16 13:16:47,210 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-16 13:16:47,210 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:21:24 2023-11-16 13:16:47,212 INFO 1947952:repeater task lastensq_os38nb starting 22.2 seconds late 2023-11-16 13:16:49,540 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-16 13:16:49,540 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:16:20 2023-11-16 13:16:49,540 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:21:25 2023-11-16 13:16:49,541 INFO 1947952:repeater task quick_os38nb starting 24.5 seconds late 2023-11-16 13:16:54,083 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 13:16:54,083 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:16:20 2023-11-16 13:16:54,083 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:21:25 2023-11-16 13:16:54,084 INFO 1947952:repeater task lastfew_vec_os38nb starting 29.1 seconds late 2023-11-16 13:16:54,703 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:16:54,703 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:16:20 2023-11-16 13:16:54,703 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:21:25 2023-11-16 13:16:54,704 INFO 1947952:repeater task 3dayplots_os38nb starting 30.7 seconds late 2023-11-16 13:16:58,208 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 13:16:58,209 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:21:24 2023-11-16 13:16:58,210 INFO 1947952:repeater task daily_netcdf starting 34.2 seconds late 2023-11-16 13:16:58,996 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 13:16:58,997 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 13:26:24 2023-11-16 13:17:04,004 INFO 1947952:repeater task noisy_rawtemps_os150 starting 40.0 seconds late 2023-11-16 13:17:06,960 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 13:17:06,960 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 13:26:24 2023-11-16 13:17:06,960 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 42.9 seconds late 2023-11-16 13:17:09,764 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 13:17:09,764 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 13:26:24 2023-11-16 13:17:09,765 INFO 1947952:repeater task noisy_rawtemps_os38 starting 45.7 seconds late 2023-11-16 13:17:12,745 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 13:17:12,746 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 13:26:24 2023-11-16 13:18:27,944 INFO 1947952:repeater task lastensq_wh300 starting 4.9 seconds late 2023-11-16 13:18:30,736 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:18:30,736 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:18:18 2023-11-16 13:18:30,736 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:20:23 2023-11-16 13:18:30,736 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-16 13:18:37,179 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 13:18:37,180 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:18:18 2023-11-16 13:18:37,180 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:20:23 2023-11-16 13:18:37,180 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.2 seconds late 2023-11-16 13:18:37,813 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:18:37,813 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:18:18 2023-11-16 13:18:37,813 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:20:23 2023-11-16 13:20:23,100 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 13:20:26,120 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 13:20:26,121 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:20:18 2023-11-16 13:20:26,121 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:22:23 2023-11-16 13:20:26,121 INFO 1947952:repeater task quick_wh300 starting 3.1 seconds late 2023-11-16 13:20:32,536 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 13:20:32,536 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:20:18 2023-11-16 13:20:32,536 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:22:23 2023-11-16 13:20:32,537 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.5 seconds late 2023-11-16 13:20:33,165 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:20:33,166 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:20:18 2023-11-16 13:20:33,166 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:22:23 2023-11-16 13:21:23,296 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.2 seconds late 2023-11-16 13:21:24,483 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 13:21:24,484 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:24,484 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.4 seconds late 2023-11-16 13:21:24,654 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 13:21:24,655 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:24,655 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.6 seconds late 2023-11-16 13:21:25,948 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 13:21:25,948 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:25,949 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.9 seconds late 2023-11-16 13:21:27,206 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 13:21:27,206 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:27,206 INFO 1947952:repeater task lastensq_os150nb starting 4.2 seconds late 2023-11-16 13:21:29,434 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 13:21:29,435 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:21:18 2023-11-16 13:21:29,435 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:26:23 2023-11-16 13:21:29,435 INFO 1947952:repeater task quick_os150nb starting 6.4 seconds late 2023-11-16 13:21:32,981 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 13:21:32,981 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:21:18 2023-11-16 13:21:32,981 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:26:23 2023-11-16 13:21:32,982 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.0 seconds late 2023-11-16 13:21:33,600 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:21:33,600 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:21:18 2023-11-16 13:21:33,600 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:26:23 2023-11-16 13:21:33,601 INFO 1947952:repeater task 3dayplots_os150nb starting 9.5 seconds late 2023-11-16 13:21:36,954 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:21:36,955 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:26:24 2023-11-16 13:21:36,955 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.9 seconds late 2023-11-16 13:21:37,906 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 13:21:37,907 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:37,907 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.9 seconds late 2023-11-16 13:21:38,072 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 13:21:38,072 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:38,072 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.0 seconds late 2023-11-16 13:21:39,323 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 13:21:39,323 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:39,323 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.3 seconds late 2023-11-16 13:21:40,574 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 13:21:40,574 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:40,574 INFO 1947952:repeater task lastensq_os38nb starting 15.6 seconds late 2023-11-16 13:21:42,698 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 13:21:42,699 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:21:20 2023-11-16 13:21:42,699 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:26:25 2023-11-16 13:21:42,699 INFO 1947952:repeater task quick_os38nb starting 17.7 seconds late 2023-11-16 13:21:47,068 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 13:21:47,069 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:21:20 2023-11-16 13:21:47,069 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:26:25 2023-11-16 13:21:47,069 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.1 seconds late 2023-11-16 13:21:47,687 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:21:47,688 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:21:20 2023-11-16 13:21:47,688 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:26:25 2023-11-16 13:21:47,688 INFO 1947952:repeater task 3dayplots_os38nb starting 23.6 seconds late 2023-11-16 13:21:51,120 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 13:21:51,120 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:26:24 2023-11-16 13:21:51,120 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.1 seconds late 2023-11-16 13:21:52,119 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 13:21:52,120 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:52,121 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.1 seconds late 2023-11-16 13:21:52,287 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 13:21:52,287 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:52,287 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.2 seconds late 2023-11-16 13:21:53,564 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 13:21:53,565 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:53,565 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.5 seconds late 2023-11-16 13:21:54,738 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 13:21:54,738 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:54,739 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 35.5 seconds late 2023-11-16 13:21:55,971 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.2 seconds 2023-11-16 13:21:55,971 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:55,971 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 36.7 seconds late 2023-11-16 13:21:57,234 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 13:21:57,235 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:57,235 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 38.0 seconds late 2023-11-16 13:21:58,512 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 13:21:58,513 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:21:58,513 INFO 1947952:repeater task plot_hbins starting 39.2 seconds late 2023-11-16 13:22:02,867 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 13:22:02,867 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 13:22:02,867 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 13:36:19 2023-11-16 13:22:07,875 INFO 1947952:repeater task 3dayplots_wh300 starting 43.8 seconds late 2023-11-16 13:22:11,699 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:22:11,699 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:26:24 2023-11-16 13:22:26,730 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-16 13:22:29,497 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:22:29,498 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:22:18 2023-11-16 13:22:29,498 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:24:23 2023-11-16 13:22:29,498 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-16 13:22:36,220 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 13:22:36,221 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:22:18 2023-11-16 13:22:36,221 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:24:23 2023-11-16 13:22:36,221 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.2 seconds late 2023-11-16 13:22:36,852 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:22:36,852 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:22:18 2023-11-16 13:22:36,852 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:24:23 2023-11-16 13:24:27,158 INFO 1947952:repeater task lastensq_wh300 starting 4.2 seconds late 2023-11-16 13:24:30,105 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 13:24:30,105 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:24:18 2023-11-16 13:24:30,106 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:26:23 2023-11-16 13:24:30,106 INFO 1947952:repeater task quick_wh300 starting 7.1 seconds late 2023-11-16 13:24:36,653 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 13:24:36,653 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:24:18 2023-11-16 13:24:36,653 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:26:23 2023-11-16 13:24:36,654 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-16 13:24:37,275 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:24:37,275 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:24:18 2023-11-16 13:24:37,276 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:26:23 2023-11-16 13:26:27,569 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.5 seconds late 2023-11-16 13:26:30,442 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 13:26:30,442 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 13:36:24 2023-11-16 13:26:30,442 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.4 seconds late 2023-11-16 13:26:33,234 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 13:26:33,234 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 13:36:24 2023-11-16 13:26:33,235 INFO 1947952:repeater task noisy_rawtemps_os38 starting 9.2 seconds late 2023-11-16 13:26:36,255 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 13:26:36,255 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 13:36:24 2023-11-16 13:26:36,255 INFO 1947952:repeater task lastensq_wh300 starting 13.3 seconds late 2023-11-16 13:26:38,708 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-16 13:26:38,708 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:26:18 2023-11-16 13:26:38,708 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:28:23 2023-11-16 13:26:38,709 INFO 1947952:repeater task quick_wh300 starting 15.7 seconds late 2023-11-16 13:26:45,394 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 13:26:45,395 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:26:18 2023-11-16 13:26:45,395 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:28:23 2023-11-16 13:26:45,395 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.4 seconds late 2023-11-16 13:26:46,030 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:26:46,031 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:26:18 2023-11-16 13:26:46,031 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:28:23 2023-11-16 13:26:46,031 INFO 1947952:repeater task 3dayplots_wh300 starting 22.0 seconds late 2023-11-16 13:26:49,843 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:26:49,843 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:31:24 2023-11-16 13:26:49,844 INFO 1947952:repeater task lastensq_os150nb starting 26.8 seconds late 2023-11-16 13:26:52,105 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 13:26:52,106 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:26:18 2023-11-16 13:26:52,106 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:31:23 2023-11-16 13:26:52,106 INFO 1947952:repeater task quick_os150nb starting 29.1 seconds late 2023-11-16 13:26:56,034 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 13:26:56,034 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:26:18 2023-11-16 13:26:56,035 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:31:23 2023-11-16 13:26:56,035 INFO 1947952:repeater task lastfew_vec_os150nb starting 33.0 seconds late 2023-11-16 13:26:56,645 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:26:56,645 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:26:18 2023-11-16 13:26:56,645 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:31:23 2023-11-16 13:26:56,645 INFO 1947952:repeater task 3dayplots_os150nb starting 32.6 seconds late 2023-11-16 13:27:00,053 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:27:00,053 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:31:24 2023-11-16 13:27:00,055 INFO 1947952:repeater task lastensq_os38nb starting 35.1 seconds late 2023-11-16 13:27:02,114 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 13:27:02,114 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:26:20 2023-11-16 13:27:02,114 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:31:25 2023-11-16 13:27:02,115 INFO 1947952:repeater task quick_os38nb starting 37.1 seconds late 2023-11-16 13:27:06,556 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 13:27:06,556 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:26:20 2023-11-16 13:27:06,556 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:31:25 2023-11-16 13:27:06,557 INFO 1947952:repeater task lastfew_vec_os38nb starting 41.6 seconds late 2023-11-16 13:27:07,178 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:27:07,179 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:26:20 2023-11-16 13:27:07,179 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:31:25 2023-11-16 13:27:07,179 INFO 1947952:repeater task 3dayplots_os38nb starting 43.1 seconds late 2023-11-16 13:27:10,571 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 13:27:10,571 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:31:24 2023-11-16 13:27:10,573 INFO 1947952:repeater task daily_netcdf starting 46.5 seconds late 2023-11-16 13:27:11,375 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 13:27:11,375 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 13:36:24 2023-11-16 13:28:26,569 INFO 1947952:repeater task lastensq_wh300 starting 3.6 seconds late 2023-11-16 13:28:29,167 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 13:28:29,168 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:28:18 2023-11-16 13:28:29,168 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:30:23 2023-11-16 13:28:29,168 INFO 1947952:repeater task quick_wh300 starting 6.2 seconds late 2023-11-16 13:28:35,897 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 13:28:35,898 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:28:18 2023-11-16 13:28:35,898 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:30:23 2023-11-16 13:28:35,898 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 13:28:36,536 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:28:36,536 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:28:18 2023-11-16 13:28:36,536 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:30:23 2023-11-16 13:30:26,822 INFO 1947952:repeater task lastensq_wh300 starting 3.8 seconds late 2023-11-16 13:30:29,750 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 13:30:29,751 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:30:18 2023-11-16 13:30:29,751 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:32:23 2023-11-16 13:30:29,751 INFO 1947952:repeater task quick_wh300 starting 6.8 seconds late 2023-11-16 13:30:35,874 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 13:30:35,874 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:30:18 2023-11-16 13:30:35,875 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:32:23 2023-11-16 13:30:35,875 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 13:30:36,509 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:30:36,509 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:30:18 2023-11-16 13:30:36,509 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:32:23 2023-11-16 13:31:26,636 INFO 1947952:repeater task 3dayplots_wh300 starting 2.6 seconds late 2023-11-16 13:31:30,451 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:31:30,451 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:36:24 2023-11-16 13:31:30,452 INFO 1947952:repeater task lastensq_os150nb starting 7.5 seconds late 2023-11-16 13:31:32,756 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 13:31:32,757 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:31:18 2023-11-16 13:31:32,757 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:36:23 2023-11-16 13:31:32,757 INFO 1947952:repeater task quick_os150nb starting 9.8 seconds late 2023-11-16 13:31:36,624 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 13:31:36,624 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:31:18 2023-11-16 13:31:36,624 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:36:23 2023-11-16 13:31:36,625 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.6 seconds late 2023-11-16 13:31:37,239 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:31:37,240 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:31:18 2023-11-16 13:31:37,240 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:36:23 2023-11-16 13:31:37,240 INFO 1947952:repeater task 3dayplots_os150nb starting 13.2 seconds late 2023-11-16 13:31:40,587 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-16 13:31:40,587 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:36:24 2023-11-16 13:31:40,588 INFO 1947952:repeater task lastensq_os38nb starting 15.6 seconds late 2023-11-16 13:31:42,756 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 13:31:42,757 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:31:20 2023-11-16 13:31:42,757 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:36:25 2023-11-16 13:31:42,757 INFO 1947952:repeater task quick_os38nb starting 17.8 seconds late 2023-11-16 13:31:47,063 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 13:31:47,063 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:31:20 2023-11-16 13:31:47,063 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:36:25 2023-11-16 13:31:47,064 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.1 seconds late 2023-11-16 13:31:47,679 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:31:47,680 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:31:20 2023-11-16 13:31:47,680 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:36:25 2023-11-16 13:31:47,680 INFO 1947952:repeater task 3dayplots_os38nb starting 23.6 seconds late 2023-11-16 13:31:51,085 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 13:31:51,085 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:36:24 2023-11-16 13:32:26,153 INFO 1947952:repeater task lastensq_wh300 starting 3.2 seconds late 2023-11-16 13:32:29,072 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 13:32:29,072 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:32:18 2023-11-16 13:32:29,072 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:34:23 2023-11-16 13:32:29,072 INFO 1947952:repeater task quick_wh300 starting 6.1 seconds late 2023-11-16 13:32:34,939 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 13:32:34,940 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:32:18 2023-11-16 13:32:34,940 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:34:23 2023-11-16 13:32:34,941 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-16 13:32:35,573 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:32:35,574 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:32:18 2023-11-16 13:32:35,574 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:34:23 2023-11-16 13:34:25,865 INFO 1947952:repeater task lastensq_wh300 starting 2.9 seconds late 2023-11-16 13:34:28,593 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 13:34:28,594 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:34:18 2023-11-16 13:34:28,594 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:36:23 2023-11-16 13:34:28,594 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-16 13:34:34,609 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 13:34:34,610 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:34:18 2023-11-16 13:34:34,610 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:36:23 2023-11-16 13:34:34,610 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.6 seconds late 2023-11-16 13:34:35,248 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:34:35,249 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:34:18 2023-11-16 13:34:35,249 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:36:23 2023-11-16 13:36:20,529 INFO 1947952:repeater task plot_tsstats_wh300 starting 1.5 seconds late 2023-11-16 13:36:21,741 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 13:36:21,742 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:21,743 INFO 1947952:repeater task grab_lastpings_wh300 starting 2.7 seconds late 2023-11-16 13:36:21,923 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 13:36:21,924 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:21,924 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.9 seconds late 2023-11-16 13:36:23,213 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 13:36:23,213 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:23,214 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.2 seconds late 2023-11-16 13:36:24,613 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.4 seconds 2023-11-16 13:36:24,614 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:24,614 INFO 1947952:repeater task lastensq_os150nb starting 1.6 seconds late 2023-11-16 13:36:26,922 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 13:36:26,923 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:36:18 2023-11-16 13:36:26,923 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:41:23 2023-11-16 13:36:26,923 INFO 1947952:repeater task quick_os150nb starting 3.9 seconds late 2023-11-16 13:36:30,702 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 13:36:30,702 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:36:18 2023-11-16 13:36:30,702 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:41:23 2023-11-16 13:36:30,703 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.7 seconds late 2023-11-16 13:36:31,311 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:36:31,312 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:36:18 2023-11-16 13:36:31,312 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:41:23 2023-11-16 13:36:31,312 INFO 1947952:repeater task 3dayplots_os150nb starting 7.3 seconds late 2023-11-16 13:36:34,720 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:36:34,720 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:41:24 2023-11-16 13:36:34,721 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.7 seconds late 2023-11-16 13:36:35,681 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 13:36:35,682 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:35,682 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.6 seconds late 2023-11-16 13:36:35,847 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 13:36:35,847 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:35,848 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.8 seconds late 2023-11-16 13:36:37,064 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 13:36:37,064 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:37,065 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.0 seconds late 2023-11-16 13:36:38,319 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 13:36:38,319 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:38,320 INFO 1947952:repeater task lastensq_os38nb starting 13.3 seconds late 2023-11-16 13:36:40,466 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 13:36:40,467 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:36:20 2023-11-16 13:36:40,467 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:41:25 2023-11-16 13:36:40,467 INFO 1947952:repeater task quick_os38nb starting 15.5 seconds late 2023-11-16 13:36:44,954 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 13:36:44,954 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:36:20 2023-11-16 13:36:44,954 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:41:25 2023-11-16 13:36:44,955 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.0 seconds late 2023-11-16 13:36:45,563 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:36:45,564 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:36:20 2023-11-16 13:36:45,564 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:41:25 2023-11-16 13:36:45,564 INFO 1947952:repeater task 3dayplots_os38nb starting 21.5 seconds late 2023-11-16 13:36:49,026 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 13:36:49,026 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:41:24 2023-11-16 13:36:49,026 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.0 seconds late 2023-11-16 13:36:50,013 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 13:36:50,014 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:50,014 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.0 seconds late 2023-11-16 13:36:50,182 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 13:36:50,182 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:50,183 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.1 seconds late 2023-11-16 13:36:51,401 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 13:36:51,401 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:51,401 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.3 seconds late 2023-11-16 13:36:52,671 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 13:36:52,671 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:36:52,672 INFO 1947952:repeater task daily_netcdf starting 28.6 seconds late 2023-11-16 13:36:53,486 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 13:36:53,487 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 13:46:24 2023-11-16 13:36:53,487 INFO 1947952:repeater task ggatime_diagnostics starting 34.4 seconds late 2023-11-16 13:37:09,540 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-16 13:37:09,540 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:37:09,541 INFO 1947952:repeater task ggatime_diagnostics starting 50.5 seconds late 2023-11-16 13:37:25,702 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-16 13:37:25,702 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:37:25,703 INFO 1947952:repeater task ggatime_diagnostics starting 66.7 seconds late 2023-11-16 13:37:41,742 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 13:37:41,743 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:37:41,743 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 82.5 seconds late 2023-11-16 13:37:43,109 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.4 seconds 2023-11-16 13:37:43,110 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:37:43,110 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 83.8 seconds late 2023-11-16 13:37:44,441 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 13:37:44,442 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:37:44,442 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 85.2 seconds late 2023-11-16 13:37:45,692 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 13:37:45,692 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:37:45,692 INFO 1947952:repeater task plot_hbins starting 86.4 seconds late 2023-11-16 13:37:49,935 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 13:37:49,935 INFO 1947952:repeater task plot_hbins ended after 4.2 seconds 2023-11-16 13:37:49,935 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 13:51:19 2023-11-16 13:37:49,936 INFO 1947952:repeater task report_generator starting 90.7 seconds late 2023-11-16 13:37:55,982 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 13:37:55,982 INFO 1947952:repeater task report_generator ended after 6.0 seconds 2023-11-16 13:37:55,982 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 14:36:19 2023-11-16 13:38:00,985 INFO 1947952:repeater task noisy_rawtemps_os150 starting 96.9 seconds late 2023-11-16 13:38:03,893 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 13:38:03,894 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 13:46:24 2023-11-16 13:38:03,894 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 99.8 seconds late 2023-11-16 13:38:06,702 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 13:38:06,702 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 13:46:24 2023-11-16 13:38:06,702 INFO 1947952:repeater task noisy_rawtemps_os38 starting 102.7 seconds late 2023-11-16 13:38:09,729 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 13:38:09,729 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 13:46:24 2023-11-16 13:38:09,729 INFO 1947952:repeater task lastensq_wh300 starting 106.7 seconds late 2023-11-16 13:38:12,646 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 13:38:12,647 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:36:18 2023-11-16 13:38:12,647 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:38:23 2023-11-16 13:38:12,647 INFO 1947952:repeater task quick_wh300 starting 109.6 seconds late 2023-11-16 13:38:19,301 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 13:38:19,302 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:36:18 2023-11-16 13:38:19,302 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:38:23 2023-11-16 13:38:19,302 INFO 1947952:repeater task lastfew_vec_wh300 starting 116.3 seconds late 2023-11-16 13:38:19,952 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 13:38:19,953 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:36:18 2023-11-16 13:38:19,953 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:38:23 2023-11-16 13:38:19,953 INFO 1947952:repeater task 3dayplots_wh300 starting 115.9 seconds late 2023-11-16 13:38:23,745 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:38:23,745 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:41:24 2023-11-16 13:38:28,754 INFO 1947952:repeater task lastensq_wh300 starting 5.8 seconds late 2023-11-16 13:38:31,396 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 13:38:31,396 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:38:18 2023-11-16 13:38:31,396 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:40:23 2023-11-16 13:38:31,397 INFO 1947952:repeater task quick_wh300 starting 8.4 seconds late 2023-11-16 13:38:37,844 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 13:38:37,844 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:38:18 2023-11-16 13:38:37,845 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:40:23 2023-11-16 13:38:37,845 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-16 13:38:38,497 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 13:38:38,497 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:38:18 2023-11-16 13:38:38,497 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:40:23 2023-11-16 13:40:23,763 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-16 13:40:26,517 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:40:26,517 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:40:18 2023-11-16 13:40:26,517 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:42:23 2023-11-16 13:40:26,518 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 13:40:32,778 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 13:40:32,778 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:40:18 2023-11-16 13:40:32,778 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:42:23 2023-11-16 13:40:32,779 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.8 seconds late 2023-11-16 13:40:33,414 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:40:33,415 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:40:18 2023-11-16 13:40:33,415 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:42:23 2023-11-16 13:41:23,542 INFO 1947952:repeater task lastensq_os150nb starting 0.5 seconds late 2023-11-16 13:41:25,838 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 13:41:25,838 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:41:18 2023-11-16 13:41:25,838 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:46:23 2023-11-16 13:41:25,839 INFO 1947952:repeater task quick_os150nb starting 2.8 seconds late 2023-11-16 13:41:29,343 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 13:41:29,344 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:41:18 2023-11-16 13:41:29,344 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:46:23 2023-11-16 13:41:29,344 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.3 seconds late 2023-11-16 13:41:29,949 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:41:29,950 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:41:18 2023-11-16 13:41:29,950 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:46:23 2023-11-16 13:41:29,950 INFO 1947952:repeater task 3dayplots_os150nb starting 5.9 seconds late 2023-11-16 13:41:33,331 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:41:33,331 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:46:24 2023-11-16 13:41:33,333 INFO 1947952:repeater task lastensq_os38nb starting 8.3 seconds late 2023-11-16 13:41:35,299 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 13:41:35,300 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:41:20 2023-11-16 13:41:35,300 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:46:25 2023-11-16 13:41:35,300 INFO 1947952:repeater task quick_os38nb starting 10.3 seconds late 2023-11-16 13:41:39,713 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 13:41:39,714 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:41:20 2023-11-16 13:41:39,714 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:46:25 2023-11-16 13:41:39,714 INFO 1947952:repeater task lastfew_vec_os38nb starting 14.7 seconds late 2023-11-16 13:41:40,317 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:41:40,317 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:41:20 2023-11-16 13:41:40,317 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:46:25 2023-11-16 13:41:40,318 INFO 1947952:repeater task 3dayplots_os38nb starting 16.3 seconds late 2023-11-16 13:41:43,702 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 13:41:43,702 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:46:24 2023-11-16 13:41:48,712 INFO 1947952:repeater task 3dayplots_wh300 starting 24.7 seconds late 2023-11-16 13:41:52,537 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:41:52,537 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:46:24 2023-11-16 13:42:27,628 INFO 1947952:repeater task lastensq_wh300 starting 4.6 seconds late 2023-11-16 13:42:30,111 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-16 13:42:30,111 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:42:18 2023-11-16 13:42:30,111 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:44:23 2023-11-16 13:42:30,111 INFO 1947952:repeater task quick_wh300 starting 7.1 seconds late 2023-11-16 13:42:36,885 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-16 13:42:36,885 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:42:18 2023-11-16 13:42:36,886 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:44:23 2023-11-16 13:42:36,886 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2023-11-16 13:42:37,509 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:42:37,509 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:42:18 2023-11-16 13:42:37,509 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:44:23 2023-11-16 13:44:27,807 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-16 13:44:30,627 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:44:30,627 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:44:18 2023-11-16 13:44:30,627 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:46:23 2023-11-16 13:44:30,628 INFO 1947952:repeater task quick_wh300 starting 7.6 seconds late 2023-11-16 13:44:37,293 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 13:44:37,294 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:44:18 2023-11-16 13:44:37,294 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:46:23 2023-11-16 13:44:37,294 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.3 seconds late 2023-11-16 13:44:37,921 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:44:37,921 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:44:18 2023-11-16 13:44:37,921 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:46:23 2023-11-16 13:46:23,199 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-16 13:46:26,274 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 13:46:26,275 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:46:18 2023-11-16 13:46:26,275 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:48:23 2023-11-16 13:46:26,275 INFO 1947952:repeater task quick_wh300 starting 3.3 seconds late 2023-11-16 13:46:32,454 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 13:46:32,454 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:46:18 2023-11-16 13:46:32,455 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:48:23 2023-11-16 13:46:32,455 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.5 seconds late 2023-11-16 13:46:33,085 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:46:33,085 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:46:18 2023-11-16 13:46:33,085 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:48:23 2023-11-16 13:46:33,086 INFO 1947952:repeater task 3dayplots_wh300 starting 9.0 seconds late 2023-11-16 13:46:36,884 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:46:36,884 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:51:24 2023-11-16 13:46:36,886 INFO 1947952:repeater task lastensq_os150nb starting 13.9 seconds late 2023-11-16 13:46:39,005 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 13:46:39,005 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:46:18 2023-11-16 13:46:39,005 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:51:23 2023-11-16 13:46:39,006 INFO 1947952:repeater task quick_os150nb starting 16.0 seconds late 2023-11-16 13:46:42,979 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-16 13:46:42,979 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:46:18 2023-11-16 13:46:42,979 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:51:23 2023-11-16 13:46:42,980 INFO 1947952:repeater task lastfew_vec_os150nb starting 20.0 seconds late 2023-11-16 13:46:43,596 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:46:43,596 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:46:18 2023-11-16 13:46:43,596 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:51:23 2023-11-16 13:46:43,597 INFO 1947952:repeater task 3dayplots_os150nb starting 19.5 seconds late 2023-11-16 13:46:47,026 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:46:47,026 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:51:24 2023-11-16 13:46:47,027 INFO 1947952:repeater task lastensq_os38nb starting 22.0 seconds late 2023-11-16 13:46:48,958 INFO 1947952:repeater task lastensq_os38nb ended after 1.9 seconds 2023-11-16 13:46:48,958 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:46:20 2023-11-16 13:46:48,958 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:51:25 2023-11-16 13:46:48,959 INFO 1947952:repeater task quick_os38nb starting 24.0 seconds late 2023-11-16 13:46:53,260 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 13:46:53,261 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:46:20 2023-11-16 13:46:53,261 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:51:25 2023-11-16 13:46:53,261 INFO 1947952:repeater task lastfew_vec_os38nb starting 28.3 seconds late 2023-11-16 13:46:53,869 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:46:53,869 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:46:20 2023-11-16 13:46:53,869 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:51:25 2023-11-16 13:46:53,870 INFO 1947952:repeater task 3dayplots_os38nb starting 29.8 seconds late 2023-11-16 13:46:57,333 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 13:46:57,334 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:51:24 2023-11-16 13:46:57,335 INFO 1947952:repeater task daily_netcdf starting 33.3 seconds late 2023-11-16 13:46:58,144 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 13:46:58,144 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 13:56:24 2023-11-16 13:47:03,149 INFO 1947952:repeater task noisy_rawtemps_os150 starting 39.1 seconds late 2023-11-16 13:47:06,069 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 13:47:06,069 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 13:56:24 2023-11-16 13:47:06,070 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 42.0 seconds late 2023-11-16 13:47:08,982 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 13:47:08,982 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 13:56:24 2023-11-16 13:47:08,983 INFO 1947952:repeater task noisy_rawtemps_os38 starting 44.9 seconds late 2023-11-16 13:47:11,937 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 13:47:11,937 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 13:56:24 2023-11-16 13:48:27,140 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-16 13:48:29,902 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:48:29,903 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:48:18 2023-11-16 13:48:29,903 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:50:23 2023-11-16 13:48:29,903 INFO 1947952:repeater task quick_wh300 starting 6.9 seconds late 2023-11-16 13:48:36,924 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-16 13:48:36,924 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:48:18 2023-11-16 13:48:36,925 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:50:23 2023-11-16 13:48:36,925 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2023-11-16 13:48:37,556 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:48:37,556 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:48:18 2023-11-16 13:48:37,556 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:50:23 2023-11-16 13:50:27,831 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-16 13:50:30,695 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 13:50:30,695 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:50:18 2023-11-16 13:50:30,695 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:52:23 2023-11-16 13:50:30,696 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-16 13:50:36,893 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 13:50:36,894 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:50:18 2023-11-16 13:50:36,894 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:52:23 2023-11-16 13:50:36,894 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2023-11-16 13:50:37,525 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:50:37,525 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:50:18 2023-11-16 13:50:37,525 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:52:23 2023-11-16 13:51:22,637 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.6 seconds late 2023-11-16 13:51:23,799 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 13:51:23,799 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:23,800 INFO 1947952:repeater task grab_lastpings_wh300 starting 4.7 seconds late 2023-11-16 13:51:23,971 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 13:51:23,971 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:23,971 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.9 seconds late 2023-11-16 13:51:25,318 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 13:51:25,318 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:25,319 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.3 seconds late 2023-11-16 13:51:26,600 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 13:51:26,600 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:26,600 INFO 1947952:repeater task lastensq_os150nb starting 3.6 seconds late 2023-11-16 13:51:29,153 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 13:51:29,154 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:51:18 2023-11-16 13:51:29,154 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 13:56:23 2023-11-16 13:51:29,154 INFO 1947952:repeater task quick_os150nb starting 6.2 seconds late 2023-11-16 13:51:32,790 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 13:51:32,790 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:51:18 2023-11-16 13:51:32,791 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 13:56:23 2023-11-16 13:51:32,791 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.8 seconds late 2023-11-16 13:51:33,394 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:51:33,394 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:51:18 2023-11-16 13:51:33,394 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 13:56:23 2023-11-16 13:51:33,395 INFO 1947952:repeater task 3dayplots_os150nb starting 9.3 seconds late 2023-11-16 13:51:36,794 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:51:36,794 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 13:56:24 2023-11-16 13:51:36,795 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.7 seconds late 2023-11-16 13:51:37,755 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 13:51:37,755 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:37,756 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.7 seconds late 2023-11-16 13:51:37,923 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 13:51:37,923 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:37,924 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.9 seconds late 2023-11-16 13:51:39,184 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 13:51:39,184 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:39,184 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.1 seconds late 2023-11-16 13:51:40,378 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 13:51:40,378 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:40,378 INFO 1947952:repeater task lastensq_os38nb starting 15.4 seconds late 2023-11-16 13:51:42,420 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 13:51:42,420 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:51:20 2023-11-16 13:51:42,421 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 13:56:25 2023-11-16 13:51:42,421 INFO 1947952:repeater task quick_os38nb starting 17.4 seconds late 2023-11-16 13:51:46,633 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 13:51:46,633 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:51:20 2023-11-16 13:51:46,633 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 13:56:25 2023-11-16 13:51:46,634 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.6 seconds late 2023-11-16 13:51:47,245 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:51:47,246 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:51:20 2023-11-16 13:51:47,246 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 13:56:25 2023-11-16 13:51:47,246 INFO 1947952:repeater task 3dayplots_os38nb starting 23.2 seconds late 2023-11-16 13:51:50,630 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 13:51:50,630 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 13:56:24 2023-11-16 13:51:50,630 INFO 1947952:repeater task plot_tsstats_os38nb starting 31.6 seconds late 2023-11-16 13:51:51,630 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 13:51:51,631 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:51,631 INFO 1947952:repeater task grab_lastpings_os38nb starting 32.6 seconds late 2023-11-16 13:51:51,798 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 13:51:51,798 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:51,799 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.7 seconds late 2023-11-16 13:51:53,089 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 13:51:53,089 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:53,089 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.0 seconds late 2023-11-16 13:51:54,332 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 13:51:54,332 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:54,333 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 35.1 seconds late 2023-11-16 13:51:55,633 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 13:51:55,633 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:55,634 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 36.4 seconds late 2023-11-16 13:51:56,917 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 13:51:56,917 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:56,918 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 37.6 seconds late 2023-11-16 13:51:58,184 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 13:51:58,185 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:51:58,185 INFO 1947952:repeater task plot_hbins starting 38.9 seconds late 2023-11-16 13:52:02,471 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 13:52:02,471 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 13:52:02,471 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 14:06:19 2023-11-16 13:52:07,479 INFO 1947952:repeater task 3dayplots_wh300 starting 43.4 seconds late 2023-11-16 13:52:11,244 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 13:52:11,244 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 13:56:24 2023-11-16 13:52:26,271 INFO 1947952:repeater task lastensq_wh300 starting 3.3 seconds late 2023-11-16 13:52:29,076 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:52:29,076 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:52:18 2023-11-16 13:52:29,076 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:54:23 2023-11-16 13:52:29,077 INFO 1947952:repeater task quick_wh300 starting 6.1 seconds late 2023-11-16 13:52:35,387 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 13:52:35,387 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:52:18 2023-11-16 13:52:35,387 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:54:23 2023-11-16 13:52:35,388 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2023-11-16 13:52:36,007 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:52:36,007 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:52:18 2023-11-16 13:52:36,007 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:54:23 2023-11-16 13:54:26,287 INFO 1947952:repeater task lastensq_wh300 starting 3.3 seconds late 2023-11-16 13:54:29,118 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:54:29,118 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:54:18 2023-11-16 13:54:29,118 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:56:23 2023-11-16 13:54:29,119 INFO 1947952:repeater task quick_wh300 starting 6.1 seconds late 2023-11-16 13:54:35,290 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 13:54:35,291 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:54:18 2023-11-16 13:54:35,291 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:56:23 2023-11-16 13:54:35,291 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2023-11-16 13:54:35,923 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:54:35,924 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:54:18 2023-11-16 13:54:35,924 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:56:23 2023-11-16 13:56:26,189 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.1 seconds late 2023-11-16 13:56:29,253 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2023-11-16 13:56:29,253 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 14:06:24 2023-11-16 13:56:29,254 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.2 seconds late 2023-11-16 13:56:32,051 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 13:56:32,051 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 14:06:24 2023-11-16 13:56:32,051 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.0 seconds late 2023-11-16 13:56:35,092 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 13:56:35,093 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 14:06:24 2023-11-16 13:56:35,093 INFO 1947952:repeater task lastensq_wh300 starting 12.1 seconds late 2023-11-16 13:56:37,910 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 13:56:37,911 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:56:18 2023-11-16 13:56:37,911 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 13:58:23 2023-11-16 13:56:37,911 INFO 1947952:repeater task quick_wh300 starting 14.9 seconds late 2023-11-16 13:56:44,341 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 13:56:44,342 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:56:18 2023-11-16 13:56:44,342 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 13:58:23 2023-11-16 13:56:44,342 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.3 seconds late 2023-11-16 13:56:44,974 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:56:44,975 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:56:18 2023-11-16 13:56:44,975 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 13:58:23 2023-11-16 13:56:44,975 INFO 1947952:repeater task 3dayplots_wh300 starting 20.9 seconds late 2023-11-16 13:56:48,840 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 13:56:48,840 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:01:24 2023-11-16 13:56:48,842 INFO 1947952:repeater task lastensq_os150nb starting 25.8 seconds late 2023-11-16 13:56:51,146 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 13:56:51,147 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:56:18 2023-11-16 13:56:51,147 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:01:23 2023-11-16 13:56:51,147 INFO 1947952:repeater task quick_os150nb starting 28.1 seconds late 2023-11-16 13:56:54,824 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 13:56:54,825 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:56:18 2023-11-16 13:56:54,825 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:01:23 2023-11-16 13:56:54,825 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.8 seconds late 2023-11-16 13:56:55,449 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 13:56:55,449 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 13:56:18 2023-11-16 13:56:55,449 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:01:23 2023-11-16 13:56:55,450 INFO 1947952:repeater task 3dayplots_os150nb starting 31.4 seconds late 2023-11-16 13:56:58,816 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 13:56:58,816 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:01:24 2023-11-16 13:56:58,817 INFO 1947952:repeater task lastensq_os38nb starting 33.8 seconds late 2023-11-16 13:57:00,750 INFO 1947952:repeater task lastensq_os38nb ended after 1.9 seconds 2023-11-16 13:57:00,750 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:56:20 2023-11-16 13:57:00,750 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:01:25 2023-11-16 13:57:00,751 INFO 1947952:repeater task quick_os38nb starting 35.8 seconds late 2023-11-16 13:57:05,143 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 13:57:05,144 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:56:20 2023-11-16 13:57:05,144 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:01:25 2023-11-16 13:57:05,144 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.1 seconds late 2023-11-16 13:57:05,758 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 13:57:05,758 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 13:56:20 2023-11-16 13:57:05,758 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:01:25 2023-11-16 13:57:05,758 INFO 1947952:repeater task 3dayplots_os38nb starting 41.7 seconds late 2023-11-16 13:57:09,153 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 13:57:09,154 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:01:24 2023-11-16 13:57:09,155 INFO 1947952:repeater task daily_netcdf starting 45.1 seconds late 2023-11-16 13:57:10,001 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 13:57:10,001 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 14:06:24 2023-11-16 13:58:25,209 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-16 13:58:28,743 INFO 1947952:repeater task lastensq_wh300 ended after 3.5 seconds 2023-11-16 13:58:28,744 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:58:18 2023-11-16 13:58:28,744 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:00:23 2023-11-16 13:58:28,744 INFO 1947952:repeater task quick_wh300 starting 5.7 seconds late 2023-11-16 13:58:35,207 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 13:58:35,208 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:58:18 2023-11-16 13:58:35,208 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:00:23 2023-11-16 13:58:35,208 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.2 seconds late 2023-11-16 13:58:35,837 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 13:58:35,838 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 13:58:18 2023-11-16 13:58:35,838 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:00:23 2023-11-16 14:00:26,130 INFO 1947952:repeater task lastensq_wh300 starting 3.1 seconds late 2023-11-16 14:00:29,106 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 14:00:29,106 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:00:18 2023-11-16 14:00:29,106 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:02:23 2023-11-16 14:00:29,106 INFO 1947952:repeater task quick_wh300 starting 6.1 seconds late 2023-11-16 14:00:35,494 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 14:00:35,494 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:00:18 2023-11-16 14:00:35,494 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:02:23 2023-11-16 14:00:35,494 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2023-11-16 14:00:36,132 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:00:36,132 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:00:18 2023-11-16 14:00:36,133 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:02:23 2023-11-16 14:01:26,251 INFO 1947952:repeater task 3dayplots_wh300 starting 2.2 seconds late 2023-11-16 14:01:30,106 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 14:01:30,106 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:06:24 2023-11-16 14:01:30,107 INFO 1947952:repeater task lastensq_os150nb starting 7.1 seconds late 2023-11-16 14:01:32,795 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-16 14:01:32,796 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:01:18 2023-11-16 14:01:32,796 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:06:23 2023-11-16 14:01:32,796 INFO 1947952:repeater task quick_os150nb starting 9.8 seconds late 2023-11-16 14:01:36,734 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 14:01:36,735 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:01:18 2023-11-16 14:01:36,735 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:06:23 2023-11-16 14:01:36,735 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.7 seconds late 2023-11-16 14:01:37,344 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:01:37,344 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:01:18 2023-11-16 14:01:37,344 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:06:23 2023-11-16 14:01:37,345 INFO 1947952:repeater task 3dayplots_os150nb starting 13.3 seconds late 2023-11-16 14:01:40,746 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:01:40,746 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:06:24 2023-11-16 14:01:40,748 INFO 1947952:repeater task lastensq_os38nb starting 15.7 seconds late 2023-11-16 14:01:42,691 INFO 1947952:repeater task lastensq_os38nb ended after 1.9 seconds 2023-11-16 14:01:42,691 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:01:20 2023-11-16 14:01:42,691 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:06:25 2023-11-16 14:01:42,692 INFO 1947952:repeater task quick_os38nb starting 17.7 seconds late 2023-11-16 14:01:46,968 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 14:01:46,968 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:01:20 2023-11-16 14:01:46,968 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:06:25 2023-11-16 14:01:46,969 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.0 seconds late 2023-11-16 14:01:47,585 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:01:47,585 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:01:20 2023-11-16 14:01:47,585 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:06:25 2023-11-16 14:01:47,586 INFO 1947952:repeater task 3dayplots_os38nb starting 23.5 seconds late 2023-11-16 14:01:51,060 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 14:01:51,061 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:06:24 2023-11-16 14:02:26,144 INFO 1947952:repeater task lastensq_wh300 starting 3.1 seconds late 2023-11-16 14:02:28,920 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 14:02:28,920 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:02:18 2023-11-16 14:02:28,920 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:04:23 2023-11-16 14:02:28,921 INFO 1947952:repeater task quick_wh300 starting 5.9 seconds late 2023-11-16 14:02:35,652 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 14:02:35,653 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:02:18 2023-11-16 14:02:35,653 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:04:23 2023-11-16 14:02:35,653 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2023-11-16 14:02:36,282 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:02:36,283 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:02:18 2023-11-16 14:02:36,283 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:04:23 2023-11-16 14:04:26,570 INFO 1947952:repeater task lastensq_wh300 starting 3.6 seconds late 2023-11-16 14:04:29,482 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 14:04:29,482 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:04:18 2023-11-16 14:04:29,482 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:06:23 2023-11-16 14:04:29,483 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-16 14:04:35,793 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 14:04:35,793 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:04:18 2023-11-16 14:04:35,793 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:06:23 2023-11-16 14:04:35,794 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.8 seconds late 2023-11-16 14:04:36,420 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:04:36,420 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:04:18 2023-11-16 14:04:36,420 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:06:23 2023-11-16 14:06:21,691 INFO 1947952:repeater task plot_tsstats_wh300 starting 2.6 seconds late 2023-11-16 14:06:22,873 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 14:06:22,873 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:22,873 INFO 1947952:repeater task grab_lastpings_wh300 starting 3.8 seconds late 2023-11-16 14:06:23,038 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 14:06:23,038 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:23,039 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.0 seconds late 2023-11-16 14:06:24,290 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:06:24,290 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:24,290 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.2 seconds late 2023-11-16 14:06:25,562 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:06:25,562 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:25,563 INFO 1947952:repeater task lastensq_os150nb starting 2.6 seconds late 2023-11-16 14:06:27,943 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 14:06:27,943 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:06:18 2023-11-16 14:06:27,943 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:11:23 2023-11-16 14:06:27,944 INFO 1947952:repeater task quick_os150nb starting 4.9 seconds late 2023-11-16 14:06:31,510 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 14:06:31,511 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:06:18 2023-11-16 14:06:31,511 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:11:23 2023-11-16 14:06:31,511 INFO 1947952:repeater task lastfew_vec_os150nb starting 8.5 seconds late 2023-11-16 14:06:32,124 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:06:32,124 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:06:18 2023-11-16 14:06:32,124 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:11:23 2023-11-16 14:06:32,125 INFO 1947952:repeater task 3dayplots_os150nb starting 8.1 seconds late 2023-11-16 14:06:35,558 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:06:35,558 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:11:24 2023-11-16 14:06:35,558 INFO 1947952:repeater task plot_tsstats_os150nb starting 16.5 seconds late 2023-11-16 14:06:36,537 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 14:06:36,538 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:36,538 INFO 1947952:repeater task grab_lastpings_os150nb starting 17.5 seconds late 2023-11-16 14:06:36,708 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 14:06:36,709 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:36,709 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.7 seconds late 2023-11-16 14:06:37,939 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 14:06:37,939 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:37,939 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.9 seconds late 2023-11-16 14:06:39,199 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 14:06:39,200 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:39,200 INFO 1947952:repeater task lastensq_os38nb starting 14.2 seconds late 2023-11-16 14:06:41,338 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 14:06:41,339 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:06:20 2023-11-16 14:06:41,339 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:11:25 2023-11-16 14:06:41,339 INFO 1947952:repeater task quick_os38nb starting 16.3 seconds late 2023-11-16 14:06:45,435 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 14:06:45,435 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:06:20 2023-11-16 14:06:45,435 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:11:25 2023-11-16 14:06:45,436 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.4 seconds late 2023-11-16 14:06:46,044 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:06:46,044 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:06:20 2023-11-16 14:06:46,045 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:11:25 2023-11-16 14:06:46,045 INFO 1947952:repeater task 3dayplots_os38nb starting 22.0 seconds late 2023-11-16 14:06:49,506 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 14:06:49,506 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:11:24 2023-11-16 14:06:49,507 INFO 1947952:repeater task plot_tsstats_os38nb starting 30.5 seconds late 2023-11-16 14:06:50,510 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 14:06:50,510 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:50,511 INFO 1947952:repeater task grab_lastpings_os38nb starting 31.5 seconds late 2023-11-16 14:06:50,681 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 14:06:50,681 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:50,681 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.6 seconds late 2023-11-16 14:06:51,918 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 14:06:51,918 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:51,919 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.9 seconds late 2023-11-16 14:06:53,168 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 14:06:53,168 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:06:53,169 INFO 1947952:repeater task daily_netcdf starting 29.1 seconds late 2023-11-16 14:06:53,966 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 14:06:53,966 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 14:16:24 2023-11-16 14:06:53,966 INFO 1947952:repeater task ggatime_diagnostics starting 34.9 seconds late 2023-11-16 14:07:10,420 INFO 1947952:repeater task ggatime_diagnostics ended after 16.5 seconds 2023-11-16 14:07:10,420 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:07:10,421 INFO 1947952:repeater task ggatime_diagnostics starting 51.4 seconds late 2023-11-16 14:07:26,172 INFO 1947952:repeater task ggatime_diagnostics ended after 15.8 seconds 2023-11-16 14:07:26,172 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:07:26,172 INFO 1947952:repeater task ggatime_diagnostics starting 67.1 seconds late 2023-11-16 14:07:41,965 INFO 1947952:repeater task ggatime_diagnostics ended after 15.8 seconds 2023-11-16 14:07:41,965 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:07:41,965 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 82.7 seconds late 2023-11-16 14:07:43,260 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 14:07:43,261 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:07:43,261 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 84.0 seconds late 2023-11-16 14:07:44,464 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.2 seconds 2023-11-16 14:07:44,464 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:07:44,465 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 85.2 seconds late 2023-11-16 14:07:45,721 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 14:07:45,721 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:07:45,721 INFO 1947952:repeater task plot_hbins starting 86.4 seconds late 2023-11-16 14:07:49,958 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 14:07:49,958 INFO 1947952:repeater task plot_hbins ended after 4.2 seconds 2023-11-16 14:07:49,958 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 14:21:19 2023-11-16 14:07:54,964 INFO 1947952:repeater task noisy_rawtemps_os150 starting 90.9 seconds late 2023-11-16 14:07:57,908 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 14:07:57,908 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 14:16:24 2023-11-16 14:07:57,908 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 93.9 seconds late 2023-11-16 14:08:00,703 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 14:08:00,703 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 14:16:24 2023-11-16 14:08:00,704 INFO 1947952:repeater task noisy_rawtemps_os38 starting 96.7 seconds late 2023-11-16 14:08:03,748 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 14:08:03,749 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 14:16:24 2023-11-16 14:08:03,749 INFO 1947952:repeater task lastensq_wh300 starting 100.7 seconds late 2023-11-16 14:08:06,366 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 14:08:06,366 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:06:18 2023-11-16 14:08:06,366 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:08:23 2023-11-16 14:08:06,367 INFO 1947952:repeater task quick_wh300 starting 103.4 seconds late 2023-11-16 14:08:12,749 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 14:08:12,750 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:06:18 2023-11-16 14:08:12,750 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:08:23 2023-11-16 14:08:12,750 INFO 1947952:repeater task lastfew_vec_wh300 starting 109.8 seconds late 2023-11-16 14:08:13,380 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:08:13,380 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:06:18 2023-11-16 14:08:13,380 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:08:23 2023-11-16 14:08:13,381 INFO 1947952:repeater task 3dayplots_wh300 starting 109.3 seconds late 2023-11-16 14:08:17,203 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:08:17,203 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:11:24 2023-11-16 14:08:27,222 INFO 1947952:repeater task lastensq_wh300 starting 4.2 seconds late 2023-11-16 14:08:30,268 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 14:08:30,269 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:08:18 2023-11-16 14:08:30,269 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:10:23 2023-11-16 14:08:30,269 INFO 1947952:repeater task quick_wh300 starting 7.3 seconds late 2023-11-16 14:08:36,581 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 14:08:36,582 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:08:18 2023-11-16 14:08:36,582 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:10:23 2023-11-16 14:08:36,582 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.6 seconds late 2023-11-16 14:08:37,216 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:08:37,216 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:08:18 2023-11-16 14:08:37,216 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:10:23 2023-11-16 14:10:27,499 INFO 1947952:repeater task lastensq_wh300 starting 4.5 seconds late 2023-11-16 14:10:30,229 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 14:10:30,230 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:10:18 2023-11-16 14:10:30,230 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:12:23 2023-11-16 14:10:30,230 INFO 1947952:repeater task quick_wh300 starting 7.2 seconds late 2023-11-16 14:10:36,963 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 14:10:36,963 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:10:18 2023-11-16 14:10:36,963 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:12:23 2023-11-16 14:10:36,964 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.0 seconds late 2023-11-16 14:10:37,602 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:10:37,602 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:10:18 2023-11-16 14:10:37,602 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:12:23 2023-11-16 14:11:27,731 INFO 1947952:repeater task 3dayplots_wh300 starting 3.7 seconds late 2023-11-16 14:11:31,529 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:11:31,529 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:16:24 2023-11-16 14:11:31,530 INFO 1947952:repeater task lastensq_os150nb starting 8.5 seconds late 2023-11-16 14:11:34,068 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 14:11:34,069 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:11:18 2023-11-16 14:11:34,069 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:16:23 2023-11-16 14:11:34,069 INFO 1947952:repeater task quick_os150nb starting 11.1 seconds late 2023-11-16 14:11:38,394 INFO 1947952:repeater task quick_os150nb ended after 4.3 seconds 2023-11-16 14:11:38,395 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:11:18 2023-11-16 14:11:38,395 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:16:23 2023-11-16 14:11:38,395 INFO 1947952:repeater task lastfew_vec_os150nb starting 15.4 seconds late 2023-11-16 14:11:39,000 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:11:39,001 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:11:18 2023-11-16 14:11:39,001 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:16:23 2023-11-16 14:11:39,001 INFO 1947952:repeater task 3dayplots_os150nb starting 15.0 seconds late 2023-11-16 14:11:42,419 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:11:42,419 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:16:24 2023-11-16 14:11:42,420 INFO 1947952:repeater task lastensq_os38nb starting 17.4 seconds late 2023-11-16 14:11:44,749 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-16 14:11:44,750 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:11:20 2023-11-16 14:11:44,750 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:16:25 2023-11-16 14:11:44,750 INFO 1947952:repeater task quick_os38nb starting 19.8 seconds late 2023-11-16 14:11:49,200 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 14:11:49,200 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:11:20 2023-11-16 14:11:49,200 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:16:25 2023-11-16 14:11:49,200 INFO 1947952:repeater task lastfew_vec_os38nb starting 24.2 seconds late 2023-11-16 14:11:49,810 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:11:49,810 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:11:20 2023-11-16 14:11:49,811 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:16:25 2023-11-16 14:11:49,811 INFO 1947952:repeater task 3dayplots_os38nb starting 25.8 seconds late 2023-11-16 14:11:53,310 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 14:11:53,310 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:16:24 2023-11-16 14:12:23,382 INFO 1947952:repeater task lastensq_wh300 starting 0.4 seconds late 2023-11-16 14:12:26,469 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 14:12:26,470 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:12:18 2023-11-16 14:12:26,470 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:14:23 2023-11-16 14:12:26,470 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 14:12:32,858 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 14:12:32,859 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:12:18 2023-11-16 14:12:32,859 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:14:23 2023-11-16 14:12:32,859 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 14:12:33,483 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:12:33,484 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:12:18 2023-11-16 14:12:33,484 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:14:23 2023-11-16 14:14:23,776 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-16 14:14:26,765 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 14:14:26,765 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:14:18 2023-11-16 14:14:26,766 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:16:23 2023-11-16 14:14:26,766 INFO 1947952:repeater task quick_wh300 starting 3.8 seconds late 2023-11-16 14:14:33,179 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 14:14:33,180 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:14:18 2023-11-16 14:14:33,180 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:16:23 2023-11-16 14:14:33,180 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2023-11-16 14:14:33,807 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:14:33,807 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:14:18 2023-11-16 14:14:33,807 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:16:23 2023-11-16 14:16:24,097 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.0 seconds late 2023-11-16 14:16:27,199 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2023-11-16 14:16:27,199 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 14:26:24 2023-11-16 14:16:27,200 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.2 seconds late 2023-11-16 14:16:30,136 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 14:16:30,137 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 14:26:24 2023-11-16 14:16:30,137 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.1 seconds late 2023-11-16 14:16:33,159 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 14:16:33,159 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 14:26:24 2023-11-16 14:16:33,160 INFO 1947952:repeater task lastensq_wh300 starting 10.2 seconds late 2023-11-16 14:16:35,827 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 14:16:35,827 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:16:18 2023-11-16 14:16:35,828 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:18:23 2023-11-16 14:16:35,828 INFO 1947952:repeater task quick_wh300 starting 12.8 seconds late 2023-11-16 14:16:41,831 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 14:16:41,832 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:16:18 2023-11-16 14:16:41,832 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:18:23 2023-11-16 14:16:41,832 INFO 1947952:repeater task lastfew_vec_wh300 starting 18.8 seconds late 2023-11-16 14:16:42,463 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:16:42,463 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:16:18 2023-11-16 14:16:42,463 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:18:23 2023-11-16 14:16:42,464 INFO 1947952:repeater task 3dayplots_wh300 starting 18.4 seconds late 2023-11-16 14:16:46,354 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 14:16:46,355 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:21:24 2023-11-16 14:16:46,356 INFO 1947952:repeater task lastensq_os150nb starting 23.4 seconds late 2023-11-16 14:16:48,827 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 14:16:48,827 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:16:18 2023-11-16 14:16:48,828 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:21:23 2023-11-16 14:16:48,828 INFO 1947952:repeater task quick_os150nb starting 25.8 seconds late 2023-11-16 14:16:52,494 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 14:16:52,495 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:16:18 2023-11-16 14:16:52,495 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:21:23 2023-11-16 14:16:52,495 INFO 1947952:repeater task lastfew_vec_os150nb starting 29.5 seconds late 2023-11-16 14:16:53,108 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:16:53,109 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:16:18 2023-11-16 14:16:53,109 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:21:23 2023-11-16 14:16:53,109 INFO 1947952:repeater task 3dayplots_os150nb starting 29.1 seconds late 2023-11-16 14:16:56,472 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:16:56,473 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:21:24 2023-11-16 14:16:56,474 INFO 1947952:repeater task lastensq_os38nb starting 31.5 seconds late 2023-11-16 14:16:58,445 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 14:16:58,446 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:16:20 2023-11-16 14:16:58,446 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:21:25 2023-11-16 14:16:58,446 INFO 1947952:repeater task quick_os38nb starting 33.4 seconds late 2023-11-16 14:17:02,956 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 14:17:02,956 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:16:20 2023-11-16 14:17:02,956 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:21:25 2023-11-16 14:17:02,957 INFO 1947952:repeater task lastfew_vec_os38nb starting 38.0 seconds late 2023-11-16 14:17:03,577 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:17:03,577 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:16:20 2023-11-16 14:17:03,577 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:21:25 2023-11-16 14:17:03,578 INFO 1947952:repeater task 3dayplots_os38nb starting 39.5 seconds late 2023-11-16 14:17:07,025 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 14:17:07,025 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:21:24 2023-11-16 14:17:07,026 INFO 1947952:repeater task daily_netcdf starting 43.0 seconds late 2023-11-16 14:17:07,852 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 14:17:07,853 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 14:26:24 2023-11-16 14:18:23,051 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 14:18:25,807 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 14:18:25,808 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:18:18 2023-11-16 14:18:25,808 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:20:23 2023-11-16 14:18:25,808 INFO 1947952:repeater task quick_wh300 starting 2.8 seconds late 2023-11-16 14:18:32,582 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-16 14:18:32,583 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:18:18 2023-11-16 14:18:32,583 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:20:23 2023-11-16 14:18:32,583 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.6 seconds late 2023-11-16 14:18:33,220 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:18:33,220 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:18:18 2023-11-16 14:18:33,220 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:20:23 2023-11-16 14:20:23,490 INFO 1947952:repeater task lastensq_wh300 starting 0.5 seconds late 2023-11-16 14:20:26,211 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 14:20:26,212 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:20:18 2023-11-16 14:20:26,212 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:22:23 2023-11-16 14:20:26,212 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-16 14:20:33,045 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-16 14:20:33,046 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:20:18 2023-11-16 14:20:33,046 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:22:23 2023-11-16 14:20:33,046 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2023-11-16 14:20:33,669 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:20:33,669 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:20:18 2023-11-16 14:20:33,670 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:22:23 2023-11-16 14:21:23,781 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.7 seconds late 2023-11-16 14:21:24,959 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 14:21:24,959 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:24,960 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.9 seconds late 2023-11-16 14:21:25,127 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 14:21:25,127 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:25,128 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.1 seconds late 2023-11-16 14:21:26,413 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:21:26,414 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:26,414 INFO 1947952:repeater task beamdiagnostics_wh300 starting 7.4 seconds late 2023-11-16 14:21:27,697 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:21:27,698 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:27,698 INFO 1947952:repeater task lastensq_os150nb starting 4.7 seconds late 2023-11-16 14:21:30,193 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 14:21:30,194 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:21:18 2023-11-16 14:21:30,194 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:26:23 2023-11-16 14:21:30,194 INFO 1947952:repeater task quick_os150nb starting 7.2 seconds late 2023-11-16 14:21:33,826 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 14:21:33,827 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:21:18 2023-11-16 14:21:33,827 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:26:23 2023-11-16 14:21:33,828 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.8 seconds late 2023-11-16 14:21:34,450 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:21:34,451 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:21:18 2023-11-16 14:21:34,451 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:26:23 2023-11-16 14:21:34,451 INFO 1947952:repeater task 3dayplots_os150nb starting 10.4 seconds late 2023-11-16 14:21:37,860 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:21:37,861 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:26:24 2023-11-16 14:21:37,861 INFO 1947952:repeater task plot_tsstats_os150nb starting 18.8 seconds late 2023-11-16 14:21:38,834 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 14:21:38,834 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:38,835 INFO 1947952:repeater task grab_lastpings_os150nb starting 19.8 seconds late 2023-11-16 14:21:39,007 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 14:21:39,008 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:39,008 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.0 seconds late 2023-11-16 14:21:40,263 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 14:21:40,263 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:40,263 INFO 1947952:repeater task beamdiagnostics_os150nb starting 21.2 seconds late 2023-11-16 14:21:41,506 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 14:21:41,506 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:41,506 INFO 1947952:repeater task lastensq_os38nb starting 16.5 seconds late 2023-11-16 14:21:43,726 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 14:21:43,726 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:21:20 2023-11-16 14:21:43,726 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:26:25 2023-11-16 14:21:43,726 INFO 1947952:repeater task quick_os38nb starting 18.7 seconds late 2023-11-16 14:21:48,344 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 14:21:48,345 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:21:20 2023-11-16 14:21:48,345 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:26:25 2023-11-16 14:21:48,345 INFO 1947952:repeater task lastfew_vec_os38nb starting 23.3 seconds late 2023-11-16 14:21:48,980 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:21:48,981 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:21:20 2023-11-16 14:21:48,981 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:26:25 2023-11-16 14:21:48,981 INFO 1947952:repeater task 3dayplots_os38nb starting 24.9 seconds late 2023-11-16 14:21:52,421 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 14:21:52,421 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:26:24 2023-11-16 14:21:52,422 INFO 1947952:repeater task plot_tsstats_os38nb starting 33.4 seconds late 2023-11-16 14:21:53,429 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 14:21:53,429 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:53,430 INFO 1947952:repeater task grab_lastpings_os38nb starting 34.4 seconds late 2023-11-16 14:21:53,598 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 14:21:53,598 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:53,599 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.5 seconds late 2023-11-16 14:21:54,885 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 14:21:54,885 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:54,886 INFO 1947952:repeater task beamdiagnostics_os38nb starting 35.8 seconds late 2023-11-16 14:21:56,144 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 14:21:56,144 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:56,146 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 36.9 seconds late 2023-11-16 14:21:57,387 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.2 seconds 2023-11-16 14:21:57,387 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:57,388 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 38.1 seconds late 2023-11-16 14:21:58,653 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 14:21:58,653 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:58,653 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 39.4 seconds late 2023-11-16 14:21:59,910 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 14:21:59,910 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:21:59,911 INFO 1947952:repeater task plot_hbins starting 40.6 seconds late 2023-11-16 14:22:04,100 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 14:22:04,100 INFO 1947952:repeater task plot_hbins ended after 4.2 seconds 2023-11-16 14:22:04,100 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 14:36:19 2023-11-16 14:22:09,107 INFO 1947952:repeater task 3dayplots_wh300 starting 45.1 seconds late 2023-11-16 14:22:12,921 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:22:12,921 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:26:24 2023-11-16 14:22:27,958 INFO 1947952:repeater task lastensq_wh300 starting 5.0 seconds late 2023-11-16 14:22:30,849 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 14:22:30,849 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:22:18 2023-11-16 14:22:30,849 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:24:23 2023-11-16 14:22:30,850 INFO 1947952:repeater task quick_wh300 starting 7.9 seconds late 2023-11-16 14:22:37,450 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 14:22:37,450 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:22:18 2023-11-16 14:22:37,451 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:24:23 2023-11-16 14:22:37,451 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.5 seconds late 2023-11-16 14:22:38,072 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:22:38,072 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:22:18 2023-11-16 14:22:38,072 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:24:23 2023-11-16 14:24:23,346 INFO 1947952:repeater task lastensq_wh300 starting 0.3 seconds late 2023-11-16 14:24:26,052 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 14:24:26,053 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:24:18 2023-11-16 14:24:26,053 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:26:23 2023-11-16 14:24:26,053 INFO 1947952:repeater task quick_wh300 starting 3.1 seconds late 2023-11-16 14:24:33,012 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-16 14:24:33,013 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:24:18 2023-11-16 14:24:33,013 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:26:23 2023-11-16 14:24:33,013 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2023-11-16 14:24:33,638 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:24:33,638 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:24:18 2023-11-16 14:24:33,638 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:26:23 2023-11-16 14:26:23,947 INFO 1947952:repeater task lastensq_wh300 starting 0.9 seconds late 2023-11-16 14:26:26,589 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 14:26:26,589 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:26:18 2023-11-16 14:26:26,589 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:28:23 2023-11-16 14:26:26,590 INFO 1947952:repeater task quick_wh300 starting 3.6 seconds late 2023-11-16 14:26:33,592 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-16 14:26:33,593 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:26:18 2023-11-16 14:26:33,593 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:28:23 2023-11-16 14:26:33,593 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2023-11-16 14:26:34,224 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:26:34,225 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:26:18 2023-11-16 14:26:34,225 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:28:23 2023-11-16 14:26:34,225 INFO 1947952:repeater task 3dayplots_wh300 starting 10.2 seconds late 2023-11-16 14:26:38,065 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:26:38,066 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:31:24 2023-11-16 14:26:38,067 INFO 1947952:repeater task lastensq_os150nb starting 15.1 seconds late 2023-11-16 14:26:40,602 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 14:26:40,602 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:26:18 2023-11-16 14:26:40,602 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:31:23 2023-11-16 14:26:40,603 INFO 1947952:repeater task quick_os150nb starting 17.6 seconds late 2023-11-16 14:26:44,432 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 14:26:44,433 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:26:18 2023-11-16 14:26:44,433 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:31:23 2023-11-16 14:26:44,433 INFO 1947952:repeater task lastfew_vec_os150nb starting 21.4 seconds late 2023-11-16 14:26:45,043 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:26:45,044 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:26:18 2023-11-16 14:26:45,044 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:31:23 2023-11-16 14:26:45,044 INFO 1947952:repeater task 3dayplots_os150nb starting 21.0 seconds late 2023-11-16 14:26:48,399 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:26:48,400 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:31:24 2023-11-16 14:26:48,401 INFO 1947952:repeater task lastensq_os38nb starting 23.4 seconds late 2023-11-16 14:26:50,760 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-16 14:26:50,761 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:26:20 2023-11-16 14:26:50,761 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:31:25 2023-11-16 14:26:50,761 INFO 1947952:repeater task quick_os38nb starting 25.8 seconds late 2023-11-16 14:26:55,615 INFO 1947952:repeater task quick_os38nb ended after 4.9 seconds 2023-11-16 14:26:55,615 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:26:20 2023-11-16 14:26:55,615 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:31:25 2023-11-16 14:26:55,616 INFO 1947952:repeater task lastfew_vec_os38nb starting 30.6 seconds late 2023-11-16 14:26:56,233 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:26:56,234 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:26:20 2023-11-16 14:26:56,234 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:31:25 2023-11-16 14:26:56,234 INFO 1947952:repeater task 3dayplots_os38nb starting 32.2 seconds late 2023-11-16 14:26:59,701 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 14:26:59,701 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:31:24 2023-11-16 14:26:59,703 INFO 1947952:repeater task daily_netcdf starting 35.7 seconds late 2023-11-16 14:27:00,460 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 14:27:00,460 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 14:36:24 2023-11-16 14:27:05,465 INFO 1947952:repeater task noisy_rawtemps_os150 starting 41.4 seconds late 2023-11-16 14:27:08,422 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.0 seconds 2023-11-16 14:27:08,423 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 14:36:24 2023-11-16 14:27:08,423 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 44.4 seconds late 2023-11-16 14:27:11,229 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 14:27:11,229 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 14:36:24 2023-11-16 14:27:11,230 INFO 1947952:repeater task noisy_rawtemps_os38 starting 47.2 seconds late 2023-11-16 14:27:14,289 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 14:27:14,289 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 14:36:24 2023-11-16 14:28:24,480 INFO 1947952:repeater task lastensq_wh300 starting 1.5 seconds late 2023-11-16 14:28:27,555 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 14:28:27,555 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:28:18 2023-11-16 14:28:27,556 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:30:23 2023-11-16 14:28:27,556 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-16 14:28:33,799 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 14:28:33,799 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:28:18 2023-11-16 14:28:33,800 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:30:23 2023-11-16 14:28:33,800 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2023-11-16 14:28:34,424 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:28:34,424 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:28:18 2023-11-16 14:28:34,424 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:30:23 2023-11-16 14:30:24,724 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-16 14:30:27,843 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 14:30:27,844 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:30:18 2023-11-16 14:30:27,844 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:32:23 2023-11-16 14:30:27,844 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 14:30:34,264 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 14:30:34,264 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:30:18 2023-11-16 14:30:34,265 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:32:23 2023-11-16 14:30:34,265 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2023-11-16 14:30:34,895 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:30:34,896 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:30:18 2023-11-16 14:30:34,896 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:32:23 2023-11-16 14:31:25,037 INFO 1947952:repeater task 3dayplots_wh300 starting 1.0 seconds late 2023-11-16 14:31:28,870 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:31:28,870 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:36:24 2023-11-16 14:31:28,872 INFO 1947952:repeater task lastensq_os150nb starting 5.9 seconds late 2023-11-16 14:31:31,343 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 14:31:31,343 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:31:18 2023-11-16 14:31:31,343 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:36:23 2023-11-16 14:31:31,344 INFO 1947952:repeater task quick_os150nb starting 8.3 seconds late 2023-11-16 14:31:34,966 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 14:31:34,967 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:31:18 2023-11-16 14:31:34,967 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:36:23 2023-11-16 14:31:34,967 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.0 seconds late 2023-11-16 14:31:35,579 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:31:35,579 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:31:18 2023-11-16 14:31:35,579 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:36:23 2023-11-16 14:31:35,580 INFO 1947952:repeater task 3dayplots_os150nb starting 11.5 seconds late 2023-11-16 14:31:38,957 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:31:38,957 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:36:24 2023-11-16 14:31:38,958 INFO 1947952:repeater task lastensq_os38nb starting 14.0 seconds late 2023-11-16 14:31:41,201 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 14:31:41,201 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:31:20 2023-11-16 14:31:41,202 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:36:25 2023-11-16 14:31:41,202 INFO 1947952:repeater task quick_os38nb starting 16.2 seconds late 2023-11-16 14:31:45,209 INFO 1947952:repeater task quick_os38nb ended after 4.0 seconds 2023-11-16 14:31:45,209 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:31:20 2023-11-16 14:31:45,209 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:36:25 2023-11-16 14:31:45,210 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.2 seconds late 2023-11-16 14:31:45,813 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:31:45,813 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:31:20 2023-11-16 14:31:45,814 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:36:25 2023-11-16 14:31:45,814 INFO 1947952:repeater task 3dayplots_os38nb starting 21.8 seconds late 2023-11-16 14:31:49,261 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 14:31:49,261 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:36:24 2023-11-16 14:32:24,344 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 14:32:27,009 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 14:32:27,009 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:32:18 2023-11-16 14:32:27,009 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:34:23 2023-11-16 14:32:27,009 INFO 1947952:repeater task quick_wh300 starting 4.0 seconds late 2023-11-16 14:32:33,444 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 14:32:33,444 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:32:18 2023-11-16 14:32:33,444 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:34:23 2023-11-16 14:32:33,444 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-16 14:32:34,071 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:32:34,072 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:32:18 2023-11-16 14:32:34,072 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:34:23 2023-11-16 14:34:24,355 INFO 1947952:repeater task lastensq_wh300 starting 1.4 seconds late 2023-11-16 14:34:27,420 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 14:34:27,421 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:34:18 2023-11-16 14:34:27,421 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:36:23 2023-11-16 14:34:27,421 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-16 14:34:33,769 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 14:34:33,769 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:34:18 2023-11-16 14:34:33,769 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:36:23 2023-11-16 14:34:33,770 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2023-11-16 14:34:34,396 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:34:34,396 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:34:18 2023-11-16 14:34:34,396 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:36:23 2023-11-16 14:36:19,681 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.6 seconds late 2023-11-16 14:36:20,848 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 14:36:20,848 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:20,848 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.8 seconds late 2023-11-16 14:36:21,014 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 14:36:21,015 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:21,015 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.0 seconds late 2023-11-16 14:36:22,293 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:36:22,293 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:22,294 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.2 seconds late 2023-11-16 14:36:23,601 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:36:23,602 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:23,602 INFO 1947952:repeater task lastensq_os150nb starting 0.6 seconds late 2023-11-16 14:36:25,722 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 14:36:25,722 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:36:18 2023-11-16 14:36:25,722 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:41:23 2023-11-16 14:36:25,723 INFO 1947952:repeater task quick_os150nb starting 2.7 seconds late 2023-11-16 14:36:29,474 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 14:36:29,474 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:36:18 2023-11-16 14:36:29,475 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:41:23 2023-11-16 14:36:29,475 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.5 seconds late 2023-11-16 14:36:30,080 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:36:30,081 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:36:18 2023-11-16 14:36:30,081 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:41:23 2023-11-16 14:36:30,081 INFO 1947952:repeater task 3dayplots_os150nb starting 6.0 seconds late 2023-11-16 14:36:33,478 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:36:33,478 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:41:24 2023-11-16 14:36:33,478 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.4 seconds late 2023-11-16 14:36:34,454 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 14:36:34,454 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:34,455 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.4 seconds late 2023-11-16 14:36:34,624 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 14:36:34,624 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:34,624 INFO 1947952:repeater task beamdiagnostics_os150nb starting 15.6 seconds late 2023-11-16 14:36:35,852 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 14:36:35,852 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:35,852 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.8 seconds late 2023-11-16 14:36:37,095 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 14:36:37,095 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:37,095 INFO 1947952:repeater task lastensq_os38nb starting 12.1 seconds late 2023-11-16 14:36:39,189 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 14:36:39,191 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:36:20 2023-11-16 14:36:39,191 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:41:25 2023-11-16 14:36:39,191 INFO 1947952:repeater task quick_os38nb starting 14.2 seconds late 2023-11-16 14:36:42,904 INFO 1947952:repeater task quick_os38nb ended after 3.7 seconds 2023-11-16 14:36:42,904 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:36:20 2023-11-16 14:36:42,904 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:41:25 2023-11-16 14:36:42,905 INFO 1947952:repeater task lastfew_vec_os38nb starting 17.9 seconds late 2023-11-16 14:36:43,515 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:36:43,515 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:36:20 2023-11-16 14:36:43,515 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:41:25 2023-11-16 14:36:43,515 INFO 1947952:repeater task 3dayplots_os38nb starting 19.5 seconds late 2023-11-16 14:36:46,979 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 14:36:46,980 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:41:24 2023-11-16 14:36:46,980 INFO 1947952:repeater task plot_tsstats_os38nb starting 27.9 seconds late 2023-11-16 14:36:47,998 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 14:36:47,998 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:47,998 INFO 1947952:repeater task grab_lastpings_os38nb starting 28.9 seconds late 2023-11-16 14:36:48,168 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 14:36:48,168 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:48,168 INFO 1947952:repeater task beamdiagnostics_os38nb starting 29.1 seconds late 2023-11-16 14:36:49,410 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 14:36:49,410 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:49,410 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.4 seconds late 2023-11-16 14:36:50,643 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 14:36:50,644 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:36:50,644 INFO 1947952:repeater task daily_netcdf starting 26.6 seconds late 2023-11-16 14:36:51,488 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 14:36:51,489 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 14:46:24 2023-11-16 14:36:51,489 INFO 1947952:repeater task ggatime_diagnostics starting 32.4 seconds late 2023-11-16 14:37:08,146 INFO 1947952:repeater task ggatime_diagnostics ended after 16.7 seconds 2023-11-16 14:37:08,146 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:37:08,146 INFO 1947952:repeater task ggatime_diagnostics starting 49.1 seconds late 2023-11-16 14:37:23,886 INFO 1947952:repeater task ggatime_diagnostics ended after 15.7 seconds 2023-11-16 14:37:23,887 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:37:23,887 INFO 1947952:repeater task ggatime_diagnostics starting 64.8 seconds late 2023-11-16 14:37:40,091 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-16 14:37:40,091 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:37:40,092 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 80.8 seconds late 2023-11-16 14:37:41,328 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.2 seconds 2023-11-16 14:37:41,328 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:37:41,328 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 82.1 seconds late 2023-11-16 14:37:42,606 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 14:37:42,607 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:37:42,607 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 83.3 seconds late 2023-11-16 14:37:43,850 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 14:37:43,851 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:37:43,851 INFO 1947952:repeater task plot_hbins starting 84.6 seconds late 2023-11-16 14:37:48,179 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 14:37:48,179 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 14:37:48,179 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 14:51:19 2023-11-16 14:37:48,180 INFO 1947952:repeater task report_generator starting 88.9 seconds late 2023-11-16 14:37:54,125 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 14:37:54,125 INFO 1947952:repeater task report_generator ended after 5.9 seconds 2023-11-16 14:37:54,125 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 15:36:19 2023-11-16 14:37:59,129 INFO 1947952:repeater task noisy_rawtemps_os150 starting 95.1 seconds late 2023-11-16 14:38:02,007 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 14:38:02,007 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 14:46:24 2023-11-16 14:38:02,008 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 98.0 seconds late 2023-11-16 14:38:04,793 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 14:38:04,793 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 14:46:24 2023-11-16 14:38:04,793 INFO 1947952:repeater task noisy_rawtemps_os38 starting 100.7 seconds late 2023-11-16 14:38:07,829 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 14:38:07,829 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 14:46:24 2023-11-16 14:38:07,829 INFO 1947952:repeater task lastensq_wh300 starting 104.8 seconds late 2023-11-16 14:38:10,450 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 14:38:10,451 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:36:18 2023-11-16 14:38:10,451 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:38:23 2023-11-16 14:38:10,451 INFO 1947952:repeater task quick_wh300 starting 107.5 seconds late 2023-11-16 14:38:16,687 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 14:38:16,687 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:36:18 2023-11-16 14:38:16,687 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:38:23 2023-11-16 14:38:16,688 INFO 1947952:repeater task lastfew_vec_wh300 starting 113.7 seconds late 2023-11-16 14:38:17,321 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:38:17,321 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:36:18 2023-11-16 14:38:17,321 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:38:23 2023-11-16 14:38:17,322 INFO 1947952:repeater task 3dayplots_wh300 starting 113.3 seconds late 2023-11-16 14:38:21,071 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 14:38:21,072 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:41:24 2023-11-16 14:38:26,078 INFO 1947952:repeater task lastensq_wh300 starting 3.1 seconds late 2023-11-16 14:38:29,326 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-16 14:38:29,327 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:38:18 2023-11-16 14:38:29,327 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:40:23 2023-11-16 14:38:29,327 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 14:38:35,600 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 14:38:35,600 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:38:18 2023-11-16 14:38:35,600 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:40:23 2023-11-16 14:38:35,601 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2023-11-16 14:38:36,232 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:38:36,233 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:38:18 2023-11-16 14:38:36,233 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:40:23 2023-11-16 14:40:26,527 INFO 1947952:repeater task lastensq_wh300 starting 3.5 seconds late 2023-11-16 14:40:29,540 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 14:40:29,540 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:40:18 2023-11-16 14:40:29,540 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:42:23 2023-11-16 14:40:29,541 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-16 14:40:35,546 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 14:40:35,546 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:40:18 2023-11-16 14:40:35,546 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:42:23 2023-11-16 14:40:35,547 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.5 seconds late 2023-11-16 14:40:36,164 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:40:36,164 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:40:18 2023-11-16 14:40:36,165 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:42:23 2023-11-16 14:41:26,289 INFO 1947952:repeater task 3dayplots_wh300 starting 2.2 seconds late 2023-11-16 14:41:30,099 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:41:30,100 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:46:24 2023-11-16 14:41:30,101 INFO 1947952:repeater task lastensq_os150nb starting 7.1 seconds late 2023-11-16 14:41:32,443 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 14:41:32,444 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:41:18 2023-11-16 14:41:32,444 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:46:23 2023-11-16 14:41:32,444 INFO 1947952:repeater task quick_os150nb starting 9.4 seconds late 2023-11-16 14:41:35,902 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 14:41:35,902 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:41:18 2023-11-16 14:41:35,902 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:46:23 2023-11-16 14:41:35,902 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.9 seconds late 2023-11-16 14:41:36,513 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:41:36,513 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:41:18 2023-11-16 14:41:36,513 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:46:23 2023-11-16 14:41:36,514 INFO 1947952:repeater task 3dayplots_os150nb starting 12.5 seconds late 2023-11-16 14:41:39,931 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:41:39,932 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:46:24 2023-11-16 14:41:39,933 INFO 1947952:repeater task lastensq_os38nb starting 14.9 seconds late 2023-11-16 14:41:42,046 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 14:41:42,047 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:41:20 2023-11-16 14:41:42,047 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:46:25 2023-11-16 14:41:42,047 INFO 1947952:repeater task quick_os38nb starting 17.0 seconds late 2023-11-16 14:41:46,562 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 14:41:46,562 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:41:20 2023-11-16 14:41:46,563 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:46:25 2023-11-16 14:41:46,563 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.6 seconds late 2023-11-16 14:41:47,177 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:41:47,178 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:41:20 2023-11-16 14:41:47,178 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:46:25 2023-11-16 14:41:47,178 INFO 1947952:repeater task 3dayplots_os38nb starting 23.1 seconds late 2023-11-16 14:41:50,542 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 14:41:50,543 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:46:24 2023-11-16 14:42:25,611 INFO 1947952:repeater task lastensq_wh300 starting 2.6 seconds late 2023-11-16 14:42:28,415 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 14:42:28,416 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:42:18 2023-11-16 14:42:28,416 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:44:23 2023-11-16 14:42:28,416 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-16 14:42:35,075 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 14:42:35,076 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:42:18 2023-11-16 14:42:35,076 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:44:23 2023-11-16 14:42:35,076 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2023-11-16 14:42:35,712 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:42:35,712 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:42:18 2023-11-16 14:42:35,713 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:44:23 2023-11-16 14:44:26,008 INFO 1947952:repeater task lastensq_wh300 starting 3.0 seconds late 2023-11-16 14:44:28,970 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 14:44:28,970 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:44:18 2023-11-16 14:44:28,970 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:46:23 2023-11-16 14:44:28,971 INFO 1947952:repeater task quick_wh300 starting 6.0 seconds late 2023-11-16 14:44:35,896 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-16 14:44:35,896 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:44:18 2023-11-16 14:44:35,897 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:46:23 2023-11-16 14:44:35,897 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 14:44:36,534 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:44:36,534 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:44:18 2023-11-16 14:44:36,534 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:46:23 2023-11-16 14:46:26,833 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.8 seconds late 2023-11-16 14:46:30,189 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.4 seconds 2023-11-16 14:46:30,189 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 14:56:24 2023-11-16 14:46:30,190 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.1 seconds late 2023-11-16 14:46:32,928 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-16 14:46:32,928 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 14:56:24 2023-11-16 14:46:32,928 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.9 seconds late 2023-11-16 14:46:36,038 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 14:46:36,038 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 14:56:24 2023-11-16 14:46:36,039 INFO 1947952:repeater task lastensq_wh300 starting 13.0 seconds late 2023-11-16 14:46:38,889 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 14:46:38,890 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:46:18 2023-11-16 14:46:38,890 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:48:23 2023-11-16 14:46:38,890 INFO 1947952:repeater task quick_wh300 starting 15.9 seconds late 2023-11-16 14:46:45,478 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 14:46:45,478 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:46:18 2023-11-16 14:46:45,478 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:48:23 2023-11-16 14:46:45,479 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.5 seconds late 2023-11-16 14:46:46,099 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:46:46,099 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:46:18 2023-11-16 14:46:46,100 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:48:23 2023-11-16 14:46:46,100 INFO 1947952:repeater task 3dayplots_wh300 starting 22.0 seconds late 2023-11-16 14:46:49,926 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:46:49,926 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:51:24 2023-11-16 14:46:49,928 INFO 1947952:repeater task lastensq_os150nb starting 26.9 seconds late 2023-11-16 14:46:52,025 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 14:46:52,025 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:46:18 2023-11-16 14:46:52,025 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:51:23 2023-11-16 14:46:52,026 INFO 1947952:repeater task quick_os150nb starting 29.0 seconds late 2023-11-16 14:46:56,187 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-16 14:46:56,187 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:46:18 2023-11-16 14:46:56,187 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:51:23 2023-11-16 14:46:56,188 INFO 1947952:repeater task lastfew_vec_os150nb starting 33.2 seconds late 2023-11-16 14:46:56,801 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:46:56,801 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:46:18 2023-11-16 14:46:56,801 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:51:23 2023-11-16 14:46:56,802 INFO 1947952:repeater task 3dayplots_os150nb starting 32.8 seconds late 2023-11-16 14:47:00,229 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:47:00,229 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:51:24 2023-11-16 14:47:00,230 INFO 1947952:repeater task lastensq_os38nb starting 35.2 seconds late 2023-11-16 14:47:02,989 INFO 1947952:repeater task lastensq_os38nb ended after 2.8 seconds 2023-11-16 14:47:02,990 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:46:20 2023-11-16 14:47:02,990 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:51:25 2023-11-16 14:47:02,990 INFO 1947952:repeater task quick_os38nb starting 38.0 seconds late 2023-11-16 14:47:08,360 INFO 1947952:repeater task quick_os38nb ended after 5.4 seconds 2023-11-16 14:47:08,361 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:46:20 2023-11-16 14:47:08,361 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:51:25 2023-11-16 14:47:08,362 INFO 1947952:repeater task lastfew_vec_os38nb starting 43.4 seconds late 2023-11-16 14:47:09,000 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:47:09,001 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:46:20 2023-11-16 14:47:09,001 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:51:25 2023-11-16 14:47:09,001 INFO 1947952:repeater task 3dayplots_os38nb starting 44.9 seconds late 2023-11-16 14:47:12,469 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 14:47:12,470 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:51:24 2023-11-16 14:47:12,471 INFO 1947952:repeater task daily_netcdf starting 48.4 seconds late 2023-11-16 14:47:13,564 INFO 1947952:repeater task daily_netcdf ended after 1.1 seconds 2023-11-16 14:47:13,565 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 14:56:24 2023-11-16 14:48:23,735 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-16 14:48:26,784 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 14:48:26,785 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:48:18 2023-11-16 14:48:26,785 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:50:23 2023-11-16 14:48:26,785 INFO 1947952:repeater task quick_wh300 starting 3.8 seconds late 2023-11-16 14:48:33,338 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 14:48:33,338 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:48:18 2023-11-16 14:48:33,338 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:50:23 2023-11-16 14:48:33,338 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-16 14:48:33,968 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:48:33,969 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:48:18 2023-11-16 14:48:33,969 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:50:23 2023-11-16 14:50:24,252 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 14:50:26,993 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 14:50:26,993 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:50:18 2023-11-16 14:50:26,993 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:52:23 2023-11-16 14:50:26,994 INFO 1947952:repeater task quick_wh300 starting 4.0 seconds late 2023-11-16 14:50:34,015 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-16 14:50:34,015 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:50:18 2023-11-16 14:50:34,015 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:52:23 2023-11-16 14:50:34,015 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 14:50:34,644 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:50:34,645 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:50:18 2023-11-16 14:50:34,645 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:52:23 2023-11-16 14:51:19,763 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.7 seconds late 2023-11-16 14:51:20,932 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 14:51:20,932 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:20,932 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.9 seconds late 2023-11-16 14:51:21,101 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 14:51:21,101 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:21,102 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.1 seconds late 2023-11-16 14:51:22,381 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:51:22,381 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:22,381 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.3 seconds late 2023-11-16 14:51:23,655 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 14:51:23,655 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:23,655 INFO 1947952:repeater task lastensq_os150nb starting 0.7 seconds late 2023-11-16 14:51:25,801 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 14:51:25,802 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:51:18 2023-11-16 14:51:25,802 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 14:56:23 2023-11-16 14:51:25,802 INFO 1947952:repeater task quick_os150nb starting 2.8 seconds late 2023-11-16 14:51:29,815 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-16 14:51:29,815 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:51:18 2023-11-16 14:51:29,816 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 14:56:23 2023-11-16 14:51:29,816 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.8 seconds late 2023-11-16 14:51:30,413 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:51:30,414 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:51:18 2023-11-16 14:51:30,414 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 14:56:23 2023-11-16 14:51:30,414 INFO 1947952:repeater task 3dayplots_os150nb starting 6.4 seconds late 2023-11-16 14:51:33,765 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:51:33,765 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 14:56:24 2023-11-16 14:51:33,766 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.7 seconds late 2023-11-16 14:51:34,740 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 14:51:34,740 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:34,740 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.7 seconds late 2023-11-16 14:51:34,906 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 14:51:34,906 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:34,906 INFO 1947952:repeater task beamdiagnostics_os150nb starting 15.9 seconds late 2023-11-16 14:51:36,137 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 14:51:36,138 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:36,138 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.1 seconds late 2023-11-16 14:51:37,400 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 14:51:37,400 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:37,401 INFO 1947952:repeater task lastensq_os38nb starting 12.4 seconds late 2023-11-16 14:51:39,509 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 14:51:39,510 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:51:20 2023-11-16 14:51:39,510 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 14:56:25 2023-11-16 14:51:39,510 INFO 1947952:repeater task quick_os38nb starting 14.5 seconds late 2023-11-16 14:51:43,969 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 14:51:43,969 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:51:20 2023-11-16 14:51:43,969 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 14:56:25 2023-11-16 14:51:43,970 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.0 seconds late 2023-11-16 14:51:44,576 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:51:44,576 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:51:20 2023-11-16 14:51:44,576 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 14:56:25 2023-11-16 14:51:44,576 INFO 1947952:repeater task 3dayplots_os38nb starting 20.5 seconds late 2023-11-16 14:51:48,042 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 14:51:48,042 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 14:56:24 2023-11-16 14:51:48,042 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.0 seconds late 2023-11-16 14:51:49,050 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 14:51:49,050 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:49,051 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.0 seconds late 2023-11-16 14:51:49,220 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 14:51:49,220 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:49,221 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.2 seconds late 2023-11-16 14:51:50,432 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 14:51:50,432 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:50,432 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.4 seconds late 2023-11-16 14:51:51,660 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 14:51:51,660 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:51,661 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 32.4 seconds late 2023-11-16 14:51:52,899 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.2 seconds 2023-11-16 14:51:52,899 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:52,900 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 33.6 seconds late 2023-11-16 14:51:54,165 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 14:51:54,165 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:54,166 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 34.9 seconds late 2023-11-16 14:51:55,405 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 14:51:55,405 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:51:55,405 INFO 1947952:repeater task plot_hbins starting 36.1 seconds late 2023-11-16 14:51:59,734 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 14:51:59,734 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 14:51:59,734 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 15:06:19 2023-11-16 14:52:04,743 INFO 1947952:repeater task 3dayplots_wh300 starting 40.7 seconds late 2023-11-16 14:52:08,716 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-16 14:52:08,716 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 14:56:24 2023-11-16 14:52:23,746 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-16 14:52:26,651 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 14:52:26,652 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:52:18 2023-11-16 14:52:26,652 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:54:23 2023-11-16 14:52:26,652 INFO 1947952:repeater task quick_wh300 starting 3.7 seconds late 2023-11-16 14:52:33,335 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 14:52:33,335 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:52:18 2023-11-16 14:52:33,335 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:54:23 2023-11-16 14:52:33,336 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-16 14:52:33,965 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:52:33,965 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:52:18 2023-11-16 14:52:33,966 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:54:23 2023-11-16 14:54:24,240 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-16 14:54:26,975 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 14:54:26,976 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:54:18 2023-11-16 14:54:26,976 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:56:23 2023-11-16 14:54:26,976 INFO 1947952:repeater task quick_wh300 starting 4.0 seconds late 2023-11-16 14:54:33,768 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-16 14:54:33,769 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:54:18 2023-11-16 14:54:33,769 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:56:23 2023-11-16 14:54:33,769 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2023-11-16 14:54:34,399 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:54:34,399 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:54:18 2023-11-16 14:54:34,399 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:56:23 2023-11-16 14:56:24,713 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.7 seconds late 2023-11-16 14:56:27,842 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2023-11-16 14:56:27,842 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 15:06:24 2023-11-16 14:56:27,843 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.8 seconds late 2023-11-16 14:56:30,903 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 3.1 seconds 2023-11-16 14:56:30,903 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 15:06:24 2023-11-16 14:56:30,903 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.9 seconds late 2023-11-16 14:56:33,933 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 14:56:33,933 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 15:06:24 2023-11-16 14:56:33,934 INFO 1947952:repeater task lastensq_wh300 starting 10.9 seconds late 2023-11-16 14:56:37,049 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 14:56:37,049 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:56:18 2023-11-16 14:56:37,049 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 14:58:23 2023-11-16 14:56:37,050 INFO 1947952:repeater task quick_wh300 starting 14.1 seconds late 2023-11-16 14:56:43,730 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 14:56:43,731 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:56:18 2023-11-16 14:56:43,731 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 14:58:23 2023-11-16 14:56:43,731 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.7 seconds late 2023-11-16 14:56:44,365 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:56:44,365 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:56:18 2023-11-16 14:56:44,365 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 14:58:23 2023-11-16 14:56:44,366 INFO 1947952:repeater task 3dayplots_wh300 starting 20.3 seconds late 2023-11-16 14:56:48,206 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 14:56:48,206 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:01:24 2023-11-16 14:56:48,208 INFO 1947952:repeater task lastensq_os150nb starting 25.2 seconds late 2023-11-16 14:56:50,684 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 14:56:50,684 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:56:18 2023-11-16 14:56:50,686 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:01:23 2023-11-16 14:56:50,687 INFO 1947952:repeater task quick_os150nb starting 27.7 seconds late 2023-11-16 14:56:54,578 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 14:56:54,579 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:56:18 2023-11-16 14:56:54,579 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:01:23 2023-11-16 14:56:54,579 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.6 seconds late 2023-11-16 14:56:55,186 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 14:56:55,186 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 14:56:18 2023-11-16 14:56:55,186 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:01:23 2023-11-16 14:56:55,186 INFO 1947952:repeater task 3dayplots_os150nb starting 31.1 seconds late 2023-11-16 14:56:58,568 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 14:56:58,568 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:01:24 2023-11-16 14:56:58,569 INFO 1947952:repeater task lastensq_os38nb starting 33.6 seconds late 2023-11-16 14:57:00,778 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 14:57:00,779 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:56:20 2023-11-16 14:57:00,779 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:01:25 2023-11-16 14:57:00,779 INFO 1947952:repeater task quick_os38nb starting 35.8 seconds late 2023-11-16 14:57:05,165 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 14:57:05,165 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:56:20 2023-11-16 14:57:05,165 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:01:25 2023-11-16 14:57:05,166 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.2 seconds late 2023-11-16 14:57:05,771 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 14:57:05,771 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 14:56:20 2023-11-16 14:57:05,771 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:01:25 2023-11-16 14:57:05,772 INFO 1947952:repeater task 3dayplots_os38nb starting 41.7 seconds late 2023-11-16 14:57:09,221 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 14:57:09,221 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:01:24 2023-11-16 14:57:09,222 INFO 1947952:repeater task daily_netcdf starting 45.2 seconds late 2023-11-16 14:57:10,034 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 14:57:10,035 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 15:06:24 2023-11-16 14:58:25,237 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-16 14:58:28,036 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 14:58:28,037 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:58:18 2023-11-16 14:58:28,037 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:00:23 2023-11-16 14:58:28,037 INFO 1947952:repeater task quick_wh300 starting 5.0 seconds late 2023-11-16 14:58:34,370 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 14:58:34,370 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:58:18 2023-11-16 14:58:34,370 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:00:23 2023-11-16 14:58:34,371 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-16 14:58:35,000 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 14:58:35,000 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 14:58:18 2023-11-16 14:58:35,000 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:00:23 2023-11-16 15:00:25,296 INFO 1947952:repeater task lastensq_wh300 starting 2.3 seconds late 2023-11-16 15:00:28,211 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:00:28,211 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:00:18 2023-11-16 15:00:28,211 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:02:23 2023-11-16 15:00:28,212 INFO 1947952:repeater task quick_wh300 starting 5.2 seconds late 2023-11-16 15:00:34,833 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 15:00:34,834 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:00:18 2023-11-16 15:00:34,834 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:02:23 2023-11-16 15:00:34,834 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.8 seconds late 2023-11-16 15:00:35,468 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:00:35,468 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:00:18 2023-11-16 15:00:35,468 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:02:23 2023-11-16 15:01:25,593 INFO 1947952:repeater task 3dayplots_wh300 starting 1.5 seconds late 2023-11-16 15:01:29,434 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 15:01:29,434 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:06:24 2023-11-16 15:01:29,435 INFO 1947952:repeater task lastensq_os150nb starting 6.4 seconds late 2023-11-16 15:01:31,813 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 15:01:31,814 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:01:18 2023-11-16 15:01:31,814 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:06:23 2023-11-16 15:01:31,814 INFO 1947952:repeater task quick_os150nb starting 8.8 seconds late 2023-11-16 15:01:35,338 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 15:01:35,339 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:01:18 2023-11-16 15:01:35,339 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:06:23 2023-11-16 15:01:35,339 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.3 seconds late 2023-11-16 15:01:35,948 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:01:35,949 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:01:18 2023-11-16 15:01:35,949 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:06:23 2023-11-16 15:01:35,949 INFO 1947952:repeater task 3dayplots_os150nb starting 11.9 seconds late 2023-11-16 15:01:39,350 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:01:39,350 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:06:24 2023-11-16 15:01:39,351 INFO 1947952:repeater task lastensq_os38nb starting 14.4 seconds late 2023-11-16 15:01:41,416 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 15:01:41,416 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:01:20 2023-11-16 15:01:41,416 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:06:25 2023-11-16 15:01:41,417 INFO 1947952:repeater task quick_os38nb starting 16.4 seconds late 2023-11-16 15:01:45,764 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 15:01:45,765 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:01:20 2023-11-16 15:01:45,765 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:06:25 2023-11-16 15:01:45,766 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.8 seconds late 2023-11-16 15:01:46,367 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:01:46,368 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:01:20 2023-11-16 15:01:46,368 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:06:25 2023-11-16 15:01:46,368 INFO 1947952:repeater task 3dayplots_os38nb starting 22.3 seconds late 2023-11-16 15:01:49,779 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 15:01:49,779 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:06:24 2023-11-16 15:02:24,859 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 15:02:27,551 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 15:02:27,551 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:02:18 2023-11-16 15:02:27,551 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:04:23 2023-11-16 15:02:27,552 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-16 15:02:34,017 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 15:02:34,017 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:02:18 2023-11-16 15:02:34,017 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:04:23 2023-11-16 15:02:34,018 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 15:02:34,657 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:02:34,658 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:02:18 2023-11-16 15:02:34,658 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:04:23 2023-11-16 15:04:24,941 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 15:04:27,776 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 15:04:27,776 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:04:18 2023-11-16 15:04:27,777 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:06:23 2023-11-16 15:04:27,777 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 15:04:34,202 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:04:34,203 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:04:18 2023-11-16 15:04:34,203 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:06:23 2023-11-16 15:04:34,203 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.2 seconds late 2023-11-16 15:04:34,840 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:04:34,840 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:04:18 2023-11-16 15:04:34,840 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:06:23 2023-11-16 15:06:20,133 INFO 1947952:repeater task plot_tsstats_wh300 starting 1.1 seconds late 2023-11-16 15:06:21,309 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 15:06:21,310 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:21,310 INFO 1947952:repeater task grab_lastpings_wh300 starting 2.3 seconds late 2023-11-16 15:06:21,482 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 15:06:21,482 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:21,482 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.4 seconds late 2023-11-16 15:06:22,801 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:06:22,802 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:22,802 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.8 seconds late 2023-11-16 15:06:24,111 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:06:24,112 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:24,112 INFO 1947952:repeater task lastensq_os150nb starting 1.1 seconds late 2023-11-16 15:06:26,583 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 15:06:26,584 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:06:18 2023-11-16 15:06:26,584 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:11:23 2023-11-16 15:06:26,584 INFO 1947952:repeater task quick_os150nb starting 3.6 seconds late 2023-11-16 15:06:30,550 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-16 15:06:30,551 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:06:18 2023-11-16 15:06:30,551 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:11:23 2023-11-16 15:06:30,551 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.6 seconds late 2023-11-16 15:06:31,229 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.7 seconds 2023-11-16 15:06:31,229 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:06:18 2023-11-16 15:06:31,229 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:11:23 2023-11-16 15:06:31,230 INFO 1947952:repeater task 3dayplots_os150nb starting 7.2 seconds late 2023-11-16 15:06:34,624 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:06:34,624 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:11:24 2023-11-16 15:06:34,625 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.6 seconds late 2023-11-16 15:06:35,604 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 15:06:35,604 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:35,604 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.6 seconds late 2023-11-16 15:06:35,773 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 15:06:35,773 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:35,773 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.7 seconds late 2023-11-16 15:06:37,051 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 15:06:37,051 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:37,051 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.0 seconds late 2023-11-16 15:06:38,299 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 15:06:38,299 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:38,300 INFO 1947952:repeater task lastensq_os38nb starting 13.3 seconds late 2023-11-16 15:06:40,413 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 15:06:40,414 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:06:20 2023-11-16 15:06:40,414 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:11:25 2023-11-16 15:06:40,414 INFO 1947952:repeater task quick_os38nb starting 15.4 seconds late 2023-11-16 15:06:44,792 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 15:06:44,792 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:06:20 2023-11-16 15:06:44,793 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:11:25 2023-11-16 15:06:44,793 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.8 seconds late 2023-11-16 15:06:45,399 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:06:45,399 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:06:20 2023-11-16 15:06:45,399 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:11:25 2023-11-16 15:06:45,399 INFO 1947952:repeater task 3dayplots_os38nb starting 21.3 seconds late 2023-11-16 15:06:48,898 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 15:06:48,898 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:11:24 2023-11-16 15:06:48,898 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.8 seconds late 2023-11-16 15:06:49,904 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 15:06:49,904 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:49,905 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.9 seconds late 2023-11-16 15:06:50,073 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 15:06:50,073 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:50,073 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.0 seconds late 2023-11-16 15:06:51,325 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 15:06:51,325 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:51,325 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.3 seconds late 2023-11-16 15:06:52,560 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 15:06:52,560 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:06:52,561 INFO 1947952:repeater task daily_netcdf starting 28.5 seconds late 2023-11-16 15:06:53,359 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 15:06:53,359 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 15:16:24 2023-11-16 15:06:53,360 INFO 1947952:repeater task ggatime_diagnostics starting 34.3 seconds late 2023-11-16 15:07:09,790 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-16 15:07:09,790 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:07:09,790 INFO 1947952:repeater task ggatime_diagnostics starting 50.7 seconds late 2023-11-16 15:07:25,958 INFO 1947952:repeater task ggatime_diagnostics ended after 16.2 seconds 2023-11-16 15:07:25,959 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:07:25,959 INFO 1947952:repeater task ggatime_diagnostics starting 66.9 seconds late 2023-11-16 15:07:41,690 INFO 1947952:repeater task ggatime_diagnostics ended after 15.7 seconds 2023-11-16 15:07:41,691 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:07:41,691 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 82.4 seconds late 2023-11-16 15:07:43,012 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 15:07:43,012 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:07:43,013 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 83.7 seconds late 2023-11-16 15:07:44,286 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 15:07:44,286 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:07:44,287 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 85.0 seconds late 2023-11-16 15:07:45,509 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 15:07:45,509 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:07:45,510 INFO 1947952:repeater task plot_hbins starting 86.2 seconds late 2023-11-16 15:07:49,866 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 15:07:49,866 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 15:07:49,866 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 15:21:19 2023-11-16 15:07:54,869 INFO 1947952:repeater task noisy_rawtemps_os150 starting 90.8 seconds late 2023-11-16 15:07:57,800 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 15:07:57,800 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 15:16:24 2023-11-16 15:07:57,801 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 93.8 seconds late 2023-11-16 15:08:00,612 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 15:08:00,612 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 15:16:24 2023-11-16 15:08:00,613 INFO 1947952:repeater task noisy_rawtemps_os38 starting 96.6 seconds late 2023-11-16 15:08:03,821 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.2 seconds 2023-11-16 15:08:03,822 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 15:16:24 2023-11-16 15:08:03,822 INFO 1947952:repeater task lastensq_wh300 starting 100.8 seconds late 2023-11-16 15:08:06,692 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:08:06,693 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:06:18 2023-11-16 15:08:06,693 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:08:23 2023-11-16 15:08:06,693 INFO 1947952:repeater task quick_wh300 starting 103.7 seconds late 2023-11-16 15:08:12,723 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 15:08:12,724 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:06:18 2023-11-16 15:08:12,724 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:08:23 2023-11-16 15:08:12,724 INFO 1947952:repeater task lastfew_vec_wh300 starting 109.7 seconds late 2023-11-16 15:08:13,349 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:08:13,349 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:06:18 2023-11-16 15:08:13,349 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:08:23 2023-11-16 15:08:13,350 INFO 1947952:repeater task 3dayplots_wh300 starting 109.3 seconds late 2023-11-16 15:08:17,099 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 15:08:17,099 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:11:24 2023-11-16 15:08:27,118 INFO 1947952:repeater task lastensq_wh300 starting 4.1 seconds late 2023-11-16 15:08:29,972 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:08:29,972 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:08:18 2023-11-16 15:08:29,972 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:10:23 2023-11-16 15:08:29,972 INFO 1947952:repeater task quick_wh300 starting 7.0 seconds late 2023-11-16 15:08:36,344 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:08:36,344 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:08:18 2023-11-16 15:08:36,344 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:10:23 2023-11-16 15:08:36,344 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.3 seconds late 2023-11-16 15:08:36,973 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:08:36,973 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:08:18 2023-11-16 15:08:36,973 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:10:23 2023-11-16 15:10:27,250 INFO 1947952:repeater task lastensq_wh300 starting 4.3 seconds late 2023-11-16 15:10:30,111 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:10:30,112 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:10:18 2023-11-16 15:10:30,112 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:12:23 2023-11-16 15:10:30,112 INFO 1947952:repeater task quick_wh300 starting 7.1 seconds late 2023-11-16 15:10:36,330 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 15:10:36,331 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:10:18 2023-11-16 15:10:36,331 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:12:23 2023-11-16 15:10:36,332 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.3 seconds late 2023-11-16 15:10:36,961 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:10:36,962 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:10:18 2023-11-16 15:10:36,962 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:12:23 2023-11-16 15:11:27,076 INFO 1947952:repeater task 3dayplots_wh300 starting 3.0 seconds late 2023-11-16 15:11:30,928 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 15:11:30,928 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:16:24 2023-11-16 15:11:30,929 INFO 1947952:repeater task lastensq_os150nb starting 7.9 seconds late 2023-11-16 15:11:33,292 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 15:11:33,292 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:11:18 2023-11-16 15:11:33,292 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:16:23 2023-11-16 15:11:33,292 INFO 1947952:repeater task quick_os150nb starting 10.3 seconds late 2023-11-16 15:11:37,008 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 15:11:37,008 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:11:18 2023-11-16 15:11:37,008 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:16:23 2023-11-16 15:11:37,009 INFO 1947952:repeater task lastfew_vec_os150nb starting 14.0 seconds late 2023-11-16 15:11:37,619 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:11:37,619 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:11:18 2023-11-16 15:11:37,619 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:16:23 2023-11-16 15:11:37,620 INFO 1947952:repeater task 3dayplots_os150nb starting 13.6 seconds late 2023-11-16 15:11:41,032 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:11:41,032 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:16:24 2023-11-16 15:11:41,033 INFO 1947952:repeater task lastensq_os38nb starting 16.0 seconds late 2023-11-16 15:11:43,109 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 15:11:43,110 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:11:20 2023-11-16 15:11:43,110 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:16:25 2023-11-16 15:11:43,111 INFO 1947952:repeater task quick_os38nb starting 18.1 seconds late 2023-11-16 15:11:47,335 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 15:11:47,335 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:11:20 2023-11-16 15:11:47,335 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:16:25 2023-11-16 15:11:47,336 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.3 seconds late 2023-11-16 15:11:47,951 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:11:47,951 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:11:20 2023-11-16 15:11:47,951 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:16:25 2023-11-16 15:11:47,952 INFO 1947952:repeater task 3dayplots_os38nb starting 23.9 seconds late 2023-11-16 15:11:51,443 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 15:11:51,443 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:16:24 2023-11-16 15:12:26,529 INFO 1947952:repeater task lastensq_wh300 starting 3.5 seconds late 2023-11-16 15:12:29,237 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 15:12:29,237 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:12:18 2023-11-16 15:12:29,237 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:14:23 2023-11-16 15:12:29,237 INFO 1947952:repeater task quick_wh300 starting 6.2 seconds late 2023-11-16 15:12:36,097 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-16 15:12:36,098 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:12:18 2023-11-16 15:12:36,098 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:14:23 2023-11-16 15:12:36,098 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.1 seconds late 2023-11-16 15:12:36,726 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:12:36,727 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:12:18 2023-11-16 15:12:36,727 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:14:23 2023-11-16 15:14:27,028 INFO 1947952:repeater task lastensq_wh300 starting 4.0 seconds late 2023-11-16 15:14:30,172 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 15:14:30,173 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:14:18 2023-11-16 15:14:30,173 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:16:23 2023-11-16 15:14:30,173 INFO 1947952:repeater task quick_wh300 starting 7.2 seconds late 2023-11-16 15:14:36,851 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 15:14:36,852 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:14:18 2023-11-16 15:14:36,852 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:16:23 2023-11-16 15:14:36,852 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.9 seconds late 2023-11-16 15:14:37,485 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:14:37,485 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:14:18 2023-11-16 15:14:37,485 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:16:23 2023-11-16 15:16:27,777 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.7 seconds late 2023-11-16 15:16:30,908 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 3.1 seconds 2023-11-16 15:16:30,908 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 15:26:24 2023-11-16 15:16:30,909 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.9 seconds late 2023-11-16 15:16:33,810 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 15:16:33,810 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 15:26:24 2023-11-16 15:16:33,811 INFO 1947952:repeater task noisy_rawtemps_os38 starting 9.8 seconds late 2023-11-16 15:16:36,851 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 15:16:36,851 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 15:26:24 2023-11-16 15:16:36,851 INFO 1947952:repeater task lastensq_wh300 starting 13.9 seconds late 2023-11-16 15:16:39,795 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:16:39,795 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:16:18 2023-11-16 15:16:39,795 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:18:23 2023-11-16 15:16:39,796 INFO 1947952:repeater task quick_wh300 starting 16.8 seconds late 2023-11-16 15:16:46,104 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 15:16:46,104 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:16:18 2023-11-16 15:16:46,105 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:18:23 2023-11-16 15:16:46,105 INFO 1947952:repeater task lastfew_vec_wh300 starting 23.1 seconds late 2023-11-16 15:16:46,737 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:16:46,737 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:16:18 2023-11-16 15:16:46,737 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:18:23 2023-11-16 15:16:46,738 INFO 1947952:repeater task 3dayplots_wh300 starting 22.7 seconds late 2023-11-16 15:16:50,608 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 15:16:50,608 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:21:24 2023-11-16 15:16:50,609 INFO 1947952:repeater task lastensq_os150nb starting 27.6 seconds late 2023-11-16 15:16:52,900 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 15:16:52,900 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:16:18 2023-11-16 15:16:52,900 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:21:23 2023-11-16 15:16:52,900 INFO 1947952:repeater task quick_os150nb starting 29.9 seconds late 2023-11-16 15:16:56,646 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 15:16:56,646 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:16:18 2023-11-16 15:16:56,646 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:21:23 2023-11-16 15:16:56,647 INFO 1947952:repeater task lastfew_vec_os150nb starting 33.6 seconds late 2023-11-16 15:16:57,256 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:16:57,257 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:16:18 2023-11-16 15:16:57,257 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:21:23 2023-11-16 15:16:57,257 INFO 1947952:repeater task 3dayplots_os150nb starting 33.2 seconds late 2023-11-16 15:17:00,572 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.3 seconds 2023-11-16 15:17:00,572 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:21:24 2023-11-16 15:17:00,574 INFO 1947952:repeater task lastensq_os38nb starting 35.6 seconds late 2023-11-16 15:17:02,732 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 15:17:02,733 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:16:20 2023-11-16 15:17:02,733 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:21:25 2023-11-16 15:17:02,733 INFO 1947952:repeater task quick_os38nb starting 37.7 seconds late 2023-11-16 15:17:07,343 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 15:17:07,343 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:16:20 2023-11-16 15:17:07,343 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:21:25 2023-11-16 15:17:07,343 INFO 1947952:repeater task lastfew_vec_os38nb starting 42.3 seconds late 2023-11-16 15:17:07,948 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:17:07,948 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:16:20 2023-11-16 15:17:07,948 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:21:25 2023-11-16 15:17:07,949 INFO 1947952:repeater task 3dayplots_os38nb starting 43.9 seconds late 2023-11-16 15:17:11,363 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 15:17:11,363 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:21:24 2023-11-16 15:17:11,364 INFO 1947952:repeater task daily_netcdf starting 47.3 seconds late 2023-11-16 15:17:12,627 INFO 1947952:repeater task daily_netcdf ended after 1.3 seconds 2023-11-16 15:17:12,628 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 15:26:24 2023-11-16 15:18:27,823 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-16 15:18:30,942 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 15:18:30,942 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:18:18 2023-11-16 15:18:30,942 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:20:23 2023-11-16 15:18:30,943 INFO 1947952:repeater task quick_wh300 starting 7.9 seconds late 2023-11-16 15:18:37,789 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-16 15:18:37,790 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:18:18 2023-11-16 15:18:37,790 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:20:23 2023-11-16 15:18:37,790 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.8 seconds late 2023-11-16 15:18:38,436 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:18:38,436 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:18:18 2023-11-16 15:18:38,436 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:20:23 2023-11-16 15:20:23,686 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-16 15:20:26,483 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 15:20:26,484 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:20:18 2023-11-16 15:20:26,484 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:22:23 2023-11-16 15:20:26,484 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 15:20:33,351 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-16 15:20:33,352 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:20:18 2023-11-16 15:20:33,352 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:22:23 2023-11-16 15:20:33,352 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-16 15:20:33,983 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:20:33,983 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:20:18 2023-11-16 15:20:33,984 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:22:23 2023-11-16 15:21:19,080 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.0 seconds late 2023-11-16 15:21:20,262 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 15:21:20,262 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:20,263 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.2 seconds late 2023-11-16 15:21:20,488 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 15:21:20,488 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:20,489 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.4 seconds late 2023-11-16 15:21:21,782 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:21:21,782 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:21,782 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.7 seconds late 2023-11-16 15:21:23,046 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:21:23,046 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:23,047 INFO 1947952:repeater task lastensq_os150nb starting 0.0 seconds late 2023-11-16 15:21:25,241 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 15:21:25,242 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:21:18 2023-11-16 15:21:25,242 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:26:23 2023-11-16 15:21:25,242 INFO 1947952:repeater task quick_os150nb starting 2.2 seconds late 2023-11-16 15:21:29,026 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 15:21:29,027 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:21:18 2023-11-16 15:21:29,027 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:26:23 2023-11-16 15:21:29,027 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.0 seconds late 2023-11-16 15:21:29,638 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:21:29,638 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:21:18 2023-11-16 15:21:29,638 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:26:23 2023-11-16 15:21:29,638 INFO 1947952:repeater task 3dayplots_os150nb starting 5.6 seconds late 2023-11-16 15:21:33,059 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:21:33,059 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:26:24 2023-11-16 15:21:33,060 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.0 seconds late 2023-11-16 15:21:34,026 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 15:21:34,026 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:34,026 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.0 seconds late 2023-11-16 15:21:34,201 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 15:21:34,201 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:34,202 INFO 1947952:repeater task beamdiagnostics_os150nb starting 15.2 seconds late 2023-11-16 15:21:35,452 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 15:21:35,452 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:35,453 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.4 seconds late 2023-11-16 15:21:36,732 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 15:21:36,732 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:36,732 INFO 1947952:repeater task lastensq_os38nb starting 11.7 seconds late 2023-11-16 15:21:38,766 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 15:21:38,767 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:21:20 2023-11-16 15:21:38,767 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:26:25 2023-11-16 15:21:38,767 INFO 1947952:repeater task quick_os38nb starting 13.8 seconds late 2023-11-16 15:21:43,642 INFO 1947952:repeater task quick_os38nb ended after 4.9 seconds 2023-11-16 15:21:43,642 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:21:20 2023-11-16 15:21:43,642 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:26:25 2023-11-16 15:21:43,643 INFO 1947952:repeater task lastfew_vec_os38nb starting 18.6 seconds late 2023-11-16 15:21:44,256 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:21:44,257 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:21:20 2023-11-16 15:21:44,257 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:26:25 2023-11-16 15:21:44,257 INFO 1947952:repeater task 3dayplots_os38nb starting 20.2 seconds late 2023-11-16 15:21:47,692 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 15:21:47,692 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:26:24 2023-11-16 15:21:47,693 INFO 1947952:repeater task plot_tsstats_os38nb starting 28.6 seconds late 2023-11-16 15:21:48,701 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 15:21:48,702 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:48,702 INFO 1947952:repeater task grab_lastpings_os38nb starting 29.7 seconds late 2023-11-16 15:21:48,868 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 15:21:48,869 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:48,869 INFO 1947952:repeater task beamdiagnostics_os38nb starting 29.8 seconds late 2023-11-16 15:21:50,112 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 15:21:50,112 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:50,113 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.1 seconds late 2023-11-16 15:21:51,282 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 15:21:51,282 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:51,283 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 32.0 seconds late 2023-11-16 15:21:52,598 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 15:21:52,598 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:52,599 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 33.3 seconds late 2023-11-16 15:21:53,871 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 15:21:53,872 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:53,872 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 34.6 seconds late 2023-11-16 15:21:55,126 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 15:21:55,127 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:21:55,127 INFO 1947952:repeater task plot_hbins starting 35.9 seconds late 2023-11-16 15:21:59,491 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 15:21:59,491 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 15:21:59,491 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 15:36:19 2023-11-16 15:22:04,499 INFO 1947952:repeater task 3dayplots_wh300 starting 40.4 seconds late 2023-11-16 15:22:08,285 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 15:22:08,285 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:26:24 2023-11-16 15:22:23,320 INFO 1947952:repeater task lastensq_wh300 starting 0.3 seconds late 2023-11-16 15:22:26,337 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 15:22:26,338 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:22:18 2023-11-16 15:22:26,338 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:24:23 2023-11-16 15:22:26,338 INFO 1947952:repeater task quick_wh300 starting 3.3 seconds late 2023-11-16 15:22:33,322 INFO 1947952:repeater task quick_wh300 ended after 7.0 seconds 2023-11-16 15:22:33,323 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:22:18 2023-11-16 15:22:33,323 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:24:23 2023-11-16 15:22:33,323 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-16 15:22:33,943 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:22:33,943 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:22:18 2023-11-16 15:22:33,943 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:24:23 2023-11-16 15:24:24,226 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-16 15:24:27,183 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 15:24:27,184 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:24:18 2023-11-16 15:24:27,184 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:26:23 2023-11-16 15:24:27,184 INFO 1947952:repeater task quick_wh300 starting 4.2 seconds late 2023-11-16 15:24:33,592 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:24:33,592 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:24:18 2023-11-16 15:24:33,592 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:26:23 2023-11-16 15:24:33,593 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2023-11-16 15:24:34,212 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:24:34,213 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:24:18 2023-11-16 15:24:34,213 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:26:23 2023-11-16 15:26:24,493 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.4 seconds late 2023-11-16 15:26:27,383 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 15:26:27,383 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 15:36:24 2023-11-16 15:26:27,384 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.3 seconds late 2023-11-16 15:26:30,181 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 15:26:30,181 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 15:36:24 2023-11-16 15:26:30,182 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.1 seconds late 2023-11-16 15:26:33,493 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.3 seconds 2023-11-16 15:26:33,493 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 15:36:24 2023-11-16 15:26:33,493 INFO 1947952:repeater task lastensq_wh300 starting 10.5 seconds late 2023-11-16 15:26:36,311 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 15:26:36,312 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:26:18 2023-11-16 15:26:36,312 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:28:23 2023-11-16 15:26:36,312 INFO 1947952:repeater task quick_wh300 starting 13.3 seconds late 2023-11-16 15:26:42,780 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 15:26:42,781 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:26:18 2023-11-16 15:26:42,781 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:28:23 2023-11-16 15:26:42,781 INFO 1947952:repeater task lastfew_vec_wh300 starting 19.8 seconds late 2023-11-16 15:26:43,416 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:26:43,416 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:26:18 2023-11-16 15:26:43,416 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:28:23 2023-11-16 15:26:43,417 INFO 1947952:repeater task 3dayplots_wh300 starting 19.4 seconds late 2023-11-16 15:26:47,189 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 15:26:47,189 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:31:24 2023-11-16 15:26:47,190 INFO 1947952:repeater task lastensq_os150nb starting 24.2 seconds late 2023-11-16 15:26:49,576 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 15:26:49,577 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:26:18 2023-11-16 15:26:49,577 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:31:23 2023-11-16 15:26:49,577 INFO 1947952:repeater task quick_os150nb starting 26.6 seconds late 2023-11-16 15:26:53,109 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 15:26:53,110 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:26:18 2023-11-16 15:26:53,110 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:31:23 2023-11-16 15:26:53,110 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.1 seconds late 2023-11-16 15:26:53,721 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:26:53,722 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:26:18 2023-11-16 15:26:53,722 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:31:23 2023-11-16 15:26:53,722 INFO 1947952:repeater task 3dayplots_os150nb starting 29.7 seconds late 2023-11-16 15:26:57,165 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:26:57,165 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:31:24 2023-11-16 15:26:57,166 INFO 1947952:repeater task lastensq_os38nb starting 32.2 seconds late 2023-11-16 15:26:59,337 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 15:26:59,337 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:26:20 2023-11-16 15:26:59,337 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:31:25 2023-11-16 15:26:59,338 INFO 1947952:repeater task quick_os38nb starting 34.3 seconds late 2023-11-16 15:27:03,729 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 15:27:03,729 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:26:20 2023-11-16 15:27:03,729 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:31:25 2023-11-16 15:27:03,730 INFO 1947952:repeater task lastfew_vec_os38nb starting 38.7 seconds late 2023-11-16 15:27:04,335 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:27:04,336 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:26:20 2023-11-16 15:27:04,336 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:31:25 2023-11-16 15:27:04,336 INFO 1947952:repeater task 3dayplots_os38nb starting 40.3 seconds late 2023-11-16 15:27:07,773 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 15:27:07,773 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:31:24 2023-11-16 15:27:07,774 INFO 1947952:repeater task daily_netcdf starting 43.7 seconds late 2023-11-16 15:27:08,621 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 15:27:08,621 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 15:36:24 2023-11-16 15:28:23,816 INFO 1947952:repeater task lastensq_wh300 starting 0.8 seconds late 2023-11-16 15:28:26,499 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 15:28:26,500 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:28:18 2023-11-16 15:28:26,500 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:30:23 2023-11-16 15:28:26,500 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 15:28:33,026 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 15:28:33,026 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:28:18 2023-11-16 15:28:33,026 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:30:23 2023-11-16 15:28:33,026 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.0 seconds late 2023-11-16 15:28:33,663 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:28:33,663 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:28:18 2023-11-16 15:28:33,663 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:30:23 2023-11-16 15:30:23,948 INFO 1947952:repeater task lastensq_wh300 starting 0.9 seconds late 2023-11-16 15:30:26,824 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:30:26,824 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:30:18 2023-11-16 15:30:26,824 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:32:23 2023-11-16 15:30:26,825 INFO 1947952:repeater task quick_wh300 starting 3.8 seconds late 2023-11-16 15:30:33,537 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 15:30:33,537 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:30:18 2023-11-16 15:30:33,537 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:32:23 2023-11-16 15:30:33,538 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.5 seconds late 2023-11-16 15:30:34,173 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:30:34,174 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:30:18 2023-11-16 15:30:34,174 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:32:23 2023-11-16 15:31:24,305 INFO 1947952:repeater task 3dayplots_wh300 starting 0.3 seconds late 2023-11-16 15:31:28,166 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 15:31:28,166 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:36:24 2023-11-16 15:31:28,167 INFO 1947952:repeater task lastensq_os150nb starting 5.2 seconds late 2023-11-16 15:31:30,440 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 15:31:30,441 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:31:18 2023-11-16 15:31:30,441 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:36:23 2023-11-16 15:31:30,441 INFO 1947952:repeater task quick_os150nb starting 7.4 seconds late 2023-11-16 15:31:33,897 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 15:31:33,897 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:31:18 2023-11-16 15:31:33,897 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:36:23 2023-11-16 15:31:33,898 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.9 seconds late 2023-11-16 15:31:34,499 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:31:34,499 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:31:18 2023-11-16 15:31:34,499 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:36:23 2023-11-16 15:31:34,500 INFO 1947952:repeater task 3dayplots_os150nb starting 10.4 seconds late 2023-11-16 15:31:37,912 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:31:37,912 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:36:24 2023-11-16 15:31:37,913 INFO 1947952:repeater task lastensq_os38nb starting 12.9 seconds late 2023-11-16 15:31:39,976 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 15:31:39,976 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:31:20 2023-11-16 15:31:39,976 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:36:25 2023-11-16 15:31:39,976 INFO 1947952:repeater task quick_os38nb starting 15.0 seconds late 2023-11-16 15:31:43,805 INFO 1947952:repeater task quick_os38nb ended after 3.8 seconds 2023-11-16 15:31:43,805 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:31:20 2023-11-16 15:31:43,806 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:36:25 2023-11-16 15:31:43,806 INFO 1947952:repeater task lastfew_vec_os38nb starting 18.8 seconds late 2023-11-16 15:31:44,406 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:31:44,407 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:31:20 2023-11-16 15:31:44,407 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:36:25 2023-11-16 15:31:44,408 INFO 1947952:repeater task 3dayplots_os38nb starting 20.4 seconds late 2023-11-16 15:31:47,848 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 15:31:47,848 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:36:24 2023-11-16 15:32:27,939 INFO 1947952:repeater task lastensq_wh300 starting 4.9 seconds late 2023-11-16 15:32:30,635 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 15:32:30,635 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:32:18 2023-11-16 15:32:30,635 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:34:23 2023-11-16 15:32:30,636 INFO 1947952:repeater task quick_wh300 starting 7.6 seconds late 2023-11-16 15:32:36,767 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 15:32:36,768 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:32:18 2023-11-16 15:32:36,768 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:34:23 2023-11-16 15:32:36,768 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.8 seconds late 2023-11-16 15:32:37,410 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:32:37,411 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:32:18 2023-11-16 15:32:37,411 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:34:23 2023-11-16 15:34:27,821 INFO 1947952:repeater task lastensq_wh300 starting 4.8 seconds late 2023-11-16 15:34:30,562 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 15:34:30,562 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:34:18 2023-11-16 15:34:30,562 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:36:23 2023-11-16 15:34:30,563 INFO 1947952:repeater task quick_wh300 starting 7.6 seconds late 2023-11-16 15:34:36,982 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:34:36,982 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:34:18 2023-11-16 15:34:36,982 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:36:23 2023-11-16 15:34:36,983 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.0 seconds late 2023-11-16 15:34:37,621 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:34:37,622 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:34:18 2023-11-16 15:34:37,622 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:36:23 2023-11-16 15:36:22,905 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.9 seconds late 2023-11-16 15:36:24,085 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 15:36:24,086 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:24,086 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.0 seconds late 2023-11-16 15:36:24,255 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 15:36:24,255 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:24,255 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.2 seconds late 2023-11-16 15:36:25,522 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:36:25,522 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:25,523 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.5 seconds late 2023-11-16 15:36:26,773 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:36:26,773 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:26,774 INFO 1947952:repeater task lastensq_os150nb starting 3.8 seconds late 2023-11-16 15:36:29,137 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 15:36:29,138 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:36:18 2023-11-16 15:36:29,138 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:41:23 2023-11-16 15:36:29,138 INFO 1947952:repeater task quick_os150nb starting 6.1 seconds late 2023-11-16 15:36:33,058 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 15:36:33,058 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:36:18 2023-11-16 15:36:33,058 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:41:23 2023-11-16 15:36:33,059 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.1 seconds late 2023-11-16 15:36:33,674 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:36:33,675 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:36:18 2023-11-16 15:36:33,675 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:41:23 2023-11-16 15:36:33,675 INFO 1947952:repeater task 3dayplots_os150nb starting 9.6 seconds late 2023-11-16 15:36:37,119 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:36:37,119 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:41:24 2023-11-16 15:36:37,119 INFO 1947952:repeater task plot_tsstats_os150nb starting 18.1 seconds late 2023-11-16 15:36:38,092 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 15:36:38,092 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:38,092 INFO 1947952:repeater task grab_lastpings_os150nb starting 19.0 seconds late 2023-11-16 15:36:38,264 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 15:36:38,265 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:38,265 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.2 seconds late 2023-11-16 15:36:39,487 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 15:36:39,487 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:39,488 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.4 seconds late 2023-11-16 15:36:40,741 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 15:36:40,741 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:40,742 INFO 1947952:repeater task lastensq_os38nb starting 15.7 seconds late 2023-11-16 15:36:42,943 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 15:36:42,944 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:36:20 2023-11-16 15:36:42,944 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:41:25 2023-11-16 15:36:42,944 INFO 1947952:repeater task quick_os38nb starting 17.9 seconds late 2023-11-16 15:36:47,254 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 15:36:47,255 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:36:20 2023-11-16 15:36:47,255 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:41:25 2023-11-16 15:36:47,255 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.3 seconds late 2023-11-16 15:36:47,861 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:36:47,861 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:36:20 2023-11-16 15:36:47,861 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:41:25 2023-11-16 15:36:47,862 INFO 1947952:repeater task 3dayplots_os38nb starting 23.8 seconds late 2023-11-16 15:36:51,363 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 15:36:51,363 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:41:24 2023-11-16 15:36:51,363 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.3 seconds late 2023-11-16 15:36:52,370 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 15:36:52,371 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:52,371 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.3 seconds late 2023-11-16 15:36:52,538 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 15:36:52,538 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:52,539 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.5 seconds late 2023-11-16 15:36:53,770 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 15:36:53,770 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:53,770 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.7 seconds late 2023-11-16 15:36:54,991 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 15:36:54,992 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:36:54,992 INFO 1947952:repeater task daily_netcdf starting 30.9 seconds late 2023-11-16 15:36:55,895 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-16 15:36:55,895 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 15:46:24 2023-11-16 15:36:55,896 INFO 1947952:repeater task ggatime_diagnostics starting 36.8 seconds late 2023-11-16 15:37:11,918 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 15:37:11,919 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:37:11,919 INFO 1947952:repeater task ggatime_diagnostics starting 52.9 seconds late 2023-11-16 15:37:27,608 INFO 1947952:repeater task ggatime_diagnostics ended after 15.7 seconds 2023-11-16 15:37:27,608 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:37:27,609 INFO 1947952:repeater task ggatime_diagnostics starting 68.6 seconds late 2023-11-16 15:37:43,458 INFO 1947952:repeater task ggatime_diagnostics ended after 15.8 seconds 2023-11-16 15:37:43,458 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:37:43,458 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 84.2 seconds late 2023-11-16 15:37:44,708 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.2 seconds 2023-11-16 15:37:44,708 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:37:44,709 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 85.4 seconds late 2023-11-16 15:37:46,009 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 15:37:46,009 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:37:46,009 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 86.7 seconds late 2023-11-16 15:37:47,325 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 15:37:47,326 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:37:47,326 INFO 1947952:repeater task plot_hbins starting 88.1 seconds late 2023-11-16 15:37:51,690 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 15:37:51,690 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 15:37:51,690 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 15:51:19 2023-11-16 15:37:51,691 INFO 1947952:repeater task report_generator starting 92.4 seconds late 2023-11-16 15:37:57,682 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 15:37:57,683 INFO 1947952:repeater task report_generator ended after 6.0 seconds 2023-11-16 15:37:57,683 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 16:36:19 2023-11-16 15:38:02,686 INFO 1947952:repeater task noisy_rawtemps_os150 starting 98.6 seconds late 2023-11-16 15:38:05,587 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 15:38:05,588 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 15:46:24 2023-11-16 15:38:05,588 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 101.5 seconds late 2023-11-16 15:38:08,395 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 15:38:08,395 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 15:46:24 2023-11-16 15:38:08,396 INFO 1947952:repeater task noisy_rawtemps_os38 starting 104.3 seconds late 2023-11-16 15:38:11,336 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 2.9 seconds 2023-11-16 15:38:11,336 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 15:46:24 2023-11-16 15:38:11,336 INFO 1947952:repeater task lastensq_wh300 starting 108.3 seconds late 2023-11-16 15:38:14,309 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 15:38:14,309 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:36:18 2023-11-16 15:38:14,310 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:38:23 2023-11-16 15:38:14,310 INFO 1947952:repeater task quick_wh300 starting 111.3 seconds late 2023-11-16 15:38:20,401 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 15:38:20,401 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:36:18 2023-11-16 15:38:20,401 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:38:23 2023-11-16 15:38:20,401 INFO 1947952:repeater task lastfew_vec_wh300 starting 117.4 seconds late 2023-11-16 15:38:21,028 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:38:21,028 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:36:18 2023-11-16 15:38:21,028 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:38:23 2023-11-16 15:38:21,029 INFO 1947952:repeater task 3dayplots_wh300 starting 117.0 seconds late 2023-11-16 15:38:24,795 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 15:38:24,795 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:41:24 2023-11-16 15:38:29,802 INFO 1947952:repeater task lastensq_wh300 starting 6.8 seconds late 2023-11-16 15:38:33,174 INFO 1947952:repeater task lastensq_wh300 ended after 3.4 seconds 2023-11-16 15:38:33,175 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:38:18 2023-11-16 15:38:33,175 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:40:23 2023-11-16 15:38:33,175 INFO 1947952:repeater task quick_wh300 starting 10.2 seconds late 2023-11-16 15:38:39,218 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 15:38:39,218 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:38:18 2023-11-16 15:38:39,218 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:40:23 2023-11-16 15:38:39,219 INFO 1947952:repeater task lastfew_vec_wh300 starting 16.2 seconds late 2023-11-16 15:38:39,852 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:38:39,852 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:38:18 2023-11-16 15:38:39,853 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:40:23 2023-11-16 15:40:25,127 INFO 1947952:repeater task lastensq_wh300 starting 2.1 seconds late 2023-11-16 15:40:28,109 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 15:40:28,110 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:40:18 2023-11-16 15:40:28,110 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:42:23 2023-11-16 15:40:28,110 INFO 1947952:repeater task quick_wh300 starting 5.1 seconds late 2023-11-16 15:40:34,748 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 15:40:34,748 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:40:18 2023-11-16 15:40:34,748 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:42:23 2023-11-16 15:40:34,748 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.7 seconds late 2023-11-16 15:40:35,386 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:40:35,386 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:40:18 2023-11-16 15:40:35,386 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:42:23 2023-11-16 15:41:25,524 INFO 1947952:repeater task 3dayplots_wh300 starting 1.5 seconds late 2023-11-16 15:41:29,341 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 15:41:29,341 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:46:24 2023-11-16 15:41:29,342 INFO 1947952:repeater task lastensq_os150nb starting 6.3 seconds late 2023-11-16 15:41:31,601 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 15:41:31,602 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:41:18 2023-11-16 15:41:31,602 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:46:23 2023-11-16 15:41:31,602 INFO 1947952:repeater task quick_os150nb starting 8.6 seconds late 2023-11-16 15:41:35,408 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 15:41:35,408 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:41:18 2023-11-16 15:41:35,408 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:46:23 2023-11-16 15:41:35,409 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.4 seconds late 2023-11-16 15:41:36,016 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:41:36,016 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:41:18 2023-11-16 15:41:36,016 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:46:23 2023-11-16 15:41:36,017 INFO 1947952:repeater task 3dayplots_os150nb starting 12.0 seconds late 2023-11-16 15:41:39,384 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:41:39,384 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:46:24 2023-11-16 15:41:39,385 INFO 1947952:repeater task lastensq_os38nb starting 14.4 seconds late 2023-11-16 15:41:41,582 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 15:41:41,583 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:41:20 2023-11-16 15:41:41,583 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:46:25 2023-11-16 15:41:41,583 INFO 1947952:repeater task quick_os38nb starting 16.6 seconds late 2023-11-16 15:41:45,832 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 15:41:45,832 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:41:20 2023-11-16 15:41:45,833 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:46:25 2023-11-16 15:41:45,833 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.8 seconds late 2023-11-16 15:41:46,452 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:41:46,452 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:41:20 2023-11-16 15:41:46,452 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:46:25 2023-11-16 15:41:46,453 INFO 1947952:repeater task 3dayplots_os38nb starting 22.4 seconds late 2023-11-16 15:41:49,801 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.3 seconds 2023-11-16 15:41:49,801 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:46:24 2023-11-16 15:42:24,890 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 15:42:27,619 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 15:42:27,620 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:42:18 2023-11-16 15:42:27,620 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:44:23 2023-11-16 15:42:27,620 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-16 15:42:33,933 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 15:42:33,933 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:42:18 2023-11-16 15:42:33,933 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:44:23 2023-11-16 15:42:33,934 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2023-11-16 15:42:34,569 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:42:34,570 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:42:18 2023-11-16 15:42:34,570 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:44:23 2023-11-16 15:44:24,868 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 15:44:27,697 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 15:44:27,698 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:44:18 2023-11-16 15:44:27,698 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:46:23 2023-11-16 15:44:27,698 INFO 1947952:repeater task quick_wh300 starting 4.7 seconds late 2023-11-16 15:44:33,853 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 15:44:33,853 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:44:18 2023-11-16 15:44:33,853 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:46:23 2023-11-16 15:44:33,853 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.9 seconds late 2023-11-16 15:44:34,491 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:44:34,491 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:44:18 2023-11-16 15:44:34,492 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:46:23 2023-11-16 15:46:24,769 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.7 seconds late 2023-11-16 15:46:27,676 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 15:46:27,676 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 15:56:24 2023-11-16 15:46:27,676 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.6 seconds late 2023-11-16 15:46:30,497 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 15:46:30,497 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 15:56:24 2023-11-16 15:46:30,498 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.4 seconds late 2023-11-16 15:46:33,787 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.3 seconds 2023-11-16 15:46:33,788 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 15:56:24 2023-11-16 15:46:33,788 INFO 1947952:repeater task lastensq_wh300 starting 10.8 seconds late 2023-11-16 15:46:36,984 INFO 1947952:repeater task lastensq_wh300 ended after 3.2 seconds 2023-11-16 15:46:36,985 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:46:18 2023-11-16 15:46:36,985 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:48:23 2023-11-16 15:46:36,985 INFO 1947952:repeater task quick_wh300 starting 14.0 seconds late 2023-11-16 15:46:43,567 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 15:46:43,568 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:46:18 2023-11-16 15:46:43,568 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:48:23 2023-11-16 15:46:43,568 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.6 seconds late 2023-11-16 15:46:44,198 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:46:44,198 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:46:18 2023-11-16 15:46:44,198 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:48:23 2023-11-16 15:46:44,199 INFO 1947952:repeater task 3dayplots_wh300 starting 20.1 seconds late 2023-11-16 15:46:48,119 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 15:46:48,120 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:51:24 2023-11-16 15:46:48,121 INFO 1947952:repeater task lastensq_os150nb starting 25.1 seconds late 2023-11-16 15:46:50,828 INFO 1947952:repeater task lastensq_os150nb ended after 2.7 seconds 2023-11-16 15:46:50,829 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:46:18 2023-11-16 15:46:50,829 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:51:23 2023-11-16 15:46:50,829 INFO 1947952:repeater task quick_os150nb starting 27.8 seconds late 2023-11-16 15:46:54,730 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 15:46:54,731 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:46:18 2023-11-16 15:46:54,731 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:51:23 2023-11-16 15:46:54,731 INFO 1947952:repeater task lastfew_vec_os150nb starting 31.7 seconds late 2023-11-16 15:46:55,344 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:46:55,344 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:46:18 2023-11-16 15:46:55,345 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:51:23 2023-11-16 15:46:55,345 INFO 1947952:repeater task 3dayplots_os150nb starting 31.3 seconds late 2023-11-16 15:46:58,843 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 15:46:58,843 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:51:24 2023-11-16 15:46:58,845 INFO 1947952:repeater task lastensq_os38nb starting 33.8 seconds late 2023-11-16 15:47:01,090 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 15:47:01,091 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:46:20 2023-11-16 15:47:01,091 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:51:25 2023-11-16 15:47:01,091 INFO 1947952:repeater task quick_os38nb starting 36.1 seconds late 2023-11-16 15:47:05,482 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 15:47:05,483 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:46:20 2023-11-16 15:47:05,483 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:51:25 2023-11-16 15:47:05,483 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.5 seconds late 2023-11-16 15:47:06,110 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:47:06,111 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:46:20 2023-11-16 15:47:06,111 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:51:25 2023-11-16 15:47:06,111 INFO 1947952:repeater task 3dayplots_os38nb starting 42.1 seconds late 2023-11-16 15:47:09,562 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 15:47:09,562 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:51:24 2023-11-16 15:47:09,563 INFO 1947952:repeater task daily_netcdf starting 45.5 seconds late 2023-11-16 15:47:10,493 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-16 15:47:10,493 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 15:56:24 2023-11-16 15:48:25,690 INFO 1947952:repeater task lastensq_wh300 starting 2.7 seconds late 2023-11-16 15:48:28,613 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:48:28,613 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:48:18 2023-11-16 15:48:28,613 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:50:23 2023-11-16 15:48:28,614 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-16 15:48:35,055 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:48:35,056 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:48:18 2023-11-16 15:48:35,056 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:50:23 2023-11-16 15:48:35,056 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.1 seconds late 2023-11-16 15:48:35,691 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:48:35,691 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:48:18 2023-11-16 15:48:35,691 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:50:23 2023-11-16 15:50:25,980 INFO 1947952:repeater task lastensq_wh300 starting 3.0 seconds late 2023-11-16 15:50:28,837 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:50:28,838 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:50:18 2023-11-16 15:50:28,838 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:52:23 2023-11-16 15:50:28,838 INFO 1947952:repeater task quick_wh300 starting 5.8 seconds late 2023-11-16 15:50:34,855 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 15:50:34,855 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:50:18 2023-11-16 15:50:34,855 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:52:23 2023-11-16 15:50:34,856 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-16 15:50:35,496 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:50:35,496 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:50:18 2023-11-16 15:50:35,496 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:52:23 2023-11-16 15:51:20,596 INFO 1947952:repeater task plot_tsstats_wh300 starting 1.5 seconds late 2023-11-16 15:51:21,777 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 15:51:21,778 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:21,778 INFO 1947952:repeater task grab_lastpings_wh300 starting 2.7 seconds late 2023-11-16 15:51:21,944 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 15:51:21,945 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:21,945 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.9 seconds late 2023-11-16 15:51:23,209 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:51:23,209 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:23,209 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.2 seconds late 2023-11-16 15:51:24,503 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 15:51:24,504 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:24,504 INFO 1947952:repeater task lastensq_os150nb starting 1.5 seconds late 2023-11-16 15:51:26,679 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 15:51:26,679 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:51:18 2023-11-16 15:51:26,679 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 15:56:23 2023-11-16 15:51:26,680 INFO 1947952:repeater task quick_os150nb starting 3.7 seconds late 2023-11-16 15:51:30,448 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 15:51:30,449 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:51:18 2023-11-16 15:51:30,449 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 15:56:23 2023-11-16 15:51:30,449 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.4 seconds late 2023-11-16 15:51:31,055 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:51:31,056 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:51:18 2023-11-16 15:51:31,056 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 15:56:23 2023-11-16 15:51:31,056 INFO 1947952:repeater task 3dayplots_os150nb starting 7.0 seconds late 2023-11-16 15:51:34,480 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:51:34,480 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 15:56:24 2023-11-16 15:51:34,480 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.4 seconds late 2023-11-16 15:51:35,449 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 15:51:35,449 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:35,450 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.4 seconds late 2023-11-16 15:51:35,616 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 15:51:35,616 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:35,616 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.6 seconds late 2023-11-16 15:51:36,913 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 15:51:36,914 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:36,914 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.9 seconds late 2023-11-16 15:51:38,122 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 15:51:38,122 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:38,122 INFO 1947952:repeater task lastensq_os38nb starting 13.1 seconds late 2023-11-16 15:51:40,317 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 15:51:40,317 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:51:20 2023-11-16 15:51:40,317 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 15:56:25 2023-11-16 15:51:40,317 INFO 1947952:repeater task quick_os38nb starting 15.3 seconds late 2023-11-16 15:51:44,668 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 15:51:44,668 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:51:20 2023-11-16 15:51:44,669 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 15:56:25 2023-11-16 15:51:44,669 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.7 seconds late 2023-11-16 15:51:45,288 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:51:45,288 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:51:20 2023-11-16 15:51:45,288 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 15:56:25 2023-11-16 15:51:45,288 INFO 1947952:repeater task 3dayplots_os38nb starting 21.2 seconds late 2023-11-16 15:51:48,754 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 15:51:48,754 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 15:56:24 2023-11-16 15:51:48,755 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.7 seconds late 2023-11-16 15:51:49,761 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 15:51:49,761 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:49,761 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.7 seconds late 2023-11-16 15:51:49,928 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 15:51:49,928 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:49,929 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.9 seconds late 2023-11-16 15:51:51,207 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 15:51:51,207 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:51,208 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.2 seconds late 2023-11-16 15:51:52,440 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 15:51:52,441 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:52,442 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 33.2 seconds late 2023-11-16 15:51:53,630 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.2 seconds 2023-11-16 15:51:53,631 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:53,631 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 34.4 seconds late 2023-11-16 15:51:54,911 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 15:51:54,912 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:54,912 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 35.6 seconds late 2023-11-16 15:51:56,207 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 15:51:56,207 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:51:56,207 INFO 1947952:repeater task plot_hbins starting 36.9 seconds late 2023-11-16 15:52:00,558 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 15:52:00,558 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 15:52:00,558 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 16:06:19 2023-11-16 15:52:05,566 INFO 1947952:repeater task 3dayplots_wh300 starting 41.5 seconds late 2023-11-16 15:52:09,338 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 15:52:09,338 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 15:56:24 2023-11-16 15:52:24,375 INFO 1947952:repeater task lastensq_wh300 starting 1.4 seconds late 2023-11-16 15:52:27,086 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 15:52:27,086 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:52:18 2023-11-16 15:52:27,087 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:54:23 2023-11-16 15:52:27,087 INFO 1947952:repeater task quick_wh300 starting 4.1 seconds late 2023-11-16 15:52:33,443 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:52:33,444 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:52:18 2023-11-16 15:52:33,444 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:54:23 2023-11-16 15:52:33,444 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.4 seconds late 2023-11-16 15:52:34,123 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 15:52:34,123 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:52:18 2023-11-16 15:52:34,123 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:54:23 2023-11-16 15:54:24,398 INFO 1947952:repeater task lastensq_wh300 starting 1.4 seconds late 2023-11-16 15:54:27,256 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 15:54:27,256 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:54:18 2023-11-16 15:54:27,256 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:56:23 2023-11-16 15:54:27,256 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-16 15:54:33,667 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:54:33,667 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:54:18 2023-11-16 15:54:33,668 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:56:23 2023-11-16 15:54:33,668 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2023-11-16 15:54:34,331 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 15:54:34,332 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:54:18 2023-11-16 15:54:34,332 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:56:23 2023-11-16 15:56:24,605 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.6 seconds late 2023-11-16 15:56:27,520 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 15:56:27,520 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 16:06:24 2023-11-16 15:56:27,521 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.5 seconds late 2023-11-16 15:56:30,343 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 15:56:30,343 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 16:06:24 2023-11-16 15:56:30,343 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.3 seconds late 2023-11-16 15:56:33,495 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.2 seconds 2023-11-16 15:56:33,495 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 16:06:24 2023-11-16 15:56:33,496 INFO 1947952:repeater task lastensq_wh300 starting 10.5 seconds late 2023-11-16 15:56:36,458 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 15:56:36,458 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:56:18 2023-11-16 15:56:36,458 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 15:58:23 2023-11-16 15:56:36,459 INFO 1947952:repeater task quick_wh300 starting 13.5 seconds late 2023-11-16 15:56:42,706 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 15:56:42,706 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:56:18 2023-11-16 15:56:42,706 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 15:58:23 2023-11-16 15:56:42,707 INFO 1947952:repeater task lastfew_vec_wh300 starting 19.7 seconds late 2023-11-16 15:56:43,337 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 15:56:43,337 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:56:18 2023-11-16 15:56:43,337 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 15:58:23 2023-11-16 15:56:43,338 INFO 1947952:repeater task 3dayplots_wh300 starting 19.3 seconds late 2023-11-16 15:56:47,233 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 15:56:47,233 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:01:24 2023-11-16 15:56:47,234 INFO 1947952:repeater task lastensq_os150nb starting 24.2 seconds late 2023-11-16 15:56:49,546 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 15:56:49,546 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:56:18 2023-11-16 15:56:49,546 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:01:23 2023-11-16 15:56:49,547 INFO 1947952:repeater task quick_os150nb starting 26.5 seconds late 2023-11-16 15:56:53,084 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 15:56:53,085 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:56:18 2023-11-16 15:56:53,085 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:01:23 2023-11-16 15:56:53,085 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.1 seconds late 2023-11-16 15:56:53,701 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 15:56:53,702 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 15:56:18 2023-11-16 15:56:53,702 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:01:23 2023-11-16 15:56:53,702 INFO 1947952:repeater task 3dayplots_os150nb starting 29.7 seconds late 2023-11-16 15:56:57,108 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 15:56:57,108 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:01:24 2023-11-16 15:56:57,109 INFO 1947952:repeater task lastensq_os38nb starting 32.1 seconds late 2023-11-16 15:56:59,206 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 15:56:59,207 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:56:20 2023-11-16 15:56:59,207 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:01:25 2023-11-16 15:56:59,207 INFO 1947952:repeater task quick_os38nb starting 34.2 seconds late 2023-11-16 15:57:03,605 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 15:57:03,606 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:56:20 2023-11-16 15:57:03,606 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:01:25 2023-11-16 15:57:03,606 INFO 1947952:repeater task lastfew_vec_os38nb starting 38.6 seconds late 2023-11-16 15:57:04,222 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 15:57:04,222 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 15:56:20 2023-11-16 15:57:04,222 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:01:25 2023-11-16 15:57:04,223 INFO 1947952:repeater task 3dayplots_os38nb starting 40.2 seconds late 2023-11-16 15:57:07,637 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 15:57:07,637 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:01:24 2023-11-16 15:57:07,639 INFO 1947952:repeater task daily_netcdf starting 43.6 seconds late 2023-11-16 15:57:09,104 INFO 1947952:repeater task daily_netcdf ended after 1.5 seconds 2023-11-16 15:57:09,104 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 16:06:24 2023-11-16 15:58:24,280 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 15:58:27,336 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 15:58:27,337 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:58:18 2023-11-16 15:58:27,337 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:00:23 2023-11-16 15:58:27,337 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-16 15:58:33,722 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 15:58:33,723 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:58:18 2023-11-16 15:58:33,723 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:00:23 2023-11-16 15:58:33,723 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2023-11-16 15:58:34,438 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 15:58:34,438 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 15:58:18 2023-11-16 15:58:34,438 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:00:23 2023-11-16 16:00:24,728 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-16 16:00:27,863 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 16:00:27,864 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:00:18 2023-11-16 16:00:27,864 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:02:23 2023-11-16 16:00:27,865 INFO 1947952:repeater task quick_wh300 starting 4.9 seconds late 2023-11-16 16:00:34,375 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 16:00:34,375 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:00:18 2023-11-16 16:00:34,375 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:02:23 2023-11-16 16:00:34,375 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-16 16:00:35,047 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 16:00:35,047 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:00:18 2023-11-16 16:00:35,047 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:02:23 2023-11-16 16:01:25,156 INFO 1947952:repeater task 3dayplots_wh300 starting 1.1 seconds late 2023-11-16 16:01:28,902 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 16:01:28,903 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:06:24 2023-11-16 16:01:28,904 INFO 1947952:repeater task lastensq_os150nb starting 5.9 seconds late 2023-11-16 16:01:31,467 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 16:01:31,468 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:01:18 2023-11-16 16:01:31,468 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:06:23 2023-11-16 16:01:31,469 INFO 1947952:repeater task quick_os150nb starting 8.5 seconds late 2023-11-16 16:01:35,732 INFO 1947952:repeater task quick_os150nb ended after 4.3 seconds 2023-11-16 16:01:35,733 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:01:18 2023-11-16 16:01:35,733 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:06:23 2023-11-16 16:01:35,733 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.7 seconds late 2023-11-16 16:01:36,347 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:01:36,347 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:01:18 2023-11-16 16:01:36,347 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:06:23 2023-11-16 16:01:36,348 INFO 1947952:repeater task 3dayplots_os150nb starting 12.3 seconds late 2023-11-16 16:01:39,733 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 16:01:39,733 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:06:24 2023-11-16 16:01:39,734 INFO 1947952:repeater task lastensq_os38nb starting 14.7 seconds late 2023-11-16 16:01:41,807 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:01:41,808 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:01:20 2023-11-16 16:01:41,808 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:06:25 2023-11-16 16:01:41,808 INFO 1947952:repeater task quick_os38nb starting 16.8 seconds late 2023-11-16 16:01:46,583 INFO 1947952:repeater task quick_os38nb ended after 4.8 seconds 2023-11-16 16:01:46,583 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:01:20 2023-11-16 16:01:46,584 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:06:25 2023-11-16 16:01:46,584 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.6 seconds late 2023-11-16 16:01:47,204 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:01:47,204 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:01:20 2023-11-16 16:01:47,204 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:06:25 2023-11-16 16:01:47,204 INFO 1947952:repeater task 3dayplots_os38nb starting 23.2 seconds late 2023-11-16 16:01:50,684 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 16:01:50,684 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:06:24 2023-11-16 16:02:25,776 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-16 16:02:28,638 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 16:02:28,638 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:02:18 2023-11-16 16:02:28,638 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:04:23 2023-11-16 16:02:28,639 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-16 16:02:35,716 INFO 1947952:repeater task quick_wh300 ended after 7.1 seconds 2023-11-16 16:02:35,716 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:02:18 2023-11-16 16:02:35,716 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:04:23 2023-11-16 16:02:35,716 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.7 seconds late 2023-11-16 16:02:36,352 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:02:36,352 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:02:18 2023-11-16 16:02:36,352 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:04:23 2023-11-16 16:04:26,652 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-16 16:04:29,665 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 16:04:29,665 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:04:18 2023-11-16 16:04:29,665 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:06:23 2023-11-16 16:04:29,666 INFO 1947952:repeater task quick_wh300 starting 6.7 seconds late 2023-11-16 16:04:36,375 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 16:04:36,376 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:04:18 2023-11-16 16:04:36,376 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:06:23 2023-11-16 16:04:36,376 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2023-11-16 16:04:37,016 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:04:37,017 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:04:18 2023-11-16 16:04:37,017 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:06:23 2023-11-16 16:06:22,293 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.2 seconds late 2023-11-16 16:06:23,474 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 16:06:23,475 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:23,475 INFO 1947952:repeater task grab_lastpings_wh300 starting 4.4 seconds late 2023-11-16 16:06:23,642 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 16:06:23,642 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:23,643 INFO 1947952:repeater task beamdiagnostics_wh300 starting 4.6 seconds late 2023-11-16 16:06:24,883 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 16:06:24,883 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:24,883 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.8 seconds late 2023-11-16 16:06:26,169 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 16:06:26,169 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:26,169 INFO 1947952:repeater task lastensq_os150nb starting 3.2 seconds late 2023-11-16 16:06:28,542 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 16:06:28,543 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:06:18 2023-11-16 16:06:28,543 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:11:23 2023-11-16 16:06:28,543 INFO 1947952:repeater task quick_os150nb starting 5.5 seconds late 2023-11-16 16:06:32,317 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 16:06:32,318 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:06:18 2023-11-16 16:06:32,318 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:11:23 2023-11-16 16:06:32,318 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.3 seconds late 2023-11-16 16:06:32,954 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:06:32,955 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:06:18 2023-11-16 16:06:32,955 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:11:23 2023-11-16 16:06:32,956 INFO 1947952:repeater task 3dayplots_os150nb starting 8.9 seconds late 2023-11-16 16:06:36,631 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.7 seconds 2023-11-16 16:06:36,631 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:11:24 2023-11-16 16:06:36,631 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.6 seconds late 2023-11-16 16:06:37,608 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 16:06:37,608 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:37,608 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.6 seconds late 2023-11-16 16:06:37,777 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 16:06:37,777 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:37,778 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.7 seconds late 2023-11-16 16:06:39,054 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 16:06:39,055 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:39,055 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.0 seconds late 2023-11-16 16:06:40,296 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 16:06:40,296 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:40,297 INFO 1947952:repeater task lastensq_os38nb starting 15.3 seconds late 2023-11-16 16:06:42,635 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-16 16:06:42,636 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:06:20 2023-11-16 16:06:42,636 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:11:25 2023-11-16 16:06:42,637 INFO 1947952:repeater task quick_os38nb starting 17.6 seconds late 2023-11-16 16:06:46,900 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 16:06:46,900 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:06:20 2023-11-16 16:06:46,900 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:11:25 2023-11-16 16:06:46,900 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.9 seconds late 2023-11-16 16:06:47,516 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:06:47,516 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:06:20 2023-11-16 16:06:47,516 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:11:25 2023-11-16 16:06:47,517 INFO 1947952:repeater task 3dayplots_os38nb starting 23.5 seconds late 2023-11-16 16:06:51,006 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 16:06:51,006 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:11:24 2023-11-16 16:06:51,006 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.0 seconds late 2023-11-16 16:06:52,016 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 16:06:52,016 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:52,016 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.0 seconds late 2023-11-16 16:06:52,187 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 16:06:52,187 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:52,187 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.1 seconds late 2023-11-16 16:06:53,526 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 16:06:53,527 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:53,527 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.5 seconds late 2023-11-16 16:06:54,754 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 16:06:54,754 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:06:54,755 INFO 1947952:repeater task daily_netcdf starting 30.7 seconds late 2023-11-16 16:06:55,568 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 16:06:55,569 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 16:16:24 2023-11-16 16:06:55,569 INFO 1947952:repeater task ggatime_diagnostics starting 36.5 seconds late 2023-11-16 16:07:11,624 INFO 1947952:repeater task ggatime_diagnostics ended after 16.1 seconds 2023-11-16 16:07:11,624 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:07:11,625 INFO 1947952:repeater task ggatime_diagnostics starting 52.6 seconds late 2023-11-16 16:07:27,579 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 16:07:27,579 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:07:27,579 INFO 1947952:repeater task ggatime_diagnostics starting 68.5 seconds late 2023-11-16 16:07:43,974 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-16 16:07:43,974 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:07:43,975 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 84.7 seconds late 2023-11-16 16:07:45,259 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 16:07:45,260 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:07:45,260 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 86.0 seconds late 2023-11-16 16:07:46,595 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 16:07:46,595 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:07:46,595 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 87.3 seconds late 2023-11-16 16:07:47,845 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.2 seconds 2023-11-16 16:07:47,845 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:07:47,846 INFO 1947952:repeater task plot_hbins starting 88.6 seconds late 2023-11-16 16:07:52,084 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 16:07:52,085 INFO 1947952:repeater task plot_hbins ended after 4.2 seconds 2023-11-16 16:07:52,085 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 16:21:19 2023-11-16 16:07:57,089 INFO 1947952:repeater task noisy_rawtemps_os150 starting 93.0 seconds late 2023-11-16 16:07:59,952 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 16:07:59,952 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 16:16:24 2023-11-16 16:07:59,952 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 95.9 seconds late 2023-11-16 16:08:02,737 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 16:08:02,737 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 16:16:24 2023-11-16 16:08:02,737 INFO 1947952:repeater task noisy_rawtemps_os38 starting 98.7 seconds late 2023-11-16 16:08:05,685 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 2.9 seconds 2023-11-16 16:08:05,685 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 16:16:24 2023-11-16 16:08:05,686 INFO 1947952:repeater task lastensq_wh300 starting 102.7 seconds late 2023-11-16 16:08:08,392 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 16:08:08,392 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:06:18 2023-11-16 16:08:08,392 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:08:23 2023-11-16 16:08:08,393 INFO 1947952:repeater task quick_wh300 starting 105.4 seconds late 2023-11-16 16:08:14,843 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 16:08:14,844 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:06:18 2023-11-16 16:08:14,844 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:08:23 2023-11-16 16:08:14,844 INFO 1947952:repeater task lastfew_vec_wh300 starting 111.8 seconds late 2023-11-16 16:08:15,483 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:08:15,484 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:06:18 2023-11-16 16:08:15,484 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:08:23 2023-11-16 16:08:15,484 INFO 1947952:repeater task 3dayplots_wh300 starting 111.4 seconds late 2023-11-16 16:08:19,243 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 16:08:19,244 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:11:24 2023-11-16 16:08:24,254 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 16:08:27,117 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 16:08:27,117 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:08:18 2023-11-16 16:08:27,118 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:10:23 2023-11-16 16:08:27,118 INFO 1947952:repeater task quick_wh300 starting 4.1 seconds late 2023-11-16 16:08:33,243 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 16:08:33,243 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:08:18 2023-11-16 16:08:33,243 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:10:23 2023-11-16 16:08:33,244 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.2 seconds late 2023-11-16 16:08:33,958 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 16:08:33,959 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:08:18 2023-11-16 16:08:33,959 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:10:23 2023-11-16 16:10:24,224 INFO 1947952:repeater task lastensq_wh300 starting 1.2 seconds late 2023-11-16 16:10:26,960 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 16:10:26,961 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:10:18 2023-11-16 16:10:26,961 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:12:23 2023-11-16 16:10:26,961 INFO 1947952:repeater task quick_wh300 starting 4.0 seconds late 2023-11-16 16:10:33,055 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 16:10:33,056 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:10:18 2023-11-16 16:10:33,056 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:12:23 2023-11-16 16:10:33,056 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2023-11-16 16:10:33,873 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.8 seconds 2023-11-16 16:10:33,874 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:10:18 2023-11-16 16:10:33,874 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:12:23 2023-11-16 16:11:23,999 INFO 1947952:repeater task lastensq_os150nb starting 1.0 seconds late 2023-11-16 16:11:26,274 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 16:11:26,274 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:11:18 2023-11-16 16:11:26,275 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:16:23 2023-11-16 16:11:26,275 INFO 1947952:repeater task quick_os150nb starting 3.3 seconds late 2023-11-16 16:11:29,969 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 16:11:29,969 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:11:18 2023-11-16 16:11:29,969 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:16:23 2023-11-16 16:11:29,970 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.0 seconds late 2023-11-16 16:11:30,584 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:11:30,585 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:11:18 2023-11-16 16:11:30,585 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:16:23 2023-11-16 16:11:30,585 INFO 1947952:repeater task 3dayplots_os150nb starting 6.5 seconds late 2023-11-16 16:11:34,006 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 16:11:34,006 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:16:24 2023-11-16 16:11:34,007 INFO 1947952:repeater task lastensq_os38nb starting 9.0 seconds late 2023-11-16 16:11:36,079 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:11:36,079 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:11:20 2023-11-16 16:11:36,079 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:16:25 2023-11-16 16:11:36,080 INFO 1947952:repeater task quick_os38nb starting 11.1 seconds late 2023-11-16 16:11:40,172 INFO 1947952:repeater task quick_os38nb ended after 4.1 seconds 2023-11-16 16:11:40,173 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:11:20 2023-11-16 16:11:40,173 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:16:25 2023-11-16 16:11:40,173 INFO 1947952:repeater task lastfew_vec_os38nb starting 15.2 seconds late 2023-11-16 16:11:40,791 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:11:40,791 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:11:20 2023-11-16 16:11:40,791 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:16:25 2023-11-16 16:11:40,792 INFO 1947952:repeater task 3dayplots_os38nb starting 16.7 seconds late 2023-11-16 16:11:44,250 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 16:11:44,250 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:16:24 2023-11-16 16:11:49,255 INFO 1947952:repeater task 3dayplots_wh300 starting 25.2 seconds late 2023-11-16 16:11:53,096 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 16:11:53,096 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:16:24 2023-11-16 16:12:23,172 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-16 16:12:26,007 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:12:26,007 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:12:18 2023-11-16 16:12:26,007 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:14:23 2023-11-16 16:12:26,008 INFO 1947952:repeater task quick_wh300 starting 3.0 seconds late 2023-11-16 16:12:32,255 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 16:12:32,255 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:12:18 2023-11-16 16:12:32,255 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:14:23 2023-11-16 16:12:32,256 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.3 seconds late 2023-11-16 16:12:32,897 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:12:32,897 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:12:18 2023-11-16 16:12:32,897 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:14:23 2023-11-16 16:14:23,182 INFO 1947952:repeater task lastensq_wh300 starting 0.2 seconds late 2023-11-16 16:14:26,001 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:14:26,001 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:14:18 2023-11-16 16:14:26,001 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:16:23 2023-11-16 16:14:26,002 INFO 1947952:repeater task quick_wh300 starting 3.0 seconds late 2023-11-16 16:14:32,142 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 16:14:32,143 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:14:18 2023-11-16 16:14:32,143 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:16:23 2023-11-16 16:14:32,143 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.1 seconds late 2023-11-16 16:14:32,781 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:14:32,781 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:14:18 2023-11-16 16:14:32,782 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:16:23 2023-11-16 16:16:23,084 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 16:16:25,856 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:16:25,856 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:16:18 2023-11-16 16:16:25,856 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:18:23 2023-11-16 16:16:25,857 INFO 1947952:repeater task quick_wh300 starting 2.9 seconds late 2023-11-16 16:16:31,835 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 16:16:31,836 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:16:18 2023-11-16 16:16:31,836 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:18:23 2023-11-16 16:16:31,836 INFO 1947952:repeater task lastfew_vec_wh300 starting 8.8 seconds late 2023-11-16 16:16:32,475 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:16:32,475 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:16:18 2023-11-16 16:16:32,475 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:18:23 2023-11-16 16:16:32,475 INFO 1947952:repeater task 3dayplots_wh300 starting 8.4 seconds late 2023-11-16 16:16:36,521 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.0 seconds 2023-11-16 16:16:36,521 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:21:24 2023-11-16 16:16:36,523 INFO 1947952:repeater task lastensq_os150nb starting 13.5 seconds late 2023-11-16 16:16:39,028 INFO 1947952:repeater task lastensq_os150nb ended after 2.5 seconds 2023-11-16 16:16:39,028 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:16:18 2023-11-16 16:16:39,028 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:21:23 2023-11-16 16:16:39,029 INFO 1947952:repeater task quick_os150nb starting 16.0 seconds late 2023-11-16 16:16:42,911 INFO 1947952:repeater task quick_os150nb ended after 3.9 seconds 2023-11-16 16:16:42,911 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:16:18 2023-11-16 16:16:42,911 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:21:23 2023-11-16 16:16:42,912 INFO 1947952:repeater task lastfew_vec_os150nb starting 19.9 seconds late 2023-11-16 16:16:43,528 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:16:43,528 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:16:18 2023-11-16 16:16:43,528 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:21:23 2023-11-16 16:16:43,529 INFO 1947952:repeater task 3dayplots_os150nb starting 19.5 seconds late 2023-11-16 16:16:46,941 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 16:16:46,941 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:21:24 2023-11-16 16:16:46,943 INFO 1947952:repeater task lastensq_os38nb starting 21.9 seconds late 2023-11-16 16:16:49,301 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-16 16:16:49,302 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:16:20 2023-11-16 16:16:49,302 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:21:25 2023-11-16 16:16:49,302 INFO 1947952:repeater task quick_os38nb starting 24.3 seconds late 2023-11-16 16:16:53,568 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 16:16:53,569 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:16:20 2023-11-16 16:16:53,569 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:21:25 2023-11-16 16:16:53,569 INFO 1947952:repeater task lastfew_vec_os38nb starting 28.6 seconds late 2023-11-16 16:16:54,193 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:16:54,194 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:16:20 2023-11-16 16:16:54,194 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:21:25 2023-11-16 16:16:54,194 INFO 1947952:repeater task 3dayplots_os38nb starting 30.1 seconds late 2023-11-16 16:16:57,715 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 16:16:57,715 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:21:24 2023-11-16 16:16:57,717 INFO 1947952:repeater task daily_netcdf starting 33.7 seconds late 2023-11-16 16:16:58,730 INFO 1947952:repeater task daily_netcdf ended after 1.0 seconds 2023-11-16 16:16:58,730 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 16:26:24 2023-11-16 16:17:03,737 INFO 1947952:repeater task noisy_rawtemps_os150 starting 39.7 seconds late 2023-11-16 16:17:06,561 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-16 16:17:06,562 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 16:26:24 2023-11-16 16:17:06,562 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 42.5 seconds late 2023-11-16 16:17:09,332 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 16:17:09,333 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 16:26:24 2023-11-16 16:17:09,333 INFO 1947952:repeater task noisy_rawtemps_os38 starting 45.3 seconds late 2023-11-16 16:17:12,408 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 16:17:12,409 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 16:26:24 2023-11-16 16:18:27,612 INFO 1947952:repeater task lastensq_wh300 starting 4.6 seconds late 2023-11-16 16:18:30,649 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 16:18:30,649 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:18:18 2023-11-16 16:18:30,650 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:20:23 2023-11-16 16:18:30,650 INFO 1947952:repeater task quick_wh300 starting 7.7 seconds late 2023-11-16 16:18:37,161 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 16:18:37,162 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:18:18 2023-11-16 16:18:37,162 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:20:23 2023-11-16 16:18:37,162 INFO 1947952:repeater task lastfew_vec_wh300 starting 14.2 seconds late 2023-11-16 16:18:37,798 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:18:37,799 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:18:18 2023-11-16 16:18:37,799 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:20:23 2023-11-16 16:20:23,087 INFO 1947952:repeater task lastensq_wh300 starting 0.1 seconds late 2023-11-16 16:20:25,840 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:20:25,840 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:20:18 2023-11-16 16:20:25,840 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:22:23 2023-11-16 16:20:25,841 INFO 1947952:repeater task quick_wh300 starting 2.8 seconds late 2023-11-16 16:20:32,179 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 16:20:32,179 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:20:18 2023-11-16 16:20:32,179 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:22:23 2023-11-16 16:20:32,180 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.2 seconds late 2023-11-16 16:20:32,811 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:20:32,812 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:20:18 2023-11-16 16:20:32,812 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:22:23 2023-11-16 16:21:22,941 INFO 1947952:repeater task plot_tsstats_wh300 starting 3.9 seconds late 2023-11-16 16:21:24,112 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 16:21:24,113 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:24,113 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.1 seconds late 2023-11-16 16:21:24,281 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 16:21:24,281 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:24,282 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.2 seconds late 2023-11-16 16:21:25,601 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 16:21:25,601 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:25,602 INFO 1947952:repeater task beamdiagnostics_wh300 starting 6.6 seconds late 2023-11-16 16:21:26,888 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 16:21:26,888 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:26,888 INFO 1947952:repeater task lastensq_os150nb starting 3.9 seconds late 2023-11-16 16:21:29,141 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 16:21:29,141 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:21:18 2023-11-16 16:21:29,142 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:26:23 2023-11-16 16:21:29,142 INFO 1947952:repeater task quick_os150nb starting 6.1 seconds late 2023-11-16 16:21:32,777 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 16:21:32,777 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:21:18 2023-11-16 16:21:32,777 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:26:23 2023-11-16 16:21:32,777 INFO 1947952:repeater task lastfew_vec_os150nb starting 9.8 seconds late 2023-11-16 16:21:33,391 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:21:33,391 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:21:18 2023-11-16 16:21:33,391 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:26:23 2023-11-16 16:21:33,392 INFO 1947952:repeater task 3dayplots_os150nb starting 9.3 seconds late 2023-11-16 16:21:36,860 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 16:21:36,860 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:26:24 2023-11-16 16:21:36,860 INFO 1947952:repeater task plot_tsstats_os150nb starting 17.8 seconds late 2023-11-16 16:21:37,830 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 16:21:37,831 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:37,831 INFO 1947952:repeater task grab_lastpings_os150nb starting 18.8 seconds late 2023-11-16 16:21:37,999 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 16:21:37,999 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:37,999 INFO 1947952:repeater task beamdiagnostics_os150nb starting 18.9 seconds late 2023-11-16 16:21:39,229 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 16:21:39,229 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:39,230 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.2 seconds late 2023-11-16 16:21:40,511 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 16:21:40,511 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:40,512 INFO 1947952:repeater task lastensq_os38nb starting 15.5 seconds late 2023-11-16 16:21:42,623 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:21:42,624 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:21:20 2023-11-16 16:21:42,624 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:26:25 2023-11-16 16:21:42,624 INFO 1947952:repeater task quick_os38nb starting 17.6 seconds late 2023-11-16 16:21:47,111 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 16:21:47,112 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:21:20 2023-11-16 16:21:47,112 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:26:25 2023-11-16 16:21:47,112 INFO 1947952:repeater task lastfew_vec_os38nb starting 22.1 seconds late 2023-11-16 16:21:47,726 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:21:47,727 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:21:20 2023-11-16 16:21:47,727 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:26:25 2023-11-16 16:21:47,727 INFO 1947952:repeater task 3dayplots_os38nb starting 23.7 seconds late 2023-11-16 16:21:51,248 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 16:21:51,248 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:26:24 2023-11-16 16:21:51,248 INFO 1947952:repeater task plot_tsstats_os38nb starting 32.2 seconds late 2023-11-16 16:21:52,243 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 16:21:52,244 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:52,244 INFO 1947952:repeater task grab_lastpings_os38nb starting 33.2 seconds late 2023-11-16 16:21:52,412 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 16:21:52,413 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:52,413 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.4 seconds late 2023-11-16 16:21:53,583 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 16:21:53,584 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:53,584 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.5 seconds late 2023-11-16 16:21:54,812 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 16:21:54,812 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:54,814 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 35.5 seconds late 2023-11-16 16:21:56,071 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 16:21:56,071 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:56,071 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 36.8 seconds late 2023-11-16 16:21:57,440 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.4 seconds 2023-11-16 16:21:57,441 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:57,441 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 38.2 seconds late 2023-11-16 16:21:58,693 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 16:21:58,693 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:21:58,693 INFO 1947952:repeater task plot_hbins starting 39.4 seconds late 2023-11-16 16:22:02,978 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 16:22:02,978 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 16:22:02,978 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 16:36:19 2023-11-16 16:22:07,986 INFO 1947952:repeater task 3dayplots_wh300 starting 43.9 seconds late 2023-11-16 16:22:11,800 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 16:22:11,801 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:26:24 2023-11-16 16:22:26,840 INFO 1947952:repeater task lastensq_wh300 starting 3.8 seconds late 2023-11-16 16:22:29,965 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 16:22:29,966 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:22:18 2023-11-16 16:22:29,966 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:24:23 2023-11-16 16:22:29,966 INFO 1947952:repeater task quick_wh300 starting 7.0 seconds late 2023-11-16 16:22:36,388 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 16:22:36,389 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:22:18 2023-11-16 16:22:36,389 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:24:23 2023-11-16 16:22:36,389 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.4 seconds late 2023-11-16 16:22:37,033 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:22:37,033 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:22:18 2023-11-16 16:22:37,033 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:24:23 2023-11-16 16:24:27,328 INFO 1947952:repeater task lastensq_wh300 starting 4.3 seconds late 2023-11-16 16:24:30,042 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 16:24:30,042 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:24:18 2023-11-16 16:24:30,042 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:26:23 2023-11-16 16:24:30,043 INFO 1947952:repeater task quick_wh300 starting 7.0 seconds late 2023-11-16 16:24:36,664 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 16:24:36,664 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:24:18 2023-11-16 16:24:36,665 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:26:23 2023-11-16 16:24:36,665 INFO 1947952:repeater task lastfew_vec_wh300 starting 13.7 seconds late 2023-11-16 16:24:37,289 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:24:37,290 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:24:18 2023-11-16 16:24:37,290 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:26:23 2023-11-16 16:26:27,568 INFO 1947952:repeater task noisy_rawtemps_os150 starting 3.5 seconds late 2023-11-16 16:26:30,499 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 16:26:30,500 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 16:36:24 2023-11-16 16:26:30,500 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 6.5 seconds late 2023-11-16 16:26:33,282 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 16:26:33,283 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 16:36:24 2023-11-16 16:26:33,283 INFO 1947952:repeater task noisy_rawtemps_os38 starting 9.2 seconds late 2023-11-16 16:26:36,563 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.3 seconds 2023-11-16 16:26:36,564 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 16:36:24 2023-11-16 16:26:36,564 INFO 1947952:repeater task lastensq_wh300 starting 13.6 seconds late 2023-11-16 16:26:39,169 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 16:26:39,169 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:26:18 2023-11-16 16:26:39,169 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:28:23 2023-11-16 16:26:39,170 INFO 1947952:repeater task quick_wh300 starting 16.2 seconds late 2023-11-16 16:26:45,688 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 16:26:45,688 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:26:18 2023-11-16 16:26:45,688 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:28:23 2023-11-16 16:26:45,688 INFO 1947952:repeater task lastfew_vec_wh300 starting 22.7 seconds late 2023-11-16 16:26:46,314 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:26:46,315 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:26:18 2023-11-16 16:26:46,315 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:28:23 2023-11-16 16:26:46,315 INFO 1947952:repeater task 3dayplots_wh300 starting 22.3 seconds late 2023-11-16 16:26:50,134 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 16:26:50,134 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:31:24 2023-11-16 16:26:50,135 INFO 1947952:repeater task lastensq_os150nb starting 27.1 seconds late 2023-11-16 16:26:52,265 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 16:26:52,266 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:26:18 2023-11-16 16:26:52,266 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:31:23 2023-11-16 16:26:52,266 INFO 1947952:repeater task quick_os150nb starting 29.3 seconds late 2023-11-16 16:26:55,762 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 16:26:55,763 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:26:18 2023-11-16 16:26:55,763 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:31:23 2023-11-16 16:26:55,763 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.8 seconds late 2023-11-16 16:26:56,370 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:26:56,371 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:26:18 2023-11-16 16:26:56,371 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:31:23 2023-11-16 16:26:56,371 INFO 1947952:repeater task 3dayplots_os150nb starting 32.3 seconds late 2023-11-16 16:26:59,788 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 16:26:59,788 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:31:24 2023-11-16 16:26:59,789 INFO 1947952:repeater task lastensq_os38nb starting 34.8 seconds late 2023-11-16 16:27:01,859 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:27:01,859 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:26:20 2023-11-16 16:27:01,860 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:31:25 2023-11-16 16:27:01,860 INFO 1947952:repeater task quick_os38nb starting 36.9 seconds late 2023-11-16 16:27:05,722 INFO 1947952:repeater task quick_os38nb ended after 3.9 seconds 2023-11-16 16:27:05,723 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:26:20 2023-11-16 16:27:05,723 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:31:25 2023-11-16 16:27:05,723 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.7 seconds late 2023-11-16 16:27:06,340 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:27:06,340 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:26:20 2023-11-16 16:27:06,340 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:31:25 2023-11-16 16:27:06,340 INFO 1947952:repeater task 3dayplots_os38nb starting 42.3 seconds late 2023-11-16 16:27:09,772 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 16:27:09,772 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:31:24 2023-11-16 16:27:09,773 INFO 1947952:repeater task daily_netcdf starting 45.7 seconds late 2023-11-16 16:27:10,566 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 16:27:10,567 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 16:36:24 2023-11-16 16:28:25,768 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-16 16:28:28,607 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:28:28,607 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:28:18 2023-11-16 16:28:28,607 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:30:23 2023-11-16 16:28:28,607 INFO 1947952:repeater task quick_wh300 starting 5.6 seconds late 2023-11-16 16:28:34,347 INFO 1947952:repeater task quick_wh300 ended after 5.7 seconds 2023-11-16 16:28:34,348 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:28:18 2023-11-16 16:28:34,348 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:30:23 2023-11-16 16:28:34,348 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2023-11-16 16:28:35,137 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.8 seconds 2023-11-16 16:28:35,138 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:28:18 2023-11-16 16:28:35,138 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:30:23 2023-11-16 16:30:25,432 INFO 1947952:repeater task lastensq_wh300 starting 2.4 seconds late 2023-11-16 16:30:28,029 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 16:30:28,030 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:30:18 2023-11-16 16:30:28,030 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:32:23 2023-11-16 16:30:28,031 INFO 1947952:repeater task quick_wh300 starting 5.0 seconds late 2023-11-16 16:30:34,134 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 16:30:34,134 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:30:18 2023-11-16 16:30:34,134 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:32:23 2023-11-16 16:30:34,135 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-16 16:30:34,917 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.8 seconds 2023-11-16 16:30:34,917 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:30:18 2023-11-16 16:30:34,918 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:32:23 2023-11-16 16:31:25,040 INFO 1947952:repeater task 3dayplots_wh300 starting 1.0 seconds late 2023-11-16 16:31:28,942 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 16:31:28,943 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:36:24 2023-11-16 16:31:28,944 INFO 1947952:repeater task lastensq_os150nb starting 5.9 seconds late 2023-11-16 16:31:31,223 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 16:31:31,223 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:31:18 2023-11-16 16:31:31,224 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:36:23 2023-11-16 16:31:31,224 INFO 1947952:repeater task quick_os150nb starting 8.2 seconds late 2023-11-16 16:31:34,651 INFO 1947952:repeater task quick_os150nb ended after 3.4 seconds 2023-11-16 16:31:34,652 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:31:18 2023-11-16 16:31:34,652 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:36:23 2023-11-16 16:31:34,652 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.7 seconds late 2023-11-16 16:31:35,269 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:31:35,270 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:31:18 2023-11-16 16:31:35,270 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:36:23 2023-11-16 16:31:35,270 INFO 1947952:repeater task 3dayplots_os150nb starting 11.2 seconds late 2023-11-16 16:31:38,726 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 16:31:38,727 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:36:24 2023-11-16 16:31:38,728 INFO 1947952:repeater task lastensq_os38nb starting 13.7 seconds late 2023-11-16 16:31:40,804 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:31:40,804 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:31:20 2023-11-16 16:31:40,804 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:36:25 2023-11-16 16:31:40,805 INFO 1947952:repeater task quick_os38nb starting 15.8 seconds late 2023-11-16 16:31:45,028 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 16:31:45,028 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:31:20 2023-11-16 16:31:45,028 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:36:25 2023-11-16 16:31:45,029 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.0 seconds late 2023-11-16 16:31:45,647 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:31:45,647 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:31:20 2023-11-16 16:31:45,647 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:36:25 2023-11-16 16:31:45,648 INFO 1947952:repeater task 3dayplots_os38nb starting 21.6 seconds late 2023-11-16 16:31:49,058 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 16:31:49,058 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:36:24 2023-11-16 16:32:24,142 INFO 1947952:repeater task lastensq_wh300 starting 1.1 seconds late 2023-11-16 16:32:26,849 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 16:32:26,849 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:32:18 2023-11-16 16:32:26,849 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:34:23 2023-11-16 16:32:26,850 INFO 1947952:repeater task quick_wh300 starting 3.9 seconds late 2023-11-16 16:32:33,076 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 16:32:33,077 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:32:18 2023-11-16 16:32:33,077 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:34:23 2023-11-16 16:32:33,077 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2023-11-16 16:32:33,698 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:32:33,698 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:32:18 2023-11-16 16:32:33,699 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:34:23 2023-11-16 16:34:23,995 INFO 1947952:repeater task lastensq_wh300 starting 1.0 seconds late 2023-11-16 16:34:26,761 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:34:26,761 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:34:18 2023-11-16 16:34:26,762 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:36:23 2023-11-16 16:34:26,762 INFO 1947952:repeater task quick_wh300 starting 3.8 seconds late 2023-11-16 16:34:32,668 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 16:34:32,668 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:34:18 2023-11-16 16:34:32,669 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:36:23 2023-11-16 16:34:32,669 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.7 seconds late 2023-11-16 16:34:33,299 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:34:33,300 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:34:18 2023-11-16 16:34:33,300 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:36:23 2023-11-16 16:36:23,585 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-16 16:36:26,318 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 16:36:26,318 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:36:18 2023-11-16 16:36:26,318 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:38:23 2023-11-16 16:36:26,318 INFO 1947952:repeater task quick_wh300 starting 3.3 seconds late 2023-11-16 16:36:32,911 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 16:36:32,911 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:36:18 2023-11-16 16:36:32,911 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:38:23 2023-11-16 16:36:32,912 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.9 seconds late 2023-11-16 16:36:33,545 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:36:33,546 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:36:18 2023-11-16 16:36:33,546 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:38:23 2023-11-16 16:36:33,546 INFO 1947952:repeater task 3dayplots_wh300 starting 9.5 seconds late 2023-11-16 16:36:37,667 INFO 1947952:repeater task 3dayplots_wh300 ended after 4.1 seconds 2023-11-16 16:36:37,667 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:41:24 2023-11-16 16:36:37,668 INFO 1947952:repeater task plot_tsstats_wh300 starting 18.6 seconds late 2023-11-16 16:36:38,842 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 16:36:38,842 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:38,843 INFO 1947952:repeater task grab_lastpings_wh300 starting 19.8 seconds late 2023-11-16 16:36:39,012 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 16:36:39,013 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:39,013 INFO 1947952:repeater task beamdiagnostics_wh300 starting 20.0 seconds late 2023-11-16 16:36:40,254 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 16:36:40,254 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:40,254 INFO 1947952:repeater task beamdiagnostics_wh300 starting 21.2 seconds late 2023-11-16 16:36:41,544 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 16:36:41,544 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:41,545 INFO 1947952:repeater task lastensq_os150nb starting 18.5 seconds late 2023-11-16 16:36:43,723 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 16:36:43,723 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:36:18 2023-11-16 16:36:43,723 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:41:23 2023-11-16 16:36:43,724 INFO 1947952:repeater task quick_os150nb starting 20.7 seconds late 2023-11-16 16:36:47,451 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 16:36:47,452 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:36:18 2023-11-16 16:36:47,452 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:41:23 2023-11-16 16:36:47,452 INFO 1947952:repeater task lastfew_vec_os150nb starting 24.5 seconds late 2023-11-16 16:36:48,058 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:36:48,058 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:36:18 2023-11-16 16:36:48,058 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:41:23 2023-11-16 16:36:48,058 INFO 1947952:repeater task 3dayplots_os150nb starting 24.0 seconds late 2023-11-16 16:36:51,526 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 16:36:51,526 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:41:24 2023-11-16 16:36:51,526 INFO 1947952:repeater task plot_tsstats_os150nb starting 32.5 seconds late 2023-11-16 16:36:52,501 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 16:36:52,502 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:52,502 INFO 1947952:repeater task grab_lastpings_os150nb starting 33.5 seconds late 2023-11-16 16:36:52,670 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 16:36:52,670 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:52,670 INFO 1947952:repeater task beamdiagnostics_os150nb starting 33.6 seconds late 2023-11-16 16:36:53,906 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 16:36:53,906 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:53,906 INFO 1947952:repeater task beamdiagnostics_os150nb starting 34.9 seconds late 2023-11-16 16:36:55,169 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 16:36:55,170 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:36:55,170 INFO 1947952:repeater task lastensq_os38nb starting 30.2 seconds late 2023-11-16 16:36:57,238 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:36:57,277 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:36:20 2023-11-16 16:36:57,277 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:41:25 2023-11-16 16:36:57,277 INFO 1947952:repeater task quick_os38nb starting 32.3 seconds late 2023-11-16 16:37:01,827 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 16:37:01,828 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:36:20 2023-11-16 16:37:01,828 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:41:25 2023-11-16 16:37:01,828 INFO 1947952:repeater task lastfew_vec_os38nb starting 36.8 seconds late 2023-11-16 16:37:02,448 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:37:02,448 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:36:20 2023-11-16 16:37:02,448 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:41:25 2023-11-16 16:37:02,449 INFO 1947952:repeater task 3dayplots_os38nb starting 38.4 seconds late 2023-11-16 16:37:05,930 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 16:37:05,930 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:41:24 2023-11-16 16:37:05,930 INFO 1947952:repeater task plot_tsstats_os38nb starting 46.9 seconds late 2023-11-16 16:37:06,933 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 16:37:06,933 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:37:06,933 INFO 1947952:repeater task grab_lastpings_os38nb starting 47.9 seconds late 2023-11-16 16:37:07,100 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 16:37:07,100 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:37:07,100 INFO 1947952:repeater task beamdiagnostics_os38nb starting 48.0 seconds late 2023-11-16 16:37:08,306 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 16:37:08,306 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:37:08,307 INFO 1947952:repeater task beamdiagnostics_os38nb starting 49.3 seconds late 2023-11-16 16:37:09,548 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 16:37:09,548 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:37:09,548 INFO 1947952:repeater task daily_netcdf starting 45.5 seconds late 2023-11-16 16:37:10,406 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-16 16:37:10,407 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 16:46:24 2023-11-16 16:37:10,407 INFO 1947952:repeater task ggatime_diagnostics starting 51.4 seconds late 2023-11-16 16:37:26,794 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-16 16:37:26,794 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:37:26,795 INFO 1947952:repeater task ggatime_diagnostics starting 67.7 seconds late 2023-11-16 16:37:42,745 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 16:37:42,746 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:37:42,746 INFO 1947952:repeater task ggatime_diagnostics starting 83.7 seconds late 2023-11-16 16:37:58,434 INFO 1947952:repeater task ggatime_diagnostics ended after 15.7 seconds 2023-11-16 16:37:58,434 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:37:58,435 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 99.2 seconds late 2023-11-16 16:37:59,685 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 16:37:59,685 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:37:59,686 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 100.4 seconds late 2023-11-16 16:38:00,962 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 16:38:00,962 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:38:00,962 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 101.7 seconds late 2023-11-16 16:38:02,243 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 16:38:02,244 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:38:02,244 INFO 1947952:repeater task plot_hbins starting 103.0 seconds late 2023-11-16 16:38:06,611 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 16:38:06,612 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 16:38:06,612 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 16:51:19 2023-11-16 16:38:06,612 INFO 1947952:repeater task report_generator starting 107.3 seconds late 2023-11-16 16:38:12,627 INFO 1947952:repeater task report_generator output: cruisename is report_dir is uhdas_dir is 0.0 sec : png archive rsync 0.0 sec : png archive rsync 0.0 sec : png archive rsync 2.0 sec : navplot: plot_rnav.py -s300 --noshow --title SR2325 --outfile /home/data/SR2325/reports/nav_plot_all /home/data/SR2325/rbin/seapath_udp/*.gps.rbin 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SR2325/reports/db_timerange --noshow /home/data/SR2325 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_topo 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/nav_plot_all_txy 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SR2325/reports /home/data/SR2325/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os150nb/png_archive/ /home/data/SR2325/reports/os150nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/os38nb/png_archive/ /home/data/SR2325/reports/os38nb_pngarchive 0.0 sec : png_archive: rsync -au /home/data/SR2325/proc/wh300/png_archive/ /home/data/SR2325/reports/wh300_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SR2325/reports/uhdas_overview.txt /home/data/SR2325 1.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SR2325/reports/ADCP_settings.txt /home/data/SR2325 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SR2325/reports/ADCP_cals.txt /home/data/SR2325 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_proc.py /home/data/SR2325/reports/SR2325_proc.py.txt 0.0 sec : pycfg: cp -p /home/data/SR2325/raw/config/SR2325_sensor.py /home/data/SR2325/reports/SR2325_sensor.py.txt 5.0 seconds elapsed html page starts here: /home/data/SR2325/reports/index.html 2023-11-16 16:38:12,628 INFO 1947952:repeater task report_generator ended after 6.0 seconds 2023-11-16 16:38:12,628 INFO 1947952:repeater task report_generator next time_to_run is 2023/11/16 17:36:19 2023-11-16 16:38:17,633 INFO 1947952:repeater task noisy_rawtemps_os150 starting 113.6 seconds late 2023-11-16 16:38:20,562 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 16:38:20,562 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 16:46:24 2023-11-16 16:38:20,563 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 116.5 seconds late 2023-11-16 16:38:23,362 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 16:38:23,362 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 16:46:24 2023-11-16 16:38:23,363 INFO 1947952:repeater task noisy_rawtemps_os38 starting 119.3 seconds late 2023-11-16 16:38:26,379 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 16:38:26,380 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 16:46:24 2023-11-16 16:38:26,380 INFO 1947952:repeater task lastensq_wh300 starting 3.4 seconds late 2023-11-16 16:38:28,958 INFO 1947952:repeater task lastensq_wh300 ended after 2.6 seconds 2023-11-16 16:38:28,958 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:38:18 2023-11-16 16:38:28,958 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:40:23 2023-11-16 16:38:28,959 INFO 1947952:repeater task quick_wh300 starting 6.0 seconds late 2023-11-16 16:38:35,253 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 16:38:35,254 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:38:18 2023-11-16 16:38:35,254 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:40:23 2023-11-16 16:38:35,254 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.3 seconds late 2023-11-16 16:38:35,935 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 16:38:35,935 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:38:18 2023-11-16 16:38:35,935 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:40:23 2023-11-16 16:40:26,209 INFO 1947952:repeater task lastensq_wh300 starting 3.2 seconds late 2023-11-16 16:40:28,895 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 16:40:28,895 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:40:18 2023-11-16 16:40:28,895 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:42:23 2023-11-16 16:40:28,896 INFO 1947952:repeater task quick_wh300 starting 5.9 seconds late 2023-11-16 16:40:34,911 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 16:40:34,911 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:40:18 2023-11-16 16:40:34,911 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:42:23 2023-11-16 16:40:34,912 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.9 seconds late 2023-11-16 16:40:35,712 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.8 seconds 2023-11-16 16:40:35,712 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:40:18 2023-11-16 16:40:35,712 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:42:23 2023-11-16 16:41:25,836 INFO 1947952:repeater task 3dayplots_wh300 starting 1.8 seconds late 2023-11-16 16:41:29,687 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 16:41:29,687 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:46:24 2023-11-16 16:41:29,688 INFO 1947952:repeater task lastensq_os150nb starting 6.7 seconds late 2023-11-16 16:41:31,856 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 16:41:31,857 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:41:18 2023-11-16 16:41:31,857 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:46:23 2023-11-16 16:41:31,857 INFO 1947952:repeater task quick_os150nb starting 8.9 seconds late 2023-11-16 16:41:35,292 INFO 1947952:repeater task quick_os150nb ended after 3.4 seconds 2023-11-16 16:41:35,293 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:41:18 2023-11-16 16:41:35,293 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:46:23 2023-11-16 16:41:35,293 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.3 seconds late 2023-11-16 16:41:35,915 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:41:35,915 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:41:18 2023-11-16 16:41:35,915 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:46:23 2023-11-16 16:41:35,916 INFO 1947952:repeater task 3dayplots_os150nb starting 11.9 seconds late 2023-11-16 16:41:39,374 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 16:41:39,374 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:46:24 2023-11-16 16:41:39,375 INFO 1947952:repeater task lastensq_os38nb starting 14.4 seconds late 2023-11-16 16:41:41,445 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:41:41,445 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:41:20 2023-11-16 16:41:41,445 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:46:25 2023-11-16 16:41:41,446 INFO 1947952:repeater task quick_os38nb starting 16.4 seconds late 2023-11-16 16:41:45,785 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 16:41:45,786 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:41:20 2023-11-16 16:41:45,786 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:46:25 2023-11-16 16:41:45,787 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.8 seconds late 2023-11-16 16:41:46,406 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:41:46,406 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:41:20 2023-11-16 16:41:46,406 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:46:25 2023-11-16 16:41:46,407 INFO 1947952:repeater task 3dayplots_os38nb starting 22.4 seconds late 2023-11-16 16:41:49,796 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 16:41:49,796 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:46:24 2023-11-16 16:42:24,884 INFO 1947952:repeater task lastensq_wh300 starting 1.9 seconds late 2023-11-16 16:42:27,896 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 16:42:27,896 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:42:18 2023-11-16 16:42:27,896 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:44:23 2023-11-16 16:42:27,897 INFO 1947952:repeater task quick_wh300 starting 4.9 seconds late 2023-11-16 16:42:33,805 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 16:42:33,805 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:42:18 2023-11-16 16:42:33,805 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:44:23 2023-11-16 16:42:33,805 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2023-11-16 16:42:34,430 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:42:34,430 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:42:18 2023-11-16 16:42:34,430 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:44:23 2023-11-16 16:44:24,728 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-16 16:44:27,581 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 16:44:27,582 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:44:18 2023-11-16 16:44:27,582 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:46:23 2023-11-16 16:44:27,582 INFO 1947952:repeater task quick_wh300 starting 4.6 seconds late 2023-11-16 16:44:34,104 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 16:44:34,104 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:44:18 2023-11-16 16:44:34,104 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:46:23 2023-11-16 16:44:34,104 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-16 16:44:34,732 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:44:34,732 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:44:18 2023-11-16 16:44:34,733 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:46:23 2023-11-16 16:46:25,017 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.0 seconds late 2023-11-16 16:46:27,945 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 16:46:27,945 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 16:56:24 2023-11-16 16:46:27,946 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.9 seconds late 2023-11-16 16:46:30,678 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.7 seconds 2023-11-16 16:46:30,679 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 16:56:24 2023-11-16 16:46:30,679 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.6 seconds late 2023-11-16 16:46:33,720 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 16:46:33,721 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 16:56:24 2023-11-16 16:46:33,721 INFO 1947952:repeater task lastensq_wh300 starting 10.7 seconds late 2023-11-16 16:46:36,563 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:46:36,563 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:46:18 2023-11-16 16:46:36,563 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:48:23 2023-11-16 16:46:36,563 INFO 1947952:repeater task quick_wh300 starting 13.6 seconds late 2023-11-16 16:46:43,403 INFO 1947952:repeater task quick_wh300 ended after 6.8 seconds 2023-11-16 16:46:43,404 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:46:18 2023-11-16 16:46:43,404 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:48:23 2023-11-16 16:46:43,404 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.4 seconds late 2023-11-16 16:46:44,022 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:46:44,023 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:46:18 2023-11-16 16:46:44,023 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:48:23 2023-11-16 16:46:44,023 INFO 1947952:repeater task 3dayplots_wh300 starting 20.0 seconds late 2023-11-16 16:46:47,803 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 16:46:47,803 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:51:24 2023-11-16 16:46:47,804 INFO 1947952:repeater task lastensq_os150nb starting 24.8 seconds late 2023-11-16 16:46:49,881 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 16:46:49,882 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:46:18 2023-11-16 16:46:49,882 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:51:23 2023-11-16 16:46:49,882 INFO 1947952:repeater task quick_os150nb starting 26.9 seconds late 2023-11-16 16:46:53,560 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 16:46:53,561 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:46:18 2023-11-16 16:46:53,561 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:51:23 2023-11-16 16:46:53,561 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.6 seconds late 2023-11-16 16:46:54,163 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:46:54,164 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:46:18 2023-11-16 16:46:54,164 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:51:23 2023-11-16 16:46:54,164 INFO 1947952:repeater task 3dayplots_os150nb starting 30.1 seconds late 2023-11-16 16:46:57,626 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 16:46:57,626 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:51:24 2023-11-16 16:46:57,628 INFO 1947952:repeater task lastensq_os38nb starting 32.6 seconds late 2023-11-16 16:46:59,606 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 16:46:59,606 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:46:20 2023-11-16 16:46:59,606 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:51:25 2023-11-16 16:46:59,607 INFO 1947952:repeater task quick_os38nb starting 34.6 seconds late 2023-11-16 16:47:03,766 INFO 1947952:repeater task quick_os38nb ended after 4.2 seconds 2023-11-16 16:47:03,766 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:46:20 2023-11-16 16:47:03,767 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:51:25 2023-11-16 16:47:03,767 INFO 1947952:repeater task lastfew_vec_os38nb starting 38.8 seconds late 2023-11-16 16:47:04,385 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:47:04,386 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:46:20 2023-11-16 16:47:04,386 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:51:25 2023-11-16 16:47:04,386 INFO 1947952:repeater task 3dayplots_os38nb starting 40.3 seconds late 2023-11-16 16:47:08,040 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.7 seconds 2023-11-16 16:47:08,041 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:51:24 2023-11-16 16:47:08,042 INFO 1947952:repeater task daily_netcdf starting 44.0 seconds late 2023-11-16 16:47:08,924 INFO 1947952:repeater task daily_netcdf ended after 0.9 seconds 2023-11-16 16:47:08,924 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 16:56:24 2023-11-16 16:48:24,096 INFO 1947952:repeater task lastensq_wh300 starting 1.1 seconds late 2023-11-16 16:48:26,937 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:48:26,938 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:48:18 2023-11-16 16:48:26,938 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:50:23 2023-11-16 16:48:26,938 INFO 1947952:repeater task quick_wh300 starting 3.9 seconds late 2023-11-16 16:48:33,054 INFO 1947952:repeater task quick_wh300 ended after 6.1 seconds 2023-11-16 16:48:33,055 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:48:18 2023-11-16 16:48:33,055 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:50:23 2023-11-16 16:48:33,055 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.1 seconds late 2023-11-16 16:48:33,687 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:48:33,688 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:48:18 2023-11-16 16:48:33,688 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:50:23 2023-11-16 16:50:23,992 INFO 1947952:repeater task lastensq_wh300 starting 1.0 seconds late 2023-11-16 16:50:26,709 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 16:50:26,710 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:50:18 2023-11-16 16:50:26,710 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:52:23 2023-11-16 16:50:26,710 INFO 1947952:repeater task quick_wh300 starting 3.7 seconds late 2023-11-16 16:50:32,753 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 16:50:32,754 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:50:18 2023-11-16 16:50:32,754 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:52:23 2023-11-16 16:50:32,754 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.8 seconds late 2023-11-16 16:50:33,392 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:50:33,392 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:50:18 2023-11-16 16:50:33,392 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:52:23 2023-11-16 16:51:23,511 INFO 1947952:repeater task plot_tsstats_wh300 starting 4.5 seconds late 2023-11-16 16:51:24,688 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 16:51:24,688 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:24,688 INFO 1947952:repeater task grab_lastpings_wh300 starting 5.6 seconds late 2023-11-16 16:51:24,857 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 16:51:24,857 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:24,858 INFO 1947952:repeater task beamdiagnostics_wh300 starting 5.8 seconds late 2023-11-16 16:51:26,100 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.2 seconds 2023-11-16 16:51:26,100 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:26,101 INFO 1947952:repeater task beamdiagnostics_wh300 starting 7.1 seconds late 2023-11-16 16:51:27,405 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 16:51:27,405 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:27,406 INFO 1947952:repeater task lastensq_os150nb starting 4.4 seconds late 2023-11-16 16:51:29,561 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 16:51:29,562 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:51:18 2023-11-16 16:51:29,562 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 16:56:23 2023-11-16 16:51:29,562 INFO 1947952:repeater task quick_os150nb starting 6.6 seconds late 2023-11-16 16:51:33,102 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 16:51:33,103 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:51:18 2023-11-16 16:51:33,103 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 16:56:23 2023-11-16 16:51:33,103 INFO 1947952:repeater task lastfew_vec_os150nb starting 10.1 seconds late 2023-11-16 16:51:33,710 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:51:33,710 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:51:18 2023-11-16 16:51:33,710 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 16:56:23 2023-11-16 16:51:33,711 INFO 1947952:repeater task 3dayplots_os150nb starting 9.7 seconds late 2023-11-16 16:51:37,151 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 16:51:37,151 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 16:56:24 2023-11-16 16:51:37,151 INFO 1947952:repeater task plot_tsstats_os150nb starting 18.1 seconds late 2023-11-16 16:51:38,113 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 16:51:38,113 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:38,114 INFO 1947952:repeater task grab_lastpings_os150nb starting 19.1 seconds late 2023-11-16 16:51:38,283 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 16:51:38,283 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:38,283 INFO 1947952:repeater task beamdiagnostics_os150nb starting 19.2 seconds late 2023-11-16 16:51:39,513 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 16:51:39,513 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:39,514 INFO 1947952:repeater task beamdiagnostics_os150nb starting 20.5 seconds late 2023-11-16 16:51:40,760 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 16:51:40,760 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:40,760 INFO 1947952:repeater task lastensq_os38nb starting 15.8 seconds late 2023-11-16 16:51:42,871 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 16:51:42,871 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:51:20 2023-11-16 16:51:42,871 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 16:56:25 2023-11-16 16:51:42,872 INFO 1947952:repeater task quick_os38nb starting 17.9 seconds late 2023-11-16 16:51:46,878 INFO 1947952:repeater task quick_os38nb ended after 4.0 seconds 2023-11-16 16:51:46,878 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:51:20 2023-11-16 16:51:46,878 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 16:56:25 2023-11-16 16:51:46,879 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.9 seconds late 2023-11-16 16:51:47,503 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 16:51:47,503 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:51:20 2023-11-16 16:51:47,503 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 16:56:25 2023-11-16 16:51:47,504 INFO 1947952:repeater task 3dayplots_os38nb starting 23.5 seconds late 2023-11-16 16:51:50,975 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 16:51:50,975 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 16:56:24 2023-11-16 16:51:50,976 INFO 1947952:repeater task plot_tsstats_os38nb starting 31.9 seconds late 2023-11-16 16:51:51,980 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 16:51:51,980 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:51,980 INFO 1947952:repeater task grab_lastpings_os38nb starting 32.9 seconds late 2023-11-16 16:51:52,148 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 16:51:52,148 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:52,148 INFO 1947952:repeater task beamdiagnostics_os38nb starting 33.1 seconds late 2023-11-16 16:51:53,372 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 16:51:53,372 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:53,373 INFO 1947952:repeater task beamdiagnostics_os38nb starting 34.3 seconds late 2023-11-16 16:51:54,598 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 16:51:54,599 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:54,600 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 35.3 seconds late 2023-11-16 16:51:55,901 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 16:51:55,902 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:55,902 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 36.6 seconds late 2023-11-16 16:51:57,176 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 16:51:57,177 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:57,177 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 37.9 seconds late 2023-11-16 16:51:58,443 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 16:51:58,443 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:51:58,444 INFO 1947952:repeater task plot_hbins starting 39.2 seconds late 2023-11-16 16:52:02,836 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 16:52:02,836 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 16:52:02,836 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 17:06:19 2023-11-16 16:52:07,844 INFO 1947952:repeater task 3dayplots_wh300 starting 43.8 seconds late 2023-11-16 16:52:11,654 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 16:52:11,654 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 16:56:24 2023-11-16 16:52:26,688 INFO 1947952:repeater task lastensq_wh300 starting 3.7 seconds late 2023-11-16 16:52:29,530 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 16:52:29,531 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:52:18 2023-11-16 16:52:29,531 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:54:23 2023-11-16 16:52:29,531 INFO 1947952:repeater task quick_wh300 starting 6.5 seconds late 2023-11-16 16:52:35,869 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 16:52:35,869 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:52:18 2023-11-16 16:52:35,869 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:54:23 2023-11-16 16:52:35,872 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 16:52:36,611 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 16:52:36,612 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:52:18 2023-11-16 16:52:36,612 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:54:23 2023-11-16 16:54:26,891 INFO 1947952:repeater task lastensq_wh300 starting 3.9 seconds late 2023-11-16 16:54:29,869 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 16:54:29,870 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:54:18 2023-11-16 16:54:29,870 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:56:23 2023-11-16 16:54:29,870 INFO 1947952:repeater task quick_wh300 starting 6.9 seconds late 2023-11-16 16:54:35,848 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 16:54:35,848 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:54:18 2023-11-16 16:54:35,848 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:56:23 2023-11-16 16:54:35,851 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.9 seconds late 2023-11-16 16:54:36,646 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.8 seconds 2023-11-16 16:54:36,647 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:54:18 2023-11-16 16:54:36,647 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:56:23 2023-11-16 16:56:26,945 INFO 1947952:repeater task noisy_rawtemps_os150 starting 2.9 seconds late 2023-11-16 16:56:29,868 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 16:56:29,868 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 17:06:24 2023-11-16 16:56:29,869 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 5.8 seconds late 2023-11-16 16:56:32,734 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.9 seconds 2023-11-16 16:56:32,735 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 17:06:24 2023-11-16 16:56:32,735 INFO 1947952:repeater task noisy_rawtemps_os38 starting 8.7 seconds late 2023-11-16 16:56:35,788 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 16:56:35,789 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 17:06:24 2023-11-16 16:56:35,789 INFO 1947952:repeater task lastensq_wh300 starting 12.8 seconds late 2023-11-16 16:56:38,698 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 16:56:38,698 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:56:18 2023-11-16 16:56:38,698 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 16:58:23 2023-11-16 16:56:38,699 INFO 1947952:repeater task quick_wh300 starting 15.7 seconds late 2023-11-16 16:56:44,867 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 16:56:44,868 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:56:18 2023-11-16 16:56:44,868 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 16:58:23 2023-11-16 16:56:44,868 INFO 1947952:repeater task lastfew_vec_wh300 starting 21.9 seconds late 2023-11-16 16:56:45,521 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 16:56:45,522 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:56:18 2023-11-16 16:56:45,522 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 16:58:23 2023-11-16 16:56:45,522 INFO 1947952:repeater task 3dayplots_wh300 starting 21.5 seconds late 2023-11-16 16:56:49,337 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 16:56:49,338 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:01:24 2023-11-16 16:56:49,339 INFO 1947952:repeater task lastensq_os150nb starting 26.3 seconds late 2023-11-16 16:56:51,416 INFO 1947952:repeater task lastensq_os150nb ended after 2.1 seconds 2023-11-16 16:56:51,416 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:56:18 2023-11-16 16:56:51,416 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:01:23 2023-11-16 16:56:51,417 INFO 1947952:repeater task quick_os150nb starting 28.4 seconds late 2023-11-16 16:56:55,013 INFO 1947952:repeater task quick_os150nb ended after 3.6 seconds 2023-11-16 16:56:55,014 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:56:18 2023-11-16 16:56:55,014 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:01:23 2023-11-16 16:56:55,014 INFO 1947952:repeater task lastfew_vec_os150nb starting 32.0 seconds late 2023-11-16 16:56:55,621 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 16:56:55,621 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 16:56:18 2023-11-16 16:56:55,621 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:01:23 2023-11-16 16:56:55,622 INFO 1947952:repeater task 3dayplots_os150nb starting 31.6 seconds late 2023-11-16 16:56:59,071 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 16:56:59,071 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:01:24 2023-11-16 16:56:59,072 INFO 1947952:repeater task lastensq_os38nb starting 34.1 seconds late 2023-11-16 16:57:01,108 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 16:57:01,108 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:56:20 2023-11-16 16:57:01,108 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:01:25 2023-11-16 16:57:01,109 INFO 1947952:repeater task quick_os38nb starting 36.1 seconds late 2023-11-16 16:57:05,026 INFO 1947952:repeater task quick_os38nb ended after 3.9 seconds 2023-11-16 16:57:05,027 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:56:20 2023-11-16 16:57:05,027 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:01:25 2023-11-16 16:57:05,027 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.0 seconds late 2023-11-16 16:57:05,679 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.7 seconds 2023-11-16 16:57:05,680 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 16:56:20 2023-11-16 16:57:05,680 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:01:25 2023-11-16 16:57:05,680 INFO 1947952:repeater task 3dayplots_os38nb starting 41.6 seconds late 2023-11-16 16:57:09,287 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.6 seconds 2023-11-16 16:57:09,287 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:01:24 2023-11-16 16:57:09,288 INFO 1947952:repeater task daily_netcdf starting 45.2 seconds late 2023-11-16 16:57:10,273 INFO 1947952:repeater task daily_netcdf ended after 1.0 seconds 2023-11-16 16:57:10,273 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 17:06:24 2023-11-16 16:58:25,480 INFO 1947952:repeater task lastensq_wh300 starting 2.5 seconds late 2023-11-16 16:58:28,380 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 16:58:28,381 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:58:18 2023-11-16 16:58:28,381 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:00:23 2023-11-16 16:58:28,381 INFO 1947952:repeater task quick_wh300 starting 5.4 seconds late 2023-11-16 16:58:34,301 INFO 1947952:repeater task quick_wh300 ended after 5.9 seconds 2023-11-16 16:58:34,301 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:58:18 2023-11-16 16:58:34,301 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:00:23 2023-11-16 16:58:34,302 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.3 seconds late 2023-11-16 16:58:34,938 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 16:58:34,938 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 16:58:18 2023-11-16 16:58:34,938 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:00:23 2023-11-16 17:00:25,234 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-16 17:00:27,983 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:00:27,984 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:00:18 2023-11-16 17:00:27,984 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:02:23 2023-11-16 17:00:27,984 INFO 1947952:repeater task quick_wh300 starting 5.0 seconds late 2023-11-16 17:00:34,016 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 17:00:34,016 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:00:18 2023-11-16 17:00:34,016 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:02:23 2023-11-16 17:00:34,017 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 17:00:34,648 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:00:34,648 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:00:18 2023-11-16 17:00:34,648 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:02:23 2023-11-16 17:01:24,780 INFO 1947952:repeater task 3dayplots_wh300 starting 0.7 seconds late 2023-11-16 17:01:28,652 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 17:01:28,652 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:06:24 2023-11-16 17:01:28,653 INFO 1947952:repeater task lastensq_os150nb starting 5.7 seconds late 2023-11-16 17:01:30,934 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 17:01:30,935 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:01:18 2023-11-16 17:01:30,935 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:06:23 2023-11-16 17:01:30,935 INFO 1947952:repeater task quick_os150nb starting 7.9 seconds late 2023-11-16 17:01:34,402 INFO 1947952:repeater task quick_os150nb ended after 3.5 seconds 2023-11-16 17:01:34,402 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:01:18 2023-11-16 17:01:34,402 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:06:23 2023-11-16 17:01:34,403 INFO 1947952:repeater task lastfew_vec_os150nb starting 11.4 seconds late 2023-11-16 17:01:35,003 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 17:01:35,004 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:01:18 2023-11-16 17:01:35,004 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:06:23 2023-11-16 17:01:35,004 INFO 1947952:repeater task 3dayplots_os150nb starting 11.0 seconds late 2023-11-16 17:01:38,427 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 17:01:38,427 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:06:24 2023-11-16 17:01:38,429 INFO 1947952:repeater task lastensq_os38nb starting 13.4 seconds late 2023-11-16 17:01:40,479 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 17:01:40,479 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:01:20 2023-11-16 17:01:40,479 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:06:25 2023-11-16 17:01:40,479 INFO 1947952:repeater task quick_os38nb starting 15.5 seconds late 2023-11-16 17:01:44,413 INFO 1947952:repeater task quick_os38nb ended after 3.9 seconds 2023-11-16 17:01:44,413 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:01:20 2023-11-16 17:01:44,413 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:06:25 2023-11-16 17:01:44,414 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.4 seconds late 2023-11-16 17:01:45,041 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 17:01:45,042 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:01:20 2023-11-16 17:01:45,042 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:06:25 2023-11-16 17:01:45,042 INFO 1947952:repeater task 3dayplots_os38nb starting 21.0 seconds late 2023-11-16 17:01:48,467 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 17:01:48,467 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:06:24 2023-11-16 17:02:23,550 INFO 1947952:repeater task lastensq_wh300 starting 0.6 seconds late 2023-11-16 17:02:26,232 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:02:26,232 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:02:18 2023-11-16 17:02:26,232 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:04:23 2023-11-16 17:02:26,233 INFO 1947952:repeater task quick_wh300 starting 3.2 seconds late 2023-11-16 17:02:32,722 INFO 1947952:repeater task quick_wh300 ended after 6.5 seconds 2023-11-16 17:02:32,722 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:02:18 2023-11-16 17:02:32,722 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:04:23 2023-11-16 17:02:32,723 INFO 1947952:repeater task lastfew_vec_wh300 starting 9.7 seconds late 2023-11-16 17:02:33,352 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:02:33,352 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:02:18 2023-11-16 17:02:33,352 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:04:23 2023-11-16 17:04:23,652 INFO 1947952:repeater task lastensq_wh300 starting 0.7 seconds late 2023-11-16 17:04:26,547 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 17:04:26,547 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:04:18 2023-11-16 17:04:26,548 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:06:23 2023-11-16 17:04:26,548 INFO 1947952:repeater task quick_wh300 starting 3.5 seconds late 2023-11-16 17:04:33,275 INFO 1947952:repeater task quick_wh300 ended after 6.7 seconds 2023-11-16 17:04:33,275 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:04:18 2023-11-16 17:04:33,275 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:06:23 2023-11-16 17:04:33,276 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.3 seconds late 2023-11-16 17:04:33,898 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:04:33,899 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:04:18 2023-11-16 17:04:33,899 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:06:23 2023-11-16 17:06:19,177 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.1 seconds late 2023-11-16 17:06:20,346 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 17:06:20,346 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:20,346 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.3 seconds late 2023-11-16 17:06:20,514 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 17:06:20,514 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:20,515 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.5 seconds late 2023-11-16 17:06:21,794 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 17:06:21,794 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:21,794 INFO 1947952:repeater task beamdiagnostics_wh300 starting 2.7 seconds late 2023-11-16 17:06:23,073 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 17:06:23,073 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:23,074 INFO 1947952:repeater task lastensq_os150nb starting 0.1 seconds late 2023-11-16 17:06:25,718 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 17:06:25,719 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:06:18 2023-11-16 17:06:25,719 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:11:23 2023-11-16 17:06:25,719 INFO 1947952:repeater task quick_os150nb starting 2.7 seconds late 2023-11-16 17:06:29,917 INFO 1947952:repeater task quick_os150nb ended after 4.2 seconds 2023-11-16 17:06:29,918 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:06:18 2023-11-16 17:06:29,918 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:11:23 2023-11-16 17:06:29,918 INFO 1947952:repeater task lastfew_vec_os150nb starting 6.9 seconds late 2023-11-16 17:06:30,526 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 17:06:30,527 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:06:18 2023-11-16 17:06:30,527 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:11:23 2023-11-16 17:06:30,527 INFO 1947952:repeater task 3dayplots_os150nb starting 6.5 seconds late 2023-11-16 17:06:33,884 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 17:06:33,885 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:11:24 2023-11-16 17:06:33,885 INFO 1947952:repeater task plot_tsstats_os150nb starting 14.8 seconds late 2023-11-16 17:06:34,853 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 17:06:34,853 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:34,854 INFO 1947952:repeater task grab_lastpings_os150nb starting 15.8 seconds late 2023-11-16 17:06:35,021 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 17:06:35,021 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:35,022 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.0 seconds late 2023-11-16 17:06:36,247 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 17:06:36,247 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:36,248 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.2 seconds late 2023-11-16 17:06:37,642 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.4 seconds 2023-11-16 17:06:37,642 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:37,643 INFO 1947952:repeater task lastensq_os38nb starting 12.6 seconds late 2023-11-16 17:06:39,909 INFO 1947952:repeater task lastensq_os38nb ended after 2.3 seconds 2023-11-16 17:06:39,910 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:06:20 2023-11-16 17:06:39,910 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:11:25 2023-11-16 17:06:39,910 INFO 1947952:repeater task quick_os38nb starting 14.9 seconds late 2023-11-16 17:06:44,307 INFO 1947952:repeater task quick_os38nb ended after 4.4 seconds 2023-11-16 17:06:44,307 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:06:20 2023-11-16 17:06:44,308 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:11:25 2023-11-16 17:06:44,308 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.3 seconds late 2023-11-16 17:06:44,930 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 17:06:44,930 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:06:20 2023-11-16 17:06:44,931 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:11:25 2023-11-16 17:06:44,931 INFO 1947952:repeater task 3dayplots_os38nb starting 20.9 seconds late 2023-11-16 17:06:48,396 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 17:06:48,396 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:11:24 2023-11-16 17:06:48,397 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.3 seconds late 2023-11-16 17:06:49,410 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 17:06:49,411 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:49,411 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.4 seconds late 2023-11-16 17:06:49,580 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 17:06:49,580 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:49,581 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.5 seconds late 2023-11-16 17:06:50,852 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.3 seconds 2023-11-16 17:06:50,852 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:50,853 INFO 1947952:repeater task beamdiagnostics_os38nb starting 31.8 seconds late 2023-11-16 17:06:52,099 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 17:06:52,099 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:06:52,100 INFO 1947952:repeater task daily_netcdf starting 28.0 seconds late 2023-11-16 17:06:52,918 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 17:06:52,918 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 17:16:24 2023-11-16 17:06:52,918 INFO 1947952:repeater task ggatime_diagnostics starting 33.9 seconds late 2023-11-16 17:07:09,274 INFO 1947952:repeater task ggatime_diagnostics ended after 16.4 seconds 2023-11-16 17:07:09,274 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:07:09,275 INFO 1947952:repeater task ggatime_diagnostics starting 50.2 seconds late 2023-11-16 17:07:25,301 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 17:07:25,301 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:07:25,301 INFO 1947952:repeater task ggatime_diagnostics starting 66.2 seconds late 2023-11-16 17:07:41,319 INFO 1947952:repeater task ggatime_diagnostics ended after 16.0 seconds 2023-11-16 17:07:41,319 INFO 1947952:repeater task ggatime_diagnostics next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:07:41,319 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 82.0 seconds late 2023-11-16 17:07:42,579 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 17:07:42,579 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:07:42,580 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 83.3 seconds late 2023-11-16 17:07:43,847 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 17:07:43,848 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:07:43,848 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 84.6 seconds late 2023-11-16 17:07:45,112 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 17:07:45,112 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:07:45,113 INFO 1947952:repeater task plot_hbins starting 85.8 seconds late 2023-11-16 17:07:49,435 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 17:07:49,435 INFO 1947952:repeater task plot_hbins ended after 4.3 seconds 2023-11-16 17:07:49,435 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 17:21:19 2023-11-16 17:07:54,441 INFO 1947952:repeater task noisy_rawtemps_os150 starting 90.4 seconds late 2023-11-16 17:07:57,378 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 17:07:57,379 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 17:16:24 2023-11-16 17:07:57,379 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 93.3 seconds late 2023-11-16 17:08:00,188 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 17:08:00,188 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 17:16:24 2023-11-16 17:08:00,188 INFO 1947952:repeater task noisy_rawtemps_os38 starting 96.1 seconds late 2023-11-16 17:08:03,265 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 17:08:03,265 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 17:16:24 2023-11-16 17:08:03,265 INFO 1947952:repeater task lastensq_wh300 starting 100.3 seconds late 2023-11-16 17:08:05,963 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:08:05,963 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:06:18 2023-11-16 17:08:05,963 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:08:23 2023-11-16 17:08:05,964 INFO 1947952:repeater task quick_wh300 starting 103.0 seconds late 2023-11-16 17:08:11,969 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 17:08:11,970 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:06:18 2023-11-16 17:08:11,970 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:08:23 2023-11-16 17:08:11,970 INFO 1947952:repeater task lastfew_vec_wh300 starting 109.0 seconds late 2023-11-16 17:08:12,599 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:08:12,600 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:06:18 2023-11-16 17:08:12,600 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:08:23 2023-11-16 17:08:12,600 INFO 1947952:repeater task 3dayplots_wh300 starting 108.6 seconds late 2023-11-16 17:08:16,341 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 17:08:16,342 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:11:24 2023-11-16 17:08:26,366 INFO 1947952:repeater task lastensq_wh300 starting 3.4 seconds late 2023-11-16 17:08:29,278 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 17:08:29,279 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:08:18 2023-11-16 17:08:29,279 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:10:23 2023-11-16 17:08:29,279 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 17:08:35,567 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 17:08:35,568 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:08:18 2023-11-16 17:08:35,568 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:10:23 2023-11-16 17:08:35,568 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.6 seconds late 2023-11-16 17:08:36,184 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:08:36,185 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:08:18 2023-11-16 17:08:36,185 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:10:23 2023-11-16 17:10:26,500 INFO 1947952:repeater task lastensq_wh300 starting 3.5 seconds late 2023-11-16 17:10:29,251 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 17:10:29,251 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:10:18 2023-11-16 17:10:29,251 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:12:23 2023-11-16 17:10:29,252 INFO 1947952:repeater task quick_wh300 starting 6.3 seconds late 2023-11-16 17:10:35,426 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 17:10:35,426 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:10:18 2023-11-16 17:10:35,426 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:12:23 2023-11-16 17:10:35,427 INFO 1947952:repeater task lastfew_vec_wh300 starting 12.4 seconds late 2023-11-16 17:10:36,051 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:10:36,052 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:10:18 2023-11-16 17:10:36,052 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:12:23 2023-11-16 17:11:26,181 INFO 1947952:repeater task 3dayplots_wh300 starting 2.1 seconds late 2023-11-16 17:11:30,085 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.9 seconds 2023-11-16 17:11:30,085 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:16:24 2023-11-16 17:11:30,087 INFO 1947952:repeater task lastensq_os150nb starting 7.1 seconds late 2023-11-16 17:11:32,403 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 17:11:32,404 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:11:18 2023-11-16 17:11:32,404 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:16:23 2023-11-16 17:11:32,404 INFO 1947952:repeater task quick_os150nb starting 9.4 seconds late 2023-11-16 17:11:36,139 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 17:11:36,139 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:11:18 2023-11-16 17:11:36,140 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:16:23 2023-11-16 17:11:36,140 INFO 1947952:repeater task lastfew_vec_os150nb starting 13.1 seconds late 2023-11-16 17:11:36,748 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 17:11:36,749 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:11:18 2023-11-16 17:11:36,749 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:16:23 2023-11-16 17:11:36,749 INFO 1947952:repeater task 3dayplots_os150nb starting 12.7 seconds late 2023-11-16 17:11:40,205 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.5 seconds 2023-11-16 17:11:40,205 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:16:24 2023-11-16 17:11:40,206 INFO 1947952:repeater task lastensq_os38nb starting 15.2 seconds late 2023-11-16 17:11:42,262 INFO 1947952:repeater task lastensq_os38nb ended after 2.1 seconds 2023-11-16 17:11:42,262 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:11:20 2023-11-16 17:11:42,262 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:16:25 2023-11-16 17:11:42,262 INFO 1947952:repeater task quick_os38nb starting 17.3 seconds late 2023-11-16 17:11:46,564 INFO 1947952:repeater task quick_os38nb ended after 4.3 seconds 2023-11-16 17:11:46,564 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:11:20 2023-11-16 17:11:46,565 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:16:25 2023-11-16 17:11:46,565 INFO 1947952:repeater task lastfew_vec_os38nb starting 21.6 seconds late 2023-11-16 17:11:47,186 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 17:11:47,187 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:11:20 2023-11-16 17:11:47,187 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:16:25 2023-11-16 17:11:47,187 INFO 1947952:repeater task 3dayplots_os38nb starting 23.1 seconds late 2023-11-16 17:11:50,669 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 17:11:50,669 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:16:24 2023-11-16 17:12:25,750 INFO 1947952:repeater task lastensq_wh300 starting 2.8 seconds late 2023-11-16 17:12:28,471 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:12:28,471 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:12:18 2023-11-16 17:12:28,471 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:14:23 2023-11-16 17:12:28,472 INFO 1947952:repeater task quick_wh300 starting 5.5 seconds late 2023-11-16 17:12:34,491 INFO 1947952:repeater task quick_wh300 ended after 6.0 seconds 2023-11-16 17:12:34,492 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:12:18 2023-11-16 17:12:34,492 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:14:23 2023-11-16 17:12:34,492 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.5 seconds late 2023-11-16 17:12:35,120 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:12:35,121 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:12:18 2023-11-16 17:12:35,121 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:14:23 2023-11-16 17:14:25,391 INFO 1947952:repeater task lastensq_wh300 starting 2.4 seconds late 2023-11-16 17:14:28,089 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:14:28,089 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:14:18 2023-11-16 17:14:28,089 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:16:23 2023-11-16 17:14:28,090 INFO 1947952:repeater task quick_wh300 starting 5.1 seconds late 2023-11-16 17:14:34,394 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 17:14:34,394 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:14:18 2023-11-16 17:14:34,394 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:16:23 2023-11-16 17:14:34,395 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-16 17:14:35,024 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:14:35,025 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:14:18 2023-11-16 17:14:35,025 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:16:23 2023-11-16 17:16:25,317 INFO 1947952:repeater task noisy_rawtemps_os150 starting 1.3 seconds late 2023-11-16 17:16:28,214 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.9 seconds 2023-11-16 17:16:28,215 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 17:26:24 2023-11-16 17:16:28,215 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 4.2 seconds late 2023-11-16 17:16:31,033 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 17:16:31,033 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 17:26:24 2023-11-16 17:16:31,034 INFO 1947952:repeater task noisy_rawtemps_os38 starting 7.0 seconds late 2023-11-16 17:16:34,096 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.1 seconds 2023-11-16 17:16:34,096 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 17:26:24 2023-11-16 17:16:34,097 INFO 1947952:repeater task lastensq_wh300 starting 11.1 seconds late 2023-11-16 17:16:36,982 INFO 1947952:repeater task lastensq_wh300 ended after 2.9 seconds 2023-11-16 17:16:36,983 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:16:18 2023-11-16 17:16:36,983 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:18:23 2023-11-16 17:16:36,983 INFO 1947952:repeater task quick_wh300 starting 14.0 seconds late 2023-11-16 17:16:43,575 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 17:16:43,575 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:16:18 2023-11-16 17:16:43,575 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:18:23 2023-11-16 17:16:43,575 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.6 seconds late 2023-11-16 17:16:44,198 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:16:44,199 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:16:18 2023-11-16 17:16:44,199 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:18:23 2023-11-16 17:16:44,199 INFO 1947952:repeater task 3dayplots_wh300 starting 20.1 seconds late 2023-11-16 17:16:47,915 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.7 seconds 2023-11-16 17:16:47,915 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:21:24 2023-11-16 17:16:47,916 INFO 1947952:repeater task lastensq_os150nb starting 24.9 seconds late 2023-11-16 17:16:50,152 INFO 1947952:repeater task lastensq_os150nb ended after 2.2 seconds 2023-11-16 17:16:50,152 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:16:18 2023-11-16 17:16:50,152 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:21:23 2023-11-16 17:16:50,153 INFO 1947952:repeater task quick_os150nb starting 27.2 seconds late 2023-11-16 17:16:53,918 INFO 1947952:repeater task quick_os150nb ended after 3.8 seconds 2023-11-16 17:16:53,918 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:16:18 2023-11-16 17:16:53,918 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:21:23 2023-11-16 17:16:53,919 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.9 seconds late 2023-11-16 17:16:54,531 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 17:16:54,531 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:16:18 2023-11-16 17:16:54,531 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:21:23 2023-11-16 17:16:54,531 INFO 1947952:repeater task 3dayplots_os150nb starting 30.5 seconds late 2023-11-16 17:16:57,894 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 17:16:57,894 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:21:24 2023-11-16 17:16:57,896 INFO 1947952:repeater task lastensq_os38nb starting 32.9 seconds late 2023-11-16 17:17:00,064 INFO 1947952:repeater task lastensq_os38nb ended after 2.2 seconds 2023-11-16 17:17:00,065 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:16:20 2023-11-16 17:17:00,065 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:21:25 2023-11-16 17:17:00,065 INFO 1947952:repeater task quick_os38nb starting 35.1 seconds late 2023-11-16 17:17:04,535 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 17:17:04,536 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:16:20 2023-11-16 17:17:04,536 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:21:25 2023-11-16 17:17:04,536 INFO 1947952:repeater task lastfew_vec_os38nb starting 39.5 seconds late 2023-11-16 17:17:05,160 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 17:17:05,160 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:16:20 2023-11-16 17:17:05,160 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:21:25 2023-11-16 17:17:05,161 INFO 1947952:repeater task 3dayplots_os38nb starting 41.1 seconds late 2023-11-16 17:17:08,613 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 17:17:08,613 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:21:24 2023-11-16 17:17:08,614 INFO 1947952:repeater task daily_netcdf starting 44.6 seconds late 2023-11-16 17:17:09,443 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 17:17:09,444 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 17:26:24 2023-11-16 17:18:24,631 INFO 1947952:repeater task lastensq_wh300 starting 1.6 seconds late 2023-11-16 17:18:27,340 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:18:27,340 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:18:18 2023-11-16 17:18:27,340 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:20:23 2023-11-16 17:18:27,341 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-16 17:18:33,601 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 17:18:33,602 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:18:18 2023-11-16 17:18:33,602 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:20:23 2023-11-16 17:18:33,602 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.6 seconds late 2023-11-16 17:18:34,227 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:18:34,227 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:18:18 2023-11-16 17:18:34,227 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:20:23 2023-11-16 17:20:24,519 INFO 1947952:repeater task lastensq_wh300 starting 1.5 seconds late 2023-11-16 17:20:27,364 INFO 1947952:repeater task lastensq_wh300 ended after 2.8 seconds 2023-11-16 17:20:27,364 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:20:18 2023-11-16 17:20:27,364 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:22:23 2023-11-16 17:20:27,365 INFO 1947952:repeater task quick_wh300 starting 4.4 seconds late 2023-11-16 17:20:33,758 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 17:20:33,759 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:20:18 2023-11-16 17:20:33,759 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:22:23 2023-11-16 17:20:33,759 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.8 seconds late 2023-11-16 17:20:34,381 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:20:34,382 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:20:18 2023-11-16 17:20:34,382 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:22:23 2023-11-16 17:21:19,494 INFO 1947952:repeater task plot_tsstats_wh300 starting 0.4 seconds late 2023-11-16 17:21:20,667 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 17:21:20,667 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:20,668 INFO 1947952:repeater task grab_lastpings_wh300 starting 1.6 seconds late 2023-11-16 17:21:20,844 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds 2023-11-16 17:21:20,844 INFO 1947952:repeater task grab_lastpings_wh300 next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:20,844 INFO 1947952:repeater task beamdiagnostics_wh300 starting 1.8 seconds late 2023-11-16 17:21:22,118 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 17:21:22,118 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:22,118 INFO 1947952:repeater task beamdiagnostics_wh300 starting 3.1 seconds late 2023-11-16 17:21:23,378 INFO 1947952:repeater task beamdiagnostics_wh300 ended after 1.3 seconds 2023-11-16 17:21:23,378 INFO 1947952:repeater task beamdiagnostics_wh300 next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:23,379 INFO 1947952:repeater task lastensq_os150nb starting 0.4 seconds late 2023-11-16 17:21:25,936 INFO 1947952:repeater task lastensq_os150nb ended after 2.6 seconds 2023-11-16 17:21:25,937 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:21:18 2023-11-16 17:21:25,937 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:26:23 2023-11-16 17:21:25,937 INFO 1947952:repeater task quick_os150nb starting 2.9 seconds late 2023-11-16 17:21:30,079 INFO 1947952:repeater task quick_os150nb ended after 4.1 seconds 2023-11-16 17:21:30,080 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:21:18 2023-11-16 17:21:30,080 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:26:23 2023-11-16 17:21:30,080 INFO 1947952:repeater task lastfew_vec_os150nb starting 7.1 seconds late 2023-11-16 17:21:30,690 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 17:21:30,690 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:21:18 2023-11-16 17:21:30,690 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:26:23 2023-11-16 17:21:30,691 INFO 1947952:repeater task 3dayplots_os150nb starting 6.6 seconds late 2023-11-16 17:21:34,042 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 17:21:34,042 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:26:24 2023-11-16 17:21:34,043 INFO 1947952:repeater task plot_tsstats_os150nb starting 15.0 seconds late 2023-11-16 17:21:35,016 INFO 1947952:repeater task plot_tsstats_os150nb ended after 1.0 seconds 2023-11-16 17:21:35,016 INFO 1947952:repeater task plot_tsstats_os150nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:35,016 INFO 1947952:repeater task grab_lastpings_os150nb starting 16.0 seconds late 2023-11-16 17:21:35,183 INFO 1947952:repeater task grab_lastpings_os150nb ended after 0.2 seconds 2023-11-16 17:21:35,183 INFO 1947952:repeater task grab_lastpings_os150nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:35,183 INFO 1947952:repeater task beamdiagnostics_os150nb starting 16.1 seconds late 2023-11-16 17:21:36,421 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.2 seconds 2023-11-16 17:21:36,422 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:36,422 INFO 1947952:repeater task beamdiagnostics_os150nb starting 17.4 seconds late 2023-11-16 17:21:37,686 INFO 1947952:repeater task beamdiagnostics_os150nb ended after 1.3 seconds 2023-11-16 17:21:37,686 INFO 1947952:repeater task beamdiagnostics_os150nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:37,686 INFO 1947952:repeater task lastensq_os38nb starting 12.7 seconds late 2023-11-16 17:21:40,089 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-16 17:21:40,089 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:21:20 2023-11-16 17:21:40,089 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:26:25 2023-11-16 17:21:40,089 INFO 1947952:repeater task quick_os38nb starting 15.1 seconds late 2023-11-16 17:21:44,613 INFO 1947952:repeater task quick_os38nb ended after 4.5 seconds 2023-11-16 17:21:44,613 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:21:20 2023-11-16 17:21:44,613 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:26:25 2023-11-16 17:21:44,613 INFO 1947952:repeater task lastfew_vec_os38nb starting 19.6 seconds late 2023-11-16 17:21:45,235 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 17:21:45,236 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:21:20 2023-11-16 17:21:45,236 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:26:25 2023-11-16 17:21:45,236 INFO 1947952:repeater task 3dayplots_os38nb starting 21.2 seconds late 2023-11-16 17:21:48,638 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 17:21:48,639 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:26:24 2023-11-16 17:21:48,639 INFO 1947952:repeater task plot_tsstats_os38nb starting 29.6 seconds late 2023-11-16 17:21:49,631 INFO 1947952:repeater task plot_tsstats_os38nb ended after 1.0 seconds 2023-11-16 17:21:49,632 INFO 1947952:repeater task plot_tsstats_os38nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:49,632 INFO 1947952:repeater task grab_lastpings_os38nb starting 30.6 seconds late 2023-11-16 17:21:49,798 INFO 1947952:repeater task grab_lastpings_os38nb ended after 0.2 seconds 2023-11-16 17:21:49,798 INFO 1947952:repeater task grab_lastpings_os38nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:49,798 INFO 1947952:repeater task beamdiagnostics_os38nb starting 30.7 seconds late 2023-11-16 17:21:51,031 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 17:21:51,031 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:51,032 INFO 1947952:repeater task beamdiagnostics_os38nb starting 32.0 seconds late 2023-11-16 17:21:52,257 INFO 1947952:repeater task beamdiagnostics_os38nb ended after 1.2 seconds 2023-11-16 17:21:52,257 INFO 1947952:repeater task beamdiagnostics_os38nb next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:52,258 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp starting 33.0 seconds late 2023-11-16 17:21:53,528 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp ended after 1.3 seconds 2023-11-16 17:21:53,528 INFO 1947952:repeater task hbinhcorrstats_hydrins_udp next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:53,528 INFO 1947952:repeater task hbinhcorrstats_trimble_udp starting 34.3 seconds late 2023-11-16 17:21:54,816 INFO 1947952:repeater task hbinhcorrstats_trimble_udp ended after 1.3 seconds 2023-11-16 17:21:54,817 INFO 1947952:repeater task hbinhcorrstats_trimble_udp next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:54,817 INFO 1947952:repeater task hbinhcorrstats_hemisphere starting 35.5 seconds late 2023-11-16 17:21:56,082 INFO 1947952:repeater task hbinhcorrstats_hemisphere ended after 1.3 seconds 2023-11-16 17:21:56,082 INFO 1947952:repeater task hbinhcorrstats_hemisphere next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:21:56,082 INFO 1947952:repeater task plot_hbins starting 36.8 seconds late 2023-11-16 17:22:00,494 INFO 1947952:repeater task plot_hbins output: /usr/local/lib/python3.8/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2023-11-16 17:22:00,494 INFO 1947952:repeater task plot_hbins ended after 4.4 seconds 2023-11-16 17:22:00,494 INFO 1947952:repeater task plot_hbins next time_to_run is 2023/11/16 17:36:19 2023-11-16 17:22:05,500 INFO 1947952:repeater task 3dayplots_wh300 starting 41.4 seconds late 2023-11-16 17:22:09,288 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 17:22:09,288 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:26:24 2023-11-16 17:22:24,322 INFO 1947952:repeater task lastensq_wh300 starting 1.3 seconds late 2023-11-16 17:22:27,336 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 17:22:27,337 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:22:18 2023-11-16 17:22:27,337 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:24:23 2023-11-16 17:22:27,337 INFO 1947952:repeater task quick_wh300 starting 4.3 seconds late 2023-11-16 17:22:33,718 INFO 1947952:repeater task quick_wh300 ended after 6.4 seconds 2023-11-16 17:22:33,718 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:22:18 2023-11-16 17:22:33,718 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:24:23 2023-11-16 17:22:33,718 INFO 1947952:repeater task lastfew_vec_wh300 starting 10.7 seconds late 2023-11-16 17:22:34,338 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:22:34,339 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:22:18 2023-11-16 17:22:34,339 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:24:23 2023-11-16 17:24:24,672 INFO 1947952:repeater task lastensq_wh300 starting 1.7 seconds late 2023-11-16 17:24:27,787 INFO 1947952:repeater task lastensq_wh300 ended after 3.1 seconds 2023-11-16 17:24:27,788 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:24:18 2023-11-16 17:24:27,788 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:26:23 2023-11-16 17:24:27,788 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 17:24:33,988 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 17:24:33,989 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:24:18 2023-11-16 17:24:33,989 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:26:23 2023-11-16 17:24:33,989 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.0 seconds late 2023-11-16 17:24:34,613 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:24:34,613 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:24:18 2023-11-16 17:24:34,614 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:26:23 2023-11-16 17:26:24,912 INFO 1947952:repeater task noisy_rawtemps_os150 starting 0.9 seconds late 2023-11-16 17:26:27,717 INFO 1947952:repeater task noisy_rawtemps_os150 ended after 2.8 seconds 2023-11-16 17:26:27,717 INFO 1947952:repeater task noisy_rawtemps_os150 next time_to_run is 2023/11/16 17:36:24 2023-11-16 17:26:27,717 INFO 1947952:repeater task noisy_rawtemps_wh300 starting 3.7 seconds late 2023-11-16 17:26:30,516 INFO 1947952:repeater task noisy_rawtemps_wh300 ended after 2.8 seconds 2023-11-16 17:26:30,516 INFO 1947952:repeater task noisy_rawtemps_wh300 next time_to_run is 2023/11/16 17:36:24 2023-11-16 17:26:30,517 INFO 1947952:repeater task noisy_rawtemps_os38 starting 6.5 seconds late 2023-11-16 17:26:33,547 INFO 1947952:repeater task noisy_rawtemps_os38 ended after 3.0 seconds 2023-11-16 17:26:33,548 INFO 1947952:repeater task noisy_rawtemps_os38 next time_to_run is 2023/11/16 17:36:24 2023-11-16 17:26:33,548 INFO 1947952:repeater task lastensq_wh300 starting 10.5 seconds late 2023-11-16 17:26:36,090 INFO 1947952:repeater task lastensq_wh300 ended after 2.5 seconds 2023-11-16 17:26:36,091 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:26:18 2023-11-16 17:26:36,091 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:28:23 2023-11-16 17:26:36,091 INFO 1947952:repeater task quick_wh300 starting 13.1 seconds late 2023-11-16 17:26:43,020 INFO 1947952:repeater task quick_wh300 ended after 6.9 seconds 2023-11-16 17:26:43,020 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:26:18 2023-11-16 17:26:43,020 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:28:23 2023-11-16 17:26:43,021 INFO 1947952:repeater task lastfew_vec_wh300 starting 20.0 seconds late 2023-11-16 17:26:43,638 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:26:43,638 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:26:18 2023-11-16 17:26:43,638 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:28:23 2023-11-16 17:26:43,639 INFO 1947952:repeater task 3dayplots_wh300 starting 19.6 seconds late 2023-11-16 17:26:47,398 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 17:26:47,399 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:31:24 2023-11-16 17:26:47,400 INFO 1947952:repeater task lastensq_os150nb starting 24.4 seconds late 2023-11-16 17:26:49,811 INFO 1947952:repeater task lastensq_os150nb ended after 2.4 seconds 2023-11-16 17:26:49,812 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:26:18 2023-11-16 17:26:49,812 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:31:23 2023-11-16 17:26:49,812 INFO 1947952:repeater task quick_os150nb starting 26.8 seconds late 2023-11-16 17:26:53,854 INFO 1947952:repeater task quick_os150nb ended after 4.0 seconds 2023-11-16 17:26:53,854 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:26:18 2023-11-16 17:26:53,854 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:31:23 2023-11-16 17:26:53,855 INFO 1947952:repeater task lastfew_vec_os150nb starting 30.9 seconds late 2023-11-16 17:26:54,465 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 17:26:54,465 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:26:18 2023-11-16 17:26:54,465 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:31:23 2023-11-16 17:26:54,466 INFO 1947952:repeater task 3dayplots_os150nb starting 30.4 seconds late 2023-11-16 17:26:57,905 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 17:26:57,905 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:31:24 2023-11-16 17:26:57,907 INFO 1947952:repeater task lastensq_os38nb starting 32.9 seconds late 2023-11-16 17:27:00,326 INFO 1947952:repeater task lastensq_os38nb ended after 2.4 seconds 2023-11-16 17:27:00,327 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:26:20 2023-11-16 17:27:00,327 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:31:25 2023-11-16 17:27:00,327 INFO 1947952:repeater task quick_os38nb starting 35.3 seconds late 2023-11-16 17:27:05,061 INFO 1947952:repeater task quick_os38nb ended after 4.7 seconds 2023-11-16 17:27:05,061 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:26:20 2023-11-16 17:27:05,061 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:31:25 2023-11-16 17:27:05,062 INFO 1947952:repeater task lastfew_vec_os38nb starting 40.1 seconds late 2023-11-16 17:27:05,685 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 17:27:05,686 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:26:20 2023-11-16 17:27:05,686 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:31:25 2023-11-16 17:27:05,686 INFO 1947952:repeater task 3dayplots_os38nb starting 41.6 seconds late 2023-11-16 17:27:09,175 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.5 seconds 2023-11-16 17:27:09,175 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:31:24 2023-11-16 17:27:09,176 INFO 1947952:repeater task daily_netcdf starting 45.1 seconds late 2023-11-16 17:27:10,001 INFO 1947952:repeater task daily_netcdf ended after 0.8 seconds 2023-11-16 17:27:10,002 INFO 1947952:repeater task daily_netcdf next time_to_run is 2023/11/16 17:36:24 2023-11-16 17:28:25,200 INFO 1947952:repeater task lastensq_wh300 starting 2.2 seconds late 2023-11-16 17:28:27,885 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:28:27,885 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:28:18 2023-11-16 17:28:27,886 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:30:23 2023-11-16 17:28:27,886 INFO 1947952:repeater task quick_wh300 starting 4.9 seconds late 2023-11-16 17:28:34,442 INFO 1947952:repeater task quick_wh300 ended after 6.6 seconds 2023-11-16 17:28:34,442 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:28:18 2023-11-16 17:28:34,442 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:30:23 2023-11-16 17:28:34,443 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.4 seconds late 2023-11-16 17:28:35,055 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:28:35,056 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:28:18 2023-11-16 17:28:35,056 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:30:23 2023-11-16 17:30:25,345 INFO 1947952:repeater task lastensq_wh300 starting 2.3 seconds late 2023-11-16 17:30:28,308 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 17:30:28,308 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:30:18 2023-11-16 17:30:28,308 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:32:23 2023-11-16 17:30:28,309 INFO 1947952:repeater task quick_wh300 starting 5.3 seconds late 2023-11-16 17:30:34,490 INFO 1947952:repeater task quick_wh300 ended after 6.2 seconds 2023-11-16 17:30:34,490 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:30:18 2023-11-16 17:30:34,490 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:32:23 2023-11-16 17:30:34,490 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.5 seconds late 2023-11-16 17:30:35,132 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:30:35,133 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:30:18 2023-11-16 17:30:35,133 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:32:23 2023-11-16 17:31:25,253 INFO 1947952:repeater task 3dayplots_wh300 starting 1.2 seconds late 2023-11-16 17:31:29,083 INFO 1947952:repeater task 3dayplots_wh300 ended after 3.8 seconds 2023-11-16 17:31:29,083 INFO 1947952:repeater task 3dayplots_wh300 next time_to_run is 2023/11/16 17:36:24 2023-11-16 17:31:29,084 INFO 1947952:repeater task lastensq_os150nb starting 6.1 seconds late 2023-11-16 17:31:31,361 INFO 1947952:repeater task lastensq_os150nb ended after 2.3 seconds 2023-11-16 17:31:31,361 INFO 1947952:repeater task lastensq_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:31:18 2023-11-16 17:31:31,361 INFO 1947952:repeater task lastensq_os150nb next time_to_run is 2023/11/16 17:36:23 2023-11-16 17:31:31,362 INFO 1947952:repeater task quick_os150nb starting 8.4 seconds late 2023-11-16 17:31:35,048 INFO 1947952:repeater task quick_os150nb ended after 3.7 seconds 2023-11-16 17:31:35,048 INFO 1947952:repeater task quick_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:31:18 2023-11-16 17:31:35,048 INFO 1947952:repeater task quick_os150nb next time_to_run is 2023/11/16 17:36:23 2023-11-16 17:31:35,048 INFO 1947952:repeater task lastfew_vec_os150nb starting 12.0 seconds late 2023-11-16 17:31:35,654 INFO 1947952:repeater task lastfew_vec_os150nb ended after 0.6 seconds 2023-11-16 17:31:35,655 INFO 1947952:repeater task lastfew_vec_os150nb: Time from /home/data/SR2325/proc/os150nb/load/wrote_ens_time: 2023/11/16 17:31:18 2023-11-16 17:31:35,655 INFO 1947952:repeater task lastfew_vec_os150nb next time_to_run is 2023/11/16 17:36:23 2023-11-16 17:31:35,655 INFO 1947952:repeater task 3dayplots_os150nb starting 11.6 seconds late 2023-11-16 17:31:39,053 INFO 1947952:repeater task 3dayplots_os150nb ended after 3.4 seconds 2023-11-16 17:31:39,054 INFO 1947952:repeater task 3dayplots_os150nb next time_to_run is 2023/11/16 17:36:24 2023-11-16 17:31:39,055 INFO 1947952:repeater task lastensq_os38nb starting 14.1 seconds late 2023-11-16 17:31:41,098 INFO 1947952:repeater task lastensq_os38nb ended after 2.0 seconds 2023-11-16 17:31:41,098 INFO 1947952:repeater task lastensq_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:31:20 2023-11-16 17:31:41,098 INFO 1947952:repeater task lastensq_os38nb next time_to_run is 2023/11/16 17:36:25 2023-11-16 17:31:41,099 INFO 1947952:repeater task quick_os38nb starting 16.1 seconds late 2023-11-16 17:31:45,664 INFO 1947952:repeater task quick_os38nb ended after 4.6 seconds 2023-11-16 17:31:45,664 INFO 1947952:repeater task quick_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:31:20 2023-11-16 17:31:45,664 INFO 1947952:repeater task quick_os38nb next time_to_run is 2023/11/16 17:36:25 2023-11-16 17:31:45,665 INFO 1947952:repeater task lastfew_vec_os38nb starting 20.7 seconds late 2023-11-16 17:31:46,281 INFO 1947952:repeater task lastfew_vec_os38nb ended after 0.6 seconds 2023-11-16 17:31:46,282 INFO 1947952:repeater task lastfew_vec_os38nb: Time from /home/data/SR2325/proc/os38nb/load/wrote_ens_time: 2023/11/16 17:31:20 2023-11-16 17:31:46,282 INFO 1947952:repeater task lastfew_vec_os38nb next time_to_run is 2023/11/16 17:36:25 2023-11-16 17:31:46,282 INFO 1947952:repeater task 3dayplots_os38nb starting 22.2 seconds late 2023-11-16 17:31:49,718 INFO 1947952:repeater task 3dayplots_os38nb ended after 3.4 seconds 2023-11-16 17:31:49,719 INFO 1947952:repeater task 3dayplots_os38nb next time_to_run is 2023/11/16 17:36:24 2023-11-16 17:32:24,810 INFO 1947952:repeater task lastensq_wh300 starting 1.8 seconds late 2023-11-16 17:32:27,839 INFO 1947952:repeater task lastensq_wh300 ended after 3.0 seconds 2023-11-16 17:32:27,839 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:32:18 2023-11-16 17:32:27,839 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:34:23 2023-11-16 17:32:27,839 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 17:32:34,113 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 17:32:34,114 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:32:18 2023-11-16 17:32:34,114 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:34:23 2023-11-16 17:32:34,114 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-16 17:32:34,739 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.6 seconds 2023-11-16 17:32:34,740 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:32:18 2023-11-16 17:32:34,740 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:34:23 2023-11-16 17:34:25,030 INFO 1947952:repeater task lastensq_wh300 starting 2.0 seconds late 2023-11-16 17:34:27,770 INFO 1947952:repeater task lastensq_wh300 ended after 2.7 seconds 2023-11-16 17:34:27,770 INFO 1947952:repeater task lastensq_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:34:18 2023-11-16 17:34:27,771 INFO 1947952:repeater task lastensq_wh300 next time_to_run is 2023/11/16 17:36:23 2023-11-16 17:34:27,771 INFO 1947952:repeater task quick_wh300 starting 4.8 seconds late 2023-11-16 17:34:34,054 INFO 1947952:repeater task quick_wh300 ended after 6.3 seconds 2023-11-16 17:34:34,055 INFO 1947952:repeater task quick_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:34:18 2023-11-16 17:34:34,055 INFO 1947952:repeater task quick_wh300 next time_to_run is 2023/11/16 17:36:23 2023-11-16 17:34:34,055 INFO 1947952:repeater task lastfew_vec_wh300 starting 11.1 seconds late 2023-11-16 17:34:34,719 INFO 1947952:repeater task lastfew_vec_wh300 ended after 0.7 seconds 2023-11-16 17:34:34,720 INFO 1947952:repeater task lastfew_vec_wh300: Time from /home/data/SR2325/proc/wh300/load/wrote_ens_time: 2023/11/16 17:34:18 2023-11-16 17:34:34,720 INFO 1947952:repeater task lastfew_vec_wh300 next time_to_run is 2023/11/16 17:36:23 2023-11-16 17:36:20,018 INFO 1947952:repeater task plot_tsstats_wh300 starting 1.0 seconds late 2023-11-16 17:36:21,203 INFO 1947952:repeater task plot_tsstats_wh300 ended after 1.2 seconds 2023-11-16 17:36:21,204 INFO 1947952:repeater task plot_tsstats_wh300 next time_to_run is 2023/11/16 17:51:19 2023-11-16 17:36:21,204 INFO 1947952:repeater task grab_lastpings_wh300 starting 2.2 seconds late 2023-11-16 17:36:21,374 INFO 1947952:repeater task grab_lastpings_wh300 ended after 0.2 seconds