2025-03-31 20:53:25,997 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 20:53:25,997 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 20:53:25,997 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 21:08:05 2025-03-31 20:53:25,998 INFO 102126:repeater task plotposmv_posmv starting 20.1 seconds late 2025-03-31 20:53:28,168 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-03-31 20:53:28,169 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 21:08:05 2025-03-31 20:53:28,169 INFO 102126:repeater task plot_hbins starting 22.3 seconds late 2025-03-31 20:53:29,249 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 20:53:29,249 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 20:53:29,249 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 21:08:05 2025-03-31 20:53:34,254 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.4 seconds late 2025-03-31 20:53:36,191 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 20:53:36,191 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 21:03:10 2025-03-31 20:53:36,191 INFO 102126:repeater task lastensq_os75nb starting 24.2 seconds late 2025-03-31 20:53:38,341 INFO 102126:repeater task lastensq_os75nb ended after 2.2 seconds 2025-03-31 20:53:38,342 INFO 102126:repeater task lastensq_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 20:53:07 2025-03-31 20:53:38,342 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 20:58:12 2025-03-31 20:53:38,342 INFO 102126:repeater task quick_os75nb starting 26.3 seconds late 2025-03-31 20:53:40,856 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 20:53:40,856 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 20:53:40,856 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 20:53:07 2025-03-31 20:53:40,856 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 20:58:12 2025-03-31 20:53:40,857 INFO 102126:repeater task lastfew_vec_os75nb starting 28.9 seconds late 2025-03-31 20:53:41,483 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 20:53:41,483 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 20:53:07 2025-03-31 20:53:41,483 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 20:58:12 2025-03-31 20:53:41,484 INFO 102126:repeater task 3dayplots_os75nb starting 30.7 seconds late 2025-03-31 20:53:45,850 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 20:53:45,850 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 20:58:10 2025-03-31 20:58:11,081 INFO 102126:repeater task 3dayplots_os75nb starting 0.3 seconds late 2025-03-31 20:58:15,504 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 20:58:15,505 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:03:10 2025-03-31 20:58:20,507 INFO 102126:repeater task lastensq_os75nb starting 8.5 seconds late 2025-03-31 20:58:22,643 INFO 102126:repeater task lastensq_os75nb ended after 2.1 seconds 2025-03-31 20:58:22,643 INFO 102126:repeater task lastensq_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 20:58:07 2025-03-31 20:58:22,643 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:03:12 2025-03-31 20:58:22,644 INFO 102126:repeater task quick_os75nb starting 10.6 seconds late 2025-03-31 20:58:25,198 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 20:58:25,198 INFO 102126:repeater task quick_os75nb ended after 2.6 seconds 2025-03-31 20:58:25,198 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 20:58:07 2025-03-31 20:58:25,198 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:03:12 2025-03-31 20:58:25,199 INFO 102126:repeater task lastfew_vec_os75nb starting 13.2 seconds late 2025-03-31 20:58:25,832 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 20:58:25,832 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 20:58:07 2025-03-31 20:58:25,832 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:03:12 2025-03-31 21:03:11,075 INFO 102126:repeater task noisy_rawtemps_os75 starting 0.3 seconds late 2025-03-31 21:03:13,008 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 21:03:13,008 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 21:13:10 2025-03-31 21:03:13,008 INFO 102126:repeater task lastensq_os75nb starting 1.0 seconds late 2025-03-31 21:03:15,172 INFO 102126:repeater task lastensq_os75nb ended after 2.2 seconds 2025-03-31 21:03:15,172 INFO 102126:repeater task lastensq_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:03:15,172 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:08:12 2025-03-31 21:03:15,172 INFO 102126:repeater task quick_os75nb starting 3.2 seconds late 2025-03-31 21:03:17,689 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:03:17,689 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:03:17,689 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:03:17,689 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:08:12 2025-03-31 21:03:17,689 INFO 102126:repeater task lastfew_vec_os75nb starting 5.7 seconds late 2025-03-31 21:03:18,316 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:03:18,316 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:03:18,316 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:08:12 2025-03-31 21:03:18,316 INFO 102126:repeater task 3dayplots_os75nb starting 7.5 seconds late 2025-03-31 21:03:22,718 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:03:22,718 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:08:10 2025-03-31 21:03:22,718 INFO 102126:repeater task daily_netcdf starting 11.9 seconds late 2025-03-31 21:03:22,908 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 21:03:22,908 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 21:13:10 2025-03-31 21:08:08,161 INFO 102126:repeater task plot_tsstats_os75nb starting 2.3 seconds late 2025-03-31 21:08:08,939 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 21:08:08,939 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 21:08:08,939 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 21:23:05 2025-03-31 21:08:08,939 INFO 102126:repeater task grab_lastpings_os75nb starting 3.1 seconds late 2025-03-31 21:08:09,030 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 21:08:09,030 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 21:23:05 2025-03-31 21:08:09,030 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.2 seconds late 2025-03-31 21:08:10,021 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:08:10,021 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 21:23:05 2025-03-31 21:08:10,021 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.2 seconds late 2025-03-31 21:08:11,015 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:08:11,015 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 21:23:05 2025-03-31 21:08:11,015 INFO 102126:repeater task hbinhcorrstats_posmv starting 5.1 seconds late 2025-03-31 21:08:12,115 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 21:08:12,115 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 21:08:12,115 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 21:23:05 2025-03-31 21:08:12,115 INFO 102126:repeater task plotposmv_posmv starting 6.2 seconds late 2025-03-31 21:08:14,264 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 21:08:14,264 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 21:23:05 2025-03-31 21:08:14,264 INFO 102126:repeater task plot_hbins starting 8.4 seconds late 2025-03-31 21:08:15,337 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 21:08:15,338 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 21:08:15,338 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 21:23:05 2025-03-31 21:08:20,339 INFO 102126:repeater task lastensq_os75nb starting 8.3 seconds late 2025-03-31 21:08:21,019 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:08:21,019 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -306.745 2025-03-31 21:08:21,019 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:13:17 2025-03-31 21:08:21,019 INFO 102126:repeater task quick_os75nb starting 9.0 seconds late 2025-03-31 21:08:23,528 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:08:23,528 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:08:23,528 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:08:23,528 INFO 102126:repeater task quick_os75nb: added 1 extra increments to time_to_run 2025-03-31 21:08:23,528 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:13:17 2025-03-31 21:08:23,529 INFO 102126:repeater task lastfew_vec_os75nb starting 11.5 seconds late 2025-03-31 21:08:24,144 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:08:24,145 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:08:24,145 INFO 102126:repeater task lastfew_vec_os75nb: added 1 extra increments to time_to_run 2025-03-31 21:08:24,145 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:13:17 2025-03-31 21:08:24,145 INFO 102126:repeater task 3dayplots_os75nb starting 13.3 seconds late 2025-03-31 21:08:28,550 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:08:28,550 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:13:10 2025-03-31 21:13:13,797 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.0 seconds late 2025-03-31 21:13:15,742 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 21:13:15,742 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 21:23:10 2025-03-31 21:13:15,742 INFO 102126:repeater task 3dayplots_os75nb starting 4.9 seconds late 2025-03-31 21:13:20,128 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:13:20,128 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:18:10 2025-03-31 21:13:20,128 INFO 102126:repeater task daily_netcdf starting 9.3 seconds late 2025-03-31 21:13:20,318 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 21:13:20,318 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 21:23:10 2025-03-31 21:13:25,324 INFO 102126:repeater task lastensq_os75nb starting 8.3 seconds late 2025-03-31 21:13:26,018 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:13:26,018 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -611.730 2025-03-31 21:13:26,019 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:18:22 2025-03-31 21:13:26,019 INFO 102126:repeater task quick_os75nb starting 9.0 seconds late 2025-03-31 21:13:28,565 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:13:28,565 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:13:28,565 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:13:28,565 INFO 102126:repeater task quick_os75nb: added 2 extra increments to time_to_run 2025-03-31 21:13:28,565 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:18:22 2025-03-31 21:13:28,565 INFO 102126:repeater task lastfew_vec_os75nb starting 11.6 seconds late 2025-03-31 21:13:29,184 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:13:29,184 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:13:29,184 INFO 102126:repeater task lastfew_vec_os75nb: added 2 extra increments to time_to_run 2025-03-31 21:13:29,184 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:18:22 2025-03-31 21:18:14,467 INFO 102126:repeater task 3dayplots_os75nb starting 3.7 seconds late 2025-03-31 21:18:18,855 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:18:18,855 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:23:10 2025-03-31 21:18:23,859 INFO 102126:repeater task lastensq_os75nb starting 1.9 seconds late 2025-03-31 21:18:24,550 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:18:24,550 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -910.265 2025-03-31 21:18:24,550 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:23:27 2025-03-31 21:18:24,551 INFO 102126:repeater task quick_os75nb starting 2.6 seconds late 2025-03-31 21:18:27,140 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:18:27,140 INFO 102126:repeater task quick_os75nb ended after 2.6 seconds 2025-03-31 21:18:27,140 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:18:27,140 INFO 102126:repeater task quick_os75nb: added 3 extra increments to time_to_run 2025-03-31 21:18:27,140 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:23:27 2025-03-31 21:18:27,140 INFO 102126:repeater task lastfew_vec_os75nb starting 5.1 seconds late 2025-03-31 21:18:27,753 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:18:27,753 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:18:27,753 INFO 102126:repeater task lastfew_vec_os75nb: added 3 extra increments to time_to_run 2025-03-31 21:18:27,753 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:23:27 2025-03-31 21:23:08,003 INFO 102126:repeater task plot_tsstats_os75nb starting 2.2 seconds late 2025-03-31 21:23:08,776 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 21:23:08,776 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 21:23:08,776 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 21:38:05 2025-03-31 21:23:08,776 INFO 102126:repeater task grab_lastpings_os75nb starting 3.0 seconds late 2025-03-31 21:23:08,867 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 21:23:08,867 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 21:38:05 2025-03-31 21:23:08,867 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.1 seconds late 2025-03-31 21:23:09,856 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:23:09,856 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 21:38:05 2025-03-31 21:23:09,857 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.0 seconds late 2025-03-31 21:23:10,852 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:23:10,852 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 21:38:05 2025-03-31 21:23:10,852 INFO 102126:repeater task daily_netcdf starting 0.0 seconds late 2025-03-31 21:23:11,041 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 21:23:11,041 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 21:33:10 2025-03-31 21:23:11,041 INFO 102126:repeater task ggatime_diagnostics starting 5.2 seconds late 2025-03-31 21:23:15,030 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-03-31 21:23:15,030 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:23:15,030 INFO 102126:repeater task ggatime_diagnostics starting 9.2 seconds late 2025-03-31 21:23:18,836 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 21:23:18,836 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:23:18,836 INFO 102126:repeater task ggatime_diagnostics starting 13.0 seconds late 2025-03-31 21:23:22,672 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 21:23:22,672 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:23:22,672 INFO 102126:repeater task hbinhcorrstats_posmv starting 16.8 seconds late 2025-03-31 21:23:23,769 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 21:23:23,769 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 21:23:23,769 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 21:38:05 2025-03-31 21:23:23,769 INFO 102126:repeater task plotposmv_posmv starting 17.9 seconds late 2025-03-31 21:23:25,920 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-03-31 21:23:25,920 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 21:38:05 2025-03-31 21:23:25,920 INFO 102126:repeater task plot_hbins starting 20.0 seconds late 2025-03-31 21:23:26,993 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 21:23:26,993 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 21:23:26,993 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 21:38:05 2025-03-31 21:23:26,993 INFO 102126:repeater task report_generator starting 21.1 seconds late 2025-03-31 21:23:30,108 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-03-31 21:23:30,108 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-03-31 21:23:30,108 INFO 102126:repeater task report_generator next time_to_run is 2025/03/31 22:23:05 2025-03-31 21:23:35,111 INFO 102126:repeater task noisy_rawtemps_os75 starting 24.3 seconds late 2025-03-31 21:23:37,037 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 21:23:37,037 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 21:33:10 2025-03-31 21:23:37,038 INFO 102126:repeater task lastensq_os75nb starting 10.0 seconds late 2025-03-31 21:23:37,712 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:23:37,712 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -1223.443 2025-03-31 21:23:37,712 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:28:32 2025-03-31 21:23:37,712 INFO 102126:repeater task quick_os75nb starting 10.7 seconds late 2025-03-31 21:23:40,267 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:23:40,267 INFO 102126:repeater task quick_os75nb ended after 2.6 seconds 2025-03-31 21:23:40,267 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:23:40,267 INFO 102126:repeater task quick_os75nb: added 4 extra increments to time_to_run 2025-03-31 21:23:40,267 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:28:32 2025-03-31 21:23:40,267 INFO 102126:repeater task lastfew_vec_os75nb starting 13.3 seconds late 2025-03-31 21:23:40,888 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:23:40,889 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:23:40,889 INFO 102126:repeater task lastfew_vec_os75nb: added 4 extra increments to time_to_run 2025-03-31 21:23:40,889 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:28:32 2025-03-31 21:23:40,889 INFO 102126:repeater task 3dayplots_os75nb starting 30.1 seconds late 2025-03-31 21:23:45,298 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:23:45,298 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:28:10 2025-03-31 21:28:15,535 INFO 102126:repeater task 3dayplots_os75nb starting 4.7 seconds late 2025-03-31 21:28:19,928 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:28:19,929 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:33:10 2025-03-31 21:28:34,941 INFO 102126:repeater task lastensq_os75nb starting 2.9 seconds late 2025-03-31 21:28:35,638 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:28:35,638 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -1521.347 2025-03-31 21:28:35,638 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:33:37 2025-03-31 21:28:35,638 INFO 102126:repeater task quick_os75nb starting 3.6 seconds late 2025-03-31 21:28:38,151 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:28:38,151 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:28:38,151 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:28:38,151 INFO 102126:repeater task quick_os75nb: added 5 extra increments to time_to_run 2025-03-31 21:28:38,151 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:33:37 2025-03-31 21:28:38,151 INFO 102126:repeater task lastfew_vec_os75nb starting 6.2 seconds late 2025-03-31 21:28:38,771 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:28:38,772 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:28:38,772 INFO 102126:repeater task lastfew_vec_os75nb: added 5 extra increments to time_to_run 2025-03-31 21:28:38,772 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:33:37 2025-03-31 21:33:13,999 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.2 seconds late 2025-03-31 21:33:15,967 INFO 102126:repeater task noisy_rawtemps_os75 ended after 2.0 seconds 2025-03-31 21:33:15,967 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 21:43:10 2025-03-31 21:33:15,967 INFO 102126:repeater task 3dayplots_os75nb starting 5.2 seconds late 2025-03-31 21:33:20,364 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:33:20,364 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:38:10 2025-03-31 21:33:20,365 INFO 102126:repeater task daily_netcdf starting 9.5 seconds late 2025-03-31 21:33:20,555 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 21:33:20,555 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 21:43:10 2025-03-31 21:33:40,575 INFO 102126:repeater task lastensq_os75nb starting 3.6 seconds late 2025-03-31 21:33:41,358 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-03-31 21:33:41,358 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -1826.981 2025-03-31 21:33:41,358 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:38:42 2025-03-31 21:33:41,358 INFO 102126:repeater task quick_os75nb starting 4.4 seconds late 2025-03-31 21:33:43,899 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:33:43,899 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:33:43,899 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:33:43,899 INFO 102126:repeater task quick_os75nb: added 6 extra increments to time_to_run 2025-03-31 21:33:43,899 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:38:42 2025-03-31 21:33:43,899 INFO 102126:repeater task lastfew_vec_os75nb starting 6.9 seconds late 2025-03-31 21:33:44,522 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:33:44,522 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:33:44,522 INFO 102126:repeater task lastfew_vec_os75nb: added 6 extra increments to time_to_run 2025-03-31 21:33:44,522 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:38:42 2025-03-31 21:38:09,775 INFO 102126:repeater task plot_tsstats_os75nb starting 4.0 seconds late 2025-03-31 21:38:10,553 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 21:38:10,553 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 21:38:10,553 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:38:10,553 INFO 102126:repeater task grab_lastpings_os75nb starting 4.7 seconds late 2025-03-31 21:38:10,643 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 21:38:10,643 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:38:10,643 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.8 seconds late 2025-03-31 21:38:11,640 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:38:11,640 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:38:11,640 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.8 seconds late 2025-03-31 21:38:12,644 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:38:12,644 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:38:12,645 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.7 seconds late 2025-03-31 21:38:13,767 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 21:38:13,767 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 21:38:13,767 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:38:13,767 INFO 102126:repeater task plotposmv_posmv starting 7.9 seconds late 2025-03-31 21:38:15,917 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 21:38:15,917 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:38:15,917 INFO 102126:repeater task plot_hbins starting 10.0 seconds late 2025-03-31 21:38:16,991 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 21:38:16,991 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 21:38:16,991 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 21:53:05 2025-03-31 21:38:21,995 INFO 102126:repeater task 3dayplots_os75nb starting 11.2 seconds late 2025-03-31 21:38:26,402 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:38:26,403 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:43:10 2025-03-31 21:38:46,423 INFO 102126:repeater task lastensq_os75nb starting 4.4 seconds late 2025-03-31 21:38:47,121 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:38:47,121 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -2132.828 2025-03-31 21:38:47,121 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:43:47 2025-03-31 21:38:47,122 INFO 102126:repeater task quick_os75nb starting 5.1 seconds late 2025-03-31 21:38:49,636 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:38:49,636 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:38:49,636 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:38:49,636 INFO 102126:repeater task quick_os75nb: added 7 extra increments to time_to_run 2025-03-31 21:38:49,636 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:43:47 2025-03-31 21:38:49,636 INFO 102126:repeater task lastfew_vec_os75nb starting 7.6 seconds late 2025-03-31 21:38:50,248 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:38:50,248 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:38:50,248 INFO 102126:repeater task lastfew_vec_os75nb: added 7 extra increments to time_to_run 2025-03-31 21:38:50,248 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:43:47 2025-03-31 21:43:15,492 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.7 seconds late 2025-03-31 21:43:17,430 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 21:43:17,430 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 21:53:10 2025-03-31 21:43:17,430 INFO 102126:repeater task 3dayplots_os75nb starting 6.6 seconds late 2025-03-31 21:43:21,803 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:43:21,803 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:48:10 2025-03-31 21:43:21,803 INFO 102126:repeater task daily_netcdf starting 11.0 seconds late 2025-03-31 21:43:21,993 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 21:43:21,994 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 21:53:10 2025-03-31 21:43:47,016 INFO 102126:repeater task lastensq_os75nb starting 0.0 seconds late 2025-03-31 21:43:47,699 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:43:47,699 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -2433.422 2025-03-31 21:43:47,700 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:48:52 2025-03-31 21:43:47,700 INFO 102126:repeater task quick_os75nb starting 0.7 seconds late 2025-03-31 21:43:50,218 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:43:50,218 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:43:50,218 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:43:50,218 INFO 102126:repeater task quick_os75nb: added 8 extra increments to time_to_run 2025-03-31 21:43:50,218 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:48:52 2025-03-31 21:43:50,219 INFO 102126:repeater task lastfew_vec_os75nb starting 3.2 seconds late 2025-03-31 21:43:50,828 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:43:50,828 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:43:50,828 INFO 102126:repeater task lastfew_vec_os75nb: added 8 extra increments to time_to_run 2025-03-31 21:43:50,828 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:48:52 2025-03-31 21:48:11,047 INFO 102126:repeater task 3dayplots_os75nb starting 0.2 seconds late 2025-03-31 21:48:15,442 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:48:15,442 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:53:10 2025-03-31 21:48:55,475 INFO 102126:repeater task lastensq_os75nb starting 3.5 seconds late 2025-03-31 21:48:56,258 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-03-31 21:48:56,258 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -2741.881 2025-03-31 21:48:56,258 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:53:57 2025-03-31 21:48:56,258 INFO 102126:repeater task quick_os75nb starting 4.3 seconds late 2025-03-31 21:48:58,774 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:48:58,774 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:48:58,774 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:48:58,774 INFO 102126:repeater task quick_os75nb: added 9 extra increments to time_to_run 2025-03-31 21:48:58,774 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:53:57 2025-03-31 21:48:58,775 INFO 102126:repeater task lastfew_vec_os75nb starting 6.8 seconds late 2025-03-31 21:48:59,393 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:48:59,393 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:48:59,393 INFO 102126:repeater task lastfew_vec_os75nb: added 9 extra increments to time_to_run 2025-03-31 21:48:59,393 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:53:57 2025-03-31 21:53:09,643 INFO 102126:repeater task plot_tsstats_os75nb starting 3.8 seconds late 2025-03-31 21:53:10,421 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 21:53:10,421 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 21:53:10,421 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 22:08:05 2025-03-31 21:53:10,421 INFO 102126:repeater task grab_lastpings_os75nb starting 4.6 seconds late 2025-03-31 21:53:10,511 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 21:53:10,511 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 22:08:05 2025-03-31 21:53:10,512 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.7 seconds late 2025-03-31 21:53:11,508 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:53:11,508 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:08:05 2025-03-31 21:53:11,508 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.7 seconds late 2025-03-31 21:53:12,507 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 21:53:12,508 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:08:05 2025-03-31 21:53:12,508 INFO 102126:repeater task daily_netcdf starting 1.7 seconds late 2025-03-31 21:53:12,697 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 21:53:12,697 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 22:03:10 2025-03-31 21:53:12,697 INFO 102126:repeater task ggatime_diagnostics starting 6.9 seconds late 2025-03-31 21:53:16,662 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-03-31 21:53:16,662 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 22:23:05 2025-03-31 21:53:16,662 INFO 102126:repeater task ggatime_diagnostics starting 10.8 seconds late 2025-03-31 21:53:20,455 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 21:53:20,455 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 22:23:05 2025-03-31 21:53:20,455 INFO 102126:repeater task ggatime_diagnostics starting 14.6 seconds late 2025-03-31 21:53:24,290 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 21:53:24,290 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 22:23:05 2025-03-31 21:53:24,290 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.4 seconds late 2025-03-31 21:53:25,400 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 21:53:25,400 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 21:53:25,400 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 22:08:05 2025-03-31 21:53:25,400 INFO 102126:repeater task plotposmv_posmv starting 19.5 seconds late 2025-03-31 21:53:27,553 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-03-31 21:53:27,553 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 22:08:05 2025-03-31 21:53:27,553 INFO 102126:repeater task plot_hbins starting 21.7 seconds late 2025-03-31 21:53:28,624 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 21:53:28,624 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 21:53:28,624 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 22:08:05 2025-03-31 21:53:33,627 INFO 102126:repeater task noisy_rawtemps_os75 starting 22.8 seconds late 2025-03-31 21:53:35,552 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 21:53:35,552 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 22:03:10 2025-03-31 21:53:35,552 INFO 102126:repeater task 3dayplots_os75nb starting 24.7 seconds late 2025-03-31 21:53:39,958 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:53:39,958 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 21:58:10 2025-03-31 21:53:59,975 INFO 102126:repeater task lastensq_os75nb starting 3.0 seconds late 2025-03-31 21:54:00,667 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:54:00,667 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -3046.381 2025-03-31 21:54:00,667 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 21:59:02 2025-03-31 21:54:00,668 INFO 102126:repeater task quick_os75nb starting 3.7 seconds late 2025-03-31 21:54:03,180 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:54:03,180 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:54:03,180 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:54:03,180 INFO 102126:repeater task quick_os75nb: added 10 extra increments to time_to_run 2025-03-31 21:54:03,180 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 21:59:02 2025-03-31 21:54:03,180 INFO 102126:repeater task lastfew_vec_os75nb starting 6.2 seconds late 2025-03-31 21:54:03,791 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:54:03,791 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:54:03,791 INFO 102126:repeater task lastfew_vec_os75nb: added 10 extra increments to time_to_run 2025-03-31 21:54:03,791 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 21:59:02 2025-03-31 21:58:14,033 INFO 102126:repeater task 3dayplots_os75nb starting 3.2 seconds late 2025-03-31 21:58:18,434 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 21:58:18,434 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:03:10 2025-03-31 21:59:03,471 INFO 102126:repeater task lastensq_os75nb starting 1.5 seconds late 2025-03-31 21:59:04,168 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 21:59:04,168 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -3349.877 2025-03-31 21:59:04,168 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:04:07 2025-03-31 21:59:04,168 INFO 102126:repeater task quick_os75nb starting 2.2 seconds late 2025-03-31 21:59:06,677 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 21:59:06,677 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 21:59:06,677 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:59:06,677 INFO 102126:repeater task quick_os75nb: added 11 extra increments to time_to_run 2025-03-31 21:59:06,677 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:04:07 2025-03-31 21:59:06,677 INFO 102126:repeater task lastfew_vec_os75nb starting 4.7 seconds late 2025-03-31 21:59:07,290 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 21:59:07,290 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 21:59:07,291 INFO 102126:repeater task lastfew_vec_os75nb: added 11 extra increments to time_to_run 2025-03-31 21:59:07,291 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:04:07 2025-03-31 22:03:12,515 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.7 seconds late 2025-03-31 22:03:14,461 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 22:03:14,461 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 22:13:10 2025-03-31 22:03:14,461 INFO 102126:repeater task 3dayplots_os75nb starting 3.6 seconds late 2025-03-31 22:03:18,876 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:03:18,876 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:08:10 2025-03-31 22:03:18,876 INFO 102126:repeater task daily_netcdf starting 8.1 seconds late 2025-03-31 22:03:19,066 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 22:03:19,067 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 22:13:10 2025-03-31 22:04:09,113 INFO 102126:repeater task lastensq_os75nb starting 2.1 seconds late 2025-03-31 22:04:09,811 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:04:09,811 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -3655.519 2025-03-31 22:04:09,811 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:09:12 2025-03-31 22:04:09,811 INFO 102126:repeater task quick_os75nb starting 2.8 seconds late 2025-03-31 22:04:12,326 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:04:12,326 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:04:12,326 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:04:12,326 INFO 102126:repeater task quick_os75nb: added 12 extra increments to time_to_run 2025-03-31 22:04:12,326 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:09:12 2025-03-31 22:04:12,327 INFO 102126:repeater task lastfew_vec_os75nb starting 5.3 seconds late 2025-03-31 22:04:12,937 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:04:12,938 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:04:12,938 INFO 102126:repeater task lastfew_vec_os75nb: added 12 extra increments to time_to_run 2025-03-31 22:04:12,938 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:09:12 2025-03-31 22:08:08,135 INFO 102126:repeater task plot_tsstats_os75nb starting 2.3 seconds late 2025-03-31 22:08:08,903 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 22:08:08,903 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 22:08:08,903 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 22:23:05 2025-03-31 22:08:08,903 INFO 102126:repeater task grab_lastpings_os75nb starting 3.1 seconds late 2025-03-31 22:08:08,994 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 22:08:08,994 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 22:23:05 2025-03-31 22:08:08,994 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.2 seconds late 2025-03-31 22:08:09,986 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:08:09,986 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:23:05 2025-03-31 22:08:09,986 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.2 seconds late 2025-03-31 22:08:10,985 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:08:10,985 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:23:05 2025-03-31 22:08:10,985 INFO 102126:repeater task hbinhcorrstats_posmv starting 5.1 seconds late 2025-03-31 22:08:12,103 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 22:08:12,103 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 22:08:12,103 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 22:23:05 2025-03-31 22:08:12,104 INFO 102126:repeater task plotposmv_posmv starting 6.2 seconds late 2025-03-31 22:08:14,209 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 22:08:14,210 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 22:23:05 2025-03-31 22:08:14,210 INFO 102126:repeater task plot_hbins starting 8.3 seconds late 2025-03-31 22:08:15,264 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 22:08:15,264 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 22:08:15,264 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 22:23:05 2025-03-31 22:08:20,270 INFO 102126:repeater task 3dayplots_os75nb starting 9.5 seconds late 2025-03-31 22:08:24,671 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:08:24,671 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:13:10 2025-03-31 22:09:14,715 INFO 102126:repeater task lastensq_os75nb starting 2.7 seconds late 2025-03-31 22:09:15,390 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:09:15,390 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -3961.121 2025-03-31 22:09:15,390 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:14:17 2025-03-31 22:09:15,390 INFO 102126:repeater task quick_os75nb starting 3.4 seconds late 2025-03-31 22:09:17,905 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:09:17,905 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:09:17,906 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:09:17,906 INFO 102126:repeater task quick_os75nb: added 13 extra increments to time_to_run 2025-03-31 22:09:17,906 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:14:17 2025-03-31 22:09:17,906 INFO 102126:repeater task lastfew_vec_os75nb starting 5.9 seconds late 2025-03-31 22:09:18,521 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:09:18,521 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:09:18,522 INFO 102126:repeater task lastfew_vec_os75nb: added 13 extra increments to time_to_run 2025-03-31 22:09:18,522 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:14:17 2025-03-31 22:13:13,727 INFO 102126:repeater task noisy_rawtemps_os75 starting 2.9 seconds late 2025-03-31 22:13:15,674 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 22:13:15,674 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 22:23:10 2025-03-31 22:13:15,674 INFO 102126:repeater task 3dayplots_os75nb starting 4.9 seconds late 2025-03-31 22:13:20,069 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:13:20,069 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:18:10 2025-03-31 22:13:20,070 INFO 102126:repeater task daily_netcdf starting 9.3 seconds late 2025-03-31 22:13:20,259 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 22:13:20,259 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 22:23:10 2025-03-31 22:14:20,311 INFO 102126:repeater task lastensq_os75nb starting 3.3 seconds late 2025-03-31 22:14:21,131 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-03-31 22:14:21,131 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -4266.717 2025-03-31 22:14:21,131 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:19:22 2025-03-31 22:14:21,132 INFO 102126:repeater task quick_os75nb starting 4.1 seconds late 2025-03-31 22:14:23,642 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:14:23,642 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:14:23,643 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:14:23,643 INFO 102126:repeater task quick_os75nb: added 14 extra increments to time_to_run 2025-03-31 22:14:23,643 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:19:22 2025-03-31 22:14:23,643 INFO 102126:repeater task lastfew_vec_os75nb starting 6.6 seconds late 2025-03-31 22:14:24,254 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:14:24,254 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:14:24,254 INFO 102126:repeater task lastfew_vec_os75nb: added 14 extra increments to time_to_run 2025-03-31 22:14:24,254 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:19:22 2025-03-31 22:18:14,469 INFO 102126:repeater task 3dayplots_os75nb starting 3.7 seconds late 2025-03-31 22:18:18,858 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:18:18,858 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:23:10 2025-03-31 22:19:23,915 INFO 102126:repeater task lastensq_os75nb starting 1.9 seconds late 2025-03-31 22:19:24,585 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:19:24,585 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -4570.321 2025-03-31 22:19:24,585 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:24:27 2025-03-31 22:19:24,585 INFO 102126:repeater task quick_os75nb starting 2.6 seconds late 2025-03-31 22:19:27,100 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:19:27,100 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:19:27,100 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:19:27,100 INFO 102126:repeater task quick_os75nb: added 15 extra increments to time_to_run 2025-03-31 22:19:27,100 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:24:27 2025-03-31 22:19:27,100 INFO 102126:repeater task lastfew_vec_os75nb starting 5.1 seconds late 2025-03-31 22:19:27,720 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:19:27,721 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:19:27,721 INFO 102126:repeater task lastfew_vec_os75nb: added 15 extra increments to time_to_run 2025-03-31 22:19:27,721 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:24:27 2025-03-31 22:23:07,921 INFO 102126:repeater task plot_tsstats_os75nb starting 2.1 seconds late 2025-03-31 22:23:08,689 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 22:23:08,689 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 22:23:08,689 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 22:38:05 2025-03-31 22:23:08,689 INFO 102126:repeater task grab_lastpings_os75nb starting 2.9 seconds late 2025-03-31 22:23:08,779 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 22:23:08,779 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 22:38:05 2025-03-31 22:23:08,780 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.0 seconds late 2025-03-31 22:23:09,770 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:23:09,770 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:38:05 2025-03-31 22:23:09,771 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.0 seconds late 2025-03-31 22:23:10,766 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:23:10,766 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:38:05 2025-03-31 22:23:10,767 INFO 102126:repeater task ggatime_diagnostics starting 4.9 seconds late 2025-03-31 22:23:14,745 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-03-31 22:23:14,745 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:23:14,746 INFO 102126:repeater task ggatime_diagnostics starting 8.9 seconds late 2025-03-31 22:23:18,525 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 22:23:18,525 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:23:18,525 INFO 102126:repeater task ggatime_diagnostics starting 12.7 seconds late 2025-03-31 22:23:22,339 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 22:23:22,340 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:23:22,340 INFO 102126:repeater task hbinhcorrstats_posmv starting 16.4 seconds late 2025-03-31 22:23:23,448 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 22:23:23,449 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 22:23:23,449 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 22:38:05 2025-03-31 22:23:23,449 INFO 102126:repeater task plotposmv_posmv starting 17.5 seconds late 2025-03-31 22:23:25,563 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 22:23:25,563 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 22:38:05 2025-03-31 22:23:25,563 INFO 102126:repeater task plot_hbins starting 19.7 seconds late 2025-03-31 22:23:26,631 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 22:23:26,631 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 22:23:26,631 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 22:38:05 2025-03-31 22:23:26,631 INFO 102126:repeater task report_generator starting 20.7 seconds late 2025-03-31 22:23:29,742 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-03-31 22:23:29,743 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-03-31 22:23:29,743 INFO 102126:repeater task report_generator next time_to_run is 2025/03/31 23:23:05 2025-03-31 22:23:34,748 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.9 seconds late 2025-03-31 22:23:36,708 INFO 102126:repeater task noisy_rawtemps_os75 ended after 2.0 seconds 2025-03-31 22:23:36,708 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 22:33:10 2025-03-31 22:23:36,709 INFO 102126:repeater task 3dayplots_os75nb starting 25.9 seconds late 2025-03-31 22:23:41,120 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:23:41,120 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:28:10 2025-03-31 22:23:41,121 INFO 102126:repeater task daily_netcdf starting 30.3 seconds late 2025-03-31 22:23:41,310 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 22:23:41,310 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 22:33:10 2025-03-31 22:24:31,351 INFO 102126:repeater task lastensq_os75nb starting 4.4 seconds late 2025-03-31 22:24:32,027 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:24:32,027 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -4877.757 2025-03-31 22:24:32,027 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:29:32 2025-03-31 22:24:32,027 INFO 102126:repeater task quick_os75nb starting 5.0 seconds late 2025-03-31 22:24:34,542 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:24:34,542 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:24:34,542 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:24:34,542 INFO 102126:repeater task quick_os75nb: added 16 extra increments to time_to_run 2025-03-31 22:24:34,542 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:29:32 2025-03-31 22:24:34,542 INFO 102126:repeater task lastfew_vec_os75nb starting 7.5 seconds late 2025-03-31 22:24:35,163 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:24:35,164 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:24:35,164 INFO 102126:repeater task lastfew_vec_os75nb: added 16 extra increments to time_to_run 2025-03-31 22:24:35,164 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:29:32 2025-03-31 22:28:15,367 INFO 102126:repeater task 3dayplots_os75nb starting 4.6 seconds late 2025-03-31 22:28:19,763 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:28:19,763 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:33:10 2025-03-31 22:29:34,831 INFO 102126:repeater task lastensq_os75nb starting 2.8 seconds late 2025-03-31 22:29:35,577 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:29:35,577 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -5181.237 2025-03-31 22:29:35,577 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:34:37 2025-03-31 22:29:35,577 INFO 102126:repeater task quick_os75nb starting 3.6 seconds late 2025-03-31 22:29:38,088 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:29:38,088 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:29:38,088 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:29:38,088 INFO 102126:repeater task quick_os75nb: added 17 extra increments to time_to_run 2025-03-31 22:29:38,088 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:34:37 2025-03-31 22:29:38,088 INFO 102126:repeater task lastfew_vec_os75nb starting 6.1 seconds late 2025-03-31 22:29:38,711 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:29:38,711 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:29:38,711 INFO 102126:repeater task lastfew_vec_os75nb: added 17 extra increments to time_to_run 2025-03-31 22:29:38,711 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:34:37 2025-03-31 22:33:13,907 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.1 seconds late 2025-03-31 22:33:15,846 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 22:33:15,846 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 22:43:10 2025-03-31 22:33:15,847 INFO 102126:repeater task 3dayplots_os75nb starting 5.0 seconds late 2025-03-31 22:33:20,247 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:33:20,247 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:38:10 2025-03-31 22:33:20,248 INFO 102126:repeater task daily_netcdf starting 9.4 seconds late 2025-03-31 22:33:20,437 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 22:33:20,437 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 22:43:10 2025-03-31 22:34:40,519 INFO 102126:repeater task lastensq_os75nb starting 3.5 seconds late 2025-03-31 22:34:41,210 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:34:41,210 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -5486.925 2025-03-31 22:34:41,210 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:39:42 2025-03-31 22:34:41,210 INFO 102126:repeater task quick_os75nb starting 4.2 seconds late 2025-03-31 22:34:43,723 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:34:43,723 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:34:43,723 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:34:43,723 INFO 102126:repeater task quick_os75nb: added 18 extra increments to time_to_run 2025-03-31 22:34:43,723 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:39:42 2025-03-31 22:34:43,723 INFO 102126:repeater task lastfew_vec_os75nb starting 6.7 seconds late 2025-03-31 22:34:44,334 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:34:44,334 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:34:44,334 INFO 102126:repeater task lastfew_vec_os75nb: added 18 extra increments to time_to_run 2025-03-31 22:34:44,334 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:39:42 2025-03-31 22:38:09,530 INFO 102126:repeater task plot_tsstats_os75nb starting 3.7 seconds late 2025-03-31 22:38:10,301 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 22:38:10,301 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 22:38:10,301 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:38:10,301 INFO 102126:repeater task grab_lastpings_os75nb starting 4.5 seconds late 2025-03-31 22:38:10,392 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 22:38:10,392 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:38:10,392 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.6 seconds late 2025-03-31 22:38:11,381 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:38:11,381 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:38:11,381 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.6 seconds late 2025-03-31 22:38:12,380 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:38:12,380 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:38:12,380 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.5 seconds late 2025-03-31 22:38:13,506 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 22:38:13,506 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 22:38:13,506 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:38:13,507 INFO 102126:repeater task plotposmv_posmv starting 7.6 seconds late 2025-03-31 22:38:15,632 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 22:38:15,632 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:38:15,632 INFO 102126:repeater task plot_hbins starting 9.7 seconds late 2025-03-31 22:38:16,703 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 22:38:16,703 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 22:38:16,703 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 22:53:05 2025-03-31 22:38:21,709 INFO 102126:repeater task 3dayplots_os75nb starting 10.9 seconds late 2025-03-31 22:38:26,108 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:38:26,109 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:43:10 2025-03-31 22:39:46,177 INFO 102126:repeater task lastensq_os75nb starting 4.2 seconds late 2025-03-31 22:39:46,861 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:39:46,861 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -5792.582 2025-03-31 22:39:46,861 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:44:47 2025-03-31 22:39:46,861 INFO 102126:repeater task quick_os75nb starting 4.9 seconds late 2025-03-31 22:39:49,370 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:39:49,370 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:39:49,370 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:39:49,370 INFO 102126:repeater task quick_os75nb: added 19 extra increments to time_to_run 2025-03-31 22:39:49,370 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:44:47 2025-03-31 22:39:49,370 INFO 102126:repeater task lastfew_vec_os75nb starting 7.4 seconds late 2025-03-31 22:39:49,979 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:39:49,980 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:39:49,980 INFO 102126:repeater task lastfew_vec_os75nb: added 19 extra increments to time_to_run 2025-03-31 22:39:49,980 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:44:47 2025-03-31 22:43:15,161 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.3 seconds late 2025-03-31 22:43:17,098 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 22:43:17,099 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 22:53:10 2025-03-31 22:43:17,099 INFO 102126:repeater task 3dayplots_os75nb starting 6.3 seconds late 2025-03-31 22:43:21,521 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:43:21,522 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:48:10 2025-03-31 22:43:21,522 INFO 102126:repeater task daily_netcdf starting 10.7 seconds late 2025-03-31 22:43:21,712 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 22:43:21,713 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 22:53:10 2025-03-31 22:44:51,779 INFO 102126:repeater task lastensq_os75nb starting 4.8 seconds late 2025-03-31 22:44:52,466 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:44:52,466 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -6098.185 2025-03-31 22:44:52,466 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:49:52 2025-03-31 22:44:52,466 INFO 102126:repeater task quick_os75nb starting 5.5 seconds late 2025-03-31 22:44:54,981 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:44:54,981 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:44:54,982 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:44:54,982 INFO 102126:repeater task quick_os75nb: added 20 extra increments to time_to_run 2025-03-31 22:44:54,982 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:49:52 2025-03-31 22:44:54,982 INFO 102126:repeater task lastfew_vec_os75nb starting 8.0 seconds late 2025-03-31 22:44:55,593 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:44:55,594 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:44:55,594 INFO 102126:repeater task lastfew_vec_os75nb: added 20 extra increments to time_to_run 2025-03-31 22:44:55,594 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:49:52 2025-03-31 22:48:15,791 INFO 102126:repeater task 3dayplots_os75nb starting 5.0 seconds late 2025-03-31 22:48:20,199 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:48:20,199 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:53:10 2025-03-31 22:49:55,293 INFO 102126:repeater task lastensq_os75nb starting 3.3 seconds late 2025-03-31 22:49:55,979 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:49:55,979 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -6401.698 2025-03-31 22:49:55,979 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 22:54:57 2025-03-31 22:49:55,980 INFO 102126:repeater task quick_os75nb starting 4.0 seconds late 2025-03-31 22:49:58,498 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:49:58,498 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:49:58,498 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:49:58,498 INFO 102126:repeater task quick_os75nb: added 21 extra increments to time_to_run 2025-03-31 22:49:58,498 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 22:54:57 2025-03-31 22:49:58,498 INFO 102126:repeater task lastfew_vec_os75nb starting 6.5 seconds late 2025-03-31 22:49:59,107 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:49:59,108 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:49:59,108 INFO 102126:repeater task lastfew_vec_os75nb: added 21 extra increments to time_to_run 2025-03-31 22:49:59,108 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 22:54:57 2025-03-31 22:53:09,293 INFO 102126:repeater task plot_tsstats_os75nb starting 3.5 seconds late 2025-03-31 22:53:10,063 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 22:53:10,063 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 22:53:10,063 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 23:08:05 2025-03-31 22:53:10,063 INFO 102126:repeater task grab_lastpings_os75nb starting 4.2 seconds late 2025-03-31 22:53:10,153 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 22:53:10,153 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 23:08:05 2025-03-31 22:53:10,154 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.3 seconds late 2025-03-31 22:53:11,141 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:53:11,141 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:08:05 2025-03-31 22:53:11,141 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.3 seconds late 2025-03-31 22:53:12,138 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 22:53:12,139 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:08:05 2025-03-31 22:53:12,139 INFO 102126:repeater task daily_netcdf starting 1.3 seconds late 2025-03-31 22:53:12,329 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 22:53:12,329 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 23:03:10 2025-03-31 22:53:12,329 INFO 102126:repeater task ggatime_diagnostics starting 6.5 seconds late 2025-03-31 22:53:16,311 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-03-31 22:53:16,312 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 23:23:05 2025-03-31 22:53:16,312 INFO 102126:repeater task ggatime_diagnostics starting 10.5 seconds late 2025-03-31 22:53:20,121 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 22:53:20,121 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 23:23:05 2025-03-31 22:53:20,121 INFO 102126:repeater task ggatime_diagnostics starting 14.3 seconds late 2025-03-31 22:53:24,011 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-03-31 22:53:24,011 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 23:23:05 2025-03-31 22:53:24,011 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.1 seconds late 2025-03-31 22:53:25,129 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 22:53:25,130 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 22:53:25,130 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 23:08:05 2025-03-31 22:53:25,130 INFO 102126:repeater task plotposmv_posmv starting 19.2 seconds late 2025-03-31 22:53:27,273 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 22:53:27,273 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 23:08:05 2025-03-31 22:53:27,273 INFO 102126:repeater task plot_hbins starting 21.4 seconds late 2025-03-31 22:53:28,345 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 22:53:28,345 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 22:53:28,345 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 23:08:05 2025-03-31 22:53:33,350 INFO 102126:repeater task noisy_rawtemps_os75 starting 22.5 seconds late 2025-03-31 22:53:35,288 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 22:53:35,288 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 23:03:10 2025-03-31 22:53:35,288 INFO 102126:repeater task 3dayplots_os75nb starting 24.5 seconds late 2025-03-31 22:53:39,685 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:53:39,685 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 22:58:10 2025-03-31 22:54:59,756 INFO 102126:repeater task lastensq_os75nb starting 2.8 seconds late 2025-03-31 22:55:00,482 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 22:55:00,482 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -6706.162 2025-03-31 22:55:00,482 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:00:02 2025-03-31 22:55:00,482 INFO 102126:repeater task quick_os75nb starting 3.5 seconds late 2025-03-31 22:55:02,998 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 22:55:02,998 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 22:55:02,999 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:55:02,999 INFO 102126:repeater task quick_os75nb: added 22 extra increments to time_to_run 2025-03-31 22:55:02,999 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:00:02 2025-03-31 22:55:02,999 INFO 102126:repeater task lastfew_vec_os75nb starting 6.0 seconds late 2025-03-31 22:55:03,610 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 22:55:03,611 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 22:55:03,611 INFO 102126:repeater task lastfew_vec_os75nb: added 22 extra increments to time_to_run 2025-03-31 22:55:03,611 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:00:02 2025-03-31 22:58:13,799 INFO 102126:repeater task 3dayplots_os75nb starting 3.0 seconds late 2025-03-31 22:58:18,191 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 22:58:18,192 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:03:10 2025-03-31 23:00:03,291 INFO 102126:repeater task lastensq_os75nb starting 1.3 seconds late 2025-03-31 23:00:04,049 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-03-31 23:00:04,049 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -7009.696 2025-03-31 23:00:04,050 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:05:07 2025-03-31 23:00:04,050 INFO 102126:repeater task quick_os75nb starting 2.1 seconds late 2025-03-31 23:00:06,563 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:00:06,563 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:00:06,563 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:00:06,563 INFO 102126:repeater task quick_os75nb: added 23 extra increments to time_to_run 2025-03-31 23:00:06,563 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:05:07 2025-03-31 23:00:06,563 INFO 102126:repeater task lastfew_vec_os75nb starting 4.6 seconds late 2025-03-31 23:00:07,175 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:00:07,175 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:00:07,175 INFO 102126:repeater task lastfew_vec_os75nb: added 23 extra increments to time_to_run 2025-03-31 23:00:07,175 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:05:07 2025-03-31 23:03:12,335 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.5 seconds late 2025-03-31 23:03:14,267 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 23:03:14,268 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 23:13:10 2025-03-31 23:03:14,268 INFO 102126:repeater task 3dayplots_os75nb starting 3.5 seconds late 2025-03-31 23:03:18,671 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:03:18,671 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:08:10 2025-03-31 23:03:18,672 INFO 102126:repeater task daily_netcdf starting 7.9 seconds late 2025-03-31 23:03:18,861 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 23:03:18,861 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 23:13:10 2025-03-31 23:05:08,963 INFO 102126:repeater task lastensq_os75nb starting 2.0 seconds late 2025-03-31 23:05:09,670 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:05:09,670 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -7315.368 2025-03-31 23:05:09,670 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:10:12 2025-03-31 23:05:09,670 INFO 102126:repeater task quick_os75nb starting 2.7 seconds late 2025-03-31 23:05:12,171 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:05:12,171 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:05:12,171 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:05:12,171 INFO 102126:repeater task quick_os75nb: added 24 extra increments to time_to_run 2025-03-31 23:05:12,172 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:10:12 2025-03-31 23:05:12,172 INFO 102126:repeater task lastfew_vec_os75nb starting 5.2 seconds late 2025-03-31 23:05:12,785 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:05:12,785 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:05:12,785 INFO 102126:repeater task lastfew_vec_os75nb: added 24 extra increments to time_to_run 2025-03-31 23:05:12,785 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:10:12 2025-03-31 23:08:07,943 INFO 102126:repeater task plot_tsstats_os75nb starting 2.1 seconds late 2025-03-31 23:08:08,721 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 23:08:08,721 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 23:08:08,721 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 23:23:05 2025-03-31 23:08:08,721 INFO 102126:repeater task grab_lastpings_os75nb starting 2.9 seconds late 2025-03-31 23:08:08,811 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 23:08:08,812 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 23:23:05 2025-03-31 23:08:08,812 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.0 seconds late 2025-03-31 23:08:09,814 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:08:09,814 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:23:05 2025-03-31 23:08:09,814 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.0 seconds late 2025-03-31 23:08:10,814 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:08:10,814 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:23:05 2025-03-31 23:08:10,815 INFO 102126:repeater task hbinhcorrstats_posmv starting 4.9 seconds late 2025-03-31 23:08:11,920 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 23:08:11,921 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 23:08:11,921 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 23:23:05 2025-03-31 23:08:11,921 INFO 102126:repeater task plotposmv_posmv starting 6.0 seconds late 2025-03-31 23:08:14,062 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 23:08:14,062 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 23:23:05 2025-03-31 23:08:14,062 INFO 102126:repeater task plot_hbins starting 8.2 seconds late 2025-03-31 23:08:15,141 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 23:08:15,141 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 23:08:15,141 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 23:23:05 2025-03-31 23:08:20,147 INFO 102126:repeater task 3dayplots_os75nb starting 9.3 seconds late 2025-03-31 23:08:24,540 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:08:24,540 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:13:10 2025-03-31 23:10:14,647 INFO 102126:repeater task lastensq_os75nb starting 2.6 seconds late 2025-03-31 23:10:15,337 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:10:15,337 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -7621.053 2025-03-31 23:10:15,337 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:15:17 2025-03-31 23:10:15,337 INFO 102126:repeater task quick_os75nb starting 3.3 seconds late 2025-03-31 23:10:17,848 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:10:17,848 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:10:17,848 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:10:17,848 INFO 102126:repeater task quick_os75nb: added 25 extra increments to time_to_run 2025-03-31 23:10:17,848 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:15:17 2025-03-31 23:10:17,848 INFO 102126:repeater task lastfew_vec_os75nb starting 5.8 seconds late 2025-03-31 23:10:18,460 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:10:18,460 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:10:18,460 INFO 102126:repeater task lastfew_vec_os75nb: added 25 extra increments to time_to_run 2025-03-31 23:10:18,460 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:15:17 2025-03-31 23:13:13,617 INFO 102126:repeater task noisy_rawtemps_os75 starting 2.8 seconds late 2025-03-31 23:13:15,554 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 23:13:15,554 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 23:23:10 2025-03-31 23:13:15,554 INFO 102126:repeater task 3dayplots_os75nb starting 4.7 seconds late 2025-03-31 23:13:19,952 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:13:19,952 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:18:10 2025-03-31 23:13:19,952 INFO 102126:repeater task daily_netcdf starting 9.1 seconds late 2025-03-31 23:13:20,141 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 23:13:20,141 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 23:23:10 2025-03-31 23:15:20,251 INFO 102126:repeater task lastensq_os75nb starting 3.3 seconds late 2025-03-31 23:15:20,939 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:15:20,939 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -7926.656 2025-03-31 23:15:20,939 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:20:22 2025-03-31 23:15:20,939 INFO 102126:repeater task quick_os75nb starting 3.9 seconds late 2025-03-31 23:15:23,451 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:15:23,451 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:15:23,451 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:15:23,451 INFO 102126:repeater task quick_os75nb: added 26 extra increments to time_to_run 2025-03-31 23:15:23,451 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:20:22 2025-03-31 23:15:23,452 INFO 102126:repeater task lastfew_vec_os75nb starting 6.5 seconds late 2025-03-31 23:15:24,073 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:15:24,073 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:15:24,073 INFO 102126:repeater task lastfew_vec_os75nb: added 26 extra increments to time_to_run 2025-03-31 23:15:24,073 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:20:22 2025-03-31 23:18:14,232 INFO 102126:repeater task 3dayplots_os75nb starting 3.4 seconds late 2025-03-31 23:18:18,629 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:18:18,629 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:23:10 2025-03-31 23:20:23,733 INFO 102126:repeater task lastensq_os75nb starting 1.7 seconds late 2025-03-31 23:20:24,411 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:20:24,411 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -8230.138 2025-03-31 23:20:24,411 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:25:27 2025-03-31 23:20:24,411 INFO 102126:repeater task quick_os75nb starting 2.4 seconds late 2025-03-31 23:20:26,922 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:20:26,922 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:20:26,922 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:20:26,922 INFO 102126:repeater task quick_os75nb: added 27 extra increments to time_to_run 2025-03-31 23:20:26,922 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:25:27 2025-03-31 23:20:26,922 INFO 102126:repeater task lastfew_vec_os75nb starting 4.9 seconds late 2025-03-31 23:20:27,559 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:20:27,559 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:20:27,559 INFO 102126:repeater task lastfew_vec_os75nb: added 27 extra increments to time_to_run 2025-03-31 23:20:27,559 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:25:27 2025-03-31 23:23:07,691 INFO 102126:repeater task plot_tsstats_os75nb starting 1.9 seconds late 2025-03-31 23:23:08,463 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 23:23:08,463 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 23:23:08,463 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 23:38:05 2025-03-31 23:23:08,463 INFO 102126:repeater task grab_lastpings_os75nb starting 2.6 seconds late 2025-03-31 23:23:08,554 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 23:23:08,554 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 23:38:05 2025-03-31 23:23:08,554 INFO 102126:repeater task beamdiagnostics_os75nb starting 2.7 seconds late 2025-03-31 23:23:09,545 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:23:09,545 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:38:05 2025-03-31 23:23:09,545 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.7 seconds late 2025-03-31 23:23:10,541 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:23:10,541 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:38:05 2025-03-31 23:23:10,541 INFO 102126:repeater task ggatime_diagnostics starting 4.7 seconds late 2025-03-31 23:23:14,544 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-03-31 23:23:14,544 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:23:14,544 INFO 102126:repeater task ggatime_diagnostics starting 8.7 seconds late 2025-03-31 23:23:18,403 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-03-31 23:23:18,403 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:23:18,404 INFO 102126:repeater task ggatime_diagnostics starting 12.6 seconds late 2025-03-31 23:23:22,254 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 23:23:22,254 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:23:22,254 INFO 102126:repeater task hbinhcorrstats_posmv starting 16.4 seconds late 2025-03-31 23:23:23,361 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 23:23:23,361 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 23:23:23,361 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 23:38:05 2025-03-31 23:23:23,361 INFO 102126:repeater task plotposmv_posmv starting 17.5 seconds late 2025-03-31 23:23:25,512 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-03-31 23:23:25,512 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 23:38:05 2025-03-31 23:23:25,512 INFO 102126:repeater task plot_hbins starting 19.6 seconds late 2025-03-31 23:23:26,583 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 23:23:26,583 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 23:23:26,583 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 23:38:05 2025-03-31 23:23:26,583 INFO 102126:repeater task report_generator starting 20.7 seconds late 2025-03-31 23:23:29,704 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-03-31 23:23:29,704 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-03-31 23:23:29,704 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 00:23:05 2025-03-31 23:23:34,710 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.9 seconds late 2025-03-31 23:23:36,649 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 23:23:36,649 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 23:33:10 2025-03-31 23:23:36,649 INFO 102126:repeater task 3dayplots_os75nb starting 25.8 seconds late 2025-03-31 23:23:41,057 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:23:41,057 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:28:10 2025-03-31 23:23:41,057 INFO 102126:repeater task daily_netcdf starting 30.2 seconds late 2025-03-31 23:23:41,247 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 23:23:41,247 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 23:33:10 2025-03-31 23:25:31,353 INFO 102126:repeater task lastensq_os75nb starting 4.4 seconds late 2025-03-31 23:25:32,048 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:25:32,048 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -8537.759 2025-03-31 23:25:32,048 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:30:32 2025-03-31 23:25:32,048 INFO 102126:repeater task quick_os75nb starting 5.0 seconds late 2025-03-31 23:25:34,575 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:25:34,575 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:25:34,575 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:25:34,575 INFO 102126:repeater task quick_os75nb: added 28 extra increments to time_to_run 2025-03-31 23:25:34,575 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:30:32 2025-03-31 23:25:34,575 INFO 102126:repeater task lastfew_vec_os75nb starting 7.6 seconds late 2025-03-31 23:25:35,183 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:25:35,184 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:25:35,184 INFO 102126:repeater task lastfew_vec_os75nb: added 28 extra increments to time_to_run 2025-03-31 23:25:35,184 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:30:32 2025-03-31 23:28:15,337 INFO 102126:repeater task 3dayplots_os75nb starting 4.5 seconds late 2025-03-31 23:28:19,738 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:28:19,739 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:33:10 2025-03-31 23:30:34,863 INFO 102126:repeater task lastensq_os75nb starting 2.9 seconds late 2025-03-31 23:30:35,553 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:30:35,553 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -8841.269 2025-03-31 23:30:35,554 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:35:37 2025-03-31 23:30:35,554 INFO 102126:repeater task quick_os75nb starting 3.6 seconds late 2025-03-31 23:30:38,089 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:30:38,090 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:30:38,090 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:30:38,090 INFO 102126:repeater task quick_os75nb: added 29 extra increments to time_to_run 2025-03-31 23:30:38,090 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:35:37 2025-03-31 23:30:38,090 INFO 102126:repeater task lastfew_vec_os75nb starting 6.1 seconds late 2025-03-31 23:30:38,709 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:30:38,709 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:30:38,709 INFO 102126:repeater task lastfew_vec_os75nb: added 29 extra increments to time_to_run 2025-03-31 23:30:38,709 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:35:37 2025-03-31 23:33:13,843 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.0 seconds late 2025-03-31 23:33:15,781 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 23:33:15,781 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 23:43:10 2025-03-31 23:33:15,782 INFO 102126:repeater task 3dayplots_os75nb starting 5.0 seconds late 2025-03-31 23:33:20,189 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:33:20,189 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:38:10 2025-03-31 23:33:20,190 INFO 102126:repeater task daily_netcdf starting 9.4 seconds late 2025-03-31 23:33:20,380 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 23:33:20,380 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 23:43:10 2025-03-31 23:35:40,500 INFO 102126:repeater task lastensq_os75nb starting 3.5 seconds late 2025-03-31 23:35:41,178 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:35:41,178 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -9146.905 2025-03-31 23:35:41,178 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:40:42 2025-03-31 23:35:41,178 INFO 102126:repeater task quick_os75nb starting 4.2 seconds late 2025-03-31 23:35:43,732 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:35:43,732 INFO 102126:repeater task quick_os75nb ended after 2.6 seconds 2025-03-31 23:35:43,732 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:35:43,732 INFO 102126:repeater task quick_os75nb: added 30 extra increments to time_to_run 2025-03-31 23:35:43,732 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:40:42 2025-03-31 23:35:43,733 INFO 102126:repeater task lastfew_vec_os75nb starting 6.7 seconds late 2025-03-31 23:35:44,346 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:35:44,346 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:35:44,346 INFO 102126:repeater task lastfew_vec_os75nb: added 30 extra increments to time_to_run 2025-03-31 23:35:44,346 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:40:42 2025-03-31 23:38:09,469 INFO 102126:repeater task plot_tsstats_os75nb starting 3.7 seconds late 2025-03-31 23:38:10,245 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 23:38:10,245 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 23:38:10,245 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:38:10,245 INFO 102126:repeater task grab_lastpings_os75nb starting 4.4 seconds late 2025-03-31 23:38:10,335 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 23:38:10,336 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:38:10,336 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.5 seconds late 2025-03-31 23:38:11,326 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:38:11,326 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:38:11,326 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.5 seconds late 2025-03-31 23:38:12,321 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:38:12,321 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:38:12,321 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.4 seconds late 2025-03-31 23:38:13,450 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 23:38:13,450 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 23:38:13,450 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:38:13,450 INFO 102126:repeater task plotposmv_posmv starting 7.5 seconds late 2025-03-31 23:38:15,614 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-03-31 23:38:15,614 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:38:15,614 INFO 102126:repeater task plot_hbins starting 9.7 seconds late 2025-03-31 23:38:16,686 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 23:38:16,687 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 23:38:16,687 INFO 102126:repeater task plot_hbins next time_to_run is 2025/03/31 23:53:05 2025-03-31 23:38:21,692 INFO 102126:repeater task 3dayplots_os75nb starting 10.9 seconds late 2025-03-31 23:38:26,086 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:38:26,086 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:43:10 2025-03-31 23:40:46,220 INFO 102126:repeater task lastensq_os75nb starting 4.2 seconds late 2025-03-31 23:40:46,905 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:40:46,905 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -9452.626 2025-03-31 23:40:46,905 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:45:47 2025-03-31 23:40:46,905 INFO 102126:repeater task quick_os75nb starting 4.9 seconds late 2025-03-31 23:40:49,412 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:40:49,412 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:40:49,412 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:40:49,412 INFO 102126:repeater task quick_os75nb: added 31 extra increments to time_to_run 2025-03-31 23:40:49,412 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:45:47 2025-03-31 23:40:49,412 INFO 102126:repeater task lastfew_vec_os75nb starting 7.4 seconds late 2025-03-31 23:40:50,037 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:40:50,037 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:40:50,037 INFO 102126:repeater task lastfew_vec_os75nb: added 31 extra increments to time_to_run 2025-03-31 23:40:50,037 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:45:47 2025-03-31 23:43:15,155 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.3 seconds late 2025-03-31 23:43:17,093 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 23:43:17,093 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/03/31 23:53:10 2025-03-31 23:43:17,094 INFO 102126:repeater task 3dayplots_os75nb starting 6.3 seconds late 2025-03-31 23:43:21,479 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:43:21,479 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:48:10 2025-03-31 23:43:21,479 INFO 102126:repeater task daily_netcdf starting 10.7 seconds late 2025-03-31 23:43:21,668 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 23:43:21,668 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/03/31 23:53:10 2025-03-31 23:45:51,803 INFO 102126:repeater task lastensq_os75nb starting 4.8 seconds late 2025-03-31 23:45:52,494 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:45:52,494 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -9758.209 2025-03-31 23:45:52,494 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:50:52 2025-03-31 23:45:52,494 INFO 102126:repeater task quick_os75nb starting 5.5 seconds late 2025-03-31 23:45:55,008 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:45:55,008 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:45:55,008 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:45:55,008 INFO 102126:repeater task quick_os75nb: added 32 extra increments to time_to_run 2025-03-31 23:45:55,009 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:50:52 2025-03-31 23:45:55,009 INFO 102126:repeater task lastfew_vec_os75nb starting 8.0 seconds late 2025-03-31 23:45:55,630 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:45:55,630 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:45:55,630 INFO 102126:repeater task lastfew_vec_os75nb: added 32 extra increments to time_to_run 2025-03-31 23:45:55,630 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:50:52 2025-03-31 23:48:15,765 INFO 102126:repeater task 3dayplots_os75nb starting 4.9 seconds late 2025-03-31 23:48:20,164 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:48:20,164 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:53:10 2025-03-31 23:50:55,323 INFO 102126:repeater task lastensq_os75nb starting 3.3 seconds late 2025-03-31 23:50:56,095 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-03-31 23:50:56,095 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -10061.728 2025-03-31 23:50:56,095 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/03/31 23:55:57 2025-03-31 23:50:56,095 INFO 102126:repeater task quick_os75nb starting 4.1 seconds late 2025-03-31 23:50:58,602 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:50:58,602 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:50:58,602 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:50:58,602 INFO 102126:repeater task quick_os75nb: added 33 extra increments to time_to_run 2025-03-31 23:50:58,602 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/03/31 23:55:57 2025-03-31 23:50:58,603 INFO 102126:repeater task lastfew_vec_os75nb starting 6.6 seconds late 2025-03-31 23:50:59,215 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:50:59,216 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:50:59,216 INFO 102126:repeater task lastfew_vec_os75nb: added 33 extra increments to time_to_run 2025-03-31 23:50:59,216 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/03/31 23:55:57 2025-03-31 23:53:09,331 INFO 102126:repeater task plot_tsstats_os75nb starting 3.5 seconds late 2025-03-31 23:53:10,102 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-03-31 23:53:10,102 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-03-31 23:53:10,102 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 00:08:05 2025-03-31 23:53:10,102 INFO 102126:repeater task grab_lastpings_os75nb starting 4.3 seconds late 2025-03-31 23:53:10,192 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-03-31 23:53:10,192 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 00:08:05 2025-03-31 23:53:10,193 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.4 seconds late 2025-03-31 23:53:11,185 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:53:11,185 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:08:05 2025-03-31 23:53:11,185 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.4 seconds late 2025-03-31 23:53:12,180 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-03-31 23:53:12,180 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:08:05 2025-03-31 23:53:12,180 INFO 102126:repeater task daily_netcdf starting 1.4 seconds late 2025-03-31 23:53:12,370 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-03-31 23:53:12,370 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 00:03:10 2025-03-31 23:53:12,370 INFO 102126:repeater task ggatime_diagnostics starting 6.6 seconds late 2025-03-31 23:53:16,350 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-03-31 23:53:16,350 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 00:23:05 2025-03-31 23:53:16,350 INFO 102126:repeater task ggatime_diagnostics starting 10.5 seconds late 2025-03-31 23:53:20,173 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 23:53:20,174 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 00:23:05 2025-03-31 23:53:20,174 INFO 102126:repeater task ggatime_diagnostics starting 14.4 seconds late 2025-03-31 23:53:24,011 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-03-31 23:53:24,011 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 00:23:05 2025-03-31 23:53:24,011 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.1 seconds late 2025-03-31 23:53:25,122 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-03-31 23:53:25,122 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-03-31 23:53:25,122 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 00:08:05 2025-03-31 23:53:25,123 INFO 102126:repeater task plotposmv_posmv starting 19.2 seconds late 2025-03-31 23:53:27,269 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-03-31 23:53:27,270 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 00:08:05 2025-03-31 23:53:27,270 INFO 102126:repeater task plot_hbins starting 21.4 seconds late 2025-03-31 23:53:28,344 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-03-31 23:53:28,344 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-03-31 23:53:28,344 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 00:08:05 2025-03-31 23:53:33,347 INFO 102126:repeater task noisy_rawtemps_os75 starting 22.5 seconds late 2025-03-31 23:53:35,286 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-03-31 23:53:35,286 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 00:03:10 2025-03-31 23:53:35,286 INFO 102126:repeater task 3dayplots_os75nb starting 24.5 seconds late 2025-03-31 23:53:39,698 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:53:39,698 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/03/31 23:58:10 2025-03-31 23:55:59,826 INFO 102126:repeater task lastensq_os75nb starting 2.8 seconds late 2025-03-31 23:56:00,523 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-03-31 23:56:00,523 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -10366.232 2025-03-31 23:56:00,523 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:01:02 2025-03-31 23:56:00,523 INFO 102126:repeater task quick_os75nb starting 3.5 seconds late 2025-03-31 23:56:03,032 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-03-31 23:56:03,032 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-03-31 23:56:03,032 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:56:03,032 INFO 102126:repeater task quick_os75nb: added 34 extra increments to time_to_run 2025-03-31 23:56:03,032 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:01:02 2025-03-31 23:56:03,032 INFO 102126:repeater task lastfew_vec_os75nb starting 6.0 seconds late 2025-03-31 23:56:03,654 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-03-31 23:56:03,654 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-03-31 23:56:03,655 INFO 102126:repeater task lastfew_vec_os75nb: added 34 extra increments to time_to_run 2025-03-31 23:56:03,655 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:01:02 2025-03-31 23:58:13,791 INFO 102126:repeater task 3dayplots_os75nb starting 3.0 seconds late 2025-03-31 23:58:18,178 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-03-31 23:58:18,179 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:03:10 2025-04-01 00:01:03,339 INFO 102126:repeater task lastensq_os75nb starting 1.3 seconds late 2025-04-01 00:01:04,039 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:01:04,039 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -10669.745 2025-04-01 00:01:04,039 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:06:07 2025-04-01 00:01:04,040 INFO 102126:repeater task quick_os75nb starting 2.0 seconds late 2025-04-01 00:01:06,550 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:01:06,550 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:01:06,551 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:01:06,551 INFO 102126:repeater task quick_os75nb: added 35 extra increments to time_to_run 2025-04-01 00:01:06,551 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:06:07 2025-04-01 00:01:06,551 INFO 102126:repeater task lastfew_vec_os75nb starting 4.6 seconds late 2025-04-01 00:01:07,161 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:01:07,161 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:01:07,161 INFO 102126:repeater task lastfew_vec_os75nb: added 35 extra increments to time_to_run 2025-04-01 00:01:07,161 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:06:07 2025-04-01 00:03:12,267 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.5 seconds late 2025-04-01 00:03:14,196 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 00:03:14,196 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 00:13:10 2025-04-01 00:03:14,197 INFO 102126:repeater task 3dayplots_os75nb starting 3.4 seconds late 2025-04-01 00:03:18,597 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:03:18,597 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:08:10 2025-04-01 00:03:18,597 INFO 102126:repeater task daily_netcdf starting 7.8 seconds late 2025-04-01 00:03:18,797 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 00:03:18,797 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 00:03:18,797 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 00:13:10 2025-04-01 00:06:08,947 INFO 102126:repeater task lastensq_os75nb starting 1.9 seconds late 2025-04-01 00:06:09,617 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:06:09,617 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -10975.353 2025-04-01 00:06:09,617 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:11:12 2025-04-01 00:06:09,617 INFO 102126:repeater task quick_os75nb starting 2.6 seconds late 2025-04-01 00:06:12,141 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:06:12,141 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:06:12,142 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:06:12,142 INFO 102126:repeater task quick_os75nb: added 36 extra increments to time_to_run 2025-04-01 00:06:12,142 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:11:12 2025-04-01 00:06:12,142 INFO 102126:repeater task lastfew_vec_os75nb starting 5.1 seconds late 2025-04-01 00:06:12,760 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:06:12,760 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:06:12,760 INFO 102126:repeater task lastfew_vec_os75nb: added 36 extra increments to time_to_run 2025-04-01 00:06:12,760 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:11:12 2025-04-01 00:08:07,859 INFO 102126:repeater task plot_tsstats_os75nb starting 2.0 seconds late 2025-04-01 00:08:08,633 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 00:08:08,633 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 00:08:08,633 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 00:23:05 2025-04-01 00:08:08,633 INFO 102126:repeater task grab_lastpings_os75nb starting 2.8 seconds late 2025-04-01 00:08:08,723 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 00:08:08,723 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 00:23:05 2025-04-01 00:08:08,724 INFO 102126:repeater task beamdiagnostics_os75nb starting 2.9 seconds late 2025-04-01 00:08:09,719 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:08:09,719 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:23:05 2025-04-01 00:08:09,720 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.9 seconds late 2025-04-01 00:08:10,716 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:08:10,716 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:23:05 2025-04-01 00:08:10,716 INFO 102126:repeater task hbinhcorrstats_posmv starting 4.8 seconds late 2025-04-01 00:08:11,834 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 00:08:11,834 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 00:08:11,834 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 00:23:05 2025-04-01 00:08:11,834 INFO 102126:repeater task plotposmv_posmv starting 5.9 seconds late 2025-04-01 00:08:13,933 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 00:08:13,933 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 00:23:05 2025-04-01 00:08:13,933 INFO 102126:repeater task plot_hbins starting 8.0 seconds late 2025-04-01 00:08:14,989 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 00:08:14,989 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 00:08:14,989 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 00:23:05 2025-04-01 00:08:19,995 INFO 102126:repeater task 3dayplots_os75nb starting 9.2 seconds late 2025-04-01 00:08:24,389 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:08:24,389 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:13:10 2025-04-01 00:11:14,551 INFO 102126:repeater task lastensq_os75nb starting 2.6 seconds late 2025-04-01 00:11:15,220 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:11:15,220 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -11280.957 2025-04-01 00:11:15,220 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:16:17 2025-04-01 00:11:15,220 INFO 102126:repeater task quick_os75nb starting 3.2 seconds late 2025-04-01 00:11:17,742 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:11:17,742 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:11:17,742 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:11:17,742 INFO 102126:repeater task quick_os75nb: added 37 extra increments to time_to_run 2025-04-01 00:11:17,742 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:16:17 2025-04-01 00:11:17,742 INFO 102126:repeater task lastfew_vec_os75nb starting 5.7 seconds late 2025-04-01 00:11:18,358 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:11:18,358 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:11:18,358 INFO 102126:repeater task lastfew_vec_os75nb: added 37 extra increments to time_to_run 2025-04-01 00:11:18,358 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:16:17 2025-04-01 00:13:13,451 INFO 102126:repeater task noisy_rawtemps_os75 starting 2.6 seconds late 2025-04-01 00:13:15,397 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 00:13:15,397 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 00:23:10 2025-04-01 00:13:15,397 INFO 102126:repeater task 3dayplots_os75nb starting 4.6 seconds late 2025-04-01 00:13:19,791 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:13:19,791 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:18:10 2025-04-01 00:13:19,791 INFO 102126:repeater task daily_netcdf starting 9.0 seconds late 2025-04-01 00:13:19,989 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 00:13:19,989 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 00:13:19,989 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 00:23:10 2025-04-01 00:16:20,155 INFO 102126:repeater task lastensq_os75nb starting 3.2 seconds late 2025-04-01 00:16:20,823 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:16:20,823 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -11586.561 2025-04-01 00:16:20,823 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:21:22 2025-04-01 00:16:20,823 INFO 102126:repeater task quick_os75nb starting 3.8 seconds late 2025-04-01 00:16:23,335 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:16:23,335 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:16:23,335 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:16:23,335 INFO 102126:repeater task quick_os75nb: added 38 extra increments to time_to_run 2025-04-01 00:16:23,335 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:21:22 2025-04-01 00:16:23,335 INFO 102126:repeater task lastfew_vec_os75nb starting 6.3 seconds late 2025-04-01 00:16:23,946 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:16:23,946 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:16:23,946 INFO 102126:repeater task lastfew_vec_os75nb: added 38 extra increments to time_to_run 2025-04-01 00:16:23,946 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:21:22 2025-04-01 00:18:14,049 INFO 102126:repeater task 3dayplots_os75nb starting 3.2 seconds late 2025-04-01 00:18:18,431 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:18:18,432 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:23:10 2025-04-01 00:21:23,593 INFO 102126:repeater task lastensq_os75nb starting 1.6 seconds late 2025-04-01 00:21:24,279 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:21:24,279 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -11889.999 2025-04-01 00:21:24,279 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:26:27 2025-04-01 00:21:24,279 INFO 102126:repeater task quick_os75nb starting 2.3 seconds late 2025-04-01 00:21:26,785 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:21:26,785 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:21:26,785 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:21:26,785 INFO 102126:repeater task quick_os75nb: added 39 extra increments to time_to_run 2025-04-01 00:21:26,785 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:26:27 2025-04-01 00:21:26,785 INFO 102126:repeater task lastfew_vec_os75nb starting 4.8 seconds late 2025-04-01 00:21:27,405 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:21:27,405 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:21:27,405 INFO 102126:repeater task lastfew_vec_os75nb: added 39 extra increments to time_to_run 2025-04-01 00:21:27,405 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:26:27 2025-04-01 00:23:07,499 INFO 102126:repeater task plot_tsstats_os75nb starting 1.7 seconds late 2025-04-01 00:23:08,263 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 00:23:08,263 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 00:23:08,263 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 00:38:05 2025-04-01 00:23:08,263 INFO 102126:repeater task grab_lastpings_os75nb starting 2.4 seconds late 2025-04-01 00:23:08,354 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 00:23:08,354 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 00:38:05 2025-04-01 00:23:08,354 INFO 102126:repeater task beamdiagnostics_os75nb starting 2.5 seconds late 2025-04-01 00:23:09,351 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:23:09,351 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:38:05 2025-04-01 00:23:09,351 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.5 seconds late 2025-04-01 00:23:10,344 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:23:10,344 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:38:05 2025-04-01 00:23:10,344 INFO 102126:repeater task ggatime_diagnostics starting 4.5 seconds late 2025-04-01 00:23:14,321 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 00:23:14,321 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:23:14,321 INFO 102126:repeater task ggatime_diagnostics starting 8.5 seconds late 2025-04-01 00:23:18,131 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 00:23:18,131 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:23:18,132 INFO 102126:repeater task ggatime_diagnostics starting 12.3 seconds late 2025-04-01 00:23:21,976 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 00:23:21,977 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:23:21,977 INFO 102126:repeater task hbinhcorrstats_posmv starting 16.1 seconds late 2025-04-01 00:23:23,097 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 00:23:23,097 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 00:23:23,098 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 00:38:05 2025-04-01 00:23:23,098 INFO 102126:repeater task plotposmv_posmv starting 17.2 seconds late 2025-04-01 00:23:25,211 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 00:23:25,211 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 00:38:05 2025-04-01 00:23:25,212 INFO 102126:repeater task plot_hbins starting 19.3 seconds late 2025-04-01 00:23:26,278 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 00:23:26,278 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 00:23:26,278 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 00:38:05 2025-04-01 00:23:26,278 INFO 102126:repeater task report_generator starting 20.4 seconds late 2025-04-01 00:23:29,399 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-04-01 00:23:29,399 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-04-01 00:23:29,399 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 01:23:05 2025-04-01 00:23:34,404 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.6 seconds late 2025-04-01 00:23:36,337 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 00:23:36,337 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 00:33:10 2025-04-01 00:23:36,338 INFO 102126:repeater task 3dayplots_os75nb starting 25.5 seconds late 2025-04-01 00:23:40,725 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:23:40,725 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:28:10 2025-04-01 00:23:40,726 INFO 102126:repeater task daily_netcdf starting 29.9 seconds late 2025-04-01 00:23:40,925 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 00:23:40,925 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 00:23:40,925 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 00:33:10 2025-04-01 00:26:31,067 INFO 102126:repeater task lastensq_os75nb starting 4.1 seconds late 2025-04-01 00:26:31,735 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:26:31,735 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -12197.473 2025-04-01 00:26:31,735 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:31:32 2025-04-01 00:26:31,735 INFO 102126:repeater task quick_os75nb starting 4.7 seconds late 2025-04-01 00:26:34,251 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:26:34,251 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:26:34,251 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:26:34,251 INFO 102126:repeater task quick_os75nb: added 40 extra increments to time_to_run 2025-04-01 00:26:34,251 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:31:32 2025-04-01 00:26:34,252 INFO 102126:repeater task lastfew_vec_os75nb starting 7.3 seconds late 2025-04-01 00:26:34,881 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:26:34,881 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:26:34,881 INFO 102126:repeater task lastfew_vec_os75nb: added 40 extra increments to time_to_run 2025-04-01 00:26:34,881 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:31:32 2025-04-01 00:28:14,975 INFO 102126:repeater task 3dayplots_os75nb starting 4.2 seconds late 2025-04-01 00:28:19,362 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:28:19,362 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:33:10 2025-04-01 00:31:34,539 INFO 102126:repeater task lastensq_os75nb starting 2.5 seconds late 2025-04-01 00:31:35,225 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:31:35,226 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -12500.945 2025-04-01 00:31:35,226 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:36:37 2025-04-01 00:31:35,226 INFO 102126:repeater task quick_os75nb starting 3.2 seconds late 2025-04-01 00:31:37,788 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:31:37,789 INFO 102126:repeater task quick_os75nb ended after 2.6 seconds 2025-04-01 00:31:37,789 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:31:37,789 INFO 102126:repeater task quick_os75nb: added 41 extra increments to time_to_run 2025-04-01 00:31:37,789 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:36:37 2025-04-01 00:31:37,789 INFO 102126:repeater task lastfew_vec_os75nb starting 5.8 seconds late 2025-04-01 00:31:38,596 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.8 seconds 2025-04-01 00:31:38,597 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:31:38,597 INFO 102126:repeater task lastfew_vec_os75nb: added 41 extra increments to time_to_run 2025-04-01 00:31:38,597 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:36:37 2025-04-01 00:33:13,671 INFO 102126:repeater task noisy_rawtemps_os75 starting 2.9 seconds late 2025-04-01 00:33:15,598 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 00:33:15,598 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 00:43:10 2025-04-01 00:33:15,599 INFO 102126:repeater task 3dayplots_os75nb starting 4.8 seconds late 2025-04-01 00:33:20,020 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:33:20,020 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:38:10 2025-04-01 00:33:20,021 INFO 102126:repeater task daily_netcdf starting 9.2 seconds late 2025-04-01 00:33:20,221 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 00:33:20,221 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 00:33:20,221 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 00:43:10 2025-04-01 00:36:40,407 INFO 102126:repeater task lastensq_os75nb starting 3.4 seconds late 2025-04-01 00:36:41,096 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:36:41,097 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -12806.813 2025-04-01 00:36:41,097 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:41:42 2025-04-01 00:36:41,097 INFO 102126:repeater task quick_os75nb starting 4.1 seconds late 2025-04-01 00:36:43,640 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:36:43,640 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:36:43,640 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:36:43,640 INFO 102126:repeater task quick_os75nb: added 42 extra increments to time_to_run 2025-04-01 00:36:43,640 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:41:42 2025-04-01 00:36:43,640 INFO 102126:repeater task lastfew_vec_os75nb starting 6.6 seconds late 2025-04-01 00:36:44,261 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:36:44,261 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:36:44,261 INFO 102126:repeater task lastfew_vec_os75nb: added 42 extra increments to time_to_run 2025-04-01 00:36:44,261 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:41:42 2025-04-01 00:38:09,341 INFO 102126:repeater task plot_tsstats_os75nb starting 3.5 seconds late 2025-04-01 00:38:10,112 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 00:38:10,112 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 00:38:10,112 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:38:10,112 INFO 102126:repeater task grab_lastpings_os75nb starting 4.3 seconds late 2025-04-01 00:38:10,203 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 00:38:10,203 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:38:10,203 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.4 seconds late 2025-04-01 00:38:11,198 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:38:11,198 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:38:11,198 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.4 seconds late 2025-04-01 00:38:12,192 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:38:12,192 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:38:12,192 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.3 seconds late 2025-04-01 00:38:13,293 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 00:38:13,293 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 00:38:13,293 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:38:13,293 INFO 102126:repeater task plotposmv_posmv starting 7.4 seconds late 2025-04-01 00:38:15,418 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 00:38:15,418 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:38:15,419 INFO 102126:repeater task plot_hbins starting 9.5 seconds late 2025-04-01 00:38:16,494 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 00:38:16,494 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 00:38:16,494 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 00:53:05 2025-04-01 00:38:21,499 INFO 102126:repeater task 3dayplots_os75nb starting 10.7 seconds late 2025-04-01 00:38:25,882 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:38:25,883 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:43:10 2025-04-01 00:41:46,063 INFO 102126:repeater task lastensq_os75nb starting 4.1 seconds late 2025-04-01 00:41:46,746 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:41:46,746 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -13112.469 2025-04-01 00:41:46,747 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:46:47 2025-04-01 00:41:46,747 INFO 102126:repeater task quick_os75nb starting 4.7 seconds late 2025-04-01 00:41:49,260 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:41:49,260 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:41:49,260 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:41:49,260 INFO 102126:repeater task quick_os75nb: added 43 extra increments to time_to_run 2025-04-01 00:41:49,260 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:46:47 2025-04-01 00:41:49,260 INFO 102126:repeater task lastfew_vec_os75nb starting 7.3 seconds late 2025-04-01 00:41:49,873 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:41:49,873 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:41:49,873 INFO 102126:repeater task lastfew_vec_os75nb: added 43 extra increments to time_to_run 2025-04-01 00:41:49,873 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:46:47 2025-04-01 00:43:14,946 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.1 seconds late 2025-04-01 00:43:16,878 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 00:43:16,879 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 00:53:10 2025-04-01 00:43:16,879 INFO 102126:repeater task 3dayplots_os75nb starting 6.1 seconds late 2025-04-01 00:43:21,272 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:43:21,273 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:48:10 2025-04-01 00:43:21,273 INFO 102126:repeater task daily_netcdf starting 10.5 seconds late 2025-04-01 00:43:21,472 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 00:43:21,472 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 00:43:21,472 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 00:53:10 2025-04-01 00:46:51,671 INFO 102126:repeater task lastensq_os75nb starting 4.7 seconds late 2025-04-01 00:46:52,358 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:46:52,359 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -13418.076 2025-04-01 00:46:52,359 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:51:52 2025-04-01 00:46:52,359 INFO 102126:repeater task quick_os75nb starting 5.4 seconds late 2025-04-01 00:46:54,871 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:46:54,871 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:46:54,871 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:46:54,871 INFO 102126:repeater task quick_os75nb: added 44 extra increments to time_to_run 2025-04-01 00:46:54,871 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:51:52 2025-04-01 00:46:54,872 INFO 102126:repeater task lastfew_vec_os75nb starting 7.9 seconds late 2025-04-01 00:46:55,491 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:46:55,492 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:46:55,492 INFO 102126:repeater task lastfew_vec_os75nb: added 44 extra increments to time_to_run 2025-04-01 00:46:55,492 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:51:52 2025-04-01 00:48:15,563 INFO 102126:repeater task 3dayplots_os75nb starting 4.7 seconds late 2025-04-01 00:48:19,964 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:48:19,964 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:53:10 2025-04-01 00:51:55,147 INFO 102126:repeater task lastensq_os75nb starting 3.1 seconds late 2025-04-01 00:51:55,863 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:51:55,863 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -13721.553 2025-04-01 00:51:55,863 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 00:56:57 2025-04-01 00:51:55,863 INFO 102126:repeater task quick_os75nb starting 3.9 seconds late 2025-04-01 00:51:58,379 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:51:58,379 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:51:58,379 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:51:58,380 INFO 102126:repeater task quick_os75nb: added 45 extra increments to time_to_run 2025-04-01 00:51:58,380 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 00:56:57 2025-04-01 00:51:58,380 INFO 102126:repeater task lastfew_vec_os75nb starting 6.4 seconds late 2025-04-01 00:51:58,989 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:51:58,990 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:51:58,990 INFO 102126:repeater task lastfew_vec_os75nb: added 45 extra increments to time_to_run 2025-04-01 00:51:58,990 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 00:56:57 2025-04-01 00:53:09,047 INFO 102126:repeater task plot_tsstats_os75nb starting 3.2 seconds late 2025-04-01 00:53:09,823 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 00:53:09,823 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 00:53:09,823 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 01:08:05 2025-04-01 00:53:09,823 INFO 102126:repeater task grab_lastpings_os75nb starting 4.0 seconds late 2025-04-01 00:53:09,913 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 00:53:09,914 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 01:08:05 2025-04-01 00:53:09,914 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.1 seconds late 2025-04-01 00:53:10,907 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:53:10,907 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:08:05 2025-04-01 00:53:10,907 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.1 seconds late 2025-04-01 00:53:11,903 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 00:53:11,903 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:08:05 2025-04-01 00:53:11,904 INFO 102126:repeater task daily_netcdf starting 1.1 seconds late 2025-04-01 00:53:12,103 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 00:53:12,103 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 00:53:12,103 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 01:03:10 2025-04-01 00:53:12,103 INFO 102126:repeater task ggatime_diagnostics starting 6.3 seconds late 2025-04-01 00:53:16,105 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 00:53:16,105 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 01:23:05 2025-04-01 00:53:16,106 INFO 102126:repeater task ggatime_diagnostics starting 10.3 seconds late 2025-04-01 00:53:19,939 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 00:53:19,939 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 01:23:05 2025-04-01 00:53:19,940 INFO 102126:repeater task ggatime_diagnostics starting 14.1 seconds late 2025-04-01 00:53:23,793 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-04-01 00:53:23,793 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 01:23:05 2025-04-01 00:53:23,793 INFO 102126:repeater task hbinhcorrstats_posmv starting 17.9 seconds late 2025-04-01 00:53:24,896 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 00:53:24,896 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 00:53:24,896 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 01:08:05 2025-04-01 00:53:24,896 INFO 102126:repeater task plotposmv_posmv starting 19.0 seconds late 2025-04-01 00:53:27,041 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 00:53:27,041 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 01:08:05 2025-04-01 00:53:27,041 INFO 102126:repeater task plot_hbins starting 21.1 seconds late 2025-04-01 00:53:28,115 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 00:53:28,115 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 00:53:28,115 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 01:08:05 2025-04-01 00:53:33,120 INFO 102126:repeater task noisy_rawtemps_os75 starting 22.3 seconds late 2025-04-01 00:53:35,040 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 00:53:35,041 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 01:03:10 2025-04-01 00:53:35,041 INFO 102126:repeater task 3dayplots_os75nb starting 24.2 seconds late 2025-04-01 00:53:39,433 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:53:39,433 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 00:58:10 2025-04-01 00:56:59,619 INFO 102126:repeater task lastensq_os75nb starting 2.6 seconds late 2025-04-01 00:57:00,290 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 00:57:00,290 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -14026.025 2025-04-01 00:57:00,290 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:02:02 2025-04-01 00:57:00,290 INFO 102126:repeater task quick_os75nb starting 3.3 seconds late 2025-04-01 00:57:02,814 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 00:57:02,814 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 00:57:02,814 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:57:02,814 INFO 102126:repeater task quick_os75nb: added 46 extra increments to time_to_run 2025-04-01 00:57:02,814 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:02:02 2025-04-01 00:57:02,814 INFO 102126:repeater task lastfew_vec_os75nb starting 5.8 seconds late 2025-04-01 00:57:03,424 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 00:57:03,424 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 00:57:03,424 INFO 102126:repeater task lastfew_vec_os75nb: added 46 extra increments to time_to_run 2025-04-01 00:57:03,424 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:02:02 2025-04-01 00:58:13,481 INFO 102126:repeater task 3dayplots_os75nb starting 2.7 seconds late 2025-04-01 00:58:17,889 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 00:58:17,889 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:03:10 2025-04-01 01:02:03,091 INFO 102126:repeater task lastensq_os75nb starting 1.1 seconds late 2025-04-01 01:02:03,762 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:02:03,762 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -14329.497 2025-04-01 01:02:03,762 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:07:07 2025-04-01 01:02:03,763 INFO 102126:repeater task quick_os75nb starting 1.8 seconds late 2025-04-01 01:02:06,274 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:02:06,274 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:02:06,274 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:02:06,274 INFO 102126:repeater task quick_os75nb: added 47 extra increments to time_to_run 2025-04-01 01:02:06,274 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:07:07 2025-04-01 01:02:06,275 INFO 102126:repeater task lastfew_vec_os75nb starting 4.3 seconds late 2025-04-01 01:02:06,894 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:02:06,894 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:02:06,894 INFO 102126:repeater task lastfew_vec_os75nb: added 47 extra increments to time_to_run 2025-04-01 01:02:06,894 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:07:07 2025-04-01 01:03:11,955 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.1 seconds late 2025-04-01 01:03:13,898 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 01:03:13,898 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 01:13:10 2025-04-01 01:03:13,898 INFO 102126:repeater task 3dayplots_os75nb starting 3.1 seconds late 2025-04-01 01:03:18,295 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:03:18,295 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:08:10 2025-04-01 01:03:18,295 INFO 102126:repeater task daily_netcdf starting 7.5 seconds late 2025-04-01 01:03:18,496 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 01:03:18,496 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 01:03:18,496 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 01:13:10 2025-04-01 01:07:08,721 INFO 102126:repeater task lastensq_os75nb starting 1.7 seconds late 2025-04-01 01:07:09,408 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:07:09,408 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -14635.126 2025-04-01 01:07:09,408 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:12:12 2025-04-01 01:07:09,408 INFO 102126:repeater task quick_os75nb starting 2.4 seconds late 2025-04-01 01:07:11,917 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:07:11,917 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:07:11,917 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:07:11,917 INFO 102126:repeater task quick_os75nb: added 48 extra increments to time_to_run 2025-04-01 01:07:11,917 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:12:12 2025-04-01 01:07:11,918 INFO 102126:repeater task lastfew_vec_os75nb starting 4.9 seconds late 2025-04-01 01:07:12,525 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:07:12,525 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:07:12,525 INFO 102126:repeater task lastfew_vec_os75nb: added 48 extra increments to time_to_run 2025-04-01 01:07:12,525 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:12:12 2025-04-01 01:08:07,571 INFO 102126:repeater task plot_tsstats_os75nb starting 1.8 seconds late 2025-04-01 01:08:08,350 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 01:08:08,350 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 01:08:08,350 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 01:23:05 2025-04-01 01:08:08,350 INFO 102126:repeater task grab_lastpings_os75nb starting 2.5 seconds late 2025-04-01 01:08:08,440 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 01:08:08,440 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 01:23:05 2025-04-01 01:08:08,441 INFO 102126:repeater task beamdiagnostics_os75nb starting 2.6 seconds late 2025-04-01 01:08:09,437 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:08:09,437 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:23:05 2025-04-01 01:08:09,438 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.6 seconds late 2025-04-01 01:08:10,431 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:08:10,431 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:23:05 2025-04-01 01:08:10,431 INFO 102126:repeater task hbinhcorrstats_posmv starting 4.5 seconds late 2025-04-01 01:08:11,545 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 01:08:11,545 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 01:08:11,545 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 01:23:05 2025-04-01 01:08:11,546 INFO 102126:repeater task plotposmv_posmv starting 5.6 seconds late 2025-04-01 01:08:13,696 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 01:08:13,696 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 01:23:05 2025-04-01 01:08:13,696 INFO 102126:repeater task plot_hbins starting 7.8 seconds late 2025-04-01 01:08:14,771 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 01:08:14,771 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 01:08:14,771 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 01:23:05 2025-04-01 01:08:19,775 INFO 102126:repeater task 3dayplots_os75nb starting 9.0 seconds late 2025-04-01 01:08:24,172 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:08:24,172 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:13:10 2025-04-01 01:12:14,379 INFO 102126:repeater task lastensq_os75nb starting 2.4 seconds late 2025-04-01 01:12:15,117 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:12:15,117 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -14940.785 2025-04-01 01:12:15,117 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:17:17 2025-04-01 01:12:15,118 INFO 102126:repeater task quick_os75nb starting 3.1 seconds late 2025-04-01 01:12:17,640 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:12:17,640 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:12:17,640 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:12:17,640 INFO 102126:repeater task quick_os75nb: added 49 extra increments to time_to_run 2025-04-01 01:12:17,640 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:17:17 2025-04-01 01:12:17,640 INFO 102126:repeater task lastfew_vec_os75nb starting 5.6 seconds late 2025-04-01 01:12:18,248 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:12:18,248 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:12:18,248 INFO 102126:repeater task lastfew_vec_os75nb: added 49 extra increments to time_to_run 2025-04-01 01:12:18,248 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:17:17 2025-04-01 01:13:13,299 INFO 102126:repeater task noisy_rawtemps_os75 starting 2.5 seconds late 2025-04-01 01:13:15,242 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 01:13:15,242 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 01:23:10 2025-04-01 01:13:15,242 INFO 102126:repeater task 3dayplots_os75nb starting 4.4 seconds late 2025-04-01 01:13:19,649 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:13:19,649 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:18:10 2025-04-01 01:13:19,649 INFO 102126:repeater task daily_netcdf starting 8.8 seconds late 2025-04-01 01:13:19,848 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 01:13:19,848 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 01:13:19,849 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 01:23:10 2025-04-01 01:17:20,067 INFO 102126:repeater task lastensq_os75nb starting 3.1 seconds late 2025-04-01 01:17:20,742 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:17:20,742 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -15246.473 2025-04-01 01:17:20,742 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:22:22 2025-04-01 01:17:20,742 INFO 102126:repeater task quick_os75nb starting 3.7 seconds late 2025-04-01 01:17:23,251 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:17:23,251 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:17:23,251 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:17:23,251 INFO 102126:repeater task quick_os75nb: added 50 extra increments to time_to_run 2025-04-01 01:17:23,251 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:22:22 2025-04-01 01:17:23,252 INFO 102126:repeater task lastfew_vec_os75nb starting 6.3 seconds late 2025-04-01 01:17:23,868 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:17:23,869 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:17:23,869 INFO 102126:repeater task lastfew_vec_os75nb: added 50 extra increments to time_to_run 2025-04-01 01:17:23,869 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:22:22 2025-04-01 01:18:13,917 INFO 102126:repeater task 3dayplots_os75nb starting 3.1 seconds late 2025-04-01 01:18:18,311 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:18:18,311 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:23:10 2025-04-01 01:22:23,547 INFO 102126:repeater task lastensq_os75nb starting 1.5 seconds late 2025-04-01 01:22:24,247 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:22:24,247 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -15549.953 2025-04-01 01:22:24,247 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:27:27 2025-04-01 01:22:24,247 INFO 102126:repeater task quick_os75nb starting 2.2 seconds late 2025-04-01 01:22:26,761 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:22:26,761 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:22:26,761 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:22:26,761 INFO 102126:repeater task quick_os75nb: added 51 extra increments to time_to_run 2025-04-01 01:22:26,761 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:27:27 2025-04-01 01:22:26,761 INFO 102126:repeater task lastfew_vec_os75nb starting 4.8 seconds late 2025-04-01 01:22:27,375 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:22:27,375 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:22:27,375 INFO 102126:repeater task lastfew_vec_os75nb: added 51 extra increments to time_to_run 2025-04-01 01:22:27,375 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:27:27 2025-04-01 01:23:07,411 INFO 102126:repeater task plot_tsstats_os75nb starting 1.6 seconds late 2025-04-01 01:23:08,185 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 01:23:08,185 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 01:23:08,185 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 01:38:05 2025-04-01 01:23:08,185 INFO 102126:repeater task grab_lastpings_os75nb starting 2.4 seconds late 2025-04-01 01:23:08,276 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 01:23:08,276 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 01:38:05 2025-04-01 01:23:08,276 INFO 102126:repeater task beamdiagnostics_os75nb starting 2.5 seconds late 2025-04-01 01:23:09,267 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:23:09,267 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:38:05 2025-04-01 01:23:09,267 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.5 seconds late 2025-04-01 01:23:10,266 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:23:10,266 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:38:05 2025-04-01 01:23:10,266 INFO 102126:repeater task ggatime_diagnostics starting 4.4 seconds late 2025-04-01 01:23:14,218 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 01:23:14,218 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:23:14,219 INFO 102126:repeater task ggatime_diagnostics starting 8.4 seconds late 2025-04-01 01:23:18,013 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 01:23:18,013 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:23:18,013 INFO 102126:repeater task ggatime_diagnostics starting 12.2 seconds late 2025-04-01 01:23:21,851 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 01:23:21,851 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:23:21,851 INFO 102126:repeater task hbinhcorrstats_posmv starting 15.9 seconds late 2025-04-01 01:23:22,957 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 01:23:22,957 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 01:23:22,957 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 01:38:05 2025-04-01 01:23:22,957 INFO 102126:repeater task plotposmv_posmv starting 17.1 seconds late 2025-04-01 01:23:25,135 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 01:23:25,135 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 01:38:05 2025-04-01 01:23:25,135 INFO 102126:repeater task plot_hbins starting 19.2 seconds late 2025-04-01 01:23:26,220 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 01:23:26,220 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 01:23:26,220 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 01:38:05 2025-04-01 01:23:26,220 INFO 102126:repeater task report_generator starting 20.3 seconds late 2025-04-01 01:23:29,329 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-04-01 01:23:29,329 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-04-01 01:23:29,329 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 02:23:05 2025-04-01 01:23:34,335 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.5 seconds late 2025-04-01 01:23:36,276 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 01:23:36,276 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 01:33:10 2025-04-01 01:23:36,277 INFO 102126:repeater task 3dayplots_os75nb starting 25.5 seconds late 2025-04-01 01:23:40,672 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:23:40,673 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:28:10 2025-04-01 01:23:40,673 INFO 102126:repeater task daily_netcdf starting 29.9 seconds late 2025-04-01 01:23:40,871 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 01:23:40,872 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 01:23:40,872 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 01:33:10 2025-04-01 01:27:31,085 INFO 102126:repeater task lastensq_os75nb starting 4.1 seconds late 2025-04-01 01:27:31,759 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:27:31,759 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -15857.491 2025-04-01 01:27:31,759 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:32:32 2025-04-01 01:27:31,759 INFO 102126:repeater task quick_os75nb starting 4.8 seconds late 2025-04-01 01:27:34,286 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:27:34,286 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:27:34,286 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:27:34,286 INFO 102126:repeater task quick_os75nb: added 52 extra increments to time_to_run 2025-04-01 01:27:34,286 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:32:32 2025-04-01 01:27:34,286 INFO 102126:repeater task lastfew_vec_os75nb starting 7.3 seconds late 2025-04-01 01:27:34,903 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:27:34,903 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:27:34,903 INFO 102126:repeater task lastfew_vec_os75nb: added 52 extra increments to time_to_run 2025-04-01 01:27:34,903 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:32:32 2025-04-01 01:28:14,944 INFO 102126:repeater task 3dayplots_os75nb starting 4.1 seconds late 2025-04-01 01:28:19,352 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:28:19,352 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:33:10 2025-04-01 01:32:34,595 INFO 102126:repeater task lastensq_os75nb starting 2.6 seconds late 2025-04-01 01:32:35,270 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:32:35,270 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -16161.001 2025-04-01 01:32:35,270 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:37:37 2025-04-01 01:32:35,270 INFO 102126:repeater task quick_os75nb starting 3.3 seconds late 2025-04-01 01:32:37,820 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:32:37,820 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:32:37,820 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:32:37,820 INFO 102126:repeater task quick_os75nb: added 53 extra increments to time_to_run 2025-04-01 01:32:37,820 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:37:37 2025-04-01 01:32:37,821 INFO 102126:repeater task lastfew_vec_os75nb starting 5.8 seconds late 2025-04-01 01:32:38,442 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:32:38,442 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:32:38,442 INFO 102126:repeater task lastfew_vec_os75nb: added 53 extra increments to time_to_run 2025-04-01 01:32:38,442 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:37:37 2025-04-01 01:33:13,477 INFO 102126:repeater task noisy_rawtemps_os75 starting 2.7 seconds late 2025-04-01 01:33:15,415 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 01:33:15,415 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 01:43:10 2025-04-01 01:33:15,416 INFO 102126:repeater task 3dayplots_os75nb starting 4.6 seconds late 2025-04-01 01:33:19,825 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:33:19,825 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:38:10 2025-04-01 01:33:19,826 INFO 102126:repeater task daily_netcdf starting 9.0 seconds late 2025-04-01 01:33:20,025 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 01:33:20,026 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 01:33:20,026 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 01:43:10 2025-04-01 01:37:40,255 INFO 102126:repeater task lastensq_os75nb starting 3.3 seconds late 2025-04-01 01:37:40,948 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:37:40,948 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -16466.661 2025-04-01 01:37:40,948 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:42:42 2025-04-01 01:37:40,948 INFO 102126:repeater task quick_os75nb starting 3.9 seconds late 2025-04-01 01:37:43,464 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:37:43,464 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:37:43,464 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:37:43,464 INFO 102126:repeater task quick_os75nb: added 54 extra increments to time_to_run 2025-04-01 01:37:43,464 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:42:42 2025-04-01 01:37:43,464 INFO 102126:repeater task lastfew_vec_os75nb starting 6.5 seconds late 2025-04-01 01:37:44,075 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:37:44,075 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:37:44,075 INFO 102126:repeater task lastfew_vec_os75nb: added 54 extra increments to time_to_run 2025-04-01 01:37:44,075 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:42:42 2025-04-01 01:38:09,095 INFO 102126:repeater task plot_tsstats_os75nb starting 3.3 seconds late 2025-04-01 01:38:09,877 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 01:38:09,877 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 01:38:09,877 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:38:09,878 INFO 102126:repeater task grab_lastpings_os75nb starting 4.1 seconds late 2025-04-01 01:38:09,968 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 01:38:09,968 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:38:09,968 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.2 seconds late 2025-04-01 01:38:10,953 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:38:10,953 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:38:10,953 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.1 seconds late 2025-04-01 01:38:11,948 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:38:11,948 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:38:11,948 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.0 seconds late 2025-04-01 01:38:13,081 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 01:38:13,081 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 01:38:13,081 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:38:13,081 INFO 102126:repeater task plotposmv_posmv starting 7.2 seconds late 2025-04-01 01:38:15,330 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 01:38:15,330 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:38:15,330 INFO 102126:repeater task plot_hbins starting 9.4 seconds late 2025-04-01 01:38:16,421 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 01:38:16,421 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 01:38:16,421 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 01:53:05 2025-04-01 01:38:21,425 INFO 102126:repeater task 3dayplots_os75nb starting 10.6 seconds late 2025-04-01 01:38:25,821 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:38:25,821 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:43:10 2025-04-01 01:42:46,043 INFO 102126:repeater task lastensq_os75nb starting 4.0 seconds late 2025-04-01 01:42:46,772 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:42:46,772 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -16772.449 2025-04-01 01:42:46,772 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:47:47 2025-04-01 01:42:46,772 INFO 102126:repeater task quick_os75nb starting 4.8 seconds late 2025-04-01 01:42:49,299 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:42:49,299 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:42:49,299 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:42:49,299 INFO 102126:repeater task quick_os75nb: added 55 extra increments to time_to_run 2025-04-01 01:42:49,299 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:47:47 2025-04-01 01:42:49,299 INFO 102126:repeater task lastfew_vec_os75nb starting 7.3 seconds late 2025-04-01 01:42:49,911 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:42:49,911 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:42:49,911 INFO 102126:repeater task lastfew_vec_os75nb: added 55 extra increments to time_to_run 2025-04-01 01:42:49,911 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:47:47 2025-04-01 01:43:14,931 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.1 seconds late 2025-04-01 01:43:16,853 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 01:43:16,853 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 01:53:10 2025-04-01 01:43:16,853 INFO 102126:repeater task 3dayplots_os75nb starting 6.0 seconds late 2025-04-01 01:43:21,260 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:43:21,260 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:48:10 2025-04-01 01:43:21,260 INFO 102126:repeater task daily_netcdf starting 10.4 seconds late 2025-04-01 01:43:21,461 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 01:43:21,461 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 01:43:21,461 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 01:53:10 2025-04-01 01:47:51,819 INFO 102126:repeater task lastensq_os75nb starting 4.8 seconds late 2025-04-01 01:47:52,588 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-04-01 01:47:52,588 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -17078.225 2025-04-01 01:47:52,588 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:52:52 2025-04-01 01:47:52,588 INFO 102126:repeater task quick_os75nb starting 5.6 seconds late 2025-04-01 01:47:55,102 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:47:55,102 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:47:55,102 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:47:55,102 INFO 102126:repeater task quick_os75nb: added 56 extra increments to time_to_run 2025-04-01 01:47:55,102 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:52:52 2025-04-01 01:47:55,102 INFO 102126:repeater task lastfew_vec_os75nb starting 8.1 seconds late 2025-04-01 01:47:55,711 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:47:55,711 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:47:55,711 INFO 102126:repeater task lastfew_vec_os75nb: added 56 extra increments to time_to_run 2025-04-01 01:47:55,711 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:52:52 2025-04-01 01:48:15,731 INFO 102126:repeater task 3dayplots_os75nb starting 4.9 seconds late 2025-04-01 01:48:20,138 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:48:20,138 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:53:10 2025-04-01 01:52:55,379 INFO 102126:repeater task lastensq_os75nb starting 3.4 seconds late 2025-04-01 01:52:56,058 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:52:56,058 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -17381.785 2025-04-01 01:52:56,058 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 01:57:57 2025-04-01 01:52:56,058 INFO 102126:repeater task quick_os75nb starting 4.1 seconds late 2025-04-01 01:52:58,577 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:52:58,578 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:52:58,578 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:52:58,578 INFO 102126:repeater task quick_os75nb: added 57 extra increments to time_to_run 2025-04-01 01:52:58,578 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 01:57:57 2025-04-01 01:52:58,578 INFO 102126:repeater task lastfew_vec_os75nb starting 6.6 seconds late 2025-04-01 01:52:59,197 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:52:59,197 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:52:59,198 INFO 102126:repeater task lastfew_vec_os75nb: added 57 extra increments to time_to_run 2025-04-01 01:52:59,198 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 01:57:57 2025-04-01 01:53:09,207 INFO 102126:repeater task plot_tsstats_os75nb starting 3.4 seconds late 2025-04-01 01:53:09,985 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 01:53:09,986 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 01:53:09,986 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 02:08:05 2025-04-01 01:53:09,986 INFO 102126:repeater task grab_lastpings_os75nb starting 4.2 seconds late 2025-04-01 01:53:10,076 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 01:53:10,077 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 02:08:05 2025-04-01 01:53:10,077 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.3 seconds late 2025-04-01 01:53:11,070 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:53:11,070 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:08:05 2025-04-01 01:53:11,071 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.3 seconds late 2025-04-01 01:53:12,072 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 01:53:12,072 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:08:05 2025-04-01 01:53:12,072 INFO 102126:repeater task daily_netcdf starting 1.3 seconds late 2025-04-01 01:53:12,271 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 01:53:12,271 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 01:53:12,271 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 02:03:10 2025-04-01 01:53:12,271 INFO 102126:repeater task ggatime_diagnostics starting 6.5 seconds late 2025-04-01 01:53:16,236 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 01:53:16,236 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 02:23:05 2025-04-01 01:53:16,236 INFO 102126:repeater task ggatime_diagnostics starting 10.4 seconds late 2025-04-01 01:53:20,047 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 01:53:20,047 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 02:23:05 2025-04-01 01:53:20,047 INFO 102126:repeater task ggatime_diagnostics starting 14.2 seconds late 2025-04-01 01:53:23,881 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 01:53:23,881 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 02:23:05 2025-04-01 01:53:23,881 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.0 seconds late 2025-04-01 01:53:24,981 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 01:53:24,982 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 01:53:24,982 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 02:08:05 2025-04-01 01:53:24,982 INFO 102126:repeater task plotposmv_posmv starting 19.1 seconds late 2025-04-01 01:53:27,136 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 01:53:27,136 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 02:08:05 2025-04-01 01:53:27,136 INFO 102126:repeater task plot_hbins starting 21.2 seconds late 2025-04-01 01:53:28,224 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 01:53:28,224 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 01:53:28,224 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 02:08:05 2025-04-01 01:53:33,227 INFO 102126:repeater task noisy_rawtemps_os75 starting 22.4 seconds late 2025-04-01 01:53:35,161 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 01:53:35,161 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 02:03:10 2025-04-01 01:53:35,162 INFO 102126:repeater task 3dayplots_os75nb starting 24.3 seconds late 2025-04-01 01:53:39,554 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:53:39,554 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 01:58:10 2025-04-01 01:57:59,775 INFO 102126:repeater task lastensq_os75nb starting 2.8 seconds late 2025-04-01 01:58:00,512 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 01:58:00,512 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -17686.181 2025-04-01 01:58:00,512 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:03:02 2025-04-01 01:58:00,512 INFO 102126:repeater task quick_os75nb starting 3.5 seconds late 2025-04-01 01:58:03,020 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 01:58:03,020 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 01:58:03,020 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:58:03,020 INFO 102126:repeater task quick_os75nb: added 58 extra increments to time_to_run 2025-04-01 01:58:03,020 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:03:02 2025-04-01 01:58:03,020 INFO 102126:repeater task lastfew_vec_os75nb starting 6.0 seconds late 2025-04-01 01:58:03,633 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 01:58:03,633 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 01:58:03,633 INFO 102126:repeater task lastfew_vec_os75nb: added 58 extra increments to time_to_run 2025-04-01 01:58:03,633 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:03:02 2025-04-01 01:58:13,645 INFO 102126:repeater task 3dayplots_os75nb starting 2.8 seconds late 2025-04-01 01:58:18,050 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 01:58:18,050 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:03:10 2025-04-01 02:03:03,312 INFO 102126:repeater task lastensq_os75nb starting 1.3 seconds late 2025-04-01 02:03:04,143 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-04-01 02:03:04,143 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -17989.718 2025-04-01 02:03:04,143 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:08:07 2025-04-01 02:03:04,143 INFO 102126:repeater task quick_os75nb starting 2.1 seconds late 2025-04-01 02:03:06,693 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:03:06,693 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:03:06,693 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:03:06,693 INFO 102126:repeater task quick_os75nb: added 59 extra increments to time_to_run 2025-04-01 02:03:06,693 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:08:07 2025-04-01 02:03:06,693 INFO 102126:repeater task lastfew_vec_os75nb starting 4.7 seconds late 2025-04-01 02:03:07,312 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:03:07,312 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:03:07,312 INFO 102126:repeater task lastfew_vec_os75nb: added 59 extra increments to time_to_run 2025-04-01 02:03:07,312 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:08:07 2025-04-01 02:03:12,318 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.5 seconds late 2025-04-01 02:03:14,261 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 02:03:14,261 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 02:13:10 2025-04-01 02:03:14,262 INFO 102126:repeater task 3dayplots_os75nb starting 3.4 seconds late 2025-04-01 02:03:18,668 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:03:18,668 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:08:10 2025-04-01 02:03:18,668 INFO 102126:repeater task daily_netcdf starting 7.9 seconds late 2025-04-01 02:03:18,868 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 02:03:18,868 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 02:03:18,868 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 02:13:10 2025-04-01 02:08:09,131 INFO 102126:repeater task lastensq_os75nb starting 2.1 seconds late 2025-04-01 02:08:09,799 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:08:09,799 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -18295.537 2025-04-01 02:08:09,799 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:13:12 2025-04-01 02:08:09,799 INFO 102126:repeater task quick_os75nb starting 2.8 seconds late 2025-04-01 02:08:12,334 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:08:12,334 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:08:12,334 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:08:12,334 INFO 102126:repeater task quick_os75nb: added 60 extra increments to time_to_run 2025-04-01 02:08:12,334 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:13:12 2025-04-01 02:08:12,335 INFO 102126:repeater task lastfew_vec_os75nb starting 5.3 seconds late 2025-04-01 02:08:12,951 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:08:12,951 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:08:12,951 INFO 102126:repeater task lastfew_vec_os75nb: added 60 extra increments to time_to_run 2025-04-01 02:08:12,951 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:13:12 2025-04-01 02:08:12,951 INFO 102126:repeater task 3dayplots_os75nb starting 2.1 seconds late 2025-04-01 02:08:17,350 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:08:17,350 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:13:10 2025-04-01 02:08:17,351 INFO 102126:repeater task plot_tsstats_os75nb starting 11.5 seconds late 2025-04-01 02:08:18,127 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 02:08:18,127 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 02:08:18,127 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 02:23:05 2025-04-01 02:08:18,127 INFO 102126:repeater task grab_lastpings_os75nb starting 12.3 seconds late 2025-04-01 02:08:18,217 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 02:08:18,217 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 02:23:05 2025-04-01 02:08:18,217 INFO 102126:repeater task beamdiagnostics_os75nb starting 12.4 seconds late 2025-04-01 02:08:19,206 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:08:19,206 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:23:05 2025-04-01 02:08:19,206 INFO 102126:repeater task beamdiagnostics_os75nb starting 13.4 seconds late 2025-04-01 02:08:20,201 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:08:20,201 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:23:05 2025-04-01 02:08:20,201 INFO 102126:repeater task hbinhcorrstats_posmv starting 14.3 seconds late 2025-04-01 02:08:21,310 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 02:08:21,310 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 02:08:21,310 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 02:23:05 2025-04-01 02:08:21,311 INFO 102126:repeater task plotposmv_posmv starting 15.4 seconds late 2025-04-01 02:08:23,408 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 02:08:23,408 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 02:23:05 2025-04-01 02:08:23,408 INFO 102126:repeater task plot_hbins starting 17.5 seconds late 2025-04-01 02:08:24,464 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 02:08:24,464 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 02:08:24,464 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 02:23:05 2025-04-01 02:13:14,739 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.9 seconds late 2025-04-01 02:13:16,670 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 02:13:16,671 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 02:23:10 2025-04-01 02:13:16,671 INFO 102126:repeater task lastensq_os75nb starting 4.7 seconds late 2025-04-01 02:13:17,339 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:13:17,339 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -18603.076 2025-04-01 02:13:17,339 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:18:17 2025-04-01 02:13:17,339 INFO 102126:repeater task quick_os75nb starting 5.3 seconds late 2025-04-01 02:13:19,864 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:13:19,864 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:13:19,864 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:13:19,864 INFO 102126:repeater task quick_os75nb: added 61 extra increments to time_to_run 2025-04-01 02:13:19,864 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:18:17 2025-04-01 02:13:19,865 INFO 102126:repeater task lastfew_vec_os75nb starting 7.9 seconds late 2025-04-01 02:13:20,494 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:13:20,494 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:13:20,494 INFO 102126:repeater task lastfew_vec_os75nb: added 61 extra increments to time_to_run 2025-04-01 02:13:20,494 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:18:17 2025-04-01 02:13:20,494 INFO 102126:repeater task 3dayplots_os75nb starting 9.7 seconds late 2025-04-01 02:13:24,889 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:13:24,889 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:18:10 2025-04-01 02:13:24,889 INFO 102126:repeater task daily_netcdf starting 14.1 seconds late 2025-04-01 02:13:25,089 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 02:13:25,089 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 02:13:25,089 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 02:23:10 2025-04-01 02:18:15,345 INFO 102126:repeater task 3dayplots_os75nb starting 4.5 seconds late 2025-04-01 02:18:19,746 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:18:19,746 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:23:10 2025-04-01 02:18:24,752 INFO 102126:repeater task lastensq_os75nb starting 7.8 seconds late 2025-04-01 02:18:25,438 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:18:25,438 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -18911.157 2025-04-01 02:18:25,438 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:23:22 2025-04-01 02:18:25,438 INFO 102126:repeater task quick_os75nb starting 8.4 seconds late 2025-04-01 02:18:27,971 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:18:27,971 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:18:27,971 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:18:27,971 INFO 102126:repeater task quick_os75nb: added 62 extra increments to time_to_run 2025-04-01 02:18:27,971 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:23:22 2025-04-01 02:18:27,971 INFO 102126:repeater task lastfew_vec_os75nb starting 11.0 seconds late 2025-04-01 02:18:28,579 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:18:28,579 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:18:28,579 INFO 102126:repeater task lastfew_vec_os75nb: added 62 extra increments to time_to_run 2025-04-01 02:18:28,579 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:23:22 2025-04-01 02:23:08,835 INFO 102126:repeater task plot_tsstats_os75nb starting 3.0 seconds late 2025-04-01 02:23:09,612 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 02:23:09,612 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 02:23:09,612 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 02:38:05 2025-04-01 02:23:09,613 INFO 102126:repeater task grab_lastpings_os75nb starting 3.8 seconds late 2025-04-01 02:23:09,703 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 02:23:09,703 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 02:38:05 2025-04-01 02:23:09,703 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.9 seconds late 2025-04-01 02:23:10,693 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:23:10,693 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:38:05 2025-04-01 02:23:10,693 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.9 seconds late 2025-04-01 02:23:11,694 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:23:11,694 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:38:05 2025-04-01 02:23:11,694 INFO 102126:repeater task daily_netcdf starting 0.9 seconds late 2025-04-01 02:23:11,895 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 02:23:11,895 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 02:23:11,895 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 02:33:10 2025-04-01 02:23:11,895 INFO 102126:repeater task ggatime_diagnostics starting 6.1 seconds late 2025-04-01 02:23:15,910 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 02:23:15,910 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:23:15,910 INFO 102126:repeater task ggatime_diagnostics starting 10.1 seconds late 2025-04-01 02:23:19,727 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 02:23:19,728 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:23:19,728 INFO 102126:repeater task ggatime_diagnostics starting 13.9 seconds late 2025-04-01 02:23:23,565 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 02:23:23,565 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:23:23,565 INFO 102126:repeater task hbinhcorrstats_posmv starting 17.7 seconds late 2025-04-01 02:23:24,682 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 02:23:24,682 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 02:23:24,682 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 02:38:05 2025-04-01 02:23:24,682 INFO 102126:repeater task plotposmv_posmv starting 18.8 seconds late 2025-04-01 02:23:26,792 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 02:23:26,792 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 02:38:05 2025-04-01 02:23:26,792 INFO 102126:repeater task plot_hbins starting 20.9 seconds late 2025-04-01 02:23:27,850 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 02:23:27,851 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 02:23:27,851 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 02:38:05 2025-04-01 02:23:27,851 INFO 102126:repeater task report_generator starting 21.9 seconds late 2025-04-01 02:23:30,962 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-04-01 02:23:30,962 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-04-01 02:23:30,962 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 03:23:05 2025-04-01 02:23:35,963 INFO 102126:repeater task noisy_rawtemps_os75 starting 25.1 seconds late 2025-04-01 02:23:37,897 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 02:23:37,897 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 02:33:10 2025-04-01 02:23:37,897 INFO 102126:repeater task lastensq_os75nb starting 15.9 seconds late 2025-04-01 02:23:38,570 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:23:38,570 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -19224.303 2025-04-01 02:23:38,570 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:28:27 2025-04-01 02:23:38,570 INFO 102126:repeater task quick_os75nb starting 16.6 seconds late 2025-04-01 02:23:41,080 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:23:41,081 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:23:41,081 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:23:41,081 INFO 102126:repeater task quick_os75nb: added 63 extra increments to time_to_run 2025-04-01 02:23:41,081 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:28:27 2025-04-01 02:23:41,081 INFO 102126:repeater task lastfew_vec_os75nb starting 19.1 seconds late 2025-04-01 02:23:41,698 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:23:41,698 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:23:41,698 INFO 102126:repeater task lastfew_vec_os75nb: added 63 extra increments to time_to_run 2025-04-01 02:23:41,698 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:28:27 2025-04-01 02:23:41,699 INFO 102126:repeater task 3dayplots_os75nb starting 30.9 seconds late 2025-04-01 02:23:46,083 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:23:46,083 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:28:10 2025-04-01 02:28:11,315 INFO 102126:repeater task 3dayplots_os75nb starting 0.5 seconds late 2025-04-01 02:28:15,720 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:28:15,720 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:33:10 2025-04-01 02:28:30,733 INFO 102126:repeater task lastensq_os75nb starting 3.7 seconds late 2025-04-01 02:28:31,400 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:28:31,400 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -19517.138 2025-04-01 02:28:31,400 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:33:32 2025-04-01 02:28:31,400 INFO 102126:repeater task quick_os75nb starting 4.4 seconds late 2025-04-01 02:28:33,915 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:28:33,916 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:28:33,916 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:28:33,916 INFO 102126:repeater task quick_os75nb: added 64 extra increments to time_to_run 2025-04-01 02:28:33,916 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:33:32 2025-04-01 02:28:33,916 INFO 102126:repeater task lastfew_vec_os75nb starting 6.9 seconds late 2025-04-01 02:28:34,538 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:28:34,538 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:28:34,538 INFO 102126:repeater task lastfew_vec_os75nb: added 64 extra increments to time_to_run 2025-04-01 02:28:34,538 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:33:32 2025-04-01 02:33:14,775 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.0 seconds late 2025-04-01 02:33:16,716 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 02:33:16,716 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 02:43:10 2025-04-01 02:33:16,717 INFO 102126:repeater task 3dayplots_os75nb starting 5.9 seconds late 2025-04-01 02:33:21,122 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:33:21,122 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:38:10 2025-04-01 02:33:21,122 INFO 102126:repeater task daily_netcdf starting 10.3 seconds late 2025-04-01 02:33:21,321 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 02:33:21,321 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 02:33:21,321 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 02:43:10 2025-04-01 02:33:36,331 INFO 102126:repeater task lastensq_os75nb starting 4.3 seconds late 2025-04-01 02:33:37,001 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:33:37,002 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -19822.737 2025-04-01 02:33:37,002 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:38:37 2025-04-01 02:33:37,002 INFO 102126:repeater task quick_os75nb starting 5.0 seconds late 2025-04-01 02:33:39,518 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:33:39,518 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:33:39,519 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:33:39,519 INFO 102126:repeater task quick_os75nb: added 65 extra increments to time_to_run 2025-04-01 02:33:39,519 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:38:37 2025-04-01 02:33:39,519 INFO 102126:repeater task lastfew_vec_os75nb starting 7.5 seconds late 2025-04-01 02:33:40,123 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:33:40,124 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:33:40,124 INFO 102126:repeater task lastfew_vec_os75nb: added 65 extra increments to time_to_run 2025-04-01 02:33:40,124 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:38:37 2025-04-01 02:38:10,364 INFO 102126:repeater task plot_tsstats_os75nb starting 4.5 seconds late 2025-04-01 02:38:11,135 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 02:38:11,135 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 02:38:11,135 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:38:11,135 INFO 102126:repeater task grab_lastpings_os75nb starting 5.3 seconds late 2025-04-01 02:38:11,226 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 02:38:11,226 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:38:11,226 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.4 seconds late 2025-04-01 02:38:12,218 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:38:12,218 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:38:12,218 INFO 102126:repeater task beamdiagnostics_os75nb starting 6.4 seconds late 2025-04-01 02:38:13,212 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:38:13,212 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:38:13,213 INFO 102126:repeater task hbinhcorrstats_posmv starting 7.3 seconds late 2025-04-01 02:38:14,313 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 02:38:14,313 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 02:38:14,313 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:38:14,313 INFO 102126:repeater task plotposmv_posmv starting 8.4 seconds late 2025-04-01 02:38:16,460 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 02:38:16,460 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:38:16,460 INFO 102126:repeater task plot_hbins starting 10.6 seconds late 2025-04-01 02:38:17,522 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 02:38:17,522 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 02:38:17,522 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 02:53:05 2025-04-01 02:38:22,527 INFO 102126:repeater task 3dayplots_os75nb starting 11.7 seconds late 2025-04-01 02:38:26,917 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:38:26,917 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:43:10 2025-04-01 02:38:41,935 INFO 102126:repeater task lastensq_os75nb starting 4.9 seconds late 2025-04-01 02:38:42,622 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:38:42,623 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -20128.340 2025-04-01 02:38:42,623 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:43:42 2025-04-01 02:38:42,623 INFO 102126:repeater task quick_os75nb starting 5.6 seconds late 2025-04-01 02:38:45,132 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:38:45,132 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:38:45,132 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:38:45,132 INFO 102126:repeater task quick_os75nb: added 66 extra increments to time_to_run 2025-04-01 02:38:45,132 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:43:42 2025-04-01 02:38:45,132 INFO 102126:repeater task lastfew_vec_os75nb starting 8.1 seconds late 2025-04-01 02:38:45,766 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:38:45,766 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:38:45,766 INFO 102126:repeater task lastfew_vec_os75nb: added 66 extra increments to time_to_run 2025-04-01 02:38:45,766 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:43:42 2025-04-01 02:43:10,994 INFO 102126:repeater task noisy_rawtemps_os75 starting 0.2 seconds late 2025-04-01 02:43:12,932 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 02:43:12,932 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 02:53:10 2025-04-01 02:43:12,933 INFO 102126:repeater task 3dayplots_os75nb starting 2.1 seconds late 2025-04-01 02:43:17,329 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:43:17,329 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:48:10 2025-04-01 02:43:17,330 INFO 102126:repeater task daily_netcdf starting 6.5 seconds late 2025-04-01 02:43:17,529 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 02:43:17,529 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 02:43:17,529 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 02:53:10 2025-04-01 02:43:42,551 INFO 102126:repeater task lastensq_os75nb starting 0.6 seconds late 2025-04-01 02:43:43,234 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:43:43,234 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -20428.957 2025-04-01 02:43:43,234 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:48:47 2025-04-01 02:43:43,235 INFO 102126:repeater task quick_os75nb starting 1.2 seconds late 2025-04-01 02:43:45,749 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:43:45,749 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:43:45,750 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:43:45,750 INFO 102126:repeater task quick_os75nb: added 67 extra increments to time_to_run 2025-04-01 02:43:45,750 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:48:47 2025-04-01 02:43:45,750 INFO 102126:repeater task lastfew_vec_os75nb starting 3.8 seconds late 2025-04-01 02:43:46,362 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:43:46,362 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:43:46,362 INFO 102126:repeater task lastfew_vec_os75nb: added 67 extra increments to time_to_run 2025-04-01 02:43:46,362 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:48:47 2025-04-01 02:48:11,611 INFO 102126:repeater task 3dayplots_os75nb starting 0.8 seconds late 2025-04-01 02:48:16,001 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:48:16,001 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:53:10 2025-04-01 02:48:51,033 INFO 102126:repeater task lastensq_os75nb starting 4.0 seconds late 2025-04-01 02:48:51,719 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:48:51,719 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -20737.438 2025-04-01 02:48:51,719 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:53:52 2025-04-01 02:48:51,719 INFO 102126:repeater task quick_os75nb starting 4.7 seconds late 2025-04-01 02:48:54,246 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:48:54,246 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:48:54,246 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:48:54,246 INFO 102126:repeater task quick_os75nb: added 68 extra increments to time_to_run 2025-04-01 02:48:54,246 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:53:52 2025-04-01 02:48:54,246 INFO 102126:repeater task lastfew_vec_os75nb starting 7.2 seconds late 2025-04-01 02:48:54,862 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:48:54,862 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:48:54,862 INFO 102126:repeater task lastfew_vec_os75nb: added 68 extra increments to time_to_run 2025-04-01 02:48:54,862 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:53:52 2025-04-01 02:53:10,087 INFO 102126:repeater task plot_tsstats_os75nb starting 4.3 seconds late 2025-04-01 02:53:10,846 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 02:53:10,847 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 02:53:10,847 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 03:08:05 2025-04-01 02:53:10,847 INFO 102126:repeater task grab_lastpings_os75nb starting 5.0 seconds late 2025-04-01 02:53:10,937 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 02:53:10,937 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 03:08:05 2025-04-01 02:53:10,937 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.1 seconds late 2025-04-01 02:53:11,930 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:53:11,930 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:08:05 2025-04-01 02:53:11,930 INFO 102126:repeater task beamdiagnostics_os75nb starting 6.1 seconds late 2025-04-01 02:53:12,925 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 02:53:12,925 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:08:05 2025-04-01 02:53:12,925 INFO 102126:repeater task daily_netcdf starting 2.1 seconds late 2025-04-01 02:53:13,124 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 02:53:13,124 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 02:53:13,124 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 03:03:10 2025-04-01 02:53:13,125 INFO 102126:repeater task ggatime_diagnostics starting 7.3 seconds late 2025-04-01 02:53:17,146 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 02:53:17,146 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 03:23:05 2025-04-01 02:53:17,146 INFO 102126:repeater task ggatime_diagnostics starting 11.3 seconds late 2025-04-01 02:53:20,956 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 02:53:20,957 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 03:23:05 2025-04-01 02:53:20,957 INFO 102126:repeater task ggatime_diagnostics starting 15.1 seconds late 2025-04-01 02:53:24,820 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-04-01 02:53:24,820 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 03:23:05 2025-04-01 02:53:24,820 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.9 seconds late 2025-04-01 02:53:25,931 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 02:53:25,931 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 02:53:25,931 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 03:08:05 2025-04-01 02:53:25,931 INFO 102126:repeater task plotposmv_posmv starting 20.0 seconds late 2025-04-01 02:53:28,072 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 02:53:28,072 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 03:08:05 2025-04-01 02:53:28,073 INFO 102126:repeater task plot_hbins starting 22.2 seconds late 2025-04-01 02:53:29,140 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 02:53:29,140 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 02:53:29,140 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 03:08:05 2025-04-01 02:53:34,145 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.3 seconds late 2025-04-01 02:53:36,076 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 02:53:36,076 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 03:03:10 2025-04-01 02:53:36,076 INFO 102126:repeater task 3dayplots_os75nb starting 25.3 seconds late 2025-04-01 02:53:40,477 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:53:40,477 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 02:58:10 2025-04-01 02:53:55,491 INFO 102126:repeater task lastensq_os75nb starting 3.5 seconds late 2025-04-01 02:53:56,174 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:53:56,174 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -21041.897 2025-04-01 02:53:56,174 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 02:58:57 2025-04-01 02:53:56,174 INFO 102126:repeater task quick_os75nb starting 4.2 seconds late 2025-04-01 02:53:58,686 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:53:58,687 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:53:58,687 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:53:58,687 INFO 102126:repeater task quick_os75nb: added 69 extra increments to time_to_run 2025-04-01 02:53:58,687 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 02:58:57 2025-04-01 02:53:58,687 INFO 102126:repeater task lastfew_vec_os75nb starting 6.7 seconds late 2025-04-01 02:53:59,304 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:53:59,304 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:53:59,304 INFO 102126:repeater task lastfew_vec_os75nb: added 69 extra increments to time_to_run 2025-04-01 02:53:59,304 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 02:58:57 2025-04-01 02:58:14,543 INFO 102126:repeater task 3dayplots_os75nb starting 3.7 seconds late 2025-04-01 02:58:18,945 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 02:58:18,945 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:03:10 2025-04-01 02:58:58,983 INFO 102126:repeater task lastensq_os75nb starting 2.0 seconds late 2025-04-01 02:58:59,675 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 02:58:59,675 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -21345.388 2025-04-01 02:58:59,675 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:04:02 2025-04-01 02:58:59,675 INFO 102126:repeater task quick_os75nb starting 2.7 seconds late 2025-04-01 02:59:02,184 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 02:59:02,185 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 02:59:02,185 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:59:02,185 INFO 102126:repeater task quick_os75nb: added 70 extra increments to time_to_run 2025-04-01 02:59:02,185 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:04:02 2025-04-01 02:59:02,185 INFO 102126:repeater task lastfew_vec_os75nb starting 5.2 seconds late 2025-04-01 02:59:02,797 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 02:59:02,798 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 02:59:02,798 INFO 102126:repeater task lastfew_vec_os75nb: added 70 extra increments to time_to_run 2025-04-01 02:59:02,798 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:04:02 2025-04-01 03:03:13,037 INFO 102126:repeater task noisy_rawtemps_os75 starting 2.2 seconds late 2025-04-01 03:03:14,955 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 03:03:14,955 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 03:13:10 2025-04-01 03:03:14,956 INFO 102126:repeater task 3dayplots_os75nb starting 4.1 seconds late 2025-04-01 03:03:19,349 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:03:19,349 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:08:10 2025-04-01 03:03:19,350 INFO 102126:repeater task daily_netcdf starting 8.5 seconds late 2025-04-01 03:03:19,548 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 03:03:19,548 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 03:03:19,548 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 03:13:10 2025-04-01 03:04:04,591 INFO 102126:repeater task lastensq_os75nb starting 2.6 seconds late 2025-04-01 03:04:05,278 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:04:05,278 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -21650.997 2025-04-01 03:04:05,278 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:09:07 2025-04-01 03:04:05,278 INFO 102126:repeater task quick_os75nb starting 3.3 seconds late 2025-04-01 03:04:07,790 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:04:07,790 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:04:07,790 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:04:07,790 INFO 102126:repeater task quick_os75nb: added 71 extra increments to time_to_run 2025-04-01 03:04:07,790 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:09:07 2025-04-01 03:04:07,790 INFO 102126:repeater task lastfew_vec_os75nb starting 5.8 seconds late 2025-04-01 03:04:08,412 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:04:08,412 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:04:08,412 INFO 102126:repeater task lastfew_vec_os75nb: added 71 extra increments to time_to_run 2025-04-01 03:04:08,412 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:09:07 2025-04-01 03:08:08,613 INFO 102126:repeater task plot_tsstats_os75nb starting 2.8 seconds late 2025-04-01 03:08:09,387 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 03:08:09,387 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 03:08:09,387 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 03:23:05 2025-04-01 03:08:09,387 INFO 102126:repeater task grab_lastpings_os75nb starting 3.6 seconds late 2025-04-01 03:08:09,477 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 03:08:09,477 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 03:23:05 2025-04-01 03:08:09,477 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.7 seconds late 2025-04-01 03:08:10,469 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:08:10,469 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:23:05 2025-04-01 03:08:10,469 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.7 seconds late 2025-04-01 03:08:11,463 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:08:11,463 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:23:05 2025-04-01 03:08:11,464 INFO 102126:repeater task hbinhcorrstats_posmv starting 5.6 seconds late 2025-04-01 03:08:12,573 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 03:08:12,573 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 03:08:12,574 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 03:23:05 2025-04-01 03:08:12,574 INFO 102126:repeater task plotposmv_posmv starting 6.7 seconds late 2025-04-01 03:08:14,722 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 03:08:14,722 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 03:23:05 2025-04-01 03:08:14,722 INFO 102126:repeater task plot_hbins starting 8.8 seconds late 2025-04-01 03:08:15,795 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 03:08:15,795 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 03:08:15,795 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 03:23:05 2025-04-01 03:08:20,799 INFO 102126:repeater task 3dayplots_os75nb starting 10.0 seconds late 2025-04-01 03:08:25,193 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:08:25,193 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:13:10 2025-04-01 03:09:10,229 INFO 102126:repeater task lastensq_os75nb starting 3.2 seconds late 2025-04-01 03:09:10,899 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:09:10,899 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -21956.634 2025-04-01 03:09:10,899 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:14:12 2025-04-01 03:09:10,900 INFO 102126:repeater task quick_os75nb starting 3.9 seconds late 2025-04-01 03:09:13,420 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:09:13,420 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:09:13,420 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:09:13,420 INFO 102126:repeater task quick_os75nb: added 72 extra increments to time_to_run 2025-04-01 03:09:13,420 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:14:12 2025-04-01 03:09:13,421 INFO 102126:repeater task lastfew_vec_os75nb starting 6.4 seconds late 2025-04-01 03:09:14,044 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:09:14,044 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:09:14,044 INFO 102126:repeater task lastfew_vec_os75nb: added 72 extra increments to time_to_run 2025-04-01 03:09:14,044 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:14:12 2025-04-01 03:13:14,253 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.4 seconds late 2025-04-01 03:13:16,202 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 03:13:16,203 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 03:23:10 2025-04-01 03:13:16,203 INFO 102126:repeater task 3dayplots_os75nb starting 5.4 seconds late 2025-04-01 03:13:20,607 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:13:20,607 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:18:10 2025-04-01 03:13:20,607 INFO 102126:repeater task daily_netcdf starting 9.8 seconds late 2025-04-01 03:13:20,807 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 03:13:20,807 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 03:13:20,807 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 03:23:10 2025-04-01 03:14:15,855 INFO 102126:repeater task lastensq_os75nb starting 3.9 seconds late 2025-04-01 03:14:16,527 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:14:16,527 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -22262.260 2025-04-01 03:14:16,528 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:19:17 2025-04-01 03:14:16,528 INFO 102126:repeater task quick_os75nb starting 4.5 seconds late 2025-04-01 03:14:19,046 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:14:19,046 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:14:19,046 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:14:19,046 INFO 102126:repeater task quick_os75nb: added 73 extra increments to time_to_run 2025-04-01 03:14:19,046 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:19:17 2025-04-01 03:14:19,046 INFO 102126:repeater task lastfew_vec_os75nb starting 7.0 seconds late 2025-04-01 03:14:19,669 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:14:19,670 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:14:19,670 INFO 102126:repeater task lastfew_vec_os75nb: added 73 extra increments to time_to_run 2025-04-01 03:14:19,670 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:19:17 2025-04-01 03:18:14,875 INFO 102126:repeater task 3dayplots_os75nb starting 4.1 seconds late 2025-04-01 03:18:19,279 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:18:19,279 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:23:10 2025-04-01 03:19:19,327 INFO 102126:repeater task lastensq_os75nb starting 2.3 seconds late 2025-04-01 03:19:20,007 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:19:20,007 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -22565.733 2025-04-01 03:19:20,007 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:24:22 2025-04-01 03:19:20,007 INFO 102126:repeater task quick_os75nb starting 3.0 seconds late 2025-04-01 03:19:22,515 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:19:22,515 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:19:22,515 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:19:22,515 INFO 102126:repeater task quick_os75nb: added 74 extra increments to time_to_run 2025-04-01 03:19:22,515 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:24:22 2025-04-01 03:19:22,516 INFO 102126:repeater task lastfew_vec_os75nb starting 5.5 seconds late 2025-04-01 03:19:23,125 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:19:23,125 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:19:23,125 INFO 102126:repeater task lastfew_vec_os75nb: added 74 extra increments to time_to_run 2025-04-01 03:19:23,125 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:24:22 2025-04-01 03:23:08,323 INFO 102126:repeater task plot_tsstats_os75nb starting 2.5 seconds late 2025-04-01 03:23:09,095 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 03:23:09,095 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 03:23:09,095 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 03:38:05 2025-04-01 03:23:09,095 INFO 102126:repeater task grab_lastpings_os75nb starting 3.3 seconds late 2025-04-01 03:23:09,186 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 03:23:09,186 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 03:38:05 2025-04-01 03:23:09,186 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.4 seconds late 2025-04-01 03:23:10,178 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:23:10,178 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:38:05 2025-04-01 03:23:10,178 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.4 seconds late 2025-04-01 03:23:11,170 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:23:11,170 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:38:05 2025-04-01 03:23:11,170 INFO 102126:repeater task daily_netcdf starting 0.4 seconds late 2025-04-01 03:23:11,370 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 03:23:11,370 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 03:23:11,370 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 03:33:10 2025-04-01 03:23:11,370 INFO 102126:repeater task ggatime_diagnostics starting 5.6 seconds late 2025-04-01 03:23:15,370 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 03:23:15,371 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:23:15,371 INFO 102126:repeater task ggatime_diagnostics starting 9.6 seconds late 2025-04-01 03:23:19,138 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 03:23:19,138 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:23:19,138 INFO 102126:repeater task ggatime_diagnostics starting 13.3 seconds late 2025-04-01 03:23:22,973 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 03:23:22,973 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:23:22,973 INFO 102126:repeater task hbinhcorrstats_posmv starting 17.1 seconds late 2025-04-01 03:23:24,091 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 03:23:24,091 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 03:23:24,091 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 03:38:05 2025-04-01 03:23:24,091 INFO 102126:repeater task plotposmv_posmv starting 18.2 seconds late 2025-04-01 03:23:26,274 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 03:23:26,275 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 03:38:05 2025-04-01 03:23:26,275 INFO 102126:repeater task plot_hbins starting 20.4 seconds late 2025-04-01 03:23:27,341 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 03:23:27,341 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 03:23:27,341 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 03:38:05 2025-04-01 03:23:27,341 INFO 102126:repeater task report_generator starting 21.4 seconds late 2025-04-01 03:23:30,460 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-04-01 03:23:30,460 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-04-01 03:23:30,460 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 04:23:05 2025-04-01 03:23:35,465 INFO 102126:repeater task noisy_rawtemps_os75 starting 24.6 seconds late 2025-04-01 03:23:37,431 INFO 102126:repeater task noisy_rawtemps_os75 ended after 2.0 seconds 2025-04-01 03:23:37,431 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 03:33:10 2025-04-01 03:23:37,431 INFO 102126:repeater task 3dayplots_os75nb starting 26.6 seconds late 2025-04-01 03:23:41,837 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:23:41,837 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:28:10 2025-04-01 03:24:26,875 INFO 102126:repeater task lastensq_os75nb starting 4.9 seconds late 2025-04-01 03:24:27,550 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:24:27,550 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -22873.281 2025-04-01 03:24:27,550 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:29:27 2025-04-01 03:24:27,550 INFO 102126:repeater task quick_os75nb starting 5.6 seconds late 2025-04-01 03:24:30,093 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:24:30,093 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:24:30,093 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:24:30,093 INFO 102126:repeater task quick_os75nb: added 75 extra increments to time_to_run 2025-04-01 03:24:30,093 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:29:27 2025-04-01 03:24:30,093 INFO 102126:repeater task lastfew_vec_os75nb starting 8.1 seconds late 2025-04-01 03:24:30,705 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:24:30,705 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:24:30,705 INFO 102126:repeater task lastfew_vec_os75nb: added 75 extra increments to time_to_run 2025-04-01 03:24:30,705 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:29:27 2025-04-01 03:28:10,899 INFO 102126:repeater task 3dayplots_os75nb starting 0.1 seconds late 2025-04-01 03:28:15,318 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:28:15,318 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:33:10 2025-04-01 03:29:30,383 INFO 102126:repeater task lastensq_os75nb starting 3.4 seconds late 2025-04-01 03:29:31,076 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:29:31,076 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -23176.789 2025-04-01 03:29:31,076 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:34:32 2025-04-01 03:29:31,076 INFO 102126:repeater task quick_os75nb starting 4.1 seconds late 2025-04-01 03:29:33,590 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:29:33,590 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:29:33,590 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:29:33,590 INFO 102126:repeater task quick_os75nb: added 76 extra increments to time_to_run 2025-04-01 03:29:33,590 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:34:32 2025-04-01 03:29:33,590 INFO 102126:repeater task lastfew_vec_os75nb starting 6.6 seconds late 2025-04-01 03:29:34,197 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:29:34,197 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:29:34,198 INFO 102126:repeater task lastfew_vec_os75nb: added 76 extra increments to time_to_run 2025-04-01 03:29:34,198 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:34:32 2025-04-01 03:33:14,398 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.6 seconds late 2025-04-01 03:33:16,334 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 03:33:16,334 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 03:43:10 2025-04-01 03:33:16,335 INFO 102126:repeater task 3dayplots_os75nb starting 5.5 seconds late 2025-04-01 03:33:20,729 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:33:20,729 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:38:10 2025-04-01 03:33:20,729 INFO 102126:repeater task daily_netcdf starting 9.9 seconds late 2025-04-01 03:33:20,928 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 03:33:20,928 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 03:33:20,929 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 03:43:10 2025-04-01 03:34:35,997 INFO 102126:repeater task lastensq_os75nb starting 4.0 seconds late 2025-04-01 03:34:36,687 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:34:36,687 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -23482.402 2025-04-01 03:34:36,687 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:39:37 2025-04-01 03:34:36,687 INFO 102126:repeater task quick_os75nb starting 4.7 seconds late 2025-04-01 03:34:39,196 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:34:39,196 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:34:39,196 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:34:39,196 INFO 102126:repeater task quick_os75nb: added 77 extra increments to time_to_run 2025-04-01 03:34:39,196 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:39:37 2025-04-01 03:34:39,196 INFO 102126:repeater task lastfew_vec_os75nb starting 7.2 seconds late 2025-04-01 03:34:39,817 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:34:39,818 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:34:39,818 INFO 102126:repeater task lastfew_vec_os75nb: added 77 extra increments to time_to_run 2025-04-01 03:34:39,818 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:39:37 2025-04-01 03:38:10,005 INFO 102126:repeater task plot_tsstats_os75nb starting 4.2 seconds late 2025-04-01 03:38:10,779 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 03:38:10,779 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 03:38:10,779 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:38:10,779 INFO 102126:repeater task grab_lastpings_os75nb starting 5.0 seconds late 2025-04-01 03:38:10,869 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 03:38:10,869 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:38:10,869 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.1 seconds late 2025-04-01 03:38:11,860 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:38:11,860 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:38:11,860 INFO 102126:repeater task beamdiagnostics_os75nb starting 6.0 seconds late 2025-04-01 03:38:12,853 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:38:12,853 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:38:12,853 INFO 102126:repeater task hbinhcorrstats_posmv starting 7.0 seconds late 2025-04-01 03:38:13,956 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 03:38:13,956 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 03:38:13,956 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:38:13,956 INFO 102126:repeater task plotposmv_posmv starting 8.1 seconds late 2025-04-01 03:38:16,099 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 03:38:16,099 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:38:16,099 INFO 102126:repeater task plot_hbins starting 10.2 seconds late 2025-04-01 03:38:17,165 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 03:38:17,165 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 03:38:17,165 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 03:53:05 2025-04-01 03:38:22,170 INFO 102126:repeater task 3dayplots_os75nb starting 11.4 seconds late 2025-04-01 03:38:26,559 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:38:26,560 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:43:10 2025-04-01 03:39:41,627 INFO 102126:repeater task lastensq_os75nb starting 4.6 seconds late 2025-04-01 03:39:42,304 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:39:42,304 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -23788.033 2025-04-01 03:39:42,305 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:44:42 2025-04-01 03:39:42,305 INFO 102126:repeater task quick_os75nb starting 5.3 seconds late 2025-04-01 03:39:44,820 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:39:44,820 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:39:44,820 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:39:44,820 INFO 102126:repeater task quick_os75nb: added 78 extra increments to time_to_run 2025-04-01 03:39:44,820 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:44:42 2025-04-01 03:39:44,820 INFO 102126:repeater task lastfew_vec_os75nb starting 7.8 seconds late 2025-04-01 03:39:45,433 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:39:45,433 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:39:45,433 INFO 102126:repeater task lastfew_vec_os75nb: added 78 extra increments to time_to_run 2025-04-01 03:39:45,433 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:44:42 2025-04-01 03:43:15,647 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.8 seconds late 2025-04-01 03:43:17,581 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 03:43:17,581 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 03:53:10 2025-04-01 03:43:17,581 INFO 102126:repeater task 3dayplots_os75nb starting 6.8 seconds late 2025-04-01 03:43:21,973 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:43:21,973 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:48:10 2025-04-01 03:43:21,973 INFO 102126:repeater task daily_netcdf starting 11.2 seconds late 2025-04-01 03:43:22,173 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 03:43:22,173 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 03:43:22,173 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 03:53:10 2025-04-01 03:44:42,247 INFO 102126:repeater task lastensq_os75nb starting 0.2 seconds late 2025-04-01 03:44:42,938 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:44:42,939 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -24088.653 2025-04-01 03:44:42,939 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:49:47 2025-04-01 03:44:42,939 INFO 102126:repeater task quick_os75nb starting 0.9 seconds late 2025-04-01 03:44:45,449 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:44:45,449 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:44:45,449 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:44:45,449 INFO 102126:repeater task quick_os75nb: added 79 extra increments to time_to_run 2025-04-01 03:44:45,449 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:49:47 2025-04-01 03:44:45,450 INFO 102126:repeater task lastfew_vec_os75nb starting 3.5 seconds late 2025-04-01 03:44:46,065 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:44:46,065 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:44:46,065 INFO 102126:repeater task lastfew_vec_os75nb: added 79 extra increments to time_to_run 2025-04-01 03:44:46,065 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:49:47 2025-04-01 03:48:11,244 INFO 102126:repeater task 3dayplots_os75nb starting 0.4 seconds late 2025-04-01 03:48:15,633 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:48:15,633 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:53:10 2025-04-01 03:49:50,711 INFO 102126:repeater task lastensq_os75nb starting 3.7 seconds late 2025-04-01 03:49:51,408 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:49:51,408 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -24397.117 2025-04-01 03:49:51,408 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:54:52 2025-04-01 03:49:51,408 INFO 102126:repeater task quick_os75nb starting 4.4 seconds late 2025-04-01 03:49:53,915 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:49:53,915 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:49:53,915 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:49:53,915 INFO 102126:repeater task quick_os75nb: added 80 extra increments to time_to_run 2025-04-01 03:49:53,915 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:54:52 2025-04-01 03:49:53,915 INFO 102126:repeater task lastfew_vec_os75nb starting 6.9 seconds late 2025-04-01 03:49:54,531 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:49:54,531 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:49:54,531 INFO 102126:repeater task lastfew_vec_os75nb: added 80 extra increments to time_to_run 2025-04-01 03:49:54,531 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:54:52 2025-04-01 03:53:09,709 INFO 102126:repeater task plot_tsstats_os75nb starting 3.9 seconds late 2025-04-01 03:53:10,489 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 03:53:10,489 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 03:53:10,489 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 04:08:05 2025-04-01 03:53:10,489 INFO 102126:repeater task grab_lastpings_os75nb starting 4.7 seconds late 2025-04-01 03:53:10,579 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 03:53:10,579 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 04:08:05 2025-04-01 03:53:10,579 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.8 seconds late 2025-04-01 03:53:11,572 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:53:11,572 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:08:05 2025-04-01 03:53:11,572 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.8 seconds late 2025-04-01 03:53:12,565 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 03:53:12,565 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:08:05 2025-04-01 03:53:12,565 INFO 102126:repeater task daily_netcdf starting 1.7 seconds late 2025-04-01 03:53:12,764 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 03:53:12,764 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 03:53:12,764 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 04:03:10 2025-04-01 03:53:12,764 INFO 102126:repeater task ggatime_diagnostics starting 6.9 seconds late 2025-04-01 03:53:16,779 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 03:53:16,779 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 04:23:05 2025-04-01 03:53:16,779 INFO 102126:repeater task ggatime_diagnostics starting 11.0 seconds late 2025-04-01 03:53:20,623 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 03:53:20,623 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 04:23:05 2025-04-01 03:53:20,624 INFO 102126:repeater task ggatime_diagnostics starting 14.8 seconds late 2025-04-01 03:53:24,467 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 03:53:24,467 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 04:23:05 2025-04-01 03:53:24,467 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.6 seconds late 2025-04-01 03:53:25,575 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 03:53:25,575 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 03:53:25,575 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 04:08:05 2025-04-01 03:53:25,575 INFO 102126:repeater task plotposmv_posmv starting 19.7 seconds late 2025-04-01 03:53:27,717 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 03:53:27,717 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 04:08:05 2025-04-01 03:53:27,717 INFO 102126:repeater task plot_hbins starting 21.8 seconds late 2025-04-01 03:53:28,785 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 03:53:28,786 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 03:53:28,786 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 04:08:05 2025-04-01 03:53:33,787 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.0 seconds late 2025-04-01 03:53:35,719 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 03:53:35,719 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 04:03:10 2025-04-01 03:53:35,719 INFO 102126:repeater task 3dayplots_os75nb starting 24.9 seconds late 2025-04-01 03:53:40,101 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:53:40,101 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 03:58:10 2025-04-01 03:54:55,169 INFO 102126:repeater task lastensq_os75nb starting 3.2 seconds late 2025-04-01 03:54:55,865 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:54:55,865 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -24701.574 2025-04-01 03:54:55,865 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 03:59:57 2025-04-01 03:54:55,866 INFO 102126:repeater task quick_os75nb starting 3.9 seconds late 2025-04-01 03:54:58,413 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 03:54:58,413 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 03:54:58,413 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:54:58,413 INFO 102126:repeater task quick_os75nb: added 81 extra increments to time_to_run 2025-04-01 03:54:58,413 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 03:59:57 2025-04-01 03:54:58,413 INFO 102126:repeater task lastfew_vec_os75nb starting 6.4 seconds late 2025-04-01 03:54:59,027 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 03:54:59,027 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 03:54:59,027 INFO 102126:repeater task lastfew_vec_os75nb: added 81 extra increments to time_to_run 2025-04-01 03:54:59,027 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 03:59:57 2025-04-01 03:58:14,195 INFO 102126:repeater task 3dayplots_os75nb starting 3.4 seconds late 2025-04-01 03:58:18,593 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 03:58:18,593 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:03:10 2025-04-01 03:59:58,677 INFO 102126:repeater task lastensq_os75nb starting 1.7 seconds late 2025-04-01 03:59:59,402 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 03:59:59,402 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -25005.083 2025-04-01 03:59:59,402 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:05:02 2025-04-01 03:59:59,402 INFO 102126:repeater task quick_os75nb starting 2.4 seconds late 2025-04-01 04:00:01,916 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:00:01,916 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:00:01,916 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:00:01,916 INFO 102126:repeater task quick_os75nb: added 82 extra increments to time_to_run 2025-04-01 04:00:01,917 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:05:02 2025-04-01 04:00:01,917 INFO 102126:repeater task lastfew_vec_os75nb starting 4.9 seconds late 2025-04-01 04:00:02,536 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:00:02,536 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:00:02,536 INFO 102126:repeater task lastfew_vec_os75nb: added 82 extra increments to time_to_run 2025-04-01 04:00:02,536 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:05:02 2025-04-01 04:03:12,715 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.9 seconds late 2025-04-01 04:03:14,638 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 04:03:14,639 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 04:13:10 2025-04-01 04:03:14,639 INFO 102126:repeater task 3dayplots_os75nb starting 3.8 seconds late 2025-04-01 04:03:19,024 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:03:19,024 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:08:10 2025-04-01 04:03:19,025 INFO 102126:repeater task daily_netcdf starting 8.2 seconds late 2025-04-01 04:03:19,224 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 04:03:19,224 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 04:03:19,224 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 04:13:10 2025-04-01 04:05:04,324 INFO 102126:repeater task lastensq_os75nb starting 2.3 seconds late 2025-04-01 04:05:05,023 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:05:05,023 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -25310.730 2025-04-01 04:05:05,023 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:10:07 2025-04-01 04:05:05,023 INFO 102126:repeater task quick_os75nb starting 3.0 seconds late 2025-04-01 04:05:07,537 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:05:07,537 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:05:07,537 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:05:07,537 INFO 102126:repeater task quick_os75nb: added 83 extra increments to time_to_run 2025-04-01 04:05:07,537 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:10:07 2025-04-01 04:05:07,537 INFO 102126:repeater task lastfew_vec_os75nb starting 5.5 seconds late 2025-04-01 04:05:08,150 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:05:08,150 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:05:08,150 INFO 102126:repeater task lastfew_vec_os75nb: added 83 extra increments to time_to_run 2025-04-01 04:05:08,150 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:10:07 2025-04-01 04:08:08,303 INFO 102126:repeater task plot_tsstats_os75nb starting 2.5 seconds late 2025-04-01 04:08:09,070 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 04:08:09,070 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 04:08:09,070 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 04:23:05 2025-04-01 04:08:09,070 INFO 102126:repeater task grab_lastpings_os75nb starting 3.3 seconds late 2025-04-01 04:08:09,161 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 04:08:09,161 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 04:23:05 2025-04-01 04:08:09,161 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.3 seconds late 2025-04-01 04:08:10,156 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:08:10,156 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:23:05 2025-04-01 04:08:10,157 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.3 seconds late 2025-04-01 04:08:11,157 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:08:11,157 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:23:05 2025-04-01 04:08:11,158 INFO 102126:repeater task hbinhcorrstats_posmv starting 5.3 seconds late 2025-04-01 04:08:12,278 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 04:08:12,278 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 04:08:12,278 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 04:23:05 2025-04-01 04:08:12,278 INFO 102126:repeater task plotposmv_posmv starting 6.4 seconds late 2025-04-01 04:08:14,373 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 04:08:14,373 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 04:23:05 2025-04-01 04:08:14,373 INFO 102126:repeater task plot_hbins starting 8.5 seconds late 2025-04-01 04:08:15,426 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 04:08:15,426 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 04:08:15,426 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 04:23:05 2025-04-01 04:08:20,431 INFO 102126:repeater task 3dayplots_os75nb starting 9.6 seconds late 2025-04-01 04:08:24,833 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:08:24,833 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:13:10 2025-04-01 04:10:09,929 INFO 102126:repeater task lastensq_os75nb starting 2.9 seconds late 2025-04-01 04:10:10,595 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:10:10,595 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -25616.335 2025-04-01 04:10:10,595 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:15:12 2025-04-01 04:10:10,595 INFO 102126:repeater task quick_os75nb starting 3.6 seconds late 2025-04-01 04:10:13,107 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:10:13,107 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:10:13,107 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:10:13,107 INFO 102126:repeater task quick_os75nb: added 84 extra increments to time_to_run 2025-04-01 04:10:13,107 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:15:12 2025-04-01 04:10:13,107 INFO 102126:repeater task lastfew_vec_os75nb starting 6.1 seconds late 2025-04-01 04:10:13,720 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:10:13,720 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:10:13,720 INFO 102126:repeater task lastfew_vec_os75nb: added 84 extra increments to time_to_run 2025-04-01 04:10:13,720 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:15:12 2025-04-01 04:13:13,869 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.1 seconds late 2025-04-01 04:13:15,804 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 04:13:15,805 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 04:23:10 2025-04-01 04:13:15,805 INFO 102126:repeater task 3dayplots_os75nb starting 5.0 seconds late 2025-04-01 04:13:20,204 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:13:20,205 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:18:10 2025-04-01 04:13:20,205 INFO 102126:repeater task daily_netcdf starting 9.4 seconds late 2025-04-01 04:13:20,404 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 04:13:20,404 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 04:13:20,404 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 04:23:10 2025-04-01 04:15:15,515 INFO 102126:repeater task lastensq_os75nb starting 3.5 seconds late 2025-04-01 04:15:16,199 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:15:16,199 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -25921.921 2025-04-01 04:15:16,200 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:20:17 2025-04-01 04:15:16,200 INFO 102126:repeater task quick_os75nb starting 4.2 seconds late 2025-04-01 04:15:18,714 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:15:18,714 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:15:18,714 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:15:18,714 INFO 102126:repeater task quick_os75nb: added 85 extra increments to time_to_run 2025-04-01 04:15:18,714 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:20:17 2025-04-01 04:15:18,714 INFO 102126:repeater task lastfew_vec_os75nb starting 6.7 seconds late 2025-04-01 04:15:19,326 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:15:19,327 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:15:19,327 INFO 102126:repeater task lastfew_vec_os75nb: added 85 extra increments to time_to_run 2025-04-01 04:15:19,327 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:20:17 2025-04-01 04:18:14,471 INFO 102126:repeater task 3dayplots_os75nb starting 3.7 seconds late 2025-04-01 04:18:18,865 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:18:18,865 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:23:10 2025-04-01 04:20:18,963 INFO 102126:repeater task lastensq_os75nb starting 2.0 seconds late 2025-04-01 04:20:19,633 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:20:19,633 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -26225.369 2025-04-01 04:20:19,634 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:25:22 2025-04-01 04:20:19,634 INFO 102126:repeater task quick_os75nb starting 2.6 seconds late 2025-04-01 04:20:22,181 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:20:22,181 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:20:22,181 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:20:22,181 INFO 102126:repeater task quick_os75nb: added 86 extra increments to time_to_run 2025-04-01 04:20:22,181 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:25:22 2025-04-01 04:20:22,181 INFO 102126:repeater task lastfew_vec_os75nb starting 5.2 seconds late 2025-04-01 04:20:22,792 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:20:22,792 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:20:22,793 INFO 102126:repeater task lastfew_vec_os75nb: added 86 extra increments to time_to_run 2025-04-01 04:20:22,793 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:25:22 2025-04-01 04:23:07,939 INFO 102126:repeater task plot_tsstats_os75nb starting 2.1 seconds late 2025-04-01 04:23:08,710 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 04:23:08,710 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 04:23:08,710 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 04:38:05 2025-04-01 04:23:08,710 INFO 102126:repeater task grab_lastpings_os75nb starting 2.9 seconds late 2025-04-01 04:23:08,801 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 04:23:08,801 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 04:38:05 2025-04-01 04:23:08,801 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.0 seconds late 2025-04-01 04:23:09,795 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:23:09,795 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:38:05 2025-04-01 04:23:09,795 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.0 seconds late 2025-04-01 04:23:10,794 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:23:10,794 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:38:05 2025-04-01 04:23:10,795 INFO 102126:repeater task ggatime_diagnostics starting 5.0 seconds late 2025-04-01 04:23:14,848 INFO 102126:repeater task ggatime_diagnostics ended after 4.1 seconds 2025-04-01 04:23:14,849 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:23:14,849 INFO 102126:repeater task ggatime_diagnostics starting 9.0 seconds late 2025-04-01 04:23:18,643 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 04:23:18,643 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:23:18,643 INFO 102126:repeater task ggatime_diagnostics starting 12.8 seconds late 2025-04-01 04:23:22,481 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 04:23:22,481 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:23:22,481 INFO 102126:repeater task hbinhcorrstats_posmv starting 16.6 seconds late 2025-04-01 04:23:23,597 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 04:23:23,598 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 04:23:23,598 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 04:38:05 2025-04-01 04:23:23,598 INFO 102126:repeater task plotposmv_posmv starting 17.7 seconds late 2025-04-01 04:23:25,780 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 04:23:25,780 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 04:38:05 2025-04-01 04:23:25,780 INFO 102126:repeater task plot_hbins starting 19.9 seconds late 2025-04-01 04:23:26,836 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 04:23:26,836 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 04:23:26,836 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 04:38:05 2025-04-01 04:23:26,836 INFO 102126:repeater task report_generator starting 20.9 seconds late 2025-04-01 04:23:29,951 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-04-01 04:23:29,951 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-04-01 04:23:29,951 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 05:23:05 2025-04-01 04:23:34,957 INFO 102126:repeater task noisy_rawtemps_os75 starting 24.1 seconds late 2025-04-01 04:23:36,888 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 04:23:36,888 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 04:33:10 2025-04-01 04:23:36,889 INFO 102126:repeater task 3dayplots_os75nb starting 26.1 seconds late 2025-04-01 04:23:41,287 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:23:41,287 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:28:10 2025-04-01 04:23:41,288 INFO 102126:repeater task daily_netcdf starting 30.5 seconds late 2025-04-01 04:23:41,486 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 04:23:41,487 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 04:23:41,487 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 04:33:10 2025-04-01 04:25:26,565 INFO 102126:repeater task lastensq_os75nb starting 4.6 seconds late 2025-04-01 04:25:27,254 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:25:27,254 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -26532.970 2025-04-01 04:25:27,255 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:30:27 2025-04-01 04:25:27,255 INFO 102126:repeater task quick_os75nb starting 5.3 seconds late 2025-04-01 04:25:29,784 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:25:29,785 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:25:29,785 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:25:29,785 INFO 102126:repeater task quick_os75nb: added 87 extra increments to time_to_run 2025-04-01 04:25:29,785 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:30:27 2025-04-01 04:25:29,785 INFO 102126:repeater task lastfew_vec_os75nb starting 7.8 seconds late 2025-04-01 04:25:30,392 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:25:30,392 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:25:30,392 INFO 102126:repeater task lastfew_vec_os75nb: added 87 extra increments to time_to_run 2025-04-01 04:25:30,392 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:30:27 2025-04-01 04:28:15,539 INFO 102126:repeater task 3dayplots_os75nb starting 4.7 seconds late 2025-04-01 04:28:19,929 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:28:19,929 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:33:10 2025-04-01 04:30:30,049 INFO 102126:repeater task lastensq_os75nb starting 3.0 seconds late 2025-04-01 04:30:30,731 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:30:30,731 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -26836.454 2025-04-01 04:30:30,731 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:35:32 2025-04-01 04:30:30,731 INFO 102126:repeater task quick_os75nb starting 3.7 seconds late 2025-04-01 04:30:33,240 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:30:33,240 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:30:33,240 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:30:33,240 INFO 102126:repeater task quick_os75nb: added 88 extra increments to time_to_run 2025-04-01 04:30:33,240 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:35:32 2025-04-01 04:30:33,240 INFO 102126:repeater task lastfew_vec_os75nb starting 6.2 seconds late 2025-04-01 04:30:33,848 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:30:33,848 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:30:33,848 INFO 102126:repeater task lastfew_vec_os75nb: added 88 extra increments to time_to_run 2025-04-01 04:30:33,848 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:35:32 2025-04-01 04:33:13,991 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.2 seconds late 2025-04-01 04:33:15,936 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 04:33:15,936 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 04:43:10 2025-04-01 04:33:15,936 INFO 102126:repeater task 3dayplots_os75nb starting 5.1 seconds late 2025-04-01 04:33:20,334 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:33:20,335 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:38:10 2025-04-01 04:33:20,335 INFO 102126:repeater task daily_netcdf starting 9.5 seconds late 2025-04-01 04:33:20,534 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 04:33:20,534 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 04:33:20,534 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 04:43:10 2025-04-01 04:35:35,667 INFO 102126:repeater task lastensq_os75nb starting 3.7 seconds late 2025-04-01 04:35:36,351 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:35:36,351 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -27142.072 2025-04-01 04:35:36,352 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:40:37 2025-04-01 04:35:36,352 INFO 102126:repeater task quick_os75nb starting 4.4 seconds late 2025-04-01 04:35:38,858 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:35:38,858 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:35:38,859 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:35:38,859 INFO 102126:repeater task quick_os75nb: added 89 extra increments to time_to_run 2025-04-01 04:35:38,859 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:40:37 2025-04-01 04:35:38,859 INFO 102126:repeater task lastfew_vec_os75nb starting 6.9 seconds late 2025-04-01 04:35:39,481 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:35:39,481 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:35:39,481 INFO 102126:repeater task lastfew_vec_os75nb: added 89 extra increments to time_to_run 2025-04-01 04:35:39,481 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:40:37 2025-04-01 04:38:09,631 INFO 102126:repeater task plot_tsstats_os75nb starting 3.8 seconds late 2025-04-01 04:38:10,408 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 04:38:10,408 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 04:38:10,408 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:38:10,408 INFO 102126:repeater task grab_lastpings_os75nb starting 4.6 seconds late 2025-04-01 04:38:10,499 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 04:38:10,499 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:38:10,499 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.7 seconds late 2025-04-01 04:38:11,511 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:38:11,512 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:38:11,512 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.7 seconds late 2025-04-01 04:38:12,507 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:38:12,507 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:38:12,508 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.6 seconds late 2025-04-01 04:38:13,614 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 04:38:13,615 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 04:38:13,615 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:38:13,615 INFO 102126:repeater task plotposmv_posmv starting 7.7 seconds late 2025-04-01 04:38:15,735 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 04:38:15,735 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:38:15,735 INFO 102126:repeater task plot_hbins starting 9.8 seconds late 2025-04-01 04:38:16,798 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 04:38:16,799 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 04:38:16,799 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 04:53:05 2025-04-01 04:38:21,803 INFO 102126:repeater task 3dayplots_os75nb starting 11.0 seconds late 2025-04-01 04:38:26,184 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:38:26,184 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:43:10 2025-04-01 04:40:41,303 INFO 102126:repeater task lastensq_os75nb starting 4.3 seconds late 2025-04-01 04:40:41,972 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:40:41,972 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -27447.709 2025-04-01 04:40:41,972 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:45:42 2025-04-01 04:40:41,972 INFO 102126:repeater task quick_os75nb starting 5.0 seconds late 2025-04-01 04:40:44,484 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:40:44,484 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:40:44,484 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:40:44,484 INFO 102126:repeater task quick_os75nb: added 90 extra increments to time_to_run 2025-04-01 04:40:44,484 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:45:42 2025-04-01 04:40:44,484 INFO 102126:repeater task lastfew_vec_os75nb starting 7.5 seconds late 2025-04-01 04:40:45,095 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:40:45,095 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:40:45,095 INFO 102126:repeater task lastfew_vec_os75nb: added 90 extra increments to time_to_run 2025-04-01 04:40:45,095 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:45:42 2025-04-01 04:43:15,233 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.4 seconds late 2025-04-01 04:43:17,168 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 04:43:17,168 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 04:53:10 2025-04-01 04:43:17,168 INFO 102126:repeater task 3dayplots_os75nb starting 6.4 seconds late 2025-04-01 04:43:21,565 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:43:21,565 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:48:10 2025-04-01 04:43:21,566 INFO 102126:repeater task daily_netcdf starting 10.7 seconds late 2025-04-01 04:43:21,766 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 04:43:21,766 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 04:43:21,766 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 04:53:10 2025-04-01 04:45:46,895 INFO 102126:repeater task lastensq_os75nb starting 4.9 seconds late 2025-04-01 04:45:47,579 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:45:47,579 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -27753.301 2025-04-01 04:45:47,579 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:50:47 2025-04-01 04:45:47,579 INFO 102126:repeater task quick_os75nb starting 5.6 seconds late 2025-04-01 04:45:50,135 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:45:50,135 INFO 102126:repeater task quick_os75nb ended after 2.6 seconds 2025-04-01 04:45:50,135 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:45:50,135 INFO 102126:repeater task quick_os75nb: added 91 extra increments to time_to_run 2025-04-01 04:45:50,135 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:50:47 2025-04-01 04:45:50,135 INFO 102126:repeater task lastfew_vec_os75nb starting 8.1 seconds late 2025-04-01 04:45:50,746 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:45:50,746 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:45:50,746 INFO 102126:repeater task lastfew_vec_os75nb: added 91 extra increments to time_to_run 2025-04-01 04:45:50,746 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:50:47 2025-04-01 04:48:10,872 INFO 102126:repeater task 3dayplots_os75nb starting 0.1 seconds late 2025-04-01 04:48:15,274 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:48:15,275 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:53:10 2025-04-01 04:50:50,413 INFO 102126:repeater task lastensq_os75nb starting 3.4 seconds late 2025-04-01 04:50:51,082 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:50:51,082 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -28056.819 2025-04-01 04:50:51,082 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 04:55:52 2025-04-01 04:50:51,082 INFO 102126:repeater task quick_os75nb starting 4.1 seconds late 2025-04-01 04:50:53,638 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:50:53,638 INFO 102126:repeater task quick_os75nb ended after 2.6 seconds 2025-04-01 04:50:53,639 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:50:53,639 INFO 102126:repeater task quick_os75nb: added 92 extra increments to time_to_run 2025-04-01 04:50:53,639 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 04:55:52 2025-04-01 04:50:53,639 INFO 102126:repeater task lastfew_vec_os75nb starting 6.6 seconds late 2025-04-01 04:50:54,250 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:50:54,250 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:50:54,250 INFO 102126:repeater task lastfew_vec_os75nb: added 92 extra increments to time_to_run 2025-04-01 04:50:54,250 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 04:55:52 2025-04-01 04:53:09,367 INFO 102126:repeater task plot_tsstats_os75nb starting 3.6 seconds late 2025-04-01 04:53:10,147 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 04:53:10,147 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 04:53:10,147 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 05:08:05 2025-04-01 04:53:10,147 INFO 102126:repeater task grab_lastpings_os75nb starting 4.3 seconds late 2025-04-01 04:53:10,237 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 04:53:10,237 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 05:08:05 2025-04-01 04:53:10,238 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.4 seconds late 2025-04-01 04:53:11,230 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:53:11,230 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:08:05 2025-04-01 04:53:11,230 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.4 seconds late 2025-04-01 04:53:12,224 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 04:53:12,224 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:08:05 2025-04-01 04:53:12,225 INFO 102126:repeater task daily_netcdf starting 1.4 seconds late 2025-04-01 04:53:12,424 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 04:53:12,424 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 04:53:12,424 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 05:03:10 2025-04-01 04:53:12,425 INFO 102126:repeater task ggatime_diagnostics starting 6.6 seconds late 2025-04-01 04:53:16,454 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 04:53:16,454 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 05:23:05 2025-04-01 04:53:16,454 INFO 102126:repeater task ggatime_diagnostics starting 10.6 seconds late 2025-04-01 04:53:20,275 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 04:53:20,276 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 05:23:05 2025-04-01 04:53:20,276 INFO 102126:repeater task ggatime_diagnostics starting 14.5 seconds late 2025-04-01 04:53:24,139 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-04-01 04:53:24,139 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 05:23:05 2025-04-01 04:53:24,139 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.2 seconds late 2025-04-01 04:53:25,251 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 04:53:25,251 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 04:53:25,251 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 05:08:05 2025-04-01 04:53:25,251 INFO 102126:repeater task plotposmv_posmv starting 19.3 seconds late 2025-04-01 04:53:27,398 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 04:53:27,399 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 05:08:05 2025-04-01 04:53:27,399 INFO 102126:repeater task plot_hbins starting 21.5 seconds late 2025-04-01 04:53:28,468 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 04:53:28,468 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 04:53:28,468 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 05:08:05 2025-04-01 04:53:33,474 INFO 102126:repeater task noisy_rawtemps_os75 starting 22.7 seconds late 2025-04-01 04:53:35,401 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 04:53:35,401 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 05:03:10 2025-04-01 04:53:35,402 INFO 102126:repeater task 3dayplots_os75nb starting 24.6 seconds late 2025-04-01 04:53:39,788 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:53:39,789 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 04:58:10 2025-04-01 04:55:54,909 INFO 102126:repeater task lastensq_os75nb starting 2.9 seconds late 2025-04-01 04:55:55,580 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 04:55:55,580 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -28361.314 2025-04-01 04:55:55,580 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:00:57 2025-04-01 04:55:55,580 INFO 102126:repeater task quick_os75nb starting 3.6 seconds late 2025-04-01 04:55:58,096 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 04:55:58,096 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 04:55:58,097 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:55:58,097 INFO 102126:repeater task quick_os75nb: added 93 extra increments to time_to_run 2025-04-01 04:55:58,097 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:00:57 2025-04-01 04:55:58,097 INFO 102126:repeater task lastfew_vec_os75nb starting 6.1 seconds late 2025-04-01 04:55:58,711 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 04:55:58,711 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 04:55:58,712 INFO 102126:repeater task lastfew_vec_os75nb: added 93 extra increments to time_to_run 2025-04-01 04:55:58,712 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:00:57 2025-04-01 04:58:13,831 INFO 102126:repeater task 3dayplots_os75nb starting 3.0 seconds late 2025-04-01 04:58:18,226 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 04:58:18,226 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:03:10 2025-04-01 05:00:58,372 INFO 102126:repeater task lastensq_os75nb starting 1.4 seconds late 2025-04-01 05:00:59,180 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-04-01 05:00:59,180 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -28664.777 2025-04-01 05:00:59,180 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:06:02 2025-04-01 05:00:59,180 INFO 102126:repeater task quick_os75nb starting 2.2 seconds late 2025-04-01 05:01:01,703 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:01:01,703 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:01:01,703 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:01:01,703 INFO 102126:repeater task quick_os75nb: added 94 extra increments to time_to_run 2025-04-01 05:01:01,703 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:06:02 2025-04-01 05:01:01,703 INFO 102126:repeater task lastfew_vec_os75nb starting 4.7 seconds late 2025-04-01 05:01:02,324 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:01:02,324 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:01:02,324 INFO 102126:repeater task lastfew_vec_os75nb: added 94 extra increments to time_to_run 2025-04-01 05:01:02,324 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:06:02 2025-04-01 05:03:12,443 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.6 seconds late 2025-04-01 05:03:14,396 INFO 102126:repeater task noisy_rawtemps_os75 ended after 2.0 seconds 2025-04-01 05:03:14,397 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 05:13:10 2025-04-01 05:03:14,397 INFO 102126:repeater task 3dayplots_os75nb starting 3.6 seconds late 2025-04-01 05:03:18,801 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:03:18,801 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:08:10 2025-04-01 05:03:18,801 INFO 102126:repeater task daily_netcdf starting 8.0 seconds late 2025-04-01 05:03:19,020 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 05:03:19,020 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 05:03:19,020 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 05:13:10 2025-04-01 05:06:04,163 INFO 102126:repeater task lastensq_os75nb starting 2.2 seconds late 2025-04-01 05:06:04,964 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-04-01 05:06:04,964 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -28970.569 2025-04-01 05:06:04,965 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:11:07 2025-04-01 05:06:04,965 INFO 102126:repeater task quick_os75nb starting 3.0 seconds late 2025-04-01 05:06:07,480 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:06:07,480 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:06:07,480 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:06:07,480 INFO 102126:repeater task quick_os75nb: added 95 extra increments to time_to_run 2025-04-01 05:06:07,480 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:11:07 2025-04-01 05:06:07,480 INFO 102126:repeater task lastfew_vec_os75nb starting 5.5 seconds late 2025-04-01 05:06:08,092 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:06:08,092 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:06:08,092 INFO 102126:repeater task lastfew_vec_os75nb: added 95 extra increments to time_to_run 2025-04-01 05:06:08,092 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:11:07 2025-04-01 05:08:08,201 INFO 102126:repeater task plot_tsstats_os75nb starting 2.4 seconds late 2025-04-01 05:08:08,974 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 05:08:08,974 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 05:08:08,974 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 05:23:05 2025-04-01 05:08:08,974 INFO 102126:repeater task grab_lastpings_os75nb starting 3.2 seconds late 2025-04-01 05:08:09,065 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 05:08:09,065 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 05:23:05 2025-04-01 05:08:09,065 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.2 seconds late 2025-04-01 05:08:10,061 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:08:10,061 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:23:05 2025-04-01 05:08:10,061 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.2 seconds late 2025-04-01 05:08:11,055 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:08:11,055 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:23:05 2025-04-01 05:08:11,055 INFO 102126:repeater task hbinhcorrstats_posmv starting 5.2 seconds late 2025-04-01 05:08:12,163 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 05:08:12,163 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 05:08:12,163 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 05:23:05 2025-04-01 05:08:12,164 INFO 102126:repeater task plotposmv_posmv starting 6.3 seconds late 2025-04-01 05:08:14,315 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 05:08:14,315 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 05:23:05 2025-04-01 05:08:14,316 INFO 102126:repeater task plot_hbins starting 8.4 seconds late 2025-04-01 05:08:15,421 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 05:08:15,421 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 05:08:15,421 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 05:23:05 2025-04-01 05:08:20,426 INFO 102126:repeater task 3dayplots_os75nb starting 9.6 seconds late 2025-04-01 05:08:24,825 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:08:24,826 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:13:10 2025-04-01 05:11:09,984 INFO 102126:repeater task lastensq_os75nb starting 3.0 seconds late 2025-04-01 05:11:10,654 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:11:10,654 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -29276.389 2025-04-01 05:11:10,654 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:16:12 2025-04-01 05:11:10,655 INFO 102126:repeater task quick_os75nb starting 3.7 seconds late 2025-04-01 05:11:13,161 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:11:13,161 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:11:13,161 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:11:13,161 INFO 102126:repeater task quick_os75nb: added 96 extra increments to time_to_run 2025-04-01 05:11:13,161 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:16:12 2025-04-01 05:11:13,161 INFO 102126:repeater task lastfew_vec_os75nb starting 6.2 seconds late 2025-04-01 05:11:13,782 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:11:13,783 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:11:13,783 INFO 102126:repeater task lastfew_vec_os75nb: added 96 extra increments to time_to_run 2025-04-01 05:11:13,783 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:16:12 2025-04-01 05:13:13,891 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.1 seconds late 2025-04-01 05:13:15,825 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 05:13:15,825 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 05:23:10 2025-04-01 05:13:15,825 INFO 102126:repeater task 3dayplots_os75nb starting 5.0 seconds late 2025-04-01 05:13:20,210 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:13:20,210 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:18:10 2025-04-01 05:13:20,210 INFO 102126:repeater task daily_netcdf starting 9.4 seconds late 2025-04-01 05:13:20,412 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 05:13:20,412 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 05:13:20,412 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 05:23:10 2025-04-01 05:16:15,567 INFO 102126:repeater task lastensq_os75nb starting 3.6 seconds late 2025-04-01 05:16:16,252 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:16:16,252 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -29581.973 2025-04-01 05:16:16,252 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:21:17 2025-04-01 05:16:16,252 INFO 102126:repeater task quick_os75nb starting 4.3 seconds late 2025-04-01 05:16:18,770 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:16:18,770 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:16:18,770 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:16:18,770 INFO 102126:repeater task quick_os75nb: added 97 extra increments to time_to_run 2025-04-01 05:16:18,770 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:21:17 2025-04-01 05:16:18,770 INFO 102126:repeater task lastfew_vec_os75nb starting 6.8 seconds late 2025-04-01 05:16:19,390 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:16:19,391 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:16:19,391 INFO 102126:repeater task lastfew_vec_os75nb: added 97 extra increments to time_to_run 2025-04-01 05:16:19,391 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:21:17 2025-04-01 05:18:14,493 INFO 102126:repeater task 3dayplots_os75nb starting 3.7 seconds late 2025-04-01 05:18:18,883 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:18:18,883 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:23:10 2025-04-01 05:21:19,051 INFO 102126:repeater task lastensq_os75nb starting 2.1 seconds late 2025-04-01 05:21:19,754 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:21:19,754 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -29885.457 2025-04-01 05:21:19,754 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:26:22 2025-04-01 05:21:19,754 INFO 102126:repeater task quick_os75nb starting 2.8 seconds late 2025-04-01 05:21:22,273 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:21:22,273 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:21:22,273 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:21:22,273 INFO 102126:repeater task quick_os75nb: added 98 extra increments to time_to_run 2025-04-01 05:21:22,273 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:26:22 2025-04-01 05:21:22,273 INFO 102126:repeater task lastfew_vec_os75nb starting 5.3 seconds late 2025-04-01 05:21:22,883 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:21:22,883 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:21:22,883 INFO 102126:repeater task lastfew_vec_os75nb: added 98 extra increments to time_to_run 2025-04-01 05:21:22,883 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:26:22 2025-04-01 05:23:07,980 INFO 102126:repeater task plot_tsstats_os75nb starting 2.2 seconds late 2025-04-01 05:23:08,747 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 05:23:08,747 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 05:23:08,747 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 05:38:05 2025-04-01 05:23:08,747 INFO 102126:repeater task grab_lastpings_os75nb starting 2.9 seconds late 2025-04-01 05:23:08,838 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 05:23:08,838 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 05:38:05 2025-04-01 05:23:08,838 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.0 seconds late 2025-04-01 05:23:09,832 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:23:09,832 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:38:05 2025-04-01 05:23:09,833 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.0 seconds late 2025-04-01 05:23:10,827 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:23:10,827 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:38:05 2025-04-01 05:23:10,827 INFO 102126:repeater task daily_netcdf starting 0.0 seconds late 2025-04-01 05:23:11,027 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 05:23:11,027 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 05:23:11,027 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 05:33:10 2025-04-01 05:23:11,027 INFO 102126:repeater task ggatime_diagnostics starting 5.2 seconds late 2025-04-01 05:23:15,026 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 05:23:15,026 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:23:15,026 INFO 102126:repeater task ggatime_diagnostics starting 9.2 seconds late 2025-04-01 05:23:18,870 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 05:23:18,870 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:23:18,871 INFO 102126:repeater task ggatime_diagnostics starting 13.1 seconds late 2025-04-01 05:23:22,733 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-04-01 05:23:22,733 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:23:22,733 INFO 102126:repeater task hbinhcorrstats_posmv starting 16.8 seconds late 2025-04-01 05:23:23,844 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 05:23:23,844 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 05:23:23,844 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 05:38:05 2025-04-01 05:23:23,845 INFO 102126:repeater task plotposmv_posmv starting 17.9 seconds late 2025-04-01 05:23:25,995 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 05:23:25,995 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 05:38:05 2025-04-01 05:23:25,995 INFO 102126:repeater task plot_hbins starting 20.1 seconds late 2025-04-01 05:23:27,067 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 05:23:27,067 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 05:23:27,067 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 05:38:05 2025-04-01 05:23:27,068 INFO 102126:repeater task report_generator starting 21.2 seconds late 2025-04-01 05:23:30,188 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-04-01 05:23:30,188 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-04-01 05:23:30,188 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 06:23:05 2025-04-01 05:23:35,192 INFO 102126:repeater task noisy_rawtemps_os75 starting 24.4 seconds late 2025-04-01 05:23:37,128 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 05:23:37,128 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 05:33:10 2025-04-01 05:23:37,128 INFO 102126:repeater task 3dayplots_os75nb starting 26.3 seconds late 2025-04-01 05:23:41,525 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:23:41,525 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:28:10 2025-04-01 05:26:26,673 INFO 102126:repeater task lastensq_os75nb starting 4.7 seconds late 2025-04-01 05:26:27,346 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:26:27,346 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -30193.079 2025-04-01 05:26:27,346 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:31:27 2025-04-01 05:26:27,347 INFO 102126:repeater task quick_os75nb starting 5.3 seconds late 2025-04-01 05:26:29,863 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:26:29,863 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:26:29,863 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:26:29,864 INFO 102126:repeater task quick_os75nb: added 99 extra increments to time_to_run 2025-04-01 05:26:29,864 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:31:27 2025-04-01 05:26:29,864 INFO 102126:repeater task lastfew_vec_os75nb starting 7.9 seconds late 2025-04-01 05:26:30,484 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:26:30,484 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:26:30,484 INFO 102126:repeater task lastfew_vec_os75nb: added 99 extra increments to time_to_run 2025-04-01 05:26:30,484 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:31:27 2025-04-01 05:28:15,585 INFO 102126:repeater task 3dayplots_os75nb starting 4.8 seconds late 2025-04-01 05:28:19,996 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:28:19,996 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:33:10 2025-04-01 05:31:30,165 INFO 102126:repeater task lastensq_os75nb starting 3.2 seconds late 2025-04-01 05:31:30,997 INFO 102126:repeater task lastensq_os75nb ended after 0.8 seconds 2025-04-01 05:31:30,997 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -30496.571 2025-04-01 05:31:30,997 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:36:32 2025-04-01 05:31:30,997 INFO 102126:repeater task quick_os75nb starting 4.0 seconds late 2025-04-01 05:31:33,511 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:31:33,511 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:31:33,511 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:31:33,511 INFO 102126:repeater task quick_os75nb: added 100 extra increments to time_to_run 2025-04-01 05:31:33,511 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:36:32 2025-04-01 05:31:33,511 INFO 102126:repeater task lastfew_vec_os75nb starting 6.5 seconds late 2025-04-01 05:31:34,120 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:31:34,120 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:31:34,120 INFO 102126:repeater task lastfew_vec_os75nb: added 100 extra increments to time_to_run 2025-04-01 05:31:34,120 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:36:32 2025-04-01 05:33:14,207 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.4 seconds late 2025-04-01 05:33:16,146 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 05:33:16,146 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 05:43:10 2025-04-01 05:33:16,147 INFO 102126:repeater task 3dayplots_os75nb starting 5.3 seconds late 2025-04-01 05:33:20,539 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:33:20,539 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:38:10 2025-04-01 05:33:20,539 INFO 102126:repeater task daily_netcdf starting 9.7 seconds late 2025-04-01 05:33:20,739 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 05:33:20,739 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 05:33:20,739 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 05:43:10 2025-04-01 05:36:35,903 INFO 102126:repeater task lastensq_os75nb starting 3.9 seconds late 2025-04-01 05:36:36,594 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:36:36,594 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -30802.309 2025-04-01 05:36:36,594 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:41:37 2025-04-01 05:36:36,594 INFO 102126:repeater task quick_os75nb starting 4.6 seconds late 2025-04-01 05:36:39,101 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:36:39,101 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:36:39,101 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:36:39,101 INFO 102126:repeater task quick_os75nb: added 101 extra increments to time_to_run 2025-04-01 05:36:39,101 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:41:37 2025-04-01 05:36:39,101 INFO 102126:repeater task lastfew_vec_os75nb starting 7.1 seconds late 2025-04-01 05:36:39,723 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:36:39,723 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:36:39,723 INFO 102126:repeater task lastfew_vec_os75nb: added 101 extra increments to time_to_run 2025-04-01 05:36:39,723 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:41:37 2025-04-01 05:38:09,803 INFO 102126:repeater task plot_tsstats_os75nb starting 4.0 seconds late 2025-04-01 05:38:10,578 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 05:38:10,578 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 05:38:10,578 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:38:10,578 INFO 102126:repeater task grab_lastpings_os75nb starting 4.8 seconds late 2025-04-01 05:38:10,669 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 05:38:10,669 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:38:10,669 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.9 seconds late 2025-04-01 05:38:11,662 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:38:11,662 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:38:11,663 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.8 seconds late 2025-04-01 05:38:12,658 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:38:12,658 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:38:12,658 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.8 seconds late 2025-04-01 05:38:13,758 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 05:38:13,758 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 05:38:13,758 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:38:13,758 INFO 102126:repeater task plotposmv_posmv starting 7.9 seconds late 2025-04-01 05:38:15,904 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 05:38:15,904 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:38:15,904 INFO 102126:repeater task plot_hbins starting 10.0 seconds late 2025-04-01 05:38:16,972 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 05:38:16,972 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 05:38:16,973 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 05:53:05 2025-04-01 05:38:21,975 INFO 102126:repeater task 3dayplots_os75nb starting 11.2 seconds late 2025-04-01 05:38:26,373 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:38:26,373 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:43:10 2025-04-01 05:41:41,539 INFO 102126:repeater task lastensq_os75nb starting 4.5 seconds late 2025-04-01 05:41:42,216 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:41:42,216 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -31107.945 2025-04-01 05:41:42,216 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:46:42 2025-04-01 05:41:42,216 INFO 102126:repeater task quick_os75nb starting 5.2 seconds late 2025-04-01 05:41:44,725 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:41:44,725 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:41:44,725 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:41:44,725 INFO 102126:repeater task quick_os75nb: added 102 extra increments to time_to_run 2025-04-01 05:41:44,725 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:46:42 2025-04-01 05:41:44,725 INFO 102126:repeater task lastfew_vec_os75nb starting 7.7 seconds late 2025-04-01 05:41:45,336 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:41:45,337 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:41:45,337 INFO 102126:repeater task lastfew_vec_os75nb: added 102 extra increments to time_to_run 2025-04-01 05:41:45,337 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:46:42 2025-04-01 05:43:15,417 INFO 102126:repeater task noisy_rawtemps_os75 starting 4.6 seconds late 2025-04-01 05:43:17,364 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 05:43:17,364 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 05:53:10 2025-04-01 05:43:17,364 INFO 102126:repeater task 3dayplots_os75nb starting 6.5 seconds late 2025-04-01 05:43:21,772 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:43:21,772 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:48:10 2025-04-01 05:43:21,772 INFO 102126:repeater task daily_netcdf starting 11.0 seconds late 2025-04-01 05:43:21,972 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 05:43:21,972 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 05:43:21,972 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 05:53:10 2025-04-01 05:46:42,149 INFO 102126:repeater task lastensq_os75nb starting 0.1 seconds late 2025-04-01 05:46:42,841 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:46:42,841 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -31408.555 2025-04-01 05:46:42,841 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:51:47 2025-04-01 05:46:42,841 INFO 102126:repeater task quick_os75nb starting 0.8 seconds late 2025-04-01 05:46:45,352 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:46:45,353 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:46:45,353 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:46:45,353 INFO 102126:repeater task quick_os75nb: added 103 extra increments to time_to_run 2025-04-01 05:46:45,353 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:51:47 2025-04-01 05:46:45,353 INFO 102126:repeater task lastfew_vec_os75nb starting 3.4 seconds late 2025-04-01 05:46:45,967 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:46:45,967 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:46:45,967 INFO 102126:repeater task lastfew_vec_os75nb: added 103 extra increments to time_to_run 2025-04-01 05:46:45,967 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:51:47 2025-04-01 05:48:11,039 INFO 102126:repeater task 3dayplots_os75nb starting 0.2 seconds late 2025-04-01 05:48:15,430 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:48:15,430 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:53:10 2025-04-01 05:51:50,626 INFO 102126:repeater task lastensq_os75nb starting 3.6 seconds late 2025-04-01 05:51:51,318 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:51:51,318 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -31717.031 2025-04-01 05:51:51,318 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 05:56:52 2025-04-01 05:51:51,318 INFO 102126:repeater task quick_os75nb starting 4.3 seconds late 2025-04-01 05:51:54,018 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:51:54,018 INFO 102126:repeater task quick_os75nb ended after 2.7 seconds 2025-04-01 05:51:54,019 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:51:54,019 INFO 102126:repeater task quick_os75nb: added 104 extra increments to time_to_run 2025-04-01 05:51:54,019 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 05:56:52 2025-04-01 05:51:54,019 INFO 102126:repeater task lastfew_vec_os75nb starting 7.0 seconds late 2025-04-01 05:51:54,636 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:51:54,636 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:51:54,636 INFO 102126:repeater task lastfew_vec_os75nb: added 104 extra increments to time_to_run 2025-04-01 05:51:54,636 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 05:56:52 2025-04-01 05:53:09,715 INFO 102126:repeater task plot_tsstats_os75nb starting 3.9 seconds late 2025-04-01 05:53:10,494 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 05:53:10,494 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 05:53:10,494 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 06:08:05 2025-04-01 05:53:10,494 INFO 102126:repeater task grab_lastpings_os75nb starting 4.7 seconds late 2025-04-01 05:53:10,585 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 05:53:10,585 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 06:08:05 2025-04-01 05:53:10,585 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.8 seconds late 2025-04-01 05:53:11,580 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:53:11,580 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:08:05 2025-04-01 05:53:11,580 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.8 seconds late 2025-04-01 05:53:12,577 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 05:53:12,577 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:08:05 2025-04-01 05:53:12,577 INFO 102126:repeater task daily_netcdf starting 1.8 seconds late 2025-04-01 05:53:12,776 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 05:53:12,776 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 05:53:12,777 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 06:03:10 2025-04-01 05:53:12,777 INFO 102126:repeater task ggatime_diagnostics starting 7.0 seconds late 2025-04-01 05:53:16,800 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 05:53:16,800 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 06:23:05 2025-04-01 05:53:16,800 INFO 102126:repeater task ggatime_diagnostics starting 11.0 seconds late 2025-04-01 05:53:20,643 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 05:53:20,643 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 06:23:05 2025-04-01 05:53:20,643 INFO 102126:repeater task ggatime_diagnostics starting 14.8 seconds late 2025-04-01 05:53:24,510 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-04-01 05:53:24,510 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 06:23:05 2025-04-01 05:53:24,510 INFO 102126:repeater task hbinhcorrstats_posmv starting 18.6 seconds late 2025-04-01 05:53:25,615 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 05:53:25,615 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 05:53:25,615 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 06:08:05 2025-04-01 05:53:25,615 INFO 102126:repeater task plotposmv_posmv starting 19.7 seconds late 2025-04-01 05:53:27,800 INFO 102126:repeater task plotposmv_posmv ended after 2.2 seconds 2025-04-01 05:53:27,801 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 06:08:05 2025-04-01 05:53:27,801 INFO 102126:repeater task plot_hbins starting 21.9 seconds late 2025-04-01 05:53:28,876 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 05:53:28,876 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 05:53:28,876 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 06:08:05 2025-04-01 05:53:33,882 INFO 102126:repeater task noisy_rawtemps_os75 starting 23.1 seconds late 2025-04-01 05:53:35,812 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 05:53:35,813 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 06:03:10 2025-04-01 05:53:35,813 INFO 102126:repeater task 3dayplots_os75nb starting 25.0 seconds late 2025-04-01 05:53:40,195 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:53:40,195 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 05:58:10 2025-04-01 05:56:55,354 INFO 102126:repeater task lastensq_os75nb starting 3.4 seconds late 2025-04-01 05:56:56,045 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 05:56:56,045 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -32021.760 2025-04-01 05:56:56,045 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:01:57 2025-04-01 05:56:56,045 INFO 102126:repeater task quick_os75nb starting 4.0 seconds late 2025-04-01 05:56:58,552 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 05:56:58,552 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 05:56:58,553 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:56:58,553 INFO 102126:repeater task quick_os75nb: added 105 extra increments to time_to_run 2025-04-01 05:56:58,553 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:01:57 2025-04-01 05:56:58,553 INFO 102126:repeater task lastfew_vec_os75nb starting 6.6 seconds late 2025-04-01 05:56:59,172 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 05:56:59,172 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 05:56:59,172 INFO 102126:repeater task lastfew_vec_os75nb: added 105 extra increments to time_to_run 2025-04-01 05:56:59,172 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:01:57 2025-04-01 05:58:14,237 INFO 102126:repeater task 3dayplots_os75nb starting 3.4 seconds late 2025-04-01 05:58:18,620 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 05:58:18,620 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:03:10 2025-04-01 06:01:58,831 INFO 102126:repeater task lastensq_os75nb starting 1.8 seconds late 2025-04-01 06:01:59,511 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:01:59,511 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -32325.237 2025-04-01 06:01:59,511 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:07:02 2025-04-01 06:01:59,511 INFO 102126:repeater task quick_os75nb starting 2.5 seconds late 2025-04-01 06:02:02,019 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:02:02,019 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:02:02,019 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:02:02,019 INFO 102126:repeater task quick_os75nb: added 106 extra increments to time_to_run 2025-04-01 06:02:02,019 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:07:02 2025-04-01 06:02:02,020 INFO 102126:repeater task lastfew_vec_os75nb starting 5.0 seconds late 2025-04-01 06:02:02,642 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:02:02,642 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:02:02,643 INFO 102126:repeater task lastfew_vec_os75nb: added 106 extra increments to time_to_run 2025-04-01 06:02:02,643 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:07:02 2025-04-01 06:03:12,707 INFO 102126:repeater task noisy_rawtemps_os75 starting 1.9 seconds late 2025-04-01 06:03:14,643 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 06:03:14,643 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 06:13:10 2025-04-01 06:03:14,643 INFO 102126:repeater task 3dayplots_os75nb starting 3.8 seconds late 2025-04-01 06:03:19,044 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:03:19,044 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:08:10 2025-04-01 06:03:19,044 INFO 102126:repeater task daily_netcdf starting 8.2 seconds late 2025-04-01 06:03:19,246 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 06:03:19,246 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 06:03:19,246 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 06:13:10 2025-04-01 06:07:04,443 INFO 102126:repeater task lastensq_os75nb starting 2.4 seconds late 2025-04-01 06:07:05,137 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:07:05,137 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -32630.849 2025-04-01 06:07:05,137 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:12:07 2025-04-01 06:07:05,137 INFO 102126:repeater task quick_os75nb starting 3.1 seconds late 2025-04-01 06:07:07,647 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:07:07,648 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:07:07,648 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:07:07,648 INFO 102126:repeater task quick_os75nb: added 107 extra increments to time_to_run 2025-04-01 06:07:07,648 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:12:07 2025-04-01 06:07:07,648 INFO 102126:repeater task lastfew_vec_os75nb starting 5.6 seconds late 2025-04-01 06:07:08,264 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:07:08,265 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:07:08,265 INFO 102126:repeater task lastfew_vec_os75nb: added 107 extra increments to time_to_run 2025-04-01 06:07:08,265 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:12:07 2025-04-01 06:08:08,319 INFO 102126:repeater task plot_tsstats_os75nb starting 2.5 seconds late 2025-04-01 06:08:09,093 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 06:08:09,093 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 06:08:09,093 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 06:23:05 2025-04-01 06:08:09,093 INFO 102126:repeater task grab_lastpings_os75nb starting 3.3 seconds late 2025-04-01 06:08:09,183 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 06:08:09,184 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 06:23:05 2025-04-01 06:08:09,184 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.4 seconds late 2025-04-01 06:08:10,178 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 06:08:10,179 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:23:05 2025-04-01 06:08:10,179 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.4 seconds late 2025-04-01 06:08:11,175 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 06:08:11,176 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:23:05 2025-04-01 06:08:11,176 INFO 102126:repeater task hbinhcorrstats_posmv starting 5.3 seconds late 2025-04-01 06:08:12,301 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 06:08:12,301 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 06:08:12,301 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 06:23:05 2025-04-01 06:08:12,301 INFO 102126:repeater task plotposmv_posmv starting 6.4 seconds late 2025-04-01 06:08:14,391 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 06:08:14,391 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 06:23:05 2025-04-01 06:08:14,391 INFO 102126:repeater task plot_hbins starting 8.5 seconds late 2025-04-01 06:08:15,447 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 06:08:15,447 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 06:08:15,447 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 06:23:05 2025-04-01 06:08:20,451 INFO 102126:repeater task 3dayplots_os75nb starting 9.6 seconds late 2025-04-01 06:08:24,850 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:08:24,850 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:13:10 2025-04-01 06:12:10,055 INFO 102126:repeater task lastensq_os75nb starting 3.1 seconds late 2025-04-01 06:12:10,746 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:12:10,746 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -32936.461 2025-04-01 06:12:10,746 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:17:12 2025-04-01 06:12:10,747 INFO 102126:repeater task quick_os75nb starting 3.7 seconds late 2025-04-01 06:12:13,259 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:12:13,259 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:12:13,259 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:12:13,259 INFO 102126:repeater task quick_os75nb: added 108 extra increments to time_to_run 2025-04-01 06:12:13,259 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:17:12 2025-04-01 06:12:13,259 INFO 102126:repeater task lastfew_vec_os75nb starting 6.3 seconds late 2025-04-01 06:12:13,870 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:12:13,870 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:12:13,870 INFO 102126:repeater task lastfew_vec_os75nb: added 108 extra increments to time_to_run 2025-04-01 06:12:13,870 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:17:12 2025-04-01 06:13:13,927 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.1 seconds late 2025-04-01 06:13:15,859 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 06:13:15,859 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 06:23:10 2025-04-01 06:13:15,859 INFO 102126:repeater task 3dayplots_os75nb starting 5.0 seconds late 2025-04-01 06:13:20,242 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:13:20,242 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:18:10 2025-04-01 06:13:20,242 INFO 102126:repeater task daily_netcdf starting 9.4 seconds late 2025-04-01 06:13:20,447 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 06:13:20,447 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 06:13:20,447 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 06:23:10 2025-04-01 06:17:15,665 INFO 102126:repeater task lastensq_os75nb starting 3.7 seconds late 2025-04-01 06:17:16,335 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:17:16,335 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -33242.070 2025-04-01 06:17:16,335 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:22:17 2025-04-01 06:17:16,336 INFO 102126:repeater task quick_os75nb starting 4.3 seconds late 2025-04-01 06:17:18,837 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:17:18,837 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:17:18,837 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:17:18,837 INFO 102126:repeater task quick_os75nb: added 109 extra increments to time_to_run 2025-04-01 06:17:18,837 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:22:17 2025-04-01 06:17:18,838 INFO 102126:repeater task lastfew_vec_os75nb starting 6.8 seconds late 2025-04-01 06:17:19,457 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:17:19,457 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:17:19,457 INFO 102126:repeater task lastfew_vec_os75nb: added 109 extra increments to time_to_run 2025-04-01 06:17:19,457 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:22:17 2025-04-01 06:18:14,503 INFO 102126:repeater task 3dayplots_os75nb starting 3.7 seconds late 2025-04-01 06:18:18,903 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:18:18,904 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:23:10 2025-04-01 06:22:19,119 INFO 102126:repeater task lastensq_os75nb starting 2.1 seconds late 2025-04-01 06:22:19,787 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:22:19,787 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -33545.525 2025-04-01 06:22:19,787 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:27:22 2025-04-01 06:22:19,787 INFO 102126:repeater task quick_os75nb starting 2.8 seconds late 2025-04-01 06:22:22,313 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:22:22,313 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:22:22,313 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:22:22,313 INFO 102126:repeater task quick_os75nb: added 110 extra increments to time_to_run 2025-04-01 06:22:22,313 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:27:22 2025-04-01 06:22:22,313 INFO 102126:repeater task lastfew_vec_os75nb starting 5.3 seconds late 2025-04-01 06:22:22,927 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:22:22,927 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:22:22,927 INFO 102126:repeater task lastfew_vec_os75nb: added 110 extra increments to time_to_run 2025-04-01 06:22:22,927 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:27:22 2025-04-01 06:23:07,967 INFO 102126:repeater task plot_tsstats_os75nb starting 2.2 seconds late 2025-04-01 06:23:08,741 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 06:23:08,741 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 06:23:08,741 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 06:38:05 2025-04-01 06:23:08,741 INFO 102126:repeater task grab_lastpings_os75nb starting 2.9 seconds late 2025-04-01 06:23:08,831 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 06:23:08,831 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 06:38:05 2025-04-01 06:23:08,832 INFO 102126:repeater task beamdiagnostics_os75nb starting 3.0 seconds late 2025-04-01 06:23:09,823 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 06:23:09,823 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:38:05 2025-04-01 06:23:09,823 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.0 seconds late 2025-04-01 06:23:10,818 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 06:23:10,819 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:38:05 2025-04-01 06:23:10,819 INFO 102126:repeater task daily_netcdf starting 0.0 seconds late 2025-04-01 06:23:11,018 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 06:23:11,018 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 06:23:11,018 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 06:33:10 2025-04-01 06:23:11,019 INFO 102126:repeater task ggatime_diagnostics starting 5.2 seconds late 2025-04-01 06:23:15,033 INFO 102126:repeater task ggatime_diagnostics ended after 4.0 seconds 2025-04-01 06:23:15,033 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:23:15,033 INFO 102126:repeater task ggatime_diagnostics starting 9.2 seconds late 2025-04-01 06:23:18,843 INFO 102126:repeater task ggatime_diagnostics ended after 3.8 seconds 2025-04-01 06:23:18,843 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:23:18,843 INFO 102126:repeater task ggatime_diagnostics starting 13.0 seconds late 2025-04-01 06:23:22,706 INFO 102126:repeater task ggatime_diagnostics ended after 3.9 seconds 2025-04-01 06:23:22,706 INFO 102126:repeater task ggatime_diagnostics next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:23:22,706 INFO 102126:repeater task hbinhcorrstats_posmv starting 16.8 seconds late 2025-04-01 06:23:23,823 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 06:23:23,824 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 06:23:23,824 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 06:38:05 2025-04-01 06:23:23,824 INFO 102126:repeater task plotposmv_posmv starting 17.9 seconds late 2025-04-01 06:23:25,929 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 06:23:25,929 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 06:38:05 2025-04-01 06:23:25,929 INFO 102126:repeater task plot_hbins starting 20.0 seconds late 2025-04-01 06:23:26,990 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 06:23:26,990 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 06:23:26,990 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 06:38:05 2025-04-01 06:23:26,990 INFO 102126:repeater task report_generator starting 21.1 seconds late 2025-04-01 06:23:30,098 INFO 102126:repeater task report_generator output: cruisename is report_dir is uhdas_dir is could not guess heading correction message. specify on command line 0.0 sec : png archive rsync 1.0 sec : db_timerange: plot_db_timeranges.py --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange --noshow /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : db_timerange: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/db_timerange 0.0 sec : png_archive: rsync -au /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/png_archive/ /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_pngarchive 0.0 sec : uhdas_overview: uhdas_info.py --overview --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/uhdas_overview.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_settings: uhdas_info.py --settings --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_settings.txt /home/data/SH-25-03_CalCOFI_Leg1 0.0 sec : adcp_cals: uhdas_info.py --cals --logfile /home/data/SH-25-03_CalCOFI_Leg1/reports/ADCP_cals.txt /home/data/SH-25-03_CalCOFI_Leg1 1.0 sec : ens_hcorr: plot_enshcorr.py --infile /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/cal/rotate/ens_hcorr.asc --outfile /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr --yearbase 2025 --titlestring ens_hcorr_None 0.0 sec : ens_hcorr: png_thumbnail.py --width 200 --outdir /home/data/SH-25-03_CalCOFI_Leg1/reports /home/data/SH-25-03_CalCOFI_Leg1/reports/os75nb_hcorr 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_proc.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_proc.toml.txt 0.0 sec : pycfg: cp -p /home/data/SH-25-03_CalCOFI_Leg1/raw/config/SH-25-03_CalCOFI_Leg1_sensor.toml /home/data/SH-25-03_CalCOFI_Leg1/reports/SH-25-03_CalCOFI_Leg1_sensor.toml.txt 3.0 seconds elapsed html page starts here: /home/data/SH-25-03_CalCOFI_Leg1/reports/index.html 2025-04-01 06:23:30,098 INFO 102126:repeater task report_generator ended after 3.1 seconds 2025-04-01 06:23:30,098 INFO 102126:repeater task report_generator next time_to_run is 2025/04/01 07:23:05 2025-04-01 06:23:35,103 INFO 102126:repeater task noisy_rawtemps_os75 starting 24.3 seconds late 2025-04-01 06:23:37,048 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 06:23:37,048 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 06:33:10 2025-04-01 06:23:37,049 INFO 102126:repeater task 3dayplots_os75nb starting 26.2 seconds late 2025-04-01 06:23:41,444 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:23:41,445 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:28:10 2025-04-01 06:27:26,651 INFO 102126:repeater task lastensq_os75nb starting 4.7 seconds late 2025-04-01 06:27:27,360 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:27:27,360 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -33853.057 2025-04-01 06:27:27,360 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:32:27 2025-04-01 06:27:27,360 INFO 102126:repeater task quick_os75nb starting 5.4 seconds late 2025-04-01 06:27:29,870 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:27:29,870 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:27:29,871 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:27:29,871 INFO 102126:repeater task quick_os75nb: added 111 extra increments to time_to_run 2025-04-01 06:27:29,871 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:32:27 2025-04-01 06:27:29,871 INFO 102126:repeater task lastfew_vec_os75nb starting 7.9 seconds late 2025-04-01 06:27:30,490 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:27:30,490 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:27:30,490 INFO 102126:repeater task lastfew_vec_os75nb: added 111 extra increments to time_to_run 2025-04-01 06:27:30,490 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:32:27 2025-04-01 06:28:15,531 INFO 102126:repeater task 3dayplots_os75nb starting 4.7 seconds late 2025-04-01 06:28:19,926 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:28:19,926 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:33:10 2025-04-01 06:32:30,153 INFO 102126:repeater task lastensq_os75nb starting 3.2 seconds late 2025-04-01 06:32:30,839 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:32:30,839 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -34156.559 2025-04-01 06:32:30,839 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:37:32 2025-04-01 06:32:30,839 INFO 102126:repeater task quick_os75nb starting 3.8 seconds late 2025-04-01 06:32:33,368 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:32:33,369 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:32:33,369 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:32:33,369 INFO 102126:repeater task quick_os75nb: added 112 extra increments to time_to_run 2025-04-01 06:32:33,369 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:37:32 2025-04-01 06:32:33,369 INFO 102126:repeater task lastfew_vec_os75nb starting 6.4 seconds late 2025-04-01 06:32:33,978 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:32:33,978 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:32:33,978 INFO 102126:repeater task lastfew_vec_os75nb: added 112 extra increments to time_to_run 2025-04-01 06:32:33,978 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:37:32 2025-04-01 06:33:14,011 INFO 102126:repeater task noisy_rawtemps_os75 starting 3.2 seconds late 2025-04-01 06:33:15,953 INFO 102126:repeater task noisy_rawtemps_os75 ended after 1.9 seconds 2025-04-01 06:33:15,953 INFO 102126:repeater task noisy_rawtemps_os75 next time_to_run is 2025/04/01 06:43:10 2025-04-01 06:33:15,954 INFO 102126:repeater task 3dayplots_os75nb starting 5.1 seconds late 2025-04-01 06:33:20,353 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:33:20,353 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:38:10 2025-04-01 06:33:20,354 INFO 102126:repeater task daily_netcdf starting 9.5 seconds late 2025-04-01 06:33:20,553 INFO 102126:repeater task daily_netcdf output: daily netcdf extraction failed: adcp_nc.py /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/adcpdb/a_sh /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/contour/os75nb_090_compr SH-25-03_CalCOFI_Leg1 os75nb --compressed --dday_range 090:091 --ship_name Bell Shimada Traceback (most recent call last): File "/usr/local/bin/adcp_nc.py", line 11, in main() File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 595, in main make_nc_compressed(dbpath, outfile, cruise_id, sonar, File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 485, in make_nc_compressed a.make_compressed(file_name, ddrange=ddrange) File "/usr/local/lib/python3.10/dist-packages/pycurrents/adcp/adcp_nc.py", line 346, in make_compressed self.pshort = self.db.get_profiles(nbins=self.nbins, ddrange=ddrange) File "pycurrents/codas/src/_codas.pyx", line 821, in pycurrents.codas._codas.DB.get_profiles File "pycurrents/codas/src/_codas.pyx", line 906, in pycurrents.codas._codas.DB.get_range_length pycurrents.codas._codas.CodasRangeAfterEnd: range is (2025, 90.0, 91.0) 2025-04-01 06:33:20,553 INFO 102126:repeater task daily_netcdf ended after 0.2 seconds 2025-04-01 06:33:20,553 INFO 102126:repeater task daily_netcdf next time_to_run is 2025/04/01 06:43:10 2025-04-01 06:37:35,797 INFO 102126:repeater task lastensq_os75nb starting 3.8 seconds late 2025-04-01 06:37:36,486 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:37:36,486 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -34462.202 2025-04-01 06:37:36,486 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:42:37 2025-04-01 06:37:36,486 INFO 102126:repeater task quick_os75nb starting 4.5 seconds late 2025-04-01 06:37:39,024 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:37:39,024 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:37:39,024 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:37:39,024 INFO 102126:repeater task quick_os75nb: added 113 extra increments to time_to_run 2025-04-01 06:37:39,024 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:42:37 2025-04-01 06:37:39,024 INFO 102126:repeater task lastfew_vec_os75nb starting 7.0 seconds late 2025-04-01 06:37:39,642 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:37:39,642 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:37:39,642 INFO 102126:repeater task lastfew_vec_os75nb: added 113 extra increments to time_to_run 2025-04-01 06:37:39,642 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:42:37 2025-04-01 06:38:09,673 INFO 102126:repeater task plot_tsstats_os75nb starting 3.9 seconds late 2025-04-01 06:38:10,445 INFO 102126:repeater task plot_tsstats_os75nb output: /usr/lib/python3/dist-packages/matplotlib/lines.py:1157: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison neq = current != val /usr/lib/python3/dist-packages/matplotlib/axis.py:124: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if labelcolor == 'inherit': 2025-04-01 06:38:10,445 INFO 102126:repeater task plot_tsstats_os75nb ended after 0.8 seconds 2025-04-01 06:38:10,445 INFO 102126:repeater task plot_tsstats_os75nb next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:38:10,446 INFO 102126:repeater task grab_lastpings_os75nb starting 4.6 seconds late 2025-04-01 06:38:10,536 INFO 102126:repeater task grab_lastpings_os75nb ended after 0.1 seconds 2025-04-01 06:38:10,536 INFO 102126:repeater task grab_lastpings_os75nb next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:38:10,536 INFO 102126:repeater task beamdiagnostics_os75nb starting 4.7 seconds late 2025-04-01 06:38:11,525 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 06:38:11,525 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:38:11,525 INFO 102126:repeater task beamdiagnostics_os75nb starting 5.7 seconds late 2025-04-01 06:38:12,519 INFO 102126:repeater task beamdiagnostics_os75nb ended after 1.0 seconds 2025-04-01 06:38:12,519 INFO 102126:repeater task beamdiagnostics_os75nb next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:38:12,519 INFO 102126:repeater task hbinhcorrstats_posmv starting 6.6 seconds late 2025-04-01 06:38:13,631 INFO 102126:repeater task hbinhcorrstats_posmv output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/hbin_diagnostics.py:185: UserWarning: FixedFormatter should only be used together with FixedLocator ax1.set_xticklabels(['',]) 2025-04-01 06:38:13,631 INFO 102126:repeater task hbinhcorrstats_posmv ended after 1.1 seconds 2025-04-01 06:38:13,631 INFO 102126:repeater task hbinhcorrstats_posmv next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:38:13,631 INFO 102126:repeater task plotposmv_posmv starting 7.7 seconds late 2025-04-01 06:38:15,766 INFO 102126:repeater task plotposmv_posmv ended after 2.1 seconds 2025-04-01 06:38:15,766 INFO 102126:repeater task plotposmv_posmv next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:38:15,766 INFO 102126:repeater task plot_hbins starting 9.9 seconds late 2025-04-01 06:38:16,831 INFO 102126:repeater task plot_hbins output: /usr/local/lib/python3.10/dist-packages/pycurrents/adcp/plot_hbin_subs.py:178: RuntimeWarning: invalid value encountered in remainder dh = np.remainder(h2 - hbase + 90, 360)-90 2025-04-01 06:38:16,831 INFO 102126:repeater task plot_hbins ended after 1.1 seconds 2025-04-01 06:38:16,831 INFO 102126:repeater task plot_hbins next time_to_run is 2025/04/01 06:53:05 2025-04-01 06:38:21,836 INFO 102126:repeater task 3dayplots_os75nb starting 11.0 seconds late 2025-04-01 06:38:26,224 INFO 102126:repeater task 3dayplots_os75nb ended after 4.4 seconds 2025-04-01 06:38:26,224 INFO 102126:repeater task 3dayplots_os75nb next time_to_run is 2025/04/01 06:43:10 2025-04-01 06:42:41,471 INFO 102126:repeater task lastensq_os75nb starting 4.5 seconds late 2025-04-01 06:42:42,153 INFO 102126:repeater task lastensq_os75nb ended after 0.7 seconds 2025-04-01 06:42:42,153 WARNING 102126:repeater task lastensq_os75nb: Expected update of /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time did not occur, dt = -34767.877 2025-04-01 06:42:42,153 INFO 102126:repeater task lastensq_os75nb next time_to_run is 2025/04/01 06:47:42 2025-04-01 06:42:42,153 INFO 102126:repeater task quick_os75nb starting 5.2 seconds late 2025-04-01 06:42:44,687 INFO 102126:repeater task quick_os75nb output: skipping: SHIP NAME None ... continuing 2025-04-01 06:42:44,687 INFO 102126:repeater task quick_os75nb ended after 2.5 seconds 2025-04-01 06:42:44,687 INFO 102126:repeater task quick_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:42:44,687 INFO 102126:repeater task quick_os75nb: added 114 extra increments to time_to_run 2025-04-01 06:42:44,687 INFO 102126:repeater task quick_os75nb next time_to_run is 2025/04/01 06:47:42 2025-04-01 06:42:44,687 INFO 102126:repeater task lastfew_vec_os75nb starting 7.7 seconds late 2025-04-01 06:42:45,314 INFO 102126:repeater task lastfew_vec_os75nb ended after 0.6 seconds 2025-04-01 06:42:45,314 INFO 102126:repeater task lastfew_vec_os75nb: Time from /home/data/SH-25-03_CalCOFI_Leg1/proc/os75nb/load/wrote_ens_time: 2025/03/31 21:03:07 2025-04-01 06:42:45,314 INFO 102126:repeater task lastfew_vec_os75nb: added 114 extra increments to time_to_run 2025-04-01 06:42:45,314 INFO 102126:repeater task lastfew_vec_os75nb next time_to_run is 2025/04/01 06:47:42 2025-04-01 06:43:05,735 INFO 102126:incr_class translate exiting, gps /home/data/SH-25-03_CalCOFI_Leg1/rbin/gpsnav 2025-04-01 06:43:05,735 INFO 102126:incr_class translate exiting, pmv /home/data/SH-25-03_CalCOFI_Leg1/rbin/posmv 2025-04-01 06:43:05,735 INFO 102126:incr_class translate exiting, hdg /home/data/SH-25-03_CalCOFI_Leg1/rbin/quadrans 2025-04-01 06:43:05,735 INFO 102126:incr_class translate exiting, gps /home/data/SH-25-03_CalCOFI_Leg1/rbin/posmv 2025-04-01 06:43:10,331 INFO 102126:DAS_while_logging cleanup_figs not_logging 2025-04-01 06:43:10,334 INFO 102126:DAS_while_logging Exiting DAS_while_logging.py 2025-04-01 06:43:35,778 INFO 558545:DAS_while_logging Starting DAS_while_logging.py 2025-04-01 06:43:35,783 INFO 558545:DAS_while_logging Cruiseid: SH-25-03_CalCOFI_Leg1 2025-04-01 06:43:35,788 INFO 558545:DAS_while_logging starting run method 2025-04-01 06:43:35,788 INFO 558545:DAS_while_logging speedlog_stopflagfile: /home/adcp/flags/DAS_speedlog.stop 2025-04-01 06:43:35,788 INFO 558545:DAS_while_logging speedlog_runflagfile: /home/adcp/flags/DAS_speedlog.running 2025-04-01 06:43:35,788 INFO 558545:DAS_while_logging speedlog command is /usr/local/bin/DAS_speedlog.py & 2025-04-01 06:43:35,989 INFO 558545:DAS_while_logging Not starting pubVL. 2025-04-01 06:43:35,989 INFO 558545:incr_class gpsnav gps 1 2025-04-01 06:43:35,989 INFO 558545:incr_class 2025-04-01 06:43:35,989 INFO 558545:incr_class posmv gps 1 2025-04-01 06:43:35,989 INFO 558545:incr_class 2025-04-01 06:43:35,989 INFO 558545:incr_class posmv pmv 0 2025-04-01 06:43:35,989 INFO 558545:incr_class 2025-04-01 06:43:35,989 INFO 558545:incr_class quadrans hdg 1 2025-04-01 06:43:35,989 INFO 558545:incr_class 2025-04-01 06:43:35,990 INFO 558545:DAS_while_logging active_procdirnames: ['os75nb'] 2025-04-01 06:43:35,990 INFO 558545:repeater task noisy_rawtemps_os75 initial time_to_run is 2025/04/01 06:53:40 2025-04-01 06:43:35,990 INFO 558545:repeater PeriodicSystem: noisy_rawtemps_os75 interval is 600 seconds cd /home/adcp/uhdas_tmp; plot_noisy_rawtemps.py --inst os75 --noshow --save_figT --outdir ~/daily_report --outdir2 ~/www/figures --outfilebase os75_noisy_temps 2025-04-01 06:43:35,990 INFO 558545:repeater task lastensq_os75nb initial time_to_run is 2025/04/01 06:48:40 2025-04-01 06:43:35,991 INFO 558545:repeater PeriodicSystem: lastensq_os75nb interval is 305 seconds cd /home/adcp/uhdas_tmp; run_lastensq.py -d os75nb --update_pygbin --averages --plotens_mpl --kt_vecprof --pingwarnN 30 --ktsdir 2025-04-01 06:43:35,991 INFO 558545:repeater task quick_os75nb initial time_to_run is 2025/04/01 06:48:40 2025-04-01 06:43:35,991 INFO 558545:repeater PeriodicSystem: quick_os75nb interval is 305 seconds cd /home/adcp/uhdas_tmp; run_quick.py -d os75nb 2025-04-01 06:43:35,991 INFO 558545:repeater task lastfew_vec_os75nb initial time_to_run is 2025/04/01 06:48:40 2025-04-01 06:43:35,991 INFO 558545:repeater PeriodicSystem: lastfew_vec_os75nb interval is 305 seconds cd /home/adcp/uhdas_tmp; run_lastfew_vec.py -d os75nb 2025-04-01 06:43:35,991 INFO 558545:repeater task 3dayplots_os75nb initial time_to_run is 2025/04/01 06:48:40 2025-04-01 06:43:35,991 INFO 558545:repeater PeriodicSystem: 3dayplots_os75nb interval is 300 seconds cd /home/adcp/uhdas_tmp; run_3dayplots_mpl.py -d os75nb --plot_amp 2025-04-01 06:43:35,991 INFO 558545:repeater task plot_tsstats_os75nb initial time_to_run is 2025/04/01 06:58:35 2025-04-01 06:43:35,991 INFO 558545:repeater PeriodicSystem: plot_tsstats_os75nb interval is 900 seconds cd /home/adcp/uhdas_tmp; run_plot_tsstats.py os75nb 2025-04-01 06:43:35,991 INFO 558545:repeater task grab_lastpings_os75nb initial time_to_run is 2025/04/01 06:58:35 2025-04-01 06:43:35,991 INFO 558545:repeater PeriodicSystem: grab_lastpings_os75nb interval is 900 seconds cd /home/adcp/uhdas_tmp; run_grab_lastpings.py -d os75nb 2025-04-01 06:43:35,992 INFO 558545:repeater task beamdiagnostics_os75nb initial time_to_run is 2025/04/01 06:58:35 2025-04-01 06:43:35,992 INFO 558545:repeater PeriodicSystem: beamdiagnostics_os75nb interval is 900 seconds cd /home/adcp/uhdas_tmp; plot_beamdiagnostics.py -m 20 -d os75nb --savefigs --webcopy -p amplitude 2025-04-01 06:43:35,992 INFO 558545:repeater task beamdiagnostics_os75nb initial time_to_run is 2025/04/01 06:58:35 2025-04-01 06:43:35,992 INFO 558545:repeater PeriodicSystem: beamdiagnostics_os75nb interval is 900 seconds cd /home/adcp/uhdas_tmp; plot_beamdiagnostics.py -m 20 -d os75nb --savefigs --webcopy -p velocity 2025-04-01 06:43:35,992 INFO 558545:repeater task daily_netcdf initial time_to_run is 2025/04/01 06:53:40 2025-04-01 06:43:35,992 INFO 558545:repeater PeriodicSystem: daily_netcdf interval is 600 seconds cd /home/adcp/uhdas_tmp; daily_netcdf.py 2025-04-01 06:43:35,992 INFO 558545:repeater task ggatime_diagnostics initial time_to_run is 2025/04/01 07:13:35 2025-04-01 06:43:35,992 INFO 558545:repeater PeriodicSystem: ggatime_diagnostics interval is 1800 seconds cd /home/adcp/uhdas_tmp; ggatime_diagnostics.py --ndays -1 --zoomname auto --yearbase 2025 --outdir ~/daily_report --outdir2 ~/www/figures --save_figT --noshow --save_hist --uhdas_dir /home/data/current_cruise 2025-04-01 06:43:35,993 INFO 558545:repeater task ggatime_diagnostics initial time_to_run is 2025/04/01 07:13:35 2025-04-01 06:43:35,993 INFO 558545:repeater PeriodicSystem: ggatime_diagnostics interval is 1800 seconds cd /home/adcp/uhdas_tmp; ggatime_diagnostics.py --ndays -1 --zoomname fixed --yearbase 2025 --outdir ~/daily_report --outdir2 ~/www/figures --save_figT --noshow --save_hist --uhdas_dir /home/data/current_cruise 2025-04-01 06:43:35,993 INFO 558545:repeater task ggatime_diagnostics initial time_to_run is 2025/04/01 07:13:35 2025-04-01 06:43:35,993 INFO 558545:repeater PeriodicSystem: ggatime_diagnostics interval is 1800 seconds cd /home/adcp/uhdas_tmp; ggatime_diagnostics.py --ndays -1 --zoomname zoom --yearbase 2025 --outdir ~/daily_report --outdir2 ~/www/figures --save_figT --noshow --save_hist --uhdas_dir /home/data/current_cruise 2025-04-01 06:43:35,994 INFO 558545:procsetup Network interface for speedlog websocket is eno2 2025-04-01 06:43:35,995 WARNING 558545:procsetup No web_speedlog IP address was found for interface eno2 2025-04-01 06:43:35,995 WARNING 558545:procsetup No web_speedlog IP address was found; using 127.0.0.1 2025-04-01 06:43:35,995 INFO 558545:uhdas_webgen /home/adcp/config/proc_cfg.py is not found 2025-04-01 06:43:35,996 INFO 558545:uhdas_webgen Copied /home/adcp/config/proc_cfg.toml to /home/adcp/www/configs/proc_cfg.toml.txt. 2025-04-01 06:43:35,996 INFO 558545:uhdas_webgen /home/adcp/config/sensor_cfg.py is not found 2025-04-01 06:43:35,996 INFO 558545:uhdas_webgen Copied /home/adcp/config/sensor_cfg.toml to /home/adcp/www/configs/sensor_cfg.toml.txt. 2025-04-01 06:43:35,996 INFO 558545:uhdas_webgen copying /home/adcp/config/cmdfiles/os75_default.cmd to /home/adcp/www/configs/os75_default.cmd.txt 2025-04-01 06:43:36,081 INFO 558545:repeater task hbinhcorrstats_posmv initial time_to_run is 2025/04/01 06:58:36 2025-04-01 06:43:36,081 INFO 558545:repeater PeriodicSystem: hbinhcorrstats_posmv interval is 900 seconds cd /home/adcp/uhdas_tmp;run_update_hbins.py;run_hbinhcorrstats.py --hcorr_inst posmv --plotdh --printstats 2025-04-01 06:43:36,081 INFO 558545:repeater task plotposmv_posmv initial time_to_run is 2025/04/01 06:58:36 2025-04-01 06:43:36,081 INFO 558545:repeater PeriodicSystem: plotposmv_posmv interval is 900 seconds cd /home/adcp/uhdas_tmp; run_plotposmv.py -n25 -d posmv 2025-04-01 06:43:36,081 INFO 558545:repeater task plot_hbins initial time_to_run is 2025/04/01 06:58:36 2025-04-01 06:43:36,081 INFO 558545:repeater PeriodicSystem: plot_hbins interval is 900 seconds cd /home/adcp/uhdas_tmp; run_plot_hbins.py -n25 --plot_hbins --plot_gph_gaps 2025-04-01 06:43:36,081 INFO 558545:DAS_while_logging staging cruise report cd /home/adcp/uhdas_tmp; uhdas_report_generator.py --hacc_cutoff 0.020 -u /home/data/SH-25-03_CalCOFI_Leg1 & 2025-04-01 06:43:36,081 INFO 558545:repeater task report_generator initial time_to_run is 2025/04/01 07:43:36 2025-04-01 06:43:36,082 INFO 558545:repeater PeriodicSystem: report_generator interval is 3600 seconds cd /home/adcp/uhdas_tmp; uhdas_report_generator.py --hacc_cutoff 0.020 -u /home/data/SH-25-03_CalCOFI_Leg1 & 2025-04-01 06:43:36,082 INFO 558545:DAS_while_logging cleanup_figs no_figure 2025-04-01 06:43:36,085 INFO 558545:repeater writing repeaterlist to /home/adcp/uhdas_tmp/repeaters.txt 2025-04-01 06:45:25,706 INFO 558545:incr_class translate exiting, gps /home/data/SH-25-03_CalCOFI_Leg1/rbin/posmv 2025-04-01 06:45:25,706 INFO 558545:incr_class translate exiting, hdg /home/data/SH-25-03_CalCOFI_Leg1/rbin/quadrans 2025-04-01 06:45:25,706 INFO 558545:incr_class translate exiting, gps /home/data/SH-25-03_CalCOFI_Leg1/rbin/gpsnav 2025-04-01 06:45:25,706 INFO 558545:incr_class translate exiting, pmv /home/data/SH-25-03_CalCOFI_Leg1/rbin/posmv 2025-04-01 06:45:26,185 INFO 558545:DAS_while_logging cleanup_figs not_logging 2025-04-01 06:45:26,187 INFO 558545:DAS_while_logging Exiting DAS_while_logging.py