As part of moving the site to Quarto I intend to write a single post to give an overview of changes in minor versions of the packages. In this case, all versions in the 0.3.x series. I will update this same post in the case of minor version updates, and start a new post when the first version in the 0.4.x series is developed. I hope this will reduce the clutter and still provide a good overview of progress. Differences between versions are listed in detail in the NEWS file.
Overview of changes
Versions 0.3.0, 0.3.1, 0.3.2 and 0.3.3 are feature and bug-fix updates. Nearly all changes are related to the interactive measurement of spectral irradiance with function acq_irrad_interactive()
. The logic of the current user interface modes in described in detail in a tutorial. The most important new feature is the automatic acquisition of time series of spectra.
Another enhancement is the possibility of storing additional information in the instrument descriptors: a description of the entrance optics used (make, model, serial number, area of diffuser, and optical fibre) and the source of the calibration data (e.g., file name or spectrometer EEPROM). This allows reliably using a given spectrometer with multiple entrance optics, with a different calibrations for each of them.
Version 0.3.2 adds function irrad_summary_table()
, which is an enhanced version of a function previously used only internally by the package.
This version also adds a repeat entry to the menu shown at the end of a measurement event. This makes it possible to quickly take a new measurement without modifying the integration time settings or protocol used, reusing the previous “dark” and “filter” reference measurements for a new “light” measurement. In “series” mode, the parameters of the series can still be modified for the repeated data acquisition.
Some enhancements to the user interface of previously available modes included changes to the wording of options in menus, a new option to skip automatic tuning of integration time and clearer progress messages.
Following the bug fix in ‘photobiologyWavebands’, in ‘ooacquire’ versions >= 0.3.1 users can choose between PAR and PhR waveband definitions. PhR is defined as a range of wavelengths while PAR is defined as a biologically effective quantity based on McCree’s definition. When used to compute photon irradiance or photon fluence PhR and PAR do not differ. However, when used to compute energy irradiance they result in different values and represent different quantities. The magnitude of the difference depends on the shape of the measured spectrum.
Data for new calibrations of two of our own spectrometers were added, and the wavelength range of some calibrations already present in earlier versions was extended to cover the full usable range of the calibration data.
Time series of spectra
I started developing this feature long ago but only now I got it working smoothly. The task was simplified by enhancements I recently implemented in packages ‘photobiology’ (0.10.16) and ‘ggspectra’ (0.3.11). A major incentive was that I needed this feature for some measurements I wanted to do.
In version 0.3.1 I implemented as a variation on the new “series” interface mode of 0.3.0 the support for high speed acquisition of time series of spectra using the special functions in the OmniDriver API. The API wrappers were already implemented in ‘rOmniDriver’ (>= 0.1.18) but initialization of the API would sometimes fail. This bug was fixed in ‘rOmniDriver’ (>= 0.1.19) and starting from ‘ooacquire’ (0.3.2) this is the minimum version of ‘rOmniDriver’ supported. In version 0.3.3 I fixed some bugs affecting specific boundary conditions for acquisition of time series and better targetted messages of progress. Version 0.3.4 (under development) will extend all the new features to the measurement of spectral fluence.
Three approaches to the acquisition of time series of spectra are supported: a) buffered, which is the fastest, as spectra are retrieved by the driver to a memory buffer, so that measurements take place non-stop, with the previous spectrum being transferred to the computer while the current one is being acquired. It is used when the step delay is set to 0 s and no HDR bracketing is used, i.e., no settings need to be modified in the spectrometer while acquiring the whole time series. b) fast, minimizes the delay with HDR bracketing. It is slower than buffered as acquisition of spectra stops and restarts when settings are modified, and because spectra are retrieved directly by R without buffering. fast acquisition is enabled when step delay is 0 s but HDR bracketing is in use. c) timed, the user sets the time interval at which the successive spectra will be acquired. This approach is enabled whenever the step delay is more than zero, with the caveat that an estimate of the time taken by individual measurements will be computed and this estimate used to limit the shortest accepted setting for step delay. If the step delay is deemed too short, it will be changed to zero and spectra acquired using either the fast or the buffered approach, or to the step delay set to the estimated duration.
Before using the new "series"
mode for acquiring spectra at a rate faster than about once per minute, please read the ‘ooacquire’ vignette that describes acquisition of spectra. This is crucial as timing considerations are far from straightforward because of how spectrometers work.
Bugs fixed
The critical bugs fixed were caused by changes to the OmniDriver API in versions 2.70 and 2.71. These changes broke data acquisition with ‘ooacquire’ because a function that had been in use was removed from the OmniDriver API. The most recent version, 2.71, is now supported, while versions 2.46 and 2.56 continue to work. Possibly versions of OmniDriver released before and in-between these also work but I have not tested ‘ooacquire’ with them.
Support for OmniDriver version 2.56 from 2018 remains important, as this seems to be the last version supporting old spectrometers, including an USB2000 that I am still using. Compatibility with Windows 10 requires a firmware update of the USB2000. (This firmware version is not recognized by OmniDriver 2.56, but this has only minor consequences for its use.)
The function that was removed from the OmniDriver API and previsouly used in ‘ooacquire’ made it possible to query the optical bench configuration from the spectrometer. When a descriptor of the spectrometer with calibration information has been included as part of ‘ooacquire’ the optical bench description attached as metadata to spectra will remain unchanged. However, in those cases when the calibration is retrieved from the spectrometer EEPROM, with the new versions of OmniDriver information about the optical bench will be missing from the metadata. When creating descriptors for new spectrometers the metadata about the optical bench will have to be entered manually. The information stored in the EEPROM and returned by this function was not always correct, so this change is not much of drawback. I need to explore if a different way of retrieving this information in the new versions of OmniDriver
An additional bug was caused, most likely, by changes in package ‘lubridate’. I noticed during testing that wrong times were in some cases retrieved from the computer. Most likely this was because of the change in the default time zone used in ‘lubridate’ or because of some major code base changes and bug fixes in ‘lubridate’ 1.9.0. These changes may have interacted with code in ‘ooacquire’ that unwisely relied on default arguments in functions imported from ‘lubridate’. It is possible that time stamps may have been wrongly recorded with some recent combinations of ‘ooacquire’ and ‘lubridate’ versions. Safest is to update both packages, and check the timing recorded in recently acquired spectra for time shifts. In case time shifts happened, I would expect differences of at least several hours, depending on the local time zone and its time offset to UTC. In this version UTC is used explicitly everywhere to keep things as simple and reliable as possible. Currently messages related to time series acquisition report times in UTC, which may be surprising. Plots include timezone information with the time stamp.
Plans for the future
Function acq_fraction_interactive()
used for the acquisition of transmittance and reflectance spectra needs to be updated to match the design and features of function acq_irrad_interactive()
used for the acquisition of spectral irradiance and spectral fluence. It also needs to be thoroughly tested, and any bugs found, fixed.
Some other enhancements are also pending: 1) better documentation and 2) unit tests for data acquisition based on the underlying functions as these do not need interactive user input. These would need to be run live with spectrometers. Current unit tests are based on stored raw counts data and test computations used to convert raw counts into physical units and the reading of raw spectral data stored in the different file formats supported.
In the future it may be necessary to remove the dependency on OmniDriver, which is proprietary, closed source software written in Java. Java itself is no longer free for commercial use since Oracle changed the more permissive licencing SUN had for Java. This licencing change can be worked around using one of the OpenJDK distributions. Another alternative is to use the free and open source SeaBreeze driver instead of OmniDriver, in its original version written in C, or the Python version of it. Python is easier to interface to R, but the C implementation could be faster.
Documentation web site includes all help pages, with output from all examples and vignettes in HTML format .
Please raise issues concerning bugs or enhancements to this package through GitHub at https://github.com/aphalo/ooacquire/issues. Pull requests are also welcome.