library(ooacquire)
acq_irrad_interactive()
R package ‘ooacquire’
Package ‘ooacquire’ is part of the R for Photobiology suite of R packages and its full documentation is available on-line. It depends on package ‘rOmniDriver’, whose full documentation is also available on-line.
Build information
Built with ‘ooacquire’ 0.5.1, ‘rOmniDriver’ 0.1.20, ‘photobiology’ 0.11.3, ‘ggspectra’ 0.3.12.9002 and ‘quarto’ 1.4.4 with ‘quarto CLI’ 1.5.56 on 2024-08-24.
Warning
The description and diagrams shown in this page are valid for ‘ooacquire’ versions \(\geq\) 0.4.4. Versions \(\leq\) 0.4.2 differ from version 0.4.4 in the user interface implemented in function acq_irrad_interactive() described in this page. Acquisition of fluence spectra with function acq_irrad_interactive() is possible only in versions \(\geq\) 0.3.4 of ‘ooacquire’. The user interface and functionality of repeated measurements was modified in version 0.4.3 and again in 0.4.4. The text of menus and prompts was slightly adjusted in versions 0.4.3-1 and 0.4.3-2, and again in 0.4.4. In addition, version 0.4.3, introduced asynchronous file saving as a still experimental feature. This version also introduced the possibility of interactively disabling the display of plots in the R session independently of saving of plots to PDF files on disk. Version 0.4.4 fixes a few bugs and refines the user interface.
1 Introduction
R package ‘ooacquire’ is a component of the R for Photobiology suite of packages and supports the near-real-time acquisition of spectral data using Ocean Optics spectrometers from Ocean Insight. Package ‘ooacquire’ imports and uses classes, methods and functions defined in other packages of the suite.
Spectral energy irradiance is expressed in \(W\,m^{-2}\,nm^{-1}\) and spectral fluence in \(J\,m^{-2}\,nm^{-1}\). The corresponding photon-based units are \(mol\,s^{-1}\,m^{-2}\,nm^{-1}\) and \(mol\,m^{-2}\,nm^{-1}\). Irradiance is a flux rate expressed per unit time and fluence is an exposure expressed per event. When illumination is constant during a short measurement event, it is customary to describe its intensity using irradiance. When illumination is a discrete pulse shorter than the duration of the measurement event, fluence is used to describe the amount of energy or number of photons per pulse. In general, which approach is preferred depends on how the output of the light source varies in time and whether a description of the temporal variation is of interest or not.
Note
Fluence can be easily computed from irradiance if we known the duration of the exposure, and for long exposures to constant irradiance, this is the preferred approach. Computing actual irradiance during a short pulse from fluence is nearly impossible unless the pulse or pulses are shaped like a square wave and we know their duration. Of course, if pulses are regularly spaced in time, have the same duration and amplitude and the measurement lasts for long enough, we can obtain an average irradiance across a train of pulses, as emitted by some light sources, e.g., because of AC mains power. PWM dimming or other non-DC power lamp drivers can be described as a wave pattern, in which case it is important for a measurement to be timed so that it includes a small number of whole cycles or many cycles. It is also possible to compute based on the frequency of repeated pulses an average irradiance from a measurement of fluence per pulse.
Function acq_irrad_interactive() implements the interactive acquisition of spectral irradiance and spectral fluence data using Ocean Optics array spectrometers. It returns the acquired data as a side effect by saving them to files on disk. The raw-counts data are acquired and converted into spectral irradiance values (if desired conversion can be disabled and later done off-line).
The acquisition of an irradiance spectrum involves in most cases the acquisition of multiple raw-counts spectra from a spectroradiometer, usually a dark reference spectrum and the spectrum corresponding to the light source being measured. When using sophisticated measurement and correction protocols, the number of raw-counts spectra needed to compute one irradiance spectrum can be six or more. Each of these raw-count spectra, can itself be computed in the spectroradiometer hardware as the average of multiple successive acquisitions of raw-counts data. The different measurement protocols supported match specific algorithms for the computation of physical quantities, corrected for dark signal and stray light and exploiting§ the full dynamic range of the array detector. The algorithms implemented are described in the vignette titled Algorithms.
The spectrometers supported by the OmniDriver driver manage the acquisition and simple computations autonomously. Commands are used to set the acquisition parameters and request specific data processing for the acquisition of the raw-counts spectra. At this point the spectrometer starts acquiring spectra continuously. However, as they are rapidly discarded, only the most recently acquired raw-counts spectrum is available for retrieval (waiting is needed after a change in settings). Thus the raw-counts spectra are retrieved one-by-one from the spectrometer to the computer and stored in RAM. From this point onwards all the computations are done in the computer to which the spectrometer is tethered.
The sooner the data are saved to disk the smaller the risk of their loss is. However, computation and writing to disk are time consuming, introducing a delay. In addition, when conditions, specially temperature of the spectrometer, are stable the same reference spectra, such as the dark raw-counts spectrum, can be reused. Delayed computations and writing of data to files makes it possible to avoid saving many copies of the same reference spectra.
Originally, ‘ooacquire’ only implemented the acquisition of individual spectra. In version 0.3.0 acquisition of time series of spectra was implemented and in subsequent versions gradually enhanced. As of version 0.4.3, the automatic acquisition of time series of spectra at frequencies ranging from the fastest allowed by the spectrometer hardware to the slowest and longest that the temporal stability of spectrometer readings allow are well supported.
Glossary of terms used in this page
I will use the following terms in the rest of this page. I will use raw-counts spectrum, as described above, when I mean each raw-data spectrum retrieved from the spectroradiometer. I will use irradiance spectrum or fluence spectrum when I mean a single spectrum in physical units of irradiance or fluence, computed from one to several raw-counts spectra. I will use measurement event to describe a single spectrum or a set of related spectra, such as time series, acquired, processed and saved to disk in the same operation. I will use measurement repeat to describe a succeeding measurement event done using the same spectrometer settings and using the same reference spectra, but with separate computation of physical quantities and separate saving to files on disk. Finally, a measurement session includes all measurements done during a single call to the function acq_irrad_interactive.
The data for each measurement event are saved as soon as computations are done on the acquired raw-spectra. Raw-counts spectra are collected into a single list object, and irradiance or fluence spectra into a separate source_spct object. These objects are saved to disk as soon as they are available in a single R data file (file name ending .rda). Optionally, plots of the spectra can be saved to PDF files. Depending on the user interface mode selected, at any time during a measurement session, summaries of the previous measurements or collecting those measurements into a single file on disk is possible.
It is possible to skip the computations and saving of irradiance or fluence spectra, and save only raw-counts spectra to disk. However, the raw-counts data, instrument descriptor, instrument settings, metadata indicating what, when, where and how was measured, as well as user’s comments. These detailed metadata provide a trace of the origin of the data and ensure reproducibility.
Plots are displayed on the screen for each measurement event (can be disabled). A measurement repeat consists of either a single light spectrum or of a time series of light spectra.
Package ‘ooacquire’ has been evolving for several years and will continue to evolve, specially with respect to interactive acquisition of spectral data. The user interface for spectral irradiance is currently more polished and complex than the interface for transmittance and reflectance spectra. This is because both myself and other users most of the time use ‘ooacquire’ to measure spectral irradiance.
The user interface
The user interfaces (UIs) implemented in functions acq_irrad_interactive() and acq_fraction_interactive() are text based rather than graphical. The design is optimized for frequent users rather than for occasional users. A key design objective has been to make the acquisition of multiple spectra per measurement session as comfortable and as fast as possible, i.e., avoid tedious repetition. This is different to what is usually meant by user friendly involving hand-holding and verbose options within the interface. The interface is designed to be friendly to frequent users and to users who acquire hundreds or even several thousand spectra per session. In spite of this, students in my courses have learnt to use it in just some minutes.
As much as possible, options are selected by typing a single character, or even just the enter key, thanks to context and state dependent defaults. The interface is as far as it can be from visually attractive, but it works well for myself, and most other people who use it.
In addition, different interface modes hide from the user specific menus or specific options from menus, allowing flexibility while avoiding a bloated user interface.
2 Starting a session
At the R console we have to load and attach the ‘ooacquire’ package and call function acq_irrad_interactive(). This function has many parameters that make it possible to change default arguments to match different situations. The idea is that the user creates a short script calling acq_irrad_interactive() with the arguments needed to tailor its behaviour to the specific type of measurement and possibly, taste.
The video below demonstrates the use of the interface for an old version of ‘ooacquire’ (will be replaced by an up-to-date one).
The UI implemented in acq_irrad_interactive() is rather terse and, thus, needs to be clearly documented. The vignettes of package ‘ooacquire’ include some examples. However, a description of the logic can be best done using diagrams.
Note
Mermaid is a markdown-like language for the easy creation of diagrams, but not yet supported in R vignettes. As diagrams are encoded using Mermaid and it includes video content, this tutorial is published as a page at the R for Photobioly web site instead of as part of the ‘ooacquire’ R package documentation.
3 Spectral quantity
Through parameter qty.out we can choose between four alternatives: "raw" (raw instrument counts for each detector pixel), "cps" (counts per second for each detector pixel), "irrad" (energy and/or photon spectral irradiance) and "fluence" (energy and/or photon spectral fluence). Spectral irradiance and spectral fluence can be computed in near real time only if irradiance calibration factors and correction method definitions are available for the spectrometer used. The raw detector counts are always returned allowing in all cases spectral irradiance and spectral fluence to be computed or recomputed at a later time. The quantity to be returned is selected when the session is started. Computation time is only very slightly less for quantity “cps” than for “irradiance” and “fluence”, and it is included for those cases when an irradiance calibration is not available. With mode "raw", computation time is only a small fraction of that when returning any of the other quantities and file saving time is also cut at least in half. Saving "raw" data can be useful when speed matters and doing the computations at a later time is possible. How fast? About 10 s to process a series of 1000 spectra and start the acquisition of the next time series.
library(ooacquire)
acq_irrad_interactive(qty.out == "fluence")4 Interface modes
Function acq_irrad_interactive() is like a Swiss Army Knife. It has multiple ‘’personalities’’ and the first step is the choice of interface.mode, which determines which interface will be shown to the user. Basically, the selected interface.mode determines which options can be modified by the user through the interactive UI and which ones are fixed at the values set by arguments passed when the function is called.
Normally, when measuring several spectra in a row, any redundant need to select choices in a menu or to enter repeatedly the same setting values by the user is distracting and time consuming. This problem is tackled using three three strategies: 1) with multiple interface modes each displaying a subset of the user interface, 2) easily triggering repeated measurements with the most recently used settings and reference readings, , and 3) by context-dependent defaults in menus and 4) automatic generation of sequential file names offered as default.
In this section I describe strategies 1) and 2). Five modes are available (Figure 1), and which one is active is selected by the argument passed to formal parameter interface.mode when acq_irrad_interactive() is called. There are five main interface modes built into acq_irrad_interactive() with "auto" as default.
%%{ init: {"graph": {"htmlLabels": true}} }%%
%%| fig-align: center
flowchart TB
A[interface.mode = ] -.-> B("simple")
A --> C(<b>auto</b>)
A -.-> D("series")
A -.-> E("manual")
A -.-> F("full")
"auto", is highlighted.
Each mode activates features by making visible the corresponding interface or disables them by hiding their interface (Table 1). Appending -attr to these mode names enables the UI for setting the values of attributes what.measured and comment (not shown in the table). For example, "simple" becomes "simple-attr".
acq_irrad_interactive().
| UI mode | Single spectra | Repeats | Time series | Collections | Summaries |
|---|---|---|---|---|---|
"simple" |
yes | yes | no | no | yes |
"auto" |
yes | yes | no | yes | yes |
"series" |
yes | yes | yes | no | no |
"manual" |
yes | yes | no | yes | yes |
"full" |
yes | yes | no | yes | yes |
Modes "simple", "auto", "series" or "full" can be used to measure spectral irradiance. Modes "manual" or "full" has to be used to measure spectral fluence. The names "auto" and "manual" refer to whether the integration time is adjusted automatically or set manually.
The interface.modes described above in Table 1 can be modified through additional arguments passed when calling function acq_irrad_interactive(). To tweak the modes we can pass a logical value (TRUE or FALSE) to parameters save.collections and\or save.summaries. In addition saving of plots to PDF files is similarly controlled through parameter save.pdfs and asynchronous (non-blocking) saves through parameter async.saves.
4.1 Selecting the interface mode
When calling function acq_irrad_interactive() we can pass arguments to tailor the UI to our current needs. For example, to use the "simple" mode instead of the default "auto" mode we can call the function as follows.
acq_irrad_interactive(interface.mode = "simple")If we do not intend to create and save collections of spectra or summaries, we can hide the menus used to obtain them, saving a few key presses. More importantly, a simpler interface with fewer options is easier and faster to use.
acq_irrad_interactive(save.collections = FALSE,
save.summaries = FALSE)If we do not want to save ready made summaries but still be able to create and save collections of spectra we can use.
acq_irrad_interactive(save.summaries = FALSE)Of course, these options are independent of each other, and can be combined as needed, and used together with all the available interface modes.
Managing computation and file-saving time overheads
Plotting many spectra in a single figure is a slow process as each spectrum can contain observations at a couple of thousands of different wavelengths. The slowest step is the rendering of the plot as a bitmap or vector graphic format. Saving the data themselves to a file on disk can also be slow for time series of many hundreds of spectra or similarly large collections of spectra. Saving plots to PDF files requires rendering them again even if they have already been displayed on screen.
There are several approaches to avoid long delays:
Acquiring time series of spectra. In the fastest mode, using buffered data acquisition, the overhead is about 0 ms for short integration times with fast spectrometers like the Maya 2000 Pro. When using this mode computations and saving of the data to files is postponed until after the end of the series. Depending of the spectrometer, in this mode it is possible to acquire hundreds of spectra per second and time series containing a few thousands of spectra. This is possible using the
"series"interface mode.Not displaying plots on the screen can save some time, specially when measuring time series or multiple repeats. This can be changed interactively and the default set by passing an argument when calling
acq_irrad_interactive().
acq_irrad_interactive(show.figs = FALSE)- Limiting the number of spectra included in a plot. A numeric argument to parameter
plot.lines.maxenables random sampling of spectra when its value (11 by default) is exceeded. Two reasons are behind the choice of 11: rendering is reasonably fast and ‘ggplot2’ supports 11 line patterns, so that it is possible to label 11 plots with their position in the series. For example to always plot 50 spectra we can use (Infcould be used, but plotting 100 spectra (\(200\,000\) observations is already quite slow, even in a fast computer):
acq_irrad_interactive(plot.lines.max = 50)- Completely disabling the saving of PDF files.
acq_irrad_interactive(save.pdfs = FALSE)Skipping conversion to physical units by setting
qty.outto"cps"or"raw". This saves always some computation time, whileqty.out = "raw"also reduces the amount of data saved to file.Asynchronous saving of files. By setting
async.savestoTRUEthe saving of data files and the rendering and saving to PDF files are done in parallel, allowing user interaction and data acquisition to continue meanwhile. To enable this feature, R package ‘mirai’ needs to be installed. The default isFALSEas this makes a difference only with long time series or possibly when saving the files to a HDD instead of to a SSD.
acq_irrad_interactive(async.saves = TRUE)5 Flow diagrams for the full UI
Function acq_irrad_interactive() is based on a loop that is repeatedly followed. All features are available within this loop to the user by interaction with menus and dialogues. The same steps are always followed, but depending on the interface.mode specific menus and dialogues are hidden and a fixed default value used, either disabling the feature or using a fixed default value instead of a value interactively entered by the user. In the subsections below the diagrams show all the features potentially available.
5.1 Session start
The UI does not differ among interface modes for the initialization steps. These steps are not part of the loop for the acquisition of individual spectra or series of spectra.
flowchart TB A((<b>start</b>)) -.-> B(select spectrometer\nand channel?) B -.-> C(session name?) --> C1(user name?) --> D(folder name?) --> E(((<b>1</b>))) A --> C
Defaults are provided and formal parameters of function acq_irrad_interactive() can be passed arguments to change these defaults (Table 2). The arguments passed to set the three names should be character strings.
acq_irrad_interactive().
| In diagram | Default | Formal parameter |
|---|---|---|
| select spectrometer | “first” | none |
| select channel | “first” | none |
| session name? | <user name>_<current datetime> | session.name |
| user name? | current user’s login name | user.name |
| folder name? | acq-irrad-<current date> | folder.name |
The loop diagrams shown in the sections below, are all active after these first steps are completed. The encircled 1, signals this connection between the diagram above and those in the sections below.
5.2 Data acquisition Loop
After completing the start-up steps the interface becomes a loop that is repeated until the close-down steps and end of session are triggered by user input. Each “walk” round the loop generates a measurement repeat.
Tip
In the diagrams below I show single boxes not only for the input of single values but also for menus containing multiple options. These menus are attribute?, acquisition parameters?, and series parameters?. Within plot and save and plot and save collection there are options about the formatting of the plot, and in the second case also about the file name used to save the collection.
We start with a complete flowchart of the UI loop for the acquisition of spectra, which shows all the possibilities, but does not correspond to any interface.mode, because all of them are based on subsets.
Once one measurement is completed and saved, a new measurement can be acquire using the same settings for the acquisition as well as reusing the same dark and filter reference spectra (repeat) or with the possibility of changing these settings and acquiring fresh dark and filter reference spectra (next).
flowchart TB
E(((<b>1</b>))) --> G(protocol?)
G --> I(object name?) --> J(attributes?) --> K(acquisition\nparameters?)
K --> L(series parameters?) --> M[[acquire\nspectra]] --> M1[[compute\nirradiance]]
%% M --pause--> MP(ready?) --> M
M -.abort.-> I
M1 --> M2[[display\nplot]] --> NM(tweak plot\nsave or discard?)
NM --save--> N0[(save\ndata and plot)]
NM -.tweak.-> M2
R(how many\nrepeats?) -.repeat.->I
N0 --> N1{collect?} --yes--> N2[(plot and save\ncollection)] --> N3{summarise?} --yes--> N4[(save\nsummaries)] --> O{next\n repeat\n or end?}
O --end--> Q(((<b>2</b>)))
O -.repeat non-stop\n or with pauses.-> R
O --next--> G
N1 --no--> N3
N3 --no--> O
J -.repeat.-> L
NM -.discard.-> O
5.3 Session end
The end of the session does not differ among interface modes (Figure 4). These steps are not part of the loop for the acquisition of individual spectra or series of spectra. In the loop diagrams shown in the section above and those below, the close down steps are the same. The encircled 2, signals this connection between the different diagrams.
flowchart TB Q(((<b>2</b>))) --> O1[(save\nsession info)] --> O2[[disconnect\nspectrometer]] --> P((<b>end</b>))
If async.saves = TRUE ids in use, any pending file saves are completed before ending the session. This may cause a short delay.
6 Available interface modes
The different interface modes differ in the user interface for the measurement loop (Figure 1). The steps to start and end a session are always the same, and not shown again in the diagrams below. These diagrams show the subsets of features enabled by default in each of the modes.
6.1 "simple" mode
The "simple" interface mode provides a minimalist UI, quite similar to the only UI available in early versions of ‘ooacquire’. It is intended for the acquisition of individual irradiance spectra.
%%{ init: {"graph": {"htmlLabels": true}} }%%
flowchart TB
E(((<b>1</b>))) --> G(protocol?)
G --> I(object name?) --> K(acquisition\nparameters?\n<i>simplified</i>)
K --> M[[acquire\nspectra]] --> M1[[compute\nirradiance]]
%% M --pause--> MP(ready?) --> M
M -.abort.-> I
M1 --> M2[[display\nplot]] --> NM(tweak plot\nsave or discard?)
NM --save--> N0[(save\ndata and plot)]
NM -.tweak.-> M2
R(how many\nrepeats?) -.repeat.-> I
N0 --> N3{summarise?} --yes--> N4[(save\nsummaries)] --> O{next\n repeat\n or end?}
O --end--> Q(((<b>2</b>)))
O -.repeat non-stop\n or with pauses.-> R
O --next--> G
N3 --no--> O
I -.repeat.-> M
NM -.discard.-> O
"simple".
Tip
The simplest possible user interface can be obtained by calling acq_irrad_interactive() as shown below, with Figure 6 showing the loop.
acq_irrad_interactive(protocols = "ld", # or some other protocol
interface.mode = "simple",
save.summaries = FALSE,
qty.out = "raw")%%{ init: {"graph": {"htmlLabels": true}} }%%
flowchart TB
E(((<b>1</b>))) --> I(object name?) --> K(acquisition\nparameters?\n<i>simplified</i>)
K --> M[[acquire\nspectra]] --> NM(save or discard?)
NM --save--> N0[(save data)]
R(how many\nrepeats?) -.repeat.-> I
N0 --> O{next\n repeat\n or end?}
O --end--> Q(((<b>2</b>)))
O -.repeat non-stop\n or with pauses.-> R
O --next--> I
I -.repeat.-> M
NM -.discard.-> O
"simple" with preset protocol, disabled summaries and acquisition of raw-counts spectra for later processing off-line.
6.2 "auto" mode
The "auto" interface mode provides a complete UI intended for the acquisition of individual spectra and possibly packing groups of spectra into collections and computing summaries. The default is to adjust the integration time automatically by measurement, and it is best suited to the measurement of sources that emit light continuously with an output that varies little in the short time.
%%{ init: {"graph": {"htmlLabels": true}} }%%
flowchart TB
E(((<b>1</b>))) --> G(protocol?)
G --> I(object name?) --> K(acquisition\nparameters?\n<i>full</i>)
K --> M[[acquire\nspectra]] --> M1[[compute\nirradiance]]
%% M --pause--> MP(ready?) --> M
M -.abort.-> I
M1 --> M2[[display\nplot]] --> NM(tweak plot\nsave or discard?)
NM --save--> N0[(save\ndata and plot)]
NM -.tweak.-> M2
R(how many\nrepeats?) -.repeat.->I
N0 --> N1{collect?} --yes--> N2[(plot and save\ncollection)] --> N3{summarise?} --yes--> N4[(save\nsummaries)] --> O{next\n repeat\n or end?}
O --end--> Q(((<b>2</b>)))
O -.repeat non-stop\n or with pauses.-> R
O --next--> G
N1 --no--> N3
N3 --no--> O
I -.repeat.-> M
NM -.discard.-> O
"auto".
6.3 "manual" mode
Mode "manual" differs from "auto" in that the default is for the user to directly set the integration time. It is most useful when measuring fluence from manually triggered light pulses, such as a xenon flash.
6.4 "series" mode
The "series" interface mode provides a complete UI intended for the acquisition of time series of spectra and possibly packing groups of time series of spectra into collections and computing summaries.
%%{ init: {"graph": {"htmlLabels": true}} }%%
flowchart TB
E(((<b>1</b>))) --> G(protocol?)
G --> I(object name?) --> K(acquisition\nparameters?)
K --> L(series parameters?) --> M[[acquire\nspectra]] --> M1[[compute\nirradiance]]
%% M --pause--> MP(ready?) --> M
M -.abort.-> I
M1 --> M2[[display\nplot]] --> NM(tweak plot\nsave or discard?)
NM --save--> N0[(save\ndata and plot)]
NM -.tweak.-> M2
R(how many\nrepeats?) -.repeat.->I
N0 --> O{next\n repeat\n or end?}
O --end--> Q(((<b>2</b>)))
O -.repeat non-stop\n or with pauses.-> R
O --next--> G
I -.repeat.-> L
NM -.discard.-> O
"series".
6.5 "full" mode
Mode “full” is in practice of little or no use except for testing as it makes all settings of the UI visible to the user.
6.6 "-attr" variants of the modes
Appending "-attr" to the name of any interface.mode enables the menu and dialogues that enable the setting of the what.measured and comment attributes of the source_spct objects into which spectra are saved. This could be represented in the diagrams by the insertion of an attributes? box immediately below object name? as shown in the first diagram showing the full logic.
Repeats
Repeats reuse the acquisition settings of the previous measurement as well as reuse the dark and filter reference spectra, i.e., repeated measurements are only for light spectra under the assumption that they will run unattended. In version 0.4.3, a sequence of multiple repeats can be requested in addition to the single repeats available in recent earlier versions.
The prompt where repeats are set is Repeats:auto/no figs./with figs./pausing//quit/GO (r/n/f/p/q/m-):. It is shown after a measurement has been taken, as the idea is that before taking a series of measurements a first measurement is inspected by the user.
In the case of a single repeat, the behaviour remains the same, and can be started with r. Three flavours of multiple repeats are available: n for non-stop with no display of figures of the spectra, f for non-stop with display of figures, and p) with a pause and prompt at the start of each repeat and display of figures. When repeats are more than one, automatically generated sequential object names are used. A pause with the usual prompt is always displayed before the first repeat but not ahead of individual repeats in the case on non-stop repeats.
As ealier q ends the measuring session, and m or <enter> starts a whole new measurement instead of repeating the last one.
When using interface.mode = "series" the series settings can be changed at the start of the repeats. This is useful as it makes it possible to use as the measurement to repeat a shorter time series or even a time series of length 1. This is unchanged from previous versions of ‘ooacquire’ but now also applies to the start of a sequence of multiple repeats.
7 Measurement protocols
The different diagrams below open the actions that take place within the measure spectra boxes in the diagrams above, previously treated as black boxes. These actions depend on the protocol used for the measurements and on whether the measurement is a fresh one (next) or one reusing settings and reference measurements (repeat). The same measurement protocols are available in the different user interface modes. We show here the protocols available by default. However, users can define their own protocols. In the diagrams below, start and end represent the start and end of a measurement event, i.e., the boundaries of the generic measure spectra boxes in the diagrams above. Within each step in most cases several spectra are acquired, with the same integration time when averaging is used and with different integration times when HDR is used. In all modes except series, the same number of raw spectra are acquired for dark, filter and light if used, and combined into a single irradiance or fluence spectrum. In the case of series and repeats, measurements are repeated in time for light so that several irradiance spectra are obtained measured at regular time intervals.
Interactively, the protocol is selected at the prompt Protocols; light, filter, dark (l/ld/lf/lfd/dl/dfl):, with the list of protocols offered possibly modified by an argument passed when calling acq_irrad_interactive() (see section User defined protocols).
In addition to interactively, the protocol to be used can be selected when calling acq_irrad_interactive() as shown below. The first example, preselects a single protocol and thus disables the menu for selecting protocols during a session.
acq_irrad_interactive(protocols = "ld")In contrast, the next example limits the protocols available through the menu during the session to the two with matching names.
acq_irrad_interactive(protocols = c("l", "ld"))7.1 Protocol l
l stands for acquisition of light scans.
flowchart LR S((<b>start</b>)) --> C C[light\nspectrum or\nspectra] --> E((<b>end</b>)) C -.series.-> C
l.
7.2 Protocol ld
ld stands for acquisition of light and dark scans sequentially in this order.
flowchart LR S((<b>start</b>)) --> A A -.repeat.-> E A[light\nspectrum or\nspectra] --next--> D[dark\nspectrum] --> E((<b>end</b>)) A -.series.-> A
ld.
7.3 Protocol lf
ld stands for acquisition of light and filter scans sequentially in this order.
flowchart LR S((<b>start</b>)) --> A A -.repeat.-> E A[light\nspectrum or\nspectra] --next--> F[filter\nspectrum] --> E((<b>end</b>)) A -.series.-> A
lf.
7.4 Protocol lfd
lfd stands for acquisition of light, filter and dark scans sequentially in this order.
flowchart LR S((<b>start</b>)) --> A A -.repeat.-> E A[light\nspectrum or\nspectra] --next--> B[filter\nspectrum] --> C[dark\nspectrum] --> E((<b>end</b>)) A -.series.-> A
lfd.
7.5 Protocol dl
dl is identical to ld except for the sequential order of the acquisition of data.
flowchart LR S((<b>start</b>)) --next--> A S -.repeat.-> C A[dark\nspectrum] --> C[light\nspectrum or\nspectra] --> E((<b>end</b>)) C -.series.-> C
dl.
7.6 Protocol dfl
dfl is identical to lfd except for the sequential order of the acquisition of data.
flowchart LR S((<b>start</b>)) --next--> A S -.repeat.-> C A[dark\nspectrum] --> B[filter\nspectrum] --> C[light\nspectrum or\nspectra] --> E((<b>end</b>)) C -.series.-> C
dfl.
7.7 User defined protocols
The protocols are defined using a list. The list for the default set of protocols is created with the code below. The names “light”, “filter”, and “dark” must be used as members of the rhs while the names on the lhs can be chosen rather freely. The order of names on the rhs determines the order in which the respective spectra are acquired.
# define measurement protocols
default.protocols <- list(l = "light",
ld = c("light", "dark"),
lf = c("light", "filter"),
lfd = c("light", "filter", "dark"),
dl = rev(c("light", "dark")),
dfl = rev(c("light", "filter", "dark"))
)To restore the simpler options available in earlier version we can pass a simpler list as argument (or alternatively, as shown above, a vector of protocol names such as c("l", "ld", "lfd")) to parameter protocols.
acq_irrad_interactive(protocols = list(l = "light",
ld = c("light", "dark"),
lfd = c("light", "filter", "dark"))
)When measuring a long time series without repeats, we could define a new protocol, "dld", and select it interactively. (In earlier versions of ‘ooacquire’ definitions called "ld" and "lfd" had to be included as they are used as defaults. In ‘ggpp’ \(\geq\) 4.3 the first in the list is used in their absence.)
acq_irrad_interactive(protocols = list(ld = c("light", "dark"),
lfd = c("light", "filter", "dark")
dld = c("dark", "light", "dark"))
)8 Object and file names
Only one name needs to be provided by the user for each measurement or sequence of repeats, the base name of the objects and files that will be created. The actual object names will be create by appending .spct and .raw_mspct and the file names by appending .spct.Rda and .pdf to the base name. As the base name needs to be a syntactically valid name in the R language, it is validated before use.
Starting from version 0.4.3, sequentially numbered names are available. They are implemented by appending a three digits number to the user-provided base name and then used as a base name as described above. Sequential numbering is started when the user inputs a base name ending in #. When a new name ending # is entered the counter is reset, and sequential numbering starts from 001 and continues as long as the user accepts the same base name, shown by default. Starting with version 0.4.4, entering a new name ending in %, behaves as # but does not reset the counter. If the new name does not end in # or %, it is used as is, sequential numbering disabled and the counter reset. In the case of repeated measurements, sequential numbering is enforced even if the user enters a base name not ending in # or %.
When sequential names are not enabled the prompt for a new name is Give a name to the spectrum: and when a sequence is in course the prompt is "Give a name to the spectrum (<base name>) : where <base name> is the previously entered name ending in #. Typing <enter> at this prompt generates the next name in the sequential series. Any other input is interpreted as a new base name.
9 Acquisition parameters
We describe the data acquisition parameters affecting the acquisition of spectral irradiance only briefly here, and refer the reader to the vignette that describes the algorithms used included in package ‘ooacquire’.
A dark measurement is also very frequently used to subtract sensor noise. Stray light is most effectively corrected by measuring it separately from the target of the measurements by means of a filter measurement. Although different protocols include different measurements, which protocol is in use does not change how the acquisition parameters are set as the same settings are used for these measurements.
Above the prompt the current value of each setting is shown. The prompt varies, as depending on the interface.mode in use some options are hidden. The integration time is labelled integ(OK) when the value has been already set or tuned, and labelled integ(??) when the tuning could need to be repeated or the value re-entered. In addition, depending on this and on the interface.mode, the default selected by typing <enter> varies, between t and g. In version 0.4.4, MEASURE was replaced by GO to make the text shorter. The full prompt is:
fixed/retune/tune/sat/range/HDR/undo/help/GO (f/t/T/s/r/h/u/?/g-):
For the integration time used for the acquisition of data, a distinction is made between the duration of a single integration event and the total added-up integration time resulting from successive integrations averaged by the spectrometer or driver. The duration of individual integrations needs to be adjusted or tuned so that full use is made of the dynamic range and signal to noise headroom of the instrument. The total measured time, can obviously be only as long or longer than a single integration. When spectral irradiance is constant in time, the longer the total time over which scans are averaged the better “random” noise is controlled, up to a point (if the light source emission is constant and bright, a total time longer than 1 s is unlikely to reduce noise any further, however, if the light is dim or varying, longer times of 30 s or more may help average-out the noise.) f is used to enter a fixed value for the integration time, while t and T are used to trigger the automatic tuning of the integration time. t and T only differ in the starting value used for the tuning (t starts from the most recently used value, while T starts from a small arbitrary value).
To increase the dynamic range we can use different integration times for different wavelength-regions of the spectrum (HDR or “bracketing”) and splice the counts per second spectra during their conversion into spectral irradiance. Dynamic range is not the only limiting factor. Stray light and thermal noise limit the signal to noise ratio. Stray light may depend on the shape of the measured spectrum and thermal noise on the temperature of the spectrometer. In normal cases, the achievable increase in dynamic range with the implemented HDR approach is approximately one order of magnitude. h makes it possible to enter HDR multipliers for the integration time. Almost always, one of this values is 1, and others larger than 1 (1 and 10 seem to work best in most cases).
Acquisition parameters include the integration time (= duration of a single integration) which is normally, but not always, adjusted by automatic tuning (i.e., by trial and error, taking into account a target margin of headroom supplied by the user), and a target duration range for the sum of multiple integration times desired, which is always explicitly supplied by the user. For bracketing the user supplies a vector of one to four multipliers to be applied to the optimal or set integration time. All parameters have defaults, and user-entered values persist between successive measurement events unless modified again by the user. r makes it possible to set the target duration range for the sum of integrations, if a single value or two equal values are entered, the total time is fixed. If a range is given, the shortest time multiple of the integration time that fits within the range is used (a range of 0 to Inf, uses the hardware limits of the spectrometer as limits).
In the case of spectral fluence, although available, several of the corrections are in practice difficult to implement because of the discontinuous light emission by the sources. If pulses can be triggered on demand and the output per pulse is reproducible, stray light correction is applicable. Adjustment of signal level and eventually bracketing need to be done based on the number of light pulses or flashes per detector integration.
Note
For additional information, please, see the help page for acq_irrad_interactive() and the vignettes of package ‘ooacquire’, especially that describing the algorithms used.
10 Series parameters
The menu is displayed only with interface mode "series" enabled. Four parameters control the timing for the acquisition of a time series of spectra: wait indicating a delay in starting the series of measurements, boundary indicated by a time unit that must be zero when starting, step giving the time step between spectra in the series, and reps the total number of spectra in the time series.
Above the prompt an estimate of the time required to acquire one spectrum is displayed. Attempts to set the time interval at values shorter than this will be overriden, either by this value or a value of zero. When no bracketing or HDR is used, acquisitions run continuously in the spectrometer, and the step can only be a multiple of the integration time plus a very small overhead dependent on the spectrometer model. In this case, the entered value for step is adjusted to fulfil this requirement. A value of zero for step is interpreted as as short as possible, resulting in spectra acquired as frequently as the spectrometer acquires them.
Above the prompt the current value of each setting is shown. All these settings have defaults that can be changed at the prompt,
wait/boundary/step-len/step-reps/undo/help/GO (w/b/s/r/u/?/g-):
The character string used to set time values must be one interpretable as a period by function lubridate::period(). When a period is expected, bare numbers are not accepte, while 30S, 5M, and 1H, as well 1H30M are all accepted as periods. In the case of boundary, none is equivalent to 0S.
When seconds are expected, bare numbers are interpreted as seconds as in versions of ‘ooacquire’ < 0.4.3, with numbers followed by S also accepted. The number of repeated steps in a time series must be an integer number between 1 and 10000.
11 Plot inspection and tweaks
The prompt displayed after a measurement event, or afters a series of measurement repeats is completed is, when the computed quantity is spectral irradiance or fluence,
fig/photons/energy/w.bands/discard+go/SAVE+GO (f/p/e/w/d/s-):
when the computed quantity is cps (counts per second),
fig/w.bands/discard+go/SAVE+GO (f/w/d/s-):
and when the computed quantity is raw (raw-counts),
discard/SAVE+NEXT (d/s-):
f toggles between displaying plots or not, p and e switch between photon and energy units in the plot, w opens a menu that makes it possible to change the wave bands shown in the plot, d discards the most recently acquired spectrum or time series of spectra (skips saving of files), s saves the data and plot to disk. After f, p, e and w the plot is redrawn and the same prompt redisplayed, making it possible to make additional tweaks to the plot before saving, discarding or continuing. The plot saved to disk is the one current when s is entered. After both d and s, if enabled, a prompt for collections and summaries is displayed, and if disabled, the “loop-end” prompt.
12 Collections and summaries
Multiple spectra can be collected into a single R object and saved to a file at any time during a session. At the same time it is possible to compute and save summaries. The prompt,
collect and/or summarize? yes/NO (y/n-):
makes this possible.
13 Loop-end
The prompt,
quit/repeat-/NEW-MEASUREMENT (q/r/m-):
offers as choices: q to quit or end the session, r to repeat the last measurement, m start a new one. A repeat reuses the same acquisition settings, and reference spectra, dark and/or filter, if they are part the protocol, while the sequence settings can be updated (see callout Repeats above for details).
14 Measuring approaches
14.1 Acquisition of individual spectra
Use cases
Each spectrum is acquired from a different light source or the output of the source varies between spectra or one changes the position or location of the entrance optics relative to the source between acquired spectra. More generally any situation when the integration time needs to be readjusted7tuned or new dark and/or filter reference spectra need to be acquired for each light measurement.
The data for each spectrum are stored as separate R objects and saved to files individually.
In this case, if measuring spectral irradiance, the interface modes "simple" or "auto" are the most convenient. If measuring spectral fluence, the interface mode "manual" has to be used.
If we wish to take a few repeats of a given spectrum, for example to make sure that the acquired spectra are not being distorted by rapid fluctuations in the light source, selecting repeat instead of new-measurement saves some time and effort, as long as we trust that reference spectra are reliable. Most likely, we will select to display of figures and possibly pause between spectra to have time to inspect the individual plots.
14.2 Acquisition of short and/or slow time series
Use cases
Each spectrum is acquired from the same light source and the output of the source varies moderately between spectra. More generally any situation where the integration time does not need to be readjusted and no new dark and/or filter reference spectra need to be acquired for each light measurement. We could be using bracketing or interested in acquiring between 10’s to a few 100’s of spectra at a rather slow pace, such as not more than 1 or 2 spectra per minute.
The data for each spectrum will be stored in separate R objects and saved to separate files as soon as acquired, identically to when acquired individually.
In this case, if measuring spectral irradiance, the interface mode "series" is the most suitable.
When measuring several repeats, the approach is to start by acquiring one light spectrum and the necessary reference spectra (dark and/or filter) and checking that the acquired spectrum is fine. To take repeats of a given spectrum, we select repeat followed by entering the number of repeats. Next, we will normally select without figures or with figures (both non-stop). If there is idle time between repeats, it is good to display figures to be able to check progress. If there is little idle time, say less than 5 s per spectrum on a fast computer, it is better to disable the display of figures.
It is possible to use repeats in modes other than "series", but using this mode makes it possible to precisely time when each repeat is acquired. The trick is to set the number of steps of the time series to one, and a boundary that determines when each acquisition starts. Say with a boundary of "2 min" one spectrum will be acquired every 2 min and the data saved to disk separately for each spectrum.
14.3 Acquisition of long and/or fast time series
Use cases
Each spectrum is acquired from the same light source and the output of the source varies moderately between spectra. More generally any situation when the integration time does not need to be readjusted and no new dark and/or filter reference spectra need to be acquired for each light measurement. We could be using bracketing or not, interested in acquiring spectra as fast as possible or maybe a moderately fast sequence of a few 1000s of spectra. In the case of very fast acquisition the computations and saving of data to disk have to be done after the whole series has been acquired, to reduce the overhead. In the case of long time series creating a separate R object and file for each spectrum can become wasteful and inconvenient.
The data for the whole time series of spectra is stored in a single R object and saved to the same files, one file for the data and one file for the plot if enabled. The number of spectra per plot is by default limited to a maximum of 11, and random sampling is used if the series contains more spectra.
In this case, the interface mode "series" is used. As described above, it is best to start by acquiring a very short time series and then selecting repeat and increasing the number of steps before triggering the start of the repeat.
Spectrometers and OmniDriver have different modes. The high speed mode is used automatically when no bracketing is used and the time per step is short or zero. In this mode the spectrometer acquires spectra non-stop in parallel with the transfer to the computer. There is minimal overhead with the Maya 2000 Pro of less than 1 ms per spectrum. This means than with an integration time of 10 ms nearly 100 spectra can be acquired per second, or 6000 spectra in 1 min. When not downloading spectra from the spectrometer this frequently, still the step between acquired spectra needs to be a multiple of the integration time so that the real time step matches the setting. This is enforced by adjusting the value entered by the user. If a different value would be accepted the spectra would be acquired at the same rate as with the adjusted value but from time to time an acquired spectrum would not the transferred to the computer.
When using bracketing the overhead is much more as the spectrometer settings need to be changed twice for each spectrum, adding an overhead that depends on the integration time. An estimate of the time needed to acquire one spectrum is displayed and step times shorter than this are not accepted as settings. In the case of high speed acquisition the time of measurement is retrieved from the actual spectrometer and is according to specifications within a \(\mu s\) of the true real time. When bracketing is used, different parts of the same spectrum are acquired at different times, using longer time steps and the time reported (and the only one available) is the time when the "light" spectrum is retrieved from the spectrometer (that can be at most later than the real time of measuremet as much as the integration time.) When using HDR, the time of retrieval of the spectrum with the shortest integration time, always measured first, is reported together with physical quantities.
Tip
When acquiring long time series of spectra, if gaps are tolerable, it is a good idea to combine time series with repeats, so that the data are saved to files in smaller batches, decreasing the risk of data loss.
Another use case when repeated time-series of spectra can be useful is when fast variation is of interest but continuous acquisition over a long time would generate excessive data. For example one may want to measure a very fast time series once every few minutes or once per hour.
Warning
One limitation to the acquisition of spectra over a long time is the drift of the spectrometer dark signal dependent on temperature and the variation in stray light as a consequence changes in the light source.
For the dark signal the best solution is to control the temperature of the spectrometer so that it remains constant. Stray light is trickier to control or compensate for. It depends on the combination of light source spectrum and spectrometer how problematic stray light is. The UV region is the most difficult in sunlight. When measuring LEDs emitting white light with a spectrometer that can measure in the UV region, a protocol without any dark or filter spectra used as reference can be satisfactory, because the algorithm uses the reading in the UV-C as an internal “dark” reference. This works better than using the very few true dark pixels built into the detector array of some spectrometers.
To get reliable measurements over hours of unattended acquisitions tests are needed as well as most likely also a thermostatic system.
Note
The new features I have added in the last year make possible many different approaches to the acquisition of spectra. I am sure some of the new features will be found more useful than others. In any case, I am finding some of the features convenient for uses I had not thought of before implementing them.
I would like to hear from users how they are using ‘ooacquire’.
15 Resources
The documentation of the packages includes installation instructions.