 |
The SKIRT project
advanced radiative transfer for astrophysics
|
|
The SKIRT project
advanced radiative transfer for astrophysics
|
#include <StoredTable.hpp>
Public Member Functions | |
| StoredTable () | |
| StoredTable (const SimulationItem *item, string filename, string axes, string quantity, bool clampFirstAxis=true, bool resource=true) | |
| ~StoredTable () | |
| template<typename... Values, typename = std::enable_if_t<CompileTimeUtils::isFloatArgList<N - 1, Values...>()>> | |
| double | cdf (Array &xv, Array &pv, Array &Pv, Range xrange, Values... values) const |
| void | open (const SimulationItem *item, string filename, string axes, string quantity, bool clampFirstAxis=true, bool resource=true) |
| template<typename... Values, typename = std::enable_if_t<CompileTimeUtils::isFloatArgList<N, Values...>()>> | |
| double | operator() (Values... values) const |
| template<typename Value , typename = std::enable_if_t<N == 1 && CompileTimeUtils::isFloatArgList<1, Value>()>> | |
| double | operator[] (Value value) const |
An instance of the StoredTable<N> class template provides access to the contents of a particular resource file in the SKIRT stored table format (i.e. a "stored table").
A stored table includes the names of the quantities on the axes (e.g. wavelength and grain size) and those being tabulated (e.g. absorption and scattering efficiencies), in addition to the grid points for each axis and the tabulated data values. All grid points and values are stored as binary data in the form of 64-bit floating-point numbers, and are always given in SI units. The format is designed so that it is easy to calculate the offset, relative to the start of the file, to any particular data value. More specifically, a stored table file is essentially a sequence of 8-byte data items. A data item can have one of three types:
The overall layout is as follows:
The values are ordered so that the quantity values for a particular point are next to each other, the first axis index varies most rapidly, and the last axis index varies least rapidly.
The StoredTable<N> template parameter N specifies the number of axes in the stored table, and thus the number of axis values needed to retrieve a tabulated quantity from the table. Each StoredTable<N> instance represents a single tabulated quantity. Accessing the full contents of a stored table resource file with multiple tabulated quantities requires a seperate StoredTable<N> instance for each of those quantities.
The default constructor creates an invalid stored table instance. The alternate constructor and the open() function associate a particular stored table resource file with the stored table instance. The number of axes in this stored table resource file must match the template parameter N. Also, the axis names and the corresponding units in the file, and one of the tabulated quantity names and its corresponding unit in the file, must match the information passed to the alternate constructor or the open() function. The destructor automatically releases the file association and any related resources.
The parenthesis operator returns the quantity represented by the StoredTable<N> for the N specified axis values, interpolated from the tabulated values. Other functions offer access for specific purposes, such as constructing a cumulative distribution function along one axis, given values for the other axes.
A StoredTable<N> instance acquires a read-only memory map on the associated stored table resource file as opposed to actually reading the file contents into memory through regular file I/O operations. This has some important, mostly positive, consequences.
Acquiring a memory map establishes a mapping between "pages" of system-defined size in the logical address space of a process and the contents of the "backing file", in this case the stored table resource file. This operation is simple and thus very fast. From then on, the operating system automatically loads pages from the backing file into physical memory as they become needed because the program addresses an item in the logical memory range of the page. Conversely, the operating system automatically removes pages from physical memory if available memory becomes tight. In effect, the operating system automatically manages a high-performance caching mechanism on stored tables.
Three important use cases benefit greatly from this mechanism. Firstly, a large resource file can be left associated with a StoredTable<N> instance for the duration of the program, even if it is used only sporadically. When memory is tight, infrequently used portions of the data will automatically be removed from memory and reloaded later if needed. Secondly, there is little overhead in constructing a StoredTable<N> instance (and possibly destroying it shortly thereafter) even when the program needs only a small portion of the file contents. And thirdly, because all StoredTable<N> instances associated with a given stored table resource file share the same memory map on that file, using a seperate instance for each quantity in the stored table incurs very little overhead.
Moreover, most operating systems share memory maps between processes. For a parallel program using MPI, this means that all processes running on the same compute node share a single memory copy of the resources they employ. Also, most operating systems keep the memory map caches alive between consecutive invocations of a program (assuming memory is available), increasing performance when, for example, interactively testing the program.
On the downside, a program requesting a huge chunk of data from a large stored table in a serial fashion would run faster using regular file I/O, because the separate page loads take more time than sequentially reading data in bulk. More importantly, performance usually degrades rapidly (to the point where the program no longer performs any useful work) when the system is forced to constantly remove and reload pages because there is not enough memory to hold the data needed for a particular phase in the program. And finally, the run-time performance of a program becomes somewhat unpredicable because the speed of accessing resources depends heavily on the previous state of the operating system caches.
|
inline |
|
inline |
|
inline |
The destructor breaks the association with a stored table resource file established by the alternate constructor or the open() function, if there is any. In practice, this simply means releasing the memory map on the associated file.
|
inline |
This function constructs both the normalized probability density function (pdf) and the corresponding normalized cumulative distribution function (cdf) for the tabulated quantity across a given range in the first axis (the xrange argument), using given fixed values for the other axes, if any (the arguments at the end of the list).
The resulting first-axis grid is constructed into xv, the corresponding pdf into pv, and the corresponding cdf into Pv. In all cases, xv[0]=xmin, xv[n]=xmax, Pv[0]=0, and Pv[n]=1. The function returns the normalization factor, i.e. the value of Pv[n] before normalization. This corresponds to the result of integrating the tabulated quantity over the given range.
If any of the axes values, including the xrange values specifying the range for the first axis, are out of range of the internal grid, extra quantity values are fabricated according to the policy set by the clampFirstAxis flag in the open() function.
If the flags specified in the stored table indicate that both the first axis and the quantity represented by the table should be interpolated logarithmically, it is assumed that the pdf behaves as a power-law between any two grid points, and the integration to determine the cdf is performed accordingly. In all other cases, piece-wise linear behavior is assumed and regular trapezium-rule integration is used.
|
inline |
This function associates a given stored table resource or input file with the stored table instance. If such an association already exists, this function throws a fatal error. Conversely, calling any of the other functions before an association exists results in undefined behavior (usually a crash).
The item argument specifies a simulation item in the hierarchy of the caller (usually the caller itself) used to retrieve an appropriate logger.
If the resource flag is true (the default value), the filename argument specifies the filename of the built-in resource, without any directory segments. If the resource flag is false, the filename argument specifies the file path of the stored table input file relative to the input path of the simulation. In both cases, the file must have the ".stab" filename extension, which will be added to the specified filename if needed.
The filename argument specifies the filename of the resource, without any directory segments. The resource file must have the ".stab" filename extension, which will be added to the specified filename if needed.
First of all, the number of axes in this stored table resource file must match the template parameter N. Furthermore, the axes names in the resource file and the corresponding units for each axis must match the information specified in the axes argument. Finally, one of the tabulated quantity names in the resource file and its corresponding unit must match the information specified in the quantity argument. For a stored table resource file with multiple tabulated quantities, the quantity argument at the same time determines which of these quantities will be associated with the stored table instance.
The string passed to the axes argument must have the syntax "name1(unit1),...,nameN(unitN)". In other words, a unit string between parenthesis follows each axis name, and the specifications for different axes are separated by a comma. For example, "lambda(m),a(m)". Whitespace is not allowed. The string passed to the quantity argument must have a similar syntax, for a single name/unit combination. Examples include "Llambda(W/m)", "Qabs(1)", and "h(J/m3)".
By default, out-of-range axes values are clamped to the corresponding outer grid point. In other words, a "nearby" quantity value at some outer grid point is used for out-of-range axes values. This behavior can be turned off for the first axis (but not for the other axes) by passing a value of false to the optional clampFirstAxis flag. In that case, the quantity value for out-of-range first-axis values is considered to be zero.
In summary, this function (1) locates the specified stored table resource file, (2) acquires a memory map on the file, (3) verifies that the stored table matches all requirements, and (4) stores relevant information in data members. If any of these steps fail, the function throws a fatal error.
|
inline |
This function returns the value of the quantity represented by this stored table for the specified axes values, interpolated over the grid points of the actual tabulated values in all dimensions. The function uses linear or logarithmic interpolation for the axes and quantity values according to the flags specified in the stored table. Out-of-range axes values are handled according to the policy set by the clampFirstAxis flag in the open() function.
|
inline |
For a one-dimensional table only, this function returns the value of the quantity represented by the stored table for the specified axes value, interpolated over the grid points of the actual tabulated values. The function uses linear or logarithmic interpolation for the axis and quantity values according to the flags specified in the stored table. Out-of-range axes values are handled according to the policy set by the clampFirstAxis flag in the open() function.