Table 2 Description of the WV-CKDs. WV-CKD is a series of lookup tables for the CKD method in the water vapor absorption region of 940 nm at 2-, 5-, and 10-\({\text{cm}}^{ - 1}\) intervals

Resolution

\(2 \;{\text{cm}}^{ - 1}\)

\(5 \;{\text{cm}}^{ - 1}\)

\(10 \;{\text{cm}}^{ - 1}\)

\(0.01 \;{\text{cm}}^{ - 1}\)

Maximum error of \(E\left( {\mu ,\lambda } \right)\): \(\left( {\varepsilon_{E}^{2} } \right)^{1/2}\)

\(11.2 \times 10^{ - 3}\)

\(7.1 \times 10^{ - 3}\)

\(5.0 \times 10^{ - 3}\)

–

Median quadrature numbers (10,000–10,900 \({\text{cm}}^{ - 1}\))

4

6

7

–

Median quadrature numbers (10,410–10,870 \({\text{cm}}^{ - 1}\))

5

6

8

–

Mean computation times (10,000–10,900 \({\text{cm}}^{ - 1}\))

2.18/\({\text{cm}}^{ - 1}\)

1.18/\({\text{cm}}^{ - 1}\)

0.734/\({\text{cm}}^{ - 1}\)

100/\({\text{cm}}^{ - 1}\)

Mean computation times (10,410–10,870 \({\text{cm}}^{ - 1}\))

2.44/\({\text{cm}}^{ - 1}\)

1.33/\({\text{cm}}^{ - 1}\)

0.848/\({\text{cm}}^{ - 1}\)

100/\({\text{cm}}^{ - 1}\)

\(D{\Phi }\) of Eq. 16 (FWHM = 5 nm)

0.176

0.280

0.394

–

\(D{\Phi }\) of Eq. 16 (FWHM = 10 nm)

0.124

0.198

0.278

–

\(D{\Phi }\) of Eq. 16 (FWHM = 15 nm)

0.102

0.162

0.228

–