SEARCH WITHIN CONTENT
Citation Information : International Journal on Smart Sensing and Intelligent Systems. Volume 13, Issue 1, Pages 1-10, DOI: https://doi.org/10.21307/ijssis-2020-009
License : (BY-NC-ND-4.0)
Received Date : 29-January-2020 / Published Online: 29-May-2020
Food quality monitoring in the production process is essential. The control of food quality and freshness is of growing interest for both consumer and food industry. Near infrared (NIR) spectroscopy is popular as it does not need any sample preparation. However, NIR spectroscopy is costly and needs reliable calibration. A non-contact, non-destructive optical process is proposed in this work to monitor the quality of the food. It is shown that the reﬂected phase information can be used to detect the quality of the fruits. The color and the spectral reﬂectance change with storage. The changes in the spectral feature due to ripening or decay of apples are used to non-destructively monitor the quality of the fruit. A closed relationship between the reﬂected phase information and degradation is obtained. The developed model is simple, low cost, and does not need extensive calibration as compared to conventional technologies currently used like NIR besides being robust to skin color or appearances of the fruit. The phase-based reﬂectance spectroscopy could revolutionize the on/inline quality monitoring of the fruits.
The food quality monitoring in the production process is essential. The quality of the food depends on many parameters which are categorized as internal and external parameters. External parameters include the color, shape, surface, hardness, softness, odor, taste, etc., whereas internal parameters include sugar, water, acidity, internal defects, etc. The processing and storage requirements of agro-based products are determined by assessing the initial quality of the fruit or the vegetables. Many industries are ﬁnding better ways to assess the quality of the food. The techniques used to monitor the food quality are classiﬁed into destructive and non-destructive methods. Industry procedures like chromatography, spectrophotometry, electrophoresis, and titration are examples of destructive techniques. Destructive methods do not allow continuous monitoring besides being expensive, slow, and time consuming. The major limitation of the destructive method is that the sample taken for testing must represent the batch accurately, else it results in a huge wastage of the food. Therefore, it is a cost inefficient solution.
A rapid and affordable method is desired by the food industry to monitor the quality of the food. In non-destructive approach, the information available from the outside of the product is correlated to the quality or freshness of the product. In non-destructive methods, the samples are not destroyed. In the study of Concina et al. (2012), the chemical proﬁle of the fruit is used to monitor the freshness using electronic noses. In the study of Shmulevich et al. (2003), accoustic measurements are used to measure the tissue elasticity which relates to the softening of the apples during storage. In non-destructive optical technique, light spectrum in the range of 400 to 1000 nm is used to determine the response of the fruit to an incident light. The response can be in the form of refraction, reﬂection, absorption, or scattering. An optical technique allows us to successive measurements of a sample and is very popular due to its ease in implementation.
Among the non-destructive techniques, for monitoring of the internal quality of fruits, near infrared (NIR) spectroscopy is widely used (Nicolaï et al., 2007). The NIR spectroscopy is now commonly used for the assessment of fresh fruits by monitoring the internal quality. NIR radiations cover the electromagnetic spectrum between 780 and 2500 nm. Most agro products behave as semi-transparent or turbid for these wavelengths, and therefore, the light passes through the agro-products and interacts with the internal structures (Askoura et al., 2016). The incident radiation may be either absorbed, transmitted, or reﬂected. The transmitted and reﬂected light is collected using a detector and the received information is then processed for quality estimation. The non-invasive and nondestructive nature of the NIR spectroscopy allows its use in industrial application, where continuous quality and process monitoring is desired. Furthermore, NIR spectroscopy allows several constituents to be measured simultaneously. However, NIR spectroscopy is limited to fruits and vegetables with homogeneous pulp (Oliveira et al., 2014). The penetration depth of the NIR spectroscopy is inadequate for internal quality monitoring of various fruits and vegetables. The transmitted or reﬂected NIR spectrum of a fruit or a vegetable has peaks of absorption band; therefore, statistical processing needs to be done for extracting the information on the internal quality of the fruit. This increases the processing complexity of the NIR spectroscopy.
In the study of Qin and Lu (2008), a novel spatially resolved hyperspectral diffuse reﬂectance imaging system is proposed to obtain the optical properties of fruits and vegetables. Hyperspectral imaging or multispectral imaging involves image formation using spectral information from a band of wavelengths transmitted or reﬂected from the same region of an object. Hyperspectral imaging using three-dimensional data cubes is used to analyze the quality of the agro-based products. Hyperspectral imaging uses images from the object, and therefore, it is non-contact and non-destructive. Furthermore, the process of image capture and analysis is relatively simple. However, the computational complexity in creating the data cube is very high and the requirement of specialized camera systems increases the cost. Furthermore, the shape and quality of the skin layer of the fruit affects the scattering, introducing distortion in the measurements. In the study of Cubeddu et al. (2001), a time-resolved approach is used to predict the ﬁrmness of fruits and vegetables. Time-resolved spectroscopy relies on the time it takes to penetrate through the fruit. The internal composition of the fruit will affect the scattering, and therefore, the time the incident pulse takes to transmit through the pulp of the fruit can be correlated to its quality. In the study of Cubeddu et al. (2001), a pulsed laser in the wavelength range of 650 to 1000 nm is injected into the fruit. The injected pulse transmits through the pulp of the fruit. The temporal delay and broadening of the pulse is measured at the other end. This technique needs two optical ﬁcube, one for the injection of the laser light and the other for the collection of the transmitted laser light. The collected light is then analyzed and correlated to the internal pulp quality. The experimental set-up for time-resolved spectroscopy is complex. Furthermore, the correlation between the received pulse and the internal pulp quality is weak.
For automatic inspection of fruits and vegetables, artiﬁcial vision systems are becoming popular (Du and Sun, 2006; Cubero et al., 2011). The computer vision algorithms are used in grading, quality estimation using external features and also monitoring of fruits during storage. The computer vision-based grading systems allow a large number of fruits to be inspected in short time with minimal human intervention and are more reliable than human inspection. Computer vision-based system relies on the quality of the captured images. The quality of the captured images in turn largely depends on the speciﬁcations of the camera being used and the illumination condition on the object. The large variations in the color, size, shape, and features in one type of fruit make the computer vision-based sorting challenging and often results in errors. Furthermore, the color of the skin of a fresh fruit and blemish on the skin of another fruit may be the same. The variations in the performance of camera and illumination conditions used severely affect the quality of the image obtained and, therefore, limit the performance of the computer-based system. For every batch of the fruit, different calibrations on the processing algorithms are required which make computer vision-based systems slow. Image processing-based algorithms always have a trade-off between speed and accuracy.
Optical techniques using reﬂectance/transmittance suffer from saturation of the detector due to high reﬂectance, variations in skin color and texture and distortions due to the variation in the shape of the reﬂecting object (Yousaf and Qin, 2014). In the study of Sarkar et al. (2019), Stokes degree of linear polarization is used to measure the internal quality of the fruits. The Stokes based polarization information is complex in computation. In this paper, a real-time low-complex optical method is proposed for the assessment of the internal quality of fruits. The proposed method relies on the reﬂection rather than transmission of the incident light. The incident light is linearly polarized and the reﬂected phase information is monitored which is correlated to the quality of the fruit. The proposed method is non-destructive and is not computationally intensive. To demonstrate the proposed reﬂectance-based phase spectroscopy, apples are used as test case. As the apples are stored, they degrade, changing the internal chemical composition. The changing chemical composition changes the scattering of the linearly incident light wave. Thus, the reﬂected phase information is shown to be dependent on the decay rate of the apples. The changes in the reﬂected phase information can be qualitatively used to predict the quality of the fruit. The use of phase information of the reﬂected light makes it independent of the color of the skin of the fruit, increasing the accuracy of the quality prediction. The rest of the paper is organized as follows: the second section presents the theory behind phase-based reﬂection and its use in monitoring the quality of the apples. The third section describes the measurement setup and the correlation between the quality of the apples and the measured reﬂected phase is derived. The paper is concluded in the fourth section.
The color and spectral reﬂectance changes are often used to monitor or sort fruits in commercial applications. The fruit color and appearance is primarily due to the proportional content of chlorophylls, cartoneoids, and anthocyanins (Merzlyak et al., 2003; Merzlyak, 2006). The red color of the apples is due to the pigment anthocyanins. The pigment content varies in different types of apples. For example, Red Star has higher anthocyanins as compared to Golden Delicious. Golden Delicious, on the contrary, has higher chlorophyll content compared to Red Star. The anthocyanins and chlorophyll content in the Red Star and Golden Delicious changes as the apples ripen. In the optical technique, the changes in the reﬂectance due to changing pigment content is monitored to determine the quality of the fruit (Chuma et al., 1977; Merzlyak, 2006). Reﬂectance spectroscopy depends on the amount of light reﬂected by the object and, therefore, also depends on the nature of the reﬂecting surface. The color, texture, and the composition of the object, thus, inﬂuence the reﬂected light. Furthermore, the changes in the internal pigmentation of the fruits change the absorption maxima, and therefore, the reﬂected light is also wavelength sensitive.
In this section, the reﬂectance properties of apple are described. Furthermore, the advantages of using phase to monitor the scattered components are elaborated.
Light transmission or reﬂection from the apple is dependent on the reﬂectance, absorption, and scattering of the region on which the light is incident. Absorption is primarily due to the pigments, carotenoids, chlorophylls, water, fats, and proteins (Cena et al., 2013). The change in the refractive index inside the apple due to membranes, air vacuoles, and changes in the pigment contents deviates the light path and, therefore, causes scattering (Sarkar et al., 2019). In most fruits and vegetables, scattering is dominant over absorption. The light distribution is highly scattered in apples.
The total reﬂected light from the apple which is a scattering medium consists of a specular component and a diffuse component. The specular reﬂection from the surface is only 3 to 5% of the incident light (Baranyai and Zude,, 2009). Therefore, the major component of the reﬂected light is diffused. The diffused reﬂectance R d(x) is expressed as (Hu et al., 2016):is the transport coefficient and is the effective attenuation coefficient. µa is the absorption coefficient and is the reduced scattering coefficient (Hu et al., 2016). The diffused reﬂectance is, thus, determined by the scattering observed by the incident light. Apple contains a bulk of their pigments in the skin above a thick layer of parenchyma which exhibits strong light scattering properties (Law and Norris, 1973). The scattering generally decreases progressively with increasing wavelength in the range of 600 to 1000 nm (Mourant et al., 1997).
For an incident polarized light, the specularly reﬂected light retains the polarization state of the incident light (Mahendru and Sarkar, 2012). The scattering of the incident light which results in diffuse reﬂection changes the polarization state of the reﬂected light. The diffused reﬂected light is partially polarized (Sarkar et al., 2011; Mishra et al., 2015). A partially polarized light is characterized by the amount of polarized component in an unpolarized light. The polarization state of the incident light is preserved in weakly scattered light as compared to multiple scattered light. The Stokes vector of an incident photon that has scattered n times depends on the energy that remains after n times scattering and is given by:
The Stokes vector which corresponds to the energy of the electric ﬁeld in the scattered light is, thus, dependent on the scattering proﬁle. The scattering of an incident light by an apple depends on the changes in the pigment content. Therefore, a linearly polarized light on reﬂection will be partially polarized. The partially polarized light is characterized by degree of polarization. The degree of polarization states the polarization component present in a partially polarized light. The changes in the scattering proﬁle by an apple due to change in a pigment content due to ripening and storage changes the degree of linear polarization.
The degree of linear polarization is used to measure the reﬂected partially polarized light (Sarkar and Theuwissen, 2013):
In comparison to the NIR spectroscopy and the computer vision-based algorithms for fruit quality monitoring, the use of phase in reﬂectance spectroscopy has the follow advantages:
The phase-based measurements only require a camera with polarization ﬁlters. Therefore, the instrumentation is less complex compared to NIR spectrocopy.
The processing complexity is simple as only a ratio of phase information is required compared to computer vision based which are calibration intensive.
The phase-based measurements are weakly dependent on the illumination condition of the object as it is ratio based. The NIR spectroscopy and the computer vision-based systems are highly inﬂuenced by the ambient illumination of the object.
The phase-based measurement are independent of the skin color of the apples. NIR spectroscopy and computer vision-based systems are highly dependent on the skin color, shape, and size of the object.
The block diagram of the optical setup used to monitor the quality of the fruit is shown in Figure 1. For the experiments, standard CMOS/CCD camera, two polarization ﬁlters, a monochromatic light source and apple (Red Star and Golden Delicious) as fruit object are used. One polarization ﬁlter is used across the light source (optional), whereas the other is used across the camera. The camera used is a monochromatic Basler camera (acA640 to 90 um, Sony ICX424 mono CCD sensor) with a resolution of 659 × 494 pixels. A DC ambient white light source (make: Holmarc) with maximum intensity of 1300 lux is used. The environmental lighting condition is ambient. The DC white light generated is unpolarized in nature. The unpolarized light is passed through a linear polarization ﬁlter (Polarizer 1) to linearly polarize the light which is then incident on the sample apple. The apples reﬂect the incident light while changing the phase of the incident light. The reﬂected light from the apple is, therefore, partially polarized. This partially polarized light is transmitted through another linear polarization ﬁlter (Polarizer 2) before being detected by the camera. The transmission axis of the polarizer 1 and 2 is varied manually to 0°, 45°, and 90° for different measurement conditions.
A sample measurement of the apples is shown in Figure 2. The polarized images are obtained by changing the transmission axis of the linear polarizer. Figure 2A is a test image without the linear polarizer ﬁlter, which is used a reference image. The images shown in Figure 2B have both polarizer 1 and 2 at 0° transmission angle. The received light intensity when both polarizer 1 and 2 at 0° transmission angle is represented as I(0, 0). In Figure 2C, the transmission axis of polarizer 1 and 2 are at 0° and 45°, respectively. The received image is represented as I(0, 45). Figure 2D represented as I(0, 90) is obtained when the transmission axis of polarizer 1 and 2 is at 0° and 90° respectively. For each measurement, a set of 100 images per observation is taken with and without a linear polarizer. An ROI of 140×140 pixel is used to obtain the average received light intensity to improve the signal-to-noise ratio (SNR). The pixel-to-pixel variations occur due to the variations in the reﬂected light reaching the camera through the linear polarizer. As per the Malus law, the light reaching the sensor is a function of the transmission axis which is evident from the images shown in Figure 2.
The reﬂectance and the polarization state of the reﬂected light will depend on the pigment content as discussed in the second section. The quality of the apples is expected to decay over time. The decaying or ripening of the apple changes the pigment content which changes the absorption and scattering characteristics of the apples. To monitor the scattering characteristics, the measurements are repeated with the same apples at different time instances over a period of two week. Figure 2 also shows the measurement results of I(0, 0), I(0, 45), and I(0, 90) for Red Star and Golden Delicious apples at day 12. The varying absorption and scattering characteristics inﬂuence the diffuse reﬂection from the apples and, thus, changes the polarization state of the reﬂected light. The varying polarization state is clearly observed in Figure 2.
Initially, the measurements are performed with single polarizer in the light transmission path. The polarizer is placed in front of the camera in Figure 1. Unpolarized light from the DC light source is incident on the apple. The pigment contents in the apple change the phase information of the reﬂected light and, therefore, partially polarizes the unpolarized incident light. The reﬂected partially polarized light is transmitted through a linear polarizer whose transmission axis is varied manually. Figure 3 shows the variation of normalized received polarization intensity with varying observation number when the transmission axis of the linear polarizer is varied by 0°, 45°, and 90° for red light and white light, respectively. Two observations are clearly made from the ﬁgure:
The received normalized polarization intensity decreases with increase in the observation number. The drop is very sharp for Red Star in the presence of red light as compared to when white light is used.
The normalized received polarization intensity is higher when white light is used as compared to when red light is used.
The differential absorption of red light by chlorophyll and anthocyanin inﬂuences the spectral reﬂectance. Therefore, the reﬂected light from an apple depends on the wavelength of the incident light. For cholorophyll, the absorption coefficient is high for light with wavelength at 675 nm. Therefore, the spectral reﬂectance of an incident light by an apples depends on the wavelength of light used as well as the storage or ripening period. The stronger reﬂection of incident white light explains the higher normalized polarization intensity as compared to red light. In case of red light, the higher absorption reduces the scattering, and therefore, the received polarization intensity is lower compared to white light.
For a less mature fruit, the chlorophyll absorption is high. With the ripening or storage, the absorption by the chlorophyll decreases, whereas the scattering increases. The reﬂected light is, therefore, more unpolarized which decreases the transmittance through a linear polarizer. This decrease is sharp when red light is used as compared to white light. The other wavelength components present in the white light are not scattered much, and therefore, the transmitted intensity is higher though the linear polarizer. The degree of linear polarization (DoLP) as expressed in Equation (3) is shown in Figure 4.
The degree of linear polarization shows that for both red and white lights, the DoLP increases with increase in an observation number. The variation in the scattering proﬁle with increase in storage time changes the DoLP, and therefore, its variation is directly correlated to the quality of the fruit. For a fresh apple, the DoLP in the presence of both white and red lights is low. As the apples degrades, the DoLP increases over time. The R-squared for red light is 0.2232, whereas for white light it is 0.5458. The change in the DoLP can be used to predict the characteristic changes in the internal quality of the apples. In the study of Sarkar et al. (2019), the DoLP was computed using two polarizers one in front of the light source, whereas the other was used in front of the detector. In the proposed, method, it is demonstrated that the changes in the DoLP for changing pigment content in the apple can be obtained using a single polarizer. The correlation between the changes in the pigment content and changes in the DoLP as measured by the R-squared coefficient is higher when a single polarizer is used as compared to (Sarkar et al., 2019). The use of single polarizer further simpliﬁes the instrumentation and, thus, can be realized for real-time measurements.
In this section, the measurements of the reﬂected light are repeated using two polarizers as in the study of Sarkar et al. (2019). Polarizer 1 is used to polarize the DC white light. This polarized light is incident on the apple which partially polarizes the light. The partially polarized light is then transmitted through polarizer 2 and detected by the detector. In the study of Sarkar et al. (2019), the DoLP was calculated using Stokes parameters which increases the computational complexity as it needs addition, subtraction to be performed on the received data besides calculating the ratios. The real-time implementation would, therefore, be difficult if the direct algorithm of Sarkar et al., (2019) is used. Thus, the algorithm of Sarkar et al., (2019) is slightly modiﬁed here to make it real-time implementable. The DoLP computation in the proposed method uses I(0, 45) information rather than conventional I(0, 90). The incident light is, therefore, 0° polarized, whereas the DoLP is computed based on the received intensity by the detector when the transmission axis of the polarizer in front of it is 0° and 45° . The received intensity proﬁle is as shown in Figure 5.
The Red Star with low absorption has higher reﬂectance compared to Golden Delicious which has higher absorption. The larger spectral reﬂectance helps in retaining the incident polarization phase in Red Star, whereas higher absorption in the case of Golden as compared to Golden Delicious, and therefore, the reﬂected light is more likely to retain the phase information of the incident light. The 45° ﬁlter rejects the reﬂected 0° polarized light. The multiple scattering in Golden Delicious, unpolarize the reﬂected light, and therefore, the transmitted light through the 45° ﬁlter is higher. I(0, 45) also increases with increase in an observation number. The differential variation in I(0, 45) is much higher than I(0, 90) reported in the study of Sarkar et al. (2019). The increased differential variation would improve the DoLP which is now computed using the modiﬁed equation as Delicious causes multiple scattering. The multiple scattering results in diffuse reﬂection which looses the phase information of the incident light. The lower content of chlorophyll in Red Star causes less absorption, and therefore, for an incident 0° polarized light, the received reﬂected light at the detector with 0° transmission axis of the linear ﬁlter in front of it is higher for Red Star as compared to Golden Delicious. I(0, 0), measured with both polarizer 1 and 2 transmission axis set at 0° for Red Star and Golden Delicious is shown in Figure 5A. It is further observed that I(0, 0) increases with increase in the observation number I(0, 45) measured with the transmission axis of 45° for polarizer 2 in front of the detector, whereas the incident light is still 0° polarized, as shown in Figure 5B. The higher reﬂectance in Red Star retains the incident phase, and therefore, the measured I(0, 45) is lower for Red Star as compared to Golden Delicious. The scattering in Red Star is weak as compared to Golden Delicious, and therefore, the reﬂected light is more likely to retain the phase information of the incident light. The 45° ﬁlter rejects the reﬂected 0° polarized light. The multiple scattering in Golden Delicious unpolarizes the reﬂected light, and therefore, the transmitted light through the 45° ﬁlter is higher. I(0, 45) also increases with increase in the observation number. The differential variation in I(0, 45) is much higher than I(0, 90) reported in the study of Sarkar et al. (2019). The increased differential variation would improve the DoLP which is now computed using modiﬁed equation as:Figure 6 for Red Star and Golden Delicious, respectively, when white light source is used. The following observations can be made:
The DoLP for Golden Delicious is lower compared to Red Star.
The DoLP decreases with increase in the observation number.
In the presence of white light, the absorption of the chlorophyll dominates, reducing the scattering component. The reduced scattering component increases I(0, 0). The Golden Delicious having a higher content of chlorophyll compared to Red Star has higher absorption than scattering. Therefore, the DoLP is lower in Golden Delicious compared to Red Star due to reduced I(0, 45). The DoLP for both Red Star and Golden Delicious apples decreases with increase in time. The increase in the storage time of the apples changes the pigment content. As the apple decays, the scattering is lowered, and therefore, the polarized component in the partially polarized light decreases, decreasing the DoLP. Thus, scattering from the pigments inﬂuences the degree of linear polarization (DoLP).
The change in the DoLP for Red Star is lower as compared to change in the DoLP of golden delicious. For the Red Star, the change in the DoLP is 4%, whereas for Golden Delicious it is 12.5% over the 14 day period of storage. Furthermore, the R-squared measures are 0.7087 and 0.3085 for Red Star and Golden Delicious, respectively. The R-squared values obtained are higher than those reported in the study of Sarkar et al. (2019). Therefore, we can conclude that it is better to use I(0, 45) for DoLP computation rather than I(0, 90).
As compared to the degree of linear polarization presented in the study of Sarkar et al. (2019), the DoLP presented here is easy to compute and is shown to reﬂect the changes in the pigmentation inside an apple due to storage or ripening. This optical method is non-destructive and non-invasive. Compared to other non-invasive and non-destructive optical methods, the proposed method is independent of the color of the skin of an apple or its shininess as the normalized phase information is used for the computation of the DoLP. Furthermore, the reﬂected phase information is independent on the shape and size of the apple used, and thus, the variations in the shape of an apple do not introduce distortions in the measurements.
To summarize DoLP has the following advantages in food quality monitoring:
In comparison to the DoP calculations proposed in the study of Sarkar et al. (2019), the proposed DoLP calculations are relatively simple and, therefore, are less computationally complex. Furthermore, the instrumentation required is simple.
The DoLP-based correlation to the internal quality of the fruit is independent of the color of the skin of an apple. Furthermore, the size and shape of the apples also does not inﬂuence the measurement. The shininess of the apples does not saturate the detector and, thus, also does not inﬂuence the measurements. The de-coloration of the fruit skin due to damage do not introduce distortion in the measurements.
The DoLP-based quality monitoring is, therefore, a true non-destructive, noninvasive real-time fruit monitoring technique. Clearly many technical aspects still need to be solved before an industrial application can take place. However, in comparison to the existing food quality monitoring methods, the proposed solution is easy to implement and low cost for real-time non-destructive online monitoring of the quality of the food.
The assessment of internal quality of fruits is desirable for grading and monitoring of internal quality of fruits in food industry. The techniques available for quality monitoring can be broadly classiﬁed into destructive and non-destructive methods. In destructive methods, the food sample is destroyed to assess the quality, and therefore, it is not preferred. Non-destructive techniques do not need sample preparation and, therefore, are ideal for in-line real-time monitoring of the quality of the fruits. In this paper, an optical-based technique is proposed which uses the phase of the reﬂected light from the object to assess the quality of the fruit. In this work, a phase-based nondestructive and non-invasive measurement of optical properties in fruits is proposed. In general, with other optical methods, it is difficult to distinguish the absorption and scattering components. The scattering component is detected using the measurement of the reﬂected phase information. The degree of linear polarization is shown to vary with decaying of the apples. Monitoring the degree of linear polarization, thus, gives an indication of the quality of the fruit. Furthermore, this technique is independent of the color or shininess of the skin and the shape of the fruit. The developed model is simple and low cost. This novel approach can also be used for diagnostic purpose of other agricultural products.