Artificial Intelligence-Based Thermal Imaging for Breast Tumor Location and Size Estimation Using Thermal Impedance

Breast cancer has the highest incidence and mortality in women worldwide. Early and accurate detection is crucial for reducing mortality rates associated with this disease. The incidence of breast cancer has increased significantly in recent years, thus requiring increased efforts, caution, and research toward finding a cure. Importantly, detecting breast cancer at an early stage is essential to avoid the side effects of radiotherapy and chemotherapy treatments, the permanent scars left by surgery, and the fatal consequences of treatment nonresponsiveness. As a result, researchers share a strong desire to find fast, accurate, and minimally invasive procedures to detect early signs of breast cancer.

Imaging in cancer diagnosis has helped to identify diseases, locate malignant tissues for biopsy, identify metastasis, and plan treatment protocols [1]. Mammography is one of the most effective current approaches for early detection of breast cancer; however, it has several limitations. Accessibility, for example, remains an issue for women, even when palpable nodules are present. This examination is essential for therapeutic decision-making but remains unavailable in numerous public cancer referral services. Furthermore, the quality of mammography varies greatly among healthcare providers, and excess radiation exposure is also risky when reexamination is required.

Artificial intelligence has been widely used in medicine, with studies being applied to diagnose various conditions, including cervical cancer [2], thyroid nodules [3], and COVID-19 [4]. In a recent study, an algorithm based on machine learning using least squares methods was developed to perform estimates related to the COVID-19 pandemic and predict the future behavior of the outbreak [4]. Specifically, artificial intelligence has been widely applied in mammography results to improve breast cancer diagnosis and reduce false positive or false negative rates [5–10].

Thermography, on the other hand, represents a low-cost, easily applicable method for early tumor detection, and the use of thermographic images is a potential alternative to mammography exams. However, disadvantages include the high operator dependence on image interpretation and the inability to determine the depth of a nodule.

Tumors are characterized by uncontrolled cell multiplication, resulting in higher rates of metabolic heat generation and blood perfusion compared to healthy tissues. The excess heat generated by the tumor is dissipated to the surrounding healthy tissue, creating a temperature gradient on the breast's surface. The use of infrared imaging in cancer research has been investigated by numerous researchers [1,11–13] and [14]. Sree et al. [1] established standardized protocols for thermographic procedures and interpretation of thermograms. In Ref. [11], the authors performed a 4-year clinical trial involving 769 patients and found a sensitivity of 97%, specificity of 14%, negative predictive value of 95%, and positive predictive value of 24% in 875 biopsied lesions. Analysis of infrared imaging performance in all the biopsied lesions revealed that specificity was statistically improved in dense breast tissue compared to fatty breast tissue. In Ref. [12], a prospective, double-blind study involving 92 patients found a sensitivity of 97%, correctly identifying 58 out of 60 neoplasms with biopsies. This report also showed that digital infrared thermal imaging is a valuable adjunct to mammography, especially in women with dense breast parenchyma. To improve the accuracy of preliminary breast cancer screening, Kapoor and Prasad [13] proposed a thermogram to analyze infrared thermal images automatically. Edge detection and Hough transform were used for asymmetry analysis of breast heat patterns. Mitra and Balaji [14] proposed an estimation of tumor position and size in human breast using thermographic images in conjunction with artificial neural networks. Das and Mishra [15] reported a numerical study on the simultaneous estimation of size, radial location, and angular location of a malignant tumor in a 3-D human breast.

The temperature profiles are obtained by solving the Pennes equation through numerical solutions based on finite elements, using the commercial software COMSOL. In a prior study, Figueiredo et al. [16] estimated breast tumor depth using the experimental surface temperature of the breast. Different methods treat breast tumor detection as an inverse heat transfer problem.

The numerical solution of bioheat transfer critically relies on knowledge of the breast's thermal properties and thermophysical parameters. Breast geometry is another critical factor. Therefore, any experimental temperature data are intrinsically dependent on these parameters. If a temperature field is required, measurements of the thermal properties would be necessary. However, estimating these parameters under natural conditions would be complicated and challenging. An alternative approach involves searching for models or practical procedures that do not require an in-depth understanding of these parameters while improving sensitivity for tumor identification. One of the techniques addressed in this work uses temperature information measured on the surface using infrared sensors or surface thermocouples to quantitatively establish an index indicating the presence of a tumor, without requiring knowledge of the tissue's physical parameters. This study was based on applying damage detection techniques to structures and developed the application of thermal impedance for tumor detection [17].

The analogy between electromechanical and thermal impedance presents a novel approach to inclusion detection. Applying thermal impedance to damage detection represents innovation and can increase the sensitivity of techniques based on thermal infrared images [18].

However, relying solely on thermal images or surface thermal impedance is not sufficient to identify the location and size of an eventual tumor.

Even when artificial intelligence is applied directly to these images, it still fails to obtain tumor location and size due to the low sensitivity associated with temperature variations and their position within the breast. Importantly, studies have been carried out to improve the remarkable screening potential of this technique [19–21]. Nonetheless, these works, which also use CNNs, aim to improve the diagnosis of tumor detection based on thermal images but still fail to detect the size and position of eventual tumors, which is fundamentally the objective of these studies.

In this scenario, additional information is required. This information can be derived from thermal models developed to establish the position and size of eventual tumors within the breast. Furthermore, data can be correlated with breast surface temperatures, whether measured or simulated, to provide the procedure for estimating tumor position and size.

As impedance significantly increases the sensitivity of superficial thermal images related to small and deep tumors, applying an artificial intelligence algorithm represents a powerful tool to produce additional information regarding tumor position and size. This can be estimated through the training of convolutional networks.

As previously mentioned, this work uses artificial intelligence, specifically deep learning, to estimate tumor location and size solely based on thermal images of the breast's skin surface. These thermal images can be acquired using an infrared camera in practical applications. To validate this technique, simulated thermographic images were used. Simulated thermographic images were obtained from direct thermal problem solutions calculated using the commercial software COMSOL Multiphysics.

Deep learning was developed in Python as the programming language and in the integrated development environment (IDE), Jupyter Notebook. Specific deep learning libraries, namely, Keras and TensorFlow, were used for deep learning programming.

Convolutional neural networks were used as a subarea of the deep learning technique to identify patterns or classes of parameters to be evaluated in simulated thermal images. Initially, a database was created with classes of different tumor locations. Tumors were positioned at other locations within the domain that simulated the breast's geometry. Then, surface thermal images were extracted from the temperature field solution. This process was referred to as the direct problem.

where ρ, c, and k are, respectively, the specific mass, the specific heat, and the heat conductivity of the tissue; ω_b is the blood perfusion rate; and ρ_b and c_b are the blood's specific mass and specific heat; T_a is the arterial temperature and T is the temperature distribution. Q_m is the heat source due to metabolic heat generation (W/m³).

Our study investigated two distinct thermal models. The first model represents a Cartesian sample (Fig. 1(a)), where the hyperparameters and sensitivity required to construct the artificial intelligence structure were evaluated. The three-dimensional schematic model of the breast in Cartesian coordinates comprises a region of healthy tissue and a spherical tumor with an arbitrary location. All surface of the Cartesian model was assumed to be exposed to the external environment, except the internal surface of the breast, in contact with the body, was kept at a constant temperature T_c equal to the body's internal temperature, that is, 37 °C (310 K). The temperature of the medium was considered room temperature, T_∞ = 20 °C (293 K).

The second model, an anatomical structure designed to mimic a natural breast, was used as a Phantom to identify the location and size of a tumor inserted into the model (Fig. 1(b)). In both models, an external heat flux was applied at the frontal surface, as shown in figures. This heat flux, $q″0$ constant and equal to 10 W/m², is needed to disturb the initial condition to calculate thermal impedance in transit conditions. Equation (1) and its respective boundary conditions are solved numerically using the finite element method. Figure 2 shows the tridimensional mesh for both models.

Considering the geometric schemes shown in Fig. 1, the tissue domain is subjected to the following boundary conditions for the Cartesian model as:

• an external heat flux applied at the frontal surface

  $−k[∂T∂z]z=0=q″0−hi(T(x,y,0)−T∞)$

  (2)
• surfaces exposed to an ambient medium of temperature T_∞ = 20 °C

  $k[∂T∂x]x=0=hi(T(0,y,z)−T∞)$

  (3)

  $−k[∂T∂x]x=L=hi(T(L,y,z)−T∞)$

  (4)

  $k[∂T∂y]y=0=hi(T(x,0,z)−T∞)$

  (5)

  $k[∂T∂y]y=R=hi(T(x,R,z)−T∞)$

  (6)
• and the internal surface of the breast, in contact with the body, is kept at a constant temperature T_c = 37 °C (310 K)

  $T(x,y,W)=Tc$

  (7)

The boundary conditions of the anatomical model are equivalent to the equations of the Cartesian model and will not be described here. The initial temperature field F(x,y,z) for this model, which corresponds to the steady-state, is shown in Sec. 5 (Figs. 3 and 4).

The characteristics of a natural breast, such as muscle, fat, lobe, nipple, areola, ducts, and irregular geometry, make the thermal model a challenge to solve analytically.

It is essential to observe that Eqs. (1) or (9) can be solved analytically if the work aims to study a 3D Cartesian model. However, the work's great interest is obtaining the position and location of the tumor in a breast with actual geometry. In this work, the anatomical model shown in Fig. 1(b) was used to simulate the breast temperature field. Application to natural breasts is immediate by obtaining a scanned breast geometry. The irregular geometry of the anatomical model, or a natural breast, was the main reason for modeling numerically and not analytically.

In our study, the geometry of a typical breast was acquired by scanning an anatomical model. The direct problem solution was obtained using finite elements through the COMSOL multiphysics software.

The values of thermophysical properties used in the simulations, including thermal conductivity k, blood perfusion w, volumetric heat generation Q_m, density ρ, and specific heat c_p of the biological tissues, are listed in Table 1. Density and specific heat are assumed to have the same value for both healthy breast tissue and tumor. Table 2 presents the mesh, element types of the three-dimensional model provided by the commercial software (COMSOL multiphysics), and statistics related to the features of the 3D model under analysis.

where q₀(ω) is the thermal excitation represented by the heat flux applied on the surface; ΔT(ω) is the response to this heat flux, measured at the surface temperature of the sample (thermal system); ΔV (ω) and I(ω) represent voltage and current variation of the electromechanical system. The variable ω corresponds to the frequency domain, obtained through the Fourier transform applied to the temporal signals of ΔT(t) and q(t) measured by heat flux and temperature transducers. As the thermal system depends on thermal properties, including thermal conductivity, thermal diffusivity, specific heat, and the medium itself, the thermal impedance must also assume different values if the medium has inclusions with other thermal properties than its surroundings.

Equation (17) does not explicitly include thermal parameters such as perfusion and metabolism. Can these parameters in human tissue in vivo change the experimental thermal impedance? If so, can a thermal impedance be defined based on them? The thermal problem resulting from metabolism, perfusion, and external thermal excitation was developed in this project, and we defined the thermal impedance related to this system.

In the practical application, where part of the breast surface is exposed to a convective medium, an analysis of the necessity for precise knowledge of the heat transfer coefficient by convection must be carried out. This analysis must consider the presence or absence of tumors.

Figures 3 and 4 illustrate the breast without and with tumor, respectively, showing thermal images, T, and impedance images, Z(t).

Observing the images, we note changes in temperature colors due to the effects of medium thermal convection. However, this effect is absent in the thermal impedance images, Figs. 3(b) and 4(b). Our findings indicate that the h parameter does not interfere with the procedure for estimating an eventual tumor.

If the size and intensity of metabolism need to be estimated, it is also necessary to analyze the previous sensitivity of the parameters that influence the temperature measurement.

Figure 5 displays the temperature and impedance evolution considering different metabolic heat generation intensities.

Spherical tumors with a radius of 3 mm were considered in the simulations. The thermal behavior of simulated variations in the metabolic heat source was examined, with tumors positioned at (x,y,z) = (80,80,60) mm. Infrared images are shown in Figs. 5(a)–5(d) and thermal impedance images of the breast surface are presented in Figs. 5(e)–5(h). According to Figs. 5(a)–5(d), it is challenging to discern the effect of variations in tumor metabolism intensities.

However, this effect is evident in Figs. 5(e)–5(h). The figures illustrate temperature curves called Q_m = 10⁴ W/m³ to Q_m = 10⁷ W/m³, representing their metabolic generations. Notably, all these curves have similar shapes and profiles. This format is symmetrical with respect to pixel 484, where the breast's impedance is sensitive to changes in metabolic intensity. This effect is highlighted in Fig. 6, where the tumor center is located. An overlap can be seen between the curves, making it impossible to distinguish them visually.

The results demonstrate that thermal impedance values undergo significant variations, as depicted in Fig. 6(b).

To provide further clarity, we determined the sensitivity coefficient related to metabolic intensity and the influence of tumor size. Figures 7(a) and 7(b) show the evolution of the sensitivity coefficients associated with the metabolism power $(Qm∂T∂Qm and Qm∂Z∂Qm)$⁠, respectively and Figs. 8(a) and 8(b) show the effect of tumor size by comparing the evolution of the sensitivity coefficients associated with the radius $(r∂T∂r and r∂Z∂r )$⁠, respectively. This effect on temperature and impedance variations can also be verified in Fig. 9.

The direct problem is established by solving the thermal problem with multiple tumors strategically placed within a Cartesian geometry that simulates the breast. Figures 10(a) and 10(b) present, respectively, the three-dimensional Cartesian breast, with dimensions of 160 × 160 × 80 mm, along with a top view indicating the tumor locations created to build the database for simulated temperature images.

Figure 10(b) shows the positioning of the simulated tumors in the x − y plane. This means their positions on the x−axis ranged from x = 10 mm to x = 80 mm, to the y coordinate, between y = 10 mm to y = 80 mm. Figures 11(e)–11(h) show the simulated tumors in the xy plane. In this plane, the tumors were introduced at a distance of 10 mm. Temperature fields calculated considering tumor depth varying, with z = 60 mm, z = 65 mm, z = 70 mm, and z = 75 mm, respectively, in the coordinate (x = 80 mm, y = 80 mm).

The physical and geometric parameters of the breast and tumors were randomly changed to construct our database. The COMSOL Multiphysics software assumed that a random variable was used. The database comprised approximately 1113 images, divided into training, test, and validation sets. Specifically, 40% of the images were part of the training set, 40% of the test set, and 20% were used for validation.

Although a detailed analysis is not presented here, several hyperparameters were adjusted to construct CNNs. These are weights, bias, number of layers, number of neurons per layer, activation function, optimizer, batch size, target size, epochs, learning rate, and momentum. Initially, we identified which parameters were most commonly used in related works. Posteriorly, we analyzed which ones achieved better results with fewer epochs. The parameters were varied one by one, then two by two, to determine the optimal configuration.

We used an automatic search method known as Bayesian optimization. Figures 12(a) and 12(b) show the neural network architecture and their convergence when the optimizers are varied, including Nadam, RMSprop, Adam Adamx, SGD and Adagrad, Adadelta, and Ftrl. Notably, the Nadam and RMSpropt optimizers showed better accuracy

We used two image sizes for comparison: 128 × 128 and 512 × 512 pixels. The number of trainable parameters by one of the main parameters trained by CNNs is the image pixels. The neural network is outlined in Table 3.

This neural network architecture consists of the conv2d-input data input, the hidden layers composed of Conv2D and MaxPooling2D, the data flattening layer flatten, and the dense data output layer (Table 4). Our initial assessment focused on verifying the ability of the neural network to estimate the location of tumors not found in the neural network learning process; this analysis was referred to as the inverse problem. We tested an estimate of the tumor coordinates located among the classes in the neural network learning process. Posteriorly, we verified the network's ability to correlate these new positions with those already known by the neural network through the learning process. A second test considered the variation of two tumor coordinates.

Initially, it was found that the neural network model based on deep learning could not predict the correct depth of tumors in positions x = 10, y = 10, located in the lower left corner.

We noticed that these tumors suffer interference. Our preliminary findings revealed that the neural network model based on deep learning could not predict the correct depth within the pixel matrix due to the proximity of the thermal image's edges. To mitigate this problem, we optimized the hyperparameters to achieve more accurate results and reduce the error function's values.

Among the numerous possibilities of hyperparameter configurations, two were chosen to create the neural network model referring to the database under analysis. For the thermal images described in Sec. 6.1, the Nadam optimizer was used, a model with four hidden layers and 512 neurons per layer, a ReLu activation function, 512 × 512 image quality, Glorot Uniform weight, and zero bias. For thermal impedance images (Sec. 6.2), we also used the Nadam optimizer, a model with four hidden layers, Lrelu, 32 neurons, and 256 × 256 image quality. Table 3 compiles this information.

Hyperparameters in CNNs are not learned directly during model training, but they significantly affect the training process and network architecture. These parameters are defined before training and are vital in determining the CNNs performance, efficiency, and behavior. Proper optimization of these hyperparameters is crucial to achieve good performance of the CNN in specific tasks. This optimization can be conducted manually or through more advanced techniques, such as Bayesian optimization. Figure 11(b) was included in this study to show how the manual search method would change only one hyperparameter. In this case, the optimizers were modified. Figure 11 indicates that the Nadam, RMSprop, and Adam optimizers achieved better accuracy for smaller epochs. In contrast, the Adadelta and Ftrl optimizers did not present any response even with 1000 neural network training epochs for the same analysis.

Figure 13 illustrates the simulated tumors (in grey) present in the neural network learning process. New tumors were then inserted between previously known grid positions to assess the network's ability to estimate tumor positions in unknown locations. The fictitious tumors (in red) were placed between y = 70 and y = 80 and between positions x = 70 and x = 80. This analysis was repeated at all discrete z locations.

The results of the location estimation using the Nadam optimizer, a model with four hidden layers and 512 neurons per layer, ReLu activation function, and 512 × 512 image quality, are displayed in Table 5.

We observed that the neural network correlated the thermal image of the new position with positions previously known by the neural network. In other words, it approximated the tumors closer to the locations known by the neural network. The tumors positioned at (x,y) = (75,75) are between two locations known by the network (x,y) = (70,70) and (x,y) = (80,80). Furthermore, in some cases, the network adjusted the z-depth to optimize the tumor position concerning the trained points.

A database was constructed using spherical tumors with a radius of R₁ = 2 mm, R₂ = 6 mm, and R₃ = 10 mm, located at z = 70 mm. The center of the tumors is located 10 mm from the breast's surface, as seen in Fig. 14. As shown in Table 3, a Nadam optimizer was used along with a model with four hidden layers and 32 neurons per layer, ReLu activation function, and 256 × 256 image quality. The initialization of bias matrices and weights were, respectively, zero and glorot uniform.

The scanned three-dimensional model of the breast was also numerically simulated using the COMSOL Multiphysics software. Its geometry is shown in Figs. 15(a) and 15(b), displaying the front and lateral views of the breast, respectively. The anatomical breast has dimensions of 144 mm width, 160 mm height, and 58.4 mm thickness.

Figure 16 shows some of the surface temperature images of the anatomically shaped breast with inclusions inserted at different positions.

The behavior of some familiar parameters in the IA universe can indicate the performance of the optimization procedure. They can be described as categorical accuracy, which computes the mean accuracy rate across all predictions; precision, which is related to the predictions concerning the labels; accuracy, which calculates how often predictions equal labels; epochs is an arbitrary cutoff, generally defined as “one pass over the entire dataset,” used to separate training into distinct phases, which is helpful for logging and periodic evaluation and loss that is typically created by instantiating a loss class and Loss function that is a parameter that computes the quantity that a model should seek to minimize during training.

Figures 17 and 18 represent the convergence of the neural network related to the anatomically shaped breast images. Estimation results using temperature images and thermal impedance images, respectively, are presented. Both figures indicate that the model converges approximately to the number of 175 epochs. However, besides the more significant oscillation, the estimation error value is evident when using temperature images. In turn, the use of impedance images (Fig. 18) allows a curve with fewer oscillations in precision values, and the error function value tends to zero in the neural network convergence values.

Table 6 depicts the estimation of tumor location with the variation of the positioning in a coordinate and changes in tumor size.

Our findings suggest that estimating tumor size is possible since thermal impedance images are sensitive to changes in the size of the metabolic heat source. Furthermore, the results showed significant improvement compared to the analysis conducted on temperature images. In this case, a small error was obtained in tumor positioning and the estimation was successful, as the estimated positions were obtained for the closest locations already known by the database.

The technique proposed in this study is based on CNNs applied to both thermographic images and thermal impedance images of the breast surface. Two three-dimensional geometries were used for the breast: Cartesian geometry and geometry with accurate shapes obtained by digital scanning of a phantom. The size (radius) and intensity of tumor metabolic heat generation do not significantly alter either the image pixel profiles or the normalized surface temperature profiles. This behavior results in low sensitivity in temperature images related to these parameters. Low sensitivity implies that temperature profiles can be used only for spatial tumor location. However, using the thermal impedance images allows us to determine tumor location and size. A phantom breast scan was used in three-dimensional numerical simulations to estimate the location and size of a simulated tumor.

• Brazilian Agencies CNPq, CAPES/PROEX, and FAPEMIG.

• Laboratory of Mechanical Projects (LPM-UFU) and Laboratory of Heat Transfer: Modeling and Experiment (LTCME-UFU).

The authors attest that all data for this study are included in the paper.

C =

tissue specific heat (J/kg.K)

c_b =

blood specific heat (J/kg.K)

CNN =

convolutional neural network

h_i =

convection coefficient (W/m².K)

I =

current (A)

k =

tissue thermal conductivity (W/m.K)

q″₀ =

external heat flux (W/m²)

Q_m =

heat source due to metabolic heat generation (W/m³)

t =

time (s)

T =

tissue temperature (K)

T_a =

arterial temperature (K)

T_c =

internal temperature (K)

T_st =

tissue temperature in steady-state conditions (K)

T_∞ =

room temperature (K)

w =

frequency (Hz)

w_b =

blood perfusion rate (1/s)

Z(ω) =

thermal impedance (K/W)

ΔV =

voltage (Volts)

ρ =

tissue specific mass (kg/m³)

ρ_b =

blood specific mass (kg/m³)