CLASSIFICATION OF THE NUTRITIONAL CONDITION OF BEAN PLANTS (Phaseolus Vulgaris) USING CONVOLUTIONAL NEURAL NETWORKS AND IMAGE ANALYSIS

ABSTRACT

Agriculture plays an essential role in Brazil, especially in the production of beans (Phaseolus vulgaris), an important source of plant protein. In this study, a convolutional neural network (CNN) model was developed to classify the nutritional status of the bean plant focusing on nitrogen (N) content, using RGB images. The experiment was conducted at USP, in Pirassununga, with five nitrogen fertilization treatments and 30 bean plant pots. Weekly images of the leaves were captured starting from 30 days after emergence (DAE). The images were processed and used to train and test different CNN configurations. The results indicated that larger sets of images and smaller blocks (10x10 pixels) increased accuracy, especially at 37 DAE. It is concluded that the proposed model is effective for nutritional monitoring, providing an efficient alternative to traditional leaf analysis.

deep learning; nitrogen; precision agriculture

INTRODUCTION

Agriculture is essential for the economic and social development of Brazil, having a direct impact on the Gross Domestic Product (GDP) and various production chains (Caligaris et al., 2022). However, for this sector to develop sustainably, it is crucial that agricultural practices are carried out responsibly and efficiently, maximizing the use of natural resources while respecting the ecological limits of the systems. Beans (Phaseolus vulgaris) play an important role both in the economy and in the nutrition of Brazilians, being one of the main sources of plant protein in the national diet (Silva et al., 2014; Oliveira & Wander, 2023). According to FAOSTAT (2019), Brazil is the second largest producer of beans in the world (CONAB, 2023).

To ensure high levels of productivity in bean cultivation, it is essential to carry out adequate nutritional management that meets the specific requirements of the plant. Among the nutrients, nitrogen (N) stands out as essential for the growth of the bean plant, as it participates in the formation of chlorophyll, which is responsible for the green coloring of the leaves and the healthy development of the plant (Zhou et al., 2023).

Nitrogen deficiency can be observed by the yellowing of the older leaves, which transfer the nutrient to the younger leaves due to the high mobility of N within the plant (Javornik et al., 2023; Woo et al., 2019). Therefore, precise application of nitrogen fertilization is essential to avoid both excess and deficiency of N, conditions that can compromise productivity (Kraeski et al., 2021).

Traditionally, the diagnosis of the nutritional status of plants is carried out through leaf analyses, methods that, although effective, have limitations in terms of cost, time, and applicability (Cheng et al., 2017). In response to these limitations, non-destructive methods based on deep learning technologies, such as convolutional neural networks (CNNs), have been gaining prominence. When trained with large volumes of data, CNNs are capable of efficiently learning and identifying complex patterns in images. In agriculture, these networks are useful for extracting features from leaves, allowing for indirect inferences about pigment content and, consequently, about the nutritional status of plants (Liu et al., 2021).

The objective of this study was to develop and evaluate a convolutional neural network (CNN) model to classify the nutritional status of the common bean (Phaseolus vulgaris) focusing on nitrogen content, using RGB images captured from 30 days after emergence (DAE), when the symptoms of nitrogen deficiency became visible. The assessment aimed to identify the architecture and the acquisition period that maximized accuracy in nutritional classification, offering a faster, more economical, and efficient approach for nutritional monitoring in precision agriculture.

MATERIAL AND METHODS

The experiment was conducted at the Faculty of Animal Science and Food Engineering (FZEA/USP), in Pirassununga, SP, at an altitude of approximately 627 m, with geographic coordinates of 21°57'27'' south latitude and 47°28'13'' west longitude, in a greenhouse equipped with a pad-fan system for temperature control. This system humidifies and cools the environment, allowing for temperature optimization according to the crop's requirements.

The bean seedlings (Phaseolus vulgaris L., cultivar BRSMG mother-of-pearl) were cultivated in pots to allow for precise control of the applied nutrient quantities. Due to the high mobility of nitrogen in the soil, the application was divided into two stages: one third of the dose was administered at planting, and the remainder was applied 20 days after germination. Irrigation was carried out daily to avoid any influence of water deficit on the plants.

The experiment included five treatments (0, 50, 100, 150, and 200% of the recommended nitrogen dose) and six repetitions, totaling 30 pots with 15 dm³ of soil each. The choice of these treatments aimed to evaluate the impact of different nitrogen doses, from complete absence (0%) to an excessive level (200%), in order to understand the response of the common bean both under deficiency and excess conditions of N. This variation in doses allowed for the investigation of the effects of nitrogen on the growth, development, and productivity of the plants, considering that the common bean is sensitive to fluctuations in this macronutrient.

The images of the leaves were captured with a Fujifilm Finepix S4500 camera, with a 30x optical zoom, mounted on a tripod positioned 80 cm from the pots, in a natural light environment to simulate field conditions. Each pot was photographed to create a large database, with acquisitions starting at 30 DAE, when the first signs of nitrogen deficiency become visible on the leaves. The images were obtained weekly, over four weeks (30, 37, 44, and 51 DAE), during the morning, between 10 a.m. and 12 p.m.

The images were processed at the Laboratory of Machines and Precision Agriculture (LAMAP) with the support of the Laboratory of Robotics and Automation in Biosystems Engineering (RAEB) at the University of São Paulo (Universidade de São Paulo - USP). A script developed in Matlab^® R2021a was used to automatically crop the images into dimensions of 10x10, 40x40, 60x20, and 80x80 pixels, Figure 1, based on previous studies indicating that the size of pixel blocks affects the accuracy of convolutional neural network models (Chen & Tsou, 2022) in recognizing visual patterns in leaves.

FIGURE 1
Size of the cutouts with each status of N.

After defining the dataset, a script was created in Matlab® software to train models using CNN architectures. Table 1 shows the main configurations of the architecture of the model based on the convolutional neural network that was used to classify the level of fertilization.

TABLE 1
Architecture of the model based on the convolutional neural network used to classify the fertilizer application dosages.

Layers	Settings
2D Convolutional	256 kernels with size 5x5
Batch Normalization	Default
2D MaxPooling	Size pooling = 2x2
2D Convolutional	128 kernels with size 3x7
Batch Normalization	Default
2D MaxPooling	Size pooling = 2x2
2D Convolutional	128 kernels with size 7x3
Batch Normalization	Default
2D Convolutional	32 kernels with size 3x3
Batch Normalization	Default
Fully Connected	Activated by softmax

To introduce non-linearity to the network and enhance its ability to identify complex patterns, the ReLU (Rectified Linear Unit) activation function was used, widely adopted in deep learning models for its effectiveness in mitigating the vanishing gradient problem and its operational simplicity (Taye, 2023). Of the generated data blocks, 60% were allocated for training, 20% for validation, and 20% for testing. Ten trainings were conducted with 20 epochs for each combination of week and block size, in order to identify the best classifier based on test accuracy.

From the final network models, confusion matrices were built for each week and each block dimension, totaling 16 matrices. Multiple cycles of CNN architecture optimization were executed to determine the ideal configuration of the parameters, aiming to maximize the classifier's performance.

The performance of the models was evaluated at the end of each cycle using metrics derived from the confusion matrices, such as accuracy, precision, recall, and f1-score, as well as the Kappa index and error, as described in the equations presented in Table 2.

TABLE 2
Formulas of the metrics involved in the confusion matrix.

Metrics	Formula	Description
Accuracy	$(\mathrm{VP}+\mathrm{VN})/\text{Total}$	Analyzes the overall effectiveness of the model.
Precision	$\mathrm{VP}/(\mathrm{VP}+\mathrm{FP})$	Proportion of true positives among all predicted positives.
Recall	$\mathrm{VP}/(\mathrm{VP}+\mathrm{FN})$	Efficiency of a model of positive samples.
F1-Score	$(2\times \text{precision}\times \text{recall})/(\text{precision}+\text{recall})$	Harmonic mean between precision and sensitivity.
Kappa Index	$\hat{K}=\frac{n_t\sum _{i=1}^c{x}_{ii}-\sum _{i=1}^c{x}_{i\oplus }{x}_{\oplus i}}{n_t^2-\sum _{i=1}^c{x}_{i\oplus }{x}_{\oplus i}}$	Evaluates the agreement between the observed and expected ratings of a classifier.
Accuracy	$\left(\text{Correct}/\text{Total}\right)\times 100$	Proportion of correct predictions relative to the total predictions.
Error	$1\text{- Accuracy}$	Proportion of incorrect predictions in relation to the total number of predictions.

• VP: true positive; VN: true negative; FP: false positive; FN: false negative.

Figure 2 presents a flowchart that synthesizes the methodology adopted, allowing for a clear visualization of the procedures carried out and the main stages of the study. This resource facilitates the understanding of the flow of activities, from data collection to analysis and evaluation of results, highlighting the critical phases for the nutritional classification of plants based on the processed images.

FIGURE 2
Graphic summary of the methodology used in the study.

For the convolutional neural network model, the options listed in Table 3 were considered to train the deep learning neural network, and it includes the hyperparameters since they are responsible for defining how the model will be trained and, consequently, how it will be able to perform the task for which it was designed (Bengio, 2012).

TABLE 3
Training options used in the convolutional model.

Number of epochs	10
Size of the minilote	8
Option for data shuffling	every-epoch
Positive scaling of the initial learning rate	0.0001

RESULTS AND DISCUSSION

Tables 2 and 3 shows the influence of the database size on the accuracy of the trained CNN. By comparing the number of images in Table 2 with the accuracy levels achieved in Table 4 for different blocks (10x10, 40x40, 60x20, and 80x80 pixels) and growth stages (30, 37, 44, and 51 DAE), there is a trend that an increase in the number of images results in better accuracy rates in CNN-based classification.

TABLE 4
Number of images applied in the CNN for each combination.

	Images
DAE	10x10	40x40	60x20	80x80
30	2995	1042	2937	2195
37	4252	3372	2748	2457
44	2638	3009	3218	1687
51	2818	3250	2708	1457

At 30 DAE, for example, accuracy increased from 78.3% with 2995 images to 81.4% with 2937 images in a block size of 60x20, demonstrating a positive correlation between the number of images and accuracy. At 37 DAE, a similar increase is noted: accuracy rose from 78.2% with 2457 images to 82.8% with 4252 images in a 10x10 block, highlighting the importance of data volume in improving accuracy. This trend reinforces that larger datasets are essential for the efficient training of CNN models, allowing for optimal accuracy levels in agricultural image analysis.

This effect is corroborated by Rezaei et al. (2024), who showed in their study on deep neural networks for disease recognition in barley that data augmentation improves performance, especially in lower-capacity networks, such as the ResNet.

Safaee et al. (2024) also observed that the number of samples significantly impacts the accuracy of soil property predictions in different deep learning models. Such studies confirm that the amount of data is a critical factor for maximizing accuracy in nutritional classification and other agricultural tasks using CNNs.

Table 5 presents the accuracy and Kappa index obtained from the confusion matrices for different combinations of block sizes and days after emergence. Based on the results, the growth stage of the plants (30, 37, 44, and 51 DAE) was not a determining factor for the performance of the classifier, as all classifiers demonstrated consistency and reliability across the stages, as evidenced by the Kappa Index.

TABLE 5
Accuracy results and Kappa Index for each combination.

	Accuracy (%)				Kappa Index
DAE	10x10	40x40	60x20	80x80	10x10	40x40	60x20	80x80
30	78.3	70.3	81.4	80.4	0.7569Aa	0.6252Aa	0.7664Aa	0.7517Aa
37	82.8	78.8	79.4	78.2	0.7530Aa	0.7298Aa	0.7409Aa	0.7235Aa
44	71.0	73.3	78.9	69.4	0.6376Aa	0.6619Aa	0.7301Aa	0.6122Aa
51	73.0	80.4	70.7	66.3	0.6612Aa	0.7521Aa	0.6327Aa	0.6618Aa

• Legend: The Kappa coefficients followed by the same uppercase letter do not differ in the column by the Z test at 5% probability; the Kappa coefficients followed by the same lowercase letter do not differ in the row by the Z test at 5% probability.

When analyzing the block sizes (10x10, 40x40, 60x20, and 80x80 pixels), it was observed that all were suitable for creating an efficient classifier. However, 10x10 pixel blocks with 37 DAE stood out for having the highest accuracy, followed by 60x20 pixel blocks with 30 DAE and 40x40 pixel blocks with 51 DAE. The 60x20 pixel block size performed well in two combinations, especially at 44 DAE. Although the data from 37 DAE recorded the highest accuracy, they revealed inconsistencies when predicting different classes of nitrogen status, while the model with data from 30 DAE showed more consistent performance at the 100% nitrogen level.

Considering the training time, computational demand, and the representativeness of the images, the blocks of 10x10 and 60x20 pixels were selected as the most suitable. The 10x10 pixel block allowed for a larger number of samples, increasing the variability and robustness of the model, while the 60x20 pixel block offered a better balance between capturing contextual information and computational feasibility. Furthermore, the choice of these sizes was based on accuracy results and Kappa Index, which indicated more consistent performance in these cases. Although larger blocks like 80x80 pixels retain more image details, they reduce the total number of samples available for training, which can negatively impact the model’s generalization. Thus, sizes 10x10 and 60x20 were prioritized for providing a more advantageous relationship between accuracy, computational efficiency, and practical applicability in the nutritional analysis of crops.

Traditionally, the nutritional status of plants is assessed through foliar laboratory analyses, methods that, although reliable, are costly and time-consuming, in addition to requiring physical sample collections. The CNN-based approach emerges as a non-destructive alternative, allowing large-scale and real-time monitoring. However, its effectiveness can be influenced by factors such as image quality, lighting variations, and dataset diversity. Future comparisons could directly quantify the performance of CNNs in relation to laboratory analyses, evaluating not only accuracy but also cost-effectiveness and feasibility for field use.

The reliability of the results is reflected in the substantial Kappa index for all analyzed periods. Recent studies in the area support the use of neural networks in precision agriculture: Supreetha et al. (2024) demonstrated their effectiveness in identifying nutritional deficiencies in rice; Ghazal et al. (2024) analyzed nitrogen stress in corn; Urfan et al. (2024) developed the DL-CRoP platform for identifying species and nutritional states; and Regazzo et al. (2024) used neural networks to predict the nutritional state in strawberries. These studies validate the results of this work, confirming the effectiveness of CNNs for nutritional diagnosis in agriculture, highlighting images as a reliable diagnostic tool.

Although the results demonstrated the effectiveness of CNNs in classifying the nutritional status of common beans, some practical challenges must be considered for their application in real field conditions. The model's sensitivity to variations in lighting and noise present in the images can impact classification accuracy, especially in uncontrolled environments. In the present study, the images were obtained under natural light in a greenhouse, partially reducing these effects. However, to enhance the applicability of the approach, color normalization techniques, data augmentation, and lighting calibration can be incorporated in future studies to mitigate these challenges. Previous works indicate that the implementation of these strategies improves the robustness of CNNs for analyzing agricultural images, allowing greater generalization of models in different scenarios (Nam & Lee, 2024).

CONCLUSIONS

According to the objectives of this study, it was demonstrated that the nitrogen nutritional status in bean leaves can be accurately identified using RGB images and classifiers based on convolutional neural networks (deep learning). The results indicated that the development period (30, 37, 44, and 51 DAE) was not a determining factor for the classifiers' performance, as all achieved substantial classification according to the Kappa Index. Additionally, all tested block sizes (10x10, 40x40, 60x20, and 80x80 pixels) were found to be suitable for developing an efficient classifier.

Only the limitations related to processing speed and computational demand were identified as potential challenges for real-time applications in the field, which may restrict the use of classifiers with larger block sizes. These results reinforce the feasibility of the proposed approach for practical applications in the nutritional diagnosis of bean crops.

ACKNOWLEDGMENTS

To the support of the Luiz de Queiroz Foundation for Agrarian Studies (Fundação de Estudos Agrários Luiz de Queiroz - FEALQ). o the support of the National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq) – Productivity Grant in Technological Development and Innovative Extension DT – 2, No. 304210/2021-8. National Council for Scientific and Technological Development (Conselho Nacional de Desenvol-vimento Científico e Tecnológico - CNPQ) - Funding Code 001.

REFERENCES

• Bengio, Y. (2012). Practical recommendations for gradient-based training of deep architectures. In G. Montavon, G. B. Ortiz, & K. R. Müller (Eds.), Neural networks: Tricks of the trade (2nd ed., pp. 437–478). Springer.
• Caligaris, B. S. A., Rangel, L. E. P., Polidoro, J. C., Farias, P. I. V. (2022). The importance of the National Fertilizer Plan for the future of agribusiness and Brazil. Agricultural Science and Technology, 31(1), 3–8.
• Chen, F., Tsou, J. Y. (2022). Assessing the effects of convolutional neural network architectural factors on model performance for remote sensing image classification: An in-depth investigation. International Journal of Applied Earth Observation and Geoinformation, 112, 102865. https://doi.org/10.1016/j.jag.2022.102865
  » https://doi.org/10.1016/j.jag.2022.102865
• Cheng, W., Sun, D.-W., Pu, H., & Wei, Q. (2017). Chemical spoilage extent traceability of two kinds of processed pork meats using one multispectral system developed by hyperspectral imaging combined with effective variable selection methods. Food Chemistry, 221, 1989–1996. https://doi.org/10.1016/j.foodchem.2016.11.093
  » https://doi.org/10.1016/j.foodchem.2016.11.093
• Companhia Nacional de Abastecimento. (2023). Acompanhamento da safra brasileira de grãos: Safra 2023/24, 2º levantamento https://www.conab.gov.br/info-agro/safras/graos/boletim-da-safra-de-graos
  » https://www.conab.gov.br/info-agro/safras/graos/boletim-da-safra-de-graos
• FAOSTAT. (2019). Crops and livestock products. Food and Agriculture Organization of the United Nations. http://www.fao.org/faostat/en/#data/QC
  » http://www.fao.org/faostat/en/#data/QC
• Ghazal, S., Kommineni, N., Munir, A. (2024). Comparative analysis of machine learning techniques using RGB imaging for nitrogen stress detection in maize. AI, 5(3), 1286–1300. https://doi.org/10.3390/ai5030062
  » https://doi.org/10.3390/ai5030062
• Javornik, T., Poljak, M., Carović-Stanlo, K., Lazarević, B. (2023). Common bean (Phaseolus vulgaris L) gas exchange capacity under nutrient deficiency. Journal of Central European Agriculture, 24(1), 216–224. https://doi.org/10.5513/JCEA01/24.1.3667
  » https://doi.org/10.5513/JCEA01/24.1.3667
• Kraeski MJ, Lopes AS, Fanaya Júnior ED, Pacheco A, Centurião MA, Arevalo ACM, França A de, Medeiros RD (2021) Manejo da irrigação, inoculação e nitrogênio no feijoeiro de inverno. Research, Society and Development 10(8): e56910817437. https://doi.org/10.33448/rsd-v10i8.17437
  » https://doi.org/10.33448/rsd-v10i8.17437
• Liu, Y., Pu, H., & Sun, D.-W. (2021). Efficient extraction of deep image features using convolutional neural network (CNN) for applications in detecting and analysing complex food matrices. Trends in Food Science & Technology, 113, 193–204. https://doi.org/10.1016/j.tifs.2021.04.042
  » https://doi.org/10.1016/j.tifs.2021.04.042
• Nam, J. H., Lee, S. C. (2024). FSDA: Frequency re-scaling in data augmentation for corruption-robust image classification. Pattern Recognition, 150, 110332. https://doi.org/10.1016/j.patcog.2024.110332
  » https://doi.org/10.1016/j.patcog.2024.110332
• Oliveira, G. M. de, & Wander, A. E. (2023). Mapeamento da cadeia produtiva do feijão-comum no Brasil. Revista Economia Política do Desenvolvimento, 14(32), 96-122. https://doi.org/10.28998/2594-598X.2023v14n32p96-122
  » https://doi.org/10.28998/2594-598X.2023v14n32p96-122
• Regazzo, J. R., Silva, T. L., Tavares, M. S., Sardinha, E. J. S., Figueiredo, C. G., Couto, J. L., Gomes, T. M., Tech, A. R. B., & Baesso, M. M. (2024). Performance of neural networks in the prediction of nitrogen nutrition in strawberry plants. AgriEngineering, 6(2), 1760-1770. https://doi.org/10.3390/agriengineering6020102
  » https://doi.org/10.3390/agriengineering6020102
• Rezaei, M., Gupta, S., Diepeveen, D., Laga, H., Jones, M. G. K., &Sohel, F. (2024). Barley disease recognition using deep neural networks. European Journal of Agronomy, 161, 127359. https://doi.org/10.1016/j.eja.2024.127359
  » https://doi.org/10.1016/j.eja.2024.127359
• Supreetha, S., Premalathamma, R., & Manjula, S. H. (2024). Deep learning techniques to detect nutrient deficiency in rice plants. In International Conference on Inventive Computation Technologies (pp. 699-705). https://doi.org/10.1109/ICICT60155.2024.10544924
  » https://doi.org/10.1109/ICICT60155.2024.10544924
• Safaee, S., Libohova, Z., Kladivko, E. J., Brown, A., Winzeler, E., Read, Q., Rahmani, S., Adhikari, K. (2024). Influence of sample size, model selection, and land use on prediction accuracy of soil properties. Geoderma Regional, 36, e00766. https://doi.org/10.1016/j.geodrs.2024.e00766
  » https://doi.org/10.1016/j.geodrs.2024.e00766
• Silva, J. B. L., Ferreira, P. A., Justino, F., Pires, L. C., Toledo, A. S. (2014). Leaf concentrations of nitrogen and phosphorus in Phaseolus vulgaris L plants under high CO2 concentration and drought stress. Engenharia Agrícola, 34(5), 935–944. https://doi.org/10.1590/S0100-69162014000500012
  » https://doi.org/10.1590/S0100-69162014000500012
• Taye, M. M. (2023). Theoretical understanding of convolutional neural network: Concepts, architectures, applications, future directions. Computation, 11(3), 52. https://doi.org/10.3390/computation11030052
  » https://doi.org/10.3390/computation11030052
• Urfan, M., Rajput, P., Mahajan, P., Sharma, S., Hakla, H. R., Kour, V., Khajuria, B., Chowdhary, R., Lehana, P. K., Karlupia, N., Abrol, P., Tran, L. S. P., & Choudhary, S. P. (2024). The Deep Learning-Crop Platform (DL-CRoP): For species-level identification and nutrient status of agricultural crops. Research, 7. https://doi.org/10.34133/research.0491
  » https://doi.org/10.34133/research.0491
• Woo, H. R., Kim, H. J., Lim, P. O., Nam, H. G. (2019). Leaf senescence: Systems and dynamics aspects. Annual Review of Plant Biology, 70(1), 347–376. https://doi.org/10.1146/annurev-arplant-050718-095859
  » https://doi.org/10.1146/annurev-arplant-050718-095859
• Zhou, H., Liu, Y., Mu, B., Wang, F., Feng, N., Zheng, D. (2023). Nitrogen limitation affects carbon and nitrogen metabolism in mung bean (Vigna radiata L.). Journal of Plant Physiology, 290, 154105. https://doi.org/10.1016/j.jplph.2023.154105https://doi.org/10.1016/j.jplph.2023.154105
  » https://doi.org/10.1016/j.jplph.2023.154105https://doi.org/10.1016/j.jplph.2023.154105

Edited by

• Area Editor:
  Welington Gonzaga do Vale

• Edited by
  SBEA

Publication Dates

• Publication in this collection
  21 July 2025
• Date of issue
  June 2025

History

• Received
  8 Nov 2024
• Accepted
  17 Apr 2025