A method for bee activities recognition from videos captured at the beehive entrance
DOI:
https://doi.org/10.54939/1859-1043.j.mst.CSCE8.2024.3-13Keywords:
Bee detection; Bee tracking; Bee activity recognition; YOLOv5; OC-SORT.Abstract
Honeybees play an important role in the ecosystem and agricultural economy. To maintain and develop healthy bee colonies, monitoring and recognizing bee activities at the beehive entrance is necessary. In this research, we extend the method in [6] to track and recognize the flight-in and flight-out activities of both pollen-bearing and non-pollen-bearing bees in videos recorded at the beehive entrance. To achieve this goal, a framework consisting of bee detection, bee tracking, and bee activity recognition is proposed. In the first step, to address the imbalance between the number of pollen-bearing and non-pollen-bearing bees, we employed a detection method combining YOLOv5 and the focal loss function. Subsequently, in the tracking step, based on the detection results of the first step, two OC-SORT-based trackers were initialized to determine the trajectories of pollen-bearing and non-pollen-bearing bees. Finally, in the activity recognition step, rules are applied to the tracked trajectories to determine the instantaneous activity states of honey bees and to recognize their overall activities. The experimental results show that the detection step obtained an overall precision of 0.972 whereas the tracking step achieved HOTA values of 77.28%, MOTA of 90.09%, and MOTP of 84.98%.
References
[1]. EFSA Panel on Animal Health and Welfare (AHAW). “Assessing the health status of managed honeybee colonies (HEALTHY‐B): a toolbox to facilitate harmonised data collection”. EFSA Journal, 14(10), e04578, (2016). DOI: https://doi.org/10.2903/j.efsa.2016.4578
[2]. G. Jocher, A. Chaurasia, A. Stoken, J. Borovec, Y. Kwon, J. Fang, K. Michael, D. Mon-tes, J. Nadar, P. Skalski, et al., ultralytics/yolov5: v6.1-tensorrt, tensorflow edge tpu and openvino export and inference, Zenodo (2022).
[3]. J. Cao, J. Pang, X. Weng, R. Khirodkar, and K. Kitani, “Observationcentric sort: Re-thinking sort for robust multi-object tracking,” in In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9686–9696, 2023. DOI: https://doi.org/10.1109/CVPR52729.2023.00934
[4]. Jiang, J. A., Wang, J. C., Huang, C. P., Lee, M. H., Liu, A. C., Lin, H. J.,... & Yang, E. C. “Foraging flight-based health indicators for honey bee colonies using automatic monitoring systems”. Computers and Electronics in Agriculture, 216, 108476, (2024). DOI: https://doi.org/10.1016/j.compag.2023.108476
[5]. Kim, S., & Kim, H. “A new metric of absolute percentage error for intermittent de-mand forecasts”. International Journal of Forecasting, 32(3), 669-679, (2016). DOI: https://doi.org/10.1016/j.ijforecast.2015.12.003
[6]. Le, T. N., Tran, D. N., Pham, H. T., Le, T. L., & Vu, H. “A Robust Multiple Honeybee Tracking Method from Videos Captured at Beehive Entrance”. In 2023 International Conference on Multimedia Analysis and Pattern Recognition (MAPR) (pp. 1-6). IEEE, (2023). DOI: https://doi.org/10.1109/MAPR59823.2023.10289105
[7]. Luiten, J., Osep, A., Dendorfer, P., Torr, P., Geiger, A., Leal-Taixé, L., & Leibe, B. Ho-ta: “A higher order metric for evaluating multi-object tracking”. International journal of computer vision, 129, 548-578, (2021). DOI: https://doi.org/10.1007/s11263-020-01375-2
[8]. Ngo, T. N., Rustia, D. J. A., Yang, E. C., & Lin, T. T. “Automated monitoring and anal-yses of honey bee pollen foraging behavior using a deep learning-based imaging sys-tem”. Computers and Electronics in Agriculture, 187, 106239, (2021). DOI: https://doi.org/10.1016/j.compag.2021.106239
[9]. Nguyen, D. T., Le, T. N., Phung, T. H., Nguyen, D. M., Nguyen, H. Q., Pham, H. T., Vu, H. & Le, T. L. “Improving pollen-bearing honey bee detection from videos cap-tured at hive entrance by combining deep learning and handling imbalance tech-niques”. Ecological Informatics, 82, 102744, (2024). DOI: https://doi.org/10.1016/j.ecoinf.2024.102744
[10]. Rozenbaum, E., Shrot, T., Daltrophe, H., Kunya, Y., & Shafir, S. “Machine learn-ing-based bee recognition and tracking for advancing insect behavior research”. Arti-ficial Intelligence Review, 57(9), 245, (2024). DOI: https://doi.org/10.1007/s10462-024-10879-z
[11]. S. A. M. Khalifa, E. H. Elshafiey, A. A. Shetaia, A. A. A. El-Wahed, A. F. Algethami, S. G. Musharraf, M. F. AlAjmi, C. Zhao, S. H. D. Masry, M. M. Abdel-Daim, M. F. Halabi, G. Kai, Y. A. Naggar, M. Bishr, M. A. M. Diab, and H. R. El-Seedi, “Overview of bee pollination and its economic value for crop production,” Insects, vol. 12, no. 8, (2021). DOI: https://doi.org/10.3390/insects12080688
[12]. Sokhai, K., & Mardy, S. “A Review on the Aspect of Beekeeping and Economic Effi-ciency”, International Journal of Integrative Research 2(2):107-114, (2024). DOI: https://doi.org/10.59890/ijir.v2i2.1223
[13]. Thi Nha Ngo, Kung-Chin Wu, En-Cheng Yang, Ta-Te Lin, “A real-time imaging system for multiple honey bee tracking and activity monitoring”, Computers and Elec-tronics in Agriculture, (2019).
[14]. U. Joshi, K. Kothiyal, Y. Kumar, and R. Bhatt, “Role of honeybees in horticultural crop productivity enhancement,” International Journal of Agricultural Sciences, vol. 17, no. AAEBSSD, pp. 314–320, (2021). DOI: https://doi.org/10.15740/HAS/IJAS/17-AAEBSSD/348-355
