ACTIVE CONTOURS METHOD IMPLEMENTATION FOR OBJECTS SELECTION IN THE MOBILE ROBOT’S WORKSPACE

SELECTION IN THE MOBILE ROBOT’S WORKSPACE

Vladyslav Yevsieiev1, Svitlana Maksymova1, Nikolay Starodubtsev1,

Amer Abu-Jassar2

1Department of Computer-Integrated Technologies, Automation and Robotics, Kharkiv

National University of Radio Electronics, Ukraine

2Faculty of Information Technology, Department of Computer Science, Ajloun National

University, Ajloun, Jordan

Abstract: This article is a study on the implementation of the active contours method using mathematical descriptions to identify objects in the work area of a mobile robot. The program, developed in Python in the PyCharm 2022.2.3 (Professional Edition) environment, is based on the principles of the active contour method, ensuring accurate selection of objects in images. Experiments conducted on matchbox contour extraction with the help of ESP32- Cam module confirm the effectiveness of the method in real-world conditions, demonstrating its potential for application in various fields of mobile robotics and computer vision.

Key words: Industry 5.0, Сomputer Vision Systems, Mobile Robots, Work zone

Introduction In the era of Industry 5.0, where digital technologies penetrate all areas of industry, robotic systems are becoming an integral part of the work process. In this context, the introduction of mobile robots into enterprise work areas promises to revolutionize the way tasks are performed, increasing efficiency and the level of automation. It should be noted that robots are becoming collaborative, that is, they work closely with people. However, successful implementation of robotic systems requires an accurate perception of the environment, making Computer Vision Systems [1]-[14], Speech Recognition Systems [15]-[19] and so on a key element in integrating mobile robots into manufacturing processes.

When developing computer vision systems, it is necessary to solve a whole series of different problems [20]-[29]. In this work we will consider the problem of object recognition in terms of the objects contour selection. In this context, this paper focuses on the implementation of the active contour method in the work area of a mobile robot. This method, an advanced technique in the field of Computer Vision, can effectively isolate objects in images, providing an accurate perception of the environment. The study is relevant in light of the rapid development of mobile robots and their role in Industry 5.0, where precision visual perception of the work area becomes an important component for increasing the productivity and efficiency of automated production processes.

Related works Let us consider some recent works by scientists related to the objects contour selection.

First of all, we note that objects contour selection problem is widely used in different scientific fields. In [30] authors propose a novel end-to-end edge-aware network, the EANet, and an edge-aware loss for getting accurate buildings from aerial images.

Scientists in [31] note that extracting parametric edge curves from point clouds is a fundamental problem in 3D vision and geometry processing. They propose to directly detect structured edges to circumvent the limitations of the previous point-wise methods. They present NerVE, a novel neural volumetric edge representation that can be easily learned through a volumetric learning framework.

At the same time, researchers [32] consider robotic welding technology that is constantly growing with the development of vision technologies. They propose a novel method to extract weld seams based on point cloud registration from various view directions that can handle randomly occluded seams in a workpiece with multiple seam parts.

The paper [33] proposes a construction progress information acquisition system based on simultaneous localization and mapping (SLAM) and edge extraction in view of the shortage of construction progress supervision and management.

Waldner, F., & Diakogiannis, F. I. [34] propose a data-driven, robust and general method to facilitate field boundary extraction from satellite images. We formulated this task as a multi-task semantic segmentation problem. Their convolutional neural network is capable of learning complex hierarchical contextual features from the image to accurately detect field boundaries and discard irrelevant boundaries, thereby outperforming conventional edge filters.

Article [35] proposes a ray-model-based straight-line extraction method for the grid map of a mobile robot, call RM-Line.

Xu, S. and co-authors [36] note that global structural features of buildings with strong coupling relationships in complex scenes are difficult to extract, such as the edges and bodies of buildings, leading to discontinuous results. Therefore, multiscale decoupled body and edge supervision network, which can consider both edge optimization and inner consistency, is proposed to solve these problems.

Study [37] propose edge computing based video pre-processing to eliminate the redundant frames, so that we migrate the partial or all the video processing task to the edge, thereby diminishing the computing, storage and network bandwidth requirements of the cloud center, and enhancing the effectiveness of video analyzes.

So, we a great amount of problems and solutions in this scientific field. And further in this article we consider object boundaries automatic selection based on the active contours method.

Object boundaries automatic selection based on the active contours method

The active contour method is an effective approach in mobile robotics for the objects boundaries selection in images. This technique provides mobile robots with the ability to adapt to a variety of object shapes in the environment, which is especially important when navigating in different environments. Using the active contour method, robots can perform object segmentation, which facilitates the perception of the environment and decision making in real time. In addition, this method takes into account the context of the scene, which improves the accuracy of edge extraction in complex scenarios where lighting conditions or the presence of noise may be variable. Integrating active contours with other sensors such as lidars or radars provides mobile robots with a more reliable perception of their surroundings and improves overall navigation. Finally, the results of edge extraction can be used to correct the robot's trajectory, making this method an important tool in ensuring safe and efficient operation of mobile robots.

Let us denote byernal

Eint the internal energy of the contour, which is usually represented in mathematical terms such as elasticity or smoothing, and it describes how much the contour “tends” to a certain shape. In the active contour method, this energy is minimized in the process of searching for the optimal position of the contour.external

E is an external energy. In the context of the active contour method in image processing, external contour energy is energy originating from external influences or data, such as the intensity of pixels in an image. This energy helps the contour to be attracted to certain parts of the image that contain objects of interest. Therefore, the general energy function can be defined in next way:externalernaltotal EEE  int

. (1)

Now we describe the evolution of the circuit by minimizing the total energy. This is achieved by changing the position of the contour in the direction opposite to the total energy gradient with respect to the contour curve. The evolution of a circuit in the active circuit method can be represented mathematically as a differential equation. The basic idea is to move the contour in the opposite direction of the total energy gradient with respect to the contour curve. Let us denote the contour curve as),( tsC , wheres is the length of the contour andt is the time parameter. Then the evolution of the contour can be represented by the equation:dC

dt

dC total



. (2)

Now let's dive into the part with the gradient of the total energy relative to the contour curve:dC

dC

dC

dE externalernaltotal  int

. (3)

Thus, we obtain the contour evolution equation:dC

dC

dt

dС externalernal  int

. (4)

This equation describes how the contour changes over time in the opposite direction to the total energy gradient. Solving this equation allows the contour to evolve and adapt to the object boundaries in the image.

Note that in the active contour methodernal

Eint includes two terms: the contour deformation term and the contour length term:

– contour deformation refers to changes in the shape or position of a closed curve (contour) in an image. In the context of the active contour method, contour deformation occurs in response to external forces and energies, such as a brightness or color gradient in an image. The closer the contour is to the boundaries of objects, the less deformation, and vice versa;

– contour length, this is a physical or mathematical characteristic of a closed curve that represents the object boundary in the image. In the active contour method, contour length is considered in the context of contour energy. As a contour approaches the objects boundaries, its length typically decreases as the method seeks to minimize the contour's energy by aligning it with external factors such as brightness or color.ernal

Eint

takes the following form:

139dxССLengthE ernal   )|(|)( 2

int





, (5)

where: – parameter regulating contour rigidity;

– parameter regulating contour deformation;)(СLength

– contourC length;C

– contour gradient.

External energyexternal

E evaluates how closely the outline matches the object we want to extract. This energy is usually based on the intensity of the image and can include additional factors such as gradients and textures. As a result,external

E can be generally described by the following expression:  С

external dsyxtexturewyxIwyxIwE ),(||),(||),( 321

, (6)

where:321 ,, www – weighting coefficients that allow to adjust the contribution of each factor;),( yxI

– image intensity gradient;),( yxtexture

– function that evaluates texture characteristics at a point),( yx .

Thus, the external energy is a weighted combination of pixel intensities, gradients, and textures along the contour. Adjusting the weighting coefficients allows to adjust the sensitivity to different aspects of the image when highlighting the contour of an object.

Let us follow the evolution of the circuit. Contour evolution, in the context of the Active Contour Model (or Snake), is the process of changing the position of a contour curve in an image over time or iterations. The purpose of this process is to adjust the outline so that it better matches the object boundaries that you want to extract in the image. Basically the circuit evolves in the direction of minimizing the total energy.dC

tСtC total

 )()1(

, (7)

where:dC

dEtotal – the gradient of the total energy with respect to the contour curve.

The active contour method is often used to segment objects in images and can be adapted to various conditions and task requirements.

Software implementation and experimentation

To check the correctness of the reasoning, we will develop a program in Python in the development environment PyCharm 2022.2.3 (Professional Edition). Let us give an example of software implementation of the above described mathematical expressions.

# Get an outline based on the selected area

points = cv2.convexHull(np.array([[rect[0], rect[1]], [rect[0], rect[1] + rect[3]], [rect[0] + rect[2], rect[1] + rect[3]], [rect[0] + rect[2], rect[1]]]))

Obtaining a contour based on a selected area is useful for isolating and analyzing a specific object or region in an image. In addition, this approach provides interactivity and precise control when analyzing images, which is important in areas where high precision and a personalized approach are required.

# Initialize the active circuit

snake = points.reshape((-1, 1, 2))

This stage allows to set the initial approximation of the object contour in the image, which is a necessary step for the evolution of the contour during the operation of the algorithm.

# Contour evolution process

epsilon = 0.1 # Parameter for approximation

snake = cv2.approxPolyDP(snake, epsilon, closed=True)

The process of contour evolution in the active contour method is a key step aimed at automatically adjusting the contour to the object boundaries in the image. The evolution of a contour occurs by minimizing the total energy, which includes internal energy (characterizing the contour shape and smoothness) and external energy (assessing the conformity of the contour to the object). T

his process allows contour adaptation to different shapes of objects in an image, making it an important tool for automated object extraction in computer vision and image analysis. The result of the developed program based on the active contours method is presented in Figure 1.

a) b)

c) d)

a),c) original image; b), d) selected object boundary;

Figure 1: The result of the developed program based on the active contours method

As you can see from the results of the experiment on identifying objects based on the method of active contours, it allows to determine the presence of an object and obtain its contour for subsequent recognition.

Conclusion

As a result of our research, we have successfully implemented the active contours method for identifying objects in the mobile robot’s workspace. A mathematical description of the method provided the basis for developing a program in Python using the PyCharm 2022.2.3 (Professional Edition) development environment. Experiments carried out, in particular on the contour extraction of a matchbox and an ESP32-Cam module, confirmed the effectiveness of the method in various scenarios, highlighting its potential for application in mobile robotics and computer vision. The results obtained confirm the capabilities of the active contours method in object selection tasks in real working environment conditions.

REFERENCES

1. Akopov, M., & et al. (2023). Choosing a Camera for 3D Mapping. Journal of Universal Science Research, 1(11), 28-38.

2. Yevsieiev, V., & et al. (2022). Software Implementation Concept Development for the Mobile Robot Control System on ESP-32CAM. In Current issues of science, prospects and challenges: collection of scientific papers «SCIENTIA» with Proceedings of the II International Scientific and Theoretical Conference, Sydney, Australia: European Scientific Platform, 2, 54-56.

3. Maksymova, S., & Velet, A. (2022). Development of an Automated System of Terminal Access to Production Equipment Using Computer Vision. In Manufacturing & Mechatronic Systems 2022: Proceedings of VIst International Conference, Kharkiv, 22-23.

4. Maksymova, S., & Akopov, M. (2023). Selection of Sensors for Building a 3D Model of the Mobile Robot's Environment. In Manufacturing & Mechatronic Systems 2023: Proceedings of VII International Conference, Kharkiv, 33-35.

5. Abu-Jassar, A. T., Al-Sharo, Y. M., Lyashenko, V., & Sotnik, S. (2021). Some Features of Classifiers Implementation for Object Recognition in Specialized Computer systems. TEM Journal: Technology, Education, Management, Informatics, 10(4), 1645-1654.

6. Deineko, Zh., & et al.. (2021). Color space image as a factor in the choice of its processing technology. Abstracts of I International scientific-practical conference «Problems of modern science and practice» (September 21-24, 2021). Boston, USA, pp. 389-394.

7. Putyatin, Y. P., & et al.. (2016) The Pre-Processing of Images Technique for the Material Samples in the Study of Natural Polymer Composites. American Journal of Engineering Research, 5(8), 221-226.

8. Lyashenko, V., & et al.. (2021). Wavelet ideology as a universal tool for data processing and analysis: some application examples. International Journal of Academic Information Systems Research (IJAISR), 5(9), 25-30.

9. Kobylin, O., & Lyashenko, V. (2014). Comparison of standard image edge detection techniques and of method based on wavelet transform. International Journal, 2(8), 572-580.

10. Rabotiahov, A., Kobylin, O., Dudar, Z., & Lyashenko, V. (2018, February). Bionic image segmentation of cytology samples method. In 2018 14th International Conference on Advanced Trends in Radioelecrtronics, Telecommunications and Computer Engineering (TCSET) (pp. 665-670). IEEE.

11. Baker, J. H., Laariedh, F., Ahmad, M. A., Lyashenko, V., Sotnik, S., & Mustafa, S. K. (2021). Some interesting features of semantic model in Robotic Science. SSRG International Journal of Engineering Trends and Technology, 69(7), 38-44.

12. Lyashenko, V., Kobylin, O., & Ahmad, M. A. (2014). General methodology for implementation of image normalization procedure using its wavelet transform. International Journal of Science and Research (IJSR), 3(11), 2870-2877.

13. Гиренко, А. В., Ляшенко, В. В., Машталир, В. П., & Путятин, Е. П. (1996). Методы корреляционного обнаружения объектов. Харьков: АО “БизнесИнформ, 112.

14. Al-Sharo, Y. M., Abu-Jassar, A. T., Sotnik, S., & Lyashenko, V. (2021). Neural Networks As A Tool For Pattern Recognition of Fasteners. International Journal of Engineering Trends and Technology, 69(10), 151-160.

15. Matarneh R., & et al. (2018). Voice Control for Flexible Medicine Robot. International Journal of Computer Trends and Technology (IJCTT), 55(1). 1-5.

16. Maksymova S., & et al. (2017). Software for Voice Control Robot: Example of Implementation. Open Access Library Journal, 4, 1-12.

17. Matarneh R., & et al. (2017). Building Robot Voice Control Training Methodology Using Artificial Neural Net. International Journal of Civil Engineering and Technology, 8(10), 523–532.

18. Maksymova S., & et al. (2017). Voice Control for an Industrial Robot as a Combination of Various Robotic Assembly Process Models. Journal of Computer and Communication, 5, 1-15.

19. Matarneh R., & et al. (2017). Speech Recognition Systems: A Comparative Review. Journal of Computer Engineering (IOSR-JCE), 19(5), 71–79.

20. Sotnik, S., Mustafa, S. K., Ahmad, M. A., Lyashenko, V., & Zeleniy, O. (2020). Some features of route planning as the basis in a mobile robot. International Journal of Emerging Trends in Engineering Research, 8(5), 2074-2079.

21. Lyashenko, V., Matarneh, R., & Kobylin, O. (2016). Contrast modification as a tool to study the structure of blood components. Journal of Environmental Science, Computer Science and Engineering & Technology, 5(3), 150-160.

22. Mousavi, S. M. H., Lyashenko, V., & Prasath, S. (2019). Analysis of a robust edge detection system in different color spaces using color and depth images. Компьютерная оптика, 43(4), 632-646.

23. Babker, A., & Lyashenko, V. (2018). Identification of megaloblastic anemia cells through the use of image processing techniques. Int J Clin Biomed Res, 4, 1-5.

24. Mousavi, S. M. H., Victorovich, L. V., Ilanloo, A., & Mirinezhad, S. Y (2022, November). Fatty Liver Level Recognition Using Particle Swarm optimization (PSO) Image Segmentation and Analysis. In 2022 12th International Conference on Computer and Knowledge Engineering (ICCKE) (pp. 237-245). IEEE.

25. Lyashenko, V., Laariedh, F., Sotnik, S., & Ahmad, M. A. (2021). Recognition of Voice Commands Based on Neural Network. TEM Journal, 10(2), 583-591.

26. Al-Sharo, Y. M., Abu-Jassar, A. T., Sotnik, S., & Lyashenko, V. (2023). Generalized Procedure for Determining the Collision-Free Trajectory for a Robotic Arm. Tikrit Journal of Engineering Sciences, 30(2), 142-151.

27. Lyashenko, V., Kobylin, O., & Selevko, O. (2020). Wavelet analysis and contrast modification in the study of cell structures images. International Journal of Advanced Trends in Computer Science and Engineering, 9(4), 4701-4706.

28. Uchqun o‘g‘li, B. S., Valentin, L., & Vyacheslav, L. (2023). Image Processing Techniques as a Tool for the Analysis of Liver Diseases. Journal of Universal Science Research, 1(8), 223-233.

29. Sotnik, S., & et al.. (2022). Analysis of Existing Infliences in Formation of Mobile Robots Trajectory. International Journal of Academic Information Systems Research, 6(1), 13-20.

30. Yang, G., & et al. (2020). EANet: Edge-aware network for the extraction of buildings from aerial images. Remote Sensing, 12(13), 2161.

31. Zhu, X., & et al. (2023). NerVE: Neural Volumetric Edges for Parametric Curve Extraction from Point Cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13601-13610.

32. Kim, J., & et al. (2021). Multiple weld seam extraction from RGB-depth images for automatic robotic welding via point cloud registration. Multimedia Tools and Applications, 80, 9703-9719.

33. Wu, F., & et al. (2020). Information collection system of construction progress based on SLAM and edge extraction. In 2020 Chinese Automation Congress (CAC), IEEE, 2827-2832.

34. Waldner, F., & Diakogiannis, F. I. (2020). Deep learning on edge: Extracting field boundaries from satellite images with a convolutional neural network. Remote sensing of environment, 245, 111741.

35. Liu, H., & Zhang, Y. (2022). RM-Line: A Ray-Model-Based Straight-Line Extraction Method for the Grid Map of Mobile Robot. Applied Sciences, 12(19), 9754.

36. Xu, S., & et al. (2023). MDBES-Net: Building Extraction From Remote Sensing Images Based on Multiscale Decoupled Body and Edge Supervision Network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 519-534.

37. Wan, S., & et al. (2022). Edge computing enabled video segmentation for realtime traffic monitoring in internet of vehicles. Pattern Recognition, 121, 108146.