Use of image processing algorithms for mine originating waste grain size determination

The utilization of mineral wastes from the mining industry is one of the most challenging phases in the raw materials life cycle. In many countries, there are piles of mineral waste materials that date back to the previous century. There is also a constant stream of accompanying mineral matter excavated during everyday mine operations. This stream of waste matter is particularly notable for deep coal mining. Grain size composition of waste mineral matter is one of the most important characteristics of coal originating waste material. This paper presents the use of image analysis for the determination of grain size composition of rock material. Three methods for edge identification have been tested: gradient magnitude, multiscale linear filtering, and Statistical Dominance Algorithm (SDA). Images acquired in laboratory conditions were pre-processed using Gaussian, Median, and Perona-Malik filtration. The image was segmented using a classic watershed algorithm; as a reference, manually segmented images were used. The results show that the SDA algorithm was the best in determining the grain edges. Therefore, the sizes determined after the application of this algorithm were closest to the groundtruth. This method can be used for the assessment of the grain size composition of mineral waste material.


Introduction
E xcavation of raw materials is inherently associated with waste material generation. This generation of waste is particularly true in the case of the coal mining industry. Though there is strong pressure for the utilization of all material taken from underground, waste piles formed in the past still exist as part of the landscape in many less developed countries. The piles often contain material that potentially can be used in different branches of industry. It is common that fuel (hard coal) can still be recovered from the waste material. Other waste utilization possibilities include aggregating and additives for construction industry production. Regardless of the way in which waste is used, the determination of grain size is important. The processing technologies use differences in mineral matter grains properties for their separation and enhancement. Among several parameters characterizing both Run--Of-Mine and waste materials, the grain size and density are most commonly used for material differentiation. Some of separation methods are particularly sensitive to grain size [1,2]. It is also known that the grain sizes help to determine coal mining originated wastes self-ignition ability [3]. The proposed method is a fast and reasonably accurate way for the determination of this parameter that can help in controlling of the sorting and enhancement process and achieving quick characterization of waste material. The method helps in the thorough assessment of stored material grain sizes distribution and mineralogy. This technique would be helpful in the rapid determination of effort required for mineral matter separation and utilization. Therefore, an attempt was made to use an image processing and analyzing method for waste mineral material characterization.
The application of image analysis and image processing methods in characterizing grain sizes in bulk materials has been investigated by scientists for more than twenty years. Several of the studies focussed on delivering appropriate measurements of grain size distribution in sampled mineral material. There are different approaches presented, which vary widely in image acquisition methods and further processing. Several authors use dedicated equipment for rock (usually coal or ore) images acquiring. Most of the equipment is designed for achieving images of separated rocks using highly contrasting backgrounds. Attention has also been paid to minimizing shadows. As a solution, a backlit screen is commonly used [4,5]. Though this method is effective, it can hardly be used in industrial conditions. One such attempt was presented in [6], where falling material is observed by an outdoor digital camera using the sky as a bright background. Other image acquisition systems let the light source be placed next to the camera [7,8]. This approach can be easily adapted to online data acquisition on, e.g., a conveyor belt [9]. However, this method does not eliminate shadows as efficiently as backlight. Another problem considered is the detection of grains' boundaries. The images taken in laboratory conditions ensure that no grains touch one another. Detection of grains is then relatively simple; typically, the use of thresholding is quite sufficient [5]. The problem becomes more complicated in industrial conditions. Even photographing the rocks against the sky does not ensure grain separationsome of the rocks will overlap one another. While separating touching grains in that situation is potentially possible using algorithm presented in [10], it is difficult to achieve using images taken from conveyor belts or rock piles. In the case of image analysis of bulk rock materials, overlapping is unavoidable. The image analysis process usually begins with a form of pre-processing [8,11,12]. This step includes noise filtering, contrast/brightness adjustments, softening (or sharpening), and colour space transformations. Next, a set of methods is used to identify grains. Typically, a variety of techniques are used for the determination of an edge of grains. After that step, segmentation is performed, allowing differentiation between rocks and measurements of their features. Edge detection is a challenging task, and there have been many different approaches to this problem reported in the literature. Among the methods, the most important are gradient magnitude analysis [13], multiscale linear filtering [14], morphological transformations [8], colour, and texture features classification using machine learning methods [15e17]. The final step is segmentation, where thresholding and watershed algorithms are commonly used [14,18]. Occasionally, the edge detection step is omitted, and a watershed algorithm is applied directly after image pre-processing.
This paper describes usage of selected image processing methods for the determination of grain edges and sizes in the pictures of coal originating wastes. A comparison of three methods are presented: the gradient magnitude, multiscale linear filtering using a Hessian matrix, and Statistical Dominance Algorithm (SDA) [19]. The influence of the image pre-processing method is also considered. This study's results are compared to the ones achieved by experts manually described waste grain boundaries.

Sample preparation and image acquisition
The coal originating waste sample was taken from the "Rymer" waste dump (Poland). The waste dump was formed with barren rock (clay slate) that was separated from coal during a wet enrichment process. The sample material (approx. 20 kg) was comprised mostly of mudstone and sandstone mixture. The minerals were present in their native and thermally processed form (Fig. 1). The sample was sieved, and both minerals were manually separated. The clay slate was later used for further tests as being more representative and commonly present at mining industry waste dumps. The aggregate was manually agitated to achieve uniform distribution of all grain sizes in the sample bulk. The material was placed in metal form (370 Â 320 Â 100 mm), and a flat surface of the rock was formed. The surface was similar to the surface commonly observed in rock waste piles. Pictures of the surface were taken with a Nikon D80 digital camera under ambient light conditions. There were ten images registered with a resolution of 3872 Â 2592 pixels. The images were saved in a lossless format to maintain high quality. The area covered by the photograph was equal to 350 Â 235 mm. The area represented by one pixel is square, measuring 0.09 Â 0.09 mm.

Image pre-processing
Every image was split into 16 smaller images for further processing. The final resolution of analysed images was 968 Â 648. As the edge detection and grain size estimation does not include required colour information analysis, all pictures were converted from RGB to HSV colour space. Only the luminance channel was used for further analysis. Histograms for grey images were corrected using Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm [20]. The algorithm performs histogram equalization within the neighbourhood (called tile) of each pixel. The result's contrast is limited by clipping the original histogram to the given value. The pixel's vicinity size is a parameter of the algorithm. Both equalized and unequalized versions were filtered by a Median, Gaussian, and Perona-Malik filter [21]. The Median filter replaces the pixel's value with the median of the values of pixels within its neighbourhood. The Gaussian filter smooths the image due to its convolution with the Gauss function. Such blurring allows eliminating tiny structures (e.g. noise) while preserving large objects. The Perona-Malik filter uses anisotropic diffusion for image filtering. The pixel values are considered values of the function being transformed by the diffusion equation. However, the filtering tries not to smooth the prominent edges visible on the image, by coupling the diffusion coefficient with the magnitude of the image gradient (Eq. (1)). The CLAHE filtering was performed, with tile sizes varying from 2 to 16 and clipping limits ranging from 1 to 8. Median filters used square structural elements with sizes varying from 3 to 11. The standard deviation of the Gaussian filtering varied from 0.75 to 2.0. All filtered images, as well as source images, were used as an input for edge detection methods. The changes of statistical parameters of the images after filtration are depicted in (Table 1, Fig. 2).

Grain edges detection
It is common to characterize edge pixels as having a large local contrast. The difficulty with edge detection in the case of aggregate images is in the nature of the material. Mineral grain can be composed of parts having different colour and intensity. It is not clear what criteria should be used to distinguish the real grain edge from artifacts caused by shadows, mineral structure, and inner edges (e.g., results of crushing). Therefore, an attempt was made to determine how wellknown methods for edge detection perform for grain boundaries detection. Three methods were used for that purpose: gradient magnitude, multilinear filtering in gradient space, and SDA. Each algorithm was tested with different parameters and the best results were taken for comparison. In the following sections, each method is summarized.

Gradient magnitude
This method attempts to identify the areas in the image where colour or grey level changes are most prominent. Calculation was performed using a Sobel operator for calculating partial derivative in x and y direction. The length of gradient vector is a measure for edge identification.
The I(x, y) denotes the image grayscale value at coordinates x and y, the v$ vx and v$ vy are partial derivative operators in x and y direction. In spite of its simplicity, this method is used in many applications and provides satisfactory results. After the gradient JOURNAL OF SUSTAINABLE MINING 2020;19:221e229 magnitude is calculated, a thresholding operation has to be performed to discriminate the desired edge characteristic. The result of application of gradient magnitude method to the sample image is depicted in Fig. 3. One can observe that most grain edges have been detected. However, there are artifacts present. Further processing should include morphological operator application and removal of small objects. Leaving only large structures enables easier segmentation.

Multi-scale linear filtering
Multi-scale linear filtering uses Hessian matrix properties to identify linear structures in the picture [22]. The method was previously used in medical image analysis [23], but its application in grain edges detection is also mentioned in [14]. The first step is the calculation of Hessian matrix:   Hessian contains information on changes of second derivatives of the image. The eigenvectors of the matrix show directions in which the second derivatives (and therefore curvature of the depicted object) changes most. The eigenvalues l 1 , l 2 contain information on the magnitude of this change. For linear structure identification, the greater absolute value of l 1 and l 2 is taken. As the line structures in the image can have different width, input image is filtered by the Gaussian function. Each filtering reveals linear details at certain levels. The complete procedure is formulated as follows. The input source images are filtered using a Gaussian filter with increasing standard deviations, s i X [s 1 , s 2 , …, s n ]. For each filtered image, Hessian matrix elements for every pixel are calculated. Next, the Hessian's eigenvalues are determined, and the eigenvalue with a larger absolute value is stored as a result for the spatial scale defined by s i : The resulting image is composed of maximum values for each pixel calculated for all space scale images: Detecting of the multi-scale linear structures is performed, with s i varying from 0.25 to 3.25 with step 1.0. The resulting images are binarized using thresholding. The sample result of processing is presented in Fig. 4. After binarization, the artifacts were filtered out. The size of the pixel complex was used as a determining factor of filtering procedure.

Statistical Dominance Algorithm
The Statistical Dominance Algorithm is based on differences in grey level between selected point and its neighbourhood. The new value of the pixel equals the number of pixels, which has a value greater than the selected point in the input image. Only pixels within the neighbourhood are taken into consideration. It is also possible to use a threshold value e only pixels of value greater than  the selected one and greater than the threshold value are counted. Despite relative simplicity of the algorithm, it demonstrates that its capability of edge detection is at least as good as the differential, more complicated methods. Formally, the algorithm can be defined as follows: where p(x,y) is a pixel from the source image, p b (x b , y b ), and is a pixel belonging to neighbourhood B(x, y) of point p(x, y). The p'(x, y) is the resulting pixel, and t stands for grey level threshold. The overall process of image transformation is the same as in previous two methods. After SDA image is obtained, it is normalized and thresholded (Fig. 5). Any artifacts that are present are removed, and image is processed by segmentation and measurements procedure.

Segmentation and measurements
The processed images were segmented using watershed algorithm. The segmentation was performed with varying parameters; the best achieved result was used for further analysis. For each segmented grain, the maximum and minimum diameter were calculated. The average of the two diameters was taken as a measure of grain size. Their ratio, conversely, was used as a shape elongation measure. Both parameters are important in mineral waste processing technologies. The results achieved using the tested methods were compared with the manually segmented image. Comparison results were related to hand made segmentation.

Results and Discussion
The edges detected by tested method were compared with the edges that were manually detected by trained personnel (Fig. 6). It can be easily observed that multiscale liner filtering and SDA perform better than the gradient magnitude method. However, even thresholded images still show significant numbers of artifacts and fail in identification of some of the rocks' grains. It should be noted that the failure is present only on parts of images, where light conditions are notably poor, and even trained personnel had difficulty in grain boundaries determination.
The images were segmented using watershed algorithm with marked starting points. The manual selection of grains resulted in 128 identified grains. The average perimeter of the grains was 25.4 mm, and the range was 3.1 mm to 115 mm. Using watershed on gradient magnitude processed image led to significant over-segmentation. The mean diameter decreased to nearly 12.6 mm, and the maximum size was 111 mm. The lowest value was 4 mm. There were nearly 500 grains identified. The different results were obtained by segmenting the images processed by a multi-scale linear filter. The over-segmentation was lower than in the case of gradient magnitude, but still, almost 400 grains were detected. However, the diameter measured was similar in value for the lowest (4 mm) and mean size (12.3 mm). The highest value was significantly higher and was equal to 126.5 mm. The best results were obtained using the SDA processed images. The mean diameter was 18.8 mm, the lowers and the biggest values 7 mm and 108.5 mm. The image was still over-segmented, but with approximately 200 grain identified, it was closest to the manually achieved records. The comparison between the methods was presented (Fig. 7).
The presented algorithms, though using different methods, aim at identification of the boundaries between grains of rock material depicted in the image. The close investigation reveals significant differences between the final result of gradient magnitude, multi-scale linear filter, and SDA approach. The gradient magnitude and SDA were able to properly identify the long segments of the borders. The multiscale linear filter gave more fragmented grains outlines. At the first sight, the longer segments of borders should lead to better segmentation of the rock grains. The results show it is only true to a certain level. The better segmentation achieved using multi- scale filter than gradient magnitude, suggests that the length and compactness of the identified boundaries have only limited influence on final result. All methods, apart from recognizing the boundaries between grains, identified some of the edge-like structures appearing within the grains. They are caused by the complex nature of the rock grains itself (inhomogeneity, internal mineral structure, shadows caused by surface unevenness). Depending on the method, they reveal a different nature. While the gradient magnitude produced mostly the short lines or arches, the multi-scale and SDA filter resulted in rather dot-like structures. It seems, that the latter cause much less confusion for watershed algorithm used for segmentation. Therefore the boundaries recognized by SDA, characterized by long segments of boundaries between grains and only dot-like artifacts within the grains give the best segmentation result. It is recommended to use the SDA algorithm for grains' boundaries detection.

Conclusions
This study attempted to use image processing methods to assess grain sizes of rocks gathered from a coal originating waste dump site. The material was taken from the "Rymer" pile. Rocks were placed in standardized metal forms. Images were taken using an ordinary camera. For further processing, the images were converted into an HSV colour space. Only the V channel was taken for further processing. Several filters were tested for image pre-processing. Among the filters, a Median, Gaussian, and Perona-Malik filter were the most promising. Finally, the best results were obtained using a Median filter. The pre-processed images were further transformed using edge detection algorithms. Gradient magnitude, multi-scale linear filtering, and statistical dominance algorithm were used for that purpose. A reference selected image was processed by trained personnel. Next, edge detection of the segmentation step was performed using a classic watershed algorithm. The best results for edge detection algorithms were obtained from SDA-processed images. The boundaries produced by SDA were composed of relatively long segments while the internal complexity of the grain structure manifested itself in the dot-like objects, allowing the successful application of watershed algorithm. Although in comparison with manually processed images, this method reveals significant over-segmentation, the results provide a good assessment of rock grain sizes. The results and the SDA method can be used in development of a method of automatic assessment of grain sizes upon the image information.

Conflicts of interest
None declared.

Ethical statement
The authors state that the research was conducted according to ethical standards.

Funding body
This research received no external funding.