Application of Entropy Method for Estimating Factor Weights in Mining-Method Selection for Development of Novel Mining-Method Selection System

Mining-method selection (MMS) is one of the most critical and complex decision-making processes in mine planning. Therefore, it has been a subject of several studies for many years culminating with the development of different systems. However, there is still more to be done to improve and/or create more efficient systems and deal with the complexity caused by many influencing factors. This study introduces the application of the entropy method for feature selection, i.e., select the most critical factors in MMS. The entropy method is applied to assess the relative importance of the factors influencing MMS by estimating their objective weights to then select the most critical. Based on the results, ore strength, host-rock strength, thickness, shape, dip, ore uniformity, mining costs, and dilution were identified as the most critical factors. This study adopts the entropy method in the data preparation step (i.e., feature selection) for developing a novelMMS system that employs recommendation system technologies. The most critical factors will be used as main variables to create the dataset to serve as a basis for developing the model for the novel-MMS system. This study is a key step to optimize the performance of the model.

Application of entropy method for estimating factor weights in mining-method selection for development of novel mining-method selection system 1. Introduction T he success of a mining project relies heavily on the feasibility of the adopted mining method that maximises profits and recovery of mineral resources while minimising environmental impacts. For this purpose, different mining methods are used by mining engineers to extract or recover mineral resources from the earth. Surface and underground mining are the two most common types of mining methods. During the mine planning and design processes, the selection of the best mining method or combinations of multiple mining methods is the most critical and complex decision-making task. Moreover, the adoption of a certain mining method can be an irreversible decision owing to the high costs involved in changing or replacing the mining method during the production stage [1]. Therefore, the mining-method selection (MMS) task requires the engagement of experienced mining engineers. Additionally, this process is considered complex and somewhat problematic because the selection of the most feasible mining method requires the consideration of several factors, including historical, social, and cultural factors, mechanical and physical characteristics of the orebody, geological and geographical conditions, as well as technological, economic, and environmental factors. Moreover, owing to the complexity of the physical characteristics and geological conditions of an orebody deposit, the extraction of the entire orebody by using a single mining method is almost impossible [1]. The factors that highly influence the selection of the surface and underground mining methods are categorised as follows [2,3]: Physical characteristics of the orebody deposit (orebody geometry): the size of the orebody (height, width, and thickness), orebody shape, orebody dip, and depth of the orebody below the surface. These factors are considered to be the most critical in choosing between surface and underground methods because they affect the entire mine design and production. Geomechanical properties, and geological and hydrologic conditions: rock material properties (strength, deformation, and weathering characteristics), grade distribution/ore uniformity, mineralogy, and petrology. These factors include the mechanical and structural geological compositions of the orebody and host rock. They play a significant role in the selection of different classes (i.e., unsupported, supported, and caving methods) of underground mining methods as well as in the selection of the ground support. Economic factors: comparative capital and mining costs of suitable methods, reserves (tonnage and grade), mine life, production rate, and productivity. These factors play an important role during the final decision-making process of MMS, determining the feasibility of the methods based on financial and economic analyses. Technological factors include recovery, selectivity, dilution, flexibility of the method to changing conditions, mechanisation or automatisation, and labour intensity. These factors are mostly related to the effects of mining methods on subsequent operations, such as processing requirements, treatment, and smelting. Environmental considerations: subsidence, stability of openings, and health and safety. These factors are interconnected to social, political, historical, and geographical factors, and affect the rejection or acceptance of the method in a certain location.
Over many years, MMS has been the focus of numerous studies. The first MMS systems were developed during the 1970s and 1980s [4,5]. The first systems were called qualitative systems, as they were basically flowcharts that served as guidelines for selecting the most suitable mining methods. Subsequently, quantitative systems were introduced to improve the qualitative systems. Quantitative systems determine the most feasible mining method by numerically ranking the influencing factors, which are then summed. The best methods are those with higher ranks. However, the relative importance of the influencing factors is not considered in these systems; thus, multi-criteria decisionmaking (MCDM)-based MMS systems were introduced. In MCDM-based systems, the feasibility of the methods is assessed based on the relative importance of the influencing factors that are often measured subjectively, i.e., based on the direct subjective opinions and professional judgements of mining engineering experts. The applicability of MCDM techniques in MMS has been proven to be effective owing to their ability to solve problems involving several and conflicting criteria, and MMS is classified by several conflicting factors [6e10]. Moreover, the most difficult and complex task in MCDM is to determine the relative importance of the criteria.
As it is evident, several studies have been done in the field of MMS culminating with the development of different systems. However, there is still more to be done to improve and/or create systems that are more efficient and deal with the complexity caused by many influencing factors. This study adopts the entropy method in the data preparation step (i.e., feature selection) for developing a novel mining-method selection (novel-MMS) system that employs recommendation system [11] technologies. Recommendation systems use different machine learning algorithms to generate models aimed to make recommendations of the most relevant items to the users based on user(s) historical information. Thus, recommendation systems find an important application in different business areas and, have proven to improve and/or optimize the decision-making process and quality, hence, boost profits and save costs [12]. Data preparation is one of the most complex, troublesome, and critical steps in the development of a recommendation system. This step consists of readying a dataset that will be used as a base for the training and implementation of the models, in which, feature selection is one of the processes involved. Many factors need to be considered during MMS and, using all factors as input variables in the dataset would negatively affect the performance of the novel-MMS model; hence, the need to identify and select the most critical factors (i.e., features/input variables). In machine learning different methods are used for feature selection to reduce the number of features in a dataset by selecting the most critical features, thus, improving the performance of the prediction models and reduce computation time [13,14]. However, these methods usually require a dataset with a big amount of historical data to effectively analyse features correlation and identify the most critical features. Getting a fair amount of information about mining projects, specifically related to MMS influential factors is one of the most difficult and challenging aspects, thus, a huge limitation of the study. Hence, this paper presents the application of the entropy method for feature selection. The entropy method is applied to assess the relative importance of the factors affecting MMS by determining their objective weights. Then, based on objective weights, unimportant factors are reduced and the most critical are selected. The most critical factors will be used as main variables to create the input dataset to serve as a basis for developing the model for the novel-MMS system.
The entropy method is considered suitable owing to its advantage of not requiring a huge amount of historical data to analyse the relative importance of the factors (i.e., features correlation). Furthermore, this method has the advantage of determining criteria (i.e., factors or features) weights without direct involvement (i.e., opinion or judgement) of decision-makers [15]. The objective weights of the factors are determined based on a decision matrix containing raw information, i.e., rates measuring the performance of the mining methods with respect to various factors. The information in the decision matrix is obtained subjectively based on literature. The use of the entropy method prevents any bias that could be caused by direct subjective decisionmaking in estimating MMS influencing factor weights, which, and may thus affect the accuracy of the results.
The remainder of this paper is organised into five main sections. Section 2 presents a review of the literature on different MMS systems, including the qualitative, quantitative, and MCDM-based systems. In Section 3, we introduce MCDM techniques and explain the application and procedures of the entropy method to estimate multi-criteria weights. Section 4 demonstrates the application of the entropy method to estimate the weights of MMS influencing factors. The results of the application of the entropy method are presented in Section 5. Finally, the discussion and concluding remarks are presented in Section 6.  [4,5].
The systems proposed by Boscov and Wright, Morrison and Laubscher can be applied to underground mining methods but differ in the category of factors considered in each system. Boscov and Wright proposed a system based on the physical and mechanical characteristics of the orebody (i.e., thickness, orebody dip, and strengths of the ore and host rock). The system suggested by Morrison is based on the orebody width, underground mine support types, and strain-energy accumulation. Laubscher proposed a system based on geotechnical parameters (rock-mass classification) aimed at mass underground mining methods [5].
The system proposed by Hartman is relatively similar to that proposed by Boscov and Wright, which is based on the physical characteristics of the orebody and the mechanical characteristics of the ore zone (i.e., shape, dip, size, and strength of the orebody) but targets both surface and underground methods [4].

Quantitative MMS systems
In 1981, Nicholas developed the first quantitative MMS system based on orebody geometry, grade distribution, and the mechanical characteristics of the orebody and host rock to select the most suitable mining methods [5]. In this system, numerical ranks are assigned to all factors to indicate the suitability of each factor for each mining method. Then, the ranks are summed for each mining method, and the method with the highest rank is selected as the most suitable method and submitted for economic evaluation [4,5]. Furthermore, to improve Nicholas' approach, an MMS tool was developed by the University of British Columbia (UBC) [16]. The UBC tool is a modified version of Nicholas' approach, with the introduction of some mechanical properties and ranks as well as modification of most of the factor ranks. Although the UBC approach is the latest and most common quantitative system, it emphasizes underground stoping methods and best represents Canadian mine design practices. Moreover, in both Nicholas' and UBC approaches, the relative importance of the factors is not considered, implying that all factors have the same degree of importance.

MCDM-based MMS systems
Currently, the trend involves the application of MCDM techniques in MMS. As several factors are related to MMS, the formulation of definite criteria for selecting methods that can simultaneously satisfy all conditions of the mining becomes complicated [17]. Therefore, several researchers developed MMS methodologies by applying MCDM techniques, wherein the relative importance of the factors is considered. Bitarafan and Ataei [17] applied fuzzy decision-making tools (fuzzy dominance and fuzzy multiple attribute decision-making methods) to select the best mining method for anomaly No. 3 of the Gol-Gohar iron mine, where the weights of criteria (i.e., influencing factors) and alternatives (i.e., mining methods) are determined in a fuzzy environment based on the most suitable mining method, i.e., block caving. Ataei et al. [16] explored the application of the analytical hierarchy process (AHP) technique to develop a suitable mining method for the Golbini No. 8 deposit. Their technique was applied to determine criteria weights as well as the best alternative, and therefore AHP was found to be a unique model in that it could identify multiple criteria, minimal data requirement, and minimal time consumption. Namin et al. [7] discussed the application of a decision-making tool based on the fuzzy technique for order of preference by similarity to ideal solution (TOPSIS) to develop the MMS tool for the Gol-e-Gohar anomaly No. 3 and Chahar Gonbad deposit. In this case, the open-pit method was identified as the best method for the deposit and the systematic evaluation of fuzzy TOPSIS of MMS was determined to reduce the risk of a poor choice. Alpay and Yavuz [8] developed a tool based on AHP and Yager's techniques to develop a computer program to analyse underground MMS problems for the EskisehireKaraburun chromite ore. The computer program could also enable decision-makers to perform sensitivity analyses after selecting the best method to observe the rate proposed method according to criteria weights. Azadeh et al. [1] developed a modified version of Nicholas' approach by using a fuzzy analytical hierarchy process (FAHP) to select the most appropriate mining method for the anomaly of the Choghart iron mine. In their approach, FAHP was applied to determine and modify criteria weights according to Nicholas' approach, and thus determine the most suitable method considering these criteria weights. Bogdanovic et al. [6] implemented an integrated approach that employed the AHP and preference-ranking organisation method for enrichment evaluations (PROMETHEE) to select the most suitable mining method for the Coka Marin underground mine. In their approach, AHP was used to assign criteria weights, while PROMETHEE was used to complete the ranking of the alternatives; sublevel caving was identified as the most suitable method. Shariati et al. [6] developed an integrated model based on FAHP and TOPSIS to select the optimum mining methods for the Angouran Mine; criteria weights were determined based on FAHP, and TOPSIS was applied to analyse the feasible alternatives, and the alternative with the highest score was selected followed by sensitivity analyses to determine the influence of criteria weights. The advantage of MCDM-based MMS methodologies is the consideration of the relative importance of the factors that are mostly determined subjectively. Furthermore, most MCDM-based methodologies are based on a specific case study, wherein the opinion and judgement of mining engineer experts are crucial to determine the subjective weights of the factors.

Multi-criteria decision-making (MCDM) methods
MCDM is a branch of operations research (OR) that attempts to solve real-life problems that involve different alternatives by considering several conflicting criteria to achieve specific goals. MCDM attempts to solve problems of selecting an alternative from a set of alternatives under several criteria, typically aiming at a single goal [18]. There are different MCDM techniques, all aiming towards breaking down complicated decisions into smaller decisions that can be analysed individually and then recombined into a weighted-sum utility score [15].
To overcome these problems, the decision maker's team performs the decision-making process based on the hierarchical structure model, wherein the first step is to define the goal and then identify the alternatives for achieving the goal and the criteria used to compare the alternatives [19]. Based on the hierarchical structure model shown in Fig. 1, a decision matrix (DM) composed of a set of m alternatives evaluated based on n decision criteria and the respective decision data were set up. During this process, criteria are weighted subjectively or objectively [15].
MCDM techniques evaluate the performance of different alternatives based on the criteria weights, wherein the best alternative is selected as the one with the highest performance rates. The weights of each criterion express their relative importance for the decision. Typically, decision-makers may define and assign subjective weights to each criterion based on their intuition and judgement, most commonly using methods, such as the utility preferences function, AHP, and fuzzy version of classical linear weighted averages [18]. However, often, decision-makers have conflicting views on the values of weights or are simply uncertain of the relative importance of each criterion. In this case, the entropy method [20] is applied to determine the objective weights of each criterion based on the DM data, wherein the preferences or judgement of decision-makers are completely or partially unavailable or even not required [21]. The entropy method also called Shannon's entropy [20], is a technique applied in MCDM to estimate objective criteria weights.

Entropy method to estimate criteria weights
The term entropy is applied in different scientific fields (e.g., physics, chemistry, biology, mathematics, psychology, and information theory); in information theory, this term plays an important role in measuring the uncertainty associated with random phenomena of the expected information content of a certain message [22]. The MCDM entropy method is applied to measure the relative importance of criteria based on DM generated from the hierarchical model. Fig. 2 illustrates the flowchart of the overall procedures of the entropy method, wherein the first step involves the generation of the DM of the problem as follows: ð1Þ where x ij is the criteria/sub-criteria rate, n is the number of criteria/sub-criteria, and m is the number of alternatives.
In the second step, the DM data are normalized by applying Equation (2) to make all the criteria comparable by transforming different scales and units among several criteria into common measurable units [21]: where r ij is the normalized criteria/sub-criteria rate. Then, the entropy (E j ) values are computed by applying Equation (3). The entropy value measures the degree of uncertainty between the set of alternatives in the DM when no preference among criteria can be established [15,21,23].
where r ij ln(r ij ) ¼ 0 if r ij ¼ 0 and h is the entropy constant. The fourth step is to calculate the diversity (D j ) or the degree of diversification based on the entropy values using Equation (4). Diversity measures the level of diversity of the evaluation of a set of alternatives for the same criterion [10,21,24]. In other words, diversity measures the variation or the degree of dispersion between the rates of different alternatives for the same criterion. The higher the variation or dispersion the higher the diversity, and, the more valuable is the criterion: Finally, the relative importance of the criteria, which are measured by the objective weight, is calculated based on Equation (5). The relative importance of the criteria is directly related to the amount of data essentially provided by a set of alternatives for the same criterion [21,23]: where w j is the degree of importance of criterion j or object weight of criterion j.

Application of the entropy method to estimate factor weights for mining method selection (MMS)
As MMS is a decision-making process that involves several conflicting factors (or criteria) for the selection of different mining methods (or alternatives), it is an appropriate method, considering the complexity of this task. For this reason, MCDM techniques have been applied in the MMS process, and several researchers [1,6e10], [17,18,25,26] have proven the advantages and applicability of different MCDM techniques. In MCDM the most complex task is to define the relative importance of the criteria which is commonly defined subjectively. That is, criteria weights are defined based on direct opinion and judgement of decision-makers or mining engineer experts. However, when direct opinions or judgement from decision-makers are unavailable (or not required), objective weights are considered the best option. Therefore, in this study, the entropy method was applied as a tool for feature selection, i.e., to analyse and select the most critical factors in MMS. The entropy method measures the relative importance of the factors influencing MMS by calculating their objective weights.
Decision matrix (DM) Table 1 present the DM created based on the approaches proposed by Miller et al. [15], developers of the UBC MMS tool, and Hartman and Mutmansky [2], who created a guideline base to compare different surface and underground mining methods. The DM is a classification system for different surface and underground mining methods, and it provides guidelines to select the most suitable mining methods; its main characteristics include the classification of factors for the MMS according to each mining method. Since the study aims to assess the relative importance of factors involved in the selection of both surface and underground mining methods, 12 mining methods or alternatives (A) included in both surface and underground methods were considered to create the DM: block caving This factor is related to the properties of the rock surrounding the ore deposit, measuring the hardness or toughness of the rock against permanent deformation. The strength of the rock (host and ore) can be from very weak, weak, fair, strong and very strong. Understanding host rock strength is crucial in MMS to ensure the safety and stability of openings (in surface and underground mining). Host rock properties play a huge role in the selection of different classes of underground mining methods (i.e., supported, unsupported and caving). Furthermore, to determine pit slopes angle (in surface mining) and the support systems (in underground mining).

Ore strength (c2):
Ore strength is related to the mechanics of the ore or even ore properties. In the selection of both surface and underground mining methods is crucial to understand the properties of the ore to determine the extraction methods (i.e., mechanical or blasting), the support systems, for equipment selection, and the stability of openings.

Ore uniformity (c3):
Ore uniformity is a geological factor corresponding to ore grade distribution throughout the ore deposit. Ore uniformity is determined based on ore grade variation from the average grade within the ore deposit. The distribution of the ore can be variable/erratic, gradational and uniform. It is variable when the grade values within the deposit change radically over a short distance and don't show any perceptible pattern in their changes. Gradational when grade values at any point within the deposit have zonal characteristics, and the grades change gradually from one to another. Uniform when grade values at any point within the deposit doesn't vary significantly from the average grade. It is important to understand the distribution of the ore to select the most suitable mining method to ensure high selectivity and recovery and low dilution. Additionally, this factor is directly related to the selectivity of

Depth (c4):
This factor corresponds to the depth of the ore deposit relative to the surface ground. An ore deposit can be shallow (<100 m), intermediate (100e600 m) and deep (>600 m). Depth is usually a key factor to select between surface and underground methods. For surface, deposits depth is applied to decide between casting the waste (in open-cast) or haulage the waste to dump sites (in open-pit) as well as applying solution mining. Additionally, some underground methods are less suitable for deep deposits owing to the limited depth capacity.

Shape (c5):
Shape refers to the form of the ore deposit which can usually be tabular, equidimensional/massive and irregular. Tabular deposits extend at least hundreds of meters along two dimensions, and substantially less along a minor dimension. Equidimensional/massive deposits have all dimensions in the same order of magnitude. In irregular deposits, the dimensions vary over short distances. It is important to understand the ore deposit shape for mining methods selection as some methods (i.e., longwall, open cast, room and pillar) are more suitable for tabular deposits than others.

Dip (c6):
The ore deposit dip is the angle of inclination of a plane measured downward, perpendicular to the strike direction. An ore deposit can be flat (<20 ), intermediate (20e55 ) and steep (>55 ).
The dip is important in the selection of both surface and underground mining methods. In surface mining, the dip is used to decide between open-cast (in flat deposits), open pit or solution mining (in intermediate or steep). Moreover, some underground mining methods (i.e., shrinkage stoping, sublevel stoping, stull stoping and caving methods) are more suitable to exploit intermediate or steep deposits because they rely on gravity for material flow and cannot be applied in flat deposits.

Thickness (c7):
This factor refers to one of the three dimensions of the ore deposit. The thickness can vary throughout ore deposits being very narrow (<3 m), narrow (3e10 m), intermediate (10e30 m), thick (30e100 m), and very thick (>100 m). The thickness of the ore deposit determines the effectiveness of some mining methods, as some methods (i.e., open pit and caving methods) are less effective in narrow deposits. Additionally, this factor affects the mechanization (and equipment selection) and the selectivity of certain mining methods.
Health and safety (c8) and stability of openings (c9): The stability of openings is one of the factors that determine the health and safety of mining operations. The health and safety of the mining operators should be a top priority objective preventing hazards that can be caused by unappropriated mining methods for a particular ore deposit. Therefore, it is important to always consider mining methods with high stability of openings providing good health and safety conditions.

Recovery (c10) and dilution (c12):
Recovery is the capability of a mining method to completely extract valuable ore from the deposit. Ore recovery is defined as the percentage of mineable reserves extracted in the mining process. On the other hand, dilution is the waste material mixed with ore during the extraction which is then sent to the processing plant. Dilution is the percentage of the waste mined and sent to the processing plant over the combined total ore and waste material milled. Recovery and dilution are usually interrelated, as some mining methods with high recovery usually involves contamination of the ore from the waste. Some mining methods have low recovery due to the need to leave the ore as structural support, whilst providing moderate to low dilution.

Flexibility (c11) and selectivity (c13):
Flexibility refers to the ability of a mining method in adapting to changes related to mining conditions, market price and technology throughout the mine life. Selectivity refers to separate extraction of ore and waste (or gangue), ensuring complete extraction of the ore with low dilution. Flexibility marries well with the selectivity of a mining method to determine the success of a project. The more flexible and selective, the more effective is the mining method.

Depth capacity (c14):
This factor measures the capability of the mining method in terms of ore deposit depth. Mining methods with limited depth capacity (i.e., open-pit, open-cast, solution mining, room and pillar, stull stoping and shrinkage) are not suitable to extract deep ore deposits, hence, the importance of considering depth capacity in MMS.

Development rate (c15):
Mine development rate is the time (or speed) spent to undertake operations (i.e., tunnelling, sinking, crosscutting, drifting, raising, stripping, construction of mine infrastructures, etc.) that prepare the mine for ore extraction. This factor directly affects the capital investment because the slower the development rate the higher the capital costs or investment. Hence, the importance of considering this factor during MMS.

Productivity (c16):
Productivity is the measure of the efficiency or performance in the mine, in terms of how well/ smart the inputs (labour, materials, equipment, capital investment, resources) are converted into outputs (gross output, value-added). This factor involves most of the parameters used to measure the efficiency of certain mining methods. Therefore, is crucial to consider productivity during the MMS process.

Ore grade (c17):
The grade is used to measure the quality of an ore deposit, the higher the grade the more valuable is the deposit. It is important to consider this factor during the MMS process to ensure the efficiency and effectiveness of mining operations. Mining methods with high operating costs are usually applied to highgrade deposits in order to be economic. Moreover, large-scale mining methods may be economically appropriate for low-grade deposits.
Mining costs (c18) and capital investment (c20): Mining costs are the expenses (mine development, rehabilitation, exploration and grade control activities, material and utility handling, maintenance, and labour costs) resulting from all operations or activities necessary to extract the ore. Mining costs are usually measured in terms of the money necessary to mine a tonne of material (ore and waste). While capital investment is the amount of money necessary to invest in the mining project in order to pursue the objectives (growing operations and generate revenue). It is crucial to consider these factors during the MMS process, and, usually, underground mining methods requires high capital investment.
Production rate (c19): The production rate corresponds to the quantity of material (ore and waste) extracted per hour, day, month, and year. The production rate of a mine highly relies on the selected mining method, thus, the need to consider this factor during the MMS process. Usually, large scale mining methods have a higher production rate and low-scale methos have otherwise.
In the DM, each row describes an alternative (A), and each column describes the performance of each alternative against each criterion (c). In addition, the DM is composed of qualitative values, most of which are presented in the qualitative classification system.
However, the entropy method is more effective and accurate for quantitative criteria values, wherein some or all pertinent decision data are available [15]; hence, the qualitative classification values must be converted into quantitative values. For this, an appropriate weighting system was applied, as shown in Fig. 3, which is composed of 10 points, from 0 to 9. First, the qualitative classification of the factors belonging to the mechanical properties and physical characteristics of the orebody is transformed into an adequate qualitative classification to be compatible with the weighing system, as presented in Table 2. Then, the weighing system depicted in Fig. 3 is applied to convert all the qualitative values in Table 1 to quantitative values, resulting in a numerical DM, as presented in Table  3.

Results
By applying Equation (2), the values in the original DM in Table 3 are normalized, resulting in a normalized matrix, as presented in Table 4.
Then, by applying Equations (3)e(5), the entropy values (E j ), diversity (D j ), and objective weights (W j ) are generated, as presented in Table 5. The entropy is indirectly related to the objective weights and is typically measured from 0 to 1. Therefore, the closer the entropy value is to 1, the higher the level of uncertainty and the smaller the objective weight of that criterion. Additionally, diversity is directly related to the objective weight; therefore, the higher the diversity in a criterion, the higher the objective weight of the same criterion. The objective weights reflect the relative importance of each factor (or criterion) in the selection of the 12 mining methods (or alternatives). In this case, the results show that the factors possess a different degree of importance, with a few more important than the others. Furthermore, mechanical properties, such as the strengths of the ore and host rock, have the highest diversity, and thus the highest degree of importance among all factors. Environmental considerations, such as health and safety and the stability of openings have the lowest diversity, i.e., the lowest degree of importance among all factors.
The results from the Entropy method, emphasizes the different level of impact that the twenty factors have in the selection of the twelve mining methods. Furthermore, to identify and select the most critical influential factors, the deviation concept was then applied. The deviation was applied to determine factors with the highest impact in MMS, i.e., the most critical factors. The deviation of each criterion weight from the mean weight value is calculated as follows: where w j is the weight of each criterion, w is the mean weight of the criteria set, and n is the number of criteria. The overall mean weight (w) was 0.05. Based on this mean weight, the deviation of each criterion weight from the mean weight was calculated. Fig. 4 depicts the results of the factors with the lowest and highest levels of impact in MMS based on the deviation concept. Moreover, based on the deviation concept, the criteria with an objective weight smaller than the mean weight produce negative deviation values and are considered to have the lowest level of impact. Furthermore, criteria with an objective weight higher than the mean weight produce positive deviation values and hence are considered to have the highest level of impact on MMS. Therefore, criteria with higher weights than the mean weight, and with the smallest entropy and the highest diversity, were identified and selected as those with the highest level of impact. In this case, eight factors were identified, where ore strength had the highest weight of 0.132, followed by host-rock strength, thickness, shape, dip, ore uniformity, mining costs, and dilution with weights of 0.115, 0.104, 0.100, 0.072, 0.068, 0.061, and 0.057, respectively.

Discussion and conclusions
The entropy method is commonly applied in decision-making problems to determine objective criteria weights for evaluating the performance of different alternatives and selecting the optimal one. Therefore, in this study, decision making was performed without the direct involvement of decision-makers. The entropy method was applied to assess the relative importance of the factors for MMS by determining their objective weights. Then, based on these objective weights, factors, such as orebody strength, hostrock strength, orebody thickness, orebody shape, orebody dip, ore uniformity, mining costs, and  (Table 1) into qualitative values in DM (Table 3), where the values 0, 1, 3, 5, 7 and 9 are described and 2, 4, 6 and 8 stand for intermediate value.   c1  c2  c3  c4  c5  c6  c7  c8  c9  c10  c11  c12  c13  c14  c15  c16  c17  c18 c19 c20 A1 4 4 mining-method dilution were identified as the factors with the highest level of impact on MMS. The results of this study emphasise the significant impact of the physical characteristics (i.e., thickness, shape, and dip of the orebody) and mechanical characteristics (i.e., strengths of the orebody and host rock) of the orebody as well as ore uniformity on the MMS, as described in different MMS systems, including Nicholas' approach [2,4] and the UBC MMS tool [16]. However, the factor of depth, which is considered important in the UBC tool when selecting between surface and underground methods, was not found to be highly important in this study because of its low diversity in the selection among the 12 mining methods (surface and underground methods). In addition, according to the results, researchers must focus on economic factors (i.e., mining costs) and technological factors (i.e., dilution), which may not be notably emphasised in the first stage of some of the MMS systems (i.e., Nicholas' approach [2,4] and the UBC MMS tool [16]). Furthermore, the results reveal that the factors for MMS do not have the same degree of importance, thereby indicating the need to create an MMS system that would at least consider the degree of importance of the different influential factors and emphasise the factors with the highest level of impact.
This study adopted the entropy method in the data preparation step (i.e., feature selection) for developing a novel mining-method selection (novel-MMS) system that employs recommendation system technologies. The entropy method was applied to analyse the level of impact of factors influencing MMS then identify the most critical factors/features. In future, the results of this study will be used as a foundation to prepare the input dataset for developing the model for the novel-MMS system. The factors that are identified as the most critical will be used as the main variables to create the input dataset. The respective weights will be used as a base to decide variables placing sequence and the weighting system of the attributes of the variables. The input dataset will be created by mining the variables attributes from mining company historical data which are collected from the Sedar 1 database website. This study is a key step for the optimization of the performance of the model for the novel-MMS system and the reduction of computational costs.
The Entropy method can analyse features/factors relative importance without needing a huge amount of historical data compared to other machine learning feature selection methods. Furthermore, this method enables estimation of the criteria weights without the direct involvement of decision-makers, thereby  reducing the risk of bias that may be caused by the use of subjective judgements of decision-makers (i.e., mining engineering experts). However, analysis from this method is entirely theoretical (i.e., based only on the information provided in the DM), which makes it somehow difficult to explain or interpret. Thus, an assessment of the relative importance of the influential factors based on the opinion and judgements of mining engineering experts might be required. Moreover, further analysis to compare and/or combine the results from both objective and subjective weights might be conducted to improve the quality of the results.