How to find the most important attribute for each class Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsHow to specify important attributes?Does high error rate in regression imply the data set is unpredictable?Random Forest, Type - Regression, Calculation of Importance ExampleDecisionTree split quality using chi-squared statisticFinding the most important questions from a questionary-> resultsselecting variable randomly at each node in a tree in Random ForestUnderstanding decision tree conceptOversampling for multi-class neural netWhy do we need a gain ratioChoosing the first node in a decision tree, basic example

When -s is used with third person singular. What's its use in this context?

3 doors, three guards, one stone

Proof involving the spectral radius and the Jordan canonical form

The logistics of corpse disposal

How do I stop a creek from eroding my steep embankment?

Is there a documented rationale why the House Ways and Means chairman can demand tax info?

How to bypass password on Windows XP account?

How do I keep my slimes from escaping their pens?

What is a Meta algorithm?

What is the correct way to use the pinch test for dehydration?

Why aren't air breathing engines used as small first stages

What do you call a plan that's an alternative plan in case your initial plan fails?

Sorting numerically

Diagram with tikz

Storing hydrofluoric acid before the invention of plastics

How can players work together to take actions that are otherwise impossible?

Is the address of a local variable a constexpr?

How can I fade player when goes inside or outside of the area?

Output the ŋarâþ crîþ alphabet song without using (m)any letters

Bonus calculation: Am I making a mountain out of a molehill?

Stars Make Stars

What are the pros and cons of Aerospike nosecones?

G-Code for resetting to 100% speed

How to find all the available tools in macOS terminal?



How to find the most important attribute for each class



Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsHow to specify important attributes?Does high error rate in regression imply the data set is unpredictable?Random Forest, Type - Regression, Calculation of Importance ExampleDecisionTree split quality using chi-squared statisticFinding the most important questions from a questionary-> resultsselecting variable randomly at each node in a tree in Random ForestUnderstanding decision tree conceptOversampling for multi-class neural netWhy do we need a gain ratioChoosing the first node in a decision tree, basic example










1












$begingroup$


I have a dataset with 28 attributes and 7 class values. I want to know if its possible to find out the most important attribute(s) for deciding the class value, for each class.



For example an answer could be: Attribute 2 is most important for class 1, Attribute 6 is most important for class 2 etc. Or an even more informed answer could be: Attribute 2 being below 0.5 is most important for class 1, Attribute 6 being above 0.75 is most important for class 2 etc



My initial approach to this was to build a decision tree on the data and find the node that had the largest information gain/gain ratio for each class and that would be the most determining factor for that class. The problem with this is that the decision tree implementations I have found don't give the information gain/gain ratio for each node and as this is time bound I don't have the time to implement my own version. My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute. Is this the right direction to go down or is their a better option?










share|improve this question











$endgroup$







  • 2




    $begingroup$
    Covariance matrix for each attribute and the output.
    $endgroup$
    – Vaalizaadeh
    Oct 16 '18 at 12:47










  • $begingroup$
    @Media could you expand on this please? I thought a covariance matrix would only let me know the correlation between attributes. If you have a solution you could add it as an answer
    $endgroup$
    – Nate
    Oct 16 '18 at 13:15










  • $begingroup$
    Important in what way? That best split your data (given that you were talking about decision trees and splits)? There are ML algorithms that return feature importances, for ex. random forests.
    $endgroup$
    – user2974951
    Oct 16 '18 at 13:37










  • $begingroup$
    Important in the way that I want to know the attributes that have the biggest say in determining each class. I believe this means the best split of the data. As far as I know random forest doesn't return feature importance for each class, or am I wrong about this?
    $endgroup$
    – Nate
    Oct 16 '18 at 13:52










  • $begingroup$
    @Nate, "My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute." ---> if you're happy with that definition of "most important" then yes. Your work is to define what you mean by "most important", but your approach (classification ONE vs ALL_OTHERS) is a good one.
    $endgroup$
    – f.g.
    Oct 16 '18 at 17:33















1












$begingroup$


I have a dataset with 28 attributes and 7 class values. I want to know if its possible to find out the most important attribute(s) for deciding the class value, for each class.



For example an answer could be: Attribute 2 is most important for class 1, Attribute 6 is most important for class 2 etc. Or an even more informed answer could be: Attribute 2 being below 0.5 is most important for class 1, Attribute 6 being above 0.75 is most important for class 2 etc



My initial approach to this was to build a decision tree on the data and find the node that had the largest information gain/gain ratio for each class and that would be the most determining factor for that class. The problem with this is that the decision tree implementations I have found don't give the information gain/gain ratio for each node and as this is time bound I don't have the time to implement my own version. My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute. Is this the right direction to go down or is their a better option?










share|improve this question











$endgroup$







  • 2




    $begingroup$
    Covariance matrix for each attribute and the output.
    $endgroup$
    – Vaalizaadeh
    Oct 16 '18 at 12:47










  • $begingroup$
    @Media could you expand on this please? I thought a covariance matrix would only let me know the correlation between attributes. If you have a solution you could add it as an answer
    $endgroup$
    – Nate
    Oct 16 '18 at 13:15










  • $begingroup$
    Important in what way? That best split your data (given that you were talking about decision trees and splits)? There are ML algorithms that return feature importances, for ex. random forests.
    $endgroup$
    – user2974951
    Oct 16 '18 at 13:37










  • $begingroup$
    Important in the way that I want to know the attributes that have the biggest say in determining each class. I believe this means the best split of the data. As far as I know random forest doesn't return feature importance for each class, or am I wrong about this?
    $endgroup$
    – Nate
    Oct 16 '18 at 13:52










  • $begingroup$
    @Nate, "My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute." ---> if you're happy with that definition of "most important" then yes. Your work is to define what you mean by "most important", but your approach (classification ONE vs ALL_OTHERS) is a good one.
    $endgroup$
    – f.g.
    Oct 16 '18 at 17:33













1












1








1





$begingroup$


I have a dataset with 28 attributes and 7 class values. I want to know if its possible to find out the most important attribute(s) for deciding the class value, for each class.



For example an answer could be: Attribute 2 is most important for class 1, Attribute 6 is most important for class 2 etc. Or an even more informed answer could be: Attribute 2 being below 0.5 is most important for class 1, Attribute 6 being above 0.75 is most important for class 2 etc



My initial approach to this was to build a decision tree on the data and find the node that had the largest information gain/gain ratio for each class and that would be the most determining factor for that class. The problem with this is that the decision tree implementations I have found don't give the information gain/gain ratio for each node and as this is time bound I don't have the time to implement my own version. My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute. Is this the right direction to go down or is their a better option?










share|improve this question











$endgroup$




I have a dataset with 28 attributes and 7 class values. I want to know if its possible to find out the most important attribute(s) for deciding the class value, for each class.



For example an answer could be: Attribute 2 is most important for class 1, Attribute 6 is most important for class 2 etc. Or an even more informed answer could be: Attribute 2 being below 0.5 is most important for class 1, Attribute 6 being above 0.75 is most important for class 2 etc



My initial approach to this was to build a decision tree on the data and find the node that had the largest information gain/gain ratio for each class and that would be the most determining factor for that class. The problem with this is that the decision tree implementations I have found don't give the information gain/gain ratio for each node and as this is time bound I don't have the time to implement my own version. My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute. Is this the right direction to go down or is their a better option?







machine-learning neural-network deep-learning feature-selection multiclass-classification






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Oct 16 '18 at 17:23









Vaalizaadeh

7,59562263




7,59562263










asked Oct 16 '18 at 12:46









NateNate

61




61







  • 2




    $begingroup$
    Covariance matrix for each attribute and the output.
    $endgroup$
    – Vaalizaadeh
    Oct 16 '18 at 12:47










  • $begingroup$
    @Media could you expand on this please? I thought a covariance matrix would only let me know the correlation between attributes. If you have a solution you could add it as an answer
    $endgroup$
    – Nate
    Oct 16 '18 at 13:15










  • $begingroup$
    Important in what way? That best split your data (given that you were talking about decision trees and splits)? There are ML algorithms that return feature importances, for ex. random forests.
    $endgroup$
    – user2974951
    Oct 16 '18 at 13:37










  • $begingroup$
    Important in the way that I want to know the attributes that have the biggest say in determining each class. I believe this means the best split of the data. As far as I know random forest doesn't return feature importance for each class, or am I wrong about this?
    $endgroup$
    – Nate
    Oct 16 '18 at 13:52










  • $begingroup$
    @Nate, "My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute." ---> if you're happy with that definition of "most important" then yes. Your work is to define what you mean by "most important", but your approach (classification ONE vs ALL_OTHERS) is a good one.
    $endgroup$
    – f.g.
    Oct 16 '18 at 17:33












  • 2




    $begingroup$
    Covariance matrix for each attribute and the output.
    $endgroup$
    – Vaalizaadeh
    Oct 16 '18 at 12:47










  • $begingroup$
    @Media could you expand on this please? I thought a covariance matrix would only let me know the correlation between attributes. If you have a solution you could add it as an answer
    $endgroup$
    – Nate
    Oct 16 '18 at 13:15










  • $begingroup$
    Important in what way? That best split your data (given that you were talking about decision trees and splits)? There are ML algorithms that return feature importances, for ex. random forests.
    $endgroup$
    – user2974951
    Oct 16 '18 at 13:37










  • $begingroup$
    Important in the way that I want to know the attributes that have the biggest say in determining each class. I believe this means the best split of the data. As far as I know random forest doesn't return feature importance for each class, or am I wrong about this?
    $endgroup$
    – Nate
    Oct 16 '18 at 13:52










  • $begingroup$
    @Nate, "My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute." ---> if you're happy with that definition of "most important" then yes. Your work is to define what you mean by "most important", but your approach (classification ONE vs ALL_OTHERS) is a good one.
    $endgroup$
    – f.g.
    Oct 16 '18 at 17:33







2




2




$begingroup$
Covariance matrix for each attribute and the output.
$endgroup$
– Vaalizaadeh
Oct 16 '18 at 12:47




$begingroup$
Covariance matrix for each attribute and the output.
$endgroup$
– Vaalizaadeh
Oct 16 '18 at 12:47












$begingroup$
@Media could you expand on this please? I thought a covariance matrix would only let me know the correlation between attributes. If you have a solution you could add it as an answer
$endgroup$
– Nate
Oct 16 '18 at 13:15




$begingroup$
@Media could you expand on this please? I thought a covariance matrix would only let me know the correlation between attributes. If you have a solution you could add it as an answer
$endgroup$
– Nate
Oct 16 '18 at 13:15












$begingroup$
Important in what way? That best split your data (given that you were talking about decision trees and splits)? There are ML algorithms that return feature importances, for ex. random forests.
$endgroup$
– user2974951
Oct 16 '18 at 13:37




$begingroup$
Important in what way? That best split your data (given that you were talking about decision trees and splits)? There are ML algorithms that return feature importances, for ex. random forests.
$endgroup$
– user2974951
Oct 16 '18 at 13:37












$begingroup$
Important in the way that I want to know the attributes that have the biggest say in determining each class. I believe this means the best split of the data. As far as I know random forest doesn't return feature importance for each class, or am I wrong about this?
$endgroup$
– Nate
Oct 16 '18 at 13:52




$begingroup$
Important in the way that I want to know the attributes that have the biggest say in determining each class. I believe this means the best split of the data. As far as I know random forest doesn't return feature importance for each class, or am I wrong about this?
$endgroup$
– Nate
Oct 16 '18 at 13:52












$begingroup$
@Nate, "My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute." ---> if you're happy with that definition of "most important" then yes. Your work is to define what you mean by "most important", but your approach (classification ONE vs ALL_OTHERS) is a good one.
$endgroup$
– f.g.
Oct 16 '18 at 17:33




$begingroup$
@Nate, "My current thought is to create multiple datasets which are all one class vs the rest and then perform attribute selection (eg. information gain) on them to find the most important attribute." ---> if you're happy with that definition of "most important" then yes. Your work is to define what you mean by "most important", but your approach (classification ONE vs ALL_OTHERS) is a good one.
$endgroup$
– f.g.
Oct 16 '18 at 17:33










4 Answers
4






active

oldest

votes


















1












$begingroup$

If you must split the dataset for each class then I suggest you try PCA. Principle Component Analysis is basically used for dimensionality reduction in the sense that it gives you the subset of attributes which represent the distribution of data best. You could use that for all classes and in that way get the attributes which affect the class data distribution the best.






share|improve this answer









$endgroup$












  • $begingroup$
    Could also 'rank' the importance of the attributes by looking at each attributes coefficient?
    $endgroup$
    – Nate
    Oct 16 '18 at 16:12


















0












$begingroup$


Or an even more informed answer could be: Attribute 2 being below 0.5
is most important for class 1, Attribute 6 being above 0.75 is most
important for class 2 etc




A way of doing this is by discretizing your continuous variables into histograms. Every bin of a histogram can thereafter be considered as a separate variable and its importance can be easily found using standard decision tree implementations such as the one in sklearn that provides a _feature_importances attribute. This will offer you insight on the important regions of each variable.



This is demonstrated in Figure 9 in this paper.



Welcome to the site, Nate!






share|improve this answer









$endgroup$












  • $begingroup$
    Ah yes good idea, the only thing is that wouldn't the _feature_importances attribute tell me the importance of the feetures for the whole model, rather than the importance per class.
    $endgroup$
    – Nate
    Oct 16 '18 at 14:23










  • $begingroup$
    when a feature is important to classify a sample into class A, it means that it is also important for not classifying the same sample in class B. I mean that you shouldn't ask "which feature is important to classify the sample in class A and which one to classify it in class B" BUT "which feature is important to classify the sample correctly". This is feature importance.
    $endgroup$
    – pcko1
    Oct 16 '18 at 14:35



















0












$begingroup$

Let's say that in this way. The easiest way to ascertain the relation among different features and the outputs is to use covariance matrix. You can even visualise the data for each class. Take a look at the following image.



enter image description here



Suppose that the vertical axis is the output and the horizontal axis is for one of the features. As you can see, having knowledge about the features informs us of the changes in the output. Now, consider the following illustration.



enter image description here



In this figure, you can see that considering this typical feature does not inform you of the changes in the output.



Another approach can be using PCA which finds the appropriate features itself. What it does is finding a linear combination of important features which are more relevant to the output.






share|improve this answer









$endgroup$




















    0












    $begingroup$

    It is difficult to get the answer to "which one is the most important" for every class, normally, the distintion between is "between classes" not "for a specific class".



    I use the feature importance from xgboost, this method allows to measure which feature participates in more trees of the boosted forest xgboost has. There is even, the possibility of plotting these importances resulting in very nice data to show.



    Feature importance in XGBoost






    share|improve this answer








    New contributor




    Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.






    $endgroup$













      Your Answer








      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "557"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader:
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      ,
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













      draft saved

      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f39759%2fhow-to-find-the-most-important-attribute-for-each-class%23new-answer', 'question_page');

      );

      Post as a guest















      Required, but never shown

























      4 Answers
      4






      active

      oldest

      votes








      4 Answers
      4






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      1












      $begingroup$

      If you must split the dataset for each class then I suggest you try PCA. Principle Component Analysis is basically used for dimensionality reduction in the sense that it gives you the subset of attributes which represent the distribution of data best. You could use that for all classes and in that way get the attributes which affect the class data distribution the best.






      share|improve this answer









      $endgroup$












      • $begingroup$
        Could also 'rank' the importance of the attributes by looking at each attributes coefficient?
        $endgroup$
        – Nate
        Oct 16 '18 at 16:12















      1












      $begingroup$

      If you must split the dataset for each class then I suggest you try PCA. Principle Component Analysis is basically used for dimensionality reduction in the sense that it gives you the subset of attributes which represent the distribution of data best. You could use that for all classes and in that way get the attributes which affect the class data distribution the best.






      share|improve this answer









      $endgroup$












      • $begingroup$
        Could also 'rank' the importance of the attributes by looking at each attributes coefficient?
        $endgroup$
        – Nate
        Oct 16 '18 at 16:12













      1












      1








      1





      $begingroup$

      If you must split the dataset for each class then I suggest you try PCA. Principle Component Analysis is basically used for dimensionality reduction in the sense that it gives you the subset of attributes which represent the distribution of data best. You could use that for all classes and in that way get the attributes which affect the class data distribution the best.






      share|improve this answer









      $endgroup$



      If you must split the dataset for each class then I suggest you try PCA. Principle Component Analysis is basically used for dimensionality reduction in the sense that it gives you the subset of attributes which represent the distribution of data best. You could use that for all classes and in that way get the attributes which affect the class data distribution the best.







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Oct 16 '18 at 14:00









      Saket Kumar SinghSaket Kumar Singh

      1362




      1362











      • $begingroup$
        Could also 'rank' the importance of the attributes by looking at each attributes coefficient?
        $endgroup$
        – Nate
        Oct 16 '18 at 16:12
















      • $begingroup$
        Could also 'rank' the importance of the attributes by looking at each attributes coefficient?
        $endgroup$
        – Nate
        Oct 16 '18 at 16:12















      $begingroup$
      Could also 'rank' the importance of the attributes by looking at each attributes coefficient?
      $endgroup$
      – Nate
      Oct 16 '18 at 16:12




      $begingroup$
      Could also 'rank' the importance of the attributes by looking at each attributes coefficient?
      $endgroup$
      – Nate
      Oct 16 '18 at 16:12











      0












      $begingroup$


      Or an even more informed answer could be: Attribute 2 being below 0.5
      is most important for class 1, Attribute 6 being above 0.75 is most
      important for class 2 etc




      A way of doing this is by discretizing your continuous variables into histograms. Every bin of a histogram can thereafter be considered as a separate variable and its importance can be easily found using standard decision tree implementations such as the one in sklearn that provides a _feature_importances attribute. This will offer you insight on the important regions of each variable.



      This is demonstrated in Figure 9 in this paper.



      Welcome to the site, Nate!






      share|improve this answer









      $endgroup$












      • $begingroup$
        Ah yes good idea, the only thing is that wouldn't the _feature_importances attribute tell me the importance of the feetures for the whole model, rather than the importance per class.
        $endgroup$
        – Nate
        Oct 16 '18 at 14:23










      • $begingroup$
        when a feature is important to classify a sample into class A, it means that it is also important for not classifying the same sample in class B. I mean that you shouldn't ask "which feature is important to classify the sample in class A and which one to classify it in class B" BUT "which feature is important to classify the sample correctly". This is feature importance.
        $endgroup$
        – pcko1
        Oct 16 '18 at 14:35
















      0












      $begingroup$


      Or an even more informed answer could be: Attribute 2 being below 0.5
      is most important for class 1, Attribute 6 being above 0.75 is most
      important for class 2 etc




      A way of doing this is by discretizing your continuous variables into histograms. Every bin of a histogram can thereafter be considered as a separate variable and its importance can be easily found using standard decision tree implementations such as the one in sklearn that provides a _feature_importances attribute. This will offer you insight on the important regions of each variable.



      This is demonstrated in Figure 9 in this paper.



      Welcome to the site, Nate!






      share|improve this answer









      $endgroup$












      • $begingroup$
        Ah yes good idea, the only thing is that wouldn't the _feature_importances attribute tell me the importance of the feetures for the whole model, rather than the importance per class.
        $endgroup$
        – Nate
        Oct 16 '18 at 14:23










      • $begingroup$
        when a feature is important to classify a sample into class A, it means that it is also important for not classifying the same sample in class B. I mean that you shouldn't ask "which feature is important to classify the sample in class A and which one to classify it in class B" BUT "which feature is important to classify the sample correctly". This is feature importance.
        $endgroup$
        – pcko1
        Oct 16 '18 at 14:35














      0












      0








      0





      $begingroup$


      Or an even more informed answer could be: Attribute 2 being below 0.5
      is most important for class 1, Attribute 6 being above 0.75 is most
      important for class 2 etc




      A way of doing this is by discretizing your continuous variables into histograms. Every bin of a histogram can thereafter be considered as a separate variable and its importance can be easily found using standard decision tree implementations such as the one in sklearn that provides a _feature_importances attribute. This will offer you insight on the important regions of each variable.



      This is demonstrated in Figure 9 in this paper.



      Welcome to the site, Nate!






      share|improve this answer









      $endgroup$




      Or an even more informed answer could be: Attribute 2 being below 0.5
      is most important for class 1, Attribute 6 being above 0.75 is most
      important for class 2 etc




      A way of doing this is by discretizing your continuous variables into histograms. Every bin of a histogram can thereafter be considered as a separate variable and its importance can be easily found using standard decision tree implementations such as the one in sklearn that provides a _feature_importances attribute. This will offer you insight on the important regions of each variable.



      This is demonstrated in Figure 9 in this paper.



      Welcome to the site, Nate!







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Oct 16 '18 at 14:10









      pcko1pcko1

      1,681418




      1,681418











      • $begingroup$
        Ah yes good idea, the only thing is that wouldn't the _feature_importances attribute tell me the importance of the feetures for the whole model, rather than the importance per class.
        $endgroup$
        – Nate
        Oct 16 '18 at 14:23










      • $begingroup$
        when a feature is important to classify a sample into class A, it means that it is also important for not classifying the same sample in class B. I mean that you shouldn't ask "which feature is important to classify the sample in class A and which one to classify it in class B" BUT "which feature is important to classify the sample correctly". This is feature importance.
        $endgroup$
        – pcko1
        Oct 16 '18 at 14:35

















      • $begingroup$
        Ah yes good idea, the only thing is that wouldn't the _feature_importances attribute tell me the importance of the feetures for the whole model, rather than the importance per class.
        $endgroup$
        – Nate
        Oct 16 '18 at 14:23










      • $begingroup$
        when a feature is important to classify a sample into class A, it means that it is also important for not classifying the same sample in class B. I mean that you shouldn't ask "which feature is important to classify the sample in class A and which one to classify it in class B" BUT "which feature is important to classify the sample correctly". This is feature importance.
        $endgroup$
        – pcko1
        Oct 16 '18 at 14:35
















      $begingroup$
      Ah yes good idea, the only thing is that wouldn't the _feature_importances attribute tell me the importance of the feetures for the whole model, rather than the importance per class.
      $endgroup$
      – Nate
      Oct 16 '18 at 14:23




      $begingroup$
      Ah yes good idea, the only thing is that wouldn't the _feature_importances attribute tell me the importance of the feetures for the whole model, rather than the importance per class.
      $endgroup$
      – Nate
      Oct 16 '18 at 14:23












      $begingroup$
      when a feature is important to classify a sample into class A, it means that it is also important for not classifying the same sample in class B. I mean that you shouldn't ask "which feature is important to classify the sample in class A and which one to classify it in class B" BUT "which feature is important to classify the sample correctly". This is feature importance.
      $endgroup$
      – pcko1
      Oct 16 '18 at 14:35





      $begingroup$
      when a feature is important to classify a sample into class A, it means that it is also important for not classifying the same sample in class B. I mean that you shouldn't ask "which feature is important to classify the sample in class A and which one to classify it in class B" BUT "which feature is important to classify the sample correctly". This is feature importance.
      $endgroup$
      – pcko1
      Oct 16 '18 at 14:35












      0












      $begingroup$

      Let's say that in this way. The easiest way to ascertain the relation among different features and the outputs is to use covariance matrix. You can even visualise the data for each class. Take a look at the following image.



      enter image description here



      Suppose that the vertical axis is the output and the horizontal axis is for one of the features. As you can see, having knowledge about the features informs us of the changes in the output. Now, consider the following illustration.



      enter image description here



      In this figure, you can see that considering this typical feature does not inform you of the changes in the output.



      Another approach can be using PCA which finds the appropriate features itself. What it does is finding a linear combination of important features which are more relevant to the output.






      share|improve this answer









      $endgroup$

















        0












        $begingroup$

        Let's say that in this way. The easiest way to ascertain the relation among different features and the outputs is to use covariance matrix. You can even visualise the data for each class. Take a look at the following image.



        enter image description here



        Suppose that the vertical axis is the output and the horizontal axis is for one of the features. As you can see, having knowledge about the features informs us of the changes in the output. Now, consider the following illustration.



        enter image description here



        In this figure, you can see that considering this typical feature does not inform you of the changes in the output.



        Another approach can be using PCA which finds the appropriate features itself. What it does is finding a linear combination of important features which are more relevant to the output.






        share|improve this answer









        $endgroup$















          0












          0








          0





          $begingroup$

          Let's say that in this way. The easiest way to ascertain the relation among different features and the outputs is to use covariance matrix. You can even visualise the data for each class. Take a look at the following image.



          enter image description here



          Suppose that the vertical axis is the output and the horizontal axis is for one of the features. As you can see, having knowledge about the features informs us of the changes in the output. Now, consider the following illustration.



          enter image description here



          In this figure, you can see that considering this typical feature does not inform you of the changes in the output.



          Another approach can be using PCA which finds the appropriate features itself. What it does is finding a linear combination of important features which are more relevant to the output.






          share|improve this answer









          $endgroup$



          Let's say that in this way. The easiest way to ascertain the relation among different features and the outputs is to use covariance matrix. You can even visualise the data for each class. Take a look at the following image.



          enter image description here



          Suppose that the vertical axis is the output and the horizontal axis is for one of the features. As you can see, having knowledge about the features informs us of the changes in the output. Now, consider the following illustration.



          enter image description here



          In this figure, you can see that considering this typical feature does not inform you of the changes in the output.



          Another approach can be using PCA which finds the appropriate features itself. What it does is finding a linear combination of important features which are more relevant to the output.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Oct 16 '18 at 17:22









          VaalizaadehVaalizaadeh

          7,59562263




          7,59562263





















              0












              $begingroup$

              It is difficult to get the answer to "which one is the most important" for every class, normally, the distintion between is "between classes" not "for a specific class".



              I use the feature importance from xgboost, this method allows to measure which feature participates in more trees of the boosted forest xgboost has. There is even, the possibility of plotting these importances resulting in very nice data to show.



              Feature importance in XGBoost






              share|improve this answer








              New contributor




              Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
              Check out our Code of Conduct.






              $endgroup$

















                0












                $begingroup$

                It is difficult to get the answer to "which one is the most important" for every class, normally, the distintion between is "between classes" not "for a specific class".



                I use the feature importance from xgboost, this method allows to measure which feature participates in more trees of the boosted forest xgboost has. There is even, the possibility of plotting these importances resulting in very nice data to show.



                Feature importance in XGBoost






                share|improve this answer








                New contributor




                Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.






                $endgroup$















                  0












                  0








                  0





                  $begingroup$

                  It is difficult to get the answer to "which one is the most important" for every class, normally, the distintion between is "between classes" not "for a specific class".



                  I use the feature importance from xgboost, this method allows to measure which feature participates in more trees of the boosted forest xgboost has. There is even, the possibility of plotting these importances resulting in very nice data to show.



                  Feature importance in XGBoost






                  share|improve this answer








                  New contributor




                  Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.






                  $endgroup$



                  It is difficult to get the answer to "which one is the most important" for every class, normally, the distintion between is "between classes" not "for a specific class".



                  I use the feature importance from xgboost, this method allows to measure which feature participates in more trees of the boosted forest xgboost has. There is even, the possibility of plotting these importances resulting in very nice data to show.



                  Feature importance in XGBoost







                  share|improve this answer








                  New contributor




                  Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.









                  share|improve this answer



                  share|improve this answer






                  New contributor




                  Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.









                  answered 34 mins ago









                  Juan Esteban de la CalleJuan Esteban de la Calle

                  18311




                  18311




                  New contributor




                  Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.





                  New contributor





                  Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.






                  Juan Esteban de la Calle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.



























                      draft saved

                      draft discarded
















































                      Thanks for contributing an answer to Data Science Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      Use MathJax to format equations. MathJax reference.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f39759%2fhow-to-find-the-most-important-attribute-for-each-class%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Францішак Багушэвіч Змест Сям'я | Біяграфія | Творчасць | Мова Багушэвіча | Ацэнкі дзейнасці | Цікавыя факты | Спадчына | Выбраная бібліяграфія | Ушанаванне памяці | У філатэліі | Зноскі | Літаратура | Спасылкі | НавігацыяЛяхоўскі У. Рупіўся дзеля Бога і людзей: Жыццёвы шлях Лявона Вітан-Дубейкаўскага // Вольскі і Памідораў з песняй пра немца Адвакат, паэт, народны заступнік Ашмянскі веснікВ Минске появится площадь Богушевича и улица Сырокомли, Белорусская деловая газета, 19 июля 2001 г.Айцец беларускай нацыянальнай ідэі паўстаў у бронзе Сяргей Аляксандравіч Адашкевіч (1918, Мінск). 80-я гады. Бюст «Францішак Багушэвіч».Яўген Мікалаевіч Ціхановіч. «Партрэт Францішка Багушэвіча»Мікола Мікалаевіч Купава. «Партрэт зачынальніка новай беларускай літаратуры Францішка Багушэвіча»Уладзімір Іванавіч Мелехаў. На помніку «Змагарам за родную мову» Барэльеф «Францішак Багушэвіч»Памяць пра Багушэвіча на Віленшчыне Страчаная сталіца. Беларускія шыльды на вуліцах Вільні«Krynica». Ideologia i przywódcy białoruskiego katolicyzmuФранцішак БагушэвічТворы на knihi.comТворы Францішка Багушэвіча на bellib.byСодаль Уладзімір. Францішак Багушэвіч на Лідчыне;Луцкевіч Антон. Жыцьцё і творчасьць Фр. Багушэвіча ў успамінах ягоных сучасьнікаў // Запісы Беларускага Навуковага таварыства. Вільня, 1938. Сшытак 1. С. 16-34.Большая российская1188761710000 0000 5537 633Xn9209310021619551927869394п

                      Беларусь Змест Назва Гісторыя Геаграфія Сімволіка Дзяржаўны лад Палітычныя партыі Міжнароднае становішча і знешняя палітыка Адміністрацыйны падзел Насельніцтва Эканоміка Культура і грамадства Сацыяльная сфера Узброеныя сілы Заўвагі Літаратура Спасылкі НавігацыяHGЯOiТоп-2011 г. (па версіі ej.by)Топ-2013 г. (па версіі ej.by)Топ-2016 г. (па версіі ej.by)Топ-2017 г. (па версіі ej.by)Нацыянальны статыстычны камітэт Рэспублікі БеларусьШчыльнасць насельніцтва па краінахhttp://naviny.by/rubrics/society/2011/09/16/ic_articles_116_175144/А. Калечыц, У. Ксяндзоў. Спробы засялення краю неандэртальскім чалавекам.І ў Менску былі мамантыА. Калечыц, У. Ксяндзоў. Старажытны каменны век (палеаліт). Першапачатковае засяленне тэрыторыіГ. Штыхаў. Балты і славяне ў VI—VIII стст.М. Клімаў. Полацкае княства ў IX—XI стст.Г. Штыхаў, В. Ляўко. Палітычная гісторыя Полацкай зямліГ. Штыхаў. Дзяржаўны лад у землях-княствахГ. Штыхаў. Дзяржаўны лад у землях-княствахБеларускія землі ў складзе Вялікага Княства ЛітоўскагаЛюблінская унія 1569 г."The Early Stages of Independence"Zapomniane prawdy25 гадоў таму было аб'яўлена, што Язэп Пілсудскі — беларус (фота)Наша вадаДакументы ЧАЭС: Забруджванне тэрыторыі Беларусі « ЧАЭС Зона адчужэнняСведения о политических партиях, зарегистрированных в Республике Беларусь // Министерство юстиции Республики БеларусьСтатыстычны бюлетэнь „Полаўзроставая структура насельніцтва Рэспублікі Беларусь на 1 студзеня 2012 года і сярэднегадовая колькасць насельніцтва за 2011 год“Индекс человеческого развития Беларуси — не было бы нижеБеларусь занимает первое место в СНГ по индексу развития с учетом гендерного факцёраНацыянальны статыстычны камітэт Рэспублікі БеларусьКанстытуцыя РБ. Артыкул 17Трансфармацыйныя задачы БеларусіВыйсце з крызісу — далейшае рэфармаванне Беларускі рубель — сусветны лідар па дэвальвацыяхПра змену коштаў у кастрычніку 2011 г.Бядней за беларусаў у СНД толькі таджыкіСярэдні заробак у верасні дасягнуў 2,26 мільёна рублёўЭканомікаГаласуем за ТОП-100 беларускай прозыСучасныя беларускія мастакіАрхитектура Беларуси BELARUS.BYА. Каханоўскі. Культура Беларусі ўсярэдзіне XVII—XVIII ст.Анталогія беларускай народнай песні, гуказапісы спеваўБеларускія Музычныя IнструментыБеларускі рок, які мы страцілі. Топ-10 гуртоў«Мясцовы час» — нязгаслая легенда беларускай рок-музыкіСЯРГЕЙ БУДКІН. МЫ НЯ ЗНАЕМ СВАЁЙ МУЗЫКІМ. А. Каладзінскі. НАРОДНЫ ТЭАТРМагнацкія культурныя цэнтрыПублічная дыскусія «Беларуская новая пьеса: без беларускай мовы ці беларуская?»Беларускія драматургі па-ранейшаму лепш ставяцца за мяжой, чым на радзіме«Працэс незалежнага кіно пайшоў, і дзяржаву турбуе яго непадкантрольнасць»Беларускія філосафы ў пошуках прасторыВсе идём в библиотекуАрхіваванаАб Нацыянальнай праграме даследавання і выкарыстання касмічнай прасторы ў мірных мэтах на 2008—2012 гадыУ космас — разам.У суседнім з Барысаўскім раёне пабудуюць Камандна-вымяральны пунктСвяты і абрады беларусаў«Мірныя бульбашы з малой краіны» — 5 непраўдзівых стэрэатыпаў пра БеларусьМ. Раманюк. Беларускае народнае адзеннеУ Беларусі скарачаецца колькасць злачынстваўЛукашэнка незадаволены мінскімі ўладамі Крадзяжы складаюць у Мінску каля 70% злачынстваў Узровень злачыннасці ў Мінскай вобласці — адзін з самых высокіх у краіне Генпракуратура аналізуе стан са злачыннасцю ў Беларусі па каэфіцыенце злачыннасці У Беларусі стабілізавалася крымінагеннае становішча, лічыць генпракурорЗамежнікі сталі здзяйсняць у Беларусі больш злачынстваўМУС Беларусі турбуе рост рэцыдыўнай злачыннасціЯ з ЖЭСа. Дазволіце вас абкрасці! Рэйтынг усіх службаў і падраздзяленняў ГУУС Мінгарвыканкама вырасАб КДБ РБГісторыя Аператыўна-аналітычнага цэнтра РБГісторыя ДКФРТаможняagentura.ruБеларусьBelarus.by — Афіцыйны сайт Рэспублікі БеларусьСайт урада БеларусіRadzima.org — Збор архітэктурных помнікаў, гісторыя Беларусі«Глобус Беларуси»Гербы и флаги БеларусиАсаблівасці каменнага веку на БеларусіА. Калечыц, У. Ксяндзоў. Старажытны каменны век (палеаліт). Першапачатковае засяленне тэрыторыіУ. Ксяндзоў. Сярэдні каменны век (мезаліт). Засяленне краю плямёнамі паляўнічых, рыбакоў і збіральнікаўА. Калечыц, М. Чарняўскі. Плямёны на тэрыторыі Беларусі ў новым каменным веку (неаліце)А. Калечыц, У. Ксяндзоў, М. Чарняўскі. Гаспадарчыя заняткі ў каменным векуЭ. Зайкоўскі. Духоўная культура ў каменным векуАсаблівасці бронзавага веку на БеларусіФарміраванне супольнасцей ранняга перыяду бронзавага векуФотографии БеларусиРоля беларускіх зямель ва ўтварэнні і ўмацаванні ВКЛВ. Фадзеева. З гісторыі развіцця беларускай народнай вышыўкіDMOZGran catalanaБольшая российскаяBritannica (анлайн)Швейцарскі гістарычны15325917611952699xDA123282154079143-90000 0001 2171 2080n9112870100577502ge128882171858027501086026362074122714179пппппп

                      ValueError: Expected n_neighbors <= n_samples, but n_samples = 1, n_neighbors = 6 (SMOTE) The 2019 Stack Overflow Developer Survey Results Are InCan SMOTE be applied over sequence of words (sentences)?ValueError when doing validation with random forestsSMOTE and multi class oversamplingLogic behind SMOTE-NC?ValueError: Error when checking target: expected dense_1 to have shape (7,) but got array with shape (1,)SmoteBoost: Should SMOTE be ran individually for each iteration/tree in the boosting?solving multi-class imbalance classification using smote and OSSUsing SMOTE for Synthetic Data generation to improve performance on unbalanced dataproblem of entry format for a simple model in KerasSVM SMOTE fit_resample() function runs forever with no result