A way to Identify tuning parameters and their possible rangeModel parameters & Hyper parameters of Neural Network & their tuning in training & validation stageMethods for string classificationsInterpretation of tuning parameters (shrinkage and nrounds) in XGBoostHyper parameters and ValidationSetHow much neural network theory required to design one?When to perform feature selection, how, and how does data affect choosing the predictive model?Tuning svm and cart hyperparametersWhy Gradient methods work in finding the parameters in Neural Networks?Data model and algorithm for recommending “related” interestsWhich is first ? Tuning the parameters or selecting the model
Drawing a german abacus as in the books of Adam Ries
A faster way to compute the largest prime factor
Prove that the countable union of countable sets is also countable
Nails holding drywall
Can a level 2 Warlock take one level in rogue, then continue advancing as a warlock?
As an international instructor, should I openly talk about my accent?
How to run Arduino codes in Java netbeans?
Contradiction proof for inequality of P and NP?
How does the mezzoloth's teleportation work?
Will I lose my paid in full property
std::unique_ptr of base class holding reference of derived class does not show warning in gcc compiler while naked pointer shows it. Why?
Can you stand up from being prone using Skirmisher outside of your turn?
Air bladders in bat-like skin wings for better lift?
Co-worker works way more than he should
Retract an already submitted recommendation letter (written for an undergrad student)
"My boss was furious with me and I have been fired" vs. "My boss was furious with me and I was fired"
What is the most expensive material in the world that could be used to create pun-pun's lute?
How to find the stem of any word?
Why does Arg'[1. + I] return -0.5?
Help with my training data
How exactly does Hawking radiation decrease the mass of black holes?
What makes accurate emulation of old systems a difficult task?
Why doesn't the standard consider a template constructor as a copy constructor?
How to find if a column is referenced in a computed column?
A way to Identify tuning parameters and their possible range
Model parameters & Hyper parameters of Neural Network & their tuning in training & validation stageMethods for string classificationsInterpretation of tuning parameters (shrinkage and nrounds) in XGBoostHyper parameters and ValidationSetHow much neural network theory required to design one?When to perform feature selection, how, and how does data affect choosing the predictive model?Tuning svm and cart hyperparametersWhy Gradient methods work in finding the parameters in Neural Networks?Data model and algorithm for recommending “related” interestsWhich is first ? Tuning the parameters or selecting the model
$begingroup$
I am a novice in Machine Learning. But when I started learning, I figure out that all the methods have some tuning parameters and those parameters take a range of possible values. By grid searching, we identify a set of these parameters that optimize some function. But is there any way to find the possible domain of the tuning parameters? This would definitely save my time and the computer's job. In addition, some methods such as xgboost have loads of tuning parameters. Is there any way to know which one to tune and which one to leave as it is. I have been using sklearn python library.
machine-learning hyperparameter-tuning
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
$endgroup$
bumped to the homepage by Community♦ 40 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I am a novice in Machine Learning. But when I started learning, I figure out that all the methods have some tuning parameters and those parameters take a range of possible values. By grid searching, we identify a set of these parameters that optimize some function. But is there any way to find the possible domain of the tuning parameters? This would definitely save my time and the computer's job. In addition, some methods such as xgboost have loads of tuning parameters. Is there any way to know which one to tune and which one to leave as it is. I have been using sklearn python library.
machine-learning hyperparameter-tuning
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
$endgroup$
bumped to the homepage by Community♦ 40 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
2
$begingroup$
This is where domain knowledge comes in, if you know something about your data beforehand you can use this to decrease model selection time.
$endgroup$
– user2974951
Sep 25 '18 at 13:23
add a comment |
$begingroup$
I am a novice in Machine Learning. But when I started learning, I figure out that all the methods have some tuning parameters and those parameters take a range of possible values. By grid searching, we identify a set of these parameters that optimize some function. But is there any way to find the possible domain of the tuning parameters? This would definitely save my time and the computer's job. In addition, some methods such as xgboost have loads of tuning parameters. Is there any way to know which one to tune and which one to leave as it is. I have been using sklearn python library.
machine-learning hyperparameter-tuning
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
$endgroup$
I am a novice in Machine Learning. But when I started learning, I figure out that all the methods have some tuning parameters and those parameters take a range of possible values. By grid searching, we identify a set of these parameters that optimize some function. But is there any way to find the possible domain of the tuning parameters? This would definitely save my time and the computer's job. In addition, some methods such as xgboost have loads of tuning parameters. Is there any way to know which one to tune and which one to leave as it is. I have been using sklearn python library.
machine-learning hyperparameter-tuning
machine-learning hyperparameter-tuning
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
asked Sep 25 '18 at 12:45
TheRimalayaTheRimalaya
1062
1062
bumped to the homepage by Community♦ 40 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 40 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
2
$begingroup$
This is where domain knowledge comes in, if you know something about your data beforehand you can use this to decrease model selection time.
$endgroup$
– user2974951
Sep 25 '18 at 13:23
add a comment |
2
$begingroup$
This is where domain knowledge comes in, if you know something about your data beforehand you can use this to decrease model selection time.
$endgroup$
– user2974951
Sep 25 '18 at 13:23
2
2
$begingroup$
This is where domain knowledge comes in, if you know something about your data beforehand you can use this to decrease model selection time.
$endgroup$
– user2974951
Sep 25 '18 at 13:23
$begingroup$
This is where domain knowledge comes in, if you know something about your data beforehand you can use this to decrease model selection time.
$endgroup$
– user2974951
Sep 25 '18 at 13:23
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
Not a complete answer, but was too long for a comment.
I always first try to see how the default parameters perform. Then from the documentation or some reading, you can see what is each parameter global influence (by influence I mean maybe increasing parameter X means complexifying the model, or parameter Y means increasing the convergence speed towards a solution). Depending on the first result you get, pick up one parameter, the one that seem to have the most influence on the model, and make it vary a bit in the way that make sense from your first results. If things improve on the validation set, keep moving the value this way, if not do the opposite. Often times you get good results without tuning every single parameter.
This is a method by hand, it is not optimal. But as you precise that you are a beginner in machine learning, I believe it is the best way to learn to "feel" what usually impact the performance of an algorithm as Xgboost and what impacts less and that therefore can be overlooked for a primary coarse tuning.
https://xgboost.readthedocs.io/en/latest/parameter.html has some nice pieces of information about what parameter impacts what. Don't hesitate to ask more precise questions about some specific parameters if you need :)
answered Sep 25 '18 at 13:03
EskappEskapp
376318
$endgroup$
add a comment |
$begingroup$
I agree with the previous comment on domain knowledge, that will certainly help. As you build experience, you will also get a "feel" for what works. Some parameters work better for NLP, other parameters are more nuanced towards image processing. That's stuff that you're only going to learn after being "in the trenches" for a while.
To build that experience, you could try to build your code in such a way so that you try multiple models, each with their own unique parameters. When I am working with a new dataset, I might create multiple loops and/or threads that each build their own model and I'll compare accuracy and loss rates across all models and then narrow down which parameters I want to adjust. That creates a little more work on your part to create this approach and then track the results, but it is a good way for you to learn about what-does-what and it will help you make better decisions in the future.
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f38766%2fa-way-to-identify-tuning-parameters-and-their-possible-range%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Not a complete answer, but was too long for a comment.
I always first try to see how the default parameters perform. Then from the documentation or some reading, you can see what is each parameter global influence (by influence I mean maybe increasing parameter X means complexifying the model, or parameter Y means increasing the convergence speed towards a solution). Depending on the first result you get, pick up one parameter, the one that seem to have the most influence on the model, and make it vary a bit in the way that make sense from your first results. If things improve on the validation set, keep moving the value this way, if not do the opposite. Often times you get good results without tuning every single parameter.
This is a method by hand, it is not optimal. But as you precise that you are a beginner in machine learning, I believe it is the best way to learn to "feel" what usually impact the performance of an algorithm as Xgboost and what impacts less and that therefore can be overlooked for a primary coarse tuning.
https://xgboost.readthedocs.io/en/latest/parameter.html has some nice pieces of information about what parameter impacts what. Don't hesitate to ask more precise questions about some specific parameters if you need :)
answered Sep 25 '18 at 13:03
EskappEskapp
376318
$endgroup$
add a comment |
$begingroup$
Not a complete answer, but was too long for a comment.
I always first try to see how the default parameters perform. Then from the documentation or some reading, you can see what is each parameter global influence (by influence I mean maybe increasing parameter X means complexifying the model, or parameter Y means increasing the convergence speed towards a solution). Depending on the first result you get, pick up one parameter, the one that seem to have the most influence on the model, and make it vary a bit in the way that make sense from your first results. If things improve on the validation set, keep moving the value this way, if not do the opposite. Often times you get good results without tuning every single parameter.
This is a method by hand, it is not optimal. But as you precise that you are a beginner in machine learning, I believe it is the best way to learn to "feel" what usually impact the performance of an algorithm as Xgboost and what impacts less and that therefore can be overlooked for a primary coarse tuning.
https://xgboost.readthedocs.io/en/latest/parameter.html has some nice pieces of information about what parameter impacts what. Don't hesitate to ask more precise questions about some specific parameters if you need :)
answered Sep 25 '18 at 13:03
EskappEskapp
376318
$endgroup$
add a comment |
$begingroup$
Not a complete answer, but was too long for a comment.
I always first try to see how the default parameters perform. Then from the documentation or some reading, you can see what is each parameter global influence (by influence I mean maybe increasing parameter X means complexifying the model, or parameter Y means increasing the convergence speed towards a solution). Depending on the first result you get, pick up one parameter, the one that seem to have the most influence on the model, and make it vary a bit in the way that make sense from your first results. If things improve on the validation set, keep moving the value this way, if not do the opposite. Often times you get good results without tuning every single parameter.
This is a method by hand, it is not optimal. But as you precise that you are a beginner in machine learning, I believe it is the best way to learn to "feel" what usually impact the performance of an algorithm as Xgboost and what impacts less and that therefore can be overlooked for a primary coarse tuning.
https://xgboost.readthedocs.io/en/latest/parameter.html has some nice pieces of information about what parameter impacts what. Don't hesitate to ask more precise questions about some specific parameters if you need :)
answered Sep 25 '18 at 13:03
EskappEskapp
376318
$endgroup$
Not a complete answer, but was too long for a comment.
I always first try to see how the default parameters perform. Then from the documentation or some reading, you can see what is each parameter global influence (by influence I mean maybe increasing parameter X means complexifying the model, or parameter Y means increasing the convergence speed towards a solution). Depending on the first result you get, pick up one parameter, the one that seem to have the most influence on the model, and make it vary a bit in the way that make sense from your first results. If things improve on the validation set, keep moving the value this way, if not do the opposite. Often times you get good results without tuning every single parameter.
This is a method by hand, it is not optimal. But as you precise that you are a beginner in machine learning, I believe it is the best way to learn to "feel" what usually impact the performance of an algorithm as Xgboost and what impacts less and that therefore can be overlooked for a primary coarse tuning.
https://xgboost.readthedocs.io/en/latest/parameter.html has some nice pieces of information about what parameter impacts what. Don't hesitate to ask more precise questions about some specific parameters if you need :)
answered Sep 25 '18 at 13:03
EskappEskapp
376318
answered Sep 25 '18 at 13:03
EskappEskapp
376318
answered Sep 25 '18 at 13:03
EskappEskapp
376318
answered Sep 25 '18 at 13:03
EskappEskapp
376318
376318
add a comment |
add a comment |
$begingroup$
I agree with the previous comment on domain knowledge, that will certainly help. As you build experience, you will also get a "feel" for what works. Some parameters work better for NLP, other parameters are more nuanced towards image processing. That's stuff that you're only going to learn after being "in the trenches" for a while.
To build that experience, you could try to build your code in such a way so that you try multiple models, each with their own unique parameters. When I am working with a new dataset, I might create multiple loops and/or threads that each build their own model and I'll compare accuracy and loss rates across all models and then narrow down which parameters I want to adjust. That creates a little more work on your part to create this approach and then track the results, but it is a good way for you to learn about what-does-what and it will help you make better decisions in the future.
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
$endgroup$
add a comment |
$begingroup$
I agree with the previous comment on domain knowledge, that will certainly help. As you build experience, you will also get a "feel" for what works. Some parameters work better for NLP, other parameters are more nuanced towards image processing. That's stuff that you're only going to learn after being "in the trenches" for a while.
To build that experience, you could try to build your code in such a way so that you try multiple models, each with their own unique parameters. When I am working with a new dataset, I might create multiple loops and/or threads that each build their own model and I'll compare accuracy and loss rates across all models and then narrow down which parameters I want to adjust. That creates a little more work on your part to create this approach and then track the results, but it is a good way for you to learn about what-does-what and it will help you make better decisions in the future.
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
$endgroup$
add a comment |
$begingroup$
I agree with the previous comment on domain knowledge, that will certainly help. As you build experience, you will also get a "feel" for what works. Some parameters work better for NLP, other parameters are more nuanced towards image processing. That's stuff that you're only going to learn after being "in the trenches" for a while.
To build that experience, you could try to build your code in such a way so that you try multiple models, each with their own unique parameters. When I am working with a new dataset, I might create multiple loops and/or threads that each build their own model and I'll compare accuracy and loss rates across all models and then narrow down which parameters I want to adjust. That creates a little more work on your part to create this approach and then track the results, but it is a good way for you to learn about what-does-what and it will help you make better decisions in the future.
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
$endgroup$
I agree with the previous comment on domain knowledge, that will certainly help. As you build experience, you will also get a "feel" for what works. Some parameters work better for NLP, other parameters are more nuanced towards image processing. That's stuff that you're only going to learn after being "in the trenches" for a while.
To build that experience, you could try to build your code in such a way so that you try multiple models, each with their own unique parameters. When I am working with a new dataset, I might create multiple loops and/or threads that each build their own model and I'll compare accuracy and loss rates across all models and then narrow down which parameters I want to adjust. That creates a little more work on your part to create this approach and then track the results, but it is a good way for you to learn about what-does-what and it will help you make better decisions in the future.
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
answered Oct 25 '18 at 17:38
I_Play_With_DataI_Play_With_Data
1,2521833
1,2521833
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f38766%2fa-way-to-identify-tuning-parameters-and-their-possible-range%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
$begingroup$
This is where domain knowledge comes in, if you know something about your data beforehand you can use this to decrease model selection time.
$endgroup$
– user2974951
Sep 25 '18 at 13:23