Assigning a value to Y for regression Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsRegression model for a count procesR tutorial for multitask regression tree for missing value prediction?How to get an intuitive value for regression module evaluation?one single value target variableCan a GAN-like architecture be used for maximizing the value of a regression predictor?Regression for non linear dataLinear Model for Linear RegressionSemi-supervised learning for regressionMultiple formulas for r squared eval metric for regressionPredicting descrete value problem in regression or classification
What does Turing mean by this statement?
How does the math work when buying airline miles?
Does silver oxide react with hydrogen sulfide?
The test team as an enemy of development? And how can this be avoided?
One-one communication
Why is std::move not [[nodiscard]] in C++20?
Is multiple magic items in one inherently imbalanced?
Does the Black Tentacles spell do damage twice at the start of turn to an already restrained creature?
Can you force honesty by using the Speak with Dead and Zone of Truth spells together?
Tips to organize LaTeX presentations for a semester
Test print coming out spongy
A term for a woman complaining about things/begging in a cute/childish way
Select every other edge (they share a common vertex)
What is the difference between a "ranged attack" and a "ranged weapon attack"?
Differences to CCompactSize and CVarInt
Getting out of while loop on console
RSA find public exponent
Monty Hall Problem-Probability Paradox
Trying to understand entropy as a novice in thermodynamics
Putting class ranking in CV, but against dept guidelines
Why are vacuum tubes still used in amateur radios?
Universal covering space of the real projective line?
Does any scripture mention that forms of God or Goddess are symbolic?
Special flights
Assigning a value to Y for regression
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsRegression model for a count procesR tutorial for multitask regression tree for missing value prediction?How to get an intuitive value for regression module evaluation?one single value target variableCan a GAN-like architecture be used for maximizing the value of a regression predictor?Regression for non linear dataLinear Model for Linear RegressionSemi-supervised learning for regressionMultiple formulas for r squared eval metric for regressionPredicting descrete value problem in regression or classification
$begingroup$
I'm creating a system to evaluate a risk level that grows as it approaches in time to the crisis event. This risk level ranges from 0 to 100, it's a self made index, totally arbitrary. I have a matrix containing features (X) for each time window. Is there any method to obtain the best model for the values of Y? I mean, knowing how Y should increase (linear, exponential, gaussian) according to the variables I have.
machine-learning regression
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
$endgroup$
bumped to the homepage by Community♦ 35 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I'm creating a system to evaluate a risk level that grows as it approaches in time to the crisis event. This risk level ranges from 0 to 100, it's a self made index, totally arbitrary. I have a matrix containing features (X) for each time window. Is there any method to obtain the best model for the values of Y? I mean, knowing how Y should increase (linear, exponential, gaussian) according to the variables I have.
machine-learning regression
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
$endgroup$
bumped to the homepage by Community♦ 35 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
3
$begingroup$
Why not use the objective, non-arbitrary measure of "time to crisis" as your Y value? Presumably you have a training dataset with that information, why replace that actual data with an uninformative mapping to a new scale?
$endgroup$
– Nuclear Wang
Feb 21 '18 at 20:14
$begingroup$
How many instances do you have in your dataset? How many features do you have?
$endgroup$
– JahKnows
May 23 '18 at 2:18
$begingroup$
Can you describe what your time windows are? Do you have multiple timesteps of features between crisis events? If so you can predict time to next crisis event and then transform the output to the desired scale.
$endgroup$
– Imran
Aug 21 '18 at 1:10
add a comment |
$begingroup$
I'm creating a system to evaluate a risk level that grows as it approaches in time to the crisis event. This risk level ranges from 0 to 100, it's a self made index, totally arbitrary. I have a matrix containing features (X) for each time window. Is there any method to obtain the best model for the values of Y? I mean, knowing how Y should increase (linear, exponential, gaussian) according to the variables I have.
machine-learning regression
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
$endgroup$
I'm creating a system to evaluate a risk level that grows as it approaches in time to the crisis event. This risk level ranges from 0 to 100, it's a self made index, totally arbitrary. I have a matrix containing features (X) for each time window. Is there any method to obtain the best model for the values of Y? I mean, knowing how Y should increase (linear, exponential, gaussian) according to the variables I have.
machine-learning regression
machine-learning regression
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
edited Feb 21 '18 at 18:37
Stephen Rauch♦
1,52551330
1,52551330
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
asked Feb 21 '18 at 18:15
Ernest Montañà OrtizErnest Montañà Ortiz
1
1
bumped to the homepage by Community♦ 35 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 35 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
3
$begingroup$
Why not use the objective, non-arbitrary measure of "time to crisis" as your Y value? Presumably you have a training dataset with that information, why replace that actual data with an uninformative mapping to a new scale?
$endgroup$
– Nuclear Wang
Feb 21 '18 at 20:14
$begingroup$
How many instances do you have in your dataset? How many features do you have?
$endgroup$
– JahKnows
May 23 '18 at 2:18
$begingroup$
Can you describe what your time windows are? Do you have multiple timesteps of features between crisis events? If so you can predict time to next crisis event and then transform the output to the desired scale.
$endgroup$
– Imran
Aug 21 '18 at 1:10
add a comment |
3
$begingroup$
Why not use the objective, non-arbitrary measure of "time to crisis" as your Y value? Presumably you have a training dataset with that information, why replace that actual data with an uninformative mapping to a new scale?
$endgroup$
– Nuclear Wang
Feb 21 '18 at 20:14
$begingroup$
How many instances do you have in your dataset? How many features do you have?
$endgroup$
– JahKnows
May 23 '18 at 2:18
$begingroup$
Can you describe what your time windows are? Do you have multiple timesteps of features between crisis events? If so you can predict time to next crisis event and then transform the output to the desired scale.
$endgroup$
– Imran
Aug 21 '18 at 1:10
3
3
$begingroup$
Why not use the objective, non-arbitrary measure of "time to crisis" as your Y value? Presumably you have a training dataset with that information, why replace that actual data with an uninformative mapping to a new scale?
$endgroup$
– Nuclear Wang
Feb 21 '18 at 20:14
$begingroup$
Why not use the objective, non-arbitrary measure of "time to crisis" as your Y value? Presumably you have a training dataset with that information, why replace that actual data with an uninformative mapping to a new scale?
$endgroup$
– Nuclear Wang
Feb 21 '18 at 20:14
$begingroup$
How many instances do you have in your dataset? How many features do you have?
$endgroup$
– JahKnows
May 23 '18 at 2:18
$begingroup$
How many instances do you have in your dataset? How many features do you have?
$endgroup$
– JahKnows
May 23 '18 at 2:18
$begingroup$
Can you describe what your time windows are? Do you have multiple timesteps of features between crisis events? If so you can predict time to next crisis event and then transform the output to the desired scale.
$endgroup$
– Imran
Aug 21 '18 at 1:10
$begingroup$
Can you describe what your time windows are? Do you have multiple timesteps of features between crisis events? If so you can predict time to next crisis event and then transform the output to the desired scale.
$endgroup$
– Imran
Aug 21 '18 at 1:10
add a comment |
3 Answers
3
active
oldest
votes
$begingroup$
it's a self made index, totally arbitrary
Therefore how it should increase is totally arbitrary too. I think the best answer to your question is that you need to think about the behavior you want to see out of your Y variable, and then write an algorithm that exhibits that behavior.
Do you want the value to suddenly explode as you reach the crisis event? use an exponential. Do you want to gradually approach larger values? Use a linear model. Do you want a function that gets larger until you reach the crisis event and then slowly dissipates? Use a gaussian.
answered Feb 21 '18 at 20:07
RyanRyan
649310
$endgroup$
add a comment |
$begingroup$
You might not want to model your data with a standard regression. It could be more useful to frame it as survival analysis.
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
$endgroup$
add a comment |
$begingroup$
First, you need to make a model to predict the crisis.
Second, you need to output be a probability of crisis instead of zero and one.
There are two kinds of algorithms (dependent on the kind of output it
creates):
Class output : Algorithms like SVM and KNN create a class output. For instance, in a binary classification problem, the outputs will be
either 0 or 1. However, today we have algorithms which can convert
these class outputs to probability. But these algorithms are not well
accepted by the statistics community.
Probability output : Algorithms like Logistic Regression, Random Forest, Gradient Boosting, Adaboost etc. give probability outputs.
Converting probability outputs to class output is just a matter of
creating a threshold probability.
So, choose an appropriate algorithm, and then the shape of output can be related to your policy. If you want a realistic, it should be linear so don't transform it. But if you want to be careful then use exponential transform.
Another way to be more precise is using the cost of a crisis and investigating in modeling. In this way just use 0,1 to decide to investigate or act as normal.
I suggest you use a combinational model.
answered Oct 20 '18 at 13:39
parvijparvij
485214
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f28128%2fassigning-a-value-to-y-for-regression%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
it's a self made index, totally arbitrary
Therefore how it should increase is totally arbitrary too. I think the best answer to your question is that you need to think about the behavior you want to see out of your Y variable, and then write an algorithm that exhibits that behavior.
Do you want the value to suddenly explode as you reach the crisis event? use an exponential. Do you want to gradually approach larger values? Use a linear model. Do you want a function that gets larger until you reach the crisis event and then slowly dissipates? Use a gaussian.
answered Feb 21 '18 at 20:07
RyanRyan
649310
$endgroup$
add a comment |
$begingroup$
it's a self made index, totally arbitrary
Therefore how it should increase is totally arbitrary too. I think the best answer to your question is that you need to think about the behavior you want to see out of your Y variable, and then write an algorithm that exhibits that behavior.
Do you want the value to suddenly explode as you reach the crisis event? use an exponential. Do you want to gradually approach larger values? Use a linear model. Do you want a function that gets larger until you reach the crisis event and then slowly dissipates? Use a gaussian.
answered Feb 21 '18 at 20:07
RyanRyan
649310
$endgroup$
add a comment |
$begingroup$
it's a self made index, totally arbitrary
Therefore how it should increase is totally arbitrary too. I think the best answer to your question is that you need to think about the behavior you want to see out of your Y variable, and then write an algorithm that exhibits that behavior.
Do you want the value to suddenly explode as you reach the crisis event? use an exponential. Do you want to gradually approach larger values? Use a linear model. Do you want a function that gets larger until you reach the crisis event and then slowly dissipates? Use a gaussian.
answered Feb 21 '18 at 20:07
RyanRyan
649310
$endgroup$
it's a self made index, totally arbitrary
Therefore how it should increase is totally arbitrary too. I think the best answer to your question is that you need to think about the behavior you want to see out of your Y variable, and then write an algorithm that exhibits that behavior.
Do you want the value to suddenly explode as you reach the crisis event? use an exponential. Do you want to gradually approach larger values? Use a linear model. Do you want a function that gets larger until you reach the crisis event and then slowly dissipates? Use a gaussian.
answered Feb 21 '18 at 20:07
RyanRyan
649310
answered Feb 21 '18 at 20:07
RyanRyan
649310
answered Feb 21 '18 at 20:07
RyanRyan
649310
answered Feb 21 '18 at 20:07
RyanRyan
649310
649310
add a comment |
add a comment |
$begingroup$
You might not want to model your data with a standard regression. It could be more useful to frame it as survival analysis.
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
$endgroup$
add a comment |
$begingroup$
You might not want to model your data with a standard regression. It could be more useful to frame it as survival analysis.
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
$endgroup$
add a comment |
$begingroup$
You might not want to model your data with a standard regression. It could be more useful to frame it as survival analysis.
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
$endgroup$
You might not want to model your data with a standard regression. It could be more useful to frame it as survival analysis.
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
answered Aug 21 '18 at 1:29
Brian SpieringBrian Spiering
4,2881129
4,2881129
add a comment |
add a comment |
$begingroup$
First, you need to make a model to predict the crisis.
Second, you need to output be a probability of crisis instead of zero and one.
There are two kinds of algorithms (dependent on the kind of output it
creates):
Class output : Algorithms like SVM and KNN create a class output. For instance, in a binary classification problem, the outputs will be
either 0 or 1. However, today we have algorithms which can convert
these class outputs to probability. But these algorithms are not well
accepted by the statistics community.
Probability output : Algorithms like Logistic Regression, Random Forest, Gradient Boosting, Adaboost etc. give probability outputs.
Converting probability outputs to class output is just a matter of
creating a threshold probability.
So, choose an appropriate algorithm, and then the shape of output can be related to your policy. If you want a realistic, it should be linear so don't transform it. But if you want to be careful then use exponential transform.
Another way to be more precise is using the cost of a crisis and investigating in modeling. In this way just use 0,1 to decide to investigate or act as normal.
I suggest you use a combinational model.
answered Oct 20 '18 at 13:39
parvijparvij
485214
$endgroup$
add a comment |
$begingroup$
First, you need to make a model to predict the crisis.
Second, you need to output be a probability of crisis instead of zero and one.
There are two kinds of algorithms (dependent on the kind of output it
creates):
Class output : Algorithms like SVM and KNN create a class output. For instance, in a binary classification problem, the outputs will be
either 0 or 1. However, today we have algorithms which can convert
these class outputs to probability. But these algorithms are not well
accepted by the statistics community.
Probability output : Algorithms like Logistic Regression, Random Forest, Gradient Boosting, Adaboost etc. give probability outputs.
Converting probability outputs to class output is just a matter of
creating a threshold probability.
So, choose an appropriate algorithm, and then the shape of output can be related to your policy. If you want a realistic, it should be linear so don't transform it. But if you want to be careful then use exponential transform.
Another way to be more precise is using the cost of a crisis and investigating in modeling. In this way just use 0,1 to decide to investigate or act as normal.
I suggest you use a combinational model.
answered Oct 20 '18 at 13:39
parvijparvij
485214
$endgroup$
add a comment |
$begingroup$
First, you need to make a model to predict the crisis.
Second, you need to output be a probability of crisis instead of zero and one.
There are two kinds of algorithms (dependent on the kind of output it
creates):
Class output : Algorithms like SVM and KNN create a class output. For instance, in a binary classification problem, the outputs will be
either 0 or 1. However, today we have algorithms which can convert
these class outputs to probability. But these algorithms are not well
accepted by the statistics community.
Probability output : Algorithms like Logistic Regression, Random Forest, Gradient Boosting, Adaboost etc. give probability outputs.
Converting probability outputs to class output is just a matter of
creating a threshold probability.
So, choose an appropriate algorithm, and then the shape of output can be related to your policy. If you want a realistic, it should be linear so don't transform it. But if you want to be careful then use exponential transform.
Another way to be more precise is using the cost of a crisis and investigating in modeling. In this way just use 0,1 to decide to investigate or act as normal.
I suggest you use a combinational model.
answered Oct 20 '18 at 13:39
parvijparvij
485214
$endgroup$
First, you need to make a model to predict the crisis.
Second, you need to output be a probability of crisis instead of zero and one.
There are two kinds of algorithms (dependent on the kind of output it
creates):
Class output : Algorithms like SVM and KNN create a class output. For instance, in a binary classification problem, the outputs will be
either 0 or 1. However, today we have algorithms which can convert
these class outputs to probability. But these algorithms are not well
accepted by the statistics community.
Probability output : Algorithms like Logistic Regression, Random Forest, Gradient Boosting, Adaboost etc. give probability outputs.
Converting probability outputs to class output is just a matter of
creating a threshold probability.
So, choose an appropriate algorithm, and then the shape of output can be related to your policy. If you want a realistic, it should be linear so don't transform it. But if you want to be careful then use exponential transform.
Another way to be more precise is using the cost of a crisis and investigating in modeling. In this way just use 0,1 to decide to investigate or act as normal.
I suggest you use a combinational model.
answered Oct 20 '18 at 13:39
parvijparvij
485214
answered Oct 20 '18 at 13:39
parvijparvij
485214
answered Oct 20 '18 at 13:39
parvijparvij
485214
answered Oct 20 '18 at 13:39
parvijparvij
485214
485214
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f28128%2fassigning-a-value-to-y-for-regression%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
$begingroup$
Why not use the objective, non-arbitrary measure of "time to crisis" as your Y value? Presumably you have a training dataset with that information, why replace that actual data with an uninformative mapping to a new scale?
$endgroup$
– Nuclear Wang
Feb 21 '18 at 20:14
$begingroup$
How many instances do you have in your dataset? How many features do you have?
$endgroup$
– JahKnows
May 23 '18 at 2:18
$begingroup$
Can you describe what your time windows are? Do you have multiple timesteps of features between crisis events? If so you can predict time to next crisis event and then transform the output to the desired scale.
$endgroup$
– Imran
Aug 21 '18 at 1:10