Basis expansion for regression using neural network?2019 Community Moderator ElectionApplying neural network for simple x^2 function for demonstration purposeNeural Network Timeseries Modeling with Predictor VariablesNeural Network accuracy and loss guarantees?Does it ever make sense for upper layers to have more nodes than lower layers?TensorFlow: Regression using Deep Neural NetworkError in Neural NetworkOptimising Kernel parameters using training data in GaussianProcessRegressor of Scikit-learnRephrase Neural Network : Find the Fittest Word for Given Meaning/Explanationtraining neural network
Is it inappropriate for a student to attend their mentor's dissertation defense?
How to tell a function to use the default argument values?
Theorists sure want true answers to this!
In 'Revenger,' what does 'cove' come from?
Bullying boss launched a smear campaign and made me unemployable
Venezuelan girlfriend wants to travel the USA to be with me. What is the process?
Can my sorcerer use a spellbook only to collect spells and scribe scrolls, not cast?
Ambiguity in the definition of entropy
Different meanings of こわい
How do I handle a potential work/personal life conflict as the manager of one of my friends?
ssTTsSTtRrriinInnnnNNNIiinngg
Plagiarism or not?
Is it logically or scientifically possible to artificially send energy to the body?
Valid term from quadratic sequence?
Can we compute the area of a quadrilateral with one right angle when we only know the lengths of any three sides?
How to add frame around section using titlesec?
Forming a German sentence with/without the verb at the end
Is it acceptable for a professor to tell male students to not think that they are smarter than female students?
Would Slavery Reparations be considered Bills of Attainder and hence Illegal?
How do I deal with an unproductive colleague in a small company?
How would I stat a creature to be immune to everything but the Magic Missile spell? (just for fun)
Probability that a draw from a normal distribution is some number greater than another draw from the same distribution
Is "remove commented out code" correct English?
What's the in-universe reasoning behind sorcerers needing material components?
Basis expansion for regression using neural network?
2019 Community Moderator ElectionApplying neural network for simple x^2 function for demonstration purposeNeural Network Timeseries Modeling with Predictor VariablesNeural Network accuracy and loss guarantees?Does it ever make sense for upper layers to have more nodes than lower layers?TensorFlow: Regression using Deep Neural NetworkError in Neural NetworkOptimising Kernel parameters using training data in GaussianProcessRegressor of Scikit-learnRephrase Neural Network : Find the Fittest Word for Given Meaning/Explanationtraining neural network
$begingroup$
I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data set.
I asked someone and he suggested that basis expansion might help. Can someone please provide a reference for the same, I am not able to find any. Also, he suggested "basis expansion using kernel method".
machine-learning neural-network deep-learning regression kernel
edited Mar 4 at 20:18
Media
7,48762263
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
$endgroup$
bumped to the homepage by Community♦ 22 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data set.
I asked someone and he suggested that basis expansion might help. Can someone please provide a reference for the same, I am not able to find any. Also, he suggested "basis expansion using kernel method".
machine-learning neural-network deep-learning regression kernel
edited Mar 4 at 20:18
Media
7,48762263
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
$endgroup$
bumped to the homepage by Community♦ 22 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data set.
I asked someone and he suggested that basis expansion might help. Can someone please provide a reference for the same, I am not able to find any. Also, he suggested "basis expansion using kernel method".
machine-learning neural-network deep-learning regression kernel
edited Mar 4 at 20:18
Media
7,48762263
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
$endgroup$
I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data set.
I asked someone and he suggested that basis expansion might help. Can someone please provide a reference for the same, I am not able to find any. Also, he suggested "basis expansion using kernel method".
machine-learning neural-network deep-learning regression kernel
machine-learning neural-network deep-learning regression kernel
edited Mar 4 at 20:18
Media
7,48762263
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
edited Mar 4 at 20:18
Media
7,48762263
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
edited Mar 4 at 20:18
Media
7,48762263
edited Mar 4 at 20:18
Media
7,48762263
edited Mar 4 at 20:18
Media
7,48762263
7,48762263
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
asked Mar 4 at 17:25
Adarsh GuptaAdarsh Gupta
63
63
bumped to the homepage by Community♦ 22 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 22 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
I guess expanding basis is somehow looking at the problem using linear algebra perspective. For explaining it using a trivial example, in linear algebra, for the 3D space, there are three bases namely, $i$, $j$ and $k$. It means for referring to any vector in 3D space you just need a linear combination of these three vectors to construct any desired vectors. Each basis for each space has a number of properties like being orthogonal or not or maybe being linearly independent and some other important properties. By adding extra features, you increase the number of features if they are linearly independent.
In machine learning, there are situations that comparing each feature with the output directly may not lead to a good decision boundary while using some features alongside each other can make decision boundaries which can lead to a better decision boundary. All of that means that if you change the input space to space with more number of features which the features are extracted using the current feature space, you can find a space that the data can be separated better. Finding a feature space such with properties can be done using feature extraction methods. Kernel methods are some of them.
Kernel trick is a kind of mathematical approach for reducing the difficulties of transforming the data in the current feature space to the appropriate feature space. Suppose you have one million rows of raw data and the feature space belongs to $R^3$. Suppose you find out that a feature space that can have a good generalisation belongs to the space which belongs to $R^95$. Trying to transform the data in hand to the desired feature space is a time-consuming task if possible. Instead of transforming the data directly, the kernel trick tries to calculate the outcome of the transformed data without transforming the data into the new space.
answered Mar 4 at 20:16
MediaMedia
7,48762263
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46652%2fbasis-expansion-for-regression-using-neural-network%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I guess expanding basis is somehow looking at the problem using linear algebra perspective. For explaining it using a trivial example, in linear algebra, for the 3D space, there are three bases namely, $i$, $j$ and $k$. It means for referring to any vector in 3D space you just need a linear combination of these three vectors to construct any desired vectors. Each basis for each space has a number of properties like being orthogonal or not or maybe being linearly independent and some other important properties. By adding extra features, you increase the number of features if they are linearly independent.
In machine learning, there are situations that comparing each feature with the output directly may not lead to a good decision boundary while using some features alongside each other can make decision boundaries which can lead to a better decision boundary. All of that means that if you change the input space to space with more number of features which the features are extracted using the current feature space, you can find a space that the data can be separated better. Finding a feature space such with properties can be done using feature extraction methods. Kernel methods are some of them.
Kernel trick is a kind of mathematical approach for reducing the difficulties of transforming the data in the current feature space to the appropriate feature space. Suppose you have one million rows of raw data and the feature space belongs to $R^3$. Suppose you find out that a feature space that can have a good generalisation belongs to the space which belongs to $R^95$. Trying to transform the data in hand to the desired feature space is a time-consuming task if possible. Instead of transforming the data directly, the kernel trick tries to calculate the outcome of the transformed data without transforming the data into the new space.
answered Mar 4 at 20:16
MediaMedia
7,48762263
$endgroup$
add a comment |
$begingroup$
I guess expanding basis is somehow looking at the problem using linear algebra perspective. For explaining it using a trivial example, in linear algebra, for the 3D space, there are three bases namely, $i$, $j$ and $k$. It means for referring to any vector in 3D space you just need a linear combination of these three vectors to construct any desired vectors. Each basis for each space has a number of properties like being orthogonal or not or maybe being linearly independent and some other important properties. By adding extra features, you increase the number of features if they are linearly independent.
In machine learning, there are situations that comparing each feature with the output directly may not lead to a good decision boundary while using some features alongside each other can make decision boundaries which can lead to a better decision boundary. All of that means that if you change the input space to space with more number of features which the features are extracted using the current feature space, you can find a space that the data can be separated better. Finding a feature space such with properties can be done using feature extraction methods. Kernel methods are some of them.
Kernel trick is a kind of mathematical approach for reducing the difficulties of transforming the data in the current feature space to the appropriate feature space. Suppose you have one million rows of raw data and the feature space belongs to $R^3$. Suppose you find out that a feature space that can have a good generalisation belongs to the space which belongs to $R^95$. Trying to transform the data in hand to the desired feature space is a time-consuming task if possible. Instead of transforming the data directly, the kernel trick tries to calculate the outcome of the transformed data without transforming the data into the new space.
answered Mar 4 at 20:16
MediaMedia
7,48762263
$endgroup$
add a comment |
$begingroup$
I guess expanding basis is somehow looking at the problem using linear algebra perspective. For explaining it using a trivial example, in linear algebra, for the 3D space, there are three bases namely, $i$, $j$ and $k$. It means for referring to any vector in 3D space you just need a linear combination of these three vectors to construct any desired vectors. Each basis for each space has a number of properties like being orthogonal or not or maybe being linearly independent and some other important properties. By adding extra features, you increase the number of features if they are linearly independent.
In machine learning, there are situations that comparing each feature with the output directly may not lead to a good decision boundary while using some features alongside each other can make decision boundaries which can lead to a better decision boundary. All of that means that if you change the input space to space with more number of features which the features are extracted using the current feature space, you can find a space that the data can be separated better. Finding a feature space such with properties can be done using feature extraction methods. Kernel methods are some of them.
Kernel trick is a kind of mathematical approach for reducing the difficulties of transforming the data in the current feature space to the appropriate feature space. Suppose you have one million rows of raw data and the feature space belongs to $R^3$. Suppose you find out that a feature space that can have a good generalisation belongs to the space which belongs to $R^95$. Trying to transform the data in hand to the desired feature space is a time-consuming task if possible. Instead of transforming the data directly, the kernel trick tries to calculate the outcome of the transformed data without transforming the data into the new space.
answered Mar 4 at 20:16
MediaMedia
7,48762263
$endgroup$
I guess expanding basis is somehow looking at the problem using linear algebra perspective. For explaining it using a trivial example, in linear algebra, for the 3D space, there are three bases namely, $i$, $j$ and $k$. It means for referring to any vector in 3D space you just need a linear combination of these three vectors to construct any desired vectors. Each basis for each space has a number of properties like being orthogonal or not or maybe being linearly independent and some other important properties. By adding extra features, you increase the number of features if they are linearly independent.
In machine learning, there are situations that comparing each feature with the output directly may not lead to a good decision boundary while using some features alongside each other can make decision boundaries which can lead to a better decision boundary. All of that means that if you change the input space to space with more number of features which the features are extracted using the current feature space, you can find a space that the data can be separated better. Finding a feature space such with properties can be done using feature extraction methods. Kernel methods are some of them.
Kernel trick is a kind of mathematical approach for reducing the difficulties of transforming the data in the current feature space to the appropriate feature space. Suppose you have one million rows of raw data and the feature space belongs to $R^3$. Suppose you find out that a feature space that can have a good generalisation belongs to the space which belongs to $R^95$. Trying to transform the data in hand to the desired feature space is a time-consuming task if possible. Instead of transforming the data directly, the kernel trick tries to calculate the outcome of the transformed data without transforming the data into the new space.
answered Mar 4 at 20:16
MediaMedia
7,48762263
edited Mar 4 at 20:23
answered Mar 4 at 20:16
MediaMedia
7,48762263
answered Mar 4 at 20:16
MediaMedia
7,48762263
answered Mar 4 at 20:16
MediaMedia
7,48762263
7,48762263
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46652%2fbasis-expansion-for-regression-using-neural-network%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
