Hfrxdjt

Is it possible to have a strip of cold climate in the middle of a planet?

Electoral considerations aside, what are potential benefits, for the US, of policy changes proposed by the tweet recognizing Golan annexation?

Added a new user on Ubuntu, set password not working?

Reverse int within the 32-bit signed integer range: [−2^31, 2^31 − 1]

Count the occurrence of each unique word in the file

Drawing ramified coverings with tikz

Why is it that I can sometimes guess the next note?

What prevents the use of a multi-segment ILS for non-straight approaches?

Longest common substring in linear time

Why can Carol Danvers change her suit colours in the first place?

Where does the bonus feat in the cleric starting package come from?

Does a 'pending' US visa application constitute a denial?

dpdt switch to spst switch

Not using 's' for he/she/it

Open a doc from terminal, but not by its name

Start making guitar arrangements

On a tidally locked planet, would time be quantized?

Does an advisor owe his/her student anything? Will an advisor keep a PhD student only out of pity?

Why electric field inside a cavity of a non-conducting sphere not zero?

Is this toilet slogan correct usage of the English language?

Must Legal Documents Be Siged In Standard Pen Colors?

What should you do when eye contact makes your subordinate uncomfortable?

How do I color the graph in datavisualization?

Has any country ever had 2 former presidents in jail simultaneously?

CNN Back Propagation without Sigmoid Derivative

Back-propagation through max pooling layersSteps for back propagation of convolutional layer in CNNBasic backpropagation questionDeriving backpropagation equations “natively” in tensor formBack Propagation Using MATLABback propagation in CNNA good reference for the back propagation algorithm?Should there be 'total derivative' symbol in the mathematical representation of back-propagation algorithm's formula?Could someone explain to me how back-prop is done for the generator in a GAN?Questions about Neural Network training (back propagation) in the book PRML (Pattern Recognition and Machine Learning)

0

$begingroup$

I'm new to CNN and trying to study some MATLAB sample codes (cause I need to know the internal calculation). I recently realized that the sample code I'm using doesn't multiply error by sigmoid's derivative in back propagation. The feed forward process has sigmoid as last layer's activation function so from my understanding, back propagation error = (outputs - target) * sigmoid's derivative(outputs). However, the author intentionally disabled this multiplication with the following code:

if cnn.loss_func == 'cros'
if cnn.layerscnn.no_of_layers.act_func == 'soft'
cnn.CalcLastLayerActDerivative = 0;
elseif cnn.layerscnn.no_of_layers.act_func == 'sigm'
cnn.CalcLastLayerActDerivative = 0;
end

end

My reference code: https://www.mathworks.com/matlabcentral/fileexchange/59223-convolution-neural-network-simple-code-simple-to-use

When cnn.CalcLastLayerActDerivative = 0, error is defined just as (outputs - target). I tried to initialize cnn.CalcLastLayerActDerivative = 1 so that sigmoid's derivative is considered in back propagation but then I got worse error rate. I'm not sure whether it's just because sigmoid's derivative is in the range [0,0.25] or I'm not understanding back propagation correctly. Does anyone know why this is happening and whether I should add sigmoid's derivative in my calculation?

Thanks!

cnn backpropagation

share

asked 5 mins ago

SylviaSylvia

1

New contributor

Sylvia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

0

$begingroup$

I'm new to CNN and trying to study some MATLAB sample codes (cause I need to know the internal calculation). I recently realized that the sample code I'm using doesn't multiply error by sigmoid's derivative in back propagation. The feed forward process has sigmoid as last layer's activation function so from my understanding, back propagation error = (outputs - target) * sigmoid's derivative(outputs). However, the author intentionally disabled this multiplication with the following code:

if cnn.loss_func == 'cros'
if cnn.layerscnn.no_of_layers.act_func == 'soft'
cnn.CalcLastLayerActDerivative = 0;
elseif cnn.layerscnn.no_of_layers.act_func == 'sigm'
cnn.CalcLastLayerActDerivative = 0;
end

end

My reference code: https://www.mathworks.com/matlabcentral/fileexchange/59223-convolution-neural-network-simple-code-simple-to-use

When cnn.CalcLastLayerActDerivative = 0, error is defined just as (outputs - target). I tried to initialize cnn.CalcLastLayerActDerivative = 1 so that sigmoid's derivative is considered in back propagation but then I got worse error rate. I'm not sure whether it's just because sigmoid's derivative is in the range [0,0.25] or I'm not understanding back propagation correctly. Does anyone know why this is happening and whether I should add sigmoid's derivative in my calculation?

Thanks!

cnn backpropagation

share

asked 5 mins ago

SylviaSylvia

1

New contributor

Sylvia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I'm new to CNN and trying to study some MATLAB sample codes (cause I need to know the internal calculation). I recently realized that the sample code I'm using doesn't multiply error by sigmoid's derivative in back propagation. The feed forward process has sigmoid as last layer's activation function so from my understanding, back propagation error = (outputs - target) * sigmoid's derivative(outputs). However, the author intentionally disabled this multiplication with the following code:

if cnn.loss_func == 'cros'
if cnn.layerscnn.no_of_layers.act_func == 'soft'
cnn.CalcLastLayerActDerivative = 0;
elseif cnn.layerscnn.no_of_layers.act_func == 'sigm'
cnn.CalcLastLayerActDerivative = 0;
end

end

My reference code: https://www.mathworks.com/matlabcentral/fileexchange/59223-convolution-neural-network-simple-code-simple-to-use

When cnn.CalcLastLayerActDerivative = 0, error is defined just as (outputs - target). I tried to initialize cnn.CalcLastLayerActDerivative = 1 so that sigmoid's derivative is considered in back propagation but then I got worse error rate. I'm not sure whether it's just because sigmoid's derivative is in the range [0,0.25] or I'm not understanding back propagation correctly. Does anyone know why this is happening and whether I should add sigmoid's derivative in my calculation?

Thanks!

cnn backpropagation

share

asked 5 mins ago

SylviaSylvia

1

New contributor

Sylvia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share

asked 5 mins ago

SylviaSylvia

1

New contributor

Sylvia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share

asked 5 mins ago

SylviaSylvia

1

New contributor

Sylvia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 5 mins ago

SylviaSylvia

1

New contributor

Sylvia is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 5 mins ago

SylviaSylvia

1