Deep learning(MLP) on multiclass classification. Model learns only one class Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsNeural net learning only one class?Random Forest Multiclass ClassificationEvaluate a model based on precision for multi class classificationdata pre-processing before feeding into a deep learning modelUnblanced classes: classifier only predict one classHow can I improve the recall of a certain class in a multiclass-classification resultsolving multi-class imbalance classification using smote and OSSValidation loss increases and validation accuracy decreasesMulticlass class classification for text documentXGBoost multiclass class balancing using weight parameter
How to override model in magento2?
Is the Standard Deduction better than Itemized when both are the same amount?
Why light coming from distant stars is not discrete?
Why are Kinder Surprise Eggs illegal in the USA?
What would be the ideal power source for a cybernetic eye?
Why am I getting the error "non-boolean type specified in a context where a condition is expected" for this request?
Is it ethical to give a final exam after the professor has quit before teaching the remaining chapters of the course?
Using et al. for a last / senior author rather than for a first author
What does F' and F" mean?
How do I stop a creek from eroding my steep embankment?
English words in a non-english sci-fi novel
Can a USB port passively 'listen only'?
When were vectors invented?
Why are both D and D# fitting into my E minor key?
What does this icon in iOS Stardew Valley mean?
Should I discuss the type of campaign with my players?
Using audio cues to encourage good posture
Should I use a zero-interest credit card for a large one-time purchase?
How to find all the available tools in mac terminal?
How does the particle を relate to the verb 行く in the structure「A を + B に行く」?
Okay to merge included columns on otherwise identical indexes?
Why is "Consequences inflicted." not a sentence?
String `!23` is replaced with `docker` in command line
Why did the IBM 650 use bi-quinary?
Deep learning(MLP) on multiclass classification. Model learns only one class
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsNeural net learning only one class?Random Forest Multiclass ClassificationEvaluate a model based on precision for multi class classificationdata pre-processing before feeding into a deep learning modelUnblanced classes: classifier only predict one classHow can I improve the recall of a certain class in a multiclass-classification resultsolving multi-class imbalance classification using smote and OSSValidation loss increases and validation accuracy decreasesMulticlass class classification for text documentXGBoost multiclass class balancing using weight parameter
0
$begingroup$
I am new to deep learning. I have imbalanced class data. I used one hot encoding and scaling to preprocess my data. I have used adamoptimizer as optimizer function and sparse categorical crossentropy as my lass function. The model always gives high accuracy on one class with very low accuracy on other classes. Here is my code:
`
#separating test data according to classes
data_test = data_final[data_final.YEAR.isin(2018)]
data_test_0 = data_test[data_test['DELAY_CLASS']==0]
test_labels_0 = data_test_0.pop('DELAY_CLASS')
data_test_1 = data_test[data_test['DELAY_CLASS']==1]
test_labels_1 = data_test_1.pop('DELAY_CLASS')
data_test_2 = data_test[data_test['DELAY_CLASS']==2]
test_labels_2 = data_test_2.pop('DELAY_CLASS')
data_test_3 = data_test[data_test['DELAY_CLASS']==3]
test_labels_3 = data_test_3.pop('DELAY_CLASS')
#Extracting continuous columns from training data
data_train = data_train[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#Extracting continuous columns from testing data
data_test = data_test[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
print("reached here")
#SMOTE
sm = SMOTE(random_state=2)
ad = ADASYN(random_state=2)
data_train, train_labels = sm.fit_sample(data_train, train_labels)
data_train = pd.DataFrame(data_train)
data_train = data_train.rename(columns = 0:'MONTH',1:'DAY_OF_MONTH',2:'DAY_OF_WEEK',3:'Dep_Hour',
4:'Arr_Hour', 5:'CRS_ELAPSED_TIME', 6:'DISTANCE',
7:'traffic',8:'O_SurfaceTemperatureFahrenheit',9:'O_CloudCoveragePercent',
10:'O_WindSpeedMph',11:'O_PrecipitationPreviousHourInches',12:'O_SnowfallInches',
13:'D_SurfaceTemperatureFahrenheit',14:'D_CloudCoveragePercent',15:'D_WindSpeedMph',
16:'D_PrecipitationPreviousHourInches',17:'D_SnowfallInches',18:'Bird_Strike')
#taking only continuous columns
cols = ['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']
#scaling
train_mean = data_train[cols].mean(axis=0)
train_std = data_train[cols].std(axis=0)
data_train[cols] = (data_train[cols] - train_mean) / train_std
data_test[cols] = (data_test[cols] - train_mean) / train_std
rain_labels = pd.Series(train_labels)
#taking continuous columns from test separated data
data_test_0 = data_test_0[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_1 = data_test_1[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK','Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_2 = data_test_2[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_3 = data_test_3[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#my model
def build_model():
model = keras.Sequential([
layers.Dense(100, activation = 'sigmoid', input_shape=[len(data_train.keys())]),
#layers.Dropout(0.5),
layers.Dense(50, activation = 'softplus'),
#layers.Dropout(0.3),
layers.Dense(25, activation = 'sigmoid'),
#layers.Dropout(0.2),
layers.Dense(4, activation = 'softmax')
])
model.compile(loss='sparse_categorical_crossentropy',#with binary crossentropy use sigmoid and 1 output neuron
optimizer= tf.train.AdamOptimizer(0.001),
metrics=['accuracy'])
return model
model = build_model()
model.fit(data_train, train_labels, epochs=5, batch_size=128)
test_loss, test_acc = model.evaluate(data_test_0, test_labels_0)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_1, test_labels_1)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_2, test_labels_2)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_3, test_labels_3)
print(test_acc)
`
The training data is flights data of 2016 and 2017 and testing data is of 2018. I have separated classes from testing data to see the class wise accuracy of testing data.
The output is:
Epoch 1/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.3231 - acc: 0.3466
Epoch 2/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2799 - acc: 0.3821
Epoch 3/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2634 - acc: 0.3939
Epoch 4/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2519 - acc: 0.4013
Epoch 5/5
1990363/1990363 [==============================] - 16s 8us/step - loss: 1.2445 - acc: 0.4068
Class 0:
44929/44929 [==============================] - 1s 12us/step
0.027710387500278218
Class 1:
10668/10668 [==============================] - 0s 11us/step
0.015935508061492312
Class 2:
33204/33204 [==============================] - 0s 9us/step
0.8956149861318866
Class 3:
274983/274983 [==============================] - 2s 9us/step
0.035293090845941046
The output remains somewhat same if I use adasyn instead of SMOTE or change layers and activation functions. Please help me out.
Thanks in advance.
deep-learning multiclass-classification mlp smote imbalanced-learn
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
0
$begingroup$
I am new to deep learning. I have imbalanced class data. I used one hot encoding and scaling to preprocess my data. I have used adamoptimizer as optimizer function and sparse categorical crossentropy as my lass function. The model always gives high accuracy on one class with very low accuracy on other classes. Here is my code:
`
#separating test data according to classes
data_test = data_final[data_final.YEAR.isin(2018)]
data_test_0 = data_test[data_test['DELAY_CLASS']==0]
test_labels_0 = data_test_0.pop('DELAY_CLASS')
data_test_1 = data_test[data_test['DELAY_CLASS']==1]
test_labels_1 = data_test_1.pop('DELAY_CLASS')
data_test_2 = data_test[data_test['DELAY_CLASS']==2]
test_labels_2 = data_test_2.pop('DELAY_CLASS')
data_test_3 = data_test[data_test['DELAY_CLASS']==3]
test_labels_3 = data_test_3.pop('DELAY_CLASS')
#Extracting continuous columns from training data
data_train = data_train[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#Extracting continuous columns from testing data
data_test = data_test[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
print("reached here")
#SMOTE
sm = SMOTE(random_state=2)
ad = ADASYN(random_state=2)
data_train, train_labels = sm.fit_sample(data_train, train_labels)
data_train = pd.DataFrame(data_train)
data_train = data_train.rename(columns = 0:'MONTH',1:'DAY_OF_MONTH',2:'DAY_OF_WEEK',3:'Dep_Hour',
4:'Arr_Hour', 5:'CRS_ELAPSED_TIME', 6:'DISTANCE',
7:'traffic',8:'O_SurfaceTemperatureFahrenheit',9:'O_CloudCoveragePercent',
10:'O_WindSpeedMph',11:'O_PrecipitationPreviousHourInches',12:'O_SnowfallInches',
13:'D_SurfaceTemperatureFahrenheit',14:'D_CloudCoveragePercent',15:'D_WindSpeedMph',
16:'D_PrecipitationPreviousHourInches',17:'D_SnowfallInches',18:'Bird_Strike')
#taking only continuous columns
cols = ['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']
#scaling
train_mean = data_train[cols].mean(axis=0)
train_std = data_train[cols].std(axis=0)
data_train[cols] = (data_train[cols] - train_mean) / train_std
data_test[cols] = (data_test[cols] - train_mean) / train_std
rain_labels = pd.Series(train_labels)
#taking continuous columns from test separated data
data_test_0 = data_test_0[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_1 = data_test_1[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK','Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_2 = data_test_2[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_3 = data_test_3[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#my model
def build_model():
model = keras.Sequential([
layers.Dense(100, activation = 'sigmoid', input_shape=[len(data_train.keys())]),
#layers.Dropout(0.5),
layers.Dense(50, activation = 'softplus'),
#layers.Dropout(0.3),
layers.Dense(25, activation = 'sigmoid'),
#layers.Dropout(0.2),
layers.Dense(4, activation = 'softmax')
])
model.compile(loss='sparse_categorical_crossentropy',#with binary crossentropy use sigmoid and 1 output neuron
optimizer= tf.train.AdamOptimizer(0.001),
metrics=['accuracy'])
return model
model = build_model()
model.fit(data_train, train_labels, epochs=5, batch_size=128)
test_loss, test_acc = model.evaluate(data_test_0, test_labels_0)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_1, test_labels_1)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_2, test_labels_2)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_3, test_labels_3)
print(test_acc)
`
The training data is flights data of 2016 and 2017 and testing data is of 2018. I have separated classes from testing data to see the class wise accuracy of testing data.
The output is:
Epoch 1/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.3231 - acc: 0.3466
Epoch 2/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2799 - acc: 0.3821
Epoch 3/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2634 - acc: 0.3939
Epoch 4/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2519 - acc: 0.4013
Epoch 5/5
1990363/1990363 [==============================] - 16s 8us/step - loss: 1.2445 - acc: 0.4068
Class 0:
44929/44929 [==============================] - 1s 12us/step
0.027710387500278218
Class 1:
10668/10668 [==============================] - 0s 11us/step
0.015935508061492312
Class 2:
33204/33204 [==============================] - 0s 9us/step
0.8956149861318866
Class 3:
274983/274983 [==============================] - 2s 9us/step
0.035293090845941046
The output remains somewhat same if I use adasyn instead of SMOTE or change layers and activation functions. Please help me out.
Thanks in advance.
deep-learning multiclass-classification mlp smote imbalanced-learn
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
0
0
0
$begingroup$
I am new to deep learning. I have imbalanced class data. I used one hot encoding and scaling to preprocess my data. I have used adamoptimizer as optimizer function and sparse categorical crossentropy as my lass function. The model always gives high accuracy on one class with very low accuracy on other classes. Here is my code:
`
#separating test data according to classes
data_test = data_final[data_final.YEAR.isin(2018)]
data_test_0 = data_test[data_test['DELAY_CLASS']==0]
test_labels_0 = data_test_0.pop('DELAY_CLASS')
data_test_1 = data_test[data_test['DELAY_CLASS']==1]
test_labels_1 = data_test_1.pop('DELAY_CLASS')
data_test_2 = data_test[data_test['DELAY_CLASS']==2]
test_labels_2 = data_test_2.pop('DELAY_CLASS')
data_test_3 = data_test[data_test['DELAY_CLASS']==3]
test_labels_3 = data_test_3.pop('DELAY_CLASS')
#Extracting continuous columns from training data
data_train = data_train[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#Extracting continuous columns from testing data
data_test = data_test[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
print("reached here")
#SMOTE
sm = SMOTE(random_state=2)
ad = ADASYN(random_state=2)
data_train, train_labels = sm.fit_sample(data_train, train_labels)
data_train = pd.DataFrame(data_train)
data_train = data_train.rename(columns = 0:'MONTH',1:'DAY_OF_MONTH',2:'DAY_OF_WEEK',3:'Dep_Hour',
4:'Arr_Hour', 5:'CRS_ELAPSED_TIME', 6:'DISTANCE',
7:'traffic',8:'O_SurfaceTemperatureFahrenheit',9:'O_CloudCoveragePercent',
10:'O_WindSpeedMph',11:'O_PrecipitationPreviousHourInches',12:'O_SnowfallInches',
13:'D_SurfaceTemperatureFahrenheit',14:'D_CloudCoveragePercent',15:'D_WindSpeedMph',
16:'D_PrecipitationPreviousHourInches',17:'D_SnowfallInches',18:'Bird_Strike')
#taking only continuous columns
cols = ['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']
#scaling
train_mean = data_train[cols].mean(axis=0)
train_std = data_train[cols].std(axis=0)
data_train[cols] = (data_train[cols] - train_mean) / train_std
data_test[cols] = (data_test[cols] - train_mean) / train_std
rain_labels = pd.Series(train_labels)
#taking continuous columns from test separated data
data_test_0 = data_test_0[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_1 = data_test_1[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK','Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_2 = data_test_2[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_3 = data_test_3[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#my model
def build_model():
model = keras.Sequential([
layers.Dense(100, activation = 'sigmoid', input_shape=[len(data_train.keys())]),
#layers.Dropout(0.5),
layers.Dense(50, activation = 'softplus'),
#layers.Dropout(0.3),
layers.Dense(25, activation = 'sigmoid'),
#layers.Dropout(0.2),
layers.Dense(4, activation = 'softmax')
])
model.compile(loss='sparse_categorical_crossentropy',#with binary crossentropy use sigmoid and 1 output neuron
optimizer= tf.train.AdamOptimizer(0.001),
metrics=['accuracy'])
return model
model = build_model()
model.fit(data_train, train_labels, epochs=5, batch_size=128)
test_loss, test_acc = model.evaluate(data_test_0, test_labels_0)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_1, test_labels_1)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_2, test_labels_2)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_3, test_labels_3)
print(test_acc)
`
The training data is flights data of 2016 and 2017 and testing data is of 2018. I have separated classes from testing data to see the class wise accuracy of testing data.
The output is:
Epoch 1/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.3231 - acc: 0.3466
Epoch 2/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2799 - acc: 0.3821
Epoch 3/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2634 - acc: 0.3939
Epoch 4/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2519 - acc: 0.4013
Epoch 5/5
1990363/1990363 [==============================] - 16s 8us/step - loss: 1.2445 - acc: 0.4068
Class 0:
44929/44929 [==============================] - 1s 12us/step
0.027710387500278218
Class 1:
10668/10668 [==============================] - 0s 11us/step
0.015935508061492312
Class 2:
33204/33204 [==============================] - 0s 9us/step
0.8956149861318866
Class 3:
274983/274983 [==============================] - 2s 9us/step
0.035293090845941046
The output remains somewhat same if I use adasyn instead of SMOTE or change layers and activation functions. Please help me out.
Thanks in advance.
deep-learning multiclass-classification mlp smote imbalanced-learn
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I am new to deep learning. I have imbalanced class data. I used one hot encoding and scaling to preprocess my data. I have used adamoptimizer as optimizer function and sparse categorical crossentropy as my lass function. The model always gives high accuracy on one class with very low accuracy on other classes. Here is my code:
`
#separating test data according to classes
data_test = data_final[data_final.YEAR.isin(2018)]
data_test_0 = data_test[data_test['DELAY_CLASS']==0]
test_labels_0 = data_test_0.pop('DELAY_CLASS')
data_test_1 = data_test[data_test['DELAY_CLASS']==1]
test_labels_1 = data_test_1.pop('DELAY_CLASS')
data_test_2 = data_test[data_test['DELAY_CLASS']==2]
test_labels_2 = data_test_2.pop('DELAY_CLASS')
data_test_3 = data_test[data_test['DELAY_CLASS']==3]
test_labels_3 = data_test_3.pop('DELAY_CLASS')
#Extracting continuous columns from training data
data_train = data_train[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#Extracting continuous columns from testing data
data_test = data_test[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
print("reached here")
#SMOTE
sm = SMOTE(random_state=2)
ad = ADASYN(random_state=2)
data_train, train_labels = sm.fit_sample(data_train, train_labels)
data_train = pd.DataFrame(data_train)
data_train = data_train.rename(columns = 0:'MONTH',1:'DAY_OF_MONTH',2:'DAY_OF_WEEK',3:'Dep_Hour',
4:'Arr_Hour', 5:'CRS_ELAPSED_TIME', 6:'DISTANCE',
7:'traffic',8:'O_SurfaceTemperatureFahrenheit',9:'O_CloudCoveragePercent',
10:'O_WindSpeedMph',11:'O_PrecipitationPreviousHourInches',12:'O_SnowfallInches',
13:'D_SurfaceTemperatureFahrenheit',14:'D_CloudCoveragePercent',15:'D_WindSpeedMph',
16:'D_PrecipitationPreviousHourInches',17:'D_SnowfallInches',18:'Bird_Strike')
#taking only continuous columns
cols = ['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']
#scaling
train_mean = data_train[cols].mean(axis=0)
train_std = data_train[cols].std(axis=0)
data_train[cols] = (data_train[cols] - train_mean) / train_std
data_test[cols] = (data_test[cols] - train_mean) / train_std
rain_labels = pd.Series(train_labels)
#taking continuous columns from test separated data
data_test_0 = data_test_0[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_1 = data_test_1[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK','Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit','D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_2 = data_test_2[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
data_test_3 = data_test_3[['MONTH','DAY_OF_MONTH','DAY_OF_WEEK',
'Dep_Hour','Arr_Hour','CRS_ELAPSED_TIME','DISTANCE','traffic','O_SurfaceTemperatureFahrenheit','O_CloudCoveragePercent','O_WindSpeedMph','O_PrecipitationPreviousHourInches','O_SnowfallInches','D_SurfaceTemperatureFahrenheit',
'D_CloudCoveragePercent','D_WindSpeedMph','D_PrecipitationPreviousHourInches','D_SnowfallInches','Bird_Strike']]
#my model
def build_model():
model = keras.Sequential([
layers.Dense(100, activation = 'sigmoid', input_shape=[len(data_train.keys())]),
#layers.Dropout(0.5),
layers.Dense(50, activation = 'softplus'),
#layers.Dropout(0.3),
layers.Dense(25, activation = 'sigmoid'),
#layers.Dropout(0.2),
layers.Dense(4, activation = 'softmax')
])
model.compile(loss='sparse_categorical_crossentropy',#with binary crossentropy use sigmoid and 1 output neuron
optimizer= tf.train.AdamOptimizer(0.001),
metrics=['accuracy'])
return model
model = build_model()
model.fit(data_train, train_labels, epochs=5, batch_size=128)
test_loss, test_acc = model.evaluate(data_test_0, test_labels_0)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_1, test_labels_1)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_2, test_labels_2)
print(test_acc)
test_loss, test_acc = model.evaluate(data_test_3, test_labels_3)
print(test_acc)
`
The training data is flights data of 2016 and 2017 and testing data is of 2018. I have separated classes from testing data to see the class wise accuracy of testing data.
The output is:
Epoch 1/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.3231 - acc: 0.3466
Epoch 2/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2799 - acc: 0.3821
Epoch 3/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2634 - acc: 0.3939
Epoch 4/5
1990363/1990363 [==============================] - 17s 8us/step - loss: 1.2519 - acc: 0.4013
Epoch 5/5
1990363/1990363 [==============================] - 16s 8us/step - loss: 1.2445 - acc: 0.4068
Class 0:
44929/44929 [==============================] - 1s 12us/step
0.027710387500278218
Class 1:
10668/10668 [==============================] - 0s 11us/step
0.015935508061492312
Class 2:
33204/33204 [==============================] - 0s 9us/step
0.8956149861318866
Class 3:
274983/274983 [==============================] - 2s 9us/step
0.035293090845941046
The output remains somewhat same if I use adasyn instead of SMOTE or change layers and activation functions. Please help me out.
Thanks in advance.
deep-learning multiclass-classification mlp smote imbalanced-learn
deep-learning multiclass-classification mlp smote imbalanced-learn
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 30 mins ago
Bhupesh_decoder
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
asked 42 mins ago
Bhupesh_decoderBhupesh_decoder
11
11
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Bhupesh_decoder is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Bhupesh_decoder is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f49454%2fdeep-learningmlp-on-multiclass-classification-model-learns-only-one-class%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Bhupesh_decoder is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
Bhupesh_decoder is a new contributor. Be nice, and check out our Code of Conduct.
Bhupesh_decoder is a new contributor. Be nice, and check out our Code of Conduct.
Bhupesh_decoder is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f49454%2fdeep-learningmlp-on-multiclass-classification-model-learns-only-one-class%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
