Hfrxdjt

I am doing my undergrad Dissertation on time series prediction, and use various models (linear /ridge regression, AR(2), Random Forest, SVR, and 4 variations of Neural Networks) to try and 'predict' (for academic only reasons) daily return data, using as input lagged returns and SMA - RSI features (using TA - Lib) built based on those returns. However, I have noticed that my NNs do not learn anything, and upon inspecting the loss graph and the vector of predictions, I noticed it only predicts a single value, with the same applying for the Ridge and AR regressions.

Also, when I try to calculate the correlation between the labels and the predictions (of the NNs) I get 'nan' as a result, no matter what I try, which I suspect has to do with the predictions. I also get wildly varying r2 scores on each re-run (even though I have set multiple seeds, both on Tensorflow backend as well as numpy) and always negative, which I cannot understand as even though my search on the internet and the sklearn's docs say it can be negative, my professor insists it cannot be, and I truly am bewildered.

What can I do about it? Isn't it obviously wrong for an entire NN to predict only a single value? Below I include the code for the ridge / AR regressions as well as the 'Vanilla NN' and a couple of useful graphs. The data itself is quite large, so I don't know if there's much of a point to include it if not asked specifically, given there are no algorithmic errors below.

def vanillaNN(X_train, y_train,X_test,y_test):
 n_cols = X_train.shape[1]
 model = Sequential()
 model.add(Dense(100,activation='relu', input_shape=(n_cols, )))
 model.add(Dropout(0.3))
 model.add(Dense(150, activation='relu'))
 model.add(Dense(50, activation='relu'))
 model.add(Dropout(0.1))
 model.add(Dense(1))

 model.compile(optimizer='adam', loss='mse', metrics=['mse'])

 history = model.fit(X_train,y_train,epochs=100,verbose=0,
 shuffle=False, validation_split=0.1) 

 # Use the last loss as the title
 plt.plot(history.history['loss'])
 plt.title('last loss:' + str(round(history.history['loss'][-1], 6)))
 plt.xlabel('Epoch')
 plt.ylabel('Loss')
 plt.show()

 # Calculate R^2 score and MSE
 .... Omitted Code ......

 # it returns those for testing purposes in the IPython shell
 return (train_scores, test_scores, y_pred_train, y_pred_test, y_train, y_test)

vNN_results=vanillaNN(train_features,train_targets,test_features,test_targets)

def AR(X_train, order=2):
 arma_train = np.array(X_train['returns'])
 armodel = ARMA(arma_train, order=(order,0))
 armodel_results = armodel.fit()
 print(armodel_results.summary())
 armodel_results.plot_predict(start=8670, end=8698)
 plt.show()
 ar_pred = armodel_results.predict(start=8699, end=9665)

 ...r2 and MSE scores omitted code...

 return [mse_ar2, r2_ar2, ar_pred]

def Elastic(X_train, y_train, X_test, y_test):
 elastic = ElasticNet()
 param_grid_elastic = 'alpha': [0.001, 0.01, 0.1, 0.5],
 'l1_ratio': [0.001, 0.01, 0.1, 0.5]
 
 grid_elastic = GridSearchCV(elastic, param_grid_elastic, 
 cv=tscv.split(X_train),scoring='neg_mean_squared_error')

 grid_elastic.fit(X_train, y_train)

 y_pred_train = grid_elastic.predict(X_train) 
 train_scores = scores(y_train, y_pred_train)

 y_pred_test = grid_elastic.predict(X_test)

 ...Omitted Code...
 return [train_scores, test_scores, y_pred_train, y_pred_test]

Please let me know if you need anything else, as this is my first post here and I am open to criticism!