Hfrxdjt

Confused by notation of atomic number Z and mass number A on periodic table of elements

Minimum value of 4 digit number divided by sum of its digits

Do I have an "anti-research" personality?

Why does nature favour the Laplacian?

Nginx subdirectory wordpress wp-login redirects to 404 not found

How to figure out whether the data is sample data or population data apart from the client's information?

Any examples of headwear for races with animal ears?

Examples of non trivial equivalence relations , I mean equivalence relations without the expression " same ... as" in their definition?

What word means to make something obsolete?

Counterexample: a pair of linearly ordered sets that are isomorphic to subsets of the other, but not isomorphic between them

Python "triplet" dictionary?

Pulling the rope with one hand is as heavy as with two hands?

Why do Ichisongas hate elephants and hippos?

What does YCWCYODFTRFDTY mean?

Are Boeing 737-800’s grounded?

Packing rectangles: Does rotation ever help?

Were there two appearances of Stan Lee?

Lock in SQL Server and Oracle

Transfer over $10k

Can not tell colimits from limits

Alternatives to Overleaf

get exit status from system() call

How can Republicans who favour free markets, consistently express anger when they don't like the outcome of that choice?

What's the polite way to say "I need to urinate"?

Training a model where each response in the observation data has a different known varience

How to start analysing and modelling data for an academic project, when not a statistician or data scientistOne-class classification in Keras using Autoencoders?Overfitting XGBoostPrevent overffitting in model stacking with training on the same targetInterpreting Results of Multivariable Regression / how to transform variables to improve resultsHow exactly does class_weight in Keras work?How to attribute variance to an input parameter?Is “adding the predictions to the real data for new training and prediction” a good idea for LSTM?CNN binary classifier: validation accuracy fluctuationImbalanced classes (balance of train, validation, and test)

0

$begingroup$

I have a dataset where each response variable is the number of successes of N Bernoulli trials with N and p (the probability of success) being different for each observation. The goal is to train a model to predict p given the predictors. However observations with a small n will have a higher variance and higher N.

Consider the following scenario to illustrate better: Assume coins with different pictures on them have a different bias and that the bias is dependent on the picture on the coin. I have a large number of coins each with a different picture on them and each with a different bias p. I want to create a model that can predict the bias of a coin given only the picture on the coin. I flip each coin a different number of times and record the number of successes and total number of flips. So my data set consists of each picture and its estimate p=successes/flips.

So my question is when training my model how should I handle this. It seems more weight should be given to observations with a higher sample size(number of flips). I don't think it makes sense to include number flips as a predictor variable because the point is to build a model which predicts p using only the picture on the coin so this difference in variance for the response for each observation should be taken into account when training the model.

I am using several types of model but mainly working with keras and xgboost

keras linear-regression xgboost training weighted-data

share|improve this question

asked 14 mins ago

dlndln

1

New contributor

dln is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

0

$begingroup$

I have a dataset where each response variable is the number of successes of N Bernoulli trials with N and p (the probability of success) being different for each observation. The goal is to train a model to predict p given the predictors. However observations with a small n will have a higher variance and higher N.

Consider the following scenario to illustrate better: Assume coins with different pictures on them have a different bias and that the bias is dependent on the picture on the coin. I have a large number of coins each with a different picture on them and each with a different bias p. I want to create a model that can predict the bias of a coin given only the picture on the coin. I flip each coin a different number of times and record the number of successes and total number of flips. So my data set consists of each picture and its estimate p=successes/flips.

So my question is when training my model how should I handle this. It seems more weight should be given to observations with a higher sample size(number of flips). I don't think it makes sense to include number flips as a predictor variable because the point is to build a model which predicts p using only the picture on the coin so this difference in variance for the response for each observation should be taken into account when training the model.

I am using several types of model but mainly working with keras and xgboost

keras linear-regression xgboost training weighted-data

share|improve this question

asked 14 mins ago

dlndln

1

New contributor

dln is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I have a dataset where each response variable is the number of successes of N Bernoulli trials with N and p (the probability of success) being different for each observation. The goal is to train a model to predict p given the predictors. However observations with a small n will have a higher variance and higher N.

Consider the following scenario to illustrate better: Assume coins with different pictures on them have a different bias and that the bias is dependent on the picture on the coin. I have a large number of coins each with a different picture on them and each with a different bias p. I want to create a model that can predict the bias of a coin given only the picture on the coin. I flip each coin a different number of times and record the number of successes and total number of flips. So my data set consists of each picture and its estimate p=successes/flips.

So my question is when training my model how should I handle this. It seems more weight should be given to observations with a higher sample size(number of flips). I don't think it makes sense to include number flips as a predictor variable because the point is to build a model which predicts p using only the picture on the coin so this difference in variance for the response for each observation should be taken into account when training the model.

I am using several types of model but mainly working with keras and xgboost

keras linear-regression xgboost training weighted-data

share|improve this question

asked 14 mins ago

dlndln

1

New contributor

dln is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|improve this question

asked 14 mins ago

dlndln

1

New contributor

dln is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this question

asked 14 mins ago

dlndln

1

New contributor

dln is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 14 mins ago

dlndln

1

New contributor

dln is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 14 mins ago

dlndln

1