Naive Monte Carlo, MCMC and their use in Bayesian TheoryWhat does it mean intuitively to know a pdf “up to a constant”?Simulating Monte Carlo with different standard deviations and interval confidenceMonte carlo optimisation (find maximum of function with multiple parameters)Question about accuracy in Monte Carlo integrationMarkov chain Monte Carlo (MCMC) for Maximum Likelihood Estimation (MLE)Is there a Monte Carlo/MCMC sampler implemented which can deal with isolated local maxima of posterior distribution?Monte Carlo approach in a distribution of a loss processGenerate random numbers for Monte Carlo simulationMonte Carlo simulation of posterior distributionHow to represent results of Monte Carlo SimulationMonte Carlo maximum likelihood vs Bayesian inference

Optimising a list searching algorithm

World War I as a war of liberals against authoritarians?

Is there a hypothetical scenario that would make Earth uninhabitable for humans, but not for (the majority of) other animals?

Violin - Can double stops be played when the strings are not next to each other?

Pronounciation of the combination "st" in spanish accents

How can an organ that provides biological immortality be unable to regenerate?

I seem to dance, I am not a dancer. Who am I?

Print last inputted byte

Generic TVP tradeoffs?

Unfrosted light bulb

Describing a chess game in a novel

Writing in a Christian voice

Help prove this basic trig identity please!

Suggestions on how to spend Shaabath (constructively) alone

HP P840 HDD RAID 5 many strange drive failures

How is the partial sum of a geometric sequence calculated?

Variable completely messes up echoed string

How are passwords stolen from companies if they only store hashes?

Comment Box for Substitution Method of Integrals

Bash - pair each line of file

Maths symbols and unicode-math input inside siunitx commands

두음법칙 - When did North and South diverge in pronunciation of initial ㄹ?

Do I need to consider instance restrictions when showing a language is in P?

Loading the leaflet Map in Lightning Web Component



Naive Monte Carlo, MCMC and their use in Bayesian Theory


What does it mean intuitively to know a pdf “up to a constant”?Simulating Monte Carlo with different standard deviations and interval confidenceMonte carlo optimisation (find maximum of function with multiple parameters)Question about accuracy in Monte Carlo integrationMarkov chain Monte Carlo (MCMC) for Maximum Likelihood Estimation (MLE)Is there a Monte Carlo/MCMC sampler implemented which can deal with isolated local maxima of posterior distribution?Monte Carlo approach in a distribution of a loss processGenerate random numbers for Monte Carlo simulationMonte Carlo simulation of posterior distributionHow to represent results of Monte Carlo SimulationMonte Carlo maximum likelihood vs Bayesian inference













1












$begingroup$


So let's suppose I have a random variable X which follows a PDF fX(x) which is known.
I can use the Naive Monte Carlo method (with unfiltered random sampling) to obtain n samples of fX(x) and get empirical PDF and estimates of its parameters.



Now suppose we have three random variables (X, Y and Z).



First case:
Suppose we know the relationship (function) M between X, Y and Z so that Z=M(X,Y). Suppose we know the marginals of X and Y and the covariance matrix between X and Y.
In this case we can use Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters.



Second case:
2.A
Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y. In this case we can get the empirical PDF of Z and estimates of its parameters directly (although not knowing M). Although I'm not sure how the dependency between X and Y is taken into account...



2.B
Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y (up to a normalizing constant). This in the context of Bayesian Theory is equivalent of knowing the prior and the likelihood and the Bayes rule.
In this case we can no longer use the Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters since we don't know M(X,Y). Here we need to resort to MCMC.



Is the above reasoning right?



Third case:
Suppose we only know the marginals of X and Y, and some observations of Z. What methods can one apply to estimate not only the PDF of Z but also the function M assuming the marginals of X and Y are general and representative.










share|cite|improve this question











$endgroup$
















    1












    $begingroup$


    So let's suppose I have a random variable X which follows a PDF fX(x) which is known.
    I can use the Naive Monte Carlo method (with unfiltered random sampling) to obtain n samples of fX(x) and get empirical PDF and estimates of its parameters.



    Now suppose we have three random variables (X, Y and Z).



    First case:
    Suppose we know the relationship (function) M between X, Y and Z so that Z=M(X,Y). Suppose we know the marginals of X and Y and the covariance matrix between X and Y.
    In this case we can use Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters.



    Second case:
    2.A
    Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y. In this case we can get the empirical PDF of Z and estimates of its parameters directly (although not knowing M). Although I'm not sure how the dependency between X and Y is taken into account...



    2.B
    Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y (up to a normalizing constant). This in the context of Bayesian Theory is equivalent of knowing the prior and the likelihood and the Bayes rule.
    In this case we can no longer use the Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters since we don't know M(X,Y). Here we need to resort to MCMC.



    Is the above reasoning right?



    Third case:
    Suppose we only know the marginals of X and Y, and some observations of Z. What methods can one apply to estimate not only the PDF of Z but also the function M assuming the marginals of X and Y are general and representative.










    share|cite|improve this question











    $endgroup$














      1












      1








      1


      1



      $begingroup$


      So let's suppose I have a random variable X which follows a PDF fX(x) which is known.
      I can use the Naive Monte Carlo method (with unfiltered random sampling) to obtain n samples of fX(x) and get empirical PDF and estimates of its parameters.



      Now suppose we have three random variables (X, Y and Z).



      First case:
      Suppose we know the relationship (function) M between X, Y and Z so that Z=M(X,Y). Suppose we know the marginals of X and Y and the covariance matrix between X and Y.
      In this case we can use Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters.



      Second case:
      2.A
      Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y. In this case we can get the empirical PDF of Z and estimates of its parameters directly (although not knowing M). Although I'm not sure how the dependency between X and Y is taken into account...



      2.B
      Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y (up to a normalizing constant). This in the context of Bayesian Theory is equivalent of knowing the prior and the likelihood and the Bayes rule.
      In this case we can no longer use the Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters since we don't know M(X,Y). Here we need to resort to MCMC.



      Is the above reasoning right?



      Third case:
      Suppose we only know the marginals of X and Y, and some observations of Z. What methods can one apply to estimate not only the PDF of Z but also the function M assuming the marginals of X and Y are general and representative.










      share|cite|improve this question











      $endgroup$




      So let's suppose I have a random variable X which follows a PDF fX(x) which is known.
      I can use the Naive Monte Carlo method (with unfiltered random sampling) to obtain n samples of fX(x) and get empirical PDF and estimates of its parameters.



      Now suppose we have three random variables (X, Y and Z).



      First case:
      Suppose we know the relationship (function) M between X, Y and Z so that Z=M(X,Y). Suppose we know the marginals of X and Y and the covariance matrix between X and Y.
      In this case we can use Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters.



      Second case:
      2.A
      Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y. In this case we can get the empirical PDF of Z and estimates of its parameters directly (although not knowing M). Although I'm not sure how the dependency between X and Y is taken into account...



      2.B
      Suppose we only know the marginals of X and Y and how the PDF of Z is related to the PDFs of X and Y (up to a normalizing constant). This in the context of Bayesian Theory is equivalent of knowing the prior and the likelihood and the Bayes rule.
      In this case we can no longer use the Naive Monte Carlo method to get empirical PDF of Z and estimates of its parameters since we don't know M(X,Y). Here we need to resort to MCMC.



      Is the above reasoning right?



      Third case:
      Suppose we only know the marginals of X and Y, and some observations of Z. What methods can one apply to estimate not only the PDF of Z but also the function M assuming the marginals of X and Y are general and representative.







      bayesian mcmc monte-carlo






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited 13 hours ago







      jpcgandre

















      asked 13 hours ago









      jpcgandrejpcgandre

      1848




      1848




















          1 Answer
          1






          active

          oldest

          votes


















          4












          $begingroup$


          "Suppose we know the marginals of $X$ and $Y$ and the covariance matrix
          between $X$ and $Y$."




          This information is not enough for simulating $(X,Y)$, except in the bivariate Normal setting, and other parameterised cases [like exponential families] when the covariance matrix suffices to define the joint distribution. In general, the distribution of $Z$ is given by
          $$mathbb P_Z(Zin mathcal A)=mathbb P_X,Y(M(X,Y)in A)=mathbb P_X,Y((X,Y)in M^-1(mathcal A))=int_M^-1(mathcal A) p_X,Y(x,y)textd(x,y)$$
          and hence depends on the joint distribution of $(X,Y)$.




          "...we only know the marginals of $X$ and $Y$ and how the PDF of $Z$
          is related to the PDFs of $X$ and $Y$. In this case we can get the
          empirical PDF of $Z$"




          This question is quite unclear or too vague, but in general wrong if $X$, $Y$, and $Z$ are dependent (for the same reason as above). Further, the empirical pdf of $Z$ is unrelated to the true pdfs of $X$ and $Y$, but requires a sample of $Z$'s.




          "This in the context of Bayesian Theory is equivalent of knowing the
          prior and the likelihood and the Bayes rule."




          In Bayesian theory there are two random variables, the parameter $theta$ and the experiment random variable $X$ (called the observation once realised as $x$). The likelihood function is a conditional density of the experiment random variable given the parameter random variable, not a marginal. And Bayes rule gives the conditional density of the (same) parameter random variable $theta$ given the experiment random variable, not a marginal.




          "...we can no longer use the Naive Monte Carlo method to get empirical
          PDF of $Z$"




          This intuition is far from 100% correct as the knowledge of the posterior density up to a constant may be sufficient to run (a) analytical calculations (e.g., with conjugate priors) and (b) regular Monte Carlo simulations. MCMC is not a sure solution for all cases (as in the doubly intractable likelihood problem).




          "Suppose we only know the marginals of $X$ and $Y$, and some observations
          of $Z$. What methods can one apply to estimate not only the PDF of $Z$ but
          also the function $M$ assuming the marginals of $X$ and $Y$ are general and
          representative."




          This question is once again too vague. Observing $Z$ allows for the estimation of its PDF by non-parametric tools, but if $X$ and $Y$ are not observed, it is difficult to imagine estimating $M$ solely from the $Z$'s and the marginal densities.






          share|cite|improve this answer











          $endgroup$








          • 1




            $begingroup$
            Hi! thank you for the time in helping me surf this wave. Regarding your first answer, I'm surprised since in all structural engineering reliability problems I studied and apply this is exactly what is done. You have the mathematical/numerical model that relates input variables with output variables and you estimate the distribution of the latter based on the marginals and covariance matrix, even if the marginals are not normally distributed. If this is not valid in general then I kindly ask you for a reference since this will potentially have a deep impact in civil engineering.
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            Regarding the answer to the Q2.A I agree, that is why I wrote "Although I'm not sure how the dependency between X and Y is taken into account...". Regarding Q2.B, I agree although for the purposes of the question, for me a conditional probability is still a probability function as the marginals are too. So given this I believe the logic I put forward is still applicable. About what you describe about conjugate priors, can you comment on Greenparker answer to stats.stackexchange.com/questions/275641/…
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            And if you have time, please comment/criticize my Q #3. Thank you
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            "It may be a matter of wording: if given only fX, fY, and ΣXY, with no further information, there is no single joint distribution with these characteristics." But (a big but indeed) if you know the function that relates observations of X and Y with outcomes of Z then you can estimate Z based on this function and the probabilistic marginals of X and Y, right?
            $endgroup$
            – jpcgandre
            10 hours ago











          • $begingroup$
            No since you first need to generate $(X,Y)$ pairwise before deducing $Z$.
            $endgroup$
            – Xi'an
            10 hours ago











          Your Answer





          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "65"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f398003%2fnaive-monte-carlo-mcmc-and-their-use-in-bayesian-theory%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          4












          $begingroup$


          "Suppose we know the marginals of $X$ and $Y$ and the covariance matrix
          between $X$ and $Y$."




          This information is not enough for simulating $(X,Y)$, except in the bivariate Normal setting, and other parameterised cases [like exponential families] when the covariance matrix suffices to define the joint distribution. In general, the distribution of $Z$ is given by
          $$mathbb P_Z(Zin mathcal A)=mathbb P_X,Y(M(X,Y)in A)=mathbb P_X,Y((X,Y)in M^-1(mathcal A))=int_M^-1(mathcal A) p_X,Y(x,y)textd(x,y)$$
          and hence depends on the joint distribution of $(X,Y)$.




          "...we only know the marginals of $X$ and $Y$ and how the PDF of $Z$
          is related to the PDFs of $X$ and $Y$. In this case we can get the
          empirical PDF of $Z$"




          This question is quite unclear or too vague, but in general wrong if $X$, $Y$, and $Z$ are dependent (for the same reason as above). Further, the empirical pdf of $Z$ is unrelated to the true pdfs of $X$ and $Y$, but requires a sample of $Z$'s.




          "This in the context of Bayesian Theory is equivalent of knowing the
          prior and the likelihood and the Bayes rule."




          In Bayesian theory there are two random variables, the parameter $theta$ and the experiment random variable $X$ (called the observation once realised as $x$). The likelihood function is a conditional density of the experiment random variable given the parameter random variable, not a marginal. And Bayes rule gives the conditional density of the (same) parameter random variable $theta$ given the experiment random variable, not a marginal.




          "...we can no longer use the Naive Monte Carlo method to get empirical
          PDF of $Z$"




          This intuition is far from 100% correct as the knowledge of the posterior density up to a constant may be sufficient to run (a) analytical calculations (e.g., with conjugate priors) and (b) regular Monte Carlo simulations. MCMC is not a sure solution for all cases (as in the doubly intractable likelihood problem).




          "Suppose we only know the marginals of $X$ and $Y$, and some observations
          of $Z$. What methods can one apply to estimate not only the PDF of $Z$ but
          also the function $M$ assuming the marginals of $X$ and $Y$ are general and
          representative."




          This question is once again too vague. Observing $Z$ allows for the estimation of its PDF by non-parametric tools, but if $X$ and $Y$ are not observed, it is difficult to imagine estimating $M$ solely from the $Z$'s and the marginal densities.






          share|cite|improve this answer











          $endgroup$








          • 1




            $begingroup$
            Hi! thank you for the time in helping me surf this wave. Regarding your first answer, I'm surprised since in all structural engineering reliability problems I studied and apply this is exactly what is done. You have the mathematical/numerical model that relates input variables with output variables and you estimate the distribution of the latter based on the marginals and covariance matrix, even if the marginals are not normally distributed. If this is not valid in general then I kindly ask you for a reference since this will potentially have a deep impact in civil engineering.
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            Regarding the answer to the Q2.A I agree, that is why I wrote "Although I'm not sure how the dependency between X and Y is taken into account...". Regarding Q2.B, I agree although for the purposes of the question, for me a conditional probability is still a probability function as the marginals are too. So given this I believe the logic I put forward is still applicable. About what you describe about conjugate priors, can you comment on Greenparker answer to stats.stackexchange.com/questions/275641/…
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            And if you have time, please comment/criticize my Q #3. Thank you
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            "It may be a matter of wording: if given only fX, fY, and ΣXY, with no further information, there is no single joint distribution with these characteristics." But (a big but indeed) if you know the function that relates observations of X and Y with outcomes of Z then you can estimate Z based on this function and the probabilistic marginals of X and Y, right?
            $endgroup$
            – jpcgandre
            10 hours ago











          • $begingroup$
            No since you first need to generate $(X,Y)$ pairwise before deducing $Z$.
            $endgroup$
            – Xi'an
            10 hours ago
















          4












          $begingroup$


          "Suppose we know the marginals of $X$ and $Y$ and the covariance matrix
          between $X$ and $Y$."




          This information is not enough for simulating $(X,Y)$, except in the bivariate Normal setting, and other parameterised cases [like exponential families] when the covariance matrix suffices to define the joint distribution. In general, the distribution of $Z$ is given by
          $$mathbb P_Z(Zin mathcal A)=mathbb P_X,Y(M(X,Y)in A)=mathbb P_X,Y((X,Y)in M^-1(mathcal A))=int_M^-1(mathcal A) p_X,Y(x,y)textd(x,y)$$
          and hence depends on the joint distribution of $(X,Y)$.




          "...we only know the marginals of $X$ and $Y$ and how the PDF of $Z$
          is related to the PDFs of $X$ and $Y$. In this case we can get the
          empirical PDF of $Z$"




          This question is quite unclear or too vague, but in general wrong if $X$, $Y$, and $Z$ are dependent (for the same reason as above). Further, the empirical pdf of $Z$ is unrelated to the true pdfs of $X$ and $Y$, but requires a sample of $Z$'s.




          "This in the context of Bayesian Theory is equivalent of knowing the
          prior and the likelihood and the Bayes rule."




          In Bayesian theory there are two random variables, the parameter $theta$ and the experiment random variable $X$ (called the observation once realised as $x$). The likelihood function is a conditional density of the experiment random variable given the parameter random variable, not a marginal. And Bayes rule gives the conditional density of the (same) parameter random variable $theta$ given the experiment random variable, not a marginal.




          "...we can no longer use the Naive Monte Carlo method to get empirical
          PDF of $Z$"




          This intuition is far from 100% correct as the knowledge of the posterior density up to a constant may be sufficient to run (a) analytical calculations (e.g., with conjugate priors) and (b) regular Monte Carlo simulations. MCMC is not a sure solution for all cases (as in the doubly intractable likelihood problem).




          "Suppose we only know the marginals of $X$ and $Y$, and some observations
          of $Z$. What methods can one apply to estimate not only the PDF of $Z$ but
          also the function $M$ assuming the marginals of $X$ and $Y$ are general and
          representative."




          This question is once again too vague. Observing $Z$ allows for the estimation of its PDF by non-parametric tools, but if $X$ and $Y$ are not observed, it is difficult to imagine estimating $M$ solely from the $Z$'s and the marginal densities.






          share|cite|improve this answer











          $endgroup$








          • 1




            $begingroup$
            Hi! thank you for the time in helping me surf this wave. Regarding your first answer, I'm surprised since in all structural engineering reliability problems I studied and apply this is exactly what is done. You have the mathematical/numerical model that relates input variables with output variables and you estimate the distribution of the latter based on the marginals and covariance matrix, even if the marginals are not normally distributed. If this is not valid in general then I kindly ask you for a reference since this will potentially have a deep impact in civil engineering.
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            Regarding the answer to the Q2.A I agree, that is why I wrote "Although I'm not sure how the dependency between X and Y is taken into account...". Regarding Q2.B, I agree although for the purposes of the question, for me a conditional probability is still a probability function as the marginals are too. So given this I believe the logic I put forward is still applicable. About what you describe about conjugate priors, can you comment on Greenparker answer to stats.stackexchange.com/questions/275641/…
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            And if you have time, please comment/criticize my Q #3. Thank you
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            "It may be a matter of wording: if given only fX, fY, and ΣXY, with no further information, there is no single joint distribution with these characteristics." But (a big but indeed) if you know the function that relates observations of X and Y with outcomes of Z then you can estimate Z based on this function and the probabilistic marginals of X and Y, right?
            $endgroup$
            – jpcgandre
            10 hours ago











          • $begingroup$
            No since you first need to generate $(X,Y)$ pairwise before deducing $Z$.
            $endgroup$
            – Xi'an
            10 hours ago














          4












          4








          4





          $begingroup$


          "Suppose we know the marginals of $X$ and $Y$ and the covariance matrix
          between $X$ and $Y$."




          This information is not enough for simulating $(X,Y)$, except in the bivariate Normal setting, and other parameterised cases [like exponential families] when the covariance matrix suffices to define the joint distribution. In general, the distribution of $Z$ is given by
          $$mathbb P_Z(Zin mathcal A)=mathbb P_X,Y(M(X,Y)in A)=mathbb P_X,Y((X,Y)in M^-1(mathcal A))=int_M^-1(mathcal A) p_X,Y(x,y)textd(x,y)$$
          and hence depends on the joint distribution of $(X,Y)$.




          "...we only know the marginals of $X$ and $Y$ and how the PDF of $Z$
          is related to the PDFs of $X$ and $Y$. In this case we can get the
          empirical PDF of $Z$"




          This question is quite unclear or too vague, but in general wrong if $X$, $Y$, and $Z$ are dependent (for the same reason as above). Further, the empirical pdf of $Z$ is unrelated to the true pdfs of $X$ and $Y$, but requires a sample of $Z$'s.




          "This in the context of Bayesian Theory is equivalent of knowing the
          prior and the likelihood and the Bayes rule."




          In Bayesian theory there are two random variables, the parameter $theta$ and the experiment random variable $X$ (called the observation once realised as $x$). The likelihood function is a conditional density of the experiment random variable given the parameter random variable, not a marginal. And Bayes rule gives the conditional density of the (same) parameter random variable $theta$ given the experiment random variable, not a marginal.




          "...we can no longer use the Naive Monte Carlo method to get empirical
          PDF of $Z$"




          This intuition is far from 100% correct as the knowledge of the posterior density up to a constant may be sufficient to run (a) analytical calculations (e.g., with conjugate priors) and (b) regular Monte Carlo simulations. MCMC is not a sure solution for all cases (as in the doubly intractable likelihood problem).




          "Suppose we only know the marginals of $X$ and $Y$, and some observations
          of $Z$. What methods can one apply to estimate not only the PDF of $Z$ but
          also the function $M$ assuming the marginals of $X$ and $Y$ are general and
          representative."




          This question is once again too vague. Observing $Z$ allows for the estimation of its PDF by non-parametric tools, but if $X$ and $Y$ are not observed, it is difficult to imagine estimating $M$ solely from the $Z$'s and the marginal densities.






          share|cite|improve this answer











          $endgroup$




          "Suppose we know the marginals of $X$ and $Y$ and the covariance matrix
          between $X$ and $Y$."




          This information is not enough for simulating $(X,Y)$, except in the bivariate Normal setting, and other parameterised cases [like exponential families] when the covariance matrix suffices to define the joint distribution. In general, the distribution of $Z$ is given by
          $$mathbb P_Z(Zin mathcal A)=mathbb P_X,Y(M(X,Y)in A)=mathbb P_X,Y((X,Y)in M^-1(mathcal A))=int_M^-1(mathcal A) p_X,Y(x,y)textd(x,y)$$
          and hence depends on the joint distribution of $(X,Y)$.




          "...we only know the marginals of $X$ and $Y$ and how the PDF of $Z$
          is related to the PDFs of $X$ and $Y$. In this case we can get the
          empirical PDF of $Z$"




          This question is quite unclear or too vague, but in general wrong if $X$, $Y$, and $Z$ are dependent (for the same reason as above). Further, the empirical pdf of $Z$ is unrelated to the true pdfs of $X$ and $Y$, but requires a sample of $Z$'s.




          "This in the context of Bayesian Theory is equivalent of knowing the
          prior and the likelihood and the Bayes rule."




          In Bayesian theory there are two random variables, the parameter $theta$ and the experiment random variable $X$ (called the observation once realised as $x$). The likelihood function is a conditional density of the experiment random variable given the parameter random variable, not a marginal. And Bayes rule gives the conditional density of the (same) parameter random variable $theta$ given the experiment random variable, not a marginal.




          "...we can no longer use the Naive Monte Carlo method to get empirical
          PDF of $Z$"




          This intuition is far from 100% correct as the knowledge of the posterior density up to a constant may be sufficient to run (a) analytical calculations (e.g., with conjugate priors) and (b) regular Monte Carlo simulations. MCMC is not a sure solution for all cases (as in the doubly intractable likelihood problem).




          "Suppose we only know the marginals of $X$ and $Y$, and some observations
          of $Z$. What methods can one apply to estimate not only the PDF of $Z$ but
          also the function $M$ assuming the marginals of $X$ and $Y$ are general and
          representative."




          This question is once again too vague. Observing $Z$ allows for the estimation of its PDF by non-parametric tools, but if $X$ and $Y$ are not observed, it is difficult to imagine estimating $M$ solely from the $Z$'s and the marginal densities.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited 8 hours ago

























          answered 12 hours ago









          Xi'anXi'an

          58.5k897362




          58.5k897362







          • 1




            $begingroup$
            Hi! thank you for the time in helping me surf this wave. Regarding your first answer, I'm surprised since in all structural engineering reliability problems I studied and apply this is exactly what is done. You have the mathematical/numerical model that relates input variables with output variables and you estimate the distribution of the latter based on the marginals and covariance matrix, even if the marginals are not normally distributed. If this is not valid in general then I kindly ask you for a reference since this will potentially have a deep impact in civil engineering.
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            Regarding the answer to the Q2.A I agree, that is why I wrote "Although I'm not sure how the dependency between X and Y is taken into account...". Regarding Q2.B, I agree although for the purposes of the question, for me a conditional probability is still a probability function as the marginals are too. So given this I believe the logic I put forward is still applicable. About what you describe about conjugate priors, can you comment on Greenparker answer to stats.stackexchange.com/questions/275641/…
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            And if you have time, please comment/criticize my Q #3. Thank you
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            "It may be a matter of wording: if given only fX, fY, and ΣXY, with no further information, there is no single joint distribution with these characteristics." But (a big but indeed) if you know the function that relates observations of X and Y with outcomes of Z then you can estimate Z based on this function and the probabilistic marginals of X and Y, right?
            $endgroup$
            – jpcgandre
            10 hours ago











          • $begingroup$
            No since you first need to generate $(X,Y)$ pairwise before deducing $Z$.
            $endgroup$
            – Xi'an
            10 hours ago













          • 1




            $begingroup$
            Hi! thank you for the time in helping me surf this wave. Regarding your first answer, I'm surprised since in all structural engineering reliability problems I studied and apply this is exactly what is done. You have the mathematical/numerical model that relates input variables with output variables and you estimate the distribution of the latter based on the marginals and covariance matrix, even if the marginals are not normally distributed. If this is not valid in general then I kindly ask you for a reference since this will potentially have a deep impact in civil engineering.
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            Regarding the answer to the Q2.A I agree, that is why I wrote "Although I'm not sure how the dependency between X and Y is taken into account...". Regarding Q2.B, I agree although for the purposes of the question, for me a conditional probability is still a probability function as the marginals are too. So given this I believe the logic I put forward is still applicable. About what you describe about conjugate priors, can you comment on Greenparker answer to stats.stackexchange.com/questions/275641/…
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            And if you have time, please comment/criticize my Q #3. Thank you
            $endgroup$
            – jpcgandre
            11 hours ago










          • $begingroup$
            "It may be a matter of wording: if given only fX, fY, and ΣXY, with no further information, there is no single joint distribution with these characteristics." But (a big but indeed) if you know the function that relates observations of X and Y with outcomes of Z then you can estimate Z based on this function and the probabilistic marginals of X and Y, right?
            $endgroup$
            – jpcgandre
            10 hours ago











          • $begingroup$
            No since you first need to generate $(X,Y)$ pairwise before deducing $Z$.
            $endgroup$
            – Xi'an
            10 hours ago








          1




          1




          $begingroup$
          Hi! thank you for the time in helping me surf this wave. Regarding your first answer, I'm surprised since in all structural engineering reliability problems I studied and apply this is exactly what is done. You have the mathematical/numerical model that relates input variables with output variables and you estimate the distribution of the latter based on the marginals and covariance matrix, even if the marginals are not normally distributed. If this is not valid in general then I kindly ask you for a reference since this will potentially have a deep impact in civil engineering.
          $endgroup$
          – jpcgandre
          11 hours ago




          $begingroup$
          Hi! thank you for the time in helping me surf this wave. Regarding your first answer, I'm surprised since in all structural engineering reliability problems I studied and apply this is exactly what is done. You have the mathematical/numerical model that relates input variables with output variables and you estimate the distribution of the latter based on the marginals and covariance matrix, even if the marginals are not normally distributed. If this is not valid in general then I kindly ask you for a reference since this will potentially have a deep impact in civil engineering.
          $endgroup$
          – jpcgandre
          11 hours ago












          $begingroup$
          Regarding the answer to the Q2.A I agree, that is why I wrote "Although I'm not sure how the dependency between X and Y is taken into account...". Regarding Q2.B, I agree although for the purposes of the question, for me a conditional probability is still a probability function as the marginals are too. So given this I believe the logic I put forward is still applicable. About what you describe about conjugate priors, can you comment on Greenparker answer to stats.stackexchange.com/questions/275641/…
          $endgroup$
          – jpcgandre
          11 hours ago




          $begingroup$
          Regarding the answer to the Q2.A I agree, that is why I wrote "Although I'm not sure how the dependency between X and Y is taken into account...". Regarding Q2.B, I agree although for the purposes of the question, for me a conditional probability is still a probability function as the marginals are too. So given this I believe the logic I put forward is still applicable. About what you describe about conjugate priors, can you comment on Greenparker answer to stats.stackexchange.com/questions/275641/…
          $endgroup$
          – jpcgandre
          11 hours ago












          $begingroup$
          And if you have time, please comment/criticize my Q #3. Thank you
          $endgroup$
          – jpcgandre
          11 hours ago




          $begingroup$
          And if you have time, please comment/criticize my Q #3. Thank you
          $endgroup$
          – jpcgandre
          11 hours ago












          $begingroup$
          "It may be a matter of wording: if given only fX, fY, and ΣXY, with no further information, there is no single joint distribution with these characteristics." But (a big but indeed) if you know the function that relates observations of X and Y with outcomes of Z then you can estimate Z based on this function and the probabilistic marginals of X and Y, right?
          $endgroup$
          – jpcgandre
          10 hours ago





          $begingroup$
          "It may be a matter of wording: if given only fX, fY, and ΣXY, with no further information, there is no single joint distribution with these characteristics." But (a big but indeed) if you know the function that relates observations of X and Y with outcomes of Z then you can estimate Z based on this function and the probabilistic marginals of X and Y, right?
          $endgroup$
          – jpcgandre
          10 hours ago













          $begingroup$
          No since you first need to generate $(X,Y)$ pairwise before deducing $Z$.
          $endgroup$
          – Xi'an
          10 hours ago





          $begingroup$
          No since you first need to generate $(X,Y)$ pairwise before deducing $Z$.
          $endgroup$
          – Xi'an
          10 hours ago


















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Cross Validated!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f398003%2fnaive-monte-carlo-mcmc-and-their-use-in-bayesian-theory%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Францішак Багушэвіч Змест Сям'я | Біяграфія | Творчасць | Мова Багушэвіча | Ацэнкі дзейнасці | Цікавыя факты | Спадчына | Выбраная бібліяграфія | Ушанаванне памяці | У філатэліі | Зноскі | Літаратура | Спасылкі | НавігацыяЛяхоўскі У. Рупіўся дзеля Бога і людзей: Жыццёвы шлях Лявона Вітан-Дубейкаўскага // Вольскі і Памідораў з песняй пра немца Адвакат, паэт, народны заступнік Ашмянскі веснікВ Минске появится площадь Богушевича и улица Сырокомли, Белорусская деловая газета, 19 июля 2001 г.Айцец беларускай нацыянальнай ідэі паўстаў у бронзе Сяргей Аляксандравіч Адашкевіч (1918, Мінск). 80-я гады. Бюст «Францішак Багушэвіч».Яўген Мікалаевіч Ціхановіч. «Партрэт Францішка Багушэвіча»Мікола Мікалаевіч Купава. «Партрэт зачынальніка новай беларускай літаратуры Францішка Багушэвіча»Уладзімір Іванавіч Мелехаў. На помніку «Змагарам за родную мову» Барэльеф «Францішак Багушэвіч»Памяць пра Багушэвіча на Віленшчыне Страчаная сталіца. Беларускія шыльды на вуліцах Вільні«Krynica». Ideologia i przywódcy białoruskiego katolicyzmuФранцішак БагушэвічТворы на knihi.comТворы Францішка Багушэвіча на bellib.byСодаль Уладзімір. Францішак Багушэвіч на Лідчыне;Луцкевіч Антон. Жыцьцё і творчасьць Фр. Багушэвіча ў успамінах ягоных сучасьнікаў // Запісы Беларускага Навуковага таварыства. Вільня, 1938. Сшытак 1. С. 16-34.Большая российская1188761710000 0000 5537 633Xn9209310021619551927869394п

          Беларусь Змест Назва Гісторыя Геаграфія Сімволіка Дзяржаўны лад Палітычныя партыі Міжнароднае становішча і знешняя палітыка Адміністрацыйны падзел Насельніцтва Эканоміка Культура і грамадства Сацыяльная сфера Узброеныя сілы Заўвагі Літаратура Спасылкі НавігацыяHGЯOiТоп-2011 г. (па версіі ej.by)Топ-2013 г. (па версіі ej.by)Топ-2016 г. (па версіі ej.by)Топ-2017 г. (па версіі ej.by)Нацыянальны статыстычны камітэт Рэспублікі БеларусьШчыльнасць насельніцтва па краінахhttp://naviny.by/rubrics/society/2011/09/16/ic_articles_116_175144/А. Калечыц, У. Ксяндзоў. Спробы засялення краю неандэртальскім чалавекам.І ў Менску былі мамантыА. Калечыц, У. Ксяндзоў. Старажытны каменны век (палеаліт). Першапачатковае засяленне тэрыторыіГ. Штыхаў. Балты і славяне ў VI—VIII стст.М. Клімаў. Полацкае княства ў IX—XI стст.Г. Штыхаў, В. Ляўко. Палітычная гісторыя Полацкай зямліГ. Штыхаў. Дзяржаўны лад у землях-княствахГ. Штыхаў. Дзяржаўны лад у землях-княствахБеларускія землі ў складзе Вялікага Княства ЛітоўскагаЛюблінская унія 1569 г."The Early Stages of Independence"Zapomniane prawdy25 гадоў таму было аб'яўлена, што Язэп Пілсудскі — беларус (фота)Наша вадаДакументы ЧАЭС: Забруджванне тэрыторыі Беларусі « ЧАЭС Зона адчужэнняСведения о политических партиях, зарегистрированных в Республике Беларусь // Министерство юстиции Республики БеларусьСтатыстычны бюлетэнь „Полаўзроставая структура насельніцтва Рэспублікі Беларусь на 1 студзеня 2012 года і сярэднегадовая колькасць насельніцтва за 2011 год“Индекс человеческого развития Беларуси — не было бы нижеБеларусь занимает первое место в СНГ по индексу развития с учетом гендерного факцёраНацыянальны статыстычны камітэт Рэспублікі БеларусьКанстытуцыя РБ. Артыкул 17Трансфармацыйныя задачы БеларусіВыйсце з крызісу — далейшае рэфармаванне Беларускі рубель — сусветны лідар па дэвальвацыяхПра змену коштаў у кастрычніку 2011 г.Бядней за беларусаў у СНД толькі таджыкіСярэдні заробак у верасні дасягнуў 2,26 мільёна рублёўЭканомікаГаласуем за ТОП-100 беларускай прозыСучасныя беларускія мастакіАрхитектура Беларуси BELARUS.BYА. Каханоўскі. Культура Беларусі ўсярэдзіне XVII—XVIII ст.Анталогія беларускай народнай песні, гуказапісы спеваўБеларускія Музычныя IнструментыБеларускі рок, які мы страцілі. Топ-10 гуртоў«Мясцовы час» — нязгаслая легенда беларускай рок-музыкіСЯРГЕЙ БУДКІН. МЫ НЯ ЗНАЕМ СВАЁЙ МУЗЫКІМ. А. Каладзінскі. НАРОДНЫ ТЭАТРМагнацкія культурныя цэнтрыПублічная дыскусія «Беларуская новая пьеса: без беларускай мовы ці беларуская?»Беларускія драматургі па-ранейшаму лепш ставяцца за мяжой, чым на радзіме«Працэс незалежнага кіно пайшоў, і дзяржаву турбуе яго непадкантрольнасць»Беларускія філосафы ў пошуках прасторыВсе идём в библиотекуАрхіваванаАб Нацыянальнай праграме даследавання і выкарыстання касмічнай прасторы ў мірных мэтах на 2008—2012 гадыУ космас — разам.У суседнім з Барысаўскім раёне пабудуюць Камандна-вымяральны пунктСвяты і абрады беларусаў«Мірныя бульбашы з малой краіны» — 5 непраўдзівых стэрэатыпаў пра БеларусьМ. Раманюк. Беларускае народнае адзеннеУ Беларусі скарачаецца колькасць злачынстваўЛукашэнка незадаволены мінскімі ўладамі Крадзяжы складаюць у Мінску каля 70% злачынстваў Узровень злачыннасці ў Мінскай вобласці — адзін з самых высокіх у краіне Генпракуратура аналізуе стан са злачыннасцю ў Беларусі па каэфіцыенце злачыннасці У Беларусі стабілізавалася крымінагеннае становішча, лічыць генпракурорЗамежнікі сталі здзяйсняць у Беларусі больш злачынстваўМУС Беларусі турбуе рост рэцыдыўнай злачыннасціЯ з ЖЭСа. Дазволіце вас абкрасці! Рэйтынг усіх службаў і падраздзяленняў ГУУС Мінгарвыканкама вырасАб КДБ РБГісторыя Аператыўна-аналітычнага цэнтра РБГісторыя ДКФРТаможняagentura.ruБеларусьBelarus.by — Афіцыйны сайт Рэспублікі БеларусьСайт урада БеларусіRadzima.org — Збор архітэктурных помнікаў, гісторыя Беларусі«Глобус Беларуси»Гербы и флаги БеларусиАсаблівасці каменнага веку на БеларусіА. Калечыц, У. Ксяндзоў. Старажытны каменны век (палеаліт). Першапачатковае засяленне тэрыторыіУ. Ксяндзоў. Сярэдні каменны век (мезаліт). Засяленне краю плямёнамі паляўнічых, рыбакоў і збіральнікаўА. Калечыц, М. Чарняўскі. Плямёны на тэрыторыі Беларусі ў новым каменным веку (неаліце)А. Калечыц, У. Ксяндзоў, М. Чарняўскі. Гаспадарчыя заняткі ў каменным векуЭ. Зайкоўскі. Духоўная культура ў каменным векуАсаблівасці бронзавага веку на БеларусіФарміраванне супольнасцей ранняга перыяду бронзавага векуФотографии БеларусиРоля беларускіх зямель ва ўтварэнні і ўмацаванні ВКЛВ. Фадзеева. З гісторыі развіцця беларускай народнай вышыўкіDMOZGran catalanaБольшая российскаяBritannica (анлайн)Швейцарскі гістарычны15325917611952699xDA123282154079143-90000 0001 2171 2080n9112870100577502ge128882171858027501086026362074122714179пппппп

          Герб Смалявічаў Апісанне | Спасылкі | НавігацыяГерб города Смолевичип