Is there an algorithm that imputes missing values based on n nearest columns? (KNN hybrid) The 2019 Stack Overflow Developer Survey Results Are In Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsWhere in the workflow should we deal with missing data?Assigning values to missing target vector values in scikit-learnImputation of missing values and dealing with categorical valuesIs there a way to standardize latitude and longitude to be used as predictors in KNN algorithm?Is there are way to impute missing values by clustering, regression and stochastic regressionIs there an upper bound for k in nearest neighbors-based methods?How does KNN work if there are duplicates?How to select variables based on the mean correlation in a correlation matrix?Multiple filtering pandas columns based on values in another columnHow to create a df that gets sum of columns based on a groupby column?

Why can I use a list index as an indexing variable in a for loop?

Circular reasoning in L'Hopital's rule

ELI5: Why do they say that Israel would have been the fourth country to land a spacecraft on the Moon and why do they call it low cost?

Is it ok to offer lower paid work as a trial period before negotiating for a full-time job?

Homework question about an engine pulling a train

How do you keep chess fun when your opponent constantly beats you?

Could an empire control the whole planet with today's comunication methods?

Make it rain characters

Why did Peik Lin say, "I'm not an animal"?

Can withdrawing asylum be illegal?

Word for: a synonym with a positive connotation?

What does Linus Torvalds mean when he says that Git "never ever" tracks a file?

Does Parliament need to approve the new Brexit delay to 31 October 2019?

Student Loan from years ago pops up and is taking my salary

Didn't get enough time to take a Coding Test - what to do now?

Nested ellipses in tikzpicture: Chomsky hierarchy

Do working physicists consider Newtonian mechanics to be "falsified"?

Intergalactic human space ship encounters another ship, character gets shunted off beyond known universe, reality starts collapsing

What can I do if neighbor is blocking my solar panels intentionally?

Are spiders unable to hurt humans, especially very small spiders?

Drawing vertical/oblique lines in Metrical tree (tikz-qtree, tipa)

Did the new image of black hole confirm the general theory of relativity?

Sub-subscripts in strings cause different spacings than subscripts

Mortgage adviser recommends a longer term than necessary combined with overpayments



Is there an algorithm that imputes missing values based on n nearest columns? (KNN hybrid)



The 2019 Stack Overflow Developer Survey Results Are In
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsWhere in the workflow should we deal with missing data?Assigning values to missing target vector values in scikit-learnImputation of missing values and dealing with categorical valuesIs there a way to standardize latitude and longitude to be used as predictors in KNN algorithm?Is there are way to impute missing values by clustering, regression and stochastic regressionIs there an upper bound for k in nearest neighbors-based methods?How does KNN work if there are duplicates?How to select variables based on the mean correlation in a correlation matrix?Multiple filtering pandas columns based on values in another columnHow to create a df that gets sum of columns based on a groupby column?










0












$begingroup$


I have a dataset of 70 columns that have missing values. Each column has a few columns (3-5) that it is significantly more correlated than the others but each column's correlated columns are very different from other columns. I would like to perform a tweaked version of KNN imputation and before I start writing this from scratch, I'd like to know if there's something similar out there so I don't go about reinventing the wheel.



On initialisation, a correlation matrix is obtained between all the variables. For every missing cell that needs to be imputed, the algorithm retrieves from the correlation matrix the top n highest correlated columns and uses only those n columns to select the k nearest neighbours and use the mean value of those neighbours.



Thanks and apologies if I am not very clear with my question. I'll try to clarify if you have any doubts.










share|improve this question







New contributor




Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$
















    0












    $begingroup$


    I have a dataset of 70 columns that have missing values. Each column has a few columns (3-5) that it is significantly more correlated than the others but each column's correlated columns are very different from other columns. I would like to perform a tweaked version of KNN imputation and before I start writing this from scratch, I'd like to know if there's something similar out there so I don't go about reinventing the wheel.



    On initialisation, a correlation matrix is obtained between all the variables. For every missing cell that needs to be imputed, the algorithm retrieves from the correlation matrix the top n highest correlated columns and uses only those n columns to select the k nearest neighbours and use the mean value of those neighbours.



    Thanks and apologies if I am not very clear with my question. I'll try to clarify if you have any doubts.










    share|improve this question







    New contributor




    Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.







    $endgroup$














      0












      0








      0





      $begingroup$


      I have a dataset of 70 columns that have missing values. Each column has a few columns (3-5) that it is significantly more correlated than the others but each column's correlated columns are very different from other columns. I would like to perform a tweaked version of KNN imputation and before I start writing this from scratch, I'd like to know if there's something similar out there so I don't go about reinventing the wheel.



      On initialisation, a correlation matrix is obtained between all the variables. For every missing cell that needs to be imputed, the algorithm retrieves from the correlation matrix the top n highest correlated columns and uses only those n columns to select the k nearest neighbours and use the mean value of those neighbours.



      Thanks and apologies if I am not very clear with my question. I'll try to clarify if you have any doubts.










      share|improve this question







      New contributor




      Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.







      $endgroup$




      I have a dataset of 70 columns that have missing values. Each column has a few columns (3-5) that it is significantly more correlated than the others but each column's correlated columns are very different from other columns. I would like to perform a tweaked version of KNN imputation and before I start writing this from scratch, I'd like to know if there's something similar out there so I don't go about reinventing the wheel.



      On initialisation, a correlation matrix is obtained between all the variables. For every missing cell that needs to be imputed, the algorithm retrieves from the correlation matrix the top n highest correlated columns and uses only those n columns to select the k nearest neighbours and use the mean value of those neighbours.



      Thanks and apologies if I am not very clear with my question. I'll try to clarify if you have any doubts.







      python k-nn






      share|improve this question







      New contributor




      Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question







      New contributor




      Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question






      New contributor




      Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 18 mins ago









      JoshuaJoshua

      1




      1




      New contributor




      Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      Joshua is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.




















          0






          active

          oldest

          votes












          Your Answer








          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "557"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );






          Joshua is a new contributor. Be nice, and check out our Code of Conduct.









          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f49221%2fis-there-an-algorithm-that-imputes-missing-values-based-on-n-nearest-columns-k%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          Joshua is a new contributor. Be nice, and check out our Code of Conduct.









          draft saved

          draft discarded


















          Joshua is a new contributor. Be nice, and check out our Code of Conduct.












          Joshua is a new contributor. Be nice, and check out our Code of Conduct.











          Joshua is a new contributor. Be nice, and check out our Code of Conduct.














          Thanks for contributing an answer to Data Science Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f49221%2fis-there-an-algorithm-that-imputes-missing-values-based-on-n-nearest-columns-k%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Францішак Багушэвіч Змест Сям'я | Біяграфія | Творчасць | Мова Багушэвіча | Ацэнкі дзейнасці | Цікавыя факты | Спадчына | Выбраная бібліяграфія | Ушанаванне памяці | У філатэліі | Зноскі | Літаратура | Спасылкі | НавігацыяЛяхоўскі У. Рупіўся дзеля Бога і людзей: Жыццёвы шлях Лявона Вітан-Дубейкаўскага // Вольскі і Памідораў з песняй пра немца Адвакат, паэт, народны заступнік Ашмянскі веснікВ Минске появится площадь Богушевича и улица Сырокомли, Белорусская деловая газета, 19 июля 2001 г.Айцец беларускай нацыянальнай ідэі паўстаў у бронзе Сяргей Аляксандравіч Адашкевіч (1918, Мінск). 80-я гады. Бюст «Францішак Багушэвіч».Яўген Мікалаевіч Ціхановіч. «Партрэт Францішка Багушэвіча»Мікола Мікалаевіч Купава. «Партрэт зачынальніка новай беларускай літаратуры Францішка Багушэвіча»Уладзімір Іванавіч Мелехаў. На помніку «Змагарам за родную мову» Барэльеф «Францішак Багушэвіч»Памяць пра Багушэвіча на Віленшчыне Страчаная сталіца. Беларускія шыльды на вуліцах Вільні«Krynica». Ideologia i przywódcy białoruskiego katolicyzmuФранцішак БагушэвічТворы на knihi.comТворы Францішка Багушэвіча на bellib.byСодаль Уладзімір. Францішак Багушэвіч на Лідчыне;Луцкевіч Антон. Жыцьцё і творчасьць Фр. Багушэвіча ў успамінах ягоных сучасьнікаў // Запісы Беларускага Навуковага таварыства. Вільня, 1938. Сшытак 1. С. 16-34.Большая российская1188761710000 0000 5537 633Xn9209310021619551927869394п

          Partai Komunis Tiongkok Daftar isi Kepemimpinan | Pranala luar | Referensi | Menu navigasidiperiksa1 perubahan tertundacpc.people.com.cnSitus resmiSurat kabar resmi"Why the Communist Party is alive, well and flourishing in China"0307-1235"Full text of Constitution of Communist Party of China"smengembangkannyas

          ValueError: Expected n_neighbors <= n_samples, but n_samples = 1, n_neighbors = 6 (SMOTE) The 2019 Stack Overflow Developer Survey Results Are InCan SMOTE be applied over sequence of words (sentences)?ValueError when doing validation with random forestsSMOTE and multi class oversamplingLogic behind SMOTE-NC?ValueError: Error when checking target: expected dense_1 to have shape (7,) but got array with shape (1,)SmoteBoost: Should SMOTE be ran individually for each iteration/tree in the boosting?solving multi-class imbalance classification using smote and OSSUsing SMOTE for Synthetic Data generation to improve performance on unbalanced dataproblem of entry format for a simple model in KerasSVM SMOTE fit_resample() function runs forever with no result