How to count grouped occurrences? Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsWhat is the difference between dcast and recast in R?How to plot two columns of single DataFrame on Y axisHow to delete entire row if values in a column are NaNHow do I add a column to a Pandas dataframe based on other rows and columns in the dataframe?Pandas Conditional Fill NaN Forward/BackwardHow can I merge 2+ DataFrame objects without duplicating column names?How to do multivariate survival analysis on dataset having only categorical variablesMerging dataframes in Pandas is taking a surprisingly long timeHow to create a df that gets sum of columns based on a groupby column?Groupby date and find number of occurrences of a value a in another column using pandas

Sorting numerically

Is a manifold-with-boundary with given interior and non-empty boundary essentially unique?

Why was the term "discrete" used in discrete logarithm?

What is the musical term for a note that continously plays through a melody?

What's the purpose of writing one's academic bio in 3rd person?

How do I keep my slimes from escaping their pens?

Do I really need recursive chmod to restrict access to a folder?

The logistics of corpse disposal

Is there a Spanish version of "dot your i's and cross your t's" that includes the letter 'ñ'?

Did Xerox really develop the first LAN?

How does a Death Domain cleric's Touch of Death feature work with Touch-range spells delivered by familiars?

Storing hydrofluoric acid before the invention of plastics

Antler Helmet: Can it work?

What is the longest distance a 13th-level monk can jump while attacking on the same turn?

How to deal with a team lead who never gives me credit?

Is there a documented rationale why the House Ways and Means chairman can demand tax info?

Stars Make Stars

What is this single-engine low-wing propeller plane?

Does surprise arrest existing movement?

How to bypass password on Windows XP account?

Java 8 stream max() function argument type Comparator vs Comparable

What do you call a phrase that's not an idiom yet?

Gastric acid as a weapon

How to motivate offshore teams and trust them to deliver?



How to count grouped occurrences?



Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsWhat is the difference between dcast and recast in R?How to plot two columns of single DataFrame on Y axisHow to delete entire row if values in a column are NaNHow do I add a column to a Pandas dataframe based on other rows and columns in the dataframe?Pandas Conditional Fill NaN Forward/BackwardHow can I merge 2+ DataFrame objects without duplicating column names?How to do multivariate survival analysis on dataset having only categorical variablesMerging dataframes in Pandas is taking a surprisingly long timeHow to create a df that gets sum of columns based on a groupby column?Groupby date and find number of occurrences of a value a in another column using pandas










1












$begingroup$


My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.










share|improve this question











$endgroup$







  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01















1












$begingroup$


My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.










share|improve this question











$endgroup$







  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01













1












1








1


2



$begingroup$


My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.










share|improve this question











$endgroup$




My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.







python pandas data-cleaning






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 1 hour ago







FirstSlack

















asked Apr 3 '18 at 7:55









FirstSlackFirstSlack

1016




1016







  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01












  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01







1




1




$begingroup$
On what basis is count generated?
$endgroup$
– Ankit Seth
Apr 3 '18 at 10:01




$begingroup$
On what basis is count generated?
$endgroup$
– Ankit Seth
Apr 3 '18 at 10:01










1 Answer
1






active

oldest

votes


















4












$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$












  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49











Your Answer








StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29840%2fhow-to-count-grouped-occurrences%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









4












$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$












  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49















4












$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$












  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49













4












4








4





$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$



You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.







share|improve this answer














share|improve this answer



share|improve this answer








edited Apr 3 '18 at 12:26

























answered Apr 3 '18 at 8:40









Kiritee GakKiritee Gak

1,3591521




1,3591521











  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49
















  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49















$begingroup$
He wants some counts to be zero also.., otherwise awesome solution..
$endgroup$
– Aditya
Apr 3 '18 at 9:06





$begingroup$
He wants some counts to be zero also.., otherwise awesome solution..
$endgroup$
– Aditya
Apr 3 '18 at 9:06













$begingroup$
I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
$endgroup$
– FirstSlack
Apr 3 '18 at 19:03





$begingroup$
I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
$endgroup$
– FirstSlack
Apr 3 '18 at 19:03













$begingroup$
Glad if it is of help! Good luck.
$endgroup$
– Kiritee Gak
Apr 4 '18 at 2:49




$begingroup$
Glad if it is of help! Good luck.
$endgroup$
– Kiritee Gak
Apr 4 '18 at 2:49

















draft saved

draft discarded
















































Thanks for contributing an answer to Data Science Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

Use MathJax to format equations. MathJax reference.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29840%2fhow-to-count-grouped-occurrences%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Францішак Багушэвіч Змест Сям'я | Біяграфія | Творчасць | Мова Багушэвіча | Ацэнкі дзейнасці | Цікавыя факты | Спадчына | Выбраная бібліяграфія | Ушанаванне памяці | У філатэліі | Зноскі | Літаратура | Спасылкі | НавігацыяЛяхоўскі У. Рупіўся дзеля Бога і людзей: Жыццёвы шлях Лявона Вітан-Дубейкаўскага // Вольскі і Памідораў з песняй пра немца Адвакат, паэт, народны заступнік Ашмянскі веснікВ Минске появится площадь Богушевича и улица Сырокомли, Белорусская деловая газета, 19 июля 2001 г.Айцец беларускай нацыянальнай ідэі паўстаў у бронзе Сяргей Аляксандравіч Адашкевіч (1918, Мінск). 80-я гады. Бюст «Францішак Багушэвіч».Яўген Мікалаевіч Ціхановіч. «Партрэт Францішка Багушэвіча»Мікола Мікалаевіч Купава. «Партрэт зачынальніка новай беларускай літаратуры Францішка Багушэвіча»Уладзімір Іванавіч Мелехаў. На помніку «Змагарам за родную мову» Барэльеф «Францішак Багушэвіч»Памяць пра Багушэвіча на Віленшчыне Страчаная сталіца. Беларускія шыльды на вуліцах Вільні«Krynica». Ideologia i przywódcy białoruskiego katolicyzmuФранцішак БагушэвічТворы на knihi.comТворы Францішка Багушэвіча на bellib.byСодаль Уладзімір. Францішак Багушэвіч на Лідчыне;Луцкевіч Антон. Жыцьцё і творчасьць Фр. Багушэвіча ў успамінах ягоных сучасьнікаў // Запісы Беларускага Навуковага таварыства. Вільня, 1938. Сшытак 1. С. 16-34.Большая российская1188761710000 0000 5537 633Xn9209310021619551927869394п

Partai Komunis Tiongkok Daftar isi Kepemimpinan | Pranala luar | Referensi | Menu navigasidiperiksa1 perubahan tertundacpc.people.com.cnSitus resmiSurat kabar resmi"Why the Communist Party is alive, well and flourishing in China"0307-1235"Full text of Constitution of Communist Party of China"smengembangkannyas

ValueError: Expected n_neighbors <= n_samples, but n_samples = 1, n_neighbors = 6 (SMOTE) The 2019 Stack Overflow Developer Survey Results Are InCan SMOTE be applied over sequence of words (sentences)?ValueError when doing validation with random forestsSMOTE and multi class oversamplingLogic behind SMOTE-NC?ValueError: Error when checking target: expected dense_1 to have shape (7,) but got array with shape (1,)SmoteBoost: Should SMOTE be ran individually for each iteration/tree in the boosting?solving multi-class imbalance classification using smote and OSSUsing SMOTE for Synthetic Data generation to improve performance on unbalanced dataproblem of entry format for a simple model in KerasSVM SMOTE fit_resample() function runs forever with no result