How to count grouped occurrences? Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsWhat is the difference between dcast and recast in R?How to plot two columns of single DataFrame on Y axisHow to delete entire row if values in a column are NaNHow do I add a column to a Pandas dataframe based on other rows and columns in the dataframe?Pandas Conditional Fill NaN Forward/BackwardHow can I merge 2+ DataFrame objects without duplicating column names?How to do multivariate survival analysis on dataset having only categorical variablesMerging dataframes in Pandas is taking a surprisingly long timeHow to create a df that gets sum of columns based on a groupby column?Groupby date and find number of occurrences of a value a in another column using pandas

Sorting numerically

Is a manifold-with-boundary with given interior and non-empty boundary essentially unique?

Why was the term "discrete" used in discrete logarithm?

What is the musical term for a note that continously plays through a melody?

What's the purpose of writing one's academic bio in 3rd person?

How do I keep my slimes from escaping their pens?

Do I really need recursive chmod to restrict access to a folder?

The logistics of corpse disposal

Is there a Spanish version of "dot your i's and cross your t's" that includes the letter 'ñ'?

Did Xerox really develop the first LAN?

How does a Death Domain cleric's Touch of Death feature work with Touch-range spells delivered by familiars?

Storing hydrofluoric acid before the invention of plastics

Antler Helmet: Can it work?

What is the longest distance a 13th-level monk can jump while attacking on the same turn?

How to deal with a team lead who never gives me credit?

Is there a documented rationale why the House Ways and Means chairman can demand tax info?

Stars Make Stars

What is this single-engine low-wing propeller plane?

Does surprise arrest existing movement?

How to bypass password on Windows XP account?

Java 8 stream max() function argument type Comparator vs Comparable

What do you call a phrase that's not an idiom yet?

Gastric acid as a weapon

How to motivate offshore teams and trust them to deliver?



How to count grouped occurrences?



Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsWhat is the difference between dcast and recast in R?How to plot two columns of single DataFrame on Y axisHow to delete entire row if values in a column are NaNHow do I add a column to a Pandas dataframe based on other rows and columns in the dataframe?Pandas Conditional Fill NaN Forward/BackwardHow can I merge 2+ DataFrame objects without duplicating column names?How to do multivariate survival analysis on dataset having only categorical variablesMerging dataframes in Pandas is taking a surprisingly long timeHow to create a df that gets sum of columns based on a groupby column?Groupby date and find number of occurrences of a value a in another column using pandas










1












$begingroup$


My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.










share|improve this question











$endgroup$







  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01















1












$begingroup$


My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.










share|improve this question











$endgroup$







  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01













1












1








1


2



$begingroup$


My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.










share|improve this question











$endgroup$




My dataset has 2 columns:



1) Gender( that has 1 or 2)



2) Rating1(That has 3 ratings (1,2,3))



rating1: 1,2,3,3,2

gender: 2,1,2,2,1


I want output as:enter image description here



I have tried this:



nutri.groupby(['Rating1','Gender']).nunique()


but I get an output which looks wrong.







python pandas data-cleaning






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 1 hour ago







FirstSlack

















asked Apr 3 '18 at 7:55









FirstSlackFirstSlack

1016




1016







  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01












  • 1




    $begingroup$
    On what basis is count generated?
    $endgroup$
    – Ankit Seth
    Apr 3 '18 at 10:01







1




1




$begingroup$
On what basis is count generated?
$endgroup$
– Ankit Seth
Apr 3 '18 at 10:01




$begingroup$
On what basis is count generated?
$endgroup$
– Ankit Seth
Apr 3 '18 at 10:01










1 Answer
1






active

oldest

votes


















4












$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$












  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49











Your Answer








StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29840%2fhow-to-count-grouped-occurrences%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









4












$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$












  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49















4












$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$












  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49













4












4








4





$begingroup$

You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.






share|improve this answer











$endgroup$



You can do that by



>>> import pandas as pd
>>> m = pd.DataFrame('gender': [1, 2, 2, 1, 1, 2, 1], 'rating': [3, 4, 2, 1, 3, 1, 5])
>>> m.groupby(['rating','gender']).size().to_frame('count').reset_index()
rating gender count
0 1 1 1
1 1 2 1
2 2 2 1
3 3 1 2
4 4 2 1
5 5 1 1


Hope this is what you want to pull.



Edit:
As mentioned, I did not account for zero values.



You need to do one more step additional to get what you want. Finding the combinations that are missing and then joining it. The one liner solution would be



>>> from itertools import product
>>> m.groupby(['rating', 'gender']).size().to_frame('count').reset_index().merge(
pd.DataFrame(list(set([i for i in product(*[m.gender, m.rating])])), columns=['gender', 'rating']),
on=['rating', 'gender'],
how='right').fillna(value=0)
rating gender count
0 1 1 1.0
1 1 2 1.0
2 2 2 1.0
3 3 1 2.0
4 4 2 1.0
5 5 1 1.0
6 2 1 0.0
7 4 1 0.0
8 3 2 0.0
9 5 2 0.0


Explanation



Get the original grouped counts as mentioned before the first edit but this time you need to join with the combinations that are missing to get the zero counts. Use itertools.product to get all combinations of gender and rating and right join it with original grouped frame on rating and gender to get merged DataFrame which has numpy.na values if no count is present and then use fillna method to fill it with zero. The only loophole is if a rating say 4 is not there in the original data, there is no combination that follows it.



Hope this helps.







share|improve this answer














share|improve this answer



share|improve this answer








edited Apr 3 '18 at 12:26

























answered Apr 3 '18 at 8:40









Kiritee GakKiritee Gak

1,3591521




1,3591521











  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49
















  • $begingroup$
    He wants some counts to be zero also.., otherwise awesome solution..
    $endgroup$
    – Aditya
    Apr 3 '18 at 9:06











  • $begingroup$
    I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
    $endgroup$
    – FirstSlack
    Apr 3 '18 at 19:03











  • $begingroup$
    Glad if it is of help! Good luck.
    $endgroup$
    – Kiritee Gak
    Apr 4 '18 at 2:49















$begingroup$
He wants some counts to be zero also.., otherwise awesome solution..
$endgroup$
– Aditya
Apr 3 '18 at 9:06





$begingroup$
He wants some counts to be zero also.., otherwise awesome solution..
$endgroup$
– Aditya
Apr 3 '18 at 9:06













$begingroup$
I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
$endgroup$
– FirstSlack
Apr 3 '18 at 19:03





$begingroup$
I was actually working on a Big Dataset and I don't really need a count for 0 If anything I will use fillna(0.0). I was interested in getting the format. like the example in above question: If the rating is 2.0, how many are men(1) and how many are women (2). In R, there is this 'dplyr ' package which works so smooth, I wanted to learn Python for job interviews.
$endgroup$
– FirstSlack
Apr 3 '18 at 19:03













$begingroup$
Glad if it is of help! Good luck.
$endgroup$
– Kiritee Gak
Apr 4 '18 at 2:49




$begingroup$
Glad if it is of help! Good luck.
$endgroup$
– Kiritee Gak
Apr 4 '18 at 2:49

















draft saved

draft discarded
















































Thanks for contributing an answer to Data Science Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

Use MathJax to format equations. MathJax reference.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29840%2fhow-to-count-grouped-occurrences%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

ValueError: Error when checking input: expected conv2d_13_input to have shape (3, 150, 150) but got array with shape (150, 150, 3)2019 Community Moderator ElectionError when checking : expected dense_1_input to have shape (None, 5) but got array with shape (200, 1)Error 'Expected 2D array, got 1D array instead:'ValueError: Error when checking input: expected lstm_41_input to have 3 dimensions, but got array with shape (40000,100)ValueError: Error when checking target: expected dense_1 to have shape (7,) but got array with shape (1,)ValueError: Error when checking target: expected dense_2 to have shape (1,) but got array with shape (0,)Keras exception: ValueError: Error when checking input: expected conv2d_1_input to have shape (150, 150, 3) but got array with shape (256, 256, 3)Steps taking too long to completewhen checking input: expected dense_1_input to have shape (13328,) but got array with shape (317,)ValueError: Error when checking target: expected dense_3 to have shape (None, 1) but got array with shape (7715, 40000)Keras exception: Error when checking input: expected dense_input to have shape (2,) but got array with shape (1,)

Ружовы пелікан Змест Знешні выгляд | Пашырэнне | Асаблівасці біялогіі | Літаратура | НавігацыяДагледжаная версіяправерана1 зменаДагледжаная версіяправерана1 змена/ 22697590 Сістэматыкана ВіківідахВыявына Вікісховішчы174693363011049382

Illegal assignment from SObject to ContactFetching String, Id from Map - Illegal Assignment Id to Field / ObjectError: Compile Error: Illegal assignment from String to BooleanError: List has no rows for assignment to SObjectError on Test Class - System.QueryException: List has no rows for assignment to SObjectRemote action problemDML requires SObject or SObject list type error“Illegal assignment from List to List”Test Class Fail: Batch Class: System.QueryException: List has no rows for assignment to SObjectMapping to a user'List has no rows for assignment to SObject' Mystery