Why "and", "&" are different and in what conditions should I use them?

I noticed from one of the mini-exercises of “working with missing data”,
In this line of code: unless I changed "and’ to “&”(from DQ instructor’s tip), it won’t work.

v_null = (mvc[v_col].isnull() and mvc[c_col].notnull()).sum()

Why is that? I already searched online from stackoverflow. With pages long discussion, it still confused me.

Is that true if we have two booleans expressions, together with some other method chaining syntax like the above example (boolean A & boolean B).syntax(), we will prefer to use “&”?
Or as someone from stackoverflow suggested–

  • If you are not dealing with arrays and are not performing math manipulations of integers, you probably want and .
  • If you have vectors of truth values that you wish to combine, use numpy with & .

or if the output gives me “value-error”, i just remember to try “&” to see if it works?

You probably went through that article? If you share which part of that post was hard to understand i can go through with you. There is no need to “try and see”.

This will explain example 1 in that post better: https://docs.python.org/3/library/stdtypes.html. The output is just list2 (y in the photo) because list1 is true (x in the photo). Read “Truth value testing” section to understand which objects in python are treated as True/False.

image

Things like 5&6 outputting 4 is because they were converted to binary representation, bitwise and-ed, then converted back

This explanation from Stack Overflow I think breaks it down very well.

The reason “&” is so commonly used with numpy and pandas is because often, you’re looking for an intersection of values.

E.g.
df[(df['player_age'] > 25) & (df['goals_scored] == 2)]

You use & in the above example because it returns the intersect, which is what you want. Above, I’m getting rows where a player’s age was above 25, and they scored two 2 goals. I’m basically getting the intersect of players who scored 2 goals and are above 25.

and wouldn’t work in the above because it’s simply a boolean logic operator. The other comments (they’re all mostly quite short!) in the link I provided explain why.

1 Like

@blueberrypudding85 Wow, that’s very clear and easy-to-understand explanation! Thank you so much!!!

2 Likes

@hanqi I did read thru that link last night. Thank you for sharing this python library documentation regarding boolean operators and bitwise operators. I think one of the answers that has 13 upvotes explains the best. At least the easiest to follow. Sorry I am not from a computer science major. At first glance of x and y => if x is false, then x, else y, it freaked me out. And when someone mentioned ‘short-circuiting’, I totally had no idea what it refers to.

Now I tried all those codes in my jupyter notebook, I selected what helped me best to understand the concepts “and” and “&”.

boolean_true and x -> x
boolean_false and x -> boolean_false

True and False -> True
(is my car red?) and (does car have wheels?) => logical true of false value
(the numeric value of the statement ‘my car is red’) & (the numeric value of the statement ‘my car has wheels’) => number and it is the intersection/overlapping condition from two boolean expressions(explained by blueberrypudding85).