# Define function for convert NaN to np.nan, '-' to False, other non zero string to True
elif value == '-':
# Create new column and extract converted values from columns 'Contributing Factors. Dissatisfaction'
# and 'Contributing Factors. Job Dissatisfaction' to 'dissatisfied'
intermediate = tafe_resignations.copy()
tafe_resignations['dissatisfied'] = intermediate.loc[:, ['Contributing Factors. Dissatisfaction', 'Contributing Factors. Job Dissatisfaction']].applymap(update_vals).any(axis=1, skipna=False)
tafe_resignations_up = tafe_resignations.copy()
What I expected to happen:
I am trying to avoid the warning in the title of this post and have spent a decent amount of time trying to figure it out. What is the proper syntax for the code above to avoid “SettingWithCopyWarning:”
What are the principles behind this?
you simply need to add .copy() to the line where you apply your function.
tafe_resignations['dissatisfied'] = intermediate.loc[:, ['Contributing Factors. Dissatisfaction', 'Contributing Factors. Job Dissatisfaction']].applymap(update_vals).any(axis=1, skipna=False).copy()
what is happening is that in one step you have:
- selected a bunch of columns and rows from an existing dataframe
- altered the data by applying a function
- saved that altered data under a different name
Python cant tell if you want to:
- alter the selected data in the existing dataframe, and then also save that selected data as a new dataframe or series
- use the selected data to create the new DF or series without altering the selected data itself.
when you use .copy() you are telling python to do option 2. So it selects the data, runs the function, creates a copy of those results, and then saves the copy as you choose without saving the changes to the original data.
Thank you very much for taking the time to spell it out for me!