The sigmoid function, f(x) = 1/(1+e^(-x)), returns [0.5,1) for all x >= 0, and (0,0.5) for all x < 0. (I believe).
In mission 241, we’re asked to apply the sigmoid function to each value x in an array, then set x to 1 if sigmoid(x) >= 0.5, and 0 if sigmoid(x) <= 0.5. But, seeing as the sigmoid function returns a number greater than 0.5 if fed a number greater than or equal to zero, and a number less than 0.5 if fed a number less than zero, why not cut out the middleman and skip the step of applying the sigmoid function to each value. Instead, why not just set each value to 1 if the value is greater than or equal to zero, and 0 if the value is less than zero?
For the purpose of mission 241, I tested this approach and it worked. See the following code:
from sklearn.linear_model import SGDClassifier from sklearn.datasets import make_classification class_data = make_classification(n_samples=100, n_features=4, random_state=1) class_features = pd.DataFrame(class_data) class_labels = pd.Series(class_data) class_features['bias'] = 1 def log_train(class_features,class_labels): lr = SGDClassifier() lr.fit(class_features,class_labels) return lr.coef_ def sigmoid_skip(linear_combination): for i in range(len(linear_combination)): if linear_combination[i] >= 0: linear_combination[i] = 1 else: linear_combination[i] = 0 return linear_combination def sigmoid(linear_combination): for i in range(len(linear_combination)): linear_combination[i] = 1/(1+np.exp(-linear_combination[i])) return linear_combination def log_feedforward(class_features,log_train_weights): linear_combination = np.dot(class_features,np.matrix.transpose(log_train_weights)) ##Comment OUT the two lines below log_predictions = sigmoid_skip(linear_combination) return log_predictions # Uncomment these lines below # log_predictions = sigmoid(linear_combination) # for i in range(len(log_predictions)): # if log_predictions[i] >= 0.5: # log_predictions[i] = 1 # else: # log_predictions[i] = 0 # return log_predictions # Uncomment this code when you're ready to test your functions. log_train_weights = log_train(class_features, class_labels) log_predictions = log_feedforward(class_features, log_train_weights)
This code runs using the approach that skips applying the sigmoid function to values and directly converts each value to zero or one. The code has the same result if one comments out the lines marked “comment out”, and uncomments the lines marked “uncomment out”.
I’m sure the sigmoid function is important, but I am a little confused as to why it’s important. I know one of the reasons it’s important is because it converts numbers to a number between zero and 1, but I imagine that can’t be the only reason why it’s important, seeing as one can simply code in something along the lines of
if x >= 0: return 1 else: return 0
The above code also must run faster than applying the sigmoid function to each value then applying another function to each value to see if its less than or greater than (or equal to) zero.
So, what exactly are the reasons why the sigmoid function is so important, and how important are such reasons to know for the purposes of being a good data scientist?