Removing Constants from an Equations Derivative

This post is about the mission on Gradient Descent, mission 237-3. I do not know how to completely remove/ignore constants in a derivative. As far as I know, constants can be moved out of the derivative equation or canceled out through division/subtraction. I do not see any of those operations done in this screen:

image

I am not aware of any rule for finding derivatives that say you can completely remove constants without any effect on the equation. What rule/principle was used to remove the constants with no effect on the equation at all?

You can’t remove a constant if it is part of multiplication or division. If it is adding or subtracting you can.

This:

\dfrac{d}{da_1}\left(a_1x_1^{(i)}-y_1^{(i)}\right)

Is the same as this:

\dfrac{d}{da_1}\left(a_1x_1^{(i)}\right) - \dfrac{d}{da_1}\left(y_1^{(i)}\right)

The second part of the equation, \dfrac{d}{da_1}\left(y_1^{(i)}\right), is a constant since we are differentiating it with respect to a_1. As we know, the derivative of a constant is equals to zero, therefore:

\dfrac{d}{da_1}\left(a_1x_1^{(i)}\right) - \dfrac{d}{da_1}\left(y_1^{(i)}\right)

Is the same as:

\dfrac{d}{da_1}\left(a_1x_1^{(i)}\right) - 0

Which we also know is the same as \dfrac{d}{da_1}\left(a_1x_1^{(i)}\right).

We have now a simple derivative:

\dfrac{d}{da_1}\left(a_1x_1^{(i)}\right) = \left(a_1x_1^{(i)}\right)

We are not actually removing the constant, it is just equal to zero. And zero is a neutral value in addition and subtraction.

I hope this helps.

1 Like

It has. To make sure, the derivative of a1*x1 is x1 because a1 is considered a variable (value changes for every iteration) and we treat x1 as a constant since its value is known through the input data.

I read a rule that for derivative of an equation cx with c as a constant, the derivative is c. Thus, for a1x1 with a1 being a variable and x1 a constant, the derivative is x1, right?

This is @otavios.s’s reply:

We are not actually removing the constant, it is just equal to zero.

I also assume that the derivative of a1*x1 is x1 because a1 is considered a variable (value changes for every iteration of Gradient Descent) and we treat x1 as a constant since its value is known through the input data i.e. stays the same for every iteration of gradient descent.

Here are lists of the rules about the derivative of a constant c and derivative of a simple linear equation ax where a is a constant and x is a variable.

For the form f'(x):
image

https://www.mathsisfun.com/calculus/derivatives-rules.html

For the form image:
image

https://math.info/Calculus/Derivatives_Simple_Functions/

Yes, that’s exactly it.