Belajar Math - Multivariable Calculus & Gradients
Fungsi multivariabel f(x,y,z), gradient vector ∇f, dan directional derivative — fondasi dari backpropagation di neural networks, optimasi multi-parameter, dan field gradient dalam game physics dan simulation.
#Pendahuluan
Setelah di episode 23 kita mempelajari integral dan Fundamental Theorem — akumulasi dan area — pada episode ini kita mempelajari multivariable calculus: kalkulus untuk fungsi dengan lebih dari satu variabel. Di dunia nyata, kebanyakan fungsi bergantung pada banyak parameter — harga tergantung pada kualitas, lokasi, dan permintaan; loss function di ML tergantung pada semua weights. Gradient vector ∇f adalah generalisasi turunan ke fungsi multivariabel.
Mengapa ini penting? Karena backpropagation di neural networks pada dasarnya adalah chain rule multivariabel — menghitung bagaimana perubahan setiap weight mempengaruhi loss. Gradient vector memberikan arah "kemiringan terbesar" dari fungsi — dan gradient descent mengikuti arah berlawanan untuk mencapai minimum.
#Fungsi Multivariabel
def f(x, y, z):
"""Fungsi 3 variabel."""
return x**2 + 2*y**2 + 3*z**2 + x*y*z
# Evaluasi
print(f"f(1, 2, 3) = {f(1, 2, 3)}")
print(f"f(0, 0, 0) = {f(0, 0, 0)}")#Gradient Vector
Gradient ∇f = (∂f/∂x, ∂f/∂y, ∂f/∂z) adalah vektor partial derivatives:
import numpy as np
def numerical_gradient(f, point, h=1e-6):
"""Hitung gradient numerik."""
grad = np.zeros(len(point))
for i in range(len(point)):
point_plus = point.copy()
point_minus = point.copy()
point_plus[i] += h
point_minus[i] -= h
grad[i] = (f(point_plus) - f(point_minus)) / (2 * h)
return grad
# f(x, y) = x² + 2y²
def f_2d(v):
return v[0]**2 + 2*v[1]**2
# ∇f = (2x, 4y)
point = np.array([3.0, 2.0])
grad = numerical_gradient(f_2d, point)
grad_analytical = np.array([2*point[0], 4*point[1]])
print(f"f(x,y) = x² + 2y²")
print(f" ∇f at (3,2): numerik = {grad}")
print(f" ∇f at (3,2): analitis = {grad_analytical}")
print(f" Match: {np.allclose(grad, grad_analytical)}")#Arah Steepest Ascent
Gradient menunjukkan arah kemiringan terbesar ke atas. Untuk mencapai minimum, ikuti arah berlawanan gradient:
import numpy as np
def f_2d(v):
return v[0]**2 + 2*v[1]**2
def gradient_2d(v):
return np.array([2*v[0], 4*v[1]])
# Steepest ascent: ikuti gradient
# Steepest descent: ikuti -gradient
point = np.array([3.0, 2.0])
grad = gradient_2d(point)
print(f"Di titik {point}:")
print(f" f = {f_2d(point):.1f}")
print(f" Gradient (steepest ascent): {grad}")
print(f" -Gradient (steepest descent): {-grad}")
print(f" Magnitude gradient: {np.linalg.norm(grad):.4f}")#Directional Derivative
Directional derivative mengukur laju perubahan f dalam arah vektor unit u:
D_u f = ∇f · u
import numpy as np
def f_2d(v):
return v[0]**2 + 2*v[1]**2
def gradient_2d(v):
return np.array([2*v[0], 4*v[1]])
point = np.array([3.0, 2.0])
grad = gradient_2d(point)
# Directional derivative dalam berbagai arah
directions = {
"right (1,0)": np.array([1.0, 0.0]),
"up (0,1)": np.array([0.0, 1.0]),
"diagonal (1,1)/√2": np.array([1.0, 1.0]) / np.sqrt(2),
"steepest ascent (normalized)": grad / np.linalg.norm(grad),
}
print(f"Directional derivatives di {point}:")
for name, direction in directions.items():
D = np.dot(grad, direction)
print(f" {name}: D_u f = {D:.4f}")#Backpropagation: Chain Rule Multivariabel
import numpy as np
# Neural network: 2 input → 1 hidden (2 neurons) → 1 output
# Activation: sigmoid
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def sigmoid_derivative(x):
s = sigmoid(x)
return s * (1 - s)
# Forward pass
np.random.seed(42)
w1 = np.random.randn(2, 2) # input → hidden
b1 = np.random.randn(2)
w2 = np.random.randn(2) # hidden → output
b2 = np.random.randn(1)[0]
x = np.array([0.5, 0.3])
y_true = 1.0
# Forward
z1 = w1.T @ x + b1
a1 = sigmoid(z1)
z2 = w2 @ a1 + b2
a2 = sigmoid(z2)
loss = 0.5 * (a2 - y_true)**2
print(f"Forward pass:")
print(f" Input: {x}")
print(f" Hidden pre-activation: {z1}")
print(f" Hidden post-activation: {a1}")
print(f" Output: {a2:.6f}")
print(f" Loss: {loss:.6f}")
# Backward pass — chain rule multivariabel
dL_da2 = a2 - y_true
da2_dz2 = sigmoid_derivative(z2)
dL_dz2 = dL_da2 * da2_dz2
# Gradients untuk w2, b2
dL_dw2 = dL_dz2 * a1
dL_db2 = dL_dz2
# Gradients untuk w1, b1
dL_da1 = dL_dz2 * w2
dL_dz1 = dL_da1 * sigmoid_derivative(z1)
dL_dw1 = np.outer(x, dL_dz1)
dL_db1 = dL_dz1
print(f"\nBackward pass — gradients:")
print(f" ∂L/∂w2 = {dL_dw2}")
print(f" ∂L/∂b2 = {dL_db2:.6f}")
print(f" ∂L/∂w1 =\n{dL_dw1}")
print(f" ∂L/∂b1 = {dL_db1}")
# Update weights
lr = 0.5
w2_new = w2 - lr * dL_dw2
b2_new = b2 - lr * dL_db2
w1_new = w1 - lr * dL_dw1
b1_new = b1 - lr * dL_db1
# Forward pass lagi
z1_new = w1_new.T @ x + b1_new
a1_new = sigmoid(z1_new)
z2_new = w2_new @ a1_new + b2_new
a2_new = sigmoid(z2_new)
loss_new = 0.5 * (a2_new - y_true)**2
print(f"\nAfter 1 gradient descent step:")
print(f" New output: {a2_new:.6f} (was {a2:.6f})")
print(f" New loss: {loss_new:.6f} (was {loss:.6f})")Note
Backpropagation hanyalah penerapan chain rule multivariabel secara sistematis — menghitung gradient loss terhadap setiap weight dalam network. PyTorch dan TensorFlow melakukan ini secara otomatis melalui automatic differentiation, tetapi memahami konsep manual membantu debugging dan optimasi.
#Gradient Descent Multivariabel
import numpy as np
def f_2d(v):
"""Rosenbrock function — fungsi optimasi klasik."""
x, y = v
return (1 - x)**2 + 100*(y - x**2)**2
def grad_2d(v):
x, y = v
dfdx = -2*(1 - x) + 100 * 2*(y - x**2) * (-2*x)
dfdy = 100 * 2*(y - x**2)
return np.array([dfdx, dfdy])
# Gradient descent
point = np.array([-1.0, 1.0])
lr = 0.001
print("Gradient descent pada Rosenbrock function:")
for i in range(5000):
grad = grad_2d(point)
point = point - lr * grad
if i % 1000 == 0:
print(f" Step {i}: f = {f_2d(point):.6f}, pos = ({point[0]:.4f}, {point[1]:.4f})")
print(f"\nMinimum: ({point[0]:.4f}, {point[1]:.4f})")
print(f"f(minimum) = {f_2d(point):.6f} (exact: 0 at (1,1))")#Penutup
Inti yang harus dibawa pulang:
- Gradient
∇f= vektor partial derivatives — menunjukkan arah steepest ascent. - Steepest descent: ikuti
-∇f— ini adalah gradient descent. - Directional derivative:
D_u f = ∇f · u— laju perubahan dalam arah u. - Backpropagation = chain rule multivariabel — menghitung gradient loss terhadap setiap weight.
- Multivariable calculus adalah fondasi dari optimasi di machine learning dan physics simulation.
Di episode 25 selanjutnya kita akan mempelajari differential equations — persamaan yang melibatkan turunan, digunakan untuk memodelkan sistem dinamis: gerak projectile, population growth, dan predator-prey model!