TensorFlow Minimize Cost, Gradient Discent Algorithm

TensorFlow Minimize Cost, Gradient Discent Algorithm

이전 포스트

TensorFlow 설치 및 PyCharm 설치

TensorFlow Linear Regression

---

Minimize Cost

cost(Loss) 최소화

Simplified Hypothesis

H(x) = Wx

cost(W) = 1mi=1m(H(xi)-yi)2

MinimizeCost.py

Source

import tensorflow as tf

import matplotlib.pyplot as plt

# tf Graph Input

X = [1., 2., 3.]

Y = [1., 2., 3.]

m = n_samples = len(X)

# Set Model weights

W = tf.placeholder(tf.float32)

# Construct a linear model

hypothesis = tf.mul(X,W) # H(x) = Wx

# Cost function

cost = tf.reduce_sum(tf.pow(hypothesis-Y,2))/(m) # cost(W)

# Initializing the Variables

init = tf.initialize_all_variables()

# For Graphs

W_val = []

cost_val = []

# Launch the Graph

sess = tf.Session()

sess.run(init)

for i in range(-50,50):

  print i*0.1, sess.run(cost, feed_dict={W: i*0.1})

  W_val.append(i*0.1)

  cost_val.append(sess.run(cost, feed_dict={W: i*0.1}))

# Graphic Display

plt.plot(W_val, cost_val, 'ro')

plt.ylabel('Cost')

plt.xlabel('W')

plt.show()

API

tf.mul(X,Y) : 곱하기 X*Y

Returns x * y element-wise.

https://www.tensorflow.org/versions/r0.9/api_docs/python/math_ops.html#mul

tf.reduce_sum() : Tensor의 차원(dimension)을 더함을 계산한다.

Computes the sum of elements across dimensions of a tensor.

https://www.tensorflow.org/versions/r0.9/api_docs/python/math_ops.html#reduce_sum

matplotlib.pyplot :  MATLAB 의 plot 을 제공합니다.

Provides a MATLAB-like plotting framework.

http://matplotlib.org/api/pyplot_api.html

Run Execution

Step과 Cost를 출력하고 있으며

Step 1에서 Cost가 0임을 알 수 있다.

GradientDescent Algorithm

경사도를 이용해 미분값이 0을 찾는 방법

W:= W-1mi=1m(W(xi)-yi)*xi

Find m = 0

Source

import tensorflow as tf

import matplotlib.pylab as plt

# tf Graph Input

x_data = [1., 2., 3.]

y_data = [1., 2., 3.]

# Try to find values for W and b that compute y_data = W * x_data

# We know that W should be 1

# random_uniform -> random function -10.0 ~ 10.0

W = tf.Variable(tf.random_uniform([1],-10.0,10.0))

# Place Holder

X = tf.placeholder(tf.float32)

Y = tf.placeholder(tf.float32)

# Our hypothesis

hypothesis = W * X

# Cost function

cost = tf.reduce_sum(tf.square(hypothesis - Y))

# Minimize

# W = W- alpha(step) * 1/m * Sigma(W*X-Y) * X

# Sigma( W * X - Y) * X -> Derivative

descent = W - tf.mul(0.1 , tf.reduce_mean(tf.mul((tf.mul(W , X) - Y),X)))

update = W.assign(descent) # Operation

# Before Starting, initalize the Variables

init = tf.initialize_all_variables()

# Launch the graph.

sess = tf.Session()

sess.run(init)

# For Graphs

W_val = []

cost_val = []

# Fit the line.

for step in xrange(100) :

      sess.run(update, feed_dict={X:x_data, Y:y_data})

      cost_val.append(sess.run(cost,feed_dict={X:x_data, Y:y_data}))

      W_val.append(sess.run(W))

      print step, sess.run(cost, feed_dict={X:x_data, Y:y_data}), sess.run(W)

# Show the Graph

plt.plot(W_val, cost_val)

plt.ylabel('Cost')

plt.xlabel('W')

plt.show()

Run Execution

Step 27 에서 W = 1 일 때, Cost = 0 인 것을 알 수 있다.

---

출처

Sung Kim http://hunkim.github.io/ml/

TensorFlow https://www.tensorflow.org/

PyCharm https://www.jetbrains.com/pycharm/

MATPLOTlib http://matplotlib.org/