NeuralNetwork

NetWork(layers...)

Connect multiple layers, and build a NeuralNetwork. NetWork also supports index. You can also add layers later using the add_layer!() Function.

Example

julia> N = NetWork(Dense(10=>5, relu), Dense(5=>1, relu))

julia> N[1]

Dense(IO:10=>5, σ:relu)

Dense(in=>out, σ; set_w = "Xavier", set_b = zeros)

Crate a traditinal Dense layer, whose forward propagation is given by: y = σ.(W * x .+ b) The input of x should be a Vactor of length in, (Sorry for you can't learn using batch. I'll implement)

Example

julia> D = Dense(5=>2, relu)
Dense(IO:5=>2, σ:relu)

julia> D(rand(Float64, 5)) |> size
(2,)

Descent(η=0.1)

Basic gradient descent optimizer with learning rate η.

Parameters

• learning rate : η

Example

Momentum(η=0.01, α=0.9, velocity)

Momentum gradient descent optimizer with learning rate η and parameter of velocity α.

Parameters

• learning rate : η
• parameter of velocity : α

Example

AdaGrad(η = 0.01)

Gradient descent optimizer with learning rate attenuation.

Parameters

• η : initial learning rate

Examples

Adam(η=0.01, β=(0.9, 0.99))

Gradient descent adaptive moment estimation optimizer.

Parameters

• η : learning rate
• β : Decay of momentums

Examples