Week 8 Answers
1. What are the challenges associated with using the Tanh(x) activation function?
a. It is not zero centered
b. Computationally expensive
c. Non-differentiable at 0
d. Saturation
Answer: D
2. Which of the following problems makes training a neural network harder while using sigmoid as the activation function?
a. Not-continuous at 0
b. Not-differentiable at 0
c. Saturation
d. Computationally expensive
Answer: C
3. Consider the Exponential ReLU (ELU) activation function, defined as:
f(x)={x,a(ex−1),x>0;x≤0}f(x) = \{x, a(e^x -1), x > 0; x \leq 0\}
where a≠0a \neq 0. Which of the following statements is true?
a. The function is discontinuous at x=0
b. The function is non-differentiable at x=0
c. Exponential ReLU can produce negative values
d. Exponential ReLU is computationally less expensive than ReLU
Answer: C
4. We have observed that the sigmoid neuron has become saturated. What might be the possible output values at this neuron?
a. 0.0666
b. 0.589
c. 0.9734
d. 0.498
e. 1
Answer: C
5. What is the gradient of the sigmoid function at saturation?
Answer: 0
6. Which of the following are common issues caused by saturating neurons in deep networks?
a. Vanishing gradients
b. Slow convergence during training
c. Overfitting
d. Increased model complexity
Answer: A
7. Given a neuron initialized with weights w1=0.9w_1 = 0.9, w2=1.7w_2 = 1.7, and inputs x1=0.4x_1 = 0.4, x2=−0.7x_2 = -0.7, calculate the output of a ReLU neuron.
Answer: 0
8. Which of the following is incorrect with respect to the batch normalization process in neural networks?
a. We normalize the output produced at each layer before feeding it into the next layer
b. Batch normalization leads to a better initialization of weights
c. Backpropagation can be used after batch normalization
d. Variance and mean are not learnable parameters
Answer: D
9. Which of the following is an advantage of unsupervised pre-training in deep learning?
a. It helps in reducing overfitting
b. Pre-trained models converge faster
c. It requires fewer computational resources
d. It improves the accuracy of the model
Answer: B
10. How can you tell if your network is suffering from the Dead ReLU problem?
a. The loss function is not decreasing during training
b. A large number of neurons have zero output
c. The accuracy of the network is not improving
d. The network is overfitting to the training data
Answer: B
