The current report explores the available supervised learning algorithms for multilayered spiking neural networks. Gradient descent based algorithms are one of the most used learning methods for rate neurons. The back-propagation version for spiking neurons firing a single spike, SpikeProp, promises the same learning abilities as for artificial neural networks. Systematic investigations on this learning method show that SpikeProp requires more computations than back-propagation and a reference start time is critical for convergence. These issues require significant improvements to the gradient descent learning method for spiking neural networks in order for an efficient algorithm to be developed. Further developments include a learning algorithm for input and output neurons with multiple spikes, and a general learning rule for recurrent networks.