Recent investigations in neuromorphic photonics, i.e. neuromorphic architectures on photonics platforms, have garnered much interest to enable high-bandwidth, low-latency, low-energy applications of neural networks in machine learning and neuromorphic computing. Although electronics can match biological time scales and exceed them, they eventually reach bandwidth limitations. Neuromorphic photonics exploits the advantages of optical electronics, including the ease of analog processing, and fully parallelism achieved by busing multiple signals on a single waveguide at the speed of light. In this chapter, we summarize silicon photonic on-chip neural network architectures that have been widely investigated from different approaches that can be grouped into three categories: (1) reservoir computing; reconfigurable architectures based on (2) Mach-Zehnder interferometers, and (3) ring-resonators. Our scope is limited to their forward propagation, and includes potential on-chip machine learning tasks and efficiency analyses of the proposed architectures.