基于非易失性存储器的高性能神经计算和神经优化的通用随机点积电路 ...

大脑执行的计算本质上是随机的。 在分子水平上，这是由于神经元离子通道的随机门控和突触间隙中递质释放的概率性质所致。 嘈杂的，不可靠的分子机制是对相同刺激的可重复呈现产生实质上不同的神经反应的原因，这反过来又会导致复杂的泊松动力学等的随机行为。尽管噪声对传统的数字电路总是有害的， 现代神经科学的研究发现神经元信号的极低信噪比（SNR）在大脑功能中发挥重要作用，比如器件或者芯片对于外界环境的适应性以及系统的稀疏性方面。

摘要

随机神经网络中的关键操作是随机点积，它已成为解决机器学习，信息理论和统计问题的最先进方法。尽管已经有很多关于点积电路和随机神经元的演示，但是结合了这两种功能的有效硬件实现仍然缺失。来自加州大学圣芭芭拉分校的研究者提出了基于无源集成金属氧化物忆阻器或嵌入式浮栅存储器的紧凑，快速，节能和可扩展的随机点积电路。电路的高性能归功于混合信号的实现，而有效的随机操作则是通过利用存储单元阵列固有的和/或外部的电路噪声来实现的。由模拟存储设备实现的权重的动态缩放允许有效实现不同的退火方法以改善功能。所提出的方法在两个代表性的应用中进行了实验验证，即通过实现用于解决四节点图划分问题的神经网络以及具有10输入和8隐藏神经元的Boltzmann机。

The key operation in stochastic neural networks, which have become the state-of-the-art approach for solving problems in machine learning, information theory, and statistics, is a stochastic dot-product. While there have been many demonstrations of dot-product circuits and, separately, of stochastic neurons, the efficient hardware implementation combining both functionalities is still missing. Here we report compact, fast, energy-efficient, and scalable stochastic dot-product circuits based on either passively integrated metal-oxide memristors or embedded floating-gate memories. The circuit’s high performance is due to mixed-signal implementation, while the efficient stochastic operation is achieved by utilizing circuit’s noise, intrinsic and/or extrinsic to the memory cell array. The dynamic scaling of weights, enabled by analog memory devices, allows for efficient realization of different annealing approaches to improve functionality. The proposed approach is experimentally verified for two representative applications, namely by implementing neural network for solving a four-node graphpartitioning problem, and a Boltzmann machine with 10-input and 8-hidden neurons.