January 2018 edition – Vol.10 no.1

2023 International Conference on Microelectronics (ICM)
from December 17 to 20, 2023, Abu Dhabi, United Arab Emirates.
More details

2023 IEEE 11th International Conference on Systems and Control (ICSC)
from December 18 to 20, 2023, Sousse, Tunisia.
More details

2024 37th International Conference on VLSI Design and 2024 23rd International Conference on Embedded Systems (VLSID)
from January 6 to 10, 2024, Kolkata, India.
More details

2024 IEEE International Solid-State Circuits Conference (ISSCC)
from February 18 to 22, 2024, San Francisco, California, USA.
More details

2024 IEEE 15th Latin America Symposium on Circuits and Systems (LASCAS)
from February 27 to March 1, 2024, Punta del Este, Uruguay.
More details

2024 IEEE Custom Integrated Circuits Conference (CICC)
from April 21 to 24, 2024, Denver, Colorado, USA.
More details

2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS)
from April 22 to 25, 2024, Abu Dhabi, United Arab Emirates.
More details

2024 9th International Conference on Integrated Circuits, Design, and Verification (ICDV)
from June 6 to 7, 2024, Hanoi, Vietnam.
More details

2024 61st ACM/IEEE Design Automation Conference (DAC)
from June 23 to 27, 2024, San Francisco, California, USA.
More details

2024 IEEE International Conference on Multimedia and Expo (ICME)
from July 15 to 19, 2024, Niagara Falls, Ontario, Canada.
More details

Prof. Jean-Pierre David
Polytechnique Montréal
Member of ReSMiQ since 2003

Jean Pierre David received a Ph.D. in Applied Sciences from the Catholic University of Louvain, Belgium. He worked as a researcher at the Laboratory of Integrated Devices and Electronic Circuits (DICE) at the Catholic University of Louvain. He was an adjunct professor at the University of Montreal, in the Laboratory of Analysis and Synthesis of Ordinary Systems (LASSO). Currently, he is Associate Professor in the Department of Electrical Engineering and a member of the Research Group in Microelectronics and Microsystems (GR2M) at École Polytechnique in Montreal, Canada. His current research interests are mainly in the design, configuration and programming of digital systems; programmable logic systems (e.g., FPGAs, processors, microcontrollers) and their applications, particularly in real-time simulations (Hardware In the Loop - HIL), telecommunications (e.g., network monitoring, 5G), and neural networks. Professor David is the author or co-author of more than 65 articles in peer-reviewed international conferences and journals and holds two patents. He is a member of IEEE and ReSMiQ.

More details

Below is a selection of publications in recent years followed by representative work.

1. Montano, F., Ould-Bachir, T. & David, J.P. (2017). An Evaluation of a High-Level Synthesis Approach to the FPGA-based Sub-microsecond Real-Time Simulation of Power Converters. IEEE Transactions on Industrial Electronics. 65(1), 636-644.
2. David, J.P. (2016). Low latency and division free Gauss-Jordan solver in floating point arithmetic. Journal of Parallel and Distributed Computing, 106, 185-193.
3. Larbanet, A., Lerebours, J. & David, J.P. (2015). Detecting very large sets of referenced files at 40/100 GbE, especially MP4 files. Digital Investigation, 14(suppl. 1), S85-S94.
4. Courbariaux, M., Bengio, Y., & David J.P. (2015) Binaryconnect: Training deep neural networks with binary weights during propagations - Advances in Neural Information Processing Systems, 2015 (+ 270 citations).
5. Daigneault, M.-A. & David, J.P. (2014). Fast description and synthesis of control-dominant circuits. Computers and Electrical Engineering, 40(4), 1199-1214.
6. Bachir, T.O., Dufour, C., Belanger, J., Mahseredjian, J. & David, J.P. (2013). A Fully Automated Reconfigurable Calculation Engine Dedicated to the Real-Time Simulation of High Switching Frequency Power Electronic Circuits. Mathematics and Computers in Simulation, 91, 167-177.
7. Ould-Bachir, T. & David, J.P. (2013). Self-alignment schemes for the implementation of addition-related floating-point operators. ACM Transactions on Reconfigurable Technology and Systems, 6(1).

Binaryconnect: Training deep neural networks
with binary weights during propagations

Deep  Neural  Networks  (DNN)  have  achieved  state-of-the-art  results  in  a  wide range of tasks,  with the best results obtained with large training sets and large models.  In the past, GPUs enabled these breakthroughs because of their greater computational speed.  In the future, faster computation at both training and test time is likely to be crucial for further progress and for consumer applications on low-power devices.  As a result, there is much interest in research and development of dedicated hardware for Deep Learning (DL). Binary weights, i.e., weights which are constrained to only two possible values (e.g. -1 or 1), would bring great benefits to specialized DL hardware by replacing many multiply-accumulate operations by simple accumulations, as multipliers are the most space and power-hungry  components  of  the  digital  implementation  of  neural  networks.   We  introduce BinaryConnect, a method which consists in training a DNN with binary weights during the forward and backward propagations, while retaining precision of  the  stored  weights  in  which  gradients  are  accumulated.   Like  other  dropout schemes,  we  show  that  BinaryConnect  acts  as  regularizer  and  we  obtain  near state-of-the-art results with BinaryConnect on the permutation-invariant MNIST, CIFAR-10 and SVHN.

The impact of such a method on specialized hardware implementations of deep networks could be major, by removing the need for about 2/3 of the multiplications, and thus potentially allowing to speed-up by a factor of 3 at training time.

Test error rates of DNNs trained on the MNIST (no convolution and no unsupervised pretraining), CIFAR-10 (no data augmentation) and SVHN, depending on the method are shown below. We see that in spite of using only a single bit per weight during propagation, performance is not worse than ordinary (no regularizer) DNNs, it is actually better, especially with the stochastic version, suggesting that BinaryConnect acts as a regularizer.