Michael Carbin
Person on 2 sites
Associate professor at MIT, where he leads the Programming Systems Group, and co-author of the lottery ticket hypothesis.
-
1 channel on ppll (opens ppll.app)
Where public figures publish, and what they are on record saying.
Updated 2018-03-09 -
5 beliefs on korrents (opens korrents.com)
-
Pruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free. (opens korrents.com)
“Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.”
-
The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start. (opens korrents.com)
“However, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance.”
-
Pruning does not merely shrink a network; it uncovers subnetworks whose original initialization was what made them trainable at all. (opens korrents.com)
“We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively.”
-
Pruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free. (opens korrents.com)