In computer vision at least, deep learning has been a revolution. More than half of what I knew in the field became obsolete almost overnight (it took about a year or two I would say) and a lot of tasks received an immediate boost in term of performances.
Yes, neural networks have been here for a while, gradually improving, but they were simply non-existent in many fields where they are now the favored solution.
There WAS a big fundamental paradigm shift in algorithmic. Many people argue that it should not be called "neural networks" but rather "differentiable functions networks". DL is not your dad's neural network, even if it looks superficially similar.
The shift is that now, if you can express your problem in terms of minimization of a continuous function, there is a new whole zoo of generic algorithms that are likely to perform well and that may benefit from throwing more CPU resources.
Sure it uses transistors in the end, but revolutions do not necessarily mean a shift in hardware technology. And, by the way, if we one day switch from transistors to things like opto-thingies, if it brings a measely 10x boost on performances, it won't be on par with the DL revolution we are witnessing.
It is indeed pretty amazing in CV. In a field nearly as old as CS itself, I'd say at least 75% of the existing techniques were made obsolete in a span of only a few years.
You could have started your PhD in 2006, made professor in 2012, and nearly everything you had learned would have been _completely_ different.
I got my diploma in 2003. Actually I got lucky that I found a client who needed a computer vision specialist to add fancy features to their DL framework so I could train myself in this new direction
Yea .. if 10x compute made all the difference we would see elastic cloud compute doing things 10x as well ;) As always these new application work in tandem of hardware and software advances; seems odd to point to one or the other.
Can you elaborate on what in CV became obsolete overnight? I took a survey course in CV but I haven't kept up. You still do facial detection, object recognization, camera calibration, image stitching the same way in 2012? Or has it changed because the processing has gotten faster and the results are near real-time?
These were at the root of many detectors. They still are for some applications but for most of them, a few layers of CNN manage to train far better and very counter-intuitive detectors.
Facial detection/recognition was based on features, this is not my specialty, I don't know if DL got better there too as their features were pretty advanced but if they are not there yet I am sure it is just a matter of time.
I can see image stitching benefiting from a deep-learned filter pass too.
Camera calibration is pretty much a solved problem by now, I don't think DL adds a lot to it.
Like I said, not everything became obsolete, but around 50% of the field was taken over my DL algorithms where, before that, hand-crafted algorithms had usually vastly superior performances.
Just to confirm for the facial recognition/ detection, modern DNN algorithms outperform the 'classic' methods that took decades of continuous improvement ...
Yes, neural networks have been here for a while, gradually improving, but they were simply non-existent in many fields where they are now the favored solution.
There WAS a big fundamental paradigm shift in algorithmic. Many people argue that it should not be called "neural networks" but rather "differentiable functions networks". DL is not your dad's neural network, even if it looks superficially similar.
The shift is that now, if you can express your problem in terms of minimization of a continuous function, there is a new whole zoo of generic algorithms that are likely to perform well and that may benefit from throwing more CPU resources.
Sure it uses transistors in the end, but revolutions do not necessarily mean a shift in hardware technology. And, by the way, if we one day switch from transistors to things like opto-thingies, if it brings a measely 10x boost on performances, it won't be on par with the DL revolution we are witnessing.