First developed as method for training rank-pooling for activity recognition in the Robots, Humans and Action Project and extended in the Learning Project, a Deep Declarative Network (DDN) allows the combination of traditional computer vision and physical models to be combined with contemporary deep learning models in a single end-to-end architecture. The key insight allowing this is the use of the implicit function theorem for differentiating the optimization problems, which is how traditional models are often expressed. While deep learning has many promises it is currently expensive, opaque, brittle and relies extensively on human labelled data. DDNs have the potential to make deep learning more reliable by establishing theory and algorithms that allow physical and mathematical models to be embedded within a deep learning framework, providing performance guarantees and interpretability.
Don’t forget to check out the CVPR 2020 workshop on DDNs , and ECCV 2020 tutorial on DDNs.
If this piques your interest, please come to our tutorial at ECCV on Friday night!
Code and JuPyter notebook tutorials are available here and a playlist of the tutorial videos are here (collaboration with Stanford and Facebook).
And to prepare for the tutorial you can also watch some of the talks from our CVPR 2020 workshop. Dylan, Liu Liu and I also have a paper on solving the blind PnP problem using DDNs.
I haven’t been involved in this project but have found the work really intriguing. Using the knowledge we already have about the world to inform how we machine learn the things we don’t know is an interesting approach to the problem that makes a lot of sense. I’m looking forward to seeing where DDNs go in the future.