To discern the hands, you don’t need to pay attention to the pendulum. Scale separation means that even if something is very “big” (extends a long way in, say, one or two dimensions), we can still start from small patches and “work our way up.” For example, take a cuckoo clock. Scale separationĪfter symmetry, another important geometric prior is scale separation. We’ll see why this matters when we cross the bridge from the domain (grids, sets, etc.) to the learning algorithm. (In this case, though not necessarily in general, the results are the same.) Transformations can be undone: If first I rotate, in some direction, by five degrees, I can then rotate in the opposite one, also by five degrees, and end up in the original position. Transformations are composable I can rotate the digit 3 by thirty degrees, then move it to the left by five units I could also do things the other way around. The next question then is: What are possible transformations? Translation we already mentioned on images, rotation or flipping are others. The other means that we have to transform that thing as well. One means that when we transform an object, the thing we’re interested in stays the same. So here we have two forms of symmetry: invariance and equivariance. If I move to the left, my location moves to the left. (Or: translation-invariant.) But say the property is location. If I move a few steps to the left, I’m still myself: The essence of being “myself” is shift- invariant. Say the property is some “essence,” or identity - what object something is. The appropriate meaning of “unchanged” depends on what sort of property we’re talking about. SymmetryĪ symmetry, in physics and mathematics, is a transformation that leaves some property of an object unchanged. In the GDL framework, two all-important geometric priors are symmetry and scale separation. Or graphs: The domain consists of collections of nodes and edges. A generic prior could come about in different ways a geometric prior, as defined by the GDL group, arises, originally, from the underlying domain of the task. Geometric priorsĪ prior, in the context of machine learning, is a constraint imposed on the learning task. īefore we get started though, let me mention the primary source for this text: Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges ( Bronstein et al. The goal of this post is to provide a high-level introduction. Finally, the rest of us, as well: Even understood at a purely conceptual level, the framework offers an exciting, inspiring view on DL architectures that – I think – is worth getting to know about as an end in itself. Secondly, everyone interested in the mathematical constructions themselves - this probably goes without saying. Who, then, should be interested in this? Researchers, for sure to them, the framework may well prove highly inspirational. Prima facie, this is a scientific endeavor: They take existing architectures and practices and show where these fit into the “DL blueprint.” DL research being all but confined to the ivory tower, though, it’s fair to assume that this is not all: From those mathematical foundations, it should be possible to derive new architectures, new techniques to fit a given task. Geometric deep learning (henceforth: GDL) is what a group of researchers, including Michael Bronstein, Joan Bruna, Taco Cohen, and Petar Velicković, call their attempt to build a framework that places deep learning (DL) on a solid mathematical basis. Geometric deep learning: An attempt at unification As for the “elucidate,” this characterization is meant to lead on to the topic of this post: the program of geometric deep learning. Interesting examples exist in several sciences, and I certainly hope to be able to showcase a few of these, on this blog at a later time. By “complement or replace,” I’m alluding to attempts to incorporate domain-specific knowledge into the training process. In this situation, one may feel grateful for approaches that aim to elucidate, complement, or replace some of the magic. Moreover, level of generality often is low. But theory and practice are strangely dissociated: If a technique does turn out to be helpful in practice, doubts may still arise to whether that is, in fact, due to the purported mechanism. Sure, papers abound that strive to mathematically prove why, for specific solutions, in specific contexts, this or that technique will yield better results. Magic, sometimes, in that it even works (or not). More fundamentally yet, magic in the impacts of architectural decisions. Magic in how hyper-parameter choices affect performance, for example. To the practitioner, it may often seem that with deep learning, there is a lot of magic involved. Geometric deep learning: An attempt at unification.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |