Publications
Kissing to Find a Match: Efficient Low-Rank Permutation Representation (Arxiv Preprint, 2023)
Hannah Dröge, Zorah Lähner, Yuval Bahat, Onofre Martorell, Felix Heide, Michael Möller
Permutation matrices play a key role in matching and assignment problems across the fields, especially in computer vision and robotics. However, memory for explicitly representing permutation matrices grows quadratically with the size of the problem, prohibiting large problem instances. In this work, we propose to tackle the curse of dimensionality of large permutation matrices by approximating them using low-rank matrix factorization, followed by a nonlinearity. To this end, we rely on the Kissing number theory to infer the minimal rank required for representing a permutation matrix of a given size, which is significantly smaller than the problem size. This leads to a drastic reduction in computation and memory costs, e.g., up to 3 orders of magnitude less memory for a problem of size n=2000, represented using 8.4 × 105 elements in two small matrices instead of using a single huge matrix with 4 × 108 elements.
The proposed representation allows for accurate representations of large permutation matrices, which in turn enables handling large problems that would have been infeasible otherwise. We demonstrate the applicability and merits of the proposed approach through a series of experiments on a range of problems that involve predicting permutation matrices, from linear and quadratic assignment to shape matching problems.
Explorable Data Consistent CT Reconstruction (BMVC, 2022)
Hannah Dröge, Yuval Bahat, Felix Heide, Michael Möller
Computed Tomography (CT) is an indispensable tool for the detection and assessment of
various medical conditions. This, however, comes at the cost of the health risks entailed in the usage of
ionizing X-ray radiation.
Using sparse-view CT aims to minimize these risks, as well as to reduce scan times, by capturing
fewer X-ray projections, which correspond to fewer projection angles. However, the lack of sufficient
projections may introduce significant ambiguity when solving the ill-posed inverse CT reconstruction problem,
which may hinder the medical interpretation of the results.
We propose a method for resolving these ambiguities, by conditioning image reconstruction on different
possible semantic meanings.
We demonstrate our method on the task of identifying malignant lung nodules in chest CT. To
this end, we exploit a pre-trained malignancy classifier for producing an array of possible
reconstructions corresponding to different malignancy levels, rather than outputting a single image
corresponding to an arbitrary medical interpretation. The data-consistency of all our method reconstructions
then facilitates performing a reliable and informed diagnosis (e.g. by a medical doctor).
Non-Smooth Energy Dissipating Networks (ICIP, 2022)
Hannah Dröge, Thomas Möllenhoff, Michael Möller
Over the past decade, deep neural networks have been shown to perform extremely well on a variety of image reconstruction tasks. Such networks do, however, fail to provide guarantees about these predictions, making them difficult to use in safety-critical applications. Recent works addressed this problem by combining model- and learning-based approaches, e.g., by forcing networks to iteratively minimize a model-based cost function via the prediction of suitable descent directions. While previous approaches were limited to continuously differentiable cost functions, this paper discusses a way to remove the restriction of differentiability. We propose to use the Moreau-Yosida regularization of such costs to make the framework of energy dissipating networks applicable. We demonstrate our framework on two exemplary applications, i.e., safeguarding energy dissipating denoising networks to the expected distribution of the noise as well as enforcing binary constraints on bar-code deblurring networks to improve their respective performances.
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization (MDPI, 2021)
Hannah Dröge, Baichuan Yuan, Rafael Llerena, Jesse T. Yen, Michael Möller, Andrea L. Bertozzi
Analyzing and understanding the movement of the mitral valve is of vital importance in cardiology, as the treatment and prevention of several serious heart diseases depend on it. Unfortu- nately, large amounts of noise as well as a highly varying image quality make the automatic tracking and segmentation of the mitral valve in two-dimensional echocardiographic videos challenging. In this paper, we present a fully automatic and unsupervised method for segmentation of the mitral valve in two-dimensional echocardiographic videos, independently of the echocardiographic view. We propose a bias-free variant of the robust non-negative matrix factorization (RNMF) along with a window-based localization approach, that is able to identify the mitral valve in several challenging situations. We improve the average f1-score on our dataset of 10 echocardiographic videos by 0.18 to a f1-score of 0.56.
Learning Or Modelling? An Analysis Of Single Image Segmentation
Based On Scribble Information (ICIP, 2021)
Hannah Dröge, Michael Möller
Single image segmentation based on scribbles is an important technique in several applications, e.g. for image editing software. In this paper, we investigate the scope of single image segmentation solely given the image and scribble information using both convolutional neural networks as well as classical model-based methods, and present three main findings: 1) Despite the success of deep learning in the semantic analysis of images, networks fail to outperform model-based approaches in the case of learning on a single image only. Even using a pretrained network for transfer learning does not yield faithful segmentations. 2) The best way to utilize an annotated data set is by exploiting a model-based approach that combines semantic features of a pretrained network with the RGB information, and 3) allowing the networks prediction to change spatially and additionally enforce this variation to be smooth via a gradient-based regularization term on the loss (double backpropagation) is the most successful strategy for pure single image learning-based segmentation.
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Inverting Gradients - How easy is it to break privacy in federated learning? (NeurIPS, 2020)
Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, Michael Möller
The idea of federated learning is to collaboratively train a neural network on a server. Each user receives the current weights of the network and in turns sends parameter updates (gradients) based on local data. This protocol has been designed not only to train neural networks data-efficiently, but also to provide privacy benefits for users, as their input data remains on device and only parameter gradients are shared. But how secure is sharing parameter gradients? Previous attacks have provided a false sense of security, by succeeding only in contrived settings - even for a single image. However, by exploiting a magnitude-invariant loss along with optimization strategies based on adversarial attacks, we show that is is actually possible to faithfully reconstruct images at high resolution from the knowledge of their parameter gradients, and demonstrate that such a break of privacy is possible even for trained deep networks. We analyze the effects of architecture as well as parameters on the difficulty of reconstructing an input image and prove that any input to a fully connected layer can be reconstructed analytically independent of the remaining architecture. Finally we discuss settings encountered in practice and show that even aggregating gradients over several iterations or several images does not guarantee the user’s privacy in federated learning applications.
Implementations
Python Implementation of SVCD (2022)
Markus Plack, Hannah Dröge
An unofficial Python implementation of the paper Spatially Varying Color Distributions for Interactive Multi-Label Segmentation by Claudia Nieuwenhuis and Daniel Cremers.
Pytorch Realized Radon Transformation (2022)
Digitally postprocessed images of "The Lung Image Database Consortium (LIDC) and
Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans."
by Armato SG 3rd et al. [1]
An open source implementation of radon transformation and filtered backprojection realized in pytorch for reconstruction for parallel beam projection of computer tomographic recordings. The operations are differentiable and can thus can be used for training via backprojection.