We present an efficient representation for sketch based image retrieval (SBIR) derived from a triplet loss convolutional neural network (CNN). We treat SBIR as a cross-domain modelling problem, in which a depiction invariant embedding of sketch and photo data is learned by regression over a siamese CNN architecture with half-shared weights and modified triplet loss function. Uniquely, we demonstrate the ability of our learned image descriptor to generalise beyond the categories of object present in our training data, forming a basis for general cross-category SBIR. We explore appropriate strategies for training, and for deriving a compact image descriptor from the learned representation suitable for indexing data on resource constrained e. g. mobile devices. We show the learned descriptors to outperform state of the art SBIR on the defacto standard Flickr15k dataset using a significantly more compact (56 bits per image, i. e. H 105KB total) search index than previous methods.
Bui Tu, Cooper Daniel, Collomosse John, Bell Mark, Green Alex, Sheridan John, Higgins Jez, Das Arindra, Keller Jared Robert, Thereaux Olivier (2020) Tamper-proofing Video with Hierarchical Attention
Autoencoder Hashing on Blockchain,IEEE Transactions on Multimedia
Institute of Electrical and Electronics Engineers
We present ARCHANGEL; a novel distributed
ledger based system for assuring the long-term integrity of digital video archives. First, we introduce a novel deep network architecture using a hierarchical attention autoencoder (HAAE) to compute temporal content hashes (TCHs) from minutes or hourlong audio-visual streams. Our TCHs are sensitive to accidental or malicious content modification (tampering). The focus of our
self-supervised HAAE is to guard against content modification such as frame truncation or corruption but ensure invariance against format shift (i.e. codec change). This is necessary due to the curatorial requirement for archives to format shift video
over time to ensure future accessibility. Second, we describe how
the TCHs (and the models used to derive them) are secured
via a proof-of-authority blockchain distributed across multiple independent archives.We report on the efficacy of ARCHANGEL within the context of a trial deployment in which the national
government archives of the United Kingdom, United States of
America, Estonia, Australia and Norway participated.