Automatic pattern analysis of bioacoustic signals: exploring shallow and deep learning frameworks (PhD)

Abstract

Sounds produced by living organisms are called bioacoustic signals. These bioacoustic signals can be analysed to track organisms like birds, amphibians and mammals in their natural habitats. This thesis presents various machine learning frameworks to automatically analyse the bioacoustic signals. One of the challenges in developing machine learning frameworks for bioacoustic pattern analysis is scarcity of labelled training data. Asaresult, there is a requirement of machine learning frameworks that can overcome this problem, and work effectively under the low-training data conditions. This thesis mainly addresses the development of such data-efficient frameworks. It also deals with the development of standard data-intensive machine learning methods for bioacoustic applications where a sufficient amount of labelled training data is available. This thesis explores the contrastive paradigms of shallow and deep learning to introduce frameworks for bioacoustic pattern analysis, in particular, bioacoustic activity detection, segmentation and classification. In shallow learning based frameworks, the concepts of dynamic kernels, semi-supervision and matrix factorization are utilised. These frameworks are demonstrated to have low training data requirements and hence, are suitable for many bioacoustic applications. On the contrary, for bioacoustic applications where enough labelled training data is readily available, deep learning frameworks are proposed to emphasize the performance. Apart from the standard deep learning methods, this thesis also explores meta-learning, in particular, deep metric learning to train large neural networks effectively in data-scarce scenarios. In this thesis,a computationally efficient variant of probabilistic sequence kernel (PSK) is proposed for the task of bioacoustic activity detection. Unlike the existing formulation of PSK, the proposed PSK does not require background modelling and utilises only a Gaussian mixture model (GMM) for bioacoustic activity class. Moreover, only a few most relevant components of this GMM are utilised for the kernel formulation, making the whole set up computationally efficient. Apart from this, an all-convolutional neural network (all-conv net) is also proposed for activity detection. This neural network consists of only convolutional layers, and utilises learned pooling or strided convolutions to down-sample the feature maps. In contrast to max-pooling, the learned pooling helps in capturing the inter-feature map correlations, leading to a better representation. Next, this thesis proposes a semi-supervised framework and a weakly supervised neural network for the task of bioacoustic signal segmentation. The proposed semi-supervised framework requires only a few strongly labelled training examples, and utilises the correlation between training examples and the test audio recordings to discriminate between the target bioacoustic events and the background. On the other hand, multi-instance learning is incorporated in the all-conv net to provide weakly supervised segmentation. Next, this thesis explores the utilisation of archetypal analysis (AA), a matrix factorization method, to model the bioacoustic data using its convex hull or extremal elements. Building on AA, a deep matrix factorization framework, referred to as deep archetypal analysis (DAA) is proposed. DAA improves the modelling capabilities of AA as it can model both extremal as well as average behaviour of the data. Both AA and DAA are employed in simplex projection based dictionary learning framework and in dynamic kernel formulations for developing bioacoustic classification frameworks. In comparison to other acoustic modelling methods, AA/DAA requires a lesser amount of data to effectively model the variations present in a class, making them appropriate for bioacoustic classification. Finally, this thesis explores deep metric learning (DML) to propose a data-efficient bioacoustic classification framework that utilises the triplet loss function with dynamically increasing margin. This dynamically varying margin allows the framework to re-use the training data without introducing redundancy in the training process. The experimental evaluation on publicly available and licensed datasets demonstrates that the proposed frameworks provide either better or comparable performance than state of-the-art bioacoustic methods.