Artificial neural networks can predict age in Greenland halibut using otolith images

Deep learning methodology can consistently forecast the age of fish from otolith imagery

A reliable estimate of the age distribution of a fish stock is important in stock assessment and stock development modeling. This concerns, for example, the detection of recruitment of young fish as well as to the monitoring of time series of the age distribution of the fish and the survival of strong year classes. A year class in this context is the part of the stock born on a certain year.

For most fish species, the most reliable method to estimate age is to count annual zones in otoliths, a three-dimensional calcified structure present in the inner ear of vertebrates. More precisely, there are three pairs of otoliths, where the largest pair, called sagitta, is used for age reading. This is done directly by microscope or by analyzing images of the otolith, either of the entire (thin) otolith or a cross-section through the core (“thick” otolith). Because the true age of wild fish is rarely known, a complicated challenge is to validate the reading method.

Analysis of otoliths is performed by trained human experts, a process that is difficult and time-consuming. The precision and bias of the age estimates vary between readers, and sometimes the same reader can estimate different ages for the same otolith when observing it at different times. Clearly, this often leads to inconsistent estimates. About 1 million otoliths are read each year around the world, so developing consistent and efficient age estimation methods is very important.

Artificial neural networks, which are effective in modeling complex relationships between high-dimensional inputs and labels in classification and regression problems, can be used for automatic age prediction. Among the different types of neural networks, convolutional neural networks (CNNs; a class of deep learning models) are suitable to process image data. The automatic analysis of otolith images using CNNs is consistent, in the sense that a trained deep-learning model would produce the same output for the same input. This is a major advantage in comparison to estimates from human readers.

This article – summarized from the original publication (Martinsen, I. et al. 2022. Age prediction by deep learning applied to Greenland halibut (Reinhardtius hippoglossoides) otolith images. PLoS ONE 17(11): e0277244) – reports on a study to analyze the performance of a CNN in predicting fish ages from 4,243 Greenland halibut images from a publicly available database and provided by the Institute of Marine Research with read ages (labels) provided by experts. For detailed information on the materials and methods used, refer to the original publication.

Results and discussion

Otoliths of Greenland halibut are visually analyzed by human experts and play an important role in determining the fish age. Otolith analysis is conducted extensively worldwide and the process is time-consuming both in terms of the analysis itself and in terms of the time used to educate the readers. Based on this research, we propose an automatic procedure based on CNNs – a class of deep learning models used in computer vision – for predicting the fish age from an image of the otolith.

The proposed procedure estimates the fish age consistently, in the sense that a trained CNN produces the same output for the same input, which is a major advantage in comparison to estimates from human readers, which are often inconsistent. The results we obtained show that the deep learning approach has great potential. The most important observation is that the age distribution of the Greenland halibut predictions matches the distribution of the ground truth labels very well.

Norway showcases halibut farm as sustainability star

Our results are very similar to the results obtained in previous studies focused on Greenland halibut, although two of these studies relied on a single test set. As we observed significant variations between the results obtained on different test set splits, the cross-validation procedures we conducted in this study demonstrated the importance of using robust techniques to validate model performances.

Length measurements are reliable and easily obtained and could be included as an additional feature to improve age predictions. Since it is known that the otolith continues to grow for the entire lifespan of the fish, it could be argued that all information provided by length measurements should be contained in the otolith images and that length is a redundant feature. Combining deep learning estimates with age estimates did improve the performance, but including the length also led to a slightly larger discrepancy between the distributions of the predicted ages and the ground labels. This is clearly undesired, as the overall age distribution is very important in stock assessment.

In many practical applications, it is important to understand the CNN decision process both to gain additional insights about the data and to check the validity of the model. For this purpose, we considered several explanation techniques for deep learning models that quantify the pixels’ relevance on the output in the form of heatmaps. While these tools can be useful in understanding the decision process of a deep learning model, they are mostly based on heuristics and the interpretation of the heatmaps is subjective.

In our case, the heatmaps were difficult to interpret, as there were few patterns that provided clear and obvious explanations behind the model output, which may be due to both the complex decision rules derived by the deep learning model and the lack of strong characterizing features in the images. A possible reason is that since different images exhibit different characteristics, there are few strong patterns common to all images of the same age group.

We identified a few interesting patterns in the resulting heatmaps. The development of fingers as the otolith grows and the relative size of the otolith core are clear indicators for fish age, both of which were emphasized by the heatmaps as stripe-like patterns around the core.

Perspectives

The successful use of a deep learning methodology to predict the age of fish from otolith images should also inspire the implementation of more appropriate imaging technology on board where the otoliths are sampled. A simple system could be a robust camera mounted where the fish samples are taken, such that the otolith could be photographed and labeled immediately. As good results are obtained with low-resolution images, the cost of such equipment would probably be acceptable and, potentially, could also be used to discriminate between different species as well as stocks.

Follow the Advocate on Twitter @GAA_Advocate

Related Posts

Health & Welfare

Nonintrusive methods for fish biomass estimation in aquaculture, part 2

Part 2 looks at acoustics, environmental DNA and resistivity counter as noninvasive methods for fish biomass estimation. 

Health & Welfare

R&D collaboration to explore using AI to count sea lice

The Danish Technological Institute and Benchmark Genetics are exploring the use of artificial intelligence (AI) to count sea lice on salmon. 

Innovation & Investment

AI platform delivers data to fish and shrimp farmers

XpertSea does more than count fish. It uses AI and computer vision to calculate growth rates and optimal harvest dates to improve aquaculture efficiency. 

Health & Welfare

Achotines laboratory home to continuing studies of tuna early life history

The Inter-American Tropical Tuna Commission Achotines Laboratory in southern Panama is the world’s only facility with nearly year-round availability of tuna eggs and larvae. A study is comparing the reproductive biology, genetics and early life history of yellowfin and Pacific bluefin tuna.