Tarjei S. Hveem

Contact information

Please contact Tarjei S. Hveem for more information


Increasing workload for pathologists worldwide as well as variability among the experts conclusions implies a need for automated methods for more nuanced grading.

histotyping illustration


Histopathology is the study of the appearance of the resected specimen and assesses relevant properties, such as the tumour grade. Tumour grading is an evaluation of the extent to which tumour cells and tumour tissue resemble normal cells and tissue, where a high degree of similarity (well-differentiated) is associated with a better prognosis for the patient than a low degree of similarity (poorly differentiated).

The analysis is carried out in HE-stained tissue sections. Tumour grade is a useful prognostic marker, but a substantial proportion of patients are classified as moderately differentiated, i.e., an intermediate group with an intermediate outcome. The need for tools that can differentiate the large group of patients who fall in the middle layer has been great.

Increasing workload for pathologists worldwide as well as a problem with significant intra- and interobserver variability implies a need for automated methods for this task.

ICGI has developed Histotyping, a fully automated histological characterisation of HE-stained sections from cancer specimens for prognostic purposes. The method is based on deep learning by convolutional neural networks trained on images of HE-stained tissue sections. The patient outcome is used to guide the training process into a system that can identify tissue patterns in the HE-sections that are distinct for patient prognosis. The resulting computer model can be applied to a new patient’s tumour sample and estimates the probability of a poor patient outcome.

deep learning visualisation of a convolutional network

Risks and prevention

To train deep convolutional neural networks is large amounts of labelled data needed. However, neither the required datasets nor the computational resources to perform these types of analyses have existed until recent years.

Neural network models have millions of features and overfitting to the training dataset is a common problem. Overfitting can, for instance, happen when the computer model identifies and exploits artefacts in the training dataset that are associated with the desired outcome but have no biological relevance. The result in such cases is that the computer model fails when evaluated on a new dataset on which it is not trained.

To increase the probability that the neural network generalises when applied to new patients from a new dataset, we have used a robust design with thousands of patients from different patient cohorts with the same cancer type (Table 1). We have developed the framework on stage I-III colorectal cancer patients from two hospitals in Norway and two clinical trials in England.

Another risk is an adaption to technical equipment such as the imaging system, i.e. when the method works well on images scanned with the scanner on which it is developed but not on images from a scanner from another vendor. To compensate for this problem, we have scanned all images with scanners from two major scanner manufacturers (Hamamatsu and Leica).

A third risk in the development of a computerised system for risk assessment based on scanned HE-sections is the dependence on lab preparation, i.e., that the method only works on scans of tissue sections prepared and stained in the lab where the technique is developed. To evaluate this, we have scanned parallel sections from one of the patient cohorts (Gloucester) that have been prepared in the pathology routine there.

We have partitioned our patient datasets to make the best use of the data. The Kaplan-Meier plots below illustrate results based on images scanned with 10x, and 40x lens analysed in corresponding neural network models and classified according to the agreement between the two models’ classifications in a test partition that has not been included in the training process.

kaplan-meier plot histotyping

This text was last modified: 06.08.2019

Chief Editor: Prof. Håvard E. Danielsen
Copyright Oslo University Hospital. Visiting address: The Norwegian Radium Hospital, Ullernchausséen 64, Oslo. Tel: 22 78 23 20