Validation and Verification of Embedded Neural Systems

Abstract

There is growing interest in neural networks in the industrial world and more and more safety related software involves or will involve them. Therefore a need for assessing the level of safety of a neural network software has become an essential task. The first year of this project gave an overview of neural network technology, which included in particular a description of the main principles on which it is based and an emphasis on the differences between software embedding neural network technology and “classical” software (see [D2]). This lead to a set of good practice tules to develop a successful neural network application, from which guidelines to assess a neural network system had been derived (see [D3]). The aim of this project is to explore more deeply the different means and techniques which will allow us to assess the ability of a neural network to perform a certain task. Firstly the work concentrates on the data set and the verifications that have to be carried out to ensure its quality. The main problems that have to be addressed in this context are typically the validity of the noise model, the possibly multivalued character of the mapping function, how representative of the real data the data set is and finally the links between the features of the data set and the features of the model (ie. of the neural network). From this study, the aim is to improve the set of assessment guidelines. Secondly, a shorter part of the thesis shows how to reason about a NN embedded in a safety related environment, that is how a safety case could be obtained for neural network software. Finally a set of issues and ‘suggestions for future research are provided. This document is necessarily limited in its scope. Indeed, we have chosen to restrict our study to the two main types of models currently used, namely, the radial basis function network and the multi-layer perceptron. We have excluded, for instance, unsupervised techniques and recurrent neural networks that raise notoriously difficult problems. However, whatever the model used, they are related in that they are data-driven. Consequently, issues relative to the data discussed in this thesis are central and generally applicable to each of these models.