H.J. Kappen , W. Wiegerinck , T. Morgan , J.R. Dorronsoro , E.
Chiozza , T.J. Harris , G. Paillet , J. Kopecz
1 Stichting Neurale Netwerken (SNN), the Netherlands
2 Augusta Technology Ltd (ATL), United Kingdom
3 Instituto de Ingeniera del Conocimiento (IIC), Spain
4 Neural Computing Applications Forum (NCAF), United Kingdom
5 Neuroptics Consulting, France
6 Zentrum f ur Neuroinformatik GmbH (ZN), Germany
1 Introduction
Interest in Artificial Neural Networks (ANNs) has been rising dramatically over the last decade. The earliest ideas in the field date back to the pioneering work of Norbert Wiener and John von Neumann in the mid-1940's. Commercial interest in the '60s was quickly broad to a hold by the realization of the limited applicability of these early neural networks. In the late '60s, the development of mathematical theorem provers and chess computers showed the strong potential of symbolic methods. During the '70s and early '80s, symbolic methods such as expert systems were the main methodology to build intelligent systems. During the mid '80s, however, it became clear that often the symbolic solutions are too 'brittle': they perform well under clearly defined conditions, but fail immediately when these conditions are only partially fulfilled. The design of expert systems turned out to be extremely complex in order guarantee good performance under all conceivable conditions. For the second time interest became focussed on neural networks. Learning from examples was soon found to be an effective way to develop systems that not only perform well under the conditions that they were trained, but also showed good generalization behavior to novel situations. The earliest commercial exploitation was by Nestor Inc. in the USA, founded by the Nobel prize-winner Leon Cooper. Other companies have followed rapidly, and a number of development tools are now sold commercially for a variety of platforms. Only in the last few years has the technology matured to the extent that companies are prepared to use it for mission-critical applications. The main advantages of ANNs stem from their ability to recognize patterns in data. This can be achieved without a priori knowledge of causal relationships, as would be necessary in knowledge-based systems. ANNs ability to generalize relationships from input patterns make them less sensitive to noisy data than other approaches. Their ability to represent non-linear relationships makes them well suited for a large variety of applications, such as some industrial control systems or financial forecasting, where linear relationships do not hold.
Fig. 1. World patents using neural networks from 1960 to 1994. 1994
data are preliminary. Source FhG-ISI Germany Although the original
inspiration for ANNs arose from neural behavior in nature, current
technology differs in having much simpler interconnections between processing
cells and much faster processing within a cell than natural systems.
The majority of ANN implementations are software simulations of parallel
computations, with tens or hundreds of `neurons' being executed in rapid
succession. The earliest neural chips are now appearing, and these are
likely to find application in particularly demanding applications,
such as image processing and real-time control.
2 SIENA
The objectives of the SIENA project were to examine the current state of commercial ANN usage across Europe and to undertake dissemination activities to excite business interest in this topic. The project received support from the European Commission under its Esprit program. However, unlike most other Esprit project, SIENA has not involved in issues of technology development. Instead, SIENA has focused on the way in which `known' technology can be deployed commercially. In some ways, the state of many of the previous Esprit projects re ect the concerns which originally prompted SIENA: high quality research but limited follow-through into real-world applications. The project was conducted over a period of 18 months by a consortium of six organizations - all SMEs - from five countries. The total project budget was 600 kECU. Half of the funding was provided by the European Commission, the remainder by the consortium members. The geographical spread of the partners allowed a reasonably broad coverage, but this was necessarily limited by the available budget. Within this over-riding constraint, we believe that we have obtained a fairly consistent picture of the current European situation. Our focus in SIENA was on working, money-making applications only. Our aim was to see what neural technology can be applied now, and to focus on these applications in our dissemination activities. In industrial laboratories and universities there exist a large number of applications and prototypes that are still in development or for other reasons have not been utilized in a commercial environment. Certainly, these research developments are important and will affect future commercial activities in neural networks. However, they were outside the scope of SIENA. We identified approximately 175 suppliers of neural network technology. Through questionnaires and surveys, we could estimate the size of the neural network market in Europe to be approximately 110-140 MECU in 1996, with an annual growth of 20-30%. We collected approximately 150 working applications in Europe, from very diverse industrial sectors. Detailed descriptions of 65 case studies were compiled and can be obtained from the SIENA Web site 7 . We identified the financial area, industrial control and marketing as the most promising areas for future applications. The results of the SIENA project are compiled in the White Paper, which will be published this fall.
3 Supplier Survey
As might be expected, the information collected on ANN suppliers and products has shown a very different picture in each of the countries surveyed. Several reasons for this situation can be identified. One is the existence of national programs in some countries, aimed to increase awareness or to foster R &D in this field. Another is, of course, the different levels of industrial maturity in the countries studied. The SIENA survey results show two groups of countries emerging with respect to suppliers. The first group is represented by countries such as the United Kingdom, the Netherlands and Germany: for the reasons mentioned above these have a relatively high supply level, both for products and consultancy. Typical countries in the second group are Spain, Belgium and Sweden: in these countries the suppliers market is slowly emerging and take-off could be expected in the near future. From the point of view of types of applications offered by the suppliers, they are somewhat fragmented. No single group of applications can be said to dominate the market, although Control, Monitoring and Modeling is certainly leading. The situation is slightly different when industry sectors are considered. Production leads here, which is in agreement with previous information, followed by some other relatively strong sectors. These are conclusions which can be drawn from the content of the SIENA suppliers data base, but they must be treated with some caution because of the characteristics of the data- gathering exercise. Apart from the limitations 7 http://www.mbfys.kun.nl/snn/siena/
Fig. 2. Application Types of budget, and the inevitable implications of clustering information under a restricted range of headings, we also encountered one other distorting factor which is worth noting. A number of suppliers consciously chose not to provide us with any information. A number expressed reservations about providing information to consortium members who are themselves supply-side companies and therefore, in some sense, competitors. More surprisingly some other companies appeared to be unable to come up with a coherent description of their company, products or services. These facts need to be borne in mind when the results are interpreted. Although considerable progress has been made, the ANN supplier picture emerging from SIENA should not be considered as the last word. A follow-on project would start from a much more adequate level and should provide more comprehensive results. Some results can be summarized comparing data on Industry sectors and Application type: { Control Monitoring and Model applications are mainly implemented in the Production industry sector. { Optimization applications are mainly implemented in the Production industry sector. { Recognition Detection and Pattern Matching as well as Image Processing and Forecasting and Prediction applications are mainly implemented in the Finance, Banking and Assurance industry sector.
Fig. 3. Industry Sector
4 Case Studies
One of the most important activities related to dissemination has been the collection of a large number of case studies of successful ANN applications. Case studies are a particularly valuable tool to encourage businesses to take up ANN technology. Cases have been collected covering a variety of ANN applications across a broad range of sectors. Each has been arranged in a standard presentation format to make it easy for potential users to locate relevant case studies and to assess the potential value of ANNs within a particular business context. We have taken care to ensure that only `real' cases are included: many have been rejected on the grounds that they were only experimental prototypes or based on marketing `hype'. In addition to general information about the application and contacts, each case study tries to answer the following questions. { What was the problem? { How did ANNs provide a solution? { What benefits were gained from the use of ANNs? { How could the solution be generalized to other problems of a similar type? The library of case studies provides an excellent starting point for future dissemination activities. Case studies have been made public on the Internet , (www.mbfys.kun.nl/snn/siena/cases).
5 Market Size
We have attempted to make an estimate of the total neural networks market in Europe. Within the SIENA budget it has been impossible to attempt an exhaustive coverage of the field. We have only been able to look at some regions of Europe, and within these regions at only a sample of the whole range of applications. Our coverage of suppliers is reasonably good, and we estimate that we have identified around 80% of commercial suppliers within our boundaries. However the number of users is very much greater than the number of suppliers, and our coverage is certainly below 10% of the total. In estimating the market size we are therefore obliged to extrapolate from the limited information we have, which reduces the accuracy of the results. Our estimates do not cover all ANN activity in Europe. Two especially prominent areas of activity are not included in our estimates: academic institutions and governmental research organizations. Although these are sometimes involved in commercial arrangements, we have chosen to omit them because it is impossible to put a value on their work in any consistent way. Our final estimates were derived from several lines of reasoning. { Suppliers were asked to give their annual turnover within one of several bands (we believed that suppliers would simply refuse to give financial in- formation if asked for too great a level of detail). From this information, and our estimate of the degree of coverage of our survey, we obtained one approximation. { We estimated the numbers of people involved in commercial ANN activities. Taken together with typical industry salary rates this gives an estimate of the labor costs incurred, which can be scaled up with overheads and profit to give a second estimate. { The SIENA partners' own involvement in the ANN market, and various discrete items of information in each region provides a third estimate. From these lines of reasoning, it was possible to produce a maximum and mini- mum estimate for each region. The individual results were then compared to the Gross Domestic Product (GDP) for the region concerned as a check on validity, and some minor adjustments made to produce a final value. A similar process was followed to produce estimates of growth in market size in the immediate future. By looking at the growth in the number of ANN suppliers and the growth in the number of products, we can see that the rate of increase in the number of suppliers is slowing, while the rate of increase in the number of products is rising. These factors, again tempered with the specialist knowledge of each SIENA partner, provide the basis for an estimate of growth. We can identify a surge of interest in ANN activity at the end of the 1980s. This led to increasing commercial activity from the early 1990s onwards, following a pattern which has also been seen in the early stages of other new technologies. It is likely that the early market is composed of two overlapping stages. Firstly there is burst of activity as a significant number of organizations (mostly large in size) try to come to grips with the new technology. Studies and trial applications are carried out to investigate the implications for the organizations future activities. Many new small companies emerge (and some also quickly disappear) during this period. It is also the period in which bizarre predictions of market size appear, based on the slope of the leading edge of the surge-of-interest curve. Once the technology becomes better known, its fields of application become clearer and a sustainable market in those fields starts to appear. Steady growth in established fields, together with the addition of new fields, provides a market impetus which overtakes the initial surge to become the `real' commercial market. At the time of transition between the two trends the market growth may appear to slow down or even fall. This is typically the time at which a `shake- out' occurs, with rationalization of the supply side through company failures or merges. A third stage occurs somewhat later when the technology has become fairly commonplace and routine. It then becomes progressively less meaningful to speak of a separate market for that particular technology, as it becomes absorbed into the mainstream as one of a number of standard techniques. This view of the market is shown diagrammatically in the following Figure. From value time total market proven applications ?surge-of-interest?
Fig. 4. the reasoning described above, we come to the following conclusions regarding the ANN market at the end of 1995: { The total ANN market size across the countries of the EU is in the range 110-140 MECU. { The annual growth rate in the ANN market is in the range 25% to 30%.
6 Concluding remarks
The SIENA project has been the first attempt in Europe to make a critical investigation of the impact of neural networks on business. We now have some quantitative knowledge about the size, maturity and growth of the market. How- ever, our surveys have been far from complete and some regions are covered better than others. In the future, we hope to extend our surveys. In addition, the SIENA results provide a good starting point for focussed dissemination activities in specific industrial sectors such as process control, financial forecasting and marketing.
7 Acknowledgement
The SIENA project (Esprit P 9811) was funded in part by the European Commission.