The Application of Manufacturing Informatics to Bioprocess Yield Improvement

Society for Industrial Microbiology News. March/April 1999
By Justin O. Neway, Ph.D.

Bioprocesses are as old as bread and wine. They have been developed over centuries to provide some of life's daily necessities. Nevertheless, the past few decades have seen a dramatic increase in the number of beneficial molecules and materials made by cultivating living cells. The commercial production of many useful pharmaceuticals, antibiotics, food and feed supplements, specialty chemicals, and fuels is now based on bioprocesses. Recent economic pressures have intensified the need to improve the productivity and quality of this important type of manufacturing process. As new technologies become available with the potential to improve process economics, it is important to assess their applicability to bioprocesses.

There are several ways to make improvements to bioprocesses. One of these is through careful examination of the data gathered during process development and manufacturing. With the right software system, informative trends and patterns can be found in the often complex interactions between the process parameters and the process outcomes. Seeing the hidden relationships can help direct needed improvements in process economics and product quality through better process control (Montague, 1998). But the technologies used to find these relationships must not be a burden to us as users. We would like to have easy access to our process data in several different databases and to the necessary analysis and visualization capabilities at the same time. Fortunately, there has been considerable improvement in recent years in the way software systems are designed and built to make them more capable and easier to use (Brooks, 1995).

In this article I will discuss a relatively new idea called "Manufacturing Informatics." I am introducing this concept here not for the sake of launching more buzzwords. What I would like to do in the rest of this article is outline a group of technologies that together are becoming a vital part of developing, operating and improving manufacturing processes. My intention is to show the utility of Manufacturing Informatics in bioprocesses and to discuss some of the issues and opportunities involved in applying it to improving bioprocess yields and product quality. I also want to head off the possibility that Bioinformatics will so dominate our thinking that we forget some of the other important applications of Informatics.

Three dimensional visualizations of bioprocess manufacturing and process development data can reveal patterns not seen in other ways. The initial dissolved oxygen uptake rates for the twenty fermentations vary in a systematic way between batches that is not evident from a two dimensional display of the same data. With point-and-click profiling tools, users can dissect such a surface to yield quantitative information. This allows statistical analysis to identify ways of improving bioprocess yield or quality. The feature article in this issue provides additional detail on Manufacturing Informatics technologies.

Bioprocess Yield Improvement
Economic pressures are impacting bioprocesses in several ways, particularly in the pharmaceutical industry. Some older pharmaceuticals are coming off patent and competition from generics is squeezing margins. The increasing cost of getting new pharmaceuticals to market is forcing drug makers to look for new ways to shorten development times. The sheer volume of material needed for some new pharmaceuticals (e.g. monoclonal antibodies) is getting larger than in the past. The high cost of building new manufacturing facilities continues to rise, driving the need to increase the productivity of existing plant and equipment.

A recently published analysis of bioprocess economics concluded that the cost of manufacturing recombinant-tissue plasminogen activator (rtPA) is 48% of its selling price. The pharmaceutical industry average gross margin (i.e. profit/revenue X 100) is 30%. This means that the total cost of goods sold, including manufacturing cost, averages 70% of revenue (Datar et al. 1993). For bioprocesses that produce pharmaceuticals, the fraction of product revenue attributable to manufacturing costs probably ranges from 25% to 50% depending on the process. Non-pharmaceutical bioprocesses that produce less expensive products such as food supplements and specialty chemicals are also under constant pressure to increase margins to stay competitive. Reducing manufacturing costs by increasing process yields is one of the most obvious ways of increasing operating margins. This can be done through reduced failure rates by meeting quality specifications more frequently, and by increasing the amount of final material made with each batch.

One of the most important goals of process development is to create a process that is sufficiently stable for commercial operation and, in the case of pharmaceuticals, regulatory approval. Once commercial operations have begun, the process development groups normally continue to support the manufacturing operation. They provide experience, brain power and other resources for training the technical services group and manufacturing operators, trouble-shooting the process, making process improvements, and assisting in regulatory audits. All these groups have a real need for the capabilities that a well designed Manufacturing Informatics software system can provide.

Some of the variability in process yields can be attributed to variability in the sampling and measurement techniques themselves. But some can also represent yield improvement opportunities if the process can be biased to give the higher yields more of the time. In effect, yield variability can mean that the process, as operated in "real life", has the capability to produce more product for some portion of the time when operated under a specific combination of approved conditions. Manufacturing processes tend to become less variable over time because the operating procedures are better defined, and the most critical process parameters are better controlled. One might therefore expect that commercial manufacturing processes would present little opportunity for yield improvement by favorably biasing the process yields because they show little variability in yield. This is not necessarily so.

I recently conducted a survey to take a look at the opportunities for yield improvement in commercial processes. Manufacturing and related professionals were asked to complete a simple questionnaire that asked "What is your professional assessment of the usual variability, as measured by the relative standard deviation (RSD), that you would expect from commercial processes?" They were also asked to report their experience and formal background. The 79 professionals who responded had corporate affiliations that included Abbott, Amgen, Baxter, Bristol Myers Squib, Cytec, Genentech, Lilly, Merck, Pfizer and SmithKline, as well as several unidentified companies and research institutions. The number of respondents was perhaps small compared to the total employed in these capacities in the industry, but the group had a representative composition. Eighty percent of the respondents had industrial experience in Process Development, Research or Manufacturing. Ninety percent of them had training in Engineering, Biochemistry or Biology. Their responses therefore provide a useful indication of actual commercial process variability.

The reported mean variability in yield for the five types of processes listed in the survey was as follows: The synthetic yield of a chemical process, RSD=15%; the fermentation yield of an antibiotic producing process, RSD=17%; the fermentation yield of a protein accumulating in inclusion bodies, RSD=23%; and the fermentation yield of a process accumulating an intracellular protein product, RSD=23%. Evidently, there is indeed some opportunity for yield improvement even in commercial chemical and antibiotic processes, and substantially greater opportunities for yield improvement in commercial bioprocesses producing pharmaceutical proteins. As bioprocesses would be expected to have the highest manufacturing costs, the economic gains are potentially large from reducing variability and biasing bioprocess yields to the high side of their range.

What is Manufacturing Informatics?
Manufacturing Informatics is the technology for finding and communicating informative relationships in the data gathered from process development and manufacturing operations. The heart of all Informatics is pattern recognition and Manufacturing Informatics is no different. In this case the patterns are relationships between the measured process variables and the process outcomes during actual operation. The goal is to produce analysis results that direct actions for better process control. This reduces process variability and allows process yields to be biased to the high side of their range. Manufacturing Informatics can also be used to reverse adverse process drift, improve product quality, and demonstrate process stability and regulatory compliance. All of these results help to speed new drugs to market and reduce their production cost by minimizing specification failures, assisting technology transfer and scale-up, and stabilizing the manufacturing process.

To better understand Manufacturing Informatics it is important to distinguish between data and information. The difference is that data are simply the raw facts without a context for interpretation. It is only when we have a context for interpretation that data become information. Information is useful because it allows us to take action and, by seeing the outcomes of our action, accumulate knowledge.

If we know, for example, that the temperature reading at a certain time and point in our process is 37.4°C, we have data. We can make no useful interpretation based on just that datum alone. However, if we also know that the temperature is rising, then we have information - we have a context for interpreting the temperature. It now becomes clear that we might want to take action to avoid exceeding a specified upper control limit of 37.5°C. If the temperature were not rising then there would be no need to take action and we could redirect the resources we would otherwise have used to reduce the temperature. Deriving and illustrating a context for the interpretation of manufacturing data is one of the most important capabilities of Manufacturing Informatics.

In the simple example above, the analysis that provided the context for interpretation was to look for a pattern or trend in the historical temperature readings. A pattern emerged of increasing temperature with time. A pattern like this is relatively easy to express in everyday language, i.e. the temperature is rising. It can also be easily illustrated using a familiar two-dimensional graph to provide an illustration of the rate of temperature increase. In short, it is widely understood how such a pattern is found in the data and what it means. But real manufacturing operations are usually far more complex. The complexity is even greater when the data from many batches is examined to look for combinations of process variables that affect process outcomes. This is the kind of complexity that Manufacturing Informatics helps us to cope with.

As a batch of material moves through a manufacturing process from start to finish, the product moves from its crude form to a pure form and then finally to a commercial form. During this sequence, the upstream operations often have an affect on the downstream operations. Models of these interactions can be built to develop an overall process control strategy (Groep et al. 1999). Such models are constructed from observations made during small-scale and pilot-scale tests but can fail when applied to full-scale operations. When this occurs, mechanistic hypotheses can be built post facto and tested using small scale experiments. But this is time consuming and costly, and there is a real risk that the small scale system will be a poor representation of the full scale process. Often there is no mechanistic model that can explain the observations because our ability to make detailed observations far exceeds our ability to explain them.

When faced with this situation it is preferable to first explore the historical data from actual full scale operations to look for informative patterns. But bioprocesses can consist of ten or twenty unit operations, each with five or more recorded process variables. Manufacturing Informatics provides the means for analyzing these data to find the combined influence of a large number of process variables on the yield or quality of the final product. It can also tell us when none of the measured variables substantially influence the process outcomes necessitating additional measurements if the process is to be properly controlled. When useful relationships are found using a subset of the data they can be tested against the remaining data. If the new process parameter settings necessary for improved performance lie within their approved ranges, then they can be implemented directly under a planned deviation. This can all be done without necessarily knowing the mechanistic relationships that give rise to the observations, as long as the underlying data are reliable and the analysis results achieve statistical significance. In this way, a Manufacturing Informatics approach can be much faster than the traditional experimental route based solely on building a mechanistic rationale. Ideally it can also be complementary to a mechanistic approach.

I have phrased my distinction for Manufacturing Informatics in a way that highlights its two most important objectives-deriving useful information about how the process really works, and communicating it to others. To achieve these objectives, four closely integrated capabilities are needed. These are data access, conditioning, analysis and visualization. The first and last of these, and the integration of all of them, are the most poorly addressed areas of Manufacturing Informatics today. Before going further, let's first take a closer look at bioprocess data itself.

The Nature of Bioprocess Data
Bioprocess data consists largely of the records of discrete and continuous measurements made throughout the course of process development and manufacturing runs. A file of all the raw data gathered from the operation of a highly instrumented bioprocessing facility for an entire year would be relatively compact, probably occupying less than two Gigabytes of memory. The data typically include two simultaneously measured components: a date or time (often with an associated batch number), and a record of some value or other datum at that instant. The arsenal of statistical methodologies developed for analysis of these kinds of data in the social sciences, chemometrics and signal processing is by now quite sophisticated (Brereton, 1992; Draper et al. 1998; Kil et al. 1996; Kurtz, 1998; Malinowski, 1991; Van de Geer, 1993). Interestingly, these methodologies are underutilized in their application to manufacturing processes.

The discrete data commonly recorded in bioprocesses is both numerical and categorical in nature. Examples of discrete numerical data might be single point measurements of the pH, temperature, color or conductivity of a pooled process stream or ingredient; the amounts of raw materials used; or the products and contaminants produced. Examples of discrete categorical data might be the batch number and date of a particular lot of material; the name of an operator; the reference number of an operating procedure; or the keywords that describe the name of an ingredient or product, an engineering change, or the particular method in use. Continuous data is numerical in nature, and is sampled fairly frequently over a period of time (hours, days, weeks, or even months depending on the process). Examples might be the time-based profiles of dissolved oxygen, pH, temperature, pressure, color, conductivity, nutrient feed rates, or exhaust gasses, etc. for a fermentation, or for any process stream. In actuality, these are discrete data values recorded at a high frequency but their relative abundance creates special analysis problems. Bioprocesses accumulate a much higher proportion of continuous data than do most other types of manufacturing processes. This is because of the relatively fine control needed to cultivate living cells and the number of sensors and control loops needed to do this continuously.

Access: Getting at the Data

Bioprocesses today actually produce two products: the manufactured material itself, and the associated data. Hardware systems for gathering and storing raw bioprocess data have become increasingly less expensive and more widespread in recent years. Process development and manufacturing equipment such as fermentors and mixing tanks now routinely come equipped with their own instrumentation, control systems, and database capabilities. Because of increasingly lower costs, these systems now gather and store large amounts of raw data on a routine basis with every batch of product made.

Supervisory Control and Data Acquisition (SCADA) or DCS (Distributed Control System) software is used to monitor instruments and to control process parameters at their setpoints. They also capture data in process data historians. After a batch has been produced, the associated data can usually be found not only in the process data historians but also in several other locations. The Quality group has a Laboratory Information Management (LIMS) database to store the data on how well the intermediate and final product met specifications. Product performance in the field is also important and these data are often stored in Adverse Event Management System (AEMS) and other similar databases. Data on the raw materials, reagents and lot numbers used in a batch are stored in the Manufacturing Execution System (MES) or the Enterprise Resources Planning (ERP) system. Other departments often have their own specialized databases using Oracle, Sybase, Access, Excel and other database software. There are also paper records to be dealt with for some of the process data until the transition to Electronic Batch Data Systems (EBDS) is complete. In fact, it is not unusual to find five or more islands of production data relating to a single product batch scattered throughout the organization.

All the manufacturing-related databases are usually connected to the corporate network. This means that the right middleware solution integrating the user interface with the corporate network can allow point-and-click access to individual databases and to subsets of the data within them. A traditional approach that assembles all the data from separate databases and abstracts it into a single data warehouse can be very costly. It is also cumbersome to implement, maintain and use because of the complexity of synchronization issues. In the pharmaceutical industry, such a solution can also be in conflict with FDA mandated change control requirements and constraints on the introduction and propagation of errors.

Middleware systems are one of the most rapidly expanding areas of recent software development efforts. This is the software functionality that connects data sources, destinations and applications (Baer, 1998). Smart middleware can also provide data conditioning and tight integration between the user interface, the databases, and the analysis and visualization capabilities needed for Manufacturing Informatics. It makes possible very large time savings compared with writing SQL queries and other software programs by hand to export, join and format portions of the data in different databases, and to import it into various applications.

Conditioning: Dealing with Bad Data
Once the manufacturing data are deposited in their respective databases, they remain largely unaltered because of the change control protection required by the FDA. This is how the public is protected from the dangers associated with changing the data and potentially introducing errors. For Manufacturing Informatics, this is a double-edged sword. On the one hand, users have a high assurance that the historical data will not change between the first time an analysis is done and when it is repeated at some future date. On the other hand, all the errors that may have been present in the original data from both equipment malfunctions and human data entry tend to remain there. Therefore, an essential requirement for Manufacturing Informatics is to provide easy ways for users to condition the data to minimize the impact of bad data.

There are essentially four kinds of "bad" data: missing data from failure to capture particular values; outliers arising from electronic or other random fluctuations; erroneous data arising from malfunctioning instruments or data-handling errors; and corrupted data arising from other error producing processes. A useful Manufacturing Informatics system will incorporate the ability to easily select amongst several methodologies for minimizing the effects of these kinds of errors. Missing data can be replaced with null values that will be ignored by the analysis method, by values that have similar variability to the remaining data in the set, or by values generated by a selected interpolation function. Outliers can be found and removed for the purposes of calculation by the use of formal outlier rules or visual methods. They can also be replaced by the same methods as those used to handle missing data. Erroneous and other corrupted data are less easy to find unless they show up as outliers. Their identification will depend more on the judgement of users. In each case, tracking methods are needed that let users know what data has been conditioned and how. Users must be able to assess the effect on their analyses of the chosen conditioning method. They must also be able to repeat the data conditioning for future analyses without writing the altered data back to the original databases. The original databases must remain pristine.

Data compression is often applied to continuous data. There are several methods used to compress data before depositing it in databases so that the amount of storage space required is minimized. An example might be where the value of a continuously measured process variable remains unchanged over a period of time. In this case, space can be saved by recording no further readings until the new reading is different from its predecessor by some threshold amount. As a result, an additional capability needed for data conditioning is to use the reverse protocol to decompress the data so that it is ready for analysis. There are additional more sophisticated data conditioning capabilities that would be useful to have in a well designed Manufacturing Informatics software system, but these are outside the scope of this article.

Analysis: Finding the Gold
The types of statistical analysis used to find informative patterns in the data range from simple summary statistics and regression analysis to more complex pattern recognition methodologies. They encompass traditional statistical process control (SPC) capabilities as well as linear modeling (e.g. ANOVA, MANOVA, etc), numerical analysis (e.g. parameterization, optimization, function fitting, etc.), and latent variable methods (e.g. cluster analysis, principal component analysis, partial lease squares analysis, etc.). Latent variable methods are extremely useful for condensing a large number of process parameters into a few key components whose performance has the greatest effect on process outcomes. All of these capabilities are available in modern statistical analysis software packages but they can often require command line programming. A useful Manufacturing Informatics system must make all these capabilities available to users in appropriate work flows operated with a point-and-click interface.

Work flows are one of the most important characteristics of a well designed Manufacturing Informatics user interface. They make the difference between a generic system that forces users to learn the details of the embedded statistics and graphic engines, compared to a truly user-friendly system. The latter makes available the most common analysis methodologies in a sequence of logical steps based on a built-in understanding of what is typical for bioprocess users. Without this, users spend a large portion of their time figuring out how to use the system and doing manual programming rather than on analysis and interpretation of the results. Furthermore, they lose the power that comes from facile interactivity between the data and the analysis and visualization capabilities. This is rather like the difference between a modern long distance phone conversation and an old fashioned trunk call. Today we simply pick up the phone at any time, dial a number and speak with a colleague in England for a cost of few cents a minute. This replaces the much less productive process of placing a trunk call with an operator and then waiting until a scheduled time for a brief, noisy and costly interchange. With the old way of doing things, inertial and logistical barriers significantly interfere with interactivity and the best that the technology has to offer.

Visualization: Seeing is Believing
It is still true that a picture is worth a thousand words - it is important to be able to "see" the results of an analysis (Tufte, 1998). This is also what allows the results to be understood and acted upon by those whose strengths lie in areas other than statistics. Traditional ways of displaying results rely on tables and two dimensional graphs. With the widespread availability of color displays and printers an additional visual "dimension" has become available. Colors can be used to point out significant features in a display of analysis results. Now that high-capability graphics software is available, innovation is underway in the development of new multi-dimensional display methods. Once again, command line programming is an obstacle, and these capabilities must instead be made available through appropriate work flows with a point-and-click interface. New techniques are also needed for illustrating the results of some of the more sophisticated data analysis methodologies. This is one area where Manufacturing Informatics can perhaps borrow from Bioinformatics. Some of the new, more innovative display methods developed for looking at genomics information could be adapted to these new uses.

Seeing the overall performance of a batch, or group of batches, in a single image is a very useful capability. This can be accomplished using a new technology called Visual Process Signatures. These are graphical representations of the relationships between key process parameters and process outcomes. They go further than the batch reports which are often used for routine manufacturing process status reporting. Instead of just showing tables of numbers and panels of graphs, these images can illustrate multiple relationships in the data in a single, informative view that can also be animated. They also provide the possibility of mathematically combining the Visual Process Signatures of different groups of batches to derive a single image that shows the key differences between, for example, the top 10% and the bottom 10% of batches as determined by a specific outcome parameter like yield.

The historical performance of a bioprocess can also be shown as a rolling average Visual Process Signature for a subset of all the batches in a manufacturing campaign. This can be done for the complete set of process variables, or for a selected group of process variables of most interest such as those with the highest correlation to the process outcome. These images can be presented as animations by showing the image of each successive group as a frame in a series of images displayed in sequence. This is a good way to make a dynamic comparison between one group of batches (a frame) and another. The same animation method can be used to show the pattern of process variables associated with the progression from poor process outcomes to good ones. These are much more powerful ways of communicating Manufacturing Informatics analysis results than the usual graphs and tables.

The Ideal Manufacturing Informatics Software System
Much of what is needed for a good Manufacturing Informatics Software System is still on the wish list. In the foregoing sections I outlined the basic problems that must be overcome to provide the necessary data access, conditioning, analysis and visualization capabilities. The right software system must make useful analysis routines available in work flows with a point-and-click interface so that they can be used more easily by personnel with minimal training in statistics or computer science. Appropriate statistical, pattern recognition, visualization and other capabilities must also be provided without the need for command line programming. Data visualization capabilities must display results in informative images allowing visual recognition of complex patterns and better communication of the analysis results.

Today, there is no one software system that addresses all of these needs. Instead, individual, stand-alone applications are being used to perform portions of these tasks with a very high cost from inefficiencies due to lack of integration and the need for programming by users. The ease of use issues in general must be addressed by the integration of all the needed capabilities in a single, easy-to-use, enterprise-wide system. This is illustrated in Figure 1.

A well designed Manufacturing Informatics Software System (MISS) operates on an enterprise-wide basis to allow multiple concurrent users interactive, point-and-click access to all the manufacturing-related databases on the corporate network. This makes it available to a wide range of users in today's cross-functional teams which can include operators, supervisors, scientists, engineers, statisticians and managers with training in different disciplines. The system should also accommodates data entry from paper records when needed.

The Future of Manufacturing Informatics
Once an easily usable software platform is available with the necessary capabilities for data access and basic Manufacturing Informatics, new statistical techniques and visualization methods can be deployed. There is a growing body of literature describing new methodologies that can be usefully applied to bioprocess manufacturing. They are often more "academic" than "commercial" in nature. The best of these are needed in a rugged, fully tested and documented form that is supported on an on-going basis.

One recent report described a way of using principal component analysis to reduce the number of process parameters control charted for statistical process control purposes (Montague et al. 1998). Looking through a large number of control charts with essentially unknown relationships to one another can be baffling to process development, manufacturing and quality assurance personnel. Instead, the data can be reduced to a few principal components and these can be charted in a much smaller number of more easily understood graphs and other images.

Another recent report describes an adaptation of signal processing techniques that can be used to extract features from strip charts of continuous bioprocess data (Stephanopoulos et al. 1997). Derived components representing these features can then be monitored using control charts and other SPC methodologies. Since bioprocesses generate such large amounts of continuous data, this is an important means of monitoring the historical performance of continuous variables that is not available by using the strip charts alone. The key features can also be extracted from continuous bioprocess data by means of curve fitting and other parameterization techniques. Regardless of the means used to derive them, these features can be used for parametric statistical analyses in combination with the other discrete process variables. This greatly extends the possibilities for improved process control through retrospective bioprocess data analysis.

Conclusion
Manufacturing Informatics is the technology for finding and communicating informative relationships in the data from process development and manufacturing operations. In practical terms, the value of bioprocess data can only be realized if users can have access to all of it interactively - process parameters, derived quantities as well as process outcomes. Manufacturing Informatics capabilities must be sufficiently easy to use so that users can focus on analysis and interpretation rather than on moving data around, formatting it, and programming the analyses. The objective is to obtain information that can be used for informed decision-making by groups that include manufacturing and process development technicians, scientists and managers. At present, there is no single, well integrated data access, analysis and visualization software systems easily usable by a wide group of scientific professionals. This hampers our ability to get useful information out of our increasingly vast databases. Important opportunities are being missed to speed new drugs to market and to improve the economics, stability and product quality of bioprocesses.

With easy access to the relevant data, a large number of useful statistical methods can be interactively applied to bioprocess data analysis. Many of these have been developed for the social sciences, chemometrics and signal processing and are underutilized in Manufacturing Informatics applications. To effectively communicate the analysis results, Visual Process Signatures can be used to show the characteristics of groups of batches in a single informative image or in animated images. Additional innovation is needed to better utilize the capabilities of modern, multidimensional graphics software to display and communicate analysis results. Some of the display capabilities recently developed for Bioinformatics might be productively borrowed for use with bioprocess data.

Real-Life Application
The Manufacturing Sciences group at a pharmaceutical company is asked to improve by 15% the yield of one of the company's newer products. This must be done within three months to meet production quotas. There is no capacity available in the pilot or the production facilities for direct testing of process improvement ideas. The manufacturing schedule cannot be interrupted without additional production setbacks. They decide on a Manufacturing Informatics approach.

• Trend graphs are generated showing the performance of final yield and the yield across each unit operation over the past few months .
• There are several examples of final yield that exceed the average by a substantial amount.
• A multivariate regression analysis of all the process parameters with process yield as the dependent variable shows that the yield of the unit operation with the highest variability is tightly correlated with final product yield. This is true only when the pH of the next step is low and the temperature of the final step is high.
• The yield of the most variable step in the process has a strong negative correlation with time spent in the holding tank preceding that step.
• By talking with the manufacturing operators they find that this hold time can be shortened by implementing a second transfer line which they proceed to do.
• The correlations are so good that they are able to correctly predict the final yield for the next three batches using the process parameter data as it comes in.
• They run the batches that follow under a planned deviation protocol using the conditions they predict will give the highest and least variable product yield. The planned deviation is successful.
• Because several acceptable batches had previously been run with this particular combination of parameters, the higher yield product from the planned deviations is expected to be acceptable for clinical use provided it meets final specifications.

References:

Baer, T. 1998. Enterprise Architecture. Just another pretty interface: II. Manufacturing Systems, p24

Brereton, R.G. 1992. Multivariate Pattern Recognition in Chemometrics, Illustrated by Case Studies (Data Handling in Science and Technology, Vol 9). Elsevier Science, Amsterdam.

Brooks, F.P. 1995. The Mythical Man-Month: Essays on Software Engineering. Anniversary Edition. Addison-Wesley, Reading Massachusetts.

Datar, R.V., T. Cartwright, and C.-G. Rosen . 1993. Process Economics of Animal Cell and Bacterial Fermentations: A Case Study Analysis of Tissue Plasminogen Activator. Bio/Technology, 11:349-357.

Draper, N.R. and H. Smith . 1998. Applied Regression Analysis. John Wiley & Sons, Inc., New York.

Groep, M.E., M.E. Gregory, L.S. Kershenbaum, and I.D.L. Bogle . 1999. Performance Modelling and Simulation of Biochemical Sequences with Interacting Unit Operations. Biotechnol. Bioeng., (In Press)

Kil, D.H. and F.B. Shin . 1996. Pattern Recognition and Prediction With Applications to Signal Processing (AIP Series in Modern Acoustics and Signal Processing). American Institute of Physics, Washington DC.

Kurtz, N.R. 1998. Statistical Analysis for the Social Sciences. Prentice Hall, New York.

Malinowski, E.R. 1991. Factor Analysis in Chemistry. John Wiley & Sons, Inc., New York.

Montague, G.A. 1998. Monitoring and Control of Fermentors. Institution of Chemical Engineers, UK.

Montague, G.A., H.G. Hiden, and G. Kornfeld . 1998. Multivariate Statistical Monitoring Procedures for Fermentation Supervision: An Industrial Case Study. 7th Conference on Computer Applications in Biotechnology, Osaka, Japan.Stephanopoulos, G.G., G. Locher, M.J. Duff, R. Kamimura, and G.G.

Stephanopoulos . 1997. Fermentation Database Mining by Pattern Recognition. Biotechnol.Bioeng., 53:443-453.

Tufte, E.D. 1998. The Visual Display of Quantitative Information. Graphics Press, Connecticut.

Van de Geer, J.P. 1993. Multivariate Analysis of Categorical Data: Applications (Advanced Quantitative Techniques in the Social Sciences, 3). Sage Publishing, New York.