Design Evaluation : Decomposition and State-space Analysis

This paper outlines the increasing demands upon evaluation activity during the engineering design process. In particular, the need to address large numbers of innovative concept options during the conceptual design phase is stressed and a six-step methodology proposed. This methodology combines and integrates techniques of inexact reasoning with the need to combine two basic human approaches to evaluation, namely decomposition and holistic. The holistic evaluation elements comprise fuzzy estimates of probability of achieving Pareto optimal status combined with state-space analysis. An example demonstrates how these methods may converge to provide an appropriate support for human evaluation of emerging designs. It is concluded that the six-step methodology exhibits validity and time reduction in terms of providing an aid to the evaluation of a large number of merging designs and their associated design characteristics.


I Introduction
The ability to rapidly evaluate design ideas is an essential element in the goal to increase design productivity.Given the need for companies to produce more innovative products in an increasingly competitive market place it follows that designers have to consider an increased number of design options if the most appropriate design is to be pursued through the product development process.Only through the generation of a relatively large number of concept design options along with a rapid and reliable means of evaluating the options will designers be able to increase design pro- ductivity whilst identi$ing and developing new innovative products.It is recognised that a significant difTicultyrvith eval- uating design options is that they are 'information poor'.That is, important decisions often have to be taken with very limited information It].This provides designers rvith a major challenge and requires the provision ofdesign tools and aids.These are likely to be implemented within a computer-based environment [2].It follows that the theoretical models under- pinning the design tools must be shown to be valid, reliable and robust [3].
Research activity being undertaken at the University of Glasgow is guided by a concurrent model of evaluation.It assumes two parallel strands, Holistic and Decomposirion.
The Holistic approach takes a complete integrated view of the design artefact and seeks to provide an evaluation of the acceptability of it.The decomposition approach, on the other hand, evaluates the design at design characteristic level and then recombines these into an overall evaluation that can then be compared with the outcome of the Holistic approach.This approach is summarised and illustrated by a six-step methodology, shown in Fig. l.The main elements of the methodology are now briefly reviewed.These are: Decomposition of Design, Holistic Approach, Pareto Optimality.
The objectives of this paper are: l.To outline the essential role of the evaluation activity throughout the product development process.
2. To report the current state of research activity aimed at understanding how design time can be significantly reduced via support of evaluation activity at the 'fuzzy'   front-end of the design process.
3. This paper therefore describes a method that models the perceived dependencies between design criteria, whilst mainaining the benefits of decomposition.This method is then linked to an approach enabling the relative estimation of the probability that a design concept will ultimately meer the requirements of the declared design criteria.This linking is shown to provide an enhanced capability to support human evaluation and selection ofconcept designs.
2 Decomposition of design Decomposition of design is well established in practice as a means of trying to simplifi the complexity of design activity [4].Indeed recent work has even reported on the strategic decomposition approach for conceptual design [5].Decomposition involves trying to deal with the complexity of design by both describing the required design as a set ofcharacteristics and also by undertaking design synthesis of sub-functions rather than trying to adopt a holistic approach.The underlying assumption is that the resulting recomposed design will satisfy holistic evaluation.That is, there is an assumption of independence between the criteria.Howeve4 this is clearly not the case in most practical situations and what remains unclear is how the relationship between design characteristics should be modelled, to reflect their dependent nature, and how they may be recombined to provide a more realistic and holistic evaluation of the complete design.The models that seek to describe decomposition take as fundamental the idea of a sub-division of design space.That is, for a given design domain, the associated design characteristics (D"r) allow all associated specifications and models to be described in terms of the values of the characteristics.A Product Design Specifi- cation (PDS) can, in turn, be viewed as comprising a set ofD,r.The activity of evaluation then consists of the evaluator making ajudgement as to whether a particular design concept will meet the target value of each D.,, in turn.This can be illus- A relation-set probability and concept probability is evaluated.
Compare the results of the decomposition approach with the holistic approach.
t*'l t-*" l f-t*'l The degree of interaction and hence the degree of match between the desired target value range and the estimated target range is given by the Design Margin (DM), as follows; t2 where, pDarT and pDrlrE are respectively the mean values and o7 and op 2r€ respectively the standard deviations of the Target and Estimation distributions.
It has been shown [2] that the relationship between the design characteristics, when each is equally imporrant, can be modelled by: Note: n = the number of design characteristics.
In situations where it is not essential that all design charac- teristic targets are achieved then one possible approach is to convert the DM's to an implied probability using the stand- ardised cumulative uniform distriburion as shown in Fig. 3.In this case the distribution has been standardised over a range of 0 to 4, thus if for example the DM has a value of 2 then the (l) (r\ implied probability @inpri.a)that the target will be achieved is 0.5.

Fig. 3: Standardised cumulative uniform distribution
The values on the x-axis represent the number of standard deviations (z) of separation between a selected value (x)   and the mean value (p,).Four standard deviations represent almost 100 7o of possible values.This value (z) is determined from the lollowing equation: Thus, from Fig. 3, if the value of z =0 then rhere is no separation between the mean and the selected value and therefore there is 100 % match and hence an implied proba- bility value of I is selected.Fig. 3 is analogous to a reliability curve.
The underlying assumption here is that a'relation-set'can be defined that links a set ofdesign characrerisrics such that if at least one were to bejudged on-target then there is a possi- bility that the design concept may develop ro rowards a suc- cessful conclusion.This is, analogous to a cut-set in a complex reiiability network.A further assumprion is that each design characteristic is independent of the others.In some situations this assumption may not be valid and ther"efore musr be the subject of continuing research.In the meantime the following equation has been shown [3] to provide a valid method to aid identification and selection of appropriate design conceprs.Furthe4 the relative importance of each design characteristic, usually defined within traditional evaluation merhods by an individual weighting, can be seen to be inappropriate when the interaction of design characteristics takes place.
A more acceptable and logical approach is to define the importance or criticality of each characteristic in terms of the desired degree of match with the design specificarion rarger levels.

Holistic evaluation
It is human nature to try to reduce complexity when at all possible, The aim is to obtain a usable answer as quickly as possible.Designers are no exception to this trait and have therefore traditionally relied upon using experience to rnake holisticjudgemenrs abour design concepts.That is theyjudge the overall acceptability ofa design concept rarher rhan using methods of decomposition.This may be acceptable if the evaiuator's experience is completely appropriate, but what happens to the reliabiliry of the merhod when faced with new or innovative concepts?To understand this point further ir is necessary to consider how a design concept progresses through the design process.We can say that a design changes state, from being an idea to a concept to an embodied design through to a detail design capable of manufacture, as it passes through the design process.So any judgement of a concept requires consideration or prediction of whether the said concept will satisfactorily progress through the process or whether it will fail to meet the demands of the design specifi- cation at some point.-fhis is effectively the holisticjudgement that is being made.State-space, in a design context, is con- cerned with analysing the probability of furure srares of a system, or design, given knowledge of the probability of the system moving benveen states [6], in our case during the design process.In the conrext ofdesign this involves evaluat- ing the probability that a concept will meet the inter-related rtquirements of a design specification and thus be judged to be in a 'working' state.As the concept progresses through phases of the design process the probability of being in a working state will vary as information becomes richer.
A view of the likliehood of a particular concepr conrinuing to be in a 'working' state at the end of the design process will go a longway to supporting the initial concepr evaluation and selection decision.An acceptable or a working state can, in this context, be defined as Pareto Optimal.That is,,4 conceptis consilercd Pareto optirnnl if in atternpting to ntoue the aalue of a particular desi,gn characteristic closer to its target aalue or ronge the effect is to rttoue another of the duign criterin away from its target ualue or acceptable range'.In effect a Pareto Optimal state is one where you cannot make an improvement in one design characteris- tic without having a negative effect on another.
In making holistic judgement an evaluator is effectively having to make a subjective (fuzzy) estimate [7,8] of the follorving: l.Probability of maintaining a Pareto optimal srate.2. Probabiliry of moving out with a Pareto optimal srare.3. Probability of remaining out with a Pareto optimal srare.4. Probabiliry of moving into a Pareto optimal state.
A human evaluator makes a holisticjudgement, using a 0 to I scale, for each concept under evaluation.For example, let us assume that the estimated probabilities are as follows: 0.9, 0.1, 0.6, 0.4.This would allow us to construct aFvzzy Tiansition Probability Matrix which when multiplied by itself a sufficient number of times will reach a steady-state condition for the concept indicating its overall likliehood of being in a Pareto optimal state at the end of the process.i-0.9 0.1l " Lon 06l (5) In this case a steady state is reached after approximately seven intervals (n), this is illustrated in  The holistic approach provides a subjective probability that the concepr being evaluated will remain pareto optimal throughout the design process.In the above case a subjective probability of 0.8 is suggested.If this figure exceeds rhar asso- ciated with other concepts, rhen this concepr would be a strong contender for selection.Of course the methodology described in this paper requires that the resuit obtained via holistic evaluation be compared with rhat produced via the decomposition approach before a final conclusion is reached.
A simple example will now be used to illustrate this comparison.

Example
The fbllowing simple example is only designed to illustrate the methodology in action and does not represent a real situation.Let us therefore assume that rve have produced 3 concepts as potential design solutions to a particular prob- lem.Let us further assume that each concept is defined bv 5 design characteristics (A-E).
Step I -In accordance with the methodology rve firsr have to define design characterisric target (D.17) values.
These will obviously be the same for each concepr.
Step 2 -The evaluator estimates the likely value of the De- sign Characteristic (D.7,.8).
Step 3ll.elation sets are determined for the Design Char- acteristics.In this example we assume that two relation sets exist, namely: {A, B, C}, {D, E}.
Step 4 and 5 -The Design Margin is evaluated and the implied probabilities gained from the interacrion DrlT and Dr1,E are used in conjuncrion rvith equa- tion 4 to obtain a measure (Concept Score) of the concept that best meets the requirements.
The results of the above are summarised in Thble l.Step 6 -Requires comparison between the above and the holistic judgement using a state-space approach.
Let us again assume that the evaluator's judge- ments about the three concepts are as summarised in Table 2.
The resultant steady-state probability of the concept remaining in Pareto optimal state is determined using the method outlined earlier in the text.We see once again that concept 2 is identified as the best ofthe three being evaluared.
Our next step is to compare this result with those attained via the decomposition approach incorporating relation sets.To ease comparison the results are summarised in Thble 3.
In the above example we see that Concept 2 has main- tained dominance of the three options but that the ranking has changed from 2,1,3 with Design Margin, ro2,3,1with the  Relation set approach and then back again to 2, 1,3 r,vith the holistic approach.

Conclusions
This paper has outlined the increasing demands upon evaluation activity during the engineering design process.In particular, the need to address large numbers of innovative concept options during the conceptual design phase has been strcssed and a six-step methodology proposed.This methodology has been shown to comprise techniques of inexact reasoning with the need to combine two basic human ap- proaches to evaluation, namely decomposition and holistic.
It is clear to the author that the methods employed are only useful if they can be implemented within a computer environment, otherwise engineers and designers are likely to fall back on traditional approaches rather than the detailed approach of the six-step methodology.Only through computer implementation will the desired time reduction be achieved rvhilst being able to process data associated with a large number of design concepts and their associated set ofdesign characteristics.This is work that is on-going within the Department of Mechanical Engineering at the University of Glasgow.A par- ticular aspect of this research is the need to better model the interaction and interdependency of design characteristics and to further test the six-step methodology in active indus- trial environments.
Fig. I : Summary of evaluation methodology = the number of design characteristics within a de- fined relation-set R = the number of relation-sets

Fig
Fig. 4: Steady-state probability To specifically report the status of work aimed ar com-

Table l :
Summary of method for determination of concept scores J ' From the results in Thble I we can see that the summation of Design Margins for each concept indicates that Concept 2 has the best overall match, with a score of 6.61.Howeve4 when we seek to take the relationship between design charac- teristics into account by using the implied probability ro evaluate the relation seti we find that Concept 2 remains the best choice but that the order of the other two concepts has changed.Note that with the Design Margin approach, rhe smallest score is best, and with the relation set approach, the biggest score is best.

Table 2 :
Summary of evaluatorjudgements

Table 3 :
Compalison of evaluation results