background image

 

 

 

 

 

 

 

 

 

 

Simulation Driven Design 

Benchmark Report 

 

Getting It Right the First Time 

 

 
 
 
 
 

October 2006 

 
 
 
 
 

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

i

 

Executive Summary 

et it done quickly. That’s the message manufacturers are hearing from the mar-
ket. They must develop more products that are more complex for their custom-
ers, but, most of all, they must get to market on time. When trying to figure out 
how to get things done more quickly, these manufacturers face a seeming para-

dox: should they take more time to perform simulations in design so they can save time 
and money in testing? Some have found the answer is not only” yes,” but that early simu-
lation assists them to hit their product development targets. How?  Interestingly enough, 
it’s actually quite simple. 

Key Business Value Findings 

 

Best in class manufacturers their hit revenue, cost, launch date, and quality targets for 
86% or more of their products. 

 

Best in class manufacturers average 1.6 fewer prototypes than all others. 

 

Best in class manufacturers of the most complex products get to market 158 days 
earlier with $1,900,000 lower product development costs. 

 

Best in class manufacturers of the simplest products get to market 21 days earlier 
with $21,000 fewer product development costs. 

Implications & Simulation 

 

All best in class manufacturers use simulation in the design phase compared to only 
75% of laggards. 

 

Best in class manufacturers are 63% more likely to provide CAD-embedded simula-
tion to their engineers. 

 

Best in class manufacturers are 48% more likely to provide technologies to transfer 
models from CAD to independent preprocessors to their analysts. 

 

Best in class performers are 42% more likely than all others to provide specific ex-
amples to users for training. 

Recommendations for Action 

 

Perform more simulation of product performance in the design phase. 

 

Provide CAD-embedded simulation capabilities to engineers. 

 

Use training materials and specific examples to get new users up to speed. 

 

Employ technologies that transfer geometry from CAD to independent pre-
processors for analysts. 

 

Track requirements and regulatory product compliance prior to design release. 

 

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group  

 

Table of Contents 

Executive Summary .............................................................................................. i

 

Chapter One:

 Issue at Hand.................................................................................1

 

The Ultimate Goal: Hit Ever-shrinking Time-to-market Windows ................... 1

 

Early Simulation Cultural Challenges? A Misconception, Not a Reality.......... 2

 

Chapter Two: 

Key Business Value Findings .........................................................5

 

Varying Prototype Costs and Time across Product Complexity ..................... 6

 

Avoiding Physical Prototypes with Virtual Prototypes .................................... 7

 

Chapter Three:

  Implications & Simulation ...........................................................8

 

All Best in Class Performers Utilize Simulation in the Design Phase............. 8

 

Familiar Environments: Engineers Access Simulation through CAD ............. 9

 

Design Reuse: Analysts Transfer from CAD to Preprocessors .................... 10

 

Tracking Configurations with the Simulation Model or Data Management... 11

 

Educating the User: Formalized Training Programs .................................... 12

 

Keeping an Eye on Requirements and Change Orders............................... 13

 

Chapter Four

: Recommendations for Action ......................................................15

 

Laggard Steps to Success........................................................................... 15

 

Industry Average Steps to Success ............................................................. 15

 

Best in Class Next Steps ............................................................................. 16

 

Appendix A: 

Research Methodology ..................................................................17

 

Appendix B:

 Related Aberdeen Research & Tools ...............................................1

 

 

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006. 
Aberdeen Group

 

 

Figures 

Figure 1: Challenges to Simulation-driven Design................................................2

 

Figure 2: Responses to Simulation-driven Design Challenges.............................3

 

Figure 3: Best in Class Hit Targets on an 86% Average or Better ........................5

 

Figure 4: Best in Class Perform More Simulations Earlier....................................8

 

Figure 5: Best in Class Provide Engineers CAD-embedded Simulation...............9

 

Figure 6: Best in Class Provide Experts Independent Simulation Tools .............10

 

Figure 7: Best in Class Track Simulation Configurations....................................12

 

Figure 8: Best in Class Performers Provide Formalized Training Programs.......13

 

Figure 9: Manufacturers Track Compliance to Requirements 
 and Change Orders...........................................................................................14

 

 

Tables 

Table 1: Top Five Business Pressures and Strategic Actions ...............................1

 

Table 2: General Characteristics of Product Complexity Categories ....................6

 

Table 3: Prototype Costs and Time per Product Complexity ................................7

 

Table 4: PACE Framework .................................................................................18

 

Table 5: Relationship between PACE and Competitive Framework ...................19

 

Table 6: Competitive Framework........................................................................19

 

 

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

Chapter One:

 

Issue at Hand 

Key Takeaways

 

 

Manufacturers are improving product performance (72%) and development efficiency 
(51%) in order to address shortened time-to-market (70%) constraints. 

 

Manufacturers identify shorter time to market (72%) as a top business pressure to 
adopt early simulation, yet cite lack of time (44%) as the top challenge. 

 

Manufacturers are responding to challenges such as lack of expertise (39%) and 
complicated product behavior (28%) with formal training programs. 

 

Manufacturers, in fact, did not identify any of the expected cultural challenges in im-
plementing early simulation as actual issues. 

 

hile the concept of using simulation early in the product development cycle 
initially emerged almost a decade ago, it remains a frequently pursued initia-
tive today. Although one would expect an elevated level of use of simulation 
in the up-front design phase by now, in fact, the pressure for shorter time to 

market and engineering cultural challenges have prevented manufacturers from succeed-
ing with this new paradigm. Yet some of these companies are overcoming these barriers 
to realize tangible business benefits. 

The Ultimate Goal: Hit Ever-shrinking Time-to-market Windows 

In one form or another, manufacturers adopting simulation earlier in the product devel-
opment process are reacting to pressure for shorter time to market by improving quality 
and development efficiencies (Table 1). 

Table 1: Top Five Business Pressures and Strategic Actions 

Business Pressures 

Strategic Actions 

Shortened time to market 

70%  Improve product performance or quality 

72% 

Customer demand for new products 

51%  Improve development efficiency 

51% 

Increasingly complex customer require-
ments 

40%  Reduce base development costs 

20% 

Accelerating product commodization 

30%  Develop markets via  breakthrough innovation  18% 

Threatening competitive products 

20%  Iterate product design more often 

16% 

Source: 

Aberdeen

Group

, October 2006 

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

2 • 

AberdeenGroup 

Based on the results of Aberdeen research, 
clearly the top pressure driving manufac-
turers to adopt earlier simulation is 

short-

ened time to market

 (70%). In response to 

this pressure, manufacturers are following 
two main strategies: 

improving product 

performance or quality

 (72%) and 

improv-

ing development efficiency 

(51%). At first 

glance, it’s not immediately clear how 
these relate to one another or how they re-
late to simulation early in the product development process. However, follow-up inter-
views with survey respondents found that they were looking to early simulation to ac-
complish two goals: to arrive at a good design earlier and to minimize time spent in the 
verification and testing phase of product development. 

In addition to their time concerns, manufacturers identify 

customer demand for new 

products

 (51%) and 

increasingly complex customer requirements

 (40%) as secondary 

business pressures driving earlier simulation. Correspondingly, they are turning to strate-
gies such as

 improving development efficiency 

(51%) and 

iterating the product design 

more often

 (16%). 

Overall, manufacturers are turning towards early simulation to save time in product de-
velopment as well as to get to better designs. However, the ultimate goal is to meet the 
shorter time-to-market windows that are today’s reality. 

Early Simulation Cultural Challenges? A Misconception, Not a Reality 

In the simulation-driven design trend, one would expect major cultural barriers to requir-
ing engineers to perform simulation earlier in product development because they are be-
ing asked to do more in the same amount of time. In reality, the expected cultural chal-
lenges were not cited as major obstacles (Figure 2). 

Figure 1: Challenges to Simulation-driven Design 

19%

26%

33%

40%

44%

0%

10%

20%

30%

40%

50%

Software and hardware costs

Physical test correlation

Complex product behavior

Lack of expertise

Lack of time

 

Source: 

Aberdeen

Group

, October 2006 

Elgin Sweeper Company 

“We are definitely pressured to get to de-
sign release more quickly in order to keep 
up with the competition. We need to get 
to market first to win market share. We’re 
turning to simulation to minimize our test-
ing phase of product development.” 

Jay Abrams, Elgin Sweeper Company

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

In fact, the top challenge, 

lack of time

 

(44%), aligns with the top business driver, 

shortened time to market

 (70%). However, 

some manufacturers are caught in a conflict 
between reducing time in the testing phase 
and spending more time in the design phase 
performing early analyses. Coming to the 
realization that a company must allow en-
gineers more time to perform analyses up-
front in order to save time and cost later in 
the development cycle in itself is a cultural challenge. 

The challenges of 

lack of expertise

 (40%) and 

complex product behavior

 (33%) for  per-

forming simulation within engineering organizations provides ample motivation for 
manufacturers to respond with formal training rollout programs (Figure2). In fact, the top 
four responses to challenges in this case reflect just that. 

Figure 2: Responses to Simulation-driven Design Challenges 

26%

29%

40%

45%

47%

0%

10%

20%

30%

40%

50%

Educate executives on benefits

Acquire easy to use software

Capture and deploy best practices

Software training

FEA concepts education

 

Source: 

Aberdeen

Group

, October 2006 

Manufacturers are 

acquiring easy-to-use software

 (29%) and providing 

software training 

(45%) in order to reduce the technological barriers of performing simulation for non-
expert and infrequent engineering users. Simultaneously, manufacturers are pursuing 

FEA concepts education

 (47%) as well as programs to 

capture and deploy best practices

 

(40%) in order increase the quality of analyses, so the results are more dependable and, 
consequently, yield better designs. These four tactics directly address the second and 
third top challenges: 

lack of expertise

 and 

complicated product behavior

The Holland Group 

“We run analyses so we can test out a de-
sign before we actually spend money on a 
prototype. Overall, while it takes more 
time to run a simulation, we’ll save time 
later by avoiding multiple rounds of pro-
totypes.” 

Tave Hass, The Holland Group

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

4 • 

AberdeenGroup 

While some cultural issues, in fact, are inherent 
in the challenges indicated by survey respon-
dents, many of the common perceptions are 
actually untrue. Less than 16% of manufactur-
ers identified issues such as 

only analysts per-

form simulation

difficult to get engineers to 

perform analyses,

 and 

little confidence in re-

sults

 as challenges to performing simulation 

upfront. 

Overall, the message is clear. Manufacturers 
are addressing the secondary challenges of lack 
of expertise and complicated product behavior 
with formal training and education programs 
and easy-to-use software. The paramount cul-
tural challenge for manufacturers is grasping 
the reality that they must spend more time per-
forming simulations upfront in order to save 
time in the testing phase. Outside of that, the 
cultural challenges to introducing simulation 
during design are minimal. 

CTS Corporation 

“Correlation to physical test is a large 
challenge for us. For one, the testing 
lab can’t perfectly match the idealized 
setup within the finite element simu-
lation tool. Also, the molded parts that 
are tested are slightly different 
because of manufacturing variation. 
And, finally, the material properties 
of our injection-molded parts are 
orthotropic, meaning they aren’t the 
same in all directions. Correctly set-
ting up the simulation to accurately 
reflect the reality of the actual mate-
rial properties is difficult.” 

Dave Pfaffenberger, CTS Corporation

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

Chapter Two: 

Key Business Value Findings 

Key Takeaways

 

 

Best in class manufacturers their hit revenue, cost, launch date, and quality targets 
for 86% or more of their products. 

 

Best in class manufacturers average 1.6 fewer prototypes than other companies. 

 

Best in class manufacturers of the most complex products get to market 158 days 
earlier with $1,900,000 fewer product development costs. 

 

Best in class manufacturers of the simplest products get to market 21 days earlier 
with $21,000 fewer product development costs. 

 

hile some manufacturers are adopting simulation early in the product devel-
opment cycle, Aberdeen research shows that they face serious challenges. 
While some are taking steps in response, their strategies and tactics are only as 

good as the results they deliver. To get a clear picture of which strategies and tactics are 
successful, Aberdeen categorized survey respondents by measuring five key performance 
indicators (KPIs) that provide 

financial, process, and quality measures

 (Figure 3). This 

classification subsequently enabled differentiation between the “best practices” of the top 
performers and the practices of lower-performing companies. 

Figure 3: Best in Class Hit Targets on an 86% Average or Better

 

87%

87%

86%

89%

91%

77%

64%

60%

58%

66%

63%

48%

46%

45%

45%

0%

20%

40%

60%

80%

100%

Product

revenue

targets

Product cost

targets

Development

cost targets

Product

launch dates

Product

quality

expectations

Best in class

Average

Laggard

 

Source: 

Aberdeen

Group

, October 2006 

Based on aggregate scores incorporating all five metrics, those companies in the top 20% 
achieved “best in class” status; those in the middle 50% were “average”; and those in the 
bottom 30% were “laggard.” As expected, companies in the different performance cate-

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

6 • 

AberdeenGroup 

gories show substantial differences – with best in class hitting all five marks at an 86% or 
better average. 

Varying Prototype Costs and Time across Product Complexity 

One of the primary reasons manufacturers 
pursue simulation early in the product 
development lifecycle is to test product 
performance virtually. Products that are 
virtually tested have a higher chance of 
passing physical prototype testing the first 
time. Overall, this translates to less time 
and lower development costs in the prod-
uct development lifecycle.  

Translating reduced prototypes into hard 
costs and time depends on the complexity 
of the product. To get a clear picture of 
how prototype costs and time varied ac-
cording to product complexity, Aberdeen 
categorized survey respondents’ products 
by measuring three key indicators: 

num-

ber of parts

length of development lifecy-

cle

, and 

number of engineering disciplines 

incorporated

. This measurement subse-

quently enabled differentiation of levels of product complexity. The following table de-
scribes the general characteristics of each of the product complexity categories from this 
study’s research (Table 2). 

Table 2: General Characteristics of Product Complexity Categories 

Product Complexity 

Number of Parts 

Length of Development 

Low 

Less than 50 

Between a week and a year 

Moderate 

Between 50 and 1,000 

Between a month and 5 years 

High 

Between 50 and 10,000 

Between 1 and 5 years 

Very High 

Between 1,000 and 100,000 

Between 1 and 20 years 

Source: 

Aberdeen

Group

, October 2006 

Based on these product complexity categories, one can see a logical progression in the 
corresponding increase in time and costs as complexity increases (Table 3). 

 

 

Power Tool Manufacturer 

“Most recently, not only were we able to 
reduce prototypes, but we were able to skip 
a prototype qualification phase altogether. 
Typically, we would develop machined 
prototypes, but in this case we went right to 
a prototype casting.” 

Ricon Corporation 

 “Our products move through a range of 
motion. We could always visualize the 
movements in our heads, but we started 
using simulation as a means to virtually 
verify the motion prior to building proto-
types.”

 

Ray Reynolds, Ricon Corporation

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

Table 3: Prototype Costs and Time per Product Complexity 

Product Complexity 

Time to Build Prototype 

Cost to Build Prototype 

Low 

13 days 

$7,600 

Moderate 

24 days 

$58,000 

High 

46 days 

$130,000 

Very High 

99 days 

$1,200,000 

Source: 

Aberdeen

Group

, October 2006 

Avoiding Physical Prototypes with Virtual Prototypes 

Is there any truth to the suggestion that using 
simulation during design eliminates unneces-
sary additional rounds of prototyping? The 
answer is “yes.” Aberdeen research finds that 
the best in class average 3.0 prototypes com-
pared to 4.6 for all other manufacturers.  

Applying this difference of 1.6 prototypes to 
the different categories of product complexity 
yields compelling results. The best in class 
manufacturers of the products with very high 
complexity get to market 158 days earlier 
with $1,900,000 lower product development 
costs than average performers. At the oppo-
site end of product complexity spectrum, the 
best in class manufacturers get to market 21 
days earlier and spend $12,000 less on prod-
uct development costs than average performers.  

Obviously there are very real benefits in early simulation that translate into a direct im-
pact on time to market and product development costs. 

 

 

 

 

 

 

 

 

 

Plastics One 

“Because we make over-molded elec-
tronics products, we’ve explored a few 
different types of simulations for differ-
ent purposes. For one, we’ve used 
mold-filling simulations to determine 
the base placements for injection-
molding gates. Also, because over-
molding can be rough on electronics, 
we explore various scenarios of failed 
electronics components so that when we 
do experience problems, we know 
which scenario actually occurred.” 

Steve Heckman, Plastics One

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

8 • 

AberdeenGroup 

Chapter Three:

  

Implications & Simulation 

Key Takeaways

 

 

All best in class manufacturers use simulation in the design phase compared to three 
out of four laggards. 

 

Best in class manufacturers are 63% more likely than all others to provide CAD-
embedded simulation to their engineers. 

 

Best in class manufacturers are 48% more likely than all others to provide technolo-
gies to transfer models from CAD to independent preprocessors to their analysts. 

 

Best in class performers are 42% more likely than all others to provide specific prod-
uct simulation examples to users for training. 

 

s noted earlier, the aggregated performance of surveyed companies determined 
whether they ranked as best in class, industry average, or laggard. In addition to 
having common performance levels, each class also shares characteristics and 
practices in four key categories – processes, organizational structure, technology 

usage, and performance measurement. 

All Best in Class Performers Utilize Simulation in the Design Phase 

While many manufacturers have focused on adopting simulation earlier in the product 
development process, best in class performers employ simulation throughout the product 
development lifecycle (Figure 6). 

Figure 4: Best in Class Perform More Simulations Earlier 

100%

88%

72%

57%

74%

90%

53%

69%

78%

0%

20%

40%

60%

80%

100%

Design phase

Test phase

Post design release

Best in class

Average

Laggard

 

Source: 

Aberdeen

Group

, October 2006 

In fact, every single one of the best in class performers surveyed for this report uses 
simulation in the design phase as opposed to roughly three out of every four laggards. 
And not only is this difference in the design phase, but the 20% difference continues in 
the test and post-design release phases.  

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

Finally, one can see that the best 
in class performers place the 
highest emphasis on performing 
simulation upfront in the design 
phase as opposed to during the 
other phases of development. 
Overall, this early use enables 
those leading manufacturers to 
reduce the number of prototypes 
necessary to pass quality tests as 
well as to avoid unnecessary 
change orders after design re-
lease. 

Familiar Environments: Engineers Access Simulation through CAD 

Given that many manufacturers are performing simulations early in the design phase of 
product development, the next question is “how is this being accomplished?” Tradition-
ally, the responsibility for completing simulations has commonly fallen on the analysis 
group dedicated to the task. However, the current, ongoing trend is to push the simpler 
and directional analyses upstream, into the hands of product engineers, as way of aug-
menting the efforts of the dedicated analyst groups. 

How can companies get engineers to take on this additional task? Common sense says to 
make it simple and easy to do. This starts with how the engineers access simulation capa-
bilities (Figure 8). 

Figure 5: Best in Class Provide Engineers CAD-embedded Simulation 

36%

36%

0%

27%

22%

30%

16%

32%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Embedded within

CAD application

Transfer from

CAD to pre-

processor

application

Only in

independent pre-

processor

Mixes of all

options

Best in class

All others

 

Source: 

Aberdeen

Group

, October 2006 

While a number of options for accessing simulations are available to engineers, only few 
are typically used. These include simulations 

embedded within the CAD application

 and 

Smith Aerospace 

“We perform analyses to make sure we are arriving 
at the most weight and cost effective design. In 
short, we want to get the biggest bang for our buck. 
Additionally, an analysis identifies all the potential 
failure modes whereas building and breaking a pro-
totype will reveal just one… the one that broke the 
prototype.” 

Jochen Hessemann, Smith Aerospace

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

10 • 

AberdeenGroup 

transferred from CAD to preprocessor applications

. The first option, embedding simula-

tion capabilities within a CAD application, keeps the engineer in a familiar environment 
and removes the additional step of transferring geometry over to another application. 
Sometimes this second option is necessary for an advanced setup such as finite element 
mesh adjustment and additional geometric idealizations. Development of simulation 
models 

only within independent preprocessors

 requires users to duplicate the design ge-

ometry. 

Overall the best in class are 
63% more likely than all other 
manufacturers to access simu-
lation capabilities directly 
within CAD applications (35% 
versus 22%). Conversely, the 
best in class never utilize inde-
pendent preprocessors. Some 
manufacturers do transfer from 
the CAD application to an independent preprocessor, but there is no significant differ-
ence between best in class and all others in this practice. All in all, the conclusion is 
clear. The best in class provide access to simulation capabilities for their engineers di-
rectly through CAD applications and through independent preprocessors only when nec-
essary.  

Design Reuse: Analysts Transfer from CAD to Preprocessors 

A similar question – how are simulations accessed by analysts – yielded dramatically 
different results (Figure 9). 

Figure 6: Best in Class Provide Experts Independent Simulation Tools 

0%

46%

0%

54%

13%

31%

10%

46%

0%

10%

20%

30%

40%

50%

60%

Embedded within

CAD application

Transfer from

CAD to pre-

processor

application

Only in

independent pre-

processor

Mixes of all

options

Best in class

All others

 

Source: 

Aberdeen

Group

, October 2006 

ARA Engineering 

“CAD embedded simulation can be very fast and accu-
rate for the fundamental assessments of your product. 
We use these in the middle of the design cycle to di-
rectionally confirm our design decisions as we pro-
ceed.” 

Roxanne Abul-Haj, ARA Engineering

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

11 

As shown, none of the best in class 
manufacturers have their analysts’ 
access simulation capabilities in 
CAD applications. Given that many 
simulation vendors purposefully do 
not expose all simulation capabili-
ties, especially the advanced ones, 
through CAD applications, analysts 
find that they can’t even setup ad-
vanced analyses. However, those 
analysts do want to take advantage 
of the modeling work that has already taken place. As a result, analysts at many manufac-
turers perform a 

transfer from CAD to a preprocessor application

 in an effort to reduce 

the overall simulation cycle. Interestingly enough, because analysts want to take advan-
tage of the already completed design work, they never develop the entire model 

only in 

an independent preprocessor.

 

On the whole, at best in class manu-
facturers compared to other compa-
nies, analysts are 48% more likely 
to transfer from the CAD applica-
tion to an independent pre-processor 
instead of accessing simulation ca-
pabilities through a CAD applica-
tion or building the simulation 
model independent of the CAD ap-
plication. 

Tracking Configurations with the Simulation Model or Data Manage-
ment 

During the setup of a simulation, many simplifications and idealizations are made to de-
signs in an effort to minimize mesh times and computation solve times. While this is a 
common occurrence, many users don’t make the effort to accurately capture the derived 
configuration of the model that was analyzed. This can cause issues if the product later 
fails and a root cause investigation is required as part of a change process because the 
details of the simulation configuration were not documented. Overall, there are a number 
of options manufacturers can use to track these simulation configurations (Figure 7). 

Smith Aerospace 

“We export geometry from the CAD tool and 
import it into a standalone package. Besides sav-
ing time, when we transfer geometry we know 
we are analyzing the exact design the engineer 
developed. Re-creating it can introduce mistakes 
in the geometry and might not accurately repre-
sent the design you’d want to analyze.” 

Jochen Hessemann, Smith Aerospace

GHSP 

“We use standalone analysis tools because the 
capabilities are more encompassing. We do take 
the geometry from the CAD tool into the stand-
alone tool so we don’t have to create the geome-
try there.” 

Mike Hoyt, GHSP

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

12 • 

AberdeenGroup 

Figure 7: Best in Class Track Simulation Configurations  

8%

20%

12%

46%

14%

18%

18%

20%

6%

39%

0%

10%

20%

30%

40%

50%

Not tracked

Spreadsheet

CAD model

Simulation

model

Data

management

Best in class

All others

 

Source: 

Aberdeen

Group

, October 2006 

In fact, most of the best in class performers use the simulation model as a means to track 
the configuration of the product that was actually analyzed. In addition, they are more 
than twice as likely to use data management to track the simulation configuration through 
a centralized data management tool. 

Overall, the conclusion is that the best in class track simulation configurations at least in 
some way, most frequently by using the simulation model or by using data management 
tools (and using them twice as frequently as other manufacturers). 

Educating the User: Formalized Training Programs 

Based on the fact that manufacturers are 
beginning to assign the simpler analyses to 
their engineers, it’s easy to see why train-
ing, education, and easy-to-use software 
are the highest ranked responses to the 
challenges of adopting simulation in the 
design phase of product development. A 
closer look reveals that the best in class 
train and educate their users by variety of 
means (Figure 8). 

Liebherr Mining 

“As far as training goes, we use all the 
available means to get users up to speed. 
We utilize outside training in addition to 
inside training. We have many users that 
are experienced and can pretty much coach 
them on the job.” 

Vladimir Pokras, Liebherr Mining

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

13 

Figure 8: Best in Class Performers Provide Formalized Training Programs 

66%

40%

74%

78%

72%

47%

52%

65%

0%

20%

40%

60%

80%

100%

Tutorials

Generic

examples

Specific

examples

Training

materials

Best in class

All others

 

Source: 

Aberdeen

Group

, October 2006 

Best in class manufacturers are 
more likely to provide three types of 
training list here more than other 
companies and two of them mark-
edly more than all other companies. 
Specifically, a large percentage of 
best in class manufacturers use tuto-
rials (66%), specific examples 
(74%), and training materials (78%). Tutorials and training materials help these compa-
nies provide both 

FEA concepts education

 and 

software training

. Specific examples help 

address concerns about evaluating 

complicated product behavior

 by showing users how 

to apply the software to the manufacturer’s specific products. Interestingly, best in class 
performers are 42% more likely to provide specific examples to their users (74% vs. 
52%) than other companies. 

Overall the conclusion is clear. The best in class performers are much more likely to pro-
vide training materials to their users to address their 

lack of expertise

 and 

complicated 

product behavior

 – the primary challenges they cited to performing simulation earlier in 

the product development cycle. 

Keeping an Eye on Requirements and Change Orders 

Overall, the ultimate goal is to 
manufacture a product that wins in 
the market or satisfies the cus-
tomer’s needs. How do manufactur-
ers ensure this is in fact the case? A 
wide variety of optional tactics are 
available (Figure 9). 

 

KONE Elevators and Escalators 

“We actually use the Web help fairly extensively. 
We email questions to our software vendor sup-
port group, and they post answers on their Web-
site.” 

Andy Jahn, KONE Elevator and Escalator

Large Aerospace and Defense Contractor 

“We get requirements from the customer regard-
ing fatigue life, deflection, weight, natural fre-
quencies, and other areas. The goal is to get as 
close to all of them as possible.” 

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

14 • 

AberdeenGroup 

Figure 9: Manufacturers Track Compliance to Requirements and Change Orders 

Measures that are tracked after product 

shipment or delivery to customer

Measures that are tracked prior to 

product shipment or delivery to customer

43%

14%

96%

14%

0%

11%

71%

36%

28%

62%

28%

8%

14%

68%

0%

20%

40%

60%

80%

100%

Requirements

compliance

Regulatory

compliance

Part

qualification

levels

Number of

change orders

Customer

satisfaction

measures

Number of

prototypes

Warranty

costs

Best in class

All others

Measures that are tracked after product 

shipment or delivery to customer

Measures that are tracked prior to 

product shipment or delivery to customer

43%

14%

96%

14%

0%

11%

71%

36%

28%

62%

28%

8%

14%

68%

0%

20%

40%

60%

80%

100%

Requirements

compliance

Regulatory

compliance

Part

qualification

levels

Number of

change orders

Customer

satisfaction

measures

Number of

prototypes

Warranty

costs

Best in class

All others

 

Source: 

Aberdeen

Group

, October 2006

 

Prior to design release, the majority of manufacturers primarily track requirements com-
pliance. Tracking compliance to product performance requirements acts as a key enabling 
mechanism for early corrective action to work-in-process design. Additionally, a large 
number of manufacturers track regulatory compliance. With the number of regulations 
increasing worldwide, this comes as no surprise. Interestingly enough, tracking part 
qualification levels, a common practice in many industries, is markedly lower for best in 
class performers compared to laggards. Overall, these measures are commonly verified 
through simulation prior to the development of a product prototype. 

After design release, many manufacturers track the number of change orders logged 
against a specific product. Companies use this measure to conduct a “post-mortem” 
judgment on the practices used during that product’s development. Specifically, best in 
class manufacturers are 55% more likely than other companies (i.e., 96% versus 62%) to 
track the number of change orders against a product. Outside of this measure, other post-
design-release measures are infrequently used. Because simulation should be able to pre-
dict product failures that are the root cause of many change orders, manufacturers use this 
“post-mortem” judgment to identify the simulation approaches used associated with that 
product as good or bad. 

The conclusion is clear. Best in class performer most commonly measure product per-
formance by evaluating requirements and regulatory compliance prior to design release 
and by tracking the number of change orders after design release. 

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

15 

Chapter Four

Recommendations for Action 

Key Takeaways

 

 

Perform more simulation of product performance in the design phase. 

 

Provide CAD-embedded simulation capabilities to engineers. 

 

Use training materials and specific examples to get users up to speed. 

 

Employ technologies that transfer geometry from CAD to independent preprocessors 
for analysts. 

 

Track requirements and regulatory product compliance prior to design release. 

 

egardless of the fact that performing more analyses upfront in the design phase 
takes more time, manufacturers are engaging in this practice to save time and 
money in creating physical prototypes, so they can hit shrinking time-to-market 

windows. The following actions can help them address the challenges of early simulation 
as well as enable them to improve their performance levels from “laggard” to “industry 
average,” or from “industry average” to “best in class,” or even from “best in class” to 
number one in their market. 

Laggard Steps to Success 

1.

 

Perform more analyses in the design phase. 

Use simulation capabilities to virtually prototype products in the design phase to 
save prototype costs and time in the testing phase. 

2.

 

Don’t force engineers to use independent preprocessors. Adopt CAD-embedded 
simulation capabilities instead. 

Requiring engineers to perform new tasks in completely such new and unfamiliar 
applications as independent preprocessors increases the barriers to success. In-
stead, provide simulation capabilities through CAD tools, an environment with 
which they are more familiar. 

3.

 

Employ training materials and specific examples of company products to bring 
users up to speed. 

Training materials will enable users to understand FEA concepts as well to use 
new simulation software applications. Examples that are specific to your business 
and products will help users better understand how apply the technology in their 
everyday work. 

Industry Average Steps to Success 

1.

 

Provide analysts with the ability to transfer geometry from CAD to independent 
preprocessors instead of CAD embedded simulation.  

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

16 • 

AberdeenGroup 

Simulation capabilities embedded within CAD applications are consciously lim-
ited in an effort to make things simpler for engineering users. Analysts need ac-
cess to the advanced capabilities in independent preprocessors. However, trans-
ferring geometry from the CAD application to these independent preprocessors 
allow them to take advantage of and reuse designs, shortening the time required 
to perform analyses. 

2.

 

Measure the number of change orders as lagging indicator of best practices. 

Manufacturers should track the number of change orders related to failed product 
performance as a post-design-release measure of the successful use of simulation 
in the design phase. It will also facilitate efforts to continuously improve the use 
of simulation. 

3.

 

Track the simulation configuration through simulation models or data manage-
ment. 

In order to understand what was simulated after the product has been released 
and launched, formally track the product configurations along with the idealiza-
tions and simplifications required by simulations through a simulation model or 
data management tool. 

Best in Class Next Steps 

1.

 

Track requirements and regulatory compliance more heavily prior to design re-
lease. 

While tracking change orders is a good lagging indicator of the successful appli-
cation of simulation to a product’s performance, measuring leading indicators 
such as requirements and regulatory compliance will result in minimizing proto-
type costs and time. 

2.

 

Reinforce use of CAD-embedded simulation capabilities — instead of independ-
ent preprocessors — for engineers. 

While using an independent pre-processor provides access to advanced simula-
tion capabilities, manufacturers should reinforce the use of the simulation capa-
bilities embedded in CAD applications. This way, engineers can avoid learning a 
new application that they will infrequently use. Overall, this practice will lower 
the barrier to performing simulation earlier in the design phase. 

 

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

17 

Appendix A: 

Research Methodology 

uring August 2006, Aberdeen Group and 

Desktop Engineering

 and 

NAFEMS 

examined the experiences and intentions of more than 270 enterprises regarding 
their mechanical design simulation methodologies. Responding engineering ex-
ecutives completed an online survey that included questions designed to deter-

mine the following: 

 

The degree to which mechanical design simulation impacts corporate strategies, op-
erations, and financial results 

 

The structure and effectiveness of existing mechanical design simulation 

 

The benefits, if any, that have been derived from mechanical design simulation initia-
tives 

Aberdeen supplemented this online survey effort with telephone interviews with select 
survey respondents, gathering additional information on mechanical design strategies, 
experiences, and results. 

The study aimed to identify emerging best practices for mechanical engineering and de-
sign and provide a framework by which readers could assess their own mechanical design 
capabilities. 

Responding enterprises included the following: 

 

Job title/function

:

 The research sample included respondents with the following job 

titles: engineering and design staff (34%), engineering and design managers (25%), 
internal consultants (11%), engineering and design directors (7%), and senior man-
agement (CEP, COO, CFO) (6%). 

 

Industry

:

 The research sample included respondents predominantly from manufac-

turing industries. Automotive manufacturers represented 22% of the sample. Manu-
facturers in aerospace and defense accounted for 21% of respondents. Industrial 
equipment manufacturers followed at 16%. Other sectors responding included com-
puter equipment and peripherals, high technology, telecommunication manufacturers, 
services, and logistics. 

 

Geography

:

 North American respondents accounted for 46% of respondents fol-

lowed closely by EMEA respondents at 43%. Remaining respondents from Asia-
Pacific region and South America represented the remaining 11% of the respondent 
pool. 

 

Company size

:

 About 33% of respondents were from small businesses (annual reve-

nues of $50 million or less), 40% were from midsize enterprises (annual revenues be-
tween $50 million and $1 billion), and 28% of respondents were from large enter-
prises (annual revenues above US$1 billion). 

Solution providers recognized as sponsors of this report were solicited after the fact and 
had no substantive influence on the direction of the 

Simulation-driven Design Benchmark 

background image

 The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

18 • 

AberdeenGroup 

Report

. Their sponsorship has made it possible for Aberdeen Group, 

Desktop Engineer-

ing,

 and 

NAMFEMS

 to make these findings available to readers at no charge. 

Table 4: PACE Framework 

PACE Key 

Aberdeen applies a methodology to benchmark research that evaluates the business pressures, actions, 
capabilities, and enablers (PACE) that indicate corporate behavior in specific business processes. These 
terms are defined as follows: 

Pressures —

 

external forces that impact an organization’s market position, competitiveness, or business 

operations (e.g., economic, political and regulatory, technology, changing customer preferences, competi-
tive) 

Actions —

 

the strategic approaches that an organization takes in response to industry pressures (e.g., align 

the corporate business model to leverage industry opportunities, such as product/service strategy, target 
markets, financial strategy, go-to-market, and sales strategy) 

Capabilities —

 

the business process competencies required to execute corporate strategy (e.g., skilled 

people, brand, market positioning, viable products/services, ecosystem partners, financing)

 

Enablers

 — the key functionality of technology solutions required to support the organization’s enabling 

business practices (e.g., development platform, applications, network connectivity, user interface, training 
and support, partner interfaces, data cleansing, and management)  

 

Source: Aberdeen Group, Month 2006 

 

background image

The Simulation Driven Design Benchmark Report 

 

 

All print and electronic rights are the property of Aberdeen Group © 2006.

 

Aberdeen Group • 

19 

Table 5: Relationship between PACE and Competitive Framework 

PACE and Competitive Framework How They Interact 

Aberdeen research indicates that companies that identify the most impactful pressures and take the most 
transformational and effective actions are most likely to achieve superior performance. The level of com-
petitive performance that a company achieves is strongly determined by the PACE choices that they make 
and how well they execute. 

Source: Aberdeen Group, Month 2006 

Table 6: Competitive Framework 

Competitive Framework Key 

The Aberdeen Competitive Framework defines enterprises as falling into one of the three following levels of 
FIELD SERVICES practices and performance: 

Laggards

 

(30%)

 — FIELD SERVICES practices that are significantly behind the average of the industry, 

and result in below average performance 

Industry norm

 

(50%)

 — FIELD SERVICES practices that represent the average or norm, and result in aver-

age industry performance. 

Best in class (20%)

 — FIELD SERVICES practices that are the best currently being employed and signifi-

cantly superior to the industry norm, and result in the top industry performance. 

Source: Aberdeen Group, Month 2006 

 

 

 

 

 

 

 

 

 

background image

The Simulation Driven Design Benchmark Report 

 

 

Aberdeen

 

Group, Inc. 

260 Franklin Street 
Boston, Massachusetts 
02110-3112 
USA 

Telephone: 617 723 7890 
Fax: 617 723 7897 
www.aberdeen.com 

© 2006 Aberdeen

 

Group, Inc. 

All rights reserved 
Month 2006

 

Founded in 1988, Aberdeen Group is the technology- 

driven research destination of choice for the global  

business executive. Aberdeen Group has over 100,000 

research members in over 36 countries around the world 

that both participate in and direct the most comprehen-

sive technology-driven value chain research in the  

market. Through its continued fact-based research, 

benchmarking, and actionable Simulation, Aberdeen 

Group offers global business and technology executives 

a unique mix of actionable research, KPIs, tools,  

and services. 
 

The information contained in this publication has been obtained from sources Aberdeen believes to be reliable, but 
is not guaranteed by Aberdeen. Aberdeen publications reflect the analyst’s judgment at the time and are subject to 
change without notice.  
The trademarks and registered trademarks of the corporations mentioned in this publication are the property of their 
respective holders. 

Appendix B:

 

Related Aberdeen Research & Tools 

Related Aberdeen research that forms a companion or reference to this report includes: 

 

The Product Innovation Agenda Benchmark Report

 (September 2005)

 

 

The Product Lifecycle Management for Small to Medium-Size Manufacturers 
Benchmark Report

 (March 2006) 

Information on these and any other Aberdeen publications can be found at 

www.Aberdeen.com

 


Document Outline