Metrics are essential to a healthy business. For engineering teams, these metrics will empower you to measure effectiveness with the intent of improving productivity and driving peak performance.
Choosing the right metrics that incentivize good habits and provide meaningful insight can be challenging.
The following metrics are the ones I used successfully during my tenure at a Fortune 100 Company where I was responsible for all engineering productivity projects for a 1,200-employee engineering department with an annual budget of $115 million.
The first two metrics measure the productivity of your Sustaining Engineering team; the next two metrics measure your R&D (NPI) team’s effectiveness. These two groups require different metrics because they have different goals and outputs.
The last two metrics measure the effectiveness of your physical product designs.
Measuring Sustaining Engineering Productivity
The Sustaining Engineering team is the portion of Engineering that focuses on supporting your current products. These engineers typically deal with design tasks driven by warrantee issues, cost reduction activities, manufacturing support, and engineering custom applications based on your standard product.
Metric 1 – measures the cost of supporting your existing product lines relative to the revenue that such products generate.
Engineering cost = (Budget or Expenditures for the year) minus discretionary spending, (such as R&D and continuing education)
Revenue = Company or Division revenue for the applicable year
When you compare this year’s ratio to last year’s ratio, the difference is your productivity gain or loss.
So, if your Engineering budget (minus discretionary items) was $39 million and your revenues were $780 million, then your engineering cost ratio, this year is 5%. If the ratio was 6% last year your engineering productivity gain was 16.66% [(last year ratio)-(this year ratio)]/(last year ratio) = (6-5)/6.
Why do we subtract discretionary spending? Discretionary spending is subtracted because if you don’t get it out of the measurement from the start, it will be eliminated by management with the intent to reduce cost and make the metric look better.
For example, it is easy to say “we are eliminating all outside training.” This will reduce cost indeed, but it will not increase your team’s productivity. In fact, in the long run, it may hurt it.
Metric 2 – measures the cost of your engineering team relative to what they accomplish.
Engineering cost = (Budget or Expenditures for the year) minus discretionary spending, (such as R&D and continuing education)
Measure of Deliverables = Number of unique projects or Quantity of items sold
Again, compare this year’s ratio to last year’s ratio to determine your productivity gain or loss.
For example, if you make custom valves for the Oil and Gas industry and last year you sold one project to Exxon Mobile containing 5 valves at a particular flow and pressure, then 2 projects to Shell each with 4 valves, but at a different flow for each project, you sold 3 projects with unique conditions OR you sold 13 valves that engineering had to support.
If your Engineering budget was $39 million, your engineering cost per “engineered” project is $13 million OR your engineering cost per valve delivered is $3 million. Again, you compare this to last year’s number to see if you are improving.
I recommend the “per unique project method” if you are offering mostly custom equipment and the per item method if you are making more standard or off-the shelf equipment or components.
The engineering team may argue that they have no control over revenues or what gets sold but, they do impact the success of the product in the market place. So, these general links to revenue/sales should not be so easily dismissed.
In the case of companies that do not have two separate engineering teams, you have to estimate the portion of expenditures dedicated to NPI and subtract that from the total. We will identify R&D/NPI productivity metrics next.
Measuring R&D/NPI Engineering Productivity
R&D is often considered discretionary spending because, in fact, a company could decide not to innovate and live off its current product line. I have known companies that have done this for several years in a row. However, for companies that have a robust R&D/NPI practice this expense, even if discretionary, can be significant and companies struggle to figure out if they are truly productive in their NPI organizations.
NOTE: The next two metrics use R&D/NPI costs as the basis for the metric. R&D/NPI costs are the cost of conducting the development process which include items like engineers’ salaries and benefits, cost of testing, any outside expert consulting used to support that development project, etc.
Metric 3 – measures the cost of R&D relative to the value that the new design is expected to deliver to your customers.
Consider how your customers use your products and what is of value. For example, if you develop electric motors, your customers most likely value your product by the number of kilowatts your motor puts out. In this case, the metric would be R&D cost/ kilowatt (kW) output of the motor or motors being developed; if you make a compressor it would be R&D cost/CFM*Pressure rating of the compressor being developed.
Think about developing a compressor family, same basic frame, similar rotating components, but it provides different behavior at different rotating speed, so you design with a falter peak efficiency curve. In this case, you get several compressors with just a little more effort than if you developed just one, hence your R&D productivity goes up. (You would divide the cost by the number of compressors in the family.)
You can also use this metric for components. If the R&D project is focused on just a section of the product, say a new stator for the motor, then the kW output is multiplied by the fraction of the total cost of the product that section represents.
Metric 4 – measures the cost of your R&D effort relative the potential financial gain it will bring to your company.
For example, if you are developing compressors and you expect each compressor will sell for $500,000 and your R&D team spent $20 million developing that compressor line, you have an R&D price productivity of 40.
It means that you have to sell X number of units (40 in this case) to cover the R&D cost. Obviously, the lower the better. You can compare this to your Sales growth plan for that product, note when X number of units are expected to be sold and compare that to prior R&D projects to see if the payback period is better or worse.
NOTE: This is not a real payback period because for that, you would need to know your profit margin, but, at the time you are taking this measurement, predicting the margin is difficult at best. This will be more of a post R&D metric based on hitting cost-targets and the subject of a future blog post, stay tuned. Also, for some long development cycles, lasting years, these metrics can still be used by scaling the denominators by the estimated percentage complete on any particular fiscal year.
Measuring the Cost Effectiveness of R&D (NPI) Designs – Are Your Components Cost Optimized?
We have talked about the productivity measurements that will help you gauge the effectiveness of your engineering teams and drive decisions that will make your team more effective. Now I’d like to discuss two metrics for measuring the cost effectiveness of R&D (NPI) designs.
The reason I’m focusing on R&D designs is because measuring potential savings due to redesigns of an existing product is easy. You take the current cost, and subtract the expected new cost. The difference is your expected savings from your redesign efforts.
But, with an R&D (NPI) project, there is no current cost. It’s a new part. How do you know whether or not your design is cost effective?
In those cases, you often use a rule of thumb to gauge cost effectiveness. You compare it to a similar part that does a similar thing and is about the same size on another project and from that you can see whether it is cost effective or not. But that’s just eyeballing it and it’s hard to quantify cost effectiveness when you are eyeballing it.
Additionally, you want to give your engineers the freedom to generate green field designs, in which case a new part may look nothing like the old part.
That’s why I recommend the following two principle metrics, which can be used in conjunction, to measure the cost effectiveness of your R&D (NPI) designs.
Metric 5 – measures the cost effectiveness of a new component relative to a similar measure for a prior component that served the same function.
The designs need not be similar but to be a fair measurement, they must serve the same purpose.
Using a full function cost estimating tool, you can obtain a cost estimate of your part or functional sub-assembly before you make it. Once you have this estimate, divide by the weight of the part or sub-assembly. Compare this ratio to the average of prior similar class designs and see if is higher or lower. If the ratio is higher, then, your design is less effective.
For example, you are designing the turbine section for a turbo charger intended for a 1.4 littler engine. Your company has never had a turbo this small. The smallest turbo you have currently is for a 2.0 liter engine. You don’t have a “similar” cost for comparison to see if you are in a reasonable cost vicinity. So, what do you do? How do you determine cost effectiveness?
Using your cost estimating tool, you predict the cost of your new intended design for the turbine section of the 1.4 liter turbo is $100 and the weight is 5 lbs., so your cost to weight ratio is 20.
By contrast, if the same sub-assembly weighs 8 lbs. for the 2.0 liter engine and when you estimate the cost of turbine section for the 2.0 liter engine turbo, using the same cost estimating tool, it turns out to be $120, then the cost to weight ratio is 15 (Cost effectiveness = (15-20)/15 = – 33%).
In this case, your new design is NOT as cost effective as your current design and perhaps there is more design work to be done.
This of course breaks down if you are designing products with a particular emphasis on weight. If you design aircrafts for example, you are constantly lowering weight and you may want to pay more for less weight, which then turns this metric on its head and in which case you should use the next metric.
Metric 6 – measures the cost of your new component relative to its value to the market regardless of its physical qualities.
Simple designs are rewarded by this metric.
What you do in the instance where you want to get rid of weight and are willing to pay more for a lighter component is measure the cost of your new component relative to the value that it brings to the market and compare that to older designs you’ve done before. This is similar to Metric 3 above.
For example, if you are developing an auxiliary power unit (APU) for an aircraft, you measure the predicted cost of the APU (the product cost this time, not the development cost) and divide that by the power output of the APU in kilowatts (kW) (because that’s how your customers will value that product) to get the cost/kW.
Then take that ratio and compare it to the ratio of prior APU designs to see whether or not you are improving.
This metric works for both components and entire products. I want to make this point because companies usually are not designing a whole new product. They are re-designing a component of it to make it better or more efficient or improve upon it in some other way.
R&D efforts are often geared towards a portion of the product not the whole product, but you can use this metric on an entire product or portions of the product and compare with prior designs to see if your cost effectiveness is getting better or worse.
For example, if you are developing a 70 kW APU and the predicted cost (using your cost estimating tool) of the support frame is $700, then your frame cost per kW is $10. You then calculate your existing 90 kW unit and 45 kW unit support frame cost per kW are $11/kW and $15/kW respectively. This gives you an average of $13/kW for your existing designs, therefore your new design at $10/kW, is an improvement.
Some companies may prefer to use the lowest $/kW ratio for comparison rather than the average. In this particular case, the new design is still an improvement.
Although not all-encompassing, a sub-set of these six metrics can be used in most industries and across most products to give you the visibility you need to drive your engineering organization to peak performance.
Use them not only to look good but also to help your Sustaining and R&D (NPI) engineering teams make better design decisions every day and quantify their success.
Tell Us About Your Experience
Are you pushing your engineering teams forward in measurable and meaningful ways? What metrics are you using? Tell us about it in the comments.
Want to Learn More?
Check out our resources for Design Cost Analysis.