Where can I find definitions for all the continuous improvement metrics?
We've provided detailed definitions and examples in our Metrics Guide (coming soon). This document is intended to serve as a comprehensive guide that you can reference as needed. It's not expected to be read from beginning to end.
Can we use estimates instead of hard data?
No. It is not enough to say "We expect the average lead time will be reduced by 10 days". You need to collect data to verify the improvement. Estimates should only be used to fill in the holes in your data.
How much data should I collect?
You should try to collect 30 data points both before and after you've made an improvement. However, this is just a rule of thumb. If a process occurs monthly, it doesn't make sense to collect data for 2 and a half years just so you can get a large sample size.
What's the difference between hours repurposed and lead time reduced?
Hours repurposed only considers the amount of time employees are actively working on a process, which is also known as "touch time". Lead time reduced considers both the amount of time employees are working on the process and wait time. Therefore, if you reduce the number of hours required to complete a task, you're usually reducing lead time as well.
Do I need to collect data for the entire year to annualize my metrics?
No. It's perfectly reasonable to collect data for a single month or quarter and then extrapolate the data over an entire year. However, it's important to make sure you're comparing "apples to apples". If you know the data will differ depending on the time of the year, you should account for this in some way. For example, if DNR employees know that hunting license purchases typically occur in the fall, it wouldn't make sense to only collect data in the spring.
Should I use calendar days or business days when calculating lead time?
Use calendar days for all lead time calculations. While business days would be slightly more accurate, it requires more effort than it is worth to calculate.
Should I report hours repurposed * average hourly wage in the annual cost fields?
No. Only report hard dollar, ongoing costs in the annual cost fields. Employee time is an indirect cost and causes confusion when the numbers are aggregated.
When should I start collecting data?
It depends. For lead time, you'll probably want to start collecting data after you've identified the scope of your project (i.e. where your process begins and ends). For hours repurposed and all cost metrics, you'll often want to wait until you've identified specifically where you plan to make improvements. While you can collect data on all process steps, this can be very time consuming and may not be necessary.
Can I use more than one metric?
Yes. In almost all scenarios, reporting on multiple metrics is encouraged.
What about other important metrics and outcomes?
There are many other metrics that might be important depending on your agency or division. When relevant, these metrics should be reported in the ‘Outcome’ section on your Project Report. While these metrics will not be aggregated, they are helpful to include in your project narrative.
Do I need to have all the data before digging into the problem and coming up with solutions?
No. While this can be helpful in some scenarios, it's not always required. Often teams will identify root causes and develop solutions while collecting data. Waiting for the data to come in is only necessary when the data will help you better understand your root causes.
I've taken the training and read through the metrics guide, and I'm still confused. What can I do?
If you have a specific question, we recommend reaching out to your agency’s continuous improvement contact or DOA's Strategic Management and Planning Team for more information. Some agencies also have coaches who can provide you with more detailed guidance and advice.