What data is collected for continuous improvement projects?
Wisconsin’s Continuous Improvement (CI) Program requires all reported projects to collect data that demonstrates the impact of their improvement. The program categorizes project metrics into six metric categories: Cost, Efficiency, Engagement, Outcome, Quality, and Stakeholder Experience. When capturing data for an improvement, teams must collect comparable data before and after implementation. Projects may collect different types of metrics based on the improvement being made, with preference given to the program’s four core impact metrics: staff hours repurposed (hours), cost savings (dollars), lead time reduced (days), and process steps eliminated (steps), when applicable. For more information on what data to collect and how to measure it, see the CI Metrics Best Practices and Examples document.
Can we use estimates instead of hard data?
No. You must collect real data to verify that an improvement occurred. It's not enough to say, “We expect lead time will be reduced by 10 days.”
Estimates may be used only to fill in gaps where complete data isn't available. However, whenever possible, use actual data to calculate averages and outcomes.
Can we submit improvements without hard data if we know something has improved?
No. We require at least one verifiable metric to demonstrate the improvement’s impact. If you're having trouble collecting data, reach out to your CI contact or the DOA Strategic Management and Planning Team. We can help you find practical ways to measure impact.
What if our improvement isn’t one of the core impact metrics?
Still report. Not all improvements need to result in cost savings, hour reductions, process steps eliminated, or days reduced. Projects that improve service quality, staff experience, customer satisfaction, or reduce errors are equally important and can be categorized under one of our six metric categories. Within our project form and data collection resources, you'll see a wide variety of project metrics are accepted. If your data set isn't represented, the Other field is available for write-ins.
How much data should I collect?
You should try to collect 30 data points both before and after you've improved. However, this is just a rule of thumb. If a process occurs monthly, it doesn't make sense to collect data for 2 and a half years just so you can get a large sample size.
Will staff members be cut if we report on hours repurposed?
Since we began publicly reporting improvement data in FY17, including repurposed staff hours, there have been no reported instances of programs reducing staff as a result of a process improvement.
Wisconsin’s improvement culture is rooted in innovation, service, and stewardship. Our focus is on implementing smart, sustainable solutions that allow our workforce to do more with existing resources, not less with fewer people. All reported improvement metrics must include how repurposed hours are being used, such as addressing backlogs, expanding services, or improving turnaround times. This ensures time savings are directly tied to public value, not job cuts.
What's the difference between hours repurposed and lead time reduced?
Hours repurposed only considers the amount of time, in hours, employees are actively working on a process, which is also known as "touch time". Lead time reduction considers the wait time between tasks or the delivery of service and is represented in days.
Do I need to collect data for the entire year to annualize my metrics?
No. It's perfectly reasonable to collect data for a single month or quarter and then extrapolate the data over an entire year. However, it's important to make sure you're comparing "apples to apples". If you know the data will differ depending on the time of the year, you should account for this in some way. For example, if DNR employees know that hunting license purchases typically occur in the fall, it wouldn't make sense to only collect data in the spring.
Should I use calendar days or business days when calculating lead time?
Use calendar days for all lead time calculations. While business days would be slightly more accurate, it requires more effort than it is worth to calculate.
Should I report hours repurposed * average hourly wage in the annual cost fields?
No. Only report hard dollar, ongoing costs in the annual cost fields. Employee time is an indirect cost and causes confusion when the numbers are aggregated.
When should I start collecting data?
It depends on the metric and the process.
For lead time, start collecting once you’ve clearly defined the process boundaries (i.e., the start and end points). For hours repurposed or cost savings, wait until you’ve identified specific changes you plan to make; this ensures your baseline is targeted and meaningful. You may collect data on all process steps, but that can be time-consuming and unnecessary unless you need that level of detail.
Can I use more than one metric?
Yes. In almost all scenarios, reporting on multiple metrics is encouraged.
What about other important metrics and outcomes?
As long as a metric is aligned with one of the six metric categories, it can be used for reporting improvement initiatives and their impact. While not every metric will be aggregated, they are helpful to include in your project narrative and have the potential to be aggregated at a later point.
Do I need to have all the data before digging into the problem and coming up with solutions?
No. While this can be helpful in some scenarios, it's not always required. Often, teams will identify root causes and develop solutions while collecting data. Waiting for the data to come in is only necessary when the data will help you better understand your root causes.
I've taken the training and read through the metrics guide, and I'm still confused. What can I do?
If you have a specific question, we recommend reaching out to your agency’s continuous improvement contact or DOA's Strategic Management and Planning Team for more information. Some agencies also have coaches who can provide you with more detailed guidance and advice.