Back to lesson

Tracking Improvement

Slide 1: Tracking Improvement

On-screen

Tracking Improvement

Deployment metrics and incident trends

Narration

Anna: If you've ever wondered whether your quick fix actually solved anything or simply moved the problem somewhere else, this module is for you. We've all attended post-mortems where action items pile up, yet nobody checks whether those items had any real impact.
Greg: That's why we track deployment metrics and incident trends. Numbers give us an unbiased view of progress. We'll look at tools like GitHub Insights, JIRA reports and monitoring dashboards that make collecting this data easier than it sounds.
Anna: We'll also talk about establishing a baseline before changes and how long it typically takes to see meaningful trends. By the end you'll know which metrics matter, common pitfalls to avoid and how these measurements help teams improve week after week.

Slide 2: Why metrics matter

On-screen

Why metrics matter

  • Show if fixes actually work
  • Reveal patterns in incidents
  • Support business cases for resources
  • Stop endless debates by showing trends

Narration

Anna: Why metrics matter focuses attention on a concrete part of the work. Show if fixes actually work, Reveal patterns in incidents, and Support business cases for resources.
Greg: In practice, ask who owns the work, what evidence proves it happened, and what handoff comes next. Use the supporting details as a checklist: Reveal patterns in incidents; Support business cases for resources; Stop endless debates by showing trends.

Slide 3: Key data points

On-screen

Key data points

  • Deployment frequency and lead time
  • Change failure rate and MTTR
  • Number and severity of incidents
  • Use GitHub Insights, JIRA and dashboards

Narration

Anna: Let's start with the basics—DORA metrics. Track deployment frequency, lead time for changes, change failure rate and mean time to recovery. These reveal how smoothly code travels from commit to production.
Greg: To gather them, use GitHub Insights or your CI/CD dashboard for deployment stats and JIRA or ServiceNow for incident logs. Establish a baseline before you roll out new processes so you can see the effect over time.
Anna: Good values differ by organisation, but watch for high failure rates or long recovery times. They often hint at inadequate testing or rushed releases. Connect these numbers to user experience: slower recovery means customers stuck on error pages longer.
Greg: Don't forget incident counts and severities. Plot everything on a timeline. If deployments spike but incident severity climbs with them, it might be time to revisit your quality gates rather than celebrate extra releases.

Slide 4: Using the insights

On-screen

Using the insights

  • Compare before and after major changes
  • Share trends in post-mortems
  • Adjust processes based on evidence
  • Build quarterly summaries for long-term progress

Narration

Anna: Once you've collected a few sprints of data, compare it to your baseline. Did your deployment lead time shrink? Are rollbacks less frequent?
Greg: If the numbers improve, highlight them in a quick dashboard demo during your post-mortems. Showing a trend line dropping from two‑week deployments to two‑day cycles can convince leadership to keep investing in automation.
Anna: When the metrics move the wrong way, dig into the timeline around each spike. Maybe a new testing tool slowed the pipeline or a Friday release pattern correlated with more incidents. Invite the team to suggest fixes rather than assign blame.
Greg: Present findings to management in plain language: "Our recovery time increased last month, likely due to rushed hotfixes. We propose adding a staging step." Real data helps secure approval for those changes and keeps everyone accountable.

Slide 5: Key takeaway

On-screen

Key takeaway

Metrics guide improvements and prove what works.

Narration

Anna: Metrics turn vague promises into measurable progress. They show whether you're really improving or just churning through tasks.
Greg: Keep your dashboards visible and review them regularly. Patterns often emerge after a month or two, so be patient. Remember, correlation doesn't imply causation—but it sure waves its arms to get your attention.
Anna: Watch for gaming. If someone deploys "fix typo" fifty times on Friday just to boost counts, you're measuring the wrong thing. Balance speed metrics with quality indicators like change failure rate.
Greg: Use these numbers to justify resources. Showing a 40 % drop in recovery time helped one team secure budget for an extra SRE. With clear data, you can pivot quickly when things don't work and celebrate when they do.