S4E1: "Measuring Success"

Posted 2019-03-04 04:59
Modified 2019-03-05 03:41
Comments 0

On gathering and applying metrics data. Contain’s Jthan’s “compile noise”.

• Just the Tip
• Notes
• 15 Clams
• Errata
• Music

Just the Tip

• Continuing a mention from the previous episode’s Just the Tip, Paden talks about chattr (“CHange ATTRibutes”).
  • You can read more about it here.
• Bonus! He mentions (rants about) pump, a DHCP and BOOTP client.
  • I mention a different pump (as part of distcc) as well.

Notes

Starts at 13m05s.

I was drinking water. Paden was drinking Glenmorangie Original again. Jthan was drinking Buffalo Trace.

• Metrics!
  • Metrics is the measuring of data about services, etc. (with the purpose of application for optimization). e.g.:
    • CPU usage
    • uptime/availability
    • Memory usage
    • Bandwidth throughput average
    • Bandwidth usage (“bandwidth accounting”)
  • Metrics are vitally important to projected models for e.g. budgeting.
  • Gathering metrics:
    • Generally speaking, logs don’t have much value to gathering metrics with one exception: AWStats (for Apache or Nginx).
    • Jthan mentions Prometheus which has a lot of flexibility and is pretty.
    • I also mentioned Sysdig which seems to have taken a shift more towards container technology.
    • For bandwidth alone, there’s a lot of tools!
      • RRDTool has been an industry standard for a long time. One can even generate graphs from the data it collects with Cacti, or you can interact with it directly via e.g. python-rrdtool or use the RRDTool-provided python bindings, or even modify RRD data (for instance, if you wanted to redact some information or minimize your data for a report summary).
      • nmon gathers more than just bandwidth, but it does gather it as well.
      • bandwidthd has been around for ages.
      • (There are “live metrics” monitoring programs as well such as iftop and iptraf, the latter of which DOES have logging but you would need to write your own parser.)
      • Jthan also wrought his campus’ IT department’s wrath upon him with ntop-ng.
  • Metrics, in addition to budgeting approval, are good for providing feedback/insight to clients and C-level officers proving service value.
  • Metrics can also allow you to identify issues, but this is perhaps better reserved for monitoring as it’s better designed for that purpose.
  • Metrics also can allow you to identify trends — finding patterns of usage and then making projections based on said patterns (and allowing you to make “educated guesses” to maximize your resource availability/load/etc.).
  • Metrics are useful for “proving and quantifying success”; as part of our role, the better we are the more invisible we are (read through that parent thread, by the way! It has a lot of useful tips!).
    • Always remember to translate the success of your department to financial gain — servers up equals +$$$, servers down equals -$$$.

15 Clams

In this segment, Jthan shares with you a little slice of life. The title is a reference to this video. (2m16s in)

Starts at 35m15s.

Storytime! Jthan hires a student and he (Jthan) as a result is a happy little clam.

He also relates how someone seems to “just know” the answers, but knowing how to find an answer is even more important.

Errata

• We didn’t mention it on-air, but another product to check out is netdata. It’s geared more towards the “monitoring” than “metrics” side though.
• Nosbig (who has been on several of our Shitshow episodes and is a regular in our IRC) reminded us that we missed one of the biggest metrics platform out there today — Elasticsearch. Duh! Thanks, Nosbig!

Music

Author r00t^2
Categories Season Four