Is your investment in AI paying off... or just generating more sh*tposts?

Is your investment in AI paying off... or just generating
more sh*tposts Michael Stahnke VP Engineering at Flox (flox.dev) @stahnma

@stahnma ⚠ I am a breathing time machine

@stahnma Don’t let cement harden around your ideas

@stahnma What is ROI?

@stahnma R Return

@stahnma O On

@stahnma I Investment

@stahnma Is this better with AI?

@stahnma To know this, I have to know something about
current ROI of Engineering

@stahnma Spoiler: We know little as an industry

@stahnma

@stahnma None of this is ROI

@stahnma ROI comes from an obsession for efficiency

@stahnma Edu

@stahnma

@stahnma 1. …

@stahnma

@stahnma Mental Model for Metrics Baseline DevOps Metrics I don’t
feel elite. Move Quickly with Confidence Throughput Quality Normalization Fungibility Rollbacks (or forwards) Deployments per Week Deploys / Dev / Week Successful deploys Narrow vs Wide Time to 5th PR Changes Understood Start Status Questions People Metric Metric Metric Themes Eng Metrics Framework Talk: https://www.youtube.com/watch?v=A1SCGGbHgZ0

@stahnma // it’s a better than average proxy metric developer_productivity
!= roi

@stahnma deploys / dev / week

@stahnma ROI at large

@stahnma This is actually easier the more zoomed out

@stahnma Investment

@stahnma Sum of payroll, tools, facilities, travel

@stahnma Return: Profit

@stahnma

@stahnma Difficulties

@stahnma Return for a specific team, area, feature

@stahnma Enter AI Tooling

@stahnma

@stahnma cyn·i·cism noun an inclination to believe that people are
motivated purely by self-interest; skepticism

@stahnma hating is easier than trying

@stahnma All developer tools come down to two promises

@stahnma Save time Save money

@stahnma if ( time == money ) { return money
}

@stahnma Other Returns

@stahnma Number one feature of AI:

@stahnma Number one feature of AI: shitpost enablement 💩

@stahnma Prompt: Generate me a meme image with the theme
of "the most effective use of AI is to make memes faster and in higher quality"

@stahnma But, AI kind of sucks at coding.

@stahnma Hating is easier than trying

@stahnma What is AI engineering good at?

@stahnma Writing code

@stahnma For some values of code

@stahnma What is it less good at?

@stahnma Software Engineering.

@stahnma Coding ✅ Software Engineering ⚠

@stahnma ‣ Writing code ‣ Testing ‣ Debugging ‣ Observability
‣ Understanding Code ‣ Software Architecture ‣ Security ‣ Work breakdown ‣ Prioritization, dependencies ‣ Dealing with ambiguity ‣ Reliability, delivery accountability ‣ Evaluating Tradeoffs ‣ Delivering Feedback ‣ Seeking and receiving feedback ‣ Effective communication ‣ Knowledge Sharing ‣ Teamwork ‣ Relationship building ‣ Handling disagreement ‣ Decision making ‣ Driving alignment ‣ Process thinking ‣ Facilitation ‣ Mentoring ‣ Business acumen ‣ Strategic work ‣ Product Thinking

@stahnma Coding isn’t the hardest part.

@stahnma Your value is not time spent in your editor

@stahnma

@stahnma Public CircleCI Software Engineer Career Matrix

@stahnma More senior-level work doesn’t require as much coding The
more senior the engineer, the less they like AI code

@stahnma What is AI good at?

@stahnma Finding Ambiguity

@stahnma It’s a super power

@stahnma One of my favorite prompts: What ambiguity remains?

@stahnma Learning new tech

@stahnma I’ve never worked with tailwind and HTMX before, could
you walk me through making a landing page?

@stahnma Finding new tools/libraries

@stahnma Tool trade-off explanations

@stahnma is there something like postgREST for mysql compatible databases?

@stahnma

@stahnma Understanding code: what would make this easier to test?

@stahnma Drives Product Thinking… if you start with specifications

@stahnma Improved documentation?

@stahnma Work Breakdown

@stahnma Rubber ducking Wow, this one time, I actually meant
ducking.

@stahnma Saving Time

@stahnma How much time can you save with a few
tab-completes?

@stahnma How much can you save by having simple functions
written?

@stahnma How much time do you spend figuring exact properties,
syntax, methods, API calls?

@stahnma How much time is spent attempting future-proof code?

@stahnma

@stahnma How much time is spent on “technical debt”?

@stahnma How much “technical debt” is created due to attempting
to future-proof?

@stahnma Code you don’t like is not technical debt

@stahnma “I don’t like AI Code because it’s not how
I would do it.”

@stahnma If you’re using AI exactly like you did traditionally
software engineering, you won’t save much time.

@stahnma Saving Money

@stahnma

@stahnma talks.stahnma.us/roi

@stahnma In Closing

@stahnma Summary • If you’re not using AI tools, you’re
may get lapped by those who are. • If you’re only using them, you probably don’t know what you’re doing. • If you’re saving time, you’re probably recovering cost. • If you’re learning and leveling up, you’re doing really well.

@stahnma

@stahnma Deterministic vs Probabilistic outcomes

@stahnma

@stahnma When the code has unknown quality and risk levels,
we need to treat it as hostile.

@stahnma Control points need to be created and developed

@stahnma Developers have been kingmakers for ~15 years

@stahnma Coming soon.

@stahnma

@stahnma CircleCI Career Matrix (Public) ROI Calculator References ROI of
Platform Engineering Talk Metrics Framework Talk Speakerdeck Let 1,000 Flower Bloom This deck

@stahnma We’re hiring We’re post-geographic.

Is your investment in AI paying off... or just ...

Is your investment in AI paying off... or just generating more sh*tposts?

More Decks by Michael Stahnke

Other Decks in Technology

Featured

Transcript