Modern Platform Engineering
…for the smaller company.
Welcome to AWSCQ.
As you can see this time we’re looking at Modern Platform Engineering (for the smaller company).
For this edition we’re very lucky to have the brilliant David Anderson as Editor.
David has been at the leading edge of the technology industry for twenty-five years.
He formed The Serverless Edge, and continues to work with clients and partners to prove out the thinking of his book, The Flywheel Effect. He is also a member of the Wardley Mapping community.
So without further ado - over to David.
Platform Engineering
Platform Engineering is currently experiencing its "moment in the sunshine," but many solutions are focused on giant corporations - "This is how we created a platform for our team of 10,000 engineers". Many of us do not have that problem. We have 100 engineers who would like fast flow, resilient systems and want to reduce our operational overhead.
I'm going to phrase the term "Modern Platform Engineering." If we need to build for fast flow, we should think about our architecture, we should think about easy-to-use standards & skills, and we should also think about Serverless First.
Why do we need Platform Engineering?
When we define Platform Engineering today, it's a reaction to low cloud maturity. It's costly for an engineering department to get good at the cloud.
It’s quite sobering when we see this in a survey. We should be getting a lot more value from the cloud. 8% of respondents qualify as highly Cloud mature and 91% of respondents feel they are wasting money in the cloud. Cost is your Architecture, so I believe we are struggling with technical strategy and cloud enablement.
https://www.hashicorp.com/state-of-the-cloud
Serverless should have made this easier.
But 10 years ago, AWS released Lambda - which makes the cloud a lot easier.... Why the disconnect?
Serverless is still not understood. It’s more than Lambda and (I believe) a different way to write software. Unfortunately, the ever-hype-fueled Tech industry just loves drama. Engineers are building incredible systems with Serverless with very low Total Cost of Ownership and fast flow. Good engineering is hard, not just because of the code, but because it’s hard to design systems.
https://www.serverless.com/blog/why-many-engineers-dont-understand-serverless
Platform Engineering is not what you think it is.
But we still need to help engineers do the right thing. This should be the aim of Platform Engineering. But, similar to DevOps, Platform Engineering has become another form of System Administration that can create barriers for developers. Platform Engineering must help, not hinder. I believe that removing the operational overhead creates more space to think about system-level concerns.
https://www.jeremydaly.com/serverless-platform-engineering/
In reality, we are always building a platform - but often, the Cloud team will try to recreate what the Cloud Provider does. Bob and his team of 4 are basically competing with AWS on "who can build a better platform." We do need to create a collection of conventions to accelerate delivery and improve flow. The platform's customer is the company, not the developer.
Your people are your platform.
Team Topologies also cover this in their Platform Manifesto:
Platform Manifesto
teams & interactions over tools & functionality
adoption & engagement over mandates & standards
rich customer experience over technical prowess
open to change and collaborate to discover user needs
unblocking internal customers via self-service patterns
aiming for superlinear impact with sublinear growth
https://teamtopologies.com/platform-manifesto
A good standard will help guide your decision-making. Charity Majors talks about the requirement in this old post from 2016 - it hasn’t changed.
https://charity.wtf/2016/05/31/wtf-is-operations-serverless/
I like to use the Well-Architected Framework. I prefer our SCORP method - I have helped many companies with this mechanism and it’s very powerful. We need to make standards easy and socialise the discussions and conversations.
Create a safe space and talk about how we can improve.
https://theserverlessedge.com/team-engineering-with-scorp/
Some Case studies.
Of course, the hard part is pulling all this together. Lee Gilmore describes it very well. I will be biased and say you need great technology leaders.
https://blog.serverlessadvocate.com/embracing-serverless-in-a-1951-founded-enterprise-a64c71c5459c
And (of course) the book also discusses this in detail. Code is a liability, but if you can shape your organization to modernize with the cloud, you will find that value that so many of the “State of the Cloud” respondents are struggling with.
https://itrevolution.com/product/the-value-flywheel-effect/
I’ve also just talked about this at AWS re:Invent 2024 in a talk titled:
SVS208: Balance consistency and developer freedom with platform engineering – Breakout
Learn how platform teams can provide opinionated security, cost, observability, reliability, and sustainability patterns while maintaining developer flexibility.
Just remember, “Platform Engineering” is not about writing terraform for Developers; it’s about creating Fast Flow in your organization. I would say your Engineers need you to:
Lower the Operational Overhead of running workloads - preference for managed services.
Create Fast Flow by making it easy to be Well-Architected - sensible defaults and toolsets.
Improve Discoverability across teams - can I find out information without having to book a meeting?
Get out of my way - let the engineers run their infrastructure, but help with the standards.
Get your Value Flywheel turning and focus on problems that will make your company even more successful.
And that’s all from us for another AWSCQ.
A huge thanks to David for a fantastic edition!
Before you go be sure to give our sponsors a click!
AWS Community Summit Events and AWSCQ are only possible with their generous support.
