SPIFFE and SPIRE At Confluent - Mohamed Omar

TL;DR

Spire simplifies workload credentials distribution and provides security isolation through trust domains, making it an ideal solution for scaling workloads in a multi-cloud environment.

Transcript

know so um does this help yeah perfect this is better okay cool uh so hey everyone uh thank you for joining uh my name is Mohammed I'm a software engineer at confluent I work with the product security team which is a part of our trust and security organization and confluent today I'll be talking about spiffy Inspire like what's our experience with ... Read More

Key Insights

• 😶‍🌫️ Confluent's infrastructure consists of multiple cloud providers, regions, gate clusters, and thousands of workloads, necessitating a scalable solution for workload credentials distribution.
• 🐕‍🦺 Spire was chosen for its ability to eliminate secret zero, automate token rotation, provide security isolation, and consolidate service-to-service authentication.
• 😤 Convincing teams to adopt Spire required building a solid case, identifying integration opportunities, and preparing a smooth adoption journey.
• 🥺 The design of Spire's topology involves evaluating security, availability, operational overhead, and bundle size concerns, ultimately leading to a multi-trust domain topology.
• 🐕‍🦺 Managing workload registration is simplified with a centralized registration service, leveraging the knowledge of workload coordinations and federation relationships.
• 👨‍💻 Confluent aims to automate infrastructure provisioning with infrastructure-as-code and explore potential enhancements like using Cloud Provider Queue Services for registration entries.

Questions & Answers

Q: Why did Confluent choose Spire for workload credentials distribution?

Confluent chose Spire because it eliminates secret zero, automates the rotation of short-lived tokens, provides security isolation, and consolidates service-to-service authentication solutions.

Q: How did Confluent convince teams to adopt Spire?

Confluent focused on building a solid case for Spire, identifying current solutions that could seamlessly integrate with it, and showcasing the benefits of improved security posture and workload scalability.

Q: How does Confluent manage workload registration in a multi-cloud environment?

Confluent uses a centralized registration service that facilitates workload registration requests to different Spire servers. This service leverages the cloud topology information and the Spire API to register workloads efficiently.

Q: How does Confluent handle the scaling and performance challenges of Spire?

Confluent designs the topology with multiple trust domains and nested deployments to ensure predictable bundle sizes and prevent database bottlenecks. The registration service helps manage registration and provides eventual consistency.

Key Insights:

• Confluent's infrastructure consists of multiple cloud providers, regions, gate clusters, and thousands of workloads, necessitating a scalable solution for workload credentials distribution.
• Spire was chosen for its ability to eliminate secret zero, automate token rotation, provide security isolation, and consolidate service-to-service authentication.
• Convincing teams to adopt Spire required building a solid case, identifying integration opportunities, and preparing a smooth adoption journey.
• The design of Spire's topology involves evaluating security, availability, operational overhead, and bundle size concerns, ultimately leading to a multi-trust domain topology.
• Managing workload registration is simplified with a centralized registration service, leveraging the knowledge of workload coordinations and federation relationships.
• Confluent aims to automate infrastructure provisioning with infrastructure-as-code and explore potential enhancements like using Cloud Provider Queue Services for registration entries.
• Performance testing and the speaker's specific contribution to the infrastructure were not covered in this presentation but could be shared in future discussions or events.

Summary & Key Takeaways

• Confluent's infrastructure spans multiple cloud providers (AWS, GCP, Azure) and hundreds of regions, with thousands of gate clusters, tens of thousands of nodes, and hundreds of thousands of dynamic workloads.

• Spire was chosen as the solution for workload credentials distribution due to its ability to scale, eliminate secret zero, and provide security isolation through trust domains and federation.

• Implementing Spire required convincing various teams within the organization, designing a topology with multiple trust domains, and building a centralized registration service for efficient workload registration.