How Ethos Powers a Cloud-Native Transformation at Adobe
Author: Rohan Kapoor, Product Manager
Imagine a scenario in a software company, where hundreds of development teams have sprawled to the cloud with their own infrastructure designs, processes, and Continuous Integration/Continuous Deployment (CI/CD) pipelines. Some of the challenges they face might include:
- Manual one-off provisioning of tools
- Hard to apply rigor to standards around security and compliance
- Rising costs due to little or no formal deployment process
- Inability to make timely org-wide changes (process and technology) due to same problems being solved repeatedly
At Adobe, we’ve solved these key challenges and more with Ethos, an internally developed platform that offers developers the capability to deploy containerized applications either via a direct-access, “Do It Yourself (DIY)” Platform as a Service (PaaS) model or an opinionated “paved path” Container as a Service (CaaS) model.
Ethos incorporates 12-factor application design principles, CI/CD methodologies, friendly user interface, security controls, and DevOps processes to help improve efficiency, enhance developer productivity, and reduce development costs.
About the Ethos Platform
Ethos is a cloud-native and cloud-agnostic platform (and principles) that streamlines the development, operation, and consumption of cloud services inside Adobe. Container deployment on Ethos has increased substantially over the last few years and most microservices deployed across the Creative, Document, and Experience Clouds at Adobe currently use the Ethos platform
Beginning as an effort to “Make software development a productive and enjoyable experience at Adobe,” Ethos has helped improve developer productivity across the company, through robust documentation, automated self-serve tools, and incorporating user feedback. The Ethos team itself uses the platform to find blind spots and measure average deployment times to help optimize the overall developer experience.
Key Capabilities of Ethos
Ethos enables capabilities that would otherwise be hard to activate, such as guard-railed CI/CD, code scaffolding, GPU support, multi-cloud support, and third-party code isolation. Ethos also unifies solutions over common platforms by unlocking synergies for logging, traceability, monitoring, and other foundational services. Some of the key capabilities provided by Ethos include:
- Simple Provisioning: Ethos makes infrastructure provisioning simple by providing an abstracted way (configuration file) and user-friendly UI tools to onboard and activate a new service, which is the building block for any application. The platform also gives developers a mechanism to integrate their application with any external services of their choosing for monitoring, logging, and others.
- Code Scaffolding: Ethos expedites development time by providing a containerized “Hello World” application to get developers started on various programming languages ranging from Java, Python, NodeJS, Go, and others.
- High-Availability and Reliability: Adobe product teams to meet customer SLAs by providing services within the platform that help ensure high-availability, reliability, and resilience in case of known or unknown events. This is achieved by having a strong SRE team, built-in fault tolerance, and distribution of containers across multiple regions fronted by common API Management
- Cluster Scalability: Ethos allows Adobe developers to configure the scalability requirements of their service so the platform can automatically scale compute and networking resources to match traffic volume.
- CI/CD Pipeline Orchestration: Ethos provides Adobe developers with CI/CD tooling and pipelines to build code, run tests in development and staging environments (CI), and safely deploy a new version of their application (CD) to production.
- Diagnostics and Support: Ethos provides timely support and valuable information to Adobe product teams through automated bots and self-service tools. Best practices and training are available as well as self-service API and service documentation tooling
- Multi-Cloud Enablement: A common CI/CD process deploys Ethos and Ethos client service on industry-leading cloud hosting providers, such as Microsoft Azure and AWS, and on Adobe-owned data centers. Global API Management enables dynamic multi-cloud request routing. This is paired with common network security, management infrastructure practices to protect and scale.
- Security: With hardened containers, CI/CD pipeline scanning, runtime container security, MFA-enabled access to internal systems, operational auditing, and traceability to enable compliance with HIPAA, PCI, or any number of government regulations, as well as our own Common Controls Framework (CCF), Ethos was designed from its inception to provide a wide variety of security guardrails for product teams, eliminating the need for each team to build security controls themselves. More information about the security controls provided by the Ethos platform will be discussed in future blog posts.
Ethos High-Level Architecture and Workflow
Let’s take a closer look at the five (5) different component workflows in Ethos:
- Enable Containerized Runtimes
a. As the diagram shows, business groups across all Adobe clouds leverage Ethos to run their microservices for solutions such as Adobe Photoshop, PDF, and Adobe Experience Platform.
2. Establish Ethos Service Workflows
a. Adobe developers using corporate Git can create a service and a basic scaffolding application using the Ethos home developer portal.
b. Secrets and Docker images are pushed automatically to their respective systems.
c. The application is deployed in Docker containers and onboarded to multiple Kubernetes clusters through a robust CI/CD pipeline that deploys these containers to multiple worker nodes.
d. Once deployed, developers can diagnose issues as well as regularly monitor the health of their service using Ethos dashboards or through direct read access to key events and metrics for their namespace.
3. Encapsulate Security Components
a. The containers are “hardened” Docker images based on Adobe business rules and published standards, which are retrieved during the container build process. Adobe’s standards encapsulate containerization best practices, implementation policies, and security features. These standards are based off 12-factor application design principles.
b. Automated image scanning ensures that Ethos is not arbitrarily pushing Docker images that might have known vulnerabilities as part of the CI/CD pipeline.
c. Kubernetes RBAC is used to isolate different teams’ deployments and permissions.
4. Deploy to Kubernetes Clusters
a. The CI/CD pipeline can deploy the container images to multiple Ethos clusters (for availability).
b. The rest of the interactions are standard Kubernetes interfaces, which are covered in more detail here.
5. Manage Ethos APIs
a. Ethos API Management offers Service Registry and traffic management capabilities, including authentication, routing, throttling, distributed tracing, and analytics insights.
b. Ingress Gateway implements the controls setup in the API management layer by enforcing throttling and authentication. It also manages the publishing of the container to corporate, public, and internal networks.
What Do We Have Planned for the Near Future?
The Ethos team continuously works to improve developer tooling and other offerings. Some of the initiatives we are working on for future releases include:
- Leveraging data and research to further streamline the overall Ethos developer experience by making it simpler and more intuitive.
- Improving reliability SLAs, cloud cost targets, issue resolution and development times among others.
- Formalizing and optimizing overall developer growth path (via education and trainings).
- Launching new and modern initiatives to make our overall offerings more robust by integrating/providing serverless capabilities, modernizing our CI/CD product offering.
- Continuously optimizing our product for scale to ease onboarding of our new Adobe family members through recent acquisitions.
- Scale resiliency with AIOps to find anomalies in data, monitor logs, and enhance network performance across all cloud products.
Watch this space for more information about our Ethos platform as we cover future topics around its development and usage. Are you interested in working with us to continue to evolve the Ethos platform? We have opportunities on our team. Visit the Adobe Careers site to view available opportunities.
