Introducing DevOps for Developers

发表于 更新于 分类于

Introducing DevOps for Developers

Introduction

An introduction to DevOps for developers and it’s purpose in software development.

给开发人员介绍 DevOps 及其在软件开发中的作用。

To enter a career in DevOps, it is helpful to have experience in both software development and IT operations. This may include experience with coding and scripting languages, as well as experience with system administration and infrastructure management.

要将 DevOps 作为职业，有软件开发和 IT 运营方面的经验是有帮助的。这可能包括对编码和脚本语言的经验，以及对系统管理和基础架构管理的经验。

Having a strong understanding of collaboration and communication tools and methodologies, such as Agile and lean principles, can be beneficial. It is also important to have a passion for continuous learning and staying up-to-date with the latest technologies and best practices in the field.

具有对协作和沟通工具和方法论的深刻理解，例如敏捷和精益原则，可能是有益的。重要的是要对持续学习和了解该领域的最新技术和最佳实践保持热情。

Understanding FIRST, tools SECOND. DevOps is a philosophy, not a technology.

What Exists Today

  • Poorly Defined

    • Because DevOps is a broad and somewhat abstract concept, it can be difficult for companies to define and implement in a concrete and consistent way.

      由于 DevOps 是一个广泛且有些抽象的概念，因此公司很难以具体和一致的方式定义和实施它。

  • Cuture Challenges

    • Additionally, because DevOps involves cultural shifts within an organization, it can be challenging for companies to adopt and integrate into their existing processes and ways of working.

      此外，由于 DevOps 涉及组织内部的文化转变，因此公司很难采用并将其整合到现有的流程和工作方式中。

  • Missing Communication

    • A lack of clear communication and collaboration between teams can lead to errors and other problems that can compromise the quality of the solutions being produced.

      团队之间缺乏明确的沟通和协作可能会导致错误和其他问题，从而危及正在生产的解决方案的质量。****

  • Suboptimal Solutions

    • Developers and IT professionals may not be working together as efficiently and effectively as possible, which can result in suboptimal solutions that do not meet the needs of the business or the end users.

      开发人员和 IT 专业人员可能没有尽可能有效地合作，这可能会导致不符合业务或最终用户需求的次优解决方案。

  • Start from Scratch

    • Forget what you’ve seen about DevOps.
    • Forget what you’ve heard about DevOps.
    • Forget what you’ve done about DevOps.

What is DevOps?

The goal of DevOps is to improve the speed and reliability of software releases by automating parts fo the software development and deployment process, and by fostering a culture of collaboration and experimentation among teams.

DevOps 的目标是通过自动化软件开发和部署过程的一部分，并通过在团队之间建立协作和实验文化来提高软件发布的速度和可靠性。

What DevOps is NOT

DevOps is not a specific tool or technology. It is not a job title or a job description, and it is not a standalone practice that can be implement in isolation.

DevOps 不是一个特定的工具或技术。它不是一个职位或职位描述，也不是一个可以独立实施的独立实践。

What DevOps IS NEVER

Tools or Technologies!

DevOps is a Philosophy

DevOps is a philosophy or mindset that emphasizes collaboration, communication, and integration between software developers and IT operations teams.

DevOps 是一种强调软件开发人员和 IT 运营团队之间协作、沟通和集成的哲学或思维方式。

Define your DevOps

We need to define what DevOps means uniquely to us before we can properly harness its power for an organization.

在我们能够为组织正确利用其力量之前，我们需要定义 DevOps 对我们来说意味着什么。

Three W’s of Strategy

  • The three W’s of strategy are:

    • Why - reasons and motivations behind the strategy.

      为什么 - 策略背后的原因和动机。

    • What - goals and objectives that the strategy is intended to achieve.

      什么 - 策略旨在实现的目标和目标。

    • Who - people who are benefiting from this.

      谁 - 正受益于此的人。

    Together, there three W’s help to provide a clear understanding of the strategy and what it aims to accomplish.

    这三个 W 有助于清楚地了解策略及其旨在实现的目标。

The Why, What & Who

  • Why do we need DevOps?
  • What do we accomplish with DevOps?
  • Who do we help with DevOps?

Why do we need DevOps?

DevOps aims to help organizations rapidly and reliably build and deploy software by automating and monitoring the process from end to end.

DevOps 旨在通过自动化和端到端的过程监控，帮助组织快速可靠地构建和部署软件。

Who do we help with DevOps?

DevOps helps organizations and teams improve their ability to deliver value to their customers and users.

DevOps 帮助组织和团队提高了向客户和用户提供价值的能力。

Setting Dev Standards

  • Standards vs. Rules

    • Developers don’t like rules but will adhere to standards for the greater good of the product.

      开发人员不喜欢规则，但会遵守标准，以便更好地为产品服务。

    • DevOps standards are guidelines or best practice that organizations can follow to implement standards successfully. These standards typically cover areas such as communication, collaboration, and automation, and can help organizations to improve their software development process and deliver better products to their customers.

      DevOps 标准是组织可以遵循以成功实施标准的准则或最佳实践。这些标准通常涵盖沟通、协作和自动化等领域，可以帮助组织改进其软件开发流程，并向客户交付更好的产品。

  • Automation Standards

    • Automating the software development and deployment process, using tools such as continuous integration and delivery(CI/CD).

      使用诸如持续集成和交付（CI/CD）之类的工具自动化软件开发和部署过程。

  • Planing Standards

    • Using agile software development methodologies, such as Scrum or Kanban, to increase collaboration and flexibility.

      使用敏捷软件开发方法，例如 Scrum 或 Kanban，以增加协作和灵活性。

  • Quality Standards

    • Implementing strict version control and configuration management processes to ensure that software is properly tested and deployed.

      实施严格的版本控制和配置管理流程，以确保软件经过适当的测试和部署。

  • Communication Standards

    • Encouraging open communication and collaboration among all members of the development team, including developers, IT operations professionals, and business stakeholders.

      鼓励开放的沟通和协作，包括开发人员、 IT 运营专业人员和业务利益相关者在内的开发团队的所有成员。

  • Improving Standards

    • Using metrics and data to track and improve the performance of the software development process.

      使用指标和数据来跟踪和改进软件开发流程的性能。

Technical Design Documents

  • The Power of TDDs

  • Technical Design Document

    • A technical design document(TDD) is a document that describes how a piece of software will be implemented and how it will work. It typically includes detailed information about the system’s architecture, modules, interfaces, and data structures, as well as information about any external dependencies or required resources.

      技术设计文档（TDD）是一种描述软件将如何实现以及它将如何工作的文档。它通常包括有关系统架构、模块、接口和数据结构的详细信息，以及有关任何外部依赖项或所需资源的信息。

  • Your Fist TDD

    • There is no standardized way to writing documentation. The best advice I’ve ever been given was focus only on the specific issue or problem.

      没有标准化的撰写文档的方法。我得到的最好的建议是只关注特定的问题或问题。

    • Ask yourself what the problem IS and tell the story around that. Set proper boundaries if you need to and include an “out-of-scope” section when you discover items outside of the current issue problem resolution path.

      问问自己问题是什么，并围绕这个问题讲述故事。如果需要，请设置适当的边界，并在发现当前问题解决路径之外的项目时包含“范围外”部分。

    • Apply existing common practices to problem solving that the organization has agreed upon. In this case, we can use the “Three W’s of Strategy” to help solve our problem.

      解决问题时使用组织已经达成一致的现有常见做法。在这种情况下，我们可以使用“三个 W 的策略”来帮助解决问题。

    • Ask yourself:

      • What problem are we trying to solve?(you should always have this)
      • What is the current process?(optional, though normally helpful)
      • What are the requirements?(optional, can help with scope)
      • How do we solve this problem?(you should always have this)

Writing a Technical Design Document

  • Problem

    • We currently have no way to monitor steam output logs from services and other things running in our Kubernetes cluster.

      我们目前没有办法监视在 Kubernetes 集群中运行的服务和其他内容的流输出日志。

  • Requirements

    • We only care about studio - no use for fields advanced indexing, etc.

      我们只关心“studio” - 没有用于字段高级索引等的用途。

    • We don’t want to have to manage infrastructure.

      我们不想管理基础架构。

    • We only want to solve simple logging for all of our services in Kubernetes to unblock when we have errors and need to find them, history, etc.

      我们只想解决 Kubernetes 中所有服务的简单日志记录，以便在出现错误并需要找到它们、历史记录等时解除阻塞。

    • We don’t want to deal with metrics right now.

      我们现在不想处理监控。

  • Self Hosted

    • Loki(with Grafana)
      • Free and includes most features.
        • The query language is very adoptable and simple.
    • Elastic with Kibana(ELK)
      • Free but limited with features(licensed)
        • Provides a lot of extensibility with searching logs.

  • Cloud Hosted

    • Grafana Cloud [Loki w/Grafana] (base free - 50GB)
    • Solarwinds Papertrail (base $7 - 1GB)
    • Elastic Cloud (base $95)
    • Solarwinds Loggly (base $79)
    • AWS OpenSearch (base $80 - 50GB + cluster instances)
    • New Relic (base free - 100GB)

  • Proposed Solution

    • Looking at the list above there is only one solution that really stands out to solving our problem the quickest with best accuracy:
      • Grafana Cloud

Problem: Fragmented Documentation

Fragmented Documentation

Title: Fragmented Documentation with Perpetuating Tribal Knowledge

标题：永久部落知识的碎片化文档

What problem are we trying to solve?

Engineering teams are frustrated because they are unable to find information regarding their day to day work.

工程团队感到沮丧，因为他们无法找到有关他们日常工作的信息。

How do we share information?

  • Wiki’s
    • Contains best practice for using shared.
  • Blog’s
    • Communicate releases or new features being introduced.
    • Generally serves engineering teams(internal).
  • Email Blasts
    • Usually an announcement or call to action for awareness of possible upcoming changes that will impact.
  • Messaging Software
    • Individual troubleshooting threads that become very difficult to parse and traverse.
    • Mostly focus on-off solutions that never get documented.

Which of these benefits us the most?

We have no real ability to track usages around where content lives and what the best place is to put it.

What do we prefer?

  • Messaging Software for realtime communication to solve new issues or untracked problems.
    • When a new problem is raised - we create a ticket for it and include Slack thread.
    • All critical information must be in the issue or ticket NOT in Slack(culture).
    • When new information is shared - we need a article or document for that(culture).
      • Developers must prompt each other to write documentation.
  • Centralized Documentation Platform
    • Simplicity, removes checking anywhere else.
    • Simplicity around organization in the documentation.
      • Each team get their own section or “area” that can be managed on their own.
    • Remove the need of managing more things.

What are the pain-points we’ve dealt with?

  • Some developers may not like writing documentation and is generally taking away time.
  • Hate duplication of documentation that says the same thing.
    • Prefer to have “links” to contextual thoughts around problems.
      • DevOps section(infrastructure).
      • Frontend section(logical changes pertaining to the developers).
      • Keeping information as close to implementation as possible.
      • Do not keep the same information in multiple places.
  • Centralized documentation can be a mess and lost.
    • No planing, no standards, no maintenance.
  • Terrible searching and filtering.
  • Not a realtime and requires more investment.

Proposed Solution

  • Culture around these issues
    • Documentation
  • Improve standards for messaging
    • Messaging
  • Improve standards around linking information
    • Emails
  • We are open to fragmentation of documentation but want to provide the best tools(Backstage) to be the GLUE of this information.

Managing Infrastructure with Pulumi

Infrastructure Standardization

Identifying Problems

  • Existing Accounts

    • AWS Account(infrastructure)
    • GitHub(infrastructure as code)
    • CircleCI(CI/CD)

  • Existing Tools

    • NodeJS(language)
    • Pulumi(automation)
    • CircleCI(CI/CD)

  • Technical Documents

    • Standardize Infrastructure
    • Standardize CI/CD
Standardize Infrastructure
  • Problem
    • We have no defined patterns or choices around how we describe services in our infrastructure.

      我们没有定义如何描述基础架构中的服务的模式或选择。

    • What are these services?
      • Frontend
      • Backends
    • What do they need?
      • Frontend
        • Store our static files
        • Ingress of some type
      • Backends
        • Container(s)
        • Ingress of some type
    • What do we need?
      • Simple interfaces that allow us to pull off the shelf deployments for specific service types.
      • Simple way of managing these resources and updating them when changed.
      • Way for DEVELOPERS not DevOps to create cloud resources.
        • No more asking for buckets in slack.

Managing CI/CD

Service Templates

Service Templates

Identifying Problems

We have services that all use similar ways to build, test and deploy. However we have to copy and paste these configuration files every time.

What we use for CI/CD

  • Docker(image)
  • CircleCI(build/testing)

What do we need?

  • DSL(domain-specific language) that allows us to configure templates for services
  • CLI to generate templates from our DSL

Solution

We want to build a CLI tool that has templates in it which we can configure with external JSON files to generate build configuration, etc.