Jekyll2024-03-23T15:08:59+07:00/feed.xmlDevOps with Dimas MaryantoTutorial Pemrograman, Backend, Frontend, Website, Mobile dan Devops Terlengkap di Indonesia.Dimas MaryantoSilabus Ansible: Pemula sampai Mahir2023-07-02T07:26:30+07:002023-07-02T07:26:30+07:00/posts/devops/automation/ansible/basic/00-silabus-ansibleHai semuanya, kembali lagi di Channel DevOps with Dimas Maryanto jadi kali ini aku mau jelasin dulu Silabus yang akan kita pelajari terkait kelas DevOps - Ansible untuk Pemula s/d Mahir
Materi yang di bahas pada section ini yaitu
Pengenalan Ansible
Tahap pertama yaitu kita akan berkenalan dengan Teknologi Ansible sebagai Automation Platform
Apa itu IT Automation?
Tool apa yang berguna untuk IT Automation?
Apa itu RedHat Ansible?
How to Ansible work’s?
Install and Configuring Ansible
Getting started with Ansible
What is Resource module?
Getting information about the remote server with Fact
Execute Ansible command with Ad-hoc
Create first Ansible playbook
Ok mungkin sekian dulu yang bisa saya sampaikan terkati Silabus Belajar IT Automation Infrastructure as a Code menggunakan RedHat Ansible untuk Pemula sampai Mahir. Sampai jumpa di kelas!!
]]>Dimas MaryantoGetting started: Create and run your first Gitlab CI/CD pipeline2023-06-18T15:30:55+07:002023-06-18T15:30:55+07:00/posts/git/gitlab/gitlab-ci/03d-getting-startedHai semuanya, sekarang kita akan mencoba menggunakan Gitlab CI/CD pipeline. Tetapi ada beberapa prerequisites yang harus dipenuhi diantaranya:
Before you start, make sure you have:
A project in GitLab that you would like to use CI/CD for.
The Maintainer or Owner role for the project.
To create and run your first pipeline:
Ensure you have runners available to run your jobs. To view available runners: Go to Settings > CI/CD and expand Runners.
If you’re using gitLab.com, you can skip this step. gitLab.com provides shared runners for you. If you don’t have a runner
Create a .gitlab-ci.yml file at the root of your repository. This file is where you define the CI/CD jobs.
When you commit the file to your repository, the runner runs your jobs. The job results are displayed in a pipeline.
Okay karena pembahasan kali ini akan lumayan panjang, jadi kita akan bagi-bagi menjadi beberapa bagian diantaranya seperti berikut:
Create a .gitlab-ci.yml file
Running the Gitlab CI pipelines
Tips for writing the .gitlab-ci.yml
Create a .gitlab-ci.yml file
File .gitlab-ci.yml basiclly it’s a YAML format that you can specify to instruction of Gitlab CI/CD. in this file, you can define:
The structure and order of jobs that the runner should execute.
The decisions the runner should make when specific conditions are encountered.
To create a .gitlab-ci.yml file you can used:
Manually, by using text-editor such as vscode, vim, nodepad, etc
Build-in text-editor by gitlab
This example shows four jobs: build-job, test-job1, test-job2, and deploy-prod. The comments listed in the echo commands are displayed in the UI when you view the jobs. The values for the predefined variables$GITLAB_USER_LOGIN and $CI_COMMIT_BRANCH are populated when the jobs run.
Then submit, commit & push
Now take a look at your pipeline and the jobs within.
Go to CI/CD > Pipelines. A pipeline with three stages should be displayed:
View a visual representation of your pipeline by selecting the pipeline ID:
View details of a job by selecting the job name.
This is build-job
This is build-test2
This is build-deploy-prod
You have successfully created your first CI/CD pipeline in GitLab. Congratulations!
Now you can get started customizing your .gitlab-ci.yml and defining more advanced jobs.
Running the Gitlab CI pipelines
By default, when you push or changed the file of .gitlab-ci.yml it’s trigger pipeline to run, but we have an other way to run it. Here is 4 deffrent method to the run pipeline:
Workflow (rules, only)
Manually run pipeline
Schedule to run pipeline
Trigger using (webhook, rest-api)
Updating file .gitlab-ci.yml
Okay kita bahas satu-per-satu, mulai dari update file .gitlab-ci.yml yang sebelumnya seperti berikut:
Ketika di push maka secara otomatis akan jalan pipeline baru seperti berikut:
The output will print look like this:
Hello, my name is Dimas Maryanto !\n i'm trying to learn CI/CD feature with Gitlab CI
Using workflow
Sekarang kita coba untuk workflow method, dengan mengupdate file .gitlab-ci.yml menggunakan property only atau rules seperti berikut:
Ketika di push maka secara otomatis akan jalan pipeline baru dan berikut adalah pipeline detailnya:
Nah terlihat disana terdapat 3 job yaitu build-job, test-job2 dan deploy-prod. Terus untuk job test-job1 kemana?
Job test-job1 hanya akan muncul jika kita membuat branch, tags atau commit message dengan prefix -release seperti berikut:
Kita coba buat tag 2023.06.18.12.51-release pada branch `main seperti berikut:
Sekarang jika kita lihat pada pipeline detail yang terbaru maka jobnya jalan seperti berikut:
Dengan dengan membuat tag tersebut, kita menjalankan kembali job sebelumnya beserta test-job1 tetapi tanpa job deploy-prod
Sedangkan untuk menjalankan yang job build-prod kita bisa membuat Merge request dari branch dengan prefix hotfix/ ke branch default yang which is main.
Buat branch baru misalnya dengan nama hotfix/issue01 dari branch main
Buat perubahan dan commit serta push
Buat merge request ke main maka hasilnya seperti berikut:
Manually run pipeline
Selain menggunakan script based, dan workflow Gitlab CI kita juga bisa menjalankan pipeline secara manual dengan click Run pipeline pada menu Pipelines seperti berikut:
Kemudian muncul form seperti berikut, jika tidak ada option yang kita define pada script .gitlab-ci.yml just click the button, hasilnya seperti berikut:
Schedule to run pipeline
Lanjutan dari run pipeline secara manual tersebut, kita juga bisa meng-schedule pipeline dengan menggunakan cron pattern caranya
Akses ke menu Pipelines -> Schedule seperti berikut:
Misalnya kita mau menjalankan pipeline tersebut setiap 5 menit, jadi pilih Interfal pattern Custom input seperti berikut */5 * * * *
Kemudian save, seperti berikut:
Dan perhatikan ke menu Pipeline 5 menit mendatang hasilnya seperti berikut:
Trigger using (webhook, rest-api)
Yang terkahir kita juga bisa me-trigger dari Webhook atau Rest-API menggunakan tools seperti curl atau bahkan Postman. Untuk membuat Gitlab CI yang bisa di triger kita membuat dulu TOKEN di menu Settings -> CI/CD -> expand Triger pipeline seperti berikut:
Kemudian input nama/description contohnya trigger-curl dan kemudian click button Add triger maka akan muncul tokennya, coba simpan token tersebut.
Setelah itu kita bisa triger pipeline dengan perintah curl berikut:
curl -X POST \--fail\-Ftoken=TOKEN \-Fref=REF_NAME \
PROTOCOL://SERVER_GITLAB/api/v4/projects/PROJECT_ID/trigger/pipeline
Sekarang coba jalankan perintah berikut di terminal
Each job contains a script section and belongs to a stage:
stage describes the sequential execution of jobs. If there are runners available, jobs in a single stage run in parallel.
Use the needs keyword to run jobs out of stage order.
You can set additional configuration to customize how your jobs and stages perform:
Use the rules keyword to specify when to run or skip jobs. The only and except legacy keywords are still supported, but can’t be used with rules in the same job.
Keep information across jobs and stages persistent in a pipeline with cache and artifacts. These keywords are ways to store dependencies and job output, even when using ephemeral runners for each job.
Use the default keyword to specify additional configurations that are applied to all jobs. This keyword is often used to define before_script and after_script sections that should run on every job.
]]>Dimas MaryantoSettings up Gitlab CI (on-premise)2023-06-17T21:12:16+07:002023-06-17T21:12:16+07:00/posts/git/gitlab/gitlab-ci/03c-setup-gitlab-ciHai semuanya, di materi kali ini kita akan membahas tentang Setup environment for Continues Integration (CI) menggunakan Gitlab CI di on-premise. Ok karena artikel ini akan lumayan panjang jadi kita akan bagi2 menjadi beberapa bagian yaitu
Requirement and prequisite
Arsitektur
Setup & Konfigurasi software
Install gitlab
Install & setting docker engine
Install gitlab runner
Register gitlab-runner to gitlab
Tryout Gitlab CI script
Ok langsung ja kita ke pembahasan pertama yaitu
Requirement and prequisite
Untuk menggunakan Gitlab CI ada beberapa hal yang perlu kita siapkan diantaranya minimal 2 virtual machine dengan specifikasi seperti berikut:
workers:os:Linux Distrodistro:CentOSvesion:'>=7.9'hardware:## sesuaikan specifikasi hardwarenya dengan kebutuhan build## karena ada beberapa bahasa pemograman membutuhkan lebih dari 4GB RAM dan jumlah cpucpu:2 CPUmemory:4GBstorage:70GBpartitions:-name:/size:20 GB-name:/varsize:'>=50GB'type:LVMnetworks:-name:private ipip4:192.168.88.102/24gateway:192.168.88.1dns:-8.8.8.8-8.8.4.4packages:-OpenSSH-Server-docker-ce-gitlab-runner
Untuk membuat virtual-machine temen-temen boleh menggunakan hypervisor provider apapun misalnya; Oracle VirtualBox, VMware player, qemu dan lain-lain. Kalo saya sendiri menggunakan proxmox yang saya install di PC dengan spesifikasi seperti berikut:
Okay sebelum kita install software-software tersebut kita lihat dulu secara architectur yang digunakan agar kita bisa memahami flownya, yaitu seperti berikut:
This diagram shows how runners are registered and how jobs are requested and handled:
sequenceDiagram
participant GitLab
participant GitLabRunner
participant Executor
opt registration
GitLabRunner ->>+ GitLab: POST /api/v4/runners with registration_token
GitLab -->>- GitLabRunner: Registered with runner_token
end
loop job requesting and handling
GitLabRunner ->>+ GitLab: POST /api/v4/jobs/request with runner_token
GitLab -->>+ GitLabRunner: job payload with job_token
GitLabRunner ->>+ Executor: Job payload
Executor ->>+ GitLab: clone sources with job_token
Executor ->>+ GitLab: download artifacts with job_token
Executor -->>- GitLabRunner: return job output and status
GitLabRunner -->>- GitLab: updating job output and status with job_token
end
GitLab Runner: The application that you install that executes GitLab CI jobs on a target computing platform.
runner: The agent that runs the code on the host platform and displays in the UI. If a runner is registered with the same token, the runner could represent a collection of runners and runner managers.
runner manager: A type of runner that can create multiple runners for autoscaling. Specific to the type of executor used.
runner worker: The process created by the runner on the host computing platform to run jobs.
GitLab Runner implements a number of executors that can be used to run your builds in different scenarios.
SSH, is using Connect using ssh protocol to execute jobs
Shell, is a simple executor that you use to execute builds locally on the machine where GitLab Runner is installed.
Hypervisor (VirtualBox, Paralles), is using VirtualMachine manager to execute the job
Docker, GitLab Runner uses the Docker executor to run jobs on Docker images.
Kubernetes, Use the Kubernetes executor to use Kubernetes clusters for your builds. The executor calls the Kubernetes cluster API and creates a pod for each GitLab CI job.
Tahap selanjutnya temen-temen bisa pilih salah-satu, klo saya sendiri lebih sering menggunakan Docker executor sebagai main gitlab runner executornya so jadi kita bisa langsung install.
Setup & Konfigurasi software
Untuk gitlab intances, kita bisa menggunakan Gitlab SaaS (online) atau on-premise (local). Untuk installasi gitlab on-premise kita bisa menggunakan yang sebelumnya jadi kita bisa skip atau bisa lanjutkan ke proses install gitlab-runner.
Untuk installing gitlab runner kita bisa menggunakan Linux Repository distribution yaitu dengan menggunakan perintah berikut:
Before you install Docker Engine for the first time on a new host machine, you need to set up the Docker repository. Afterward, you can install and update Docker from the repository. Install the yum-utils package (which provides the yum-config-manager utility) and set up the repository.
Setelah itu pastikan configurasi selinux di update menjadi permissive dengan mengedit file /etc/selinux/config untuk property SELINUX=enforce menjadi SELINUX=permissive dan kemudian restart.
Jangan lupa tambahkan rule pada firewall seperti berikut:
Setelah di install docker dan gitlab-runner package, kita Register gitlab-runner ke Gitlab Repository ada pun yang harus di perhatikan adalah
Registered as Global
Registered by group
Registered by project
Pilih yang mana? ini tergantung dari kebutuhan ada yang ingin semua project pake gitlab runner brati kita register sebagai Global (Menu Admin -> Runners), ada yang per project (Menu Your project -> Settings -> CI/CD -> Runners) jadi kita pilih by project. Karena disini saya mau general kita pilih yang Global. yang kita perlukan adalah URL dan Registration token seperti berikut:
Sekarang kita register, gitlab runner agent ke gitlab dengan menggunakan perintah berikut:
Jika sudah maka gitlab-runner akan terdaftar seperti berikut:
Tryout Gitlab CI script
Setelah kita install package gitlab, gitlab-runner, executor (docker) dan register gitlab runner agent ke gitlab instance tahap selanjutnya kita coba test menggunakan simple configuration. Berikut adalah step-by-step nya
Buat repository, contohnya name test-gitlab-ci kemudian visiblilty kita buat public serta untuk project configuration check pada initilization repo with README click Create project
Kemudian coba buat file baru dengan nama .gitlab-ci.yml seperti berikut:
Dan coba commit, serta push ke branch main. Jika sudah temen-temen bisa lihat di menu Pipeline seperti berikut:
Terdapat 1 pipeline yang sedang berjalan, kita coba lihat detailnya dengan click button running maka seperti berikut:
Dalam detail pipeline tersebut terlihat sedang ada 1 job yang sedang berjalan, kita coba lihat detail dengan click button job1 maka seperti berikut:
Nah terlihat pada job, menjalankan script yang kita tulis pada file .gitlab-ci.yml menghasilkan output:
$ echo'Hi my name is Dimas Maryanto'
Hi my name is Dimas Maryanto
Cleaning up project directory and file based variables
00:01
Job succeeded
Nah ini artinya, kita sudah bisa menggunakan Continues Integration dengan menggunakan Gitlab CI.
]]>Dimas MaryantoIntroduction of Gitlab DevSecOps Platform2023-06-17T20:36:43+07:002023-06-17T20:36:43+07:00/posts/git/gitlab/gitlab-ci/03b-introduction-gitlab-ciHai semuanya, di materi kali ini kita akan membahas salah satu tools implementasi dari Continues Integration (CI) yaitu menggunakan Gitlab DevSecOps Platform.
Okay karena pembahasan kali ini akan lumayan panjang jadi kita bagi-bagi menjadi beberapa section diantaranya:
Gitlab DevSecOps Platform
Gitlab CI/CD workflow
What can Gitlab CI/CD do?
Gitlab DevOps Platform
Gitlab Features
Fundamentally changing the way Development, Security, and Ops teams collaborate and build software - GitLab provides all of the essential DevSecOps tools in one DevSecOps platform.
From idea to production, GitLab helps teams improve cycle time from weeks to minutes, reduce development costs, speed time to market, and deliver more secure and compliant applications.
Planning
Regardless of your process, GitLab provides powerful planning tools to keep everyone synchronized.
Create
Create, view, and manage code and project data through powerful branching tools.
Verify
Keep strict quality standards for production code with automatic testing and reporting.
Package
Create a consistent and dependable software supply chain with built-in package management.
Secure
Security capabilities, integrated into your development lifecycle.
Deploy
GitLab’s integrated CD solution allows you to ship code with zero-touch, be it on one or one thousand servers.
Monitor
Help reduce the severity and frequency of incidents.
Govern
Manage security vulnerabilities, policies, and compliance across your organization.
Gitlab CI/CD workflow
Gitlab CI menawarkan 3 konsep yaitu Continues Integration, Continues Delivery dan Continues Deployment. Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more, so you can test, build, and publish your software with no third-party application or integration needed because Gitlab CI is the part of Gitlab.
GitLab CI/CD fits in a common development workflow.
You can start by discussing a code implementation in an issue and working locally on your proposed changes.
Then you can push your commits to a feature branch in a remote repository that’s hosted in GitLab.
The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD:
Runs automated scripts (sequentially or in parallel) to:
Build and test your application.
Preview the changes in a Review App, the same as you would see on your localhost.
After the implementation works as expected:
Get your code reviewed and approved.
Merge the feature branch into the default branch.
GitLab CI/CD deploys your changes automatically to a production environment.
If something goes wrong, you can roll back your changes.
This workflow shows the major steps in the GitLab process. You don’t need any external tools to deliver your software and you can visualize all the steps in the GitLab UI.
What can Gitlab CI/CD do?
If you look deeper into the workflow, you can see the features available in GitLab at each stage of the DevOps lifecycle.
Stages continues integration:
Code quality, To measure/analyze your source code’s quality and complexity.
Unit tests, To test your source code’s with automated
Dependency scanning, To measure/analyze your dependencies is verify and not contains vulnerability
Stages continues delivery/deployment:
Build & Package, To compile then wrap your source code’s into package, it’s can be executeable from anywere
Shipping, After build script succeed, we can store that artifact to registry such as Container registry, NPM registry, Maven repository and etc.
Deploy, After artifact stored on registry/repository, we can used that object to run the application
]]>Dimas MaryantoWhat is Continues Integration/Continues Delivery (CI/CD)?2023-06-17T16:52:30+07:002023-06-17T16:52:30+07:00/posts/git/gitlab/gitlab-ci/03a-what-is-cicdHai semuanya, di materi kali ini kita akan membahas seputar Continues Integration and Continues Delivery alias CI/CD dalam suatu SDLC atau singkatan dari Software Development Lifecycle.
Pada suatu project/product berbasis software development ini memiliki beberapa tahapan mulai dari coding, testing, dan deployment. Hal ini menjadi tantangan tersendiri jika melibatkan beberapa team/divisi/perusahaan dalam suatu development project/product tersebut.
Saat ini banyak perusahaan/instansi menggunakan proses CI/CD untuk menghilankan kerumitan (simplify) dalam proses release suatu project/product agar bisa lebih cepet dan autopilot (automation deployment).
Sebelum kita membahas lebih jauh, ada beberapa hal yang kita perlu pahami terlebih dahulu tentang CI/CD diantaranya:
What is Continues Integration & Continues Delivery/Deployment (CI/CD)?
What is the difference between CI and CD?
How does CI/CD relate to DevOps?
Benefit of implemented CI/CD in SDLC
What are the benefits of CI/CD?
CI/CD software tools
Okay tanpa berlama lama yuk kita bahas satu-per-satu
What is Continues Integration & Continues Delivery/Deployment (CI/CD)?
CI and CD stand for continuous integration and continuous delivery/continuous deployment. CI/CD is a method to frequently deliver apps to customers by introducing automation into the stages of app development.
Main concepts attributed to CI/CD are
Continuous integration,
Continuous delivery, and
Continuous deployment.
CI/CD is a solution to the problems integrating new code can cause for development and operations teams.
Specifically, CI/CD introduces ongoing automation and continuous monitoring throughout the lifecycle of apps, from integration and testing phases to delivery and deployment.
Taken together, these connected practices are often referred to as a “CI/CD pipeline” and are supported by development and operations teams working together in an agile way.
Why is CI/CD important?
CI/CD allows organizations to ship software quickly and efficiently. CI/CD facilitates an effective process for getting products to market faster than ever before, continuously delivering code into production, and ensuring an ongoing flow of new features and bug fixes via the most efficient delivery method.
What is the difference between CI and CD?
Kita telah mengenal Continues integration dan Continues Delivery/Deployment alias CI/CD, tetapi apa sih sebetulnya perbedaan diantaranya? yuk kita bahas satu-per-satu mulai dari Continues Integration.
Continuous Integration, Consider an application that has its code stored in a Git repository. Developers push code changes every day, multiple times a day. For every push to the repository, you can create a set of scripts to build and test your application automatically. These scripts help decrease the chances that you introduce errors in your application.
Each change submitted to an application, even to development branches, is built and tested automatically and continuously. These tests ensure the changes pass all tests, guidelines, and code compliance standards you established for your application.
Continuous Delivery, is a step beyond Continuous Integration. Not only is your application built and tested each time a code change is pushed to the codebase, the application is also deployed continuously. However, with continuous delivery, you trigger the deployments manually.
Continuous Delivery checks the code automatically, but it requires human intervention to manually and strategically trigger the deployment of the changes.
Continuous Deployment, is another step beyond Continuous Integration, similar to Continuous Delivery. The difference is that instead of deploying your application manually, you set it to be deployed automatically. Human intervention is not required.
For more details, here is the diagram ci/cd workflow
How does CI/CD relate to DevOps?
DevOps is a set of practices and tools designed to increase an organization’s ability to deliver applications and services faster than traditional software development processes.
The increased speed of DevOps helps an organization serve its customers more successfully and be more competitive in the market. In a DevOps environment, successful organizations “bake security in” to all phases of the development life cycle, a practice called DevSecOps.
The key practice of DevSecOps is integrating security into all DevOps workflows. By conducting security activities early and consistently throughout the software development life cycle (SDLC), organizations can ensure that they catch vulnerabilities as early as possible, and are better able to make informed decisions about risk and mitigation.
In more traditional security practices, security is not addressed until the production stage, which is no longer compatible with the faster and more agile DevOps approach.
Today, security tools must fit seamlessly into the developer workflow and the CI/CD pipeline in order to keep pace with DevOps and not slow development velocity.
The CI/CD pipeline is part of the broader DevOps/DevSecOps framework. In order to successfully implement and run a CI/CD pipeline, organizations need tools to prevent points of friction that slow down integration and delivery.
Teams require an integrated toolchain of technologies to facilitate collaborative and unimpeded development efforts.
What are the benefits of CI/CD?
Automated testing enables continuous delivery, which ensures software quality and security and increases the profitability of code in production.
CI/CD pipelines enable a much shorter time to market for new product features, creating happier customers and lowering strain on development.
The great increase in overall speed of delivery enabled by CI/CD pipelines improves an organization’s competitive edge.
Automation frees team members to focus on what they do best, yielding the best end products.
Organizations with a successful CI/CD pipeline can attract great talent. By moving away from traditional waterfall methods, engineers and developers are no longer bogged down with repetitive activities that are often highly dependent on the completion of other tasks.
CI/CD software tools
Ada beberapa software tools yang dapat digunakan untuk membuat CI/CD Pipeline. Diantaranya sebagai berikut:
Jenkins
Jenkins ini bersifat open source dan menggunakan bahasa pemrograman Java. Tool ini dapat digunakan pada berbagai sistem operasi seperti Windows, macOS, dan Linux serta dapat di deploy cloud maupun on-premise
Gitlab CI
Gitlab CI adalah bagian dari Gitlab Platform yang dibuat oleh gitlab itu sendiri dengan menggunakan bahasa Golang (go). Tool ini dapat digunakan di berbagai macam sistem operasi seperti Windows, macOS, dan linux. Kita bisa mengguankan Gitlab CI yang telah tersedia (Shared runner) atau self managed (Private runner)
Github Action
Github Action adalah bagian dari Github Platform yang dibuat oleh github yang sekarang telah di akuisisi oleh Microsoft. Tools ini hanya dapat di gunakan secara cloud (github.com) dan kita bisa gunakan secara gratis/free
Travis CI
Travis CI adalah cloud-based CI/CD yang biasanya digunakan sebelum lahirnya Github Action yang memiliki jargon Code less dan deploy more. Tools ini juga selain hanya tersedia di cloud juga saat ini menjadi platform CI/CD yang berbayar
]]>Dimas MaryantoSilabus Gitlab CI: Pemula s/d Mahir2023-06-02T20:18:43+07:002023-06-02T20:18:43+07:00/posts/git/gitlab/gitlab-ci/03-silabus-gitlab-ciHai semuanya, selamat datang kembali di kelas GitOps dengan Gitlab untuk Pemula s/d Mahir materi kali ini kita membahas tentang Continues Integration (CI) dengan menggunakan Gitlab Platform yaitu Gitlab CI.
Pada materi kali ada beberapa topic yang akan kita pelajari yaitu
Basic concepts of Gitlab CI
GitLab CI/CD uses a number of concepts to describe and run your build and deploy.
What is Gitlab CI
Seting up Gitlab & Gitlab Runner
Build a pipeline
Using CI/CD Variables
Predefined variables
User defined variables (custom)
Using environments
Job artifacts
Cache depedency
Job/Pipeline efficiency
Test case for Unit testing automation
Configuration of Gitlab CI
Schedule pipeline
Pipeline triggers
Merge request pipeline
Opimize Gitlab and Gitlab Runner for large repositories
Feature of Gitlab CI
Auto DevOps, Set up your app’s entire lifecycle.
CI services, Link Docker containers with your base image.
Auto Deploy, Deploy your application to a production environment in a Kubernetes cluster.
Build docker image
Canary deployments
Deploy boards
Gitlab release
Code quality
Biasanya Gitlab CI digunakan untuk melakukan Automation seperti build, test, deploy and monitor suatu aplikasi yang dibuat oleh Developer. Karena materinya banyak sekali jadi kita akan breakdown semua meteri tersebut mejadi beberapa bagian. So mungkin sekian dulu apa yang bisa saya sampaikan terkait Silabus belajar Gitlab CI untuk Automation. 👋
]]>Dimas MaryantoWorking with Deployment object2023-04-29T11:51:37+07:002023-04-29T11:51:37+07:00/posts/devops/orchestration/kubernetes/workloads/05b-deploymentHai semuanya, materi sebelumnya kita sudah mencoba menggunakan Kubernetes workload resources dengan Object Deployment yang paling simple. Sekarang kita akan membahas yang lebih detail tentang Object Deployment. Karena pembahasan kali ini lumayan panjang jadi kita bagi2 jadi beberapa bagian diantaranya:
What is Deployment Object?
Create a Deployment
Interaction with Deployment object
Updating a Deployment
Rolling Back a Deployment
Scalling the pods with a Deployment
Pause and Resume rollout of a Deployment
Deployment status
type: progressing
type: complete
type: failed
type: pending
Okay tanpa berlama-lama yuk langsung aja kita bahas materi yang pertama:
What is Deployment Object
Seperti yang telah saya bahas di materi sebelumnya, Deployment adalah salah satu Workload resources yang digunakan untuk me-manage Pod dan container yang paling cocok untuk aplikasi Stateless atau tidak menyimpan data.
A Deployment provides declarative updates for Pods and ReplicaSets.
You describe a desired state in a Deployment, and the Deployment Controller changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.
The following are typical use cases for Deployments:
Create a Deployment to rollout a ReplicaSet. The ReplicaSet creates Pods in the background. Check the status of the rollout to see if it succeeds or not.
Declare the new state of the Pods by updating the PodTemplateSpec of the Deployment. A new ReplicaSet is created and the Deployment manages moving the Pods from the old ReplicaSet to the new one at a controlled rate. Each new ReplicaSet updates the revision of the Deployment.
Rollback to an earlier Deployment revision if the current state of the Deployment is not stable. Each rollback updates the revision of the Deployment.
Scale up the Deployment to facilitate more load.
Pause the rollout of a Deployment to apply multiple fixes to its PodTemplateSpec and then resume it to start a new rollout.
Use the status of the Deployment as an indicator that a rollout has stuck.
Clean up older ReplicaSets that you don’t need anymore.
Create Deployment using yaml file
Untuk membuat object deployment bisa menggunakan imperative command (kubectl create deploy) dan declarative seperti membuat file extension .yaml atau .json yang kemudian di execute menggunakan perintah kubectl apply -f filename.yaml.
Berikut adalah basic Deployment spec yang digunakan untuk membuat/menjalankan 3 buah pod dengan container image nginx seperti berikut:
Kemudian coba jalankan menggunakan perintah:
kubectl apply -f basic-deployment.yaml
Jika kita lihat hasilnya seperti berikut:
➡ kubectl apply -f 03-workloads/01-basic-deploy/basic-deployment.yaml
deployment.apps/nginx-deploy created
➜ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 3/3 3 3 3m21s
➜ kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-deploy-c9bcb48d4 3 3 3 3m39s
➜ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deploy-c9bcb48d4-5s7zs 1/1 Running 0 82s
nginx-deploy-c9bcb48d4-nrsr7 1/1 Running 0 3m54s
nginx-deploy-c9bcb48d4-rbc8s 1/1 Running 0 82s
In this example:
A Deployment named nginx-deploy is created, indicated by the .metadata.name field. This name will become the basis for the ReplicaSets and Pods which are created later. See Writing a Deployment Spec for more details.
The Deployment creates a ReplicaSet that creates three replicated Pods, indicated by the .spec.replicas field.
The .spec.selector field defines how the created ReplicaSet finds which Pods to manage. In this case, you select a label that is defined in the Pod template (app: nginx, env: test). However, more sophisticated selection rules are possible, as long as the Pod template itself satisfies the rule.
The template field contains the following sub-fields:
The Pods are labeled app: nginx using the .metadata.labels field.
The Pod template’s specification, or .template.spec field, indicates that the Pods run one container, nginx, which runs the nginx Docker Hub image at version [mainline].
Create one container and name it nginx using the .spec.template.spec.containers[0].name field.
Interaction with Development object
Untuk berinteraksi dengan object deployment ada beberapa function/command yang kita bisa gunakan seperti
Melihat log pada suatu pod,
Meng-execute command pada suatu pod dari sekumpulan pod,
Merestart seluruh pod dalam deployment,
Updating a deployment spec,
Rolling back to previews history version,
Scalling deployment
Proportional scalling deployment (autoscalling)
Okay sekarang kita coba satu-per-satu yuk, Untuk melihat logs pada suatu pod yang ada pada suatu deployment kita bisa menggunakan perintah kubectl logs deploy/<deploy-name> contohnya seperti berikut:
kubectl logs deploy/nginx-deploy
Jika dijalankan maka hasilnya seperti berikut:
➜ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 3/3 3 3 111s
➜ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deploy-c9bcb48d4-dzg7p 1/1 Running 0 7m41s
nginx-deploy-c9bcb48d4-nrq4k 1/1 Running 0 7m41s
nginx-deploy-c9bcb48d4-qgf9d 1/1 Running 0 7m41s
➜ kubectl logs deploy/nginx-deploy
Found 3 pods, using pod/nginx-deploy-c9bcb48d4-dzg7p
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/04/09 02:31:40 [notice] 1#1: using the "epoll" event method
2023/04/09 02:31:40 [notice] 1#1: nginx/1.23.4
2023/04/09 02:31:40 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
2023/04/09 02:31:40 [notice] 1#1: OS: Linux 5.10.57
2023/04/09 02:31:40 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/04/09 02:31:40 [notice] 1#1: start worker processes
2023/04/09 02:31:40 [notice] 1#1: start worker process 29
2023/04/09 02:31:40 [notice] 1#1: start worker process 30
Nah jika temen-temen perhatikan, pada deployment nginx-deploy memiliki lebih dari 1 pod tetapi yang ditampilkan adalah pod yang pertama by default yaitu pod/nginx-deploy-c9bcb48d4-dzg7p
Selanjutnya, jika kita mau meng-execute command dalam container dari object deployment bisa menggunakan perintah kubectl exec deploy/<deploy-name> -- <command> seperti berikut contohnya:
➜ kubectl exec deploy/nginx-deploy -- curl https://jsonplaceholder.typicode.com/todos/1 -v
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 104.21.54.83:443...
> GET /todos/1 HTTP/2
> Host: jsonplaceholder.typicode.com
> user-agent: curl/7.74.0
> accept: */*>
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=vGRPnjpH6UP1fT0COF95QYK5RIuA2jrZPx1H6jsOVkWrVhLGjRAZYkd1izYBf04YbE4JXWKihCdw1YLrZJZBxAIMLCCW7ViJ6QUbOx%2F06hsARokb7XjEJ%2FUoacL9So%2BACJ%2BGsdUqgOuXf9w5HYUp"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 7b4f65c4a8254b80-SIN
< alt-svc: h3=":443";ma=86400, h3-29=":443";ma=86400
<
{[83 bytes data]
{"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
100 83 100 83 0 0 1136 0 --:--:-- --:--:-- --:--:-- 1136
* Connection #0 to host jsonplaceholder.typicode.com left intact
Kemudian jika kita mau me-restart semua pod sekaligus dalam object deployment kita bisa menggunakan perintah kubectl rollout restart deploy/<deploy-name> seperti berikut contohnya:
Okay nah itu adalah salah satu interacation yang paling simple pada object deployment, Untuk update deployment spec, rolling back to previews version akan di bahas pada section selanjutnya ya supaya tidak terlalu panjang.
Updating a Deployment
Ada beberapa cara melakukan update suatu Deployment object pada kubernetes
Using kubectl set command
Using kubectl edit deploy/<deploy-name>
Using declarative file kemudian menjalakan kubectl apply -f command
For example, saya mau update image deployment versionnya dari mainline menjadi stable-alpine jadi perintahnya seperti berikut:
kubectl set image deploy/nginx-deploy nginx-deploy=nginx:stable-alpine
Atau alternative lainnya, kita bisa edit secara langsung di kubernetes clusternya dengan menggunakan perintah
kubectl edit deploy nginx-deploy
Nah jika kita jalankan outputnya seperti berikut:
➜ kubectl set image deploy/nginx-deploy nginx-deploy=nginx:stable-alpine
deployment.apps/nginx-deploy image updated
➜ kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-deploy-5c7b8c97f6 3 3 3 2m8s
nginx-deploy-c9bcb48d4 0 0 0 168m
➜ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deploy-5c7b8c97f6-jgt2p 1/1 Running 0 2m54s
nginx-deploy-5c7b8c97f6-k7jnf 1/1 Running 0 3m4s
nginx-deploy-5c7b8c97f6-mzg4m 1/1 Running 0 3m16s
➜ kubernetes git:(main) kubectl describe pod/nginx-deploy-5c7b8c97f6-jgt2p
Name: nginx-deploy-5c7b8c97f6-jgt2p
Namespace: default
Node: minikube/192.168.105.9
Labels: app=nginx
env=test
pod-template-hash=5c7b8c97f6
Status: Running
IP: 10.244.0.5
IPs:
IP: 10.244.0.5
Controlled By: ReplicaSet/nginx-deploy-5c7b8c97f6
Containers:
nginx-deploy:
Image: nginx:stable-alpine
Port: 80/TCP
State: Running
Started: Fri, 07 Apr 2023 17:29:38 +0700
Ready: True
Restart Count: 0
Environment: <none>
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Events:
Type Reason Age From Message
-------------------------
Normal Scheduled 3m32s default-scheduler Successfully assigned default/nginx-deploy-5c7b8c97f6-jgt2p to minikube
Normal Pulling 3m32s kubelet Pulling image "nginx:stable-alpine"
Normal Pulled 3m22s kubelet Successfully pulled image "nginx:stable-alpine"in 9.54521492s (9.545224254s including waiting)
Normal Created 3m22s kubelet Created container nginx-deploy
Normal Started 3m22s kubelet Started container nginx-deploy
Get more details on your updated Deployment:
After the rollout succeeds, you can view the Deployment by running kubectl get deployments. The output is similar to this:
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 3/3 3 3 3h6m
Run kubectl get rs to see that the Deployment updated the Pods by creating a new ReplicaSet and scaling it up to 3 replicas, as well as scaling down the old ReplicaSet to 0 replicas.
NAME DESIRED CURRENT READY AGE
nginx-deploy-5c7b8c97f6 3 3 3 6m33s
nginx-deploy-c9bcb48d4 0 0 0 173m
Run kubectl get pod, Running get pods should now show only the new Pods:
NAME READY STATUS RESTARTS AGE
nginx-deploy-5c7b8c97f6-jgt2p 1/1 Running 0 7m30s
nginx-deploy-5c7b8c97f6-k7jnf 1/1 Running 0 7m40s
nginx-deploy-5c7b8c97f6-mzg4m 1/1 Running 0 7m52s
Next time you want to update these Pods, you only need to update the Deployment’s Pod template again.
Deployment ensures that only a certain number of Pods are down while they are being updated. By default, it ensures that at least 75% of the desired number of Pods are up (25% max unavailable).
For example, if you look at the above Deployment closely, you will see that it first creates a new Pod, then deletes an old Pod, and creates another new one. It does not kill old Pods until a sufficient number of new Pods have come up, and does not create new Pods until a sufficient number of old Pods have been killed. It makes sure that at least 3 Pods are available and that at max 4 Pods in total are available. In case of a Deployment with 4 replicas, the number of Pods would be between 3 and 5.
Rolling Back a Deployment
Sometimes, you may want to rollback a Deployment; for example, when the Deployment is not stable, such as crash looping. By default, all of the Deployment’s rollout history is kept in the system so that you can rollback anytime you want (you can change that by modifying revision history limit).
Suppose that you made a typo while updating the Deployment, by putting the image name as nginx:1.161 instead of nginx:1.16.1:
kubectl set image deployment/nginx-deploy nginx-deploy=nginx:1.161
Berikut outputnya:
➜ kubectl set image deployment/nginx-deploy nginx-deploy=nginx:1.161
deployment.apps/nginx-deploy image updated
➜ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 3/3 1 3 7m13s
➜ kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-deploy-5c7b8c97f6 3 3 3 7m11s
nginx-deploy-65cbcc896b 1 1 0 36s
nginx-deploy-c9bcb48d4 0 0 0 7m27s
➜ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deploy-5c7b8c97f6-6xzfr 1/1 Running 0 7m34s
nginx-deploy-5c7b8c97f6-tvfxv 1/1 Running 0 7m32s
nginx-deploy-5c7b8c97f6-vt5lf 1/1 Running 0 7m33s
nginx-deploy-65cbcc896b-7q6mc 0/1 ErrImagePull 0 59s
The rollout gets stuck. You can verify it by checking the rollout status:
kubectl rollout status deploy/nginx-deploy
The output look like:
➜ kubectl rollout status deploy/nginx-deploy
Waiting for deployment "nginx-deploy" rollout to finish: 1 out of 3 new replicas have been updated...
Press Ctrl-C to stop the above rollout status watch. First, check the revisions of this Deployment:
Follow the steps given below to rollback the Deployment from the current version to the previous version, which is version 3.
Now you’ve decided to undo the current rollout and rollback to the previous revision:
kubectl rollout undo deploy/nginx-deploy
The output is similar to this:
➜ kubectl rollout undo deploy/nginx-deploy
deployment.apps/nginx-deploy rolled back
➜ kubectl rollout status deploy/nginx-deploy
deployment "nginx-deploy" successfully rolled out
➜ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 3/3 3 3 15m
➜ kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-deploy-5c7b8c97f6 3 3 3 15m
nginx-deploy-65cbcc896b 0 0 0 9m4s
nginx-deploy-c9bcb48d4 0 0 0 15m
➜ kubectl describe deploy nginx-deploy
Name: nginx-deploy
Namespace: default
Labels: app=nginx
env=test
Annotations: deployment.kubernetes.io/revision: 4
Selector: app=nginx,env=test
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=nginx
env=test
Containers:
nginx-deploy:
Image: nginx:stable-alpine
Port: 80/TCP
Host Port: 0/TCP
NewReplicaSet: nginx-deploy-5c7b8c97f6 (3/3 replicas created)
Events:
Type Reason Age From Message
-------------------------
Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set nginx-deploy-c9bcb48d4 to 3
Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set nginx-deploy-5c7b8c97f6 to 1
Normal ScalingReplicaSet 16m deployment-controller Scaled down replica set nginx-deploy-c9bcb48d4 to 2 from 3
Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set nginx-deploy-5c7b8c97f6 to 2 from 1
Normal ScalingReplicaSet 16m deployment-controller Scaled down replica set nginx-deploy-c9bcb48d4 to 1 from 2
Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set nginx-deploy-5c7b8c97f6 to 3 from 2
Normal ScalingReplicaSet 16m deployment-controller Scaled down replica set nginx-deploy-c9bcb48d4 to 0 from 1
Normal ScalingReplicaSet 9m42s deployment-controller Scaled up replica set nginx-deploy-65cbcc896b to 1
Normal ScalingReplicaSet 97s deployment-controller Scaled down replica set nginx-deploy-65cbcc896b to 0 from 1
Alternatively, you can rollback to a specific revision by specifying it with --to-revision:
➜ kubectl rollout undo deploy/nginx-deploy --to-revision 1
deployment.apps/nginx-deploy rolled back
➜ kubectl rollout status deploy/nginx-deploy
deployment "nginx-deploy" successfully rolled out
➜ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 3/3 3 3 19m
➜ kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-deploy-5c7b8c97f6 0 0 0 19m
nginx-deploy-65cbcc896b 0 0 0 12m
nginx-deploy-c9bcb48d4 3 3 3 19m
➜ kubectl describe deploy nginx-deploy
Name: nginx-deploy
Namespace: default
Labels: app=nginx
env=test
Annotations: deployment.kubernetes.io/revision: 5
Selector: app=nginx,env=test
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=nginx
env=test
Containers:
nginx-deploy:
Image: nginx:mainline
Port: 80/TCP
Host Port: 0/TCP
NewReplicaSet: nginx-deploy-c9bcb48d4 (3/3 replicas created)
Events:
Type Reason Age From Message
-------------------------
Normal ScalingReplicaSet 19m deployment-controller Scaled up replica set nginx-deploy-c9bcb48d4 to 3
Normal ScalingReplicaSet 19m deployment-controller Scaled up replica set nginx-deploy-5c7b8c97f6 to 1
Normal ScalingReplicaSet 19m deployment-controller Scaled down replica set nginx-deploy-c9bcb48d4 to 2 from 3
Normal ScalingReplicaSet 19m deployment-controller Scaled up replica set nginx-deploy-5c7b8c97f6 to 2 from 1
Normal ScalingReplicaSet 19m deployment-controller Scaled down replica set nginx-deploy-c9bcb48d4 to 1 from 2
Normal ScalingReplicaSet 19m deployment-controller Scaled up replica set nginx-deploy-5c7b8c97f6 to 3 from 2
Normal ScalingReplicaSet 19m deployment-controller Scaled down replica set nginx-deploy-c9bcb48d4 to 0 from 1
Normal ScalingReplicaSet 12m deployment-controller Scaled up replica set nginx-deploy-65cbcc896b to 1
Normal ScalingReplicaSet 4m54s deployment-controller Scaled down replica set nginx-deploy-65cbcc896b to 0 from 1
Normal ScalingReplicaSet 61s (x6 over 63s) deployment-controller (combined from similar events): Scaled down replica set nginx-deploy-5c7b8c97f6 to 0 from 1
Scalling the pods with a Deployment
Selain kita bisa meng-update specifikasi pod dan container, kita juga bisa melakukan scalling suatu Deployment object dengan menggunakan beberapa cara yaitu
Using kubectl scale deploy/<deploy-name> --replicas=<number-of-instance> command
Using kubectl edit deploy/<deploy-name>
Update deployment.yaml file then using kubectl apply -f
For example, saya mau update secara langsung dengan perintahnya seperti berikut:
kubectl scale deploy/nginx-deploy --replicas=5
Jika kita coba jalankan maka seperti berikut:
➡ kubectl scale deploy/nginx-deploy --replicas=5
deployment.apps/nginx-deploy scaled
➜ kubernetes git:(main) kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 5/5 5 5 3m20s
➜ kubernetes git:(main) kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-deploy-c9bcb48d4 5 5 5 3m34s
➜ kubernetes git:(main) kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deploy-c9bcb48d4-blnpp 1/1 Running 0 69s
nginx-deploy-c9bcb48d4-bsgsr 1/1 Running 0 3m46s
nginx-deploy-c9bcb48d4-rh76q 1/1 Running 0 3m46s
nginx-deploy-c9bcb48d4-sg97l 1/1 Running 0 3m46s
nginx-deploy-c9bcb48d4-tvtq8 1/1 Running 0 69s
RollingUpdate a Deployment
RollingUpdate Deployments support running multiple versions of an application at the same time. When you or an autoscaler scales a RollingUpdate Deployment that is in the middle of a rollout (either in progress or paused), the Deployment controller balances the additional replicas in the existing active ReplicaSets (ReplicaSets with Pods) in order to mitigate risk. This is called proportional scaling.
For example, you are running a Deployment with 10 replicas, maxSurge=3, and maxUnavailable=2. Configuration look like:
Kemudian coba jalankan dengan perintah berikut:
kubectl apply -f rollingupdate-deployment.yaml
Maka outputnya seperti berikut:
➜ kubectl apply -f 03-workloads/01-basic-deploy/rollingupdate-deployment.yaml
deployment.apps/rollingupdate-nginx-deploy created
➜ kubectl get deploy rollingupdate-nginx-deploy
NAME READY UP-TO-DATE AVAILABLE AGE
rollingupdate-nginx-deploy 10/10 10 10 2m25s
NAME DESIRED CURRENT READY AGE
rollingupdate-nginx-deploy-cd8ddf7d8 10 10 10 9s
➜ kubernetes git:(main) ✗ kubectl get pod
NAME READY STATUS RESTARTS AGE
rollingupdate-nginx-deploy-cd8ddf7d8-7ffkl 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-7p5x2 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-f2pcm 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-fxvbb 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-hw67p 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-j2zls 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-rwln8 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-tw8m5 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-v5kt2 1/1 Running 0 30s
rollingupdate-nginx-deploy-cd8ddf7d8-zzt44 1/1 Running 0 30s
Sekarang kita coba update image menjadi latest dengan perintah berikut:
kubectl set image deploy/rollingupdate-nginx-deploy nginx=nginx:latest &&\
kubectl get rs -w
Maka outputnya seperti berikut:
➜ kubectl set image deploy/rollingupdate-nginx-deploy nginx=nginx:latest &&\
kubectl get rs -w
deployment.apps/rollingupdate-nginx-deploy image updated
NAME DESIRED CURRENT READY AGE
rollingupdate-nginx-deploy-cd8ddf7d8 10 10 10 0s
rollingupdate-nginx-deploy-cd8ddf7d8 8 8 8 1s
rollingupdate-nginx-deploy-77599d4db8 10 10 8 3s
rollingupdate-nginx-deploy-cd8ddf7d8 2 2 2 4s
rollingupdate-nginx-deploy-cd8ddf7d8 0 0 0 4s
rollingupdate-nginx-deploy-77599d4db8 10 10 10 5s
Pause and Resume rollout of a Deployment
When you update a Deployment, or plan to, you can pause rollouts for that Deployment before you trigger one or more updates. When you’re ready to apply those changes, you resume rollouts for the Deployment. This approach allows you to apply multiple fixes in between pausing and resuming without triggering unnecessary rollouts.
For example:
Get the rollout status:
kubectl get rs
Jika dijalankan outputnya seperti berikut:
➜ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
rollingupdate-nginx-deploy 10/10 10 10 33m
➜ kubectl get rs
NAME DESIRED CURRENT READY AGE
rollingupdate-nginx-deploy-77599d4db8 10 10 10 26m
Then when you update the deployment, such as image version using kubectl set image deploy/rollingupdate-nginx-deploy nginx=nginx:1.16.1 The output look like:
➜ kubectl rollout pause deploy/rollingupdate-nginx-deploy
deployment.apps/rollingupdate-nginx-deploy paused
➜ kubectl set image deploy/rollingupdate-nginx-deploy nginx=nginx:1.16.1
deployment.apps/rollingupdate-nginx-deploy image updated
➜ kubectl rollout history deploy/rollingupdate-nginx-deploy
deployment.apps/rollingupdate-nginx-deploy
REVISION CHANGE-CAUSE
1 <none>
2 <none>
➜ kubectl rollout status deploy/rollingupdate-nginx-deploy
Waiting for deployment "rollingupdate-nginx-deploy" rollout to finish: 0 out of 10 new replicas have been updated...
➜ kubernetes git:(main) kubectl describe deploy rollingupdate-nginx-deploy
Name: rollingupdate-nginx-deploy
Namespace: default
CreationTimestamp: Sat, 08 Apr 2023 15:52:55 +0700
Labels: app=nginx
env=test
Annotations: deployment.kubernetes.io/revision: 2
Selector: app=nginx,env=test
Replicas: 10 desired | 0 updated | 10 total | 10 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 2 max unavailable, 3 max surge
Pod Template:
Labels: app=nginx
env=test
Containers:
nginx:
Image: nginx:1.16.1
Port: 80/TCP
Host Port: 0/TCP
Conditions:
Type Status Reason
----------------
Available True MinimumReplicasAvailable
Progressing Unknown DeploymentPaused
OldReplicaSets: rollingupdate-nginx-deploy-77599d4db8 (10/10 replicas created)
NewReplicaSet: <none>
Notice that no new rollout started, because the object has been pause. to solve this you need run resume rollout with this command:
➜ kubectl rollout status deploy/rollingupdate-nginx-deploy
deployment "rollingupdate-nginx-deploy" successfully rolled out
➜ kubectl rollout history deploy/rollingupdate-nginx-deploy
deployment.apps/rollingupdate-nginx-deploy
REVISION CHANGE-CAUSE
1 <none>
2 <none>
3 <none>
➜ kubectl describe deploy/rollingupdate-nginx-deploy
Name: rollingupdate-nginx-deploy
Namespace: default
CreationTimestamp: Sat, 08 Apr 2023 15:52:55 +0700
Labels: app=nginx
env=test
Annotations: deployment.kubernetes.io/revision: 3
Selector: app=nginx,env=test
Replicas: 10 desired | 10 updated | 10 total | 10 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 2 max unavailable, 3 max surge
Pod Template:
Labels: app=nginx
env=test
Containers:
nginx:
Image: nginx:1.16.1
Port: 80/TCP
Host Port: 0/TCP
Conditions:
Type Status Reason
----------------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
NewReplicaSet: rollingupdate-nginx-deploy-679cf9c85d (10/10 replicas created)
Status of a Deployment object
A Deployment enters various states during its lifecycle.
It can be progressing while rolling out a new ReplicaSet,
it can be complete,
or it can fail to progress.
To check status you can use this command:
kubectl describe deploy/<object-name>
Untuk informasi status temen-temen bisa check di property conditions: seperti berikut:
~ ➡ kubectl describe deploy/nginx-deploy
# <...>
Conditions:
Type Status Reason
----------------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
]]>Dimas MaryantoStudy cases: Microservice apps (Springboot Rest API)2023-02-26T16:19:54+07:002023-02-26T16:19:54+07:00/posts/devops/orchestration/kubernetes/pods/04b-study-cases-microservice-backend-appsHai semuanya, di materi study cases untuk Pod and Container specification kali ini adalah lanjutan dari sebelumnya yang lebih advanced lagi yaitu Create and build microservices dengan framework Springboot Rest API dengan architecture seperti berikut:
Okay nah terlihat sedikit berbeda dengan application monolith sebelumnya, disini setiap service akan saling berkomunikasi dengan menggunakan protocol yang lightweight (ringan) seperti Rest API, grpc, messaging bus, database shared dan lain-lain. Pada study kasus kali ini terlihat pada diagram tersebut masih menggunakan physical / virtual-machine deployement kita akan migrasikan menggunakan orchestration container system dengan Kubernetes. Adapun tahap-tahap yang perlu kita lakukan yaitu
Develop aplikasi
How code works (Code Review)
The new architecture for orchestration container system
Containerize apps
Deploy to Kubernetes
Running as a Pod with namespace
Connecting other service from the another namespace
Specify container probes (health check)
Specify resource request and limit
Implement API Gateway using nginx reverse proxy
Ok tanpa berlama-lama yuk langsung aja kita bahas materi yang pertama:
Development aplikasi
Dalam mendevelop aplikasi, ada beberapa hal yang perlu di persiapakan yaitu Software Development Kit dan backing service seperti Database, source-code version control dan DevTools yaitu
Git
Java Development Kit (JDK) versi 17 keatas
MySQL Database
PostgreSQL Database
Apache Maven
Text editor seperti InteliJ IDEA, Visual Studio Code dan lain-lain
Rest API client seperti Postman, curl dan lain-lain
Docker
Jadi temen-temen perlu install software development kita tersebut untuk cara installnya sudah pernah saya bahas di kelas DevOps - Docker untuk Pemula s/d Mahir. Jika sudah temen bisa check dengan command seperti berikut:
~ » java -version
java version "19.0.1" 2022-10-18
Java(TM) SE Runtime Environment (build 19.0.1+10-21)
Java HotSpot(TM) 64-Bit Server VM (build 19.0.1+10-21, mixed mode, sharing)
~ » mvn -v
Apache Maven 3.9.0 (9b58d2bad23a66be161c4664ef21ce219c2c8584)
Maven home: /usr/local/Cellar/maven/3.9.0/libexec
Java version: 19.0.1, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk-19.jdk/Contents/Home
Default locale: en_ID, platform encoding: UTF-8
OS name: "mac os x", version: "13.2", arch: "x86_64", family: "mac"
~ » psql --version
psql (PostgreSQL) 14.7 (Homebrew)
~ » mysql --version
mysql Ver 8.0.32 for macos13.0 on x86_64 (Homebrew)
~ » curl --version
curl 7.86.0 (x86_64-apple-darwin22.0) libcurl/7.86.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.47.0
Release-Date: 2022-10-26
Setelah temen-temen menginstall semua Software Development Kit, kemudian yang kita butuhkan adalah source-code. Untuk source-code temen-temen bisa clone dari github repo berikut perintahnya seperti berikut:
Setelah di clone kita coba jalankan projectnya, tetapi pertama kita perlu configure dulu databasenya. supaya gak ribet kita akan menggunakan docker-compose.yaml seperti berikut:
Kemudian coba jalankan dengan perintah berikut:
docker compose up -d&&\
docker compose ps
Maka hasilnya seperti berikut:
devops/k8s-springboot-microservice [main●] » docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
k8s-springboot-microservice-mysql-1 mysql:8.0 "docker-entrypoint.s…" mysql 47 seconds ago Up 41 seconds 0.0.0.0:3306->3306/tcp, 33060/tcp
k8s-springboot-microservice-postgres-1 postgres:15 "docker-entrypoint.s…" postgres 47 seconds ago Up 42 seconds 0.0.0.0:5432->5432/tcp
Setelah semua database running, sekarang kita jalankan masing-masing service dengan menggunakan perintah mvn clean -pl <module-name> spring-boot:run seperti berikut
mvn clean -pl customer spring-boot:run
maka outputnya seperti berikut:
devops/k8s-springboot-microservice [main●] » mvn clean -pl customer spring-boot:run
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/ ___)| |_)| | | | | ||(_| | ))))' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.0.2)
2023-02-12T12:05:11.295+07:00 INFO 9448 --- [ restartedMain] c.m.d.udemy.customer.MainApplication : Starting MainApplication using Java 19.0.1 with PID 9448 (/Users/dimasm93/Developer/dimas-maryanto.com/youtube/_projects/devops/k8s-springboot-microservice/customer/target/classes started by dimasm93 in /Users/dimasm93/Developer/dimas-maryanto.com/youtube/_projects/devops/k8s-springboot-microservice/customer)
2023-02-12T12:05:11.298+07:00 INFO 9448 --- [ restartedMain] c.m.d.udemy.customer.MainApplication : No active profile set, falling back to 1 default profile: "default"
2023-02-12T12:05:11.392+07:00 INFO 9448 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
2023-02-12T12:05:13.303+07:00 INFO 9448 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9090 (http)
2023-02-12T12:05:13.319+07:00 INFO 9448 --- [ restartedMain] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2023-02-12T12:05:13.319+07:00 INFO 9448 --- [ restartedMain] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.5]
2023-02-12T12:05:13.400+07:00 INFO 9448 --- [ restartedMain] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2023-02-12T12:05:13.402+07:00 INFO 9448 --- [ restartedMain] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2007 ms
2023-02-12T12:05:13.612+07:00 INFO 9448 --- [ restartedMain] o.f.c.internal.license.VersionPrinter : Flyway Community Edition 9.5.1 by Redgate
2023-02-12T12:05:14.208+07:00 INFO 9448 --- [ restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-02-12T12:05:14.248+07:00 INFO 9448 --- [ restartedMain] o.f.c.i.database.base.BaseDatabaseType : Database: jdbc:mysql://localhost:3306/customer4p1 (MySQL 8.0)
2023-02-12T12:05:14.373+07:00 INFO 9448 --- [ restartedMain] o.f.core.internal.command.DbValidate : Successfully validated 1 migration (execution time 00:00.030s)
2023-02-12T12:05:14.435+07:00 INFO 9448 --- [ restartedMain] o.f.c.i.s.JdbcTableSchemaHistory : Creating Schema History table `customer4p1`.`flyway_schema_history` ...
2023-02-12T12:05:14.575+07:00 INFO 9448 --- [ restartedMain] o.f.core.internal.command.DbMigrate : Current version of schema `customer4p1`: << Empty Schema >>
2023-02-12T12:05:14.590+07:00 INFO 9448 --- [ restartedMain] o.f.core.internal.command.DbMigrate : Migrating schema `customer4p1` to version "20230211170102 - schema-customer"
2023-02-12T12:05:14.694+07:00 INFO 9448 --- [ restartedMain] o.f.core.internal.command.DbMigrate : Successfully applied 1 migration to schema `customer4p1`, now at version v20230211170102 (execution time 00:00.131s)
2023-02-12T12:05:14.806+07:00 INFO 9448 --- [ restartedMain] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-02-12T12:05:16.662+07:00 INFO 9448 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9090 (http) with context path ''
2023-02-12T12:05:16.666+07:00 INFO 9448 --- [ restartedMain] o.s.b.d.a.OptionalLiveReloadServer : LiveReload server is running on port 35729
2023-02-12T12:05:16.686+07:00 WARN 9448 --- [ restartedMain] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2023-02-12T12:05:16.706+07:00 INFO 9448 --- [ restartedMain] c.m.d.udemy.customer.MainApplication : Started MainApplication in 6.034 seconds (process running for 6.653)
Sekarang kita coba check service custemer bisa meresponse api yang kita request dengan menggunakan perintah curl seperti berikut:
curl --location--request GET 'localhost:9090/api/customer/v1/findById/cust01'
Maka jika dijalankan outputnya seperti berikut:
Last login: Sun Feb 12 11:33:28 on ttys002
~ » curl --location--request GET 'localhost:9090/api/customer/v1/findById/cust01'-v
Note: Unnecessary use of -X or --request, GET is already inferred.
* Trying 127.0.0.1:9090...
* Connected to localhost (127.0.0.1) port 9090 (#0)> GET /api/customer/v1/findById/cust01 HTTP/1.1
< HTTP/1.1 200
< Content-Type: application/json
< Transfer-Encoding: chunked
< Date: Sun, 12 Feb 2023 05:12:26 GMT
{"id":"cust01","userId":"dimasm93","fullname":"Dimas Maryanto","alamat":"Bandung, Jawa Barat"}%
Itu artinya sudah ok, selanjutnya kita coba jalankan service order dengan perintah seperti berikut:
mvn clean -pl orders spring-boot:run
Maka outputnya seperti berikut:
devops/k8s-springboot-microservice [main●] » mvn clean -pl orders spring-boot:run
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/ ___)| |_)| | | | | ||(_| | ))))' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.0.2)
2023-02-12T12:16:07.346+07:00 INFO 11255 --- [ restartedMain] c.m.dimas.udemy.orders.MainApplication : Starting MainApplication using Java 19.0.1 with PID 11255 (/Users/dimasm93/Developer/dimas-maryanto.com/youtube/_projects/devops/k8s-springboot-microservice/orders/target/classes started by dimasm93 in /Users/dimasm93/Developer/dimas-maryanto.com/youtube/_projects/devops/k8s-springboot-microservice/orders)
2023-02-12T12:16:07.350+07:00 INFO 11255 --- [ restartedMain] c.m.dimas.udemy.orders.MainApplication : No active profile set, falling back to 1 default profile: "default"
2023-02-12T12:16:09.296+07:00 INFO 11255 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9091 (http)
2023-02-12T12:16:09.322+07:00 INFO 11255 --- [ restartedMain] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2023-02-12T12:16:09.608+07:00 INFO 11255 --- [ restartedMain] o.f.c.internal.license.VersionPrinter : Flyway Community Edition 9.5.1 by Redgate
2023-02-12T12:16:10.187+07:00 INFO 11255 --- [ restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-02-12T12:16:10.217+07:00 INFO 11255 --- [ restartedMain] o.f.c.i.database.base.BaseDatabaseType : Database: jdbc:postgresql://localhost:5432/order4p1 (PostgreSQL 15.2)
2023-02-12T12:16:10.364+07:00 INFO 11255 --- [ restartedMain] o.f.core.internal.command.DbValidate : Successfully validated 1 migration (execution time 00:00.047s)
2023-02-12T12:16:10.460+07:00 INFO 11255 --- [ restartedMain] o.f.c.i.s.JdbcTableSchemaHistory : Creating Schema History table "public"."flyway_schema_history" ...
2023-02-12T12:16:10.605+07:00 INFO 11255 --- [ restartedMain] o.f.core.internal.command.DbMigrate : Current version of schema "public": << Empty Schema >>
2023-02-12T12:16:10.627+07:00 INFO 11255 --- [ restartedMain] o.f.core.internal.command.DbMigrate : Migrating schema "public" to version "20230211171555 - create-order"
2023-02-12T12:16:10.696+07:00 INFO 11255 --- [ restartedMain] o.f.core.internal.command.DbMigrate : Successfully applied 1 migration to schema "public", now at version v20230211171555 (execution time 00:00.107s)
2023-02-12T12:16:12.807+07:00 INFO 11255 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9091 (http) with context path ''
2023-02-12T12:16:12.813+07:00 WARN 11255 --- [ restartedMain] o.s.b.d.a.OptionalLiveReloadServer : Unable to start LiveReload server
2023-02-12T12:16:12.866+07:00 INFO 11255 --- [ restartedMain] c.m.dimas.udemy.orders.MainApplication : Started MainApplication in 6.225 seconds (process running for 6.76)
Sekarang kita coba check service order dengan menggunakan perintah curl berikut:
curl --location--request POST 'localhost:9091/api/order/v1/checkout'\--header'Content-Type: application/json'\--data-raw'{
"userId": "cust01",
"item": "Macbook Pro 13\" (A1723)",
"qty": "2"
}'-v
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:9091...
* Connected to localhost (127.0.0.1) port 9091 (#0)> POST /api/order/v1/checkout HTTP/1.1
> Content-Type: application/json
> Content-Length: 82
>* Mark bundle as not supporting multiuse
< HTTP/1.1 200
< Content-Type: application/json
< Date: Sun, 12 Feb 2023 05:19:32 GMT
{"id":"1a02d9bd-8413-43f8-b6ac-bf3b7f8f3438","createdDate":"2023-02-12T12:19:31.905439","customer":{"id":"cust01","userId":"dimasm93","fullname":"Dimas Maryanto","alamat":"Bandung, Jawa Barat"},"item":"Macbook Pro 13\" (A1723)","qty":2}%
Ok ini artinya sudah okay semua.
How code works (Code Review)
Setelah kita mencoba menjalankan program tersebut tahap selanjutnya adalah memahami bagaimana aplikasi bisa running dengan cara code review. Hal ini juga menjadi yang terpenting ketika proses deliver dari Programmer/Developer ke DevOps Engineer untuk di deploy ke environment.
Sebagai seorang DevOps kita harus mengetahui dan mengerti setiap service dari microservice yang telah di deliver oleh programmer untuk di implementasikan, Dengan cara melakukan assesment, diskusi, atau technical meeting dengan team Developer/Programmer. Salah satu prosedure yang biasanya saya lakukan adalah
Developer/Programmer menjelaskan overview architecture service
Developer/Programmer menjelaskan service communication
Developer/Programmer menjelaskan how to configure communication between service
DevOps menyimpulkan & Memberikan saran terkait perancangan architecture baru
DevOps mengimplementasikan perancangan architecture tersebut ke environment
Developer/Programmer dan DevOps melakukan Testing functional secara berdampingan
DevOps melakukan Performance/Stress testing
Okay kita tidak akan membahas semuanya ya, karena keterbatasan waktu. Jadi kita bahas beberapa yang dirasa penting seperti point no 1, 2, 3, dan 4. Pada point no 1, kita sudah bahas dibahas section awal jadi kita skip. Selanjutnya kita bahas point no 2 yaitu Service Communication.
Service communication ini pada dasarnya menjelaskan setiap service memiliki dependency ke mana saja dan seperti apa komunikasinya. Pada dasarnya service communication ada beberapa cara yaitu menggunakan Shared Database, Rest API, RPC (khususnya grpc), dan Messaging bus. Untuk service customer dan orders ini basicly kita menggunakan Rest API, Okay untuk lebih jelas kita lihat diagram berikut:
Customer API - find by id
Orders API - create new order
Jadi klo kita perhatikan diagram no 1, flownya sangat simple hanya menggunakan database tetapi untuk no 2 selain database perlu call service customerAPI melalui Rest API. Yang jadi pertanyaan selanjutnya bagaimana konfigurasi koneksinya? Okay sekarang kita coba bedah kodingnya / code review.
Kalo kita lihat di project orders seperti berikut:
// orders/src/main/java/com/maryanto/dimas/udemy/orders/controller/OrderController.javapackagecom.maryanto.dimas.udemy.orders.controller;@Slf4j@RestController@RequestMapping("/api/order/v1")publicclassOrderController{@PostMapping("/checkout")publicResponseEntity<?>placeOrder(@RequestBodyRequestOrderDTOorder){ResponseEntity<CustomerDTO>responseCustomer=this.serviceCustomer.findById(order.getUserId());if(responseCustomer.getStatusCode()!=HttpStatus.OK){returnResponseEntity.badRequest().body("Customer not found!");}CustomerDTOcustomer=responseCustomer.getBody();OrderpurchaseOrder=newOrder();// set value here!try{purchaseOrder=this.repo.save(purchaseOrder);OrderDTOoutput=newOrderDTO();// set value here!returnResponseEntity.ok(output);}catch(Exceptionex){log.error("Can't proses checkout",ex);returnResponseEntity.internalServerError().body("Transaction can't be processed!!! \nPlease report to adminstrator");}}}// orders/src/main/java/com/maryanto/dimas/udemy/orders/service/CustomerService.javapackagecom.maryanto.dimas.udemy.orders.service;@ServicepublicclassCustomerService{@AutowiredpublicCustomerService(RestTemplaterest,@Value("${services.customer.host}")Stringhost,@Value("${services.customer.port}")Stringport,@Value("${services.customer.proto}")Stringproto,@Value("${services.customer.context-path}")StringcontextPath){this.customerHost=host;this.customerPort=port;this.customerProto=proto;this.customerContextPath=contextPath;this.rest=rest;}publicResponseEntity<CustomerDTO>findById(Stringid){StringbaseUrl=String.format("%s://%s:%s%s/api/customer/v1/findById/",this.customerProto,this.customerHost,this.customerPort,this.customerContextPath);returnthis.rest.getForEntity(baseUrl+"{userId}",CustomerDTO.class,id);}}
Dan selain itu juga, berikut adalah file application.yaml untuk menyimpan semua environment variable yang dipanggil pada source code tersebut:
Okay sekarang perhatikan penggalan code diatas, jadi cara kita mengconfigurasi koneksi ke service customerAPI yaitu menggunakan envionment variable yang tertera pada application.yaml seperti SERVICE_CUSTOMER_HOST, SERVICE_CUSTOMER_PORT, SERVICE_CUSTOMER_CONTEXT_PATH, SERVICE_CUSTOMER_PROTO So kita bisa pasang/override nilainya pada saat dijalankan diatas container.
Nah setelah kita breakdown cara kerja atau code review, semoga temen-temen bisa memahami dan mulai merumuskan architecture yang bisa menunjang workload tersebut.
The new architecture
Setelah kita melihat, diskusi, technical meeting, serta code review tahap selanjutnya adalah merumuskan architecture yang sesuai dengan workload dari service tersebut.
Karena aplikasi yang dibuat merupakan microservice architecture yang memiliki backing service (dependency) seperti database, atau bahkan service lainnya Maka kita perlu buat scope dan boundaries dari services tersebut. Adapun scope dan boundaries tersebut dibagi menjadi Primary dan Secondary backing service
Primary backing service yaitu Aplikasi yang tidak akan bisa startup ketika di running dalam suatu environment jika tanpa/menggunakan service tersebut (Mandatory), sedangkan Secondary backing service yaitu Aplikasi masih bisa berjalan tetapi secara functional belum lengkap.
Karena tujuan kita adalah deploy ke orchestration container system menggunakan kubernetes jadi, langsung aja disini saya gambarkan menggunakan pendekatan kubernetes object resource seperti berikut:
Berdasarkan diagram tersebut kita akan bagi menjadi 3 namespace yaitu default, orders dan customer. Dalam masing-masing namespace memiliki service yang kita pisahkan berdasarkan scopenya yaitu
Namespace default, terdiri dari pod dengan workload: nginx berfungsi untuk routing (API Gateway) ke service-service seperti order dan customer
Namespace customer, terdiri dari pod dengan workload: customerAPI dan MySQL sebagai primary backing service
Namespace order, terdiri dari pod dengan workload: orderAPI dan PostgreSQL sebagai primary backing service.
Containerize apps
Setelah kita men-design architecturenya untuk deploy ke orchestration container system seperti kubernetes. Tahap awal meng-implementasikan semua konsep tersebut adalah melakukan kontainerisasi (container image). Di tahap ini adalah paling dasar sebelum kita deploy diatas kubernetes, jika service/aplikasi tidak bisa dibuild ke container image maka sudah dipastikan tidak akan bisa lanjut ke tahap selanjutnya.
Okay langsung aja kita mulai buat containernya. Tetapi sebelum itu kita lihat lagi bagaimana cara deploy manual seperti section sebelumnya. Apa yang kita perlukan untuk menjalankan service tersebut???
Java Development Kit (jdk-17 or later)
Binary execute (jar)
Untuk vendor JDK yang kita gunakan di local environment menggunakan Oracle JDK 19, nah ini sebisa mungkin untuk versi dari SDK harus sama persis dengan yang terdapat di container. Karena Oracle JDK tidak tersedia secara public di docker hub, kita akan menggunakan vendor yang open-source yaitu OpenJDK dengan version yang sama yaitu openjdk-19
Sedangkan untuk binary execute atau file bundle yang telah dicomple kita bisa peroleh dengan menjalankan perintah:
Okay setelah semua kebutuhan terpenuhi, tahap selanjutnya kita buat Dockerfile seperti berikut:
Dockerfile untuk customerAPI, simpan dalam folder customer/Dockerfile seperti berikut:
Dockerfile untuk orderAPI, simpan dalam folder order/Dockerfile seperti berikut:
Dan yang terakhir, tambahkan service customerAPI dan orderAPI dalam docker-compose.yaml seperti berikut:
Sekerang coba jalankan perintah berikut:
docker compose build
Jika dijalankan hasilnya seperti berikut:
devops/k8s-springboot-microservice [main] » docker compose build customerAPI orderAPI
[+] Building 5.1s (7/7) FINISHED
=>[internal] load metadata for docker.io/library/openjdk:19-oraclelinux8 4.7s
=>[1/2] FROM docker.io/library/openjdk:19-oraclelinux8@sha256:a5f2327c217367af3729670628c7ad66799cd890bde4e14fad02c2ae59552424 0.0s
=>[internal] load build context 0.0s
=>=> transferring context: 145B 0.0s
=> CACHED [2/2] ADD target/orders-api.jar spring-boot.jar 0.0s
=> exporting to image 0.1s
=>=> naming to repository.dimas-maryanto.com:8087/dimmaryanto93/example/order-api:latest
=>[internal] load metadata for docker.io/library/openjdk:19-oraclelinux8 4.7s
=>[1/2] FROM docker.io/library/openjdk:19-oraclelinux8@sha256:a5f2327c217367af3729670628c7ad66799cd890bde4e14fad02c2ae59552424 0.0s
=>[internal] load build context 0.0s
=>=> transferring context: 145B
=> CACHED [2/2] ADD target/customer-api.jar spring-boot.jar 0.0s
=> exporting to image 1.8s
=>=> naming to repository.dimas-maryanto.com:8087/dimmaryanto93/example/customer-api:latest
devops/k8s-springboot-microservice [main] » docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
repository.dimas-maryanto.com:8087/dimmaryanto93/example/order-api latest f27ba96797db 17 minutes ago 522MB
repository.dimas-maryanto.com:8087/dimmaryanto93/example/customer-api latest 9d589fe904f0 26 minutes ago 523MB
Setelah container image di build, sekarang coba jalankan dengan perintah berikut:
docker compose up -d
Maka seperti berikut outputnya:
devops/k8s-springboot-microservice [main] » docker compose up -d[+] Running 7/7
⠿ Network k8s-springboot-microservice_default Created 0.2s
⠿ Volume "k8s-springboot-microservice_pg_data" Created 0.0s
⠿ Volume "k8s-springboot-microservice_mysql_data" Created 0.0s
⠿ Container k8s-springboot-microservice-mysql-1 Started 3.2s
⠿ Container k8s-springboot-microservice-postgres-1 Started 3.0s
⠿ Container k8s-springboot-microservice-customerAPI-1 Started 4.9s
⠿ Container k8s-springboot-microservice-orderAPI-1 Started 4.9s
devops/k8s-springboot-microservice [main] » docker compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
k8s-springboot-microservice-mysql-1 repository.dimas-maryanto.com:8086/mysql:8.0 "docker-entrypoint.s…" mysql About a minute ago Up 58 seconds 0.0.0.0:3306->3306/tcp, 33060/tcp
k8s-springboot-microservice-orderAPI-1 repository.dimas-maryanto.com:8087/dimmaryanto93/example/order-api:latest "java -Djava.securit…" orderAPI About a minute ago Up 56 seconds 0.0.0.0:9091->9091/tcp
k8s-springboot-microservice-postgres-1 repository.dimas-maryanto.com:8086/postgres:15 "docker-entrypoint.s…" postgres About a minute ago Up 58 seconds 0.0.0.0:5432->5432/tcp
Jika semua service sudah running dengan baik, sekarang coba temen-temen test lagi service tersebut apakah berjalan dengan baik di atas single container? jika sudah ok. Yeeey selamat temen-temen udah melewati level 1.
Berikutnya adalah temen-temen bisa push ke container registry. boleh ke DockerHub atau private registry dengan menggunakan perintah berikut:
Setelah container image di-build dan publish ke container registry, tahap selanjutnya adalah deploy ke kubernetes cluster. Tetapi sebelum itu siapkan dulu kubernetes cluster untuk workload tersebut. Settingan cluster untuk microservice ini agak berbeda dengan sebelumnya yaitu seperti berikut:
~ » minikube start -p springboot-microservice \--cpus 2 \--memory 4G \--insecure-registry 192.168.88.50:8086 \--nodes 3
😄 [springboot-microservice] minikube v1.29.0 on Darwin 13.2.1
✨ Using the hyperkit driver based on user configuration
👍 Starting control plane node springboot-microservice in cluster springboot-microservice
🔥 Creating hyperkit VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
📦 Preparing Kubernetes v1.26.1 on containerd 1.6.15 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring CNI (Container Networking Interface) ...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🔎 Verifying Kubernetes components...
🌟 Enabled addons: default-storageclass, storage-provisioner
👍 Starting worker node springboot-microservice-m02 in cluster springboot-microservice
🔥 Creating hyperkit VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
👍 Starting worker node springboot-microservice-m03 in cluster springboot-microservice
🔥 Creating hyperkit VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
🔎 Verifying Kubernetes components...
🏄 Done! kubectl is now configured to use "springboot-microservice" cluster and "default" namespace by default
~ » minikube profile springboot-microservice
✅ minikube profile was successfully set to springboot-microservice
~ » minikube addons enable registry-creds
❗ registry-creds is a 3rd party addon and is not maintained or verified by minikube maintainers, enable at your own risk.
❗ registry-creds does not currently have an associated maintainer.
▪ Using image docker.io/upmcenterprises/registry-creds:1.10
🌟 The 'registry-creds' addon is enabled
~ » minikube addons configure registry-creds
Do you want to enable AWS Elastic Container Registry? [y/n]: n
Do you want to enable Google Container Registry? [y/n]: n
Do you want to enable Docker Registry? [y/n]: y
-- Enter docker registry server url: 192.168.88.50:8086
-- Enter docker registry username: admin
-- Enter docker registry password:
Do you want to enable Azure Container Registry? [y/n]: n
✅ registry-creds was successfully configured
~ » kubectl get node
NAME STATUS ROLES AGE VERSION
springboot-microservice Ready control-plane 4m52s v1.26.1
springboot-microservice-m02 Ready <none> 3m4s v1.26.1
springboot-microservice-m03 Ready <none> 86s v1.26.1
~ » minikube addons enable metrics-server
💡 metrics-server is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
▪ Using image registry.k8s.io/metrics-server/metrics-server:v0.6.2
🌟 The 'metrics-server' addon is enabled
Jika pada pod registry-creds error, temen-temen bisa menggunakan alternative lain yaitu dengan membuat sebuah secret dengan perintah serperti berikut:
<your-registry-server> is your Private Docker Registry FQDN. Use https://index.docker.io/v1/ for DockerHub otherwise 192.168.88.50:8086 for private registry
<your-name> is your Docker username.
<your-pword> is your Docker password.
<your-email> is your Docker email.
Dan setelah itu, jika mau menjalankan pod dari insecure registry temen-temen perlu nambahkan property imagePullSecrets seperti berikut contohnya:
Setelah cluster kubernetes ready, kita coba membuat simple 2 pod yang simple menggunakan nginx dan httpd setelah itu kita coba test cni antara ke dua pod tersebut dengan perintah seperti berikut:
~ » kubectl run web1 --image 192.168.88.50:8086/nginx:mainline --port 80 &&\
kubectl expose pod/web1 --port 80 --type ClusterIP
pod/web1 created
service/web1 exposed
~ » kubectl get pod
NAME READY STATUS RESTARTS AGE
web1 1/1 Running 0 56s
~ » kubectl run web2 --image 192.168.88.50:8086/httpd:latest --port 80 &&\
kubectl expose pod/web2 --port 80 --type ClusterIP
pod/web2 created
service/web2 exposed
~ » kubectl get pod
NAME READY STATUS RESTARTS AGE
web1 1/1 Running 0 71s
web2 1/1 Running 0 30s
~ » kubectl exec web1 -- curl http://web2
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 45 100 45 0 0 918 0 --:--:-- --:--:<html><body><h1>It works!</h1></body></html>
----:--:-- 918
Jika temen-temen perhatikan pada saat membuat cluster, mengapa cluster ini menggunakan memory yang lebih besar yaitu 4G di setiap nodenya, Jadi menentukan resource tidak hanya di sisi Pod dan Containernya saja tetapi juga pada cluster nodenya juga kita harus sesuaikan.
Deploy to Kubernetes cluster
Setelah cluster kubernetes siap dan kita coba test network connection antar pod bisa juga, tahap selanjutnya adalah kita buat Kubernetes resourcesnya sepert Pod, dan Service yang masing-masing di kelompokan berdasarkan Namespace berdasarkan design architecture Kubernetes resources sebelumnya.
Seperti berikut untuk kubernetes resources dengan namespace customer-module
Kemudian kita coba jalankan dengan perintah berikut:
kubectl apply -f kubernetes/ns-customer-api.yaml
Maka hasilnya seperti berikut:
~ » kubectl apply -f kubernetes/ns-customer-api.yaml
namespace/customer-module created
configmap/mysql created
secret/mysql created
pod/mysql created
service/mysql created
pod/customer-api created
service/customer-api created
~ » kubectl get pod -n customer-module
NAME READY STATUS RESTARTS AGE
customer-api 1/1 Running 0 13s
mysql 1/1 Running 0 6m36s
~ » kubectl logs customer-api -n customer-module
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/ ___)| |_)| | | | | ||(_| | ))))' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.0.2)
2023-02-25T06:49:26.684Z INFO 1 --- [ main] c.m.d.udemy.customer.MainApplication : Starting MainApplication v0.0.1-SNAPSHOT using Java 19 with PID 1 (/spring-boot.jar started by root in /)
2023-02-25T06:49:31.550Z INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9090 (http)
2023-02-25T06:49:33.287Z INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-02-25T06:49:33.356Z INFO 1 --- [ main] o.f.c.i.database.base.BaseDatabaseType : Database: jdbc:mysql://mysql:3306/customer_api (MySQL 8.0)
2023-02-25T06:49:33.728Z INFO 1 --- [ main] o.f.core.internal.command.DbValidate : Successfully validated 1 migration (execution time 00:00.090s)
2023-02-25T06:49:33.815Z INFO 1 --- [ main] o.f.c.i.s.JdbcTableSchemaHistory : Creating Schema History table `customer_api`.`flyway_schema_history` ...
2023-02-25T06:49:34.140Z INFO 1 --- [ main] o.f.core.internal.command.DbMigrate : Current version of schema `customer_api`: << Empty Schema >>
2023-02-25T06:49:34.184Z INFO 1 --- [ main] o.f.core.internal.command.DbMigrate : Migrating schema `customer_api` to version "20230211170102 - schema-customer"
2023-02-25T06:49:34.298Z INFO 1 --- [ main] o.f.core.internal.command.DbMigrate : Successfully applied 1 migration to schema `customer_api`, now at version v20230211170102 (execution time 00:00.190s)
2023-02-25T06:49:38.606Z INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9090 (http) with context path ''
2023-02-25T06:49:38.639Z WARN 1 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2023-02-25T06:49:38.685Z INFO 1 --- [ main] c.m.d.udemy.customer.MainApplication : Started MainApplication in 15.321 seconds (process running for 17.433)
~ » kubectl exec mysql -n customer-module -it -- mysql -e 'show tables;' --database customer_api -u customer_api -p
Enter password:
+------------------------+
| Tables_in_customer_api |
+------------------------+
| customer |
| flyway_schema_history |
+------------------------+
~ » kubectl exec customer-api -n customer-module -- curl --location --request GET 'localhost:9090/api/customer/v1/findById/cust01' -v
Note: Unnecessary use of -X or --request, GET is already inferred.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /api/customer/v1/findById/cust01 HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.61.1
> Accept: */*
>
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0< HTTP/1.1 200
< Content-Type: application/json
< Transfer-Encoding: chunked
< Date: Sat, 25 Feb 2023 07:14:25 GMT
<
{ [100 bytes data]
100 94 0 94 0 0 30 0 --:--:--
{"id":"cust01","userId":"dimasm93","fullname":"Dimas Maryanto","alamat":"Bandung, Jawa Barat"}
Selanjutnya kita deploy untuk namespace orders seperti berikut:
Kemudian kita coba jalankan dengan perintah berikut:
kubectl apply -f kubernetes/ns-orders-api.yaml
Jika dijalankan maka hasilnya seperti berikut:
~ » kubectl apply -f kubernetes/ns-orders-api.yaml
namespace/orders-module created
configmap/postgresql created
secret/postgresql created
pod/postgresql created
service/postgresql created
pod/orders-api created
service/orders-api created
~ » kubectl get pod -n orders-module
NAME READY STATUS RESTARTS AGE
orders-api 1/1 Running 0 15s
postgresql 1/1 Running 0 15s
~ » kubectl logs orders-api -n orders-module
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/ ___)| |_)| | | | | ||(_| | ))))' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.0.2)
2023-02-25T07:05:01.905Z INFO 1 --- [ main] c.m.dimas.udemy.orders.MainApplication : Starting MainApplication v0.0.1-SNAPSHOT using Java 19 with PID 1 (/spring-boot.jar started by root in /)
2023-02-25T07:05:06.506Z INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 9091 (http)
2023-02-25T07:05:07.808Z INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-02-25T07:05:07.888Z INFO 1 --- [ main] o.f.c.i.database.base.BaseDatabaseType : Database: jdbc:postgresql://postgresql:5432/order_api (PostgreSQL 15.1)
2023-02-25T07:05:07.985Z INFO 1 --- [ main] o.f.core.internal.command.DbValidate : Successfully validated 1 migration (execution time 00:00.041s)
2023-02-25T07:05:08.013Z INFO 1 --- [ main] o.f.c.i.s.JdbcTableSchemaHistory : Creating Schema History table "public"."flyway_schema_history" ...
2023-02-25T07:05:08.127Z INFO 1 --- [ main] o.f.core.internal.command.DbMigrate : Current version of schema "public": << Empty Schema >>
2023-02-25T07:05:08.160Z INFO 1 --- [ main] o.f.core.internal.command.DbMigrate : Migrating schema "public" to version "20230211171555 - create-order"
2023-02-25T07:05:08.225Z INFO 1 --- [ main] o.f.core.internal.command.DbMigrate : Successfully applied 1 migration to schema "public", now at version v20230211171555 (execution time 00:00.121s)
2023-02-25T07:05:24.876Z INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 9091 (http) with context path ''
2023-02-25T07:05:25.288Z INFO 1 --- [ main] c.m.dimas.udemy.orders.MainApplication : Started MainApplication in 25.583 seconds (process running for 28.329)
~ » kubectl exec orders-api -n orders-module -- curl --location --request POST 'localhost:9091/api/order/v1/checkout' \
--header 'Content-Type: application/json' \
--data-raw '{"userId": "cust01",
"item": "Macbook Pro 13\" (A1723)",
"qty": "2"}' -v
Note: Unnecessary use of -X or --request, POST is already inferred.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9091 (#0)
> POST /api/order/v1/checkout HTTP/1.1
>
} [82 bytes data]
* upload completely sent off: 82 out of 82 bytes
< HTTP/1.1 500
< Content-Type: application/json
{"timestamp":"2023-02-25T08:05:26.037+00:00","status":500,"error":"Internal Server Error","path":"/api/order/v1100 204 0 122 100 82 163 109 --:--:-- --:--:-- --:--:-- 273
* Closing connection 0
Connecting other service from the another namespace
Setelah kita deploy ke kubernetes cluster, jika temen-temen perhatikan kembali output yang dihasilkan dari service orders-api kita melakukan request ke endpoint /api/order/v1/checkout hasilnya HTTP/Status 500, Internal Server Error klo kita lihat dari log errornya pada service tersebut seperti berikut:
~ » kubectl logs orders-api -n orders-module
ith path [] threw exception [Request processing failed: org.springframework.web.client.ResourceAccessException: I/O error on GET request for"http://localhost:9090/api/customer/v1/findById/cust01": Connection refused] with root cause
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:535) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:585) ~[na:na]
at java.base/java.net.Socket.connect(Socket.java:666) ~[na:na]
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:178) ~[na:na]
Nah jika temen-temen perhatikan pada log tersebut, terlihat ada error gak bisa connect ke localhost:9090/api/xxx/xxx dari orders-api, Sedangkan kita mau ngambil data dari service customer-api. Lantas gimana caranya?
Kita sebelumnya udah membuat service untuk masing-masing pod seperti berikut:
~ » kubectl get service -n orders-module
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
orders-api NodePort 10.109.82.183 <none> 9091:31963/TCP 144m
postgresql NodePort 10.100.81.138 <none> 5432:32759/TCP 144m
~ » kubectl get service -n customer-module
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
customer-api NodePort 10.99.232.190 <none> 9090:31920/TCP 122m
mysql NodePort 10.96.9.233 <none> 3306:32554/TCP 172m
Nah jadi kalau dari kasus sebelumnya kita mau koneksi ke database maka kita bisa menggunakan hostname dari nama service tersebut misalnya jdbc:postgresql://postgresql:5432/xxxx nah serupa dengan hal tersebut dengan mengkases service customer-api dari orders-api tapi bagaimana dengan berbeda namespase.
Kita bisa menggunakan Kubernetes DNS queries untuk memangil service lain yang terletak pada namespace lain, seperti berikut:
Kita coba panggil service customer-api dari pod order-api dengan simple curl seperti berikut:
kubectl exec orders-api -n orders-module -- curl --location--request GET 'http://customer-api.customer-module:9090/api/customer/v1/findById/cust01'-v
Coba klo kita jalankan outputnya seperti berikut:
~ » kubectl exec orders-api -n orders-module -- curl --location--request GET 'http://customer-api.customer-module:9090/api/customer/v1/findById/cust01'-v
Note: Unnecessary use of -X or --request, GET is already inferred.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.99.232.190...
* TCP_NODELAY set* Connected to customer-api.customer-module (10.99.232.190) port 9090 (#0)> GET /api/customer/v1/findById/cust01 HTTP/1.1
> Host: customer-api.customer-module:9090
> User-Agent: curl/7.61.1
> Accept: */*>
< HTTP/1.1 200
< Content-Type: application/json
< Transfer-Encoding: chunked
100 94 0 94 0 0 209 0 --:--:-- --:--:-- --:--:-- 209
* Connection #0 to host customer-api.customer-module left intact{"id":"cust01","userId":"dimasm93","fullname":"Dimas Maryanto","alamat":"Bandung, Jawa Barat"}
Nah sekarang sudah okay bisa mendapatkan response, kita coba update specifikasi podnya dengan menambahkan variable SERVICE_CUSTOMER_HOST, SERVICE_CUSTOMER_PORT, SERVICE_CUSTOMER_PROTO dengan membuat configmap dan tambahkan ke spec pod orders-api seperti berikut:
Sekarang kita coba jalankan dengan perintah berikut:
~ » kubectl delete pod/orders-api -n orders-module
pod "orders-api" deleted
~ » kubectl apply -f kubernetes/ns-orders-api.yaml
configmap/orders-api created
pod/orders-api created
~ » kubectl get pod -n orders-module
NAME READY STATUS RESTARTS AGE
orders-api 1/1 Running 0 40s
postgresql 1/1 Running 0 166m
~ » kubectl logs orders-api -n orders-module
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/ ___)| |_)| | | | | ||(_| | ))))' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.0.2)
2023-02-25T09:51:19.825Z INFO 1 --- [ main] c.m.dimas.udemy.orders.MainApplication : Started MainApplication in 42.845 seconds (process running for 46.726)
~ » kubectl exec orders-api -n orders-module -- curl --location --request POST 'localhost:9091/api/order/v1/checkout' \
--header 'Content-Type: application/json' \
--data-raw '{"userId": "cust01",
"item": "Macbook Pro 13\" (A1723)",
"qty": "2"}' -v
* TCP_NODELAY set
* Connected to localhost (::1) port 9091 (#0)
> POST /api/order/v1/checkout HTTP/1.1
> Host: localhost:9091
> User-Agent: curl/7.61.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 82
>
} [82 bytes data]
* upload completely sent off: 82 out of 82 bytes
100 82 0 0 100 82 0 19 0:00:04 0:00:04 --:--:-- 19
{"id":"bb9f7d1e-92ef-4307-b412-ce112fbc01f6","createdDate":"2023-02-25T09:52:58.443768159","customer":{"id":"cust01","userId":"dimasm93","fullname":"Dimas Maryanto","alamat":"Bandung, Jawa Barat"},"item":"Macbook Pro 13\" (A1723)","qty":2}
< HTTP/1.1 200
Okay sampai sini kita sudah berhasil membuat microservice berjalan dengan baik di kubernetes, selanjutnya adalah kita buat feature kubernetes lebih advance lagi.
Specify container probes
Okay, sebelumnya kita khan sudah deploy container springboot untuk service customer-api dan orders-api ke Kubernetes cluster, hanya masih belum optimal contohnya startup service masih lama, kemudian jika database connection mati belum ada feature auto recover. Nah jadi untuk menambahkan feature tersebut ada yang perlu kita tambahkan pada framework springboot tersebut yaitu menggunakan lib spring-boot-starter-actuator yang kita tambahkan ke file pom.xml seperti berikut:
<?xml version="1.0" encoding="UTF-8"?><projectxmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"><!-- project settings --><dependencies><!-- other dependency --><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-actuator</artifactId></dependency></dependencies></project>
Dan edit/tambahkan property management.health pada file src/main/resources/application.yaml seperti berikut:
## other propertiesmanagement:endpoint:health:enabled:trueprobes:enabled:truegroup:liveness:include:-dbshow-components:alwayslogging:level:org.springframework:fatalorg.flywaydb.core.internal.license:fatalorg.hibernate:fatal
Jika sudah coba jalankan dengan menggunakan perintah berikut
Jika sudah seperti itu outputnya, lakukan hal yang sama dengan service orders-api dan kemudian coba re-build container image kemudian push menggunakan perintah berikut:
docker compose build &&\
docker compose push
Setelah kita push container image dari kedua service tersebut, sekarang kita bisa tambahkan feature container probe di kubernetes resource object pod seperti berikut:
Sekarang coba jalankan dengan perintah berikut:
kubectl delete pod customer-api -n customer-module &&\
kubectl delete pod orders-api -n orders-module
kubectl apply -f kubernetes/pod-container-probes.yaml
Maka hasilnya seperti berikut:
~ » kubectl delete -f kubernetes/pod-container-probes.yaml
pod "orders-api" deleted
pod "customer-api" deleted
~ » kubectl apply -f kubernetes/pod-container-probes.yaml
pod/orders-api created
pod/customer-api created
~ » kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
customer-module customer-api 0/1 ContainerCreating 0 6s
customer-module mysql 1/1 Running 0 9m29s
orders-module orders-api 0/1 Running 0 6s
orders-module postgresql 1/1 Running 0 9m12s
~ » kubectl describe pod customer-api -n customer-module
Name: customer-api
Namespace: customer-module
Priority: 0
Service Account: default
Node: springboot-microservice-m03/192.168.64.25
Start Time: Sat, 25 Feb 2023 23:53:15 +0700
Labels: app=customer-api
project=customer
tier=backend
Status: Running
IP: 10.244.2.9
IPs:
IP: 10.244.2.9
Events:
Type Reason Age From Message
-------------------------
Normal Scheduled 20s default-scheduler Successfully assigned customer-module/customer-api to springboot-microservice-m03
Normal Pulled 18s kubelet Container image "192.168.88.50:8086/dimmaryanto93/example/customer-api:v2" already present on machine
Normal Created 17s kubelet Created container customer-api
Normal Started 17s kubelet Started container customer-api
~ » kubectl get pod -n customer-module
NAME READY STATUS RESTARTS AGE
customer-api 1/1 Running 0 57s
mysql 1/1 Running 0 19m
~ » kubectl get pod -n orders-module
NAME READY STATUS RESTARTS AGE
orders-api 0/1 Running 0 14s
postgresql 1/1 Running 0 20m
~ » kubectl describe pod orders-api -n orders-module
Name: orders-api
Namespace: orders-module
Priority: 0
Service Account: default
Node: springboot-microservice-m03/192.168.64.25
Start Time: Sun, 26 Feb 2023 00:03:58 +0700
Labels: app=orders-api
project=orders-api
tier=backend
Annotations: <none>
Status: Running
IP: 10.244.2.14
IPs:
IP: 10.244.2.14
Events:
Type Reason Age From Message
-------------------------
Normal Scheduled 26s default-scheduler Successfully assigned orders-module/orders-api to springboot-microservice-m03
Normal Pulled 25s kubelet Container image "192.168.88.50:8086/dimmaryanto93/example/order-api:v2" already present on machine
Normal Created 25s kubelet Created container orders-api
Normal Started 25s kubelet Started container orders-api
~ » kubectl get pod -n orders-module
NAME READY STATUS RESTARTS AGE
orders-api 1/1 Running 0 70s
postgresql 1/1 Running 0 20m
Specify resource request and limit
Tahap selanjutnya untuk mengoptimalkan penggunaan resource node/worker kita harus specifikasi resource request dan limit. Resource request dan limit ini sangat membantu untuk menambahkan alert pada system kubernetes agar pod tidak menggunakan resource yang berlebih atau masih bisa mencukupi resource yang ada (resource availablity).
Untuk menentukan resource request dan limit ini kita harus mengetahui dulu titik minimum dari workload aplikasi kita dan titik optimal (rata-rata operational) dengan cara melihat activity process, metrics untuk penggunaan cpus dan memory. Sebagai contoh disini saya menggunakan Activity Monitor untuk melihat penggunaan system memory dan cpus.
Berikut adalah cpu yang terpakai dari workload customer-api:
Berikut adalah memory yang terpakai:
Atau selain itu juga kita bisa expose dari service tersebut dengan menambahkan beberapa property metrics pada src/main/resources/application.yaml seperti berikut:
## other propertiesmanagement:endpoints:web:exposure:include:health,metrics
Temen-temen bisa check menggunakan beberapa property seperti jvm.memory.used, system.cpu.usage, process.cpu.usage dan lain-lain. Okay selanjutnya kita build ulang container imagenya supaya metrics tersebut terexpose dengan perintah
Okay jadi disini saya simpulkan bahwa batas minimum atau resource request yaitu cpu = 100m dan memory = 250Mi sedangkan untuk maximum limit yaitu cpu = 1500m dan memory = 2000Mi. Sekarang kita coba update file pod.yaml menjadi seperti berikut:
Sekarang kita coba jalankan file tersebut dengan perintah berikut:
kubectl delete pod customer-api -n customer-module &&\
kubectl delete pod orders-api -n orders-module
kubectl apply -f kubernetes/pod-resource-request-limit.yaml
Maka hasilnya seperti berikut:
~ » kubectl delete pod customer-api -n customer-module &&\
kubectl delete pod orders-api -n orders-module
pod "customer-api" deleted
pod "orders-api" deleted
~ » kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
springboot-microservice 497m 24% 1360Mi 34%
springboot-microservice-m02 266m 13% 1571Mi 40%
springboot-microservice-m03 491m 24% 919Mi 23%
~ » kubectl apply -f kubernetes/pod-resource-request-limit.yaml
pod/customer-api created
pod/orders-api created
~ » kubectl get pod -n customer-module
NAME READY STATUS RESTARTS AGE
customer-api 1/1 Running 0 55s
mysql 1/1 Running 0 6m53s
~ » kubectl top pod customer-api -n customer-module
NAME CPU(cores) MEMORY(bytes)
customer-api 536m 260Mi
~ » kubectl get pod -n orders-module
NAME READY STATUS RESTARTS AGE
orders-api 1/1 Running 0 91s
postgresql 1/1 Running 0 7m6s
~ » kubectl top pod orders-api -n orders-module
NAME CPU(cores) MEMORY(bytes)
orders-api 9m 273Mi
~ » kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
springboot-microservice 151m 7% 1349Mi 34%
springboot-microservice-m02 62m 3% 1861Mi 47%
springboot-microservice-m03 45m 2% 1184Mi 30%
Implement API Gateway using nginx reverse proxy
Okay sebelumnya kita sudah deploy workload ke kubernetes cluster, kemudian juga sudah lakukan tuning workloadnya supaya bisa berjalan dengan baik di kubernetes dengan menambahkan container probe dan resource request & limit. Ini adalah ujung dari studikasus ini yaitu kita akan memasang API Gateway. okay temen-temen ada yang tau apa itu API Gateway? mengapa harus menggunakan API Gateway?
An API gateway is an API management tool that sits between a client and a collection of backend services. An API gateway acts as a reverse proxy to accept all application programming interface (API) calls, aggregate the various services required to fulfill them, and return the appropriate result.
Nah karena kita punya banyak services (microservices) jadi pada implementasinya exposing service tersebut tidak satu-per-satu atau setiap service memiliki port sendiri tetapi biasanya menggunakan methode DMZ (Demilitarized Zone) atau satu gerbang untuk beberapa service jika kita ilustrasikan seperti berikut:
Untuk implementasi API Gateway ini ada lumayan banyak salah satunya adalah apisix, nginx, KrakenD dan lain-lain. Untuk kasus kali ini kita akan menggunakan yang simple dulu ya yaitu Nginx reverse proxy dengan configurasi seperti berikut:
Sekarang coba jalankan dengan perintah berikut:
kubectl apply -f kubernetes/api-gateway.yaml
Maka hasilnya seperti berikut:
~ » kubectl apply -f kubernetes/api-gateway
configmap/api-gateway unchanged
pod/api-gateway created
service/api-gateway configured
~ » kubectl get pod,service
NAME READY STATUS RESTARTS AGE
pod/api-gateway 1/1 Running 0 54s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/api-gateway NodePort 10.107.243.20 <none> 80:30001/TCP 167m
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 27h
~ » kubectl cluster-info
Kubernetes control plane is running at https://192.168.64.23:8443
CoreDNS is running at https://192.168.64.23:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
~ » curl http://192.168.64.23:30001/customers/api/customer/v1/findById/cust01 -v* Trying 192.168.64.23:30001...
* Connected to 192.168.64.23 (192.168.64.23) port 30001 (#0)> GET /customer/api/customer/v1/findById/cust01 HTTP/1.1
{"id":"cust01","userId":"dimasm93","fullname":"Dimas Maryanto","alamat":"Bandung, Jawa Barat"}
~ » curl --location--request POST 'http://192.168.64.23:30001/orders/api/order/v1/checkout'\--header'Content-Type: application/json'\--data-raw'{
"userId": "cust01",
"item": "Macbook Pro 13\" (A1723)",
"qty": "2"
}'-v
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 192.168.64.23:30001...
* Connected to 192.168.64.23 (192.168.64.23) port 30001 (#0)> POST /order/api/order/v1/checkout HTTP/1.1
{"id":"2c25434d-00ec-46e6-b37f-c080291019b2","createdDate":"2023-02-26T09:15:03.570489716","customer":{"id":"cust01","userId":"dimasm93","fullname":"Dimas Maryanto","alamat":"Bandung, Jawa Barat"},"item":"Macbook Pro 13\" (A1723)","qty":2}%
Nah jadi disini kesimpulannya yang kita expose adalah ip 192.168.64.23 dengan port 30001 saja ke router/loadbalancer yang selanjutnya kita bisa lakukan untuk port forwarding ke ip public.
]]>Dimas MaryantoWelcome to the Nutanix HCF (Hybrid Cloud Fundamental)2023-02-05T12:00:40+07:002023-02-05T12:00:40+07:00/posts/cloud-computing/nutanix/ahv/01-welcome-nutanix-classHai semuanya, selamat datang di kelas ‘SRE - Nutanix AHV untuk Pemula sampai Mahir’ di section pertama kali ini kita akan membahas technology yang biasanya digunakan oleh team Infra/Operation yaitu Hyperconverged Infrastructure.
Nah dengan kecanggihan teknology jaman sekarang, technologies dapat membantu mendorong kemajuan suatu perusahan dalam meningkatan performa dalam menentukan goals. Selain itu juga dengan technology perusahaan dapat menentukan flexibility, choice, agility, and cost efficiency supaya memastikan bahwa perusahaan tersebut bisa competitive in the marketplace and innovate, bringing new services and products to their customers faster than ever.
Okay untuk memenuhi semua tantangan tersebut, jadi kita akan membahas dari yang funamental yaitu The Nutanix Cloud Fundamental cource, it designed for familiarize the features, components and capabilities of Nutanix cluster as if this was your first week on the job with a freshly installed Nutanix cluster. Sebelum kita mulai practice lab dengan Nutanix cluster kita akan bahas dulu
What is Nutanix?
What we can do with Nutanix?
Why Nutanix?
Okay tanpa berlama-lama yuk langsung aja kita bahas materi yang pertama:
What is Nutanix?
Nutanix, Inc. is an American cloud computing company that sells software, cloud services (such as desktops as a service, disaster recovery as a service, and cloud monitoring), and software-defined storage. Nutanix was founded on September 23, 2009 by Dheeraj Pandey, Mohit Aron and Ajeet Singh.
Jadi Nutanix adalah suatu perusahaan dengan main operation bisnis software yang meng-combine storage, computing, and virtualization menjadi satu kesatuan yang disebut Hyperconverged Infrastructure. Produk utama dari Nutanix yaitu Acropolis, Prism, Era, Frame, and Files. Serta pada tahun 2015 nutanix membuat Linux KVM based hypervisor yang disebut AHV (Acropolis HyperVisor) untuk memanage computer infrastructure lebih mudah.
Pada awal Nutanix berdiri fokus utama bisnis perusahaan adalah berjualan Hardware appliances sama seperti IBM, Lenovo, Dell dan lain-lain tetapi pada 2021 beralih fokus menjadi subscription software. Dimana saat ini Nutanix menjadi partner beberapa vendor hardware tersebut, dengan menjadikan software Nutanix bisa dijalankan diatas hardware vendor seperti IBM, Lenovo.
What we can do with Nutanix?
The Nutanix Cloud Platform comprises hybrid cloud infrastructure, multicloud management along with unified storage, database services, and desktop services, to support any application and workload, in any location.
Nutanix menawarkan beberapa solusi untuk workload seperti Cloud, Nutanix Database Service, Unified storage dan Applications.
Database: Database as a Services, Database operations dan lain-lain
Unified storage: Files storage, Object storage, dan lain-lain
Application: AI/ML, Big Data & Analitics dan lain-lain
End User Computing: VDI, VMware, Citrix dan lain-lain
Why Nutanix?
Mungkin dari temen-temen bertanya Nutanix ini software Virtualisasi mengapa tidak menggunakan VMWare atau sejenisnya? the answer is yes but not simple as that.
We solve the toughest cloud challenges, Nutanix unifies public cloud simplicity and agility with private cloud performance and security. Whether on-premises or hybrid, you’ll ensure business continuity through centralized management, one-click operations, and AI-driven automation. It’s your choice.
Your apps: IDC says there will be 500 million new apps by 2023. Nutanix makes it easy to deploy, manage and provision them.
Your infrastructure: Compute, storage, virtualization, and networking are hyperconverged and driven by software on the Nutanix Cloud Platform.
Your cloud: Whether hybrid, multicloud, public or private, you can move apps effortlessly anywhere you want with zero data loss.
Jika kita lihat, Nutanix ada 4 prinsip yaitu
The power of simplicity: We empower a single team to manage applications, data and security across your multicloud environments.
We’re data centric: Deploy, manage and scale a single software-driven storage platform that spans multiple clouds and datacenters.
Unmatched flexibility: Our software is server, hypervisor and cloud agnostic so you can build an infrastructure that matches your needs.
Substantial value: In five years, we can reduce operating costs by 62% and deliver 477% ROI. We’re obsessed with your long-term success.
]]>Dimas MaryantoSilabus SRE - Nutanix AHV: Pemula sampai Mahir2023-02-04T20:23:05+07:002023-02-04T20:23:05+07:00/posts/cloud-computing/nutanix/ahv/00-silabus-nutanix-ahvHai semuanya, di cource ini kita akan membahas salah satu Profesi IT yaitu Site Reliability Engineering atau yang biasa kita kenal dengan SRE Engineer terkait dengan Nutanix HCI (Hyper-Converged Infrastructure).
Mungkin dari temen-temen masih asing atau mungkin sudah pernah mendengar istilah SRE Engineer tetapi gak teralu mengerti seperti apa profesi tersebut, Di instrustri IT khususnya tech company banyak sekali loh profesi-profesi selain programmer yang relate dengan IT salah satunya adalah Site Reliability Engineering. Nah terus apasih Site Reliability Engineering?
Site Reliability Engineering atau singkatan dari SRE adalah salah satu pekerjaan di bidang IT yang specialis menangani reliabilitas suatu aplikasi atau software agar bisa berjalan dengan baik pada suatu infrastruktur yang sesuai.
Mungkin dari temen-temen masih belum tergambar magsudnya seperti apa? Okay nah jika temen-temen banyangkan dalam suatu development lifecycle biasanya terdiri beberapa team yaitu developer/programmer, bisnis analis, project/product manager, dan operation. Berikut adalah tugas masing-masing team:
Bisnis analis: bertugas merumuskan design berdasarkan user requirement yang di translate ke diagram agar mudah di terjemahkan oleh developer
Developer: bertugas menterjemahkan hasil design yang dibuat oleh bisnis analis menjadi software atau kode program
Operation: bertugas untuk mendeliver software atau kode program yang telah dibuat oleh programmer ke environment atau yang biasa disebut infrastructure.
Project/Product manager: bertugas untuk menjaga timeline, distribute task ke masing-masing team dan memastikan feature yang dibuat sesuai dengan user requirement.
Nah salah satu problem yang biasa ditemui di perusahaan besar adalah orang Developer/Programer gak ngerti infra sedangkan orang Operation gak ngerti Programming jadi begitu mau deliver software dari Developer/Programmer ke team Operation apakah langsung bisa jalan??? tentu saja tidak nah maka dari itu disini ada yang namanya SRE atau Site Reliability Engineering. Jadi secara umum tanggung jawab Site Reliability Engineering adalah untuk membantu atau sebagai penengah antara team Developer/Programmer dan team Operation.
Nah kira-kira udah ada gambaran sedikit khan tanggung jawab SRE seperti apa, nah jadi dikelas ini kita bahas dengan studi kasus menggunakan Nutanix HCI platform. Untuk memenuhi semua itu temen-temen perlu memahami dulu platform yang akan digunakan contohnya menggunakan Nutanix Hyper-Converged Infrastructure dan juga beberapa prinsip Developer/Programmer.
Materi yang akan dibahas di kelas ini adalah gabungan dari 3 bidang yaitu Infrastructur, Operation, Development dan DevOps knowledge seperti berikut:
Infrastructur knowledge
What is Nutanix HCI
Overview Hardware, Servers, dan Networking
Introduction to Nutanix AHV
Nutanix Prism Element
Nutanix Prism Central
Virtual Machine (VM)
Monitoring cluster
Lifecycle management
Operation knowledge
System operation
OS Server
Management files
Linux bash/shell script
Networking tools
Development knowlage
What is Programming language?
Types of programming language?
What is tools they used to develop application/software?
How to work with programmer?
DevOps knowlage
What is DevOps?
What is tools they used to automate deployment of application/software that developer created?
What the different between DevOps and SRE?
Okay mungkin sekian dulu yang bisa saya bahas terkait silabus yang akan kita pelajari di kelas ‘SRE - Nutanix AHV: Pemula sampai Mahir’, semoga temen-temen tertarik mengikuti kelas ini sampai akhir sampai jumpa dikelas!!!
