Posts

architect-associate

 https://www.cloud13.ch/2025/04/18/oracle-cloud-infrastructure-2025-architect-associate-study-guide/

How can I check EKS release checklist to make a workplan for upgrading

 1. Check major changes  https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions-standard.html#kubernetes-1.31 2. Check deprecated APIs  https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-31 3. https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html 4. https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html 5. https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html 6. Amazon EBS CSI Driver for mounting volumes 7. AMI 

How to config hosts automatically link specific templates

 In zabbix server Go to Configuration -> Actions -> Create action

Zabbix agent

 https://www.zabbix.com/download_agents ------------------- create a startup.sh /path/to/zabbix_agentd -c /path/to/conf/zabbix_agentd.conf ------------------- create shutdown.sh ps -ef|grep /path/to/zabbix_agentd |grep -v grep |awk '{print $2}'  |xargs kill -9 -------------------- zabbix_agentd.conf Include=/path/to/zabbix_agent/conf/zabbix_agentd/*.conf Timeout=10 HostnameItem=system.hostname ServerActive={10.1.2.123_zabbix_server_ip} ListenPort=10050 Server=10.1.2.123 LogFileSize=10 LogFile=/path/to/logs/zabbix_agentd.log PidFile=/path/to/zabbix_agent/zabbix_agentd.pid --------------------- /path/to/conf/zabbix_agentd/example.conf example.conf ----------- UserParameter=system.test,who|wc -l UserParameter=check_cpu,./custom_script.sh UserParameter=mysql.replication.discovery[*], mysql -h"$1" -P"$2" -sNX -e "show slave status" UserParameter=mysql.slave_status, mysql -h"$1" -P"$2" -sNX -e "show slave status" -----------...

Security groups for ALB

Public ALB: -----------------------rules-------- - Inbound rules: ipv4 http tcp 80 0.0.0.0/0 ipv6 http tcp 80  ::/0 ipv4 https tcp 443 0.0.0.0/0 ipv6 https tcp 443 ::/0 - Outbound rules:  Custom TCP TCP 30000 - xxxxx      eks_cluster_security_group1 Custom TCP TCP 30000 - xxxxx      eks_cluster_security_group2 -----------------------AZ 3 different zones-------- -----------------Listeners and rules---------------- http:80   -> redirect to https://#{host}:443/#{path}?#{query} status code: HTTP_301 ------ https:443   -> Forward to target group if path pattern is /xxx/* then forward to target1 if header is zzzz then forward or redirect to target2 if 

Compress gc logs and move gc log

#!/bin/bash SOURCE_DIR="/your/path/dir" DEST_DIR="/your/dest/dir" find "$SOURCE_DIR" -type f -name "*.gc.*" -mtime +30 -not -name "*.gz" -exec gzip {} \; mkdir -p "$DEST_DIR" mv "$SOURCE_DIR"/*.gz "$DEST_DIR"/ 2>/dev/null

Amazon Linux 2023 - User data configuration for launch templates to connect to the EKS cluster

  Amazon Linux 2023 (AL2023) introduces a new node initialization process nodeadm that uses a YAML configuration schema. If you’re using self-managed node groups or an AMI with a launch template, you’ll now need to provide additional cluster metadata explicitly when creating a new node group MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="//" --// Content-Type: application/node.eks.aws --- apiVersion: node.eks.aws/v1alpha1 kind: NodeConfig spec: cluster: name: my-cluster-name-prd apiServerEndpoint: https://D59F.xyz.ap-northest-2.eks.amazonaws.com certificateAuthority: Y2VydGlmaWNhdGVBdXRob3JpdHk= cidr: 172.10.0.0/16 kubelet: config: clusterDNS: - 172.10.0.10 flags: - --node-labels=app=my-app,environment=production --// Content-Type: text/x-shellscript; charset="us-ascii" #!/bin/bash set -o errexit set -o pipefail set -o nounset # Install additional packages yum install -y htop jq iptables-services --//--

Expand EBS volume and extend logical volume

 1. Modify the target EBS volume to increase it's size 2.  sudo pvs  --> to check the pv that needs to be resized inside the instance 3. sudo pvresize /dev/xxxxxxx  displayed in the step 2 4. sudo pvs  -> to the the new volume size 5. sudo vgs  -> to see the detailed vgs 6. sudo lvextend -L +20G /dev/vg_name/lv_name 7. df -kh  -> to check the current value before  extending 8. sudo xfs_growfs /dev/vg_name/lv_name df -h to check the current value after extending #IF YOUR filesystem is ext4, please change the xfs_growfs to another value

Understand Persistent Volume, Volume Group, Logical volume

 We can have multiple persitent volumes to be attached to an EC2 instance Then we will group some of the volumes into a volume group After that we can create multiple logical volumes from that volume group

Resize EBS volume is AWS

 sudo file -s /dev/nvme0n1 -> xfs sudo lsblk if the file system is mounted directly to the EBS volume, then we can use the command below  sudo xfs_growfs -d /your-folder-path df -hT sudo lsblk  ---------- Can not resize down the EBS volume Before making changes, please take a snapshot

Wp

------------------Sheet Workplan----------------- Date 1. Overview     Purpose:     Contents: Environment:  WorkTargets Current issues, Analysis Result, Joblist Reviewer: 2. Working Schedule:     Service impact     Checker     Supervisor:    3. Pre-backup job Backup target Backup method Backup time Remark 4. Working process and recover process 5. Emergency contact list 6. Othes ------------------Sheet Job Detail list----------------- ------------------Sheet Job process----------------- No: Prework Job Title: Setup Job Content: Details about the job Worker: The person in charge, or a group of people Start-time: End-time: Time:  System job:          Where: Servers, urls          Commands:  Checkers:  Who reviews the Job  Checking method:           Where:           Commands: Check expected:  ...

Elasticsearch version migration

1. Prepare files and packages and plugins 2. Extract elasticsearch and install plugin       sudo bin/elasticsearch-plugin install file:///path/to/plugin.zip 3. Compress the elasticsearch after installing plugins 4. Move the compressed elasticsearch to the remote servers 5. Extract the compressed elasticsearch on the remote servers 6. Update concurrent recovery for the current cluster 7. Disable shard allocation 8. Node flush curl -X POST "localhost:9200/_flush/synced?pretty" GET /_cluster/health?pretty 9. Shutdown 10. Startup and tail logs 11. health_check node GET /_cluster/health?pretty 12. enable shard allocation host=$1 port=$2 curl -XPUT "http://YY.XX.ZZZ.YXZ:9200/_cluster/settings" -H 'Content-Type: application/json' -d '{ "transient" : { "cluster.routing.allocation.enable" : null } }' curl -XPUT "http://YY.XX.ZZZ.YXZ:9200/_cluster/settings" -H 'Content-Type: application/json' -d '{ ...

Reroute index shard from one node to another node

  POST /_cluster/reroute {    "commands": [    {       "move": {          "index": "my_index",          "shard": 0,          "from_node":  "node1",          "to_node": "node2"        }    }   ]  } { "commands": [ { "move": { "index": "my_index", "shard": 0, "from_node": "node1", "to_node": "node2" } } ] }

awk tutorial

 https://linuxhandbook.com/awk-command-tutorial/

Fix argocd error: error upgrading connection: error dialing backend: tls: failed to verify certificate: x509

 (base) nhbo@nhbo-LOQ-15IAX9:~$ kubectl port-forward svc/argocd-server -n argocd 8080:443 error: error upgrading connection: error dialing backend: tls: failed to verify certificate: x509: certificate is valid for 192.168.1.5, 2402:800:61c2:10de:d38d:2b9f:6969:e8e8, 2402:800:61c2:10de:73a5:a67f:7a26:b577, not 192.168.1.7 (base) nhbo@nhbo-LOQ-15IAX9:~$ sudo microk8s refresh-certs --cert ca.crt

Introduction to installation of Action Runner Controller with docker in docker

 We must install docker in the worker node --------- helm install arc     --namespace "${NAMESPACE}"     --create-namespace     oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller ------- helm install "${INSTALLATION_NAME}"     --namespace "${NAMESPACE}"     --create-namespace     --set githubConfigUrl="${GITHUB_CONFIG_URL}"     --set githubConfigSecret.github_token="${GITHUB_PAT}"   --set containerMode.type="dind"  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set --set containerMode.type="dind" : This will allow the container to use the docker in the host 

Fixing the DeepSpeed Import Error While Fine-Tuning the Qwen Model

While fine-tuning the Qwen model, I encountered an error in the finetune.py script: ImportError : cannot import name 'deepspeed' from 'transformers.deepspeed' After some investigation, I discovered the issue stems from a recent update in the Transformers library. The transformers.deepspeed module has been deprecated and replaced by transformers.integrations . To fix this, you need to update the import statement in your script. The Fix Replace this: from transformers.deepspeed import deepspeed With this: from transformers.integrations import deepspeed This small change resolved the error, allowing the fine-tuning process to proceed smoothly. Additional Resources For more details, refer to the discussion in the official Transformers GitHub repository: Issue #34582 Remember to keep your library versions up-to-date to avoid similar issues in the future! 

VPC, Private subnet, public subnet, Routing table and InternetGateway, NAT gateway

 Trước tiên chúng ta cần hiểu về khái niệm VPC  VPC là Một thành phố độc lập với các thành phố khác Sau đó chúng ta xây dựng 2 quận nhỏ trong thành phố này gọi là 2 subnets Về cơ bản thì 2 subnet này có dải mạng khác nhau nhưng do cùng trong thành phố nên chúng có thể nói chuyện và kết nối với nhau Tuy nhiên thì chúng vẫn không thể kết nối với các thiết bị hoặc tài nguyên khác bên ngoài thành phố Điều tiếp theo là chúng ta tạo ra một cái Internet gateway đóng vai trò như một cái cửa ngõ ra ngoài internet, để bước ra ngoài thành phố Và chúng ta sẽ lắp cái cửa ngõ này cho VPC, vì lúc ban đầu tạo ra thì cái cổng này nó đứng độc lập không gắn với bất kỳ VPC nào Tuy việc gắn cổng cho thành phố rồi nhưng các Subnet vẫn k thể kết nối internet Điều tiếp theo là cần phải tạo ra các routing table, nó đóng vai trò như những tuyến đường Các routing tables ban đầu đc tạo ra nó sẽ gắn vào VPC, sau đó thì trong Routing table nó sẽ định nghĩa các tuyến đường ví dụ như là đi từ Source là dải I...

Persistent Storage for Kubernetes with StorageClass

In Kubernetes, managing persistent storage involves creating a StorageClass , which defines the types of storage to be used for persistent volumes (PV). Here's a step-by-step explanation of how to use dynamic provisioning for persistent storage in a Kubernetes cluster. Step 1: Declare a StorageClass A StorageClass is a way to describe the provisioning of storage in a Kubernetes cluster. It defines parameters such as the storage provider (e.g., Amazon EBS, NFS, etc.) and other configuration details. Depending on your cluster, there may already be a default StorageClass . If not, you will need to manually declare one. Example of defining a StorageClass for Amazon EBS: apiVersion : storage.k8s.io/v1 kind : StorageClass metadata : name : ebs-sc provisioner : kubernetes.io/aws-ebs parameters : type : gp2 Step 2: Create a PersistentVolumeClaim (PVC) Once the StorageClass is defined, you can create a PersistentVolumeClaim (PVC) that requests a specific amount of storage from a S...

RAG (Retriever-Augmented Generation) System Overview

main.py - Setting Up Qdrant Collection In this part, you are setting up the Qdrant client and creating a collection where embeddings will be stored: Initialize Qdrant Client : You connect to the Qdrant service using the API key and Qdrant Cloud URL. Create Collection : You define a new collection ( book_data ) in Qdrant with a specified vector size (1024) and distance metric (Cosine similarity). Result : You confirm that the collection is created successfully. from qdrant_client import QdrantClient from qdrant_client.models import Distance, VectorParams qdrant_client = QdrantClient( url = "https://your-qdrant-cloud-url" , api_key = "your-api-key" ) qdrant_client.create_collection( collection_name = "book_data" , # Name of the collection vectors_config =VectorParams( size = 1024 , distance =Distance.COSINE) # Vector size and distance metric ) print ( "Collection 'book_data' created successfully!" ) add_data.py - Addin...