Compress gc logs and move gc log 

#!/bin/bash

SOURCE_DIR="/your/path/dir"
DEST_DIR="/your/dest/dir"


find "$SOURCE_DIR" -type f -name "*.gc.*" -mtime +30 -not -name "*.gz" -exec gzip {} \;
mkdir -p "$DEST_DIR"


mv "$SOURCE_DIR"/*.gz  "$DEST_DIR"/  2>/dev/null

Comments

Post a Comment

Popular posts from this blog

Fixing the DeepSpeed Import Error While Fine-Tuning the Qwen Model

While fine-tuning the Qwen model, I encountered an error in the finetune.py script: ImportError : cannot import name 'deepspeed' from 'transformers.deepspeed' After some investigation, I discovered the issue stems from a recent update in the Transformers library. The transformers.deepspeed module has been deprecated and replaced by transformers.integrations . To fix this, you need to update the import statement in your script. The Fix Replace this: from transformers.deepspeed import deepspeed With this: from transformers.integrations import deepspeed This small change resolved the error, allowing the fine-tuning process to proceed smoothly. Additional Resources For more details, refer to the discussion in the official Transformers GitHub repository: Issue #34582 Remember to keep your library versions up-to-date to avoid similar issues in the future! 
Read more

Amazon Linux 2023 - User data configuration for launch templates to connect to the EKS cluster

  Amazon Linux 2023 (AL2023) introduces a new node initialization process nodeadm that uses a YAML configuration schema. If you’re using self-managed node groups or an AMI with a launch template, you’ll now need to provide additional cluster metadata explicitly when creating a new node group MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="//" --// Content-Type: application/node.eks.aws --- apiVersion: node.eks.aws/v1alpha1 kind: NodeConfig spec: cluster: name: my-cluster-name-prd apiServerEndpoint: https://D59F.xyz.ap-northest-2.eks.amazonaws.com certificateAuthority: Y2VydGlmaWNhdGVBdXRob3JpdHk= cidr: 172.10.0.0/16 kubelet: config: clusterDNS: - 172.10.0.10 flags: - --node-labels=app=my-app,environment=production --// Content-Type: text/x-shellscript; charset="us-ascii" #!/bin/bash set -o errexit set -o pipefail set -o nounset # Install additional packages yum install -y htop jq iptables-services --//--
Read more

How to create ISM policy and rotate logs in opensearch

  { "id" : "k8s_ingress-nginx" , "policy" : { "policy_id" : "k8s_ingress-nginx" , "description" : "Move indices into a cold state after 30 days and delete them after a year." , "default_state" : "hot" , "states" : [ { "name" : "hot" , "actions" : [ { "replica_count" : { "number_of_replicas" : 1 } } ] , "transitions" : [ { "state_name" : "cold" , "conditions" : { "min_index_age" : "3d" } ...
Read more