Operations Runbook

Purpose

This document describes operational procedures for maintaining and recovering the RentOne production environment.


Server Recovery Checklist

When the platform is inaccessible:

1. Check Server Availability
2. Check MongoDB
3. Check Eureka
4. Check Gateway
5. Check Backend Services
6. Check Nginx
7. Verify Domains

Check MongoDB

Status:

sudo systemctl status mongod

Restart:

sudo systemctl restart mongod

Logs:

sudo journalctl -u mongod -f

Check Nginx

Status:

sudo systemctl status nginx

Restart:

sudo systemctl restart nginx

Validate Configuration:

sudo nginx -t

Reload:

sudo systemctl reload nginx

Check Service Registry

sudo systemctl status backend-rentone-service-registry

Restart:

sudo systemctl restart backend-rentone-service-registry

Check API Gateway

sudo systemctl status backend-rentone-api-gateway

Restart:

sudo systemctl restart backend-rentone-api-gateway

Check All RentOne Services

systemctl list-units --type=service | grep rentone

Expected Output:

backend-rentone-api-gateway.service               loaded active running Backend-rentone-api-gateway Service
backend-rentone-cloud-app.service                 loaded active running Backend-rentone-cloud-app Service
backend-rentone-commerce-feedback-service.service loaded active running Backend-rentone-commerce-feedback-service Service
backend-rentone-commerce-service.service          loaded active running Backend-rentone-commerce-service Service
backend-rentone-file-mgmt.service                 loaded active running Backend-rentone-file-mgmt Service
backend-rentone-inventory-system.service          loaded active running Backend-rentone-inventory-system Service
backend-rentone-notifications.service             loaded active running Backend-rentone-notifications Service
backend-rentone-operation-service.service         loaded active running Backend-rentone-operation-service Service
backend-rentone-payment-gateway.service           loaded active running Backend-rentone-payment-gateway Service
backend-rentone-product-catalog.service           loaded active running Backend-rentone-product-catalog Service
backend-rentone-service-registry.service          loaded active running Backend-rentone-service-registry Service
backend-rentone-tenants.service                   loaded active running Backend-rentone-tenants Service
backend-rentone-users.service                     loaded active running Backend-rentone-users Service

All services should display:

loaded active running

If any service shows:

failed
inactive
activating
deactivating

investigate the service logs:

journalctl -u <service-name> -f

Restart All Services

sudo systemctl restart backend-rentone-service-registry
sudo systemctl restart backend-rentone-api-gateway
sudo systemctl restart backend-rentone-tenants
sudo systemctl restart backend-rentone-cloud-app
sudo systemctl restart backend-rentone-users
sudo systemctl restart backend-rentone-product-catalog
sudo systemctl restart backend-rentone-inventory-system
sudo systemctl restart backend-rentone-commerce-service
sudo systemctl restart backend-rentone-payment-gateway
sudo systemctl restart backend-rentone-file-mgmt
sudo systemctl restart backend-rentone-notifications
sudo systemctl restart backend-rentone-commerce-feedback-service
sudo systemctl restart backend-rentone-operation-service

SSL Certificate Renewal

Check:

sudo certbot certificates

Renew:

sudo certbot renew

Reload Nginx:

sudo systemctl reload nginx

Disk Usage

Check:

df -h

Largest Directories:

du -sh /*

Memory Usage

Check:

free -h

Processes:

top

or

htop

Production Domains

Customer Portal:

https://rentone.co.in

Admin Console:

https://console.admin.rentone.co.in

Backend API:

https://cloud.rentone.co.in

CDN:

https://cdn.rentone.co.in

Jenkins:

https://jenkins.rentone.co.in

Support Information

Infrastructure Provider: Akamai Linode

Operating System: Ubuntu 24.04.4 LTS

Java: OpenJDK 21

MongoDB: 7.0.31

Architecture: Spring Boot Microservices