Skip to main content

Rollback

When a newly deployed version turns out to have bugs or performance problems, you can return to the previous version with a single command in the Command Execution tab of the operations modal.

Rollback is needed in situations like these:

  • When unexpected errors occur after deploying a new version.
  • When performance degradation or memory leaks are discovered.
  • When compatibility issues with external services arise.
  • When serious bug reports come in from users.

Rollback Overview​

Rollback methods differ by runtime.

  • Kubernetes: There is no dedicated rollback button. Roll back by running kubectl rollout undo directly in the Command Execution tab of the operations modal.
  • Docker: There is no dedicated rollback UI. To roll back, redeploy a previous image tag from the Deploy stage.

Kubernetes Rollback​

The KIOPS UI has no dedicated Immediate Rollback button or Deployment Management tab (the service:deploy:rollback permission is defined, but the UI is not implemented). Kubernetes rollback is performed by running kubectl commands directly in the Command Execution tab of the operations modal.

Step 1: Open Operations and the Command Execution Tab​

  1. Click the Operations card for the service on the [Service Management] page.
  2. Select the Command Execution tab in the operations modal.

Step 2: Check Deployment History​

List the available revisions:

kubectl rollout history deployment/<deployment-name> -n <namespace>

Example Rollout History Output​

deployment.apps/my-app
REVISION  CHANGE-CAUSE
3         kubectl set image deployment/my-app api=registry/api:v1.0.2
4         kubectl set image deployment/my-app api=registry/api:v1.0.3
5         kubectl set image deployment/my-app api=registry/api:v1.0.4
  • The output is not a formatted table — it is the raw standard output of the kubectl command.
  • The entry with the highest REVISION number is the version currently deployed; the number just below it is the previous version.
  • The CHANGE-CAUSE column shows the command or annotation recorded at deployment time.

Step 3: Run the Rollback​

To revert to the previous revision:

kubectl rollout undo deployment/<deployment-name> -n <namespace>

To revert to a specific revision, add the --to-revision=<revision-number> flag:

kubectl rollout undo deployment/<deployment-name> --to-revision=<revision-number> -n <namespace>

Step 4: Verify Rollback Result​

After rollback completes, open the Pod List tab in the operations modal and verify the new ReplicaSet has started normally.

  • All Pod statuses are Running
  • Service responses are normal (Health Check passes)
  • No errors in logs
  • Key features work properly

Docker · Podman Rollback​

There is no dedicated rollback UI in the Docker·Podman environment. You roll back by selecting a previously successful image tag and redeploying.

Step 1: Find the Previous Image Tag​

  1. Select the service on the [Service Management] page.
  2. Check the build history or container registry for a previously working image tag. If the tag format is ${BRANCH}-${SHORT_SHA}, you can find it from the previous commit hash.

Step 2: Redeploy the Previous Image​

  1. Click the Deploy stage in the pipeline.
  2. In the deploy dialog, change the Image Version to the previous version.
  3. Make sure the Auto-backup before deploy option is enabled. With it on, the state right before redeploy is backed up so you can restore it if the redeploy fails.
  4. Click the Deploy button.

Step 3: Check Container Status​

In the operations modal's Container List tab, verify the new container is Running and the logs contain no errors.

Verify State After Rollback​

After a manual rollback, KIOPS's deployment history view and the actual cluster state may be temporarily out of sync. Verify the actual Pod state in the Pod List tab, and re-fetch the latest history with kubectl rollout history in the Command Execution tab if needed.

Rollback Precautions​

Database Migrations​

Be sure to check before rolling back.

  • Schema Changes: Verify if tables/columns were added or changed in the new version.
  • Data Compatibility: Verify if data saved in the new format is compatible with the previous version.
  • Migration Rollback: Review if the DB migration also needs to be rolled back.
Destructive Migration Warning

If destructive migrations like column deletion or type changes were applied, simple rollback may be difficult. Check DB backups and consult with your DBA.

Configuration Changes​

Settings to verify when rolling back.

  • Environment Variables: Are all environment variables needed by the previous version present?
  • ConfigMap: Are configuration values compatible with the previous version?
  • Secret: Were authentication credentials changed?

External Service Integration​

  • If external API versions changed, verify compatibility.
  • Check communication protocols with other microservices.

Handling Rollback Failures​

If the Previous Image Is Unavailable​

  • Deleted from registry: Rebuild from the previous version source code.
  • Tag overwritten: Try to restore using the image digest (SHA). If you check the image digest (SHA256) in Harbor or Docker Hub, you can restore to a specific version even if the tag was overwritten.

If Docker Redeploy Fails​

If the auto-backup-before-deploy option was enabled, you can restore the containers/volumes to the state captured at backup time. Open the [Backup Management] page, select the corresponding backup, and run the restore.

If Issues Persist After Rollback​

  1. Log Analysis: Identify the root cause from application logs.
  2. Environment Check: Review environment variables, ConfigMap, and Secret settings.
  3. External Dependencies: Check database and external API status.
  4. Earlier Version: Roll back further from the Command Execution tab if needed.

Best Practices to Prevent Rollbacks​

Recommendations to reduce situations requiring rollback.

Pre-deployment Verification Checklist​

  • Build: Verify tests passing and security scan passing.
  • Staging: Perform functional testing and performance testing.
  • Canary: Real environment verification with partial traffic.
  • Production: Set up monitoring dashboard and alert configuration.

Stable Rollout Settings (Kubernetes)​

Recommended settings for stable rollouts in Kubernetes.

  • Pod startup safety: Configure progressDeadlineSeconds.
  • Health Checks: Configure readinessProbe and livenessProbe.
  • Resource protection: Configure resources.limits.
  • Gradual rollout: Setting maxUnavailable: 0 keeps existing Pods until new Pods reach Ready status. If issues occur, new Pods never become Ready and the existing Pods continue to serve traffic.