Overview
The error "Error code 14205 (filesystem out of space) returned" occurs when attempting to enroll a node in a cluster, causing both appliances to show as master. This issue is often due to a full log partition on the secondary node, misconfigured cluster interfaces, and firmware version mismatches. The problem typically arises when the secondary node cannot communicate or synchronize with the primary node, leading to a split-brain scenario.
Information
Error Message: "Error code 14205 (filesystem out of space) returned"
Cause: The error is caused by a full log partition on the secondary node, preventing configuration saves and cluster synchronization. Misconfigured cluster interfaces and firmware version mismatches exacerbate the issue.
Resolution Steps:
-
Free Up Disk Space on the Secondary Node:
- SSH into the secondary node and check disk usage:
df -h | grep -E 'log|tms|snapshots' - Remove core dumps and old log files:
rm -f /var/opt/tms/snapshots/.running/mgmtd/core.* 2>/dev/null find /var/log -name *.gz -size +1M -delete - Verify free space:
df -h | grep -E 'log|tms|snapshots'
- SSH into the secondary node and check disk usage:
-
Correct Cluster Interface Configuration:
- Ensure eth2 is the only interface with the cluster role for internal communication.
- Set eth1 as the cluster master interface for the VIP.
-
Ensure Both Nodes Run the Same Firmware Version:
- Upgrade or downgrade one of the appliances to match the other (preferably to the latest stable version).
-
Perform a Factory Reset on the Standby Unit:
- Isolate the standby unit and perform a factory reset to address potential database corruption.
- Reconfigure the appliance as a standby and point it to the master unit.
-
Re-establish Cluster and Verify Health:
- Break and re-form the cluster to ensure a proper election.
- Verify cluster status to ensure one node is master and the other is standby.
Important: Ensure both nodes are on the same subnet and can communicate with each other. Monitor disk usage and adjust log retention policies to prevent future space issues.
Frequently Asked Questions
- How do I know if this error applies to my situation?
- You'll see the error message "Error code 14205 (filesystem out of space) returned" when attempting to enroll a node in a cluster, and both appliances may show as master.
- What should I do if the log partition is still full after deleting logs?
- If the log partition remains full, consider resizing or formatting it via the system setup interface. Be aware that formatting will erase all logs.
- How can I ensure both nodes are running the same firmware version?
- Check the firmware version in the Web UI under System > About or by running
show versionin the CLI. Upgrade or downgrade one of the appliances to match the other. - What if the cluster still won't form after following the steps?
- If the cluster still won't form, wipe the configuration from the standby node and let it re-sync from the master. Ensure all network configurations are correct and both nodes can communicate.
Priyanka Bhotika
Comments