Split the issue into multiple tickets

Oct 13, 2025 by pinkandtrash.com 143 views

Introduction

As a Celestia mainnet node operator, it's essential to ensure that your node is running smoothly and efficiently. However, recent issues with peer connectivity and stability have raised concerns about the performance of Celestia mainnet archive nodes. In this article, we'll delve into the issues faced by node operators and explore potential solutions to improve the stability of peering.

Background

Celestia mainnet archive nodes have been running without issues for a while, but over the past two to three weeks, both nodes have experienced difficulties syncing from other peers. The problem has become more pronounced, making it challenging to find reliable peers. Node operators have tried various seeds and peers from the official Celestia repository, but with limited success.

Symptoms and Errors

The issues are characterized by the following symptoms and errors:

Peer connection errors: The node is unable to establish stable connections with peers, resulting in errors such as "use of closed network connection" and "invalid peer."
Seed connection errors: The node is also experiencing issues connecting to seeds, with errors like "auth failure: secret conn failed" and "i/o timeout."
High bandwidth usage: Even when the node is not syncing blocks, it's still consuming a significant amount of bandwidth, often reaching the container limit.

Configurations and Logs

The node is running with the command celestia-appd start --home /home/celestia/.datadir, and the current configuration file is attached as config.toml.txt. Additionally, logs from the node are available as celestia-logs.txt.

Potential Solutions

To improve the stability of peering, node operators can try the following:

Use a reliable seed: Instead of relying on random seeds, use a trusted seed like 12ad7c73c7e1f2460941326937a039139aa78884@celestia-mainnet-seed.itrocket.net:40656.
Add a reliable peer: Add a trusted peer like d535cbf8d0efd9100649aa3f53cb5cbab33ef2d6@celestia-mainnet-peer.itrocket.net:40656 to the node's configuration.
Reset the addrbook.json: Reset the addrbook.json file to the one provided by ITRocket to ensure that the node is using a clean and up-to-date peer list.
Monitor bandwidth usage: Keep an eye on bandwidth usage and adjust the container limit as needed to prevent excessive consumption.

Normal Bandwidth Usage

The observed bandwidth usage of 10-30 Mb/s during block syncing is relatively normal. However, the high bandwidth usage when the node is not syncing blocks is concerning. Node operators should investigate this issue further to determine the cause and take corrective action.

Conclusion

The issues faced by Celestia mainnet archive nodes are complex and multifaceted. By understanding the symptoms and errors, node operators can take steps to improve the stability of peering. By using a reliable seed, adding a trusted peer, resetting the addrbook.json, and monitoring bandwidth usage, node operators can help ensure that their nodes are running smoothly and efficiently.

Recommendations

Based on the analysis, we recommend the following:

Split the issue into multiple tickets: Break down the issue into smaller, more manageable tickets to facilitate easier debugging and resolution.
Structure the issue in a more organized way: Use a more structured approach to present the issue, including clear descriptions of symptoms, errors, and potential solutions.
Provide additional logs and configurations: Share more logs and configurations to help developers and node operators better understand the issue and provide more effective solutions.

Introduction

In our previous article, we explored the issues faced by Celestia mainnet archive nodes, including peer connection errors, seed connection errors, and high bandwidth usage. We also discussed potential solutions to improve the stability of peering. In this Q&A article, we'll address some of the most frequently asked questions related to these issues.

Q: What are the most common causes of peer connection errors?

A: Peer connection errors can be caused by a variety of factors, including:

Network connectivity issues: Problems with your internet connection or the connection between your node and the peer can cause errors.
Peer node issues: The peer node may be experiencing technical difficulties, such as a crash or a network issue.
Configuration issues: Misconfigured settings or incorrect peer addresses can cause errors.

Q: How can I troubleshoot peer connection errors?

A: To troubleshoot peer connection errors, follow these steps:

Check your network connection: Ensure that your internet connection is stable and working correctly.
Verify peer addresses: Double-check that the peer addresses in your configuration file are correct and up-to-date.
Check peer node status: Verify that the peer node is online and functioning correctly.
Monitor logs: Review your node's logs to identify any error messages or clues that may indicate the cause of the issue.

Q: What are the most common causes of seed connection errors?

A: Seed connection errors can be caused by:

Authentication issues: Problems with the authentication process between your node and the seed can cause errors.
Seed node issues: The seed node may be experiencing technical difficulties, such as a crash or a network issue.
Configuration issues: Misconfigured settings or incorrect seed addresses can cause errors.

Q: How can I troubleshoot seed connection errors?

A: To troubleshoot seed connection errors, follow these steps:

Verify seed addresses: Double-check that the seed addresses in your configuration file are correct and up-to-date.
Check seed node status: Verify that the seed node is online and functioning correctly.
Monitor logs: Review your node's logs to identify any error messages or clues that may indicate the cause of the issue.
Check authentication settings: Ensure that your node's authentication settings are correct and configured properly.

Q: Why is my node consuming high bandwidth when not syncing blocks?

A: High bandwidth usage when not syncing blocks can be caused by:

Peer node activity: The peer node may be actively participating in the network, causing your node to consume more bandwidth.
Network activity: Your node may be receiving or sending data from other nodes or services, consuming bandwidth.
Configuration issues: Misconfigured settings or incorrect peer addresses can cause your node to consume more bandwidth than necessary.

Q: How can I reduce high bandwidth usage?

A: To reduce high bandwidth usage, follow these steps:

Verify peer addresses: Double-check that the peer addresses in your configuration file are correct and up-to-date.
Check peer node activity: Verify that the peer node is not actively participating in the network.
Monitor logs: Review your node's logs to identify any error messages or clues that may indicate the cause of the issue.
Adjust configuration settings: Adjust your node's configuration settings to reduce bandwidth consumption.

Conclusion

In this Q&A article, we've addressed some of the most frequently asked questions related to peer connection errors, seed connection errors, and high bandwidth usage. By understanding the causes and potential solutions to these issues, node operators can take steps to improve the stability of peering and ensure that their nodes are running efficiently and effectively.