This guide outlines common problems affecting non-validator nodes, with practical diagnostics and resolution steps. Rapid troubleshooting is critical for maintaining RPC availability and reliable access to stablecoin transaction data.Documentation Index
Fetch the complete documentation index at: https://docs.plasma.org/llms.txt
Use this file to discover all available pages before exploring further.
Most issues stem from system resource limits, network connectivity with consensus endpoints, or misconfiguration. Always begin with basic health checks before investigating Plasma-specific edge cases.
Synchronisation Issues
Node Lags Behind
Node Lags Behind
If your node falls behind the network head:
- Check system load: CPU, memory, and I/O must be sufficient for Plasma’s high-frequency block production.
- Verify consensus endpoint latency: High latency impacts block ingestion speed.
- Monitor database performance: Slow queries or lock contention can bottleneck sync.
block_height vs network head, state application time per block, and latency to each consensus endpoint.Sync Stalls Completely
Sync Stalls Completely
If syncing halts:
- Check disk space: Full disks halt database writes.
- Verify endpoint connectivity: Ensure DNS resolution, firewall rules, and routing are not blocking consensus traffic.
- Check container resource limits: Insufficient memory or CPU can crash sync processes.
Poor Peer Connectivity
Poor Peer Connectivity
Limited or unstable connections to consensus endpoints reduce data freshness for RPC consumers.Check: IP restrictions or routing policies, endpoint connection count and uptime, and failover behavior across multiple endpoints.Key metrics: Active connections, disconnect rate, and block arrival lag.
Network Connectivity
Port and Firewall Access
Ensure required ports are open for both:- Consensus communication
- RPC serving to applications
- Port reachability from outside and inside container
- Allowed outbound traffic for consensus sync
DNS Failures
Unresolved consensus domains break sync.- Confirm DNS resolution for all Plasma service domains
- Monitor resolver latency and reliability
- Add fallback resolvers as needed
Proxy and NAT Issues
Complex networking environments (VPNs, proxies, NATs) may interfere with:- Consensus sync
- RPC inbound access
- Proxy authentication rules
- NAT port forwarding for inbound RPC
- Routing paths to Plasma infrastructure
Configuration Errors
Invalid Non-Validator Config
Invalid Non-Validator Config
Startup errors are often caused by incorrect or missing consensus endpoints, malformed URLs or JWT tokens, or deprecated flags.Check logs for: Configuration parse errors, chain ID mismatches, and unknown flags or options.
Docker Issues
Docker Issues
Misconfigured containers can prevent normal operation.Validate: Volume mounts for database persistence, network access inside container, resource limits (CPU, memory), and image version compatibility.
Environment Variable Conflicts
Environment Variable Conflicts
Conflicting or misordered env vars may override intended settings.Verify: Consensus endpoint URLs and tokens, network and chain ID, and feature flags (e.g. custom gas support). Document environment variable usage to avoid drift between deployments.
Resource Limits
Disk Space
Monitor disk usage growth due to:- Blockchain data
- Logs and temporary files
- Implement log rotation
- Schedule cleanup jobs
- Monitor usage alerts
File Descriptor
Low ulimits can break:- Consensus connections
- RPC sessions
ulimit, systemd, or container runtime configs.
Diagnostics
Log Analysis
Use logs to trace:- Sync progress
- RPC errors
- Consensus connectivity
- Resource-related crashes
- Consensus synchronisation logs: Track connectivity and sync progress with Plasma.
- Transaction processing logs: Monitor transaction handling and errors.
- RPC request logs: Analyse payment application request patterns and performance.
- Resource utilisation logs: Identify bottlenecks affecting payment processing.
Performance Profiling
Profile non-validator workloads to identify bottlenecks in:- RPC performance
- DB query latency
- Consensus state application
- Balance check throughput
- Transfer confirmation latency
- Resource usage under load