Troubleshooting Prysm

Select a configuration

Operating system:

Linux, MacOS, Arm64

Windows

Network:

Mainnet

Sepolia

Holesky

Execution client:

Geth

Nethermind

Besu

EN-BN connection:

HTTP-JWT

IPC

Troubleshooting checklist

If you're running into unexpected outputs or errors, use the following checklist to check the status of your configuration. In many cases, this checklist will help you independently resolve your issue. If you still need help after completing this checklist, you can generate a troubleshooting report below and paste it into Discord when asking for support.

Thank you!

Prysmatic Labs is a small but mighty team of engineers who are working hard to support The Merge. Using this checklist is very helpful because it can reduce the time that it takes to gather information, allowing us to focus on Merge engineering and monitoring. Thank you for helping us!

1. Select a configuration above

If you end up generating a troubleshooting report, your report will include your selected configuration.

2. Check Quickstart Configurations

Many common issues are resolved by the steps in our Quickstart guide. We recommend reviewing the quickstart guide before continuing below.

3. Execution node sync status

Execution client:

You can check your Nethermind execution node's sync status by navigating to http://localhost:8545/healthchecks-ui or by running curl localhost:8545/health from a separate terminal window. A sync status of false indicates that your node is fully synced.

Nethermind

Operating system:

Linux, MacOS, Arm64

You can check your Besu execution node's sync status by running curl -H 'Content-Type: application/json' -X POST http://localhost:8545 -d '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":51}' from a separate terminal window. A sync status of false indicates that your node is fully synced.

Windows

You can check your Besu execution node's sync status by running curl -H "Content-Type: application/json" -X POST http://localhost:8545 -d "{""jsonrpc"":""2.0"",""method"":""eth_syncing"",""params"":[],""id"":51}" from a separate terminal window. A sync status of false indicates that your node is fully synced.

Besu

Operating system:

Linux, MacOS, Arm64

You can check your Geth execution node's sync status by running geth attach (IPC) or geth attach http://localhost:8545 (HTTP) from a separate terminal. Then type eth.syncing. A sync status of false indicates that your node is fully synced.

Windows

You can check your Geth execution node's sync status by running geth attach ipc:\\.\pipe\geth.ipc (IPC) or geth attach http://localhost:8545 (HTTP) from a separate terminal. Then type eth.syncing. A sync status of false indicates that your node is fully synced.

4. Execution node peer connectivity

Execution client:

You can check your Nethermind execution node's peer connectivity by navigating to http://localhost:8545/healthchecks-ui or by running curl localhost:8545/health a separate terminal window. A health status of Healthy indicates that your node is connected to peers.

Nethermind

You should periodically see more than a few peers reported through Besu's log output. Refer to Besu's Monitor peer connections documentation for more detailed peer health monitoring guidance.

Besu

You should periodically see more than a few peers reported through Geth's log output. Look for output in the format of peercount=12. Refer to Geth's Connecting To The Network documentation for more detailed peer health monitoring guidance.

5. Execution node version

Execution client:

Use geth version to check Geth's version. See Geth's releases page and join their Discord to stay up to date as we approach Mainnet Merge.

Geth

Review Nethermind's log output to see what version you're using. See Nethermind's releases page and join their Discord to stay up to date as we approach Mainnet Merge.

Nethermind

Review Besu's log output to see what version you're using. See Besu's releases page and join their Discord to stay up to date as we approach Mainnet Merge.

6. Beacon node sync status

You can check your beacon node's sync status by running curl http://localhost:3500/eth/v1/node/syncing | jq from a separate terminal window. When you see "is_syncing":false, your beacon node is fully synchronized with the beacon chain. When you see "is_optimistic":false, your beacon node sees that your execution node is either 1) not yet started, 2) hasn't synced past the merge block or 3) fully synchronized with the execution-layer blockchain.

7. Beacon node peer connectivity

You should periodically see more than a few peers reported through your beacon node's log output. Look for output in the format of peers=12. You can issue curl http://localhost:8080/healthz from a separate terminal window to check connectivity status. If you see currentConnectionError: no contract code at given address, your execution node may still be syncing. Otherwise, if you don't see any errors, your beacon node is connected to peers.

8. Beacon node version

Ensure that you're using the latest stable Prysm release. Check Prysm's version by issuing the following command: prysm.sh beacon-chain --version (Linux) prysm.bat beacon-chain --version (Windows).

9. Beacon node ↔ execution node connectivity

Issue curl http://localhost:3500/eth/v1alpha1/node/eth1/connections from a separate terminal window. If you see currentConnectionError: no contract code at given address, your execution node may still be syncing. Otherwise, if you don't see any errors, your beacon node is connected to your execution node. This output can be interpreted as "EN-BN connection is healthy": {"currentAddress":"http://localhost:8551","currentConnectionError":"","addresses":["http://localhost:8551"],"connectionErrors":[]}

10. Fee recipient configuration

Prysm will output an error if you attempt to provide an invalid Ethereum wallet address as a fee recipient address. You'll see warnings if a fee recipient address hasn't been provided. See Configure Fee Recipient for more information.

11. Validator status

Network:

Paste your validator's public key (available in your deposit_data-*.json file) into a blockchain explorer like beaconcha.in to check the status of your validator.

Mainnet

Paste your validator's public key (available in your deposit_data-*.json file) into a blockchain explorer like beaconcha.in to check the status of your validator.

Holesky

Running a validator on Sepolia is currently unsupported as Sepolia is a permissioned network, so there's nothing to do here.

12. Troubleshooting scenarios and solutions

Running into unexpected output, warnings, or errors? Although this is unintuitive, many errors and warnings are expected and have been identified in the below list of troubleshooting scenarios and solutions. We gratefully ask that you review this before asking for support.

13. Troubleshooting report

Issue still not resolved? Generate a troubleshooting report below. Head over to Discord and paste your report for additional troubleshooting assistance.

Troubleshooting scenarios and solutions

Common troubleshooting scenarios and solutions are detailed below.

Troubleshooting your validator node

Scenario	Solution
You see Waiting for keymanager to initialize validator client with web UI...	You'll usually see this message when your beacon node is trying to interact with a validator client instance before the beacon node is fully synced. This is a known limitation. When your beacon node is finished syncing, this message should go away. Visit Check Node and Validator Status to learn how to check the sync status of your beacon node.
Everything seems fine, but your validator balance is going down.	If your validator client is running fine without errors but you're seeing your validator balance decrease, your beacon node may be experiencing issues with connectivity, stability, or synchronization. Check your beacon node logs to see if there are any errors or crashes.
Can't import accounts, stuck in a loop. You see Could not import accounts: could not write accounts: file already exists without proper 0600 permissions	This usually happens when the account you're using doesn't have permission to read and write to the target directory. See this GitHub issue for a workaround.
level=warning msg="Failed to update proposer settings" error="could not submit signed registrations to beacon node: rpc error: code = InvalidArgument desc = Could not register block builder: could not register validator(s): unsupported error code: 502: did not receive 200 response from API: Builder API validator registration unsuccessful" prefix=validator slot=5069888	Possibly an issue with mev-boost/relay if you constantly see this.
Could not determine if beacon chain started: could not setup beacon chain ChainStart streaming client: rpc error: code = Unavailable desc = connection closed: could not connect" prefix=validator	Your validator is not connected to your beacon node properly.
Could not request attestation to sign at slot error=rpc error: code = Unavailable desc = the node is currently optimistic and cannot serve validators	This usually means that your execution client isn't yet synchronized. Visit Check Node and Validator Status.

Troubleshooting your beacon node

Scenario	Solution
Waiting for peers / peer disconnected / no active peers: Waiting for enough suitable peers before syncing... msg="Peer disconnected" active=0	Peers will continuously disconnect and reconnect, so don't worry about Peer disconnected messages. If your beacon node is struggling to find peers: Your beacon node might be suffering from connectivity problems. Visit Improve P2P connectivity for connectivity troubleshooting guidance. Ensure that your firewall isn't restricting any outbound ports for Prysm. You may be using an incorrect genesis state or network flag. Every test network requires its own genesis state and network flag. Visit our Quickstart for the latest test network parameters. Warning: If you are using Windows and you are losing peers it could be an issue with Time Sync program (net time), one user's solution was using the google time servers and changing it to 10 min sync interval. issue 13431 and a similar issue here issue 8144
Beacon node is stuck during initial sync.	If your node seems stuck (either doing nothing, or stuck in a loop) while syncing, a restart will usually resolve the problem. If you're on Windows, try selecting your console output window and hitting ENTER - this can "unpause" a paused console output stream.
Node is currently optimistic and cannot serve validators: level=error msg="Could not request attestation to sign at slot" error="rpc error: code = Unavailable desc = the node is currently optimistic and cannot serve validators" prefix=validator pubKey=0x01234 slot=65740	This usually means that your execution client isn't yet synchronized. Visit Check Node and Validator Status to learn how to check the sync status of your execution client.
Could not read JWT secret, Could not access JWT secret	You, your terminal window, or the script you're using may not have the permissions required in order to read or write your JWT token. Try elevating privileges or running as Administrator (if you're on Windows).
could not get ancestor state: failed to unmarshal encoding: incorrect size	This usually indicates that your beacon node's data has become corrupt. Try restarting your beacon node with a new or cleared data directory. Consider using Checkpoint Sync to reduce sync time.
could not process block: could not process block header: parent root 0x... does not match the latest block header signing root in state	This error is being actively investigated. See this open issue for more information.
Could not check configuration values between execution and consensus client" error="method not found" prefix=powchain	Geth users see this error when they're using an old Geth binary. Make sure you're using the latest stable release of Geth.
could not check configuration values between execution and consensus client error: timeout from http.Client 403 signature invalid Could not connect to execution client endpoint" error="could not make initial request to verify execution chain ID: 401 Unauthorized level=error msg="Could not connect to execution client endpoint" error="could not make initial request to verify execution chain ID: Post "http://localhost:8551/": dial tcp 127.0.0.1:8551: connect: connection refused" prefix=powchain warning msg="Batch is not processed" error="could not process block in batch: timeout from http.Client: received an undefined ee error. warning msg="Batch is not processed" error="could not process block in batch: got an unexpected error in JSON-RPC response: 403 Forbidden: signature is invalid	These errors are usually caused by invalid JWT configuration. If you're using HTTP-JWT to connect your EN-BN, ensure that both EN and BN are configured to use the same JWT secret. Different files are OK (eg when your EN and BN are on different machines), but the secret within each JWT file should be the same. See Configure JWT authentication for more information.
weak-subjectivity-checkpoint not provided. Prysm recommends providing a weak subjectivity checkpoint for nodes synced from genesis	You can safely ignore this warning - it will be removed in an upcoming Prysm release. See How to configure Checkpoint Sync if you'd like to learn more about checkpoint sync.
beacon node doesn't have a parent in db with root...	If you see blocks advancing, then this can usually be ignored. If you don't see blocks advancing, there are likely other warnings and/or errors that will help you troubleshoot.
ERROR sync: Could not handle p2p pubsub error=could not process block: could not validate new payload: timeout from http.Client: received an undefined ee error topic=/eth2/4a26c58b/beacon_block/ssz_snappy	Your execution client is taking too long to process a payload, if you continuously see this error it might be an indication that your execution client is stressed.
If you see: ERROR main: could not get ancestor state: failed to retrieve state from db: failed to unmarshal encoding: incorrect size	Your db might be corrupted
ERROR p2p: Failed to find peers error=unable to find requisite number of peers for topic /eth2/4a26c58b/beacon_attestation_15/ssz_snappy - only 0 out of 1 peers were able to be found	Node appears to be having peering issues. See Configure ports and firewalls for improved peer-to-peer connectivity for more information.
ERROR p2p: Failed to broadcast attestation error=could not publish message: unable to find requisite number of peers for topic /eth2/4a26c58b/beacon_attestation_15/ssz_snappy, 0 peers found to publish to: context deadline exceeded	Node appears to be having peering issues. See Configure ports and firewalls for improved peer-to-peer connectivity for more information.
level=warning msg="Block is not processed" error="could not process block: could not validate new payload: timeout from http.Client: received an undefined ee error" prefix=initial-sync	Your execution client is taking too long to process a payload, if you continuously see this error it might be an indication that your execution client is stressed.
ERROR powchain: could not get eth1 block hash: could not query header with height 11184524: not found	Your execution client might still be syncing. Syncing can be a fairly long process but if it’s already been over a week, we recommend that you investigate further and feel free to ask us questions on our Discord server.

Troubleshooting your execution node

Scenario	Solution
chain not synced beyond EIP-155	This usually means that your execution client needs more time to "catch up", which you don't need to worry about. If you see that your node is connected to peers and is advancing, your node is healthy.
Fatal: Failed to register the Ethereum service: genesis not found in chain	This can happen when your Geth tries to use an old data directory. Try restarting Geth with a new data directory specified.
Can't find peers on mainnet	Some users have reported peer-to-peer connectivity issues that were caused by old binaries or old data directories. Try using a new data directory, make sure you're using the latest version of your execution client software, and review Configure ports and firewalls for improved network connectivity for port configuration guidance.
I'm running on mainnet, but I see a testnet ( Holesky, etc) specified in my output logs	Your execution client may be using an old binary, or an old database. Try using a new data directory, and make sure you're using the latest version of your execution client software.
the method engine_exchangeTransitionConfigurationV1 does not exist/is not available	Users have resolved this Geth error by 1) updating to the latest version of Geth and 2) ensuring that both Prysm and Geth are configured to use JWT (if you're connecting your beacon node to Geth over HTTP). Configuring Geth to use a fresh data directory may also resolve this warning.
ERROR powchain: Unable to process past deposit contract logs, perhaps your execution client is not fully synced error=Receipt not available for 'To' block '14957457'.	This usually means that your execution client needs more time to "catch up", which you don't need to worry about. If you see that your node is connected to peers and is advancing, your node is healthy.
403 signature invalid	This is usually caused by invalid JWT configuration. If you're using HTTP-JWT to connect your EN-BN, ensure that both EN and BN are configured to use the same JWT secret. Different files are OK (eg when your EN and BN are on different machines), but the secret within each JWT file should be the same. See Configure JWT authentication for more information.
Beacon client online, but never received consensus updates	Your beacon node probably needs more time to sync. See [github.com/ethereum/go-ethereum/issues/25753](https://github.com/ethereum/go-ethereum/issues/25753) for more information.
If you see: “State heal in progress”	Geth goes into state heal during sync it just takes a long time for it to go through should start syncing on beacon node after the state heal.
Forkchoice requested sync to new head	Your execution client is syncing now.

Generate troubleshooting report

1. Complete the above troubleshooting checklist
2. Fill in the below form
3. Click Generate troubleshooting report
4. Copy and paste the generated report text into Discord when asking for support

Execution node startup command

Tip: We recommend redacting wallet addresses, IP addresses, and other personal information as a general operational security best practice.

Beacon node startup command

Validator node startup command

Unexpected output

Tip: Paste the ~100 lines of output before and including the output you're asking about.Generate troubleshooting report

Complete the checklist above before generating...