Skip to main content

Troubleshooting Prysm

Select a configuration#

Troubleshooting checklist#

If you're running into unexpected outputs or errors, use the following checklist to check the status of your configuration. In many cases, this checklist will help you independently resolve your issue. If you still need help after completing this checklist, you can generate a troubleshooting report below and paste it into Discord when asking for support.

Thank you!

Prysmatic Labs is a small but mighty team of engineers who are working hard to support The Merge. Using this checklist is very helpful because it can reduce the time that it takes to gather information, allowing us to focus on Merge engineering and monitoring. Thank you for helping us!

If you end up generating a troubleshooting report, your report will include your selected configuration.

Many common issues are resolved by the steps in our Prepare for The Merge checklist. We recommend completing this Merge preparation checklist before proceeding here.

You can check your Geth execution node's sync status by running geth attach and then eth.syncing from a separate terminal window. A sync status of false indicates that your node is fully synced.

You should periodically see more than a few peers reported through Geth's log output. Look for output in the format of peercount=12. Refer to Geth's Connecting To The Network documentation for more detailed peer health monitoring guidance.

Use geth version to check Geth's version. See Geth's releases page and join their Discord to stay up to date as we approach Mainnet Merge.

You can check your beacon node's sync status by running curl http://localhost:3500/eth/v1/node/syncing | jq from a separate terminal window. When you see "is_syncing":false, your beacon node is fully synchronized with the beacon chain. When you see "is_optimistic":false, your beacon node sees that your execution node is either 1) not yet started, 2) hasn't synced past the merge block or 3) fully synchronized with the execution-layer blockchain.

You should periodically see more than a few peers reported through your beacon node's log output. Look for output in the format of peers=12. You can issue curl http://localhost:8080/healthz from a separate terminal window to check connectivity status. If you see currentConnectionError: no contract code at given address, your execution node may still be syncing. Otherwise, if you don't see any errors, your beacon node is connected to peers.

Ensure that you're using the latest stable Prysm release. Check Prysm's version by issuing the following command: prysm.sh beacon-chain --version (Linux) prysm.bat beacon-chain --version (Windows).

Issue curl http://localhost:3500/eth/v1alpha1/node/eth1/connections from a separate terminal window. If you see currentConnectionError: no contract code at given address, your execution node may still be syncing. Otherwise, if you don't see any errors, your beacon node is connected to your execution node. This output can be interpreted as "EN-BN connection is healthy": {"currentAddress":"http://localhost:8551","currentConnectionError":"","addresses":["http://localhost:8551"],"connectionErrors":[]}

Prysm will output an error if you attempt to provide an invalid Ethereum wallet address as a fee recipient address. You'll see warnings if a fee recipient address hasn't been provided. See Configure Fee Recipient for more information.

Paste your validator's public key (available in your deposit_data-*.json file) into a blockchain explorer like beaconcha.in to check the status of your validator.

Running into unexpected output, warnings, or errors? Although this is unintuitive, many errors and warnings are expected and have been identified in the below list of troubleshooting scenarios and solutions. We gratefully ask that you review this before asking for support.

Issue still not resolved? Generate a troubleshooting report below. Head over to Discord and paste your report for additional troubleshooting assistance.

Troubleshooting scenarios and solutions#

Common troubleshooting scenarios and solutions are detailed below.

Troubleshooting your validator node#

ScenarioSolution
You see Waiting for keymanager to initialize validator client with web UI...You'll usually see this message when your beacon node is trying to interact with a validator client instance before the beacon node is fully synced. This is a known limitation. When your beacon node is finished syncing, this message should go away. Visit Check Node and Validator Status to learn how to check the sync status of your beacon node.
Everything seems fine, but your validator balance is going down.If your validator client is running fine without errors but you're seeing your validator balance decrease, your beacon node may be experiencing issues with connectivity, stability, or synchronization. Check your beacon node logs to see if there are any errors or crashes.
Can't import accounts, stuck in a loop. You see Could not import accounts: could not write accounts: file already exists without proper 0600 permissionsThis usually happens when the account you're using doesn't have permission to read and write to the target directory. See this GitHub issue for a workaround.

Troubleshooting your beacon node#

ScenarioSolution
Waiting for peers / peer disconnected / no active peers: Waiting for enough suitable peers before syncing... msg="Peer disconnected" active=0Peers will continuously disconnect and reconnect, so don't worry about Peer disconnected messages. If your beacon node is struggling to find peers:
  • Your beacon node might be suffering from connectivity problems. Visit Improve P2P connectivity for connectivity troubleshooting guidance. Ensure that your firewall isn't restricting any outbound ports for Prysm.
  • You may be using an incorrect genesis state or network flag. Every test network requires its own genesis state and network flag. Visit our Quickstart for the latest test network parameters.
Beacon node is stuck during initial sync.If your node seems stuck (either doing nothing, or stuck in a loop) while syncing, a restart will usually resolve the problem. If you're on Windows, try selecting your console output window and hitting ENTER - this can "unpause" a paused console output stream.
Node is currently optimistic and cannot serve validators: level=error msg="Could not request attestation to sign at slot" error="rpc error: code = Unavailable desc = the node is currently optimistic and cannot serve validators" prefix=validator pubKey=0x01234 slot=65740This usually means that your execution client isn't yet synchronized. Visit Check Node and Validator Status to learn how to check the sync status of your execution client.
Could not read JWT secret, Could not access JWT secretYou, your terminal window, or the script you're using may not have the permissions required in order to read or write your JWT token. Try elevating privileges or running as Administrator (if you're on Windows).
could not get ancestor state: failed to unmarshal encoding: incorrect sizeThis usually indicates that your beacon node's data has become corrupt. Try restarting your beacon node with a new or cleared data directory. Consider using Checkpoint Sync to reduce sync time.
could not process block: could not process block header: parent root 0x... does not match the latest block header signing root in stateThis error is being actively investigated. See this open issue for more information.
Could not check configuration values between execution and consensus client" error="method not found" prefix=powchainGeth users see this error when they're using an old Geth binary. Make sure you're using the latest stable release of Geth.
could not check configuration values between execution and consensus client error: timeout from http.Client

403 signature invalid

Could not connect to execution client endpoint" error="could not make initial request to verify execution chain ID: 401 Unauthorized

level=error msg="Could not connect to execution client endpoint" error="could not make initial request to verify execution chain ID: Post "http://localhost:8551/": dial tcp 127.0.0.1:8551: connect: connection refused" prefix=powchain

warning msg="Batch is not processed" error="could not process block in batch: timeout from http.Client: received an undefined ee error.

warning msg="Batch is not processed" error="could not process block in batch: got an unexpected error in JSON-RPC response: 403 Forbidden: signature is invalid
These errors are usually caused by invalid JWT configuration. If you're using HTTP-JWT to connect your EN-BN, ensure that both EN and BN are configured to use the same JWT secret. Different files are OK (eg when your EN and BN are on different machines), but the secret within each JWT file should be the same. See Configure JWT authentication for more information.
weak-subjectivity-checkpoint not provided. Prysm recommends providing a weak subjectivity checkpoint for nodes synced from genesisYou can safely ignore this warning - it will be removed in an upcoming Prysm release. See How to configure Checkpoint Sync if you'd like to learn more about checkpoint sync.
beacon node doesn't have a parent in db with root...If you see blocks advancing, then this can usually be ignored. If you don't see blocks advancing, there are likely other warnings and/or errors that will help you troubleshoot.

Troubleshooting your execution node#

ScenarioSolution
chain not synced beyond EIP-155This usually means that your execution client needs more time to "catch up", which you don't need to worry about. If you see that your node is connected to peers and is advancing, your node is healthy.
Fatal: Failed to register the Ethereum service: genesis not found in chainThis can happen when your Geth tries to use an old data directory. Try restarting Geth with a new data directory specified.
Can't find peers on mainnetSome users have reported peer-to-peer connectivity issues that were caused by old binaries or old data directories. Try using a new data directory, make sure you're using the latest version of your execution client software, and review Configure ports and firewalls for improved network connectivity for port configuration guidance.
I'm running on mainnet, but I see a testnet (Goerli, Prater, etc) specified in my output logsYour execution client may be using an old binary, or an old database. Try using a new data directory, and make sure you're using the latest version of your execution client software.
the method engine_exchangeTransitionConfigurationV1 does not exist/is not availableUsers have resolved this Geth error by 1) updating to the latest version of Geth and 2) ensuring that both Prysm and Geth are configured to use JWT (if you're connecting your beacon node to Geth over HTTP). Configuring Geth to use a fresh data directory may also resolve this warning.
ERROR powchain: Unable to process past deposit contract logs, perhaps your execution client is not fully synced error=Receipt not available for 'To' block '14957457'.This usually means that your execution client needs more time to "catch up", which you don't need to worry about. If you see that your node is connected to peers and is advancing, your node is healthy.
403 signature invalidThis is usually caused by invalid JWT configuration. If you're using HTTP-JWT to connect your EN-BN, ensure that both EN and BN are configured to use the same JWT secret. Different files are OK (eg when your EN and BN are on different machines), but the secret within each JWT file should be the same. See Configure JWT authentication for more information.

Generate troubleshooting report#

  1. Complete the above troubleshooting checklist
  2. Fill in the below form
  3. Click Generate troubleshooting report
  4. Copy and paste the report into Discord when asking for support

Execution node startup command

Tip: We recommend redacting wallet addresses, IP addresses, and other personal information as a general operational security best practice.

Beacon node startup command

Validator node startup command

Unexpected output

Tip: Paste the ~100 lines of output before and including the output you're asking about.Generate troubleshooting report
Complete the checklist above before generating...