Troubleshooting Subnet Deployments
If you run into trouble deploying your Subnet, use this document for tips to resolve common issues.
Deployment Times Out
During a local deployment, your network may fail to start. Your error may look something like this:
[~]$ avalanche subnet deploy mySubnet
✔ Local Network
Deploying [mySubnet] to Local Network
Backend controller started, pid: 26388, output at: /Users/user/.metal-cli/runs/server_20221231_111605/metal-cli-backend
VMs ready.
Starting network...
..................................................................................
..................................................................................
......Error: failed to query network health: rpc error: code = DeadlineExceeded desc = context deadline exceeded
Metal-CLI only supports running one local Metal network at a time. If other instances of MetalGo are running concurrently, your Metal-CLI network fails to start.
To test for this error, start by shutting down any Metal nodes started by Metal-CLI.
avalanche network clean --hard
Next, look for any lingering MetalGo processes with:
ps aux | grep metalgo
If any processes are running, you need to stop them before you can launch your VM with Metal-CLI.
If you're running a validator node on the same box you're using Metal-CLI, don't end any of these lingering MetalGo processes. This may shut down your validator and could affect your validation uptime.
Incompatible RPC Version for Custom VM
If you're locally deploying a custom VM, you may run into this error message.
[~]$ avalanche subnet deploy mySubnet
✔ Local Network
Deploying [mySubnet] to Local Network
Backend controller started, pid: 26388, output at: /Users/user/.metal-cli/runs/server_20221231_111605/metal-cli-backend
VMs ready.
Starting network...
.........
Blockchain has been deployed. Wait until network acknowledges...
..................................................................................
..................................................................................
......Error: failed to query network health: rpc error: code = DeadlineExceeded desc = context deadline exceeded
This error has many possible causes, however you should be aware of the possibility of an RPC protocol version mismatch.
MetalGo communicates with custom VMs over RPC using gRPC. gRPC defines a protocol specification shared by both MetalGo and the VM. Both components must be running the same RPC version for VM deployment to work. You can view MetalGo's RPC compatibility broken down by release version.
Your custom VM's RPC version is set by the version of MetalGo that you import. By default, Metal-CLI creates local Metal networks that run the latest MetalGo release.
Here's an example with real numbers from the MetalGo compatibility page. If the latest MetalGo release is version v1.9.4, then Metal-CLI deploys a network with RPC version 20. For your deploy to be successful, your VM must also have RPC version 20. Because only MetalGo version v1.9.4 supports RPC version 20, your VM must import MetalGo version v1.9.4. If the latest release were v1.9.3, your VM could safely import versions v1.9.2 or v1.9.3 because both versions support RPC version 19.
If your VM has an RPC version mismatch, you have two options. First, you can update the version of
MetalGo you use in your VM. This is the correct long-term approach. However, there is a
short-term workaround. Metal-CLI supports deploying older versions of MetalGo by using the
--metalgo-version flag. Both the subnet deploy and
network start commands support setting the MetalGo
version explicitly. Although it's very important to keep your version of MetalGo up-to-date,
this workaround helps you avoid broken builds in the short term. However, you need to upgrade
to the latest MetalGo version when deploying publicly to the Tahoe Testnet or Mainnet.
Updates to MetalGo's RPC version are not tied to its semantic version scheme. Minor MetalGo version bumps may include a breaking RPC version bump.