Skip to main content

Deploy a Permissioned Subnet on Mainnet

danger

Deploying a Subnet to Mainnet has many risks. Doing so safely requires a laser focus on security. This tutorial does its best to point out common pitfalls, but there may be other risks not discussed here.

This tutorial is an educational resource and provides no guarantees that following it results in a secure deployment. Additionally, this tutorial takes some shortcuts that aid the understanding of the deployment process at the expense of security. The text highlights these shortcuts and they shouldn't be used for a production deployment.

After managing a successful Subnet deployment on the Tahoe Testnet, you're ready to deploy your Subnet on Mainnet. If you haven't done so, first Deploy a Subnet on Testnet.

This tutorial shows how to do the following on Mainnet.

  • Create a Subnet.
  • Deploy a virtual machine based on Subnet-EVM.
  • Add a node as a validator to the Subnet.
  • Join a node to the newly created Subnet.
note

All IDs in this article are for illustration purposes only. They are guaranteed to be different in your own run-through of this tutorial.

Prerequisites​

danger

Although only one validator is strictly required to run a Subnet, running with less than five validators is extremely dangerous and guarantees network downtime. Plan to support at least five validators in your production network.

Getting Your Mainnet NodeIDs​

You need to collect the NodeIDs for each of your validators. This tutorial uses these NodeIDs in several commands.

To get the NodeID of a Mainnet node, call the info.getNodeID endpoint. For example:

curl -X POST --data '{
    "jsonrpc":"2.0",
    "id"     :1,
    "method" :"info.getNodeID"
}' -H 'content-type:application/json;' 127.0.0.1:9650/ext/info

The response should look something like:

{
  "jsonrpc": "2.0",
  "result": {
    "nodeID": "NodeID-5mb46qkSBj81k9g9e4VFjGGSbaaSLFRzD"
  },
  "id": 1
}

In the sample response, NodeID-5mb46qkSBj81k9g9e4VFjGGSbaaSLFRzD is the NodeID. Note that the NodeID- prefix is part of the NodeID.

Setting up Your Ledger​

In the interest of security, all Metal-CLI Mainnet operations require the use of a connected Ledger device. You must unlock your Ledger and run the Metal App.

Metal-CLI supports the Ledger Nano X, Nano S, and Nano S Plus.

Ledger devices support TX signing for any address inside a sequence automatically generated by the device.

By default, Metal-CLI uses the first address of the derivation, and that address needs funds to issue the TXs to create the Subnet and add validators.

To get the first Mainnet address of your Ledger device, first make sure it is connected, unblocked, and running the Metal app. Then execute the key list command:

metal key list --ledger 0 --mainnet
+--------+---------+-------------------------+-----------------------------------------------+---------+---------+
|  KIND  |  NAME   |          CHAIN          |                    ADDRESS                    | BALANCE | NETWORK |
+--------+---------+-------------------------+-----------------------------------------------+---------+---------+
| ledger | index 0 | P-Chain (Bech32 format) | P-metal1ucykh6ls8thqpuwhg3vp8vvu6spg5e8tp8a25j|      11 | Mainnet |
+--------+---------+-------------------------+-----------------------------------------------+---------+---------+

The command prints the P-Chain address for Mainnet, P-metal1ucykh6ls8thqpuwhg3vp8vvu6spg5e8tp8a25j, and its balance. You should fund this address with at least 2.5 METAL to cover TX fees. the TX fee for creating your Subnet costs 2 METAL. Adding validators costs 0.001 METAL each. For more details, see Fees)

note

You can use the key list command to get any Ledger address in the derivation sequence by changing the index parameter from 0 to the one desired, or to a list of them (for example: 2, or 0,4,7). Also, you can ask for addresses on Tahoe with the --tahoe parameter, and local networks with the --local parameter.

Funding the Ledger​

A new Ledger device has no funds on the addresses it controls. You'll need to send funds to it by exporting them from the C-Chain to the P-Chain using Metal's Web Wallet.

You can load the Ledger's C-Chain address in the web wallet or load in a different private key. You can transfer funds from the C-Chain to the P-Chain by clicking on the Cross Chain on the left side of the web wallet.

Deploy the Subnet​

With your Ledger unlocked and running the Metal app, run

metal subnet deploy testsubnet

This is going to start a new prompt series.

Use the arrow keys to navigate: ↓ ↑ → ←
? Choose a network to deploy on:
    Local Network
    Tahoe
  ▸ Mainnet

This tutorial is about deploying to Mainnet, so navigate with the arrow keys to Mainnet and hit enter.

✔ Mainnet
Deploying [testsubnet] to Mainnet

After that, CLI shows the Mainnet Ledger address used to fund the deployment:

Ledger address: P-metal1ucykh6ls8thqpuwhg3vp8vvu6spg5e8tp8a25j

The deployment requires running a createSubnet transaction and a createBlockchain transaction, and so this first Ledger address must have the funds to issue both operations.

This tutorial creates a permissioned Subnet. As such, you must specify which P-Chain addresses can control the Subnet. These addresses are known as Control Keys. The CLI can automatically set your Ledger's address as the sole control key or the user may specify a custom list.

danger

In production Subnets, you should always use multiple control keys running in a multisig configuration. This tutorial uses a single control key for illustrative purposes only.

For instructions on controlling your Subnet with a multisig, see the Multisig Deployment Tutorial.

Configure which addresses may make changes to the subnet.
These addresses are known as your control keys. You are going to also
set how many control keys are required to make a subnet change (the threshold).
Use the arrow keys to navigate: ↓ ↑ → ←
? How would you like to set your control keys?:
  ▸ Use ledger address
    Custom list

For this tutorial, opt to use the first Ledger address, so enter at Use ledger address. Only this address is going to be able to add or remove validators, or create blockchains on the Subnet.

Your Subnet's control keys: [P-metal1ucykh6ls8thqpuwhg3vp8vvu6spg5e8tp8a25j]
Your subnet auth keys for chain creation: [P-metal1ucykh6ls8thqpuwhg3vp8vvu6spg5e8tp8a25j]

Next, the CLI generates a TX for creating the SubnetID and asks the user to sign it by using the Ledger.

*** Please sign subnet creation hash on the ledger device ***

This activates a Please review window on the Ledger. Navigate to the Ledger's APPROVE window by using the Ledger's right button, and then authorize the request by pressing both left and right buttons.

If the Ledger doesn't have enough funds, the user may see an error message:

*** Please sign subnet creation hash on the ledger device ***
Error: insufficient funds: provided UTXOs need 1000000000 more units of asset "U8iRqJoiJm8xZHAacmvYyZVwqQx6uDNtQeP3CQ6fcgQk3JqnK"

If successful, the CLI next asks you to sign a chain creation TX.

Subnet has been created with ID: 2UUCLbdGqawRDND7kHjwq3zXXMPdiycG2bkyuRzYMnuTSDr6db. Now creating blockchain...
*** Please sign blockchain creation hash on the ledger device ***

This activates a Please review window on the Ledger. Navigate to the Ledger's APPROVE window by using the Ledger's right button, and then authorize the request by pressing both left and right buttons.

After that, CLI creates the blockchain within the Subnet, and a summary window for the deployment appears:

+--------------------+------------------------------------------------------------------------------------+
| DEPLOYMENT RESULTS |                                                                                    |
+--------------------+------------------------------------------------------------------------------------+
| Chain Name         | testsubnet                                                                         |
+--------------------+------------------------------------------------------------------------------------+
| Subnet ID          | 2UUCLbdGqawRDND7kHjwq3zXXMPdiycG2bkyuRzYMnuTSDr6db                                 |
+--------------------+------------------------------------------------------------------------------------+
| VM ID              | rW1esjm6gy4BtGvxKMpHB2M28MJGFNsqHRY9AmnchdcgeB3ii                                  |
+--------------------+------------------------------------------------------------------------------------+
| Blockchain ID      | wNoEemzDEr54Zy3iNn66yjUxXmZS9LKsZYSUciL89274mHsjG                                  |
+--------------------+------------------------------------------------------------------------------------+
| RPC URL            | http://127.0.0.1:9650/ext/bc/wNoEemzDEr54Zy3iNn66yjUxXmZS9LKsZYSUciL89274mHsjG/rpc |
+--------------------+------------------------------------------------------------------------------------+
| P-Chain TXID       | wNoEemzDEr54Zy3iNn66yjUxXmZS9LKsZYSUciL89274mHsjG                                  |
+--------------------+------------------------------------------------------------------------------------+

Well done. You have just created your own Subnet running on Mainnet. Now it's time to add your validators.

Add a Validator​

info

Adding a validator on a Subnet requires that the node is already a validator on the primary network, which means that your node has fully bootstrapped.

See here on how to become a validator.

This new Subnet is cool, but it doesn't have any dedicated validators and can't process transactions yet.

To add a validator to your Subnet, you must add the desired node's NodeID to the Subnet's validator allow list by issuing an addValidator transaction and instruct the node to start syncing the Subnet by updating its local configs. You need to repeat this process for every validator you add to the network.

You can run the addValidator transactions from the same box that deployed the Subnet.

Submit addValidator TX to Mainnet​

Start by running the addValidator command and adding the name of the Subnet. To be clear, this does not start or run a validator, it only whitelists the node as a recognized validator on the Subnet.

metal subnet addValidator testsubnet

First choose Mainnet as the network to add the Subnet validator to.

Use the arrow keys to navigate: ↓ ↑ → ←
? Choose a network to add validator to.:
    Tahoe
  ▸ Mainnet

Because this operation issues a new transaction, the CLI needs the control keys to sign the operation. Because this tutorial only uses one control key in the Subnet, the process looks slightly different if you use multiple controls keys. The address needs to pay a TX fee of 0.001 METAL.

Your subnet auth keys for add validator tx creation: [P-metal1ucykh6ls8thqpuwhg3vp8vvu6spg5e8tp8a25j]

Set NodeID​

Now enter the NodeID of the validator.

Next, we need the NodeID of the validator you want to whitelist.

Check https://docs.metalblockchain.org/apis/metalgo/apis/info#infogetnodeid for instructions about how
to query the NodeID from your node
(Edit host IP address and port to match your deployment, if needed).
✔ What is the NodeID of the validator you'd like to whitelist?: NodeID-7Xhw2mDxuDS44j42TCB6U5579esbSt3Lg█

Note, this ID is intentionally modified to prevent replication.

Set Stake Weight​

Select 30 as the stake weight. You can learn more about the stake weight parameter in addSubnetValidator under the weight section.

danger

The stake weight of all your validators should sum to at least 100.

Use the arrow keys to navigate: ↓ ↑ → ←
? What stake weight would you like to assign to the validator?:
    Default (20)
  ▸ Custom
✔ What stake weight would you like to assign to the validator?: 30

Set Validation Start Time​

Next, specify when the validator is going to start validating. The time must be in the future. You can use the custom option to enter a specific date in YYYY-MM-DD HH:MM:SS format. Follow the default this time:

Use the arrow keys to navigate: ↓ ↑ → ←
? Start time:
  ▸ Start in one minute
    Custom
danger

If the join command isn't successfully completed before this time elapses and the validator's stake weight is >20% of the Subnet, the Subnet may have down time.

Production Subnets should ensure they have sufficient time to setup their validators before validation begins.

Set Validation Duration​

Finally, specify how long it's going to be validating:

✔ Start in one minute
Use the arrow keys to navigate: ↓ ↑ → ←
? How long should your validator validate for?:
  ▸ Until primary network validator expires
    Custom

If you choose Custom here, you need to enter a duration, which is a time span expressed in hours. For example, say 200 days = 24 \* 200 = 4800h

✔ How long should this validator be validating? Enter a duration, e.g. 8760h: 4800h

The CLI shows the user an actual date:

? Your validator is going to finish staking by 2023-06-10 13:07:58:
  ▸ Yes
    No

Confirm if correct.

Issue the TX​

At this point the prompt series is complete and the CLI attempts the transaction:

NodeID: NodeID-7Xhw2mDxuDS44j42TCB6U5579esbSt3Lg
Network: Mainnet
Start time: 2022-11-22 13:07:58
End time: 2023-06-10 13:07:58
Weight: 30
Inputs complete, issuing transaction to add the provided validator information...

The CLI shows the Ledger address:

Ledger address: P-metal1ucykh6ls8thqpuwhg3vp8vvu6spg5e8tp8a25j

At this point, if the Ledger address isn't the control key for the Subnet, the user receives an error:

Error: wallet doesn't contain subnet auth keys
exit status 1

If the Ledger doesn't have enough funds, the following error message appears:

*** Please sign subnet creation hash on the ledger device ***
Error: insufficient funds: provided UTXOs need 1000000 more units of asset "U8iRqJoiJm8xZHAacmvYyZVwqQx6uDNtQeP3CQ6fcgQk3JqnK"

Otherwise, both the CLI and the Ledger request to sign the TX:

*** Please sign add validator hash on the ledger device ***

This activates a Please review window on the Ledger. Navigate to the Ledger's APPROVE window by using the Ledger's right button, and then authorize the request by pressing both left and right buttons.

This might take a couple of seconds. After, it prints:

Transaction successful, transaction ID: r3tJ4Wr2CWA8AaticmFrKdKgAs5AhW2wwWTaQHRBZKwJhsXzb

This means the node is now a validator on the given Subnet on Mainnet! However, your work isn't complete. You must finish the Join a Subnet section otherwise your Subnet risks downtime.

You can get the P-Chain TX id information on Metal Explorer

Subnet Export​

Because you need to setup multiple validators on multiple different machines, you need to export your Subnet's configuration and import it on each validator.

metal subnet export testsubnet
✔ Enter file path to write export data to: /tmp/testsubnet-export.dat

The file is in text format and you shouldn't change it. You can use it to import the configuration on a different machine.

Subnet Import​

To import a VM configuration, move the file you exported in the previous section to your desired machine and issue the import command with the path to the file.

metal subnet import /tmp/testsubnet-export.dat
Subnet imported successfully

After this the whole Subnet configuration should be available on the target machine:

metal subnet list
+---------------+---------------+----------+-----------+----------+
|    SUBNET     |     CHAIN     | CHAIN ID |   TYPE    | DEPLOYED |
+---------------+---------------+----------+-----------+----------+
| testsubnet    | testsubnet    |     3333 | SubnetEVM | No       |
+---------------+---------------+----------+-----------+----------+

Join a Subnet​

To configure your validator to sync a specific Subnet, run the join command. This is a bit of a special command. The join command is going to either just print the required instructions for your already running node or is going to attempt to edit your config files automatically. If you are running the CLI on the same box as your validator, you should run the commands in automated mode. If you don't want to run Metal-CLI on the same box as your validator, use the manual commands.

metal subnet join testsubnet

Select Mainnet​

First ask for network for which the validator is joining. Choose Mainnet:

Use the arrow keys to navigate: ↓ ↑ → ←
? Choose a network to validate on (this command only supports public networks):
    Tahoe
  ▸ Mainnet

Check Validator List Status​

Because you deployed a permissioned Subnet, not just any primary network validator may validate the Subnet. The Subnet's control key holders must have first added the desired NodeID to the Subnet's validator list with the addValidator command described in the preceding section. Metal-CLI checks that your node is authorized to validate the target Subnet.

Would you like to check if your node is allowed to join this subnet?
If not, the subnet's control key holder must call metal subnet
addValidator with your NodeID.
Use the arrow keys to navigate: ↓ ↑ → ←
? Check whitelist?:
  ▸ Yes
    No

If you followed the steps in Add a Validator, everything should work as expected.

Setup Node Automatically​

There are now two choices possible: automatic and Manual configuration. As mentioned earlier, "Automatic" attempts to edit your config file and sets up your plugin directory, while "Manual" just prints the required config to the screen. To run in automatic mode, you must be running Metal-CLI on the same box as the validator node.

Select automatic.

Set Config File​

✔ Automatic
✔ Path to your existing config file (or where it's going to be generated): config.json

Provide a path to a config file. If this command runs on the box where your validator is running, then you could point this to the actually used config file, for example /etc/metalgo/config.json - just make sure the tool has write access to the file. Or you could just copy the file later. In any case, the tool is going to either try to edit the existing file specified by the given path, or create a new file. Again, set write permissions.

Set Plugin Directory​

Next, provide the plugin directory. Each VM runs its own binary, called a plugin. Therefore, you need to copy your VM's plugin binary into MetalGo's plugin directory. This directory depends on your MetalGo install location.

✔ Path to your metalgo plugin dir (likely metalgo/build/plugins): /home/user/go/src/github.com/!metal!blockchain/metalgo/build/plugins

The tool doesn't know where exactly it's located so it requires the full path. With the path given, it's going to copy the VM binary to the provided location:

✔ Path to your metalgo plugin dir (likely metalgo/build/plugins): /home/user/go/src/github.com/!metal!blockchain/metalgo/build/plugins█
VM binary written to /home/user/go/src/github.com/!metal!blockchain/metalgo/build/plugins/tGBrMADESojmu5Et9CpbGCrmVf9fiAJtZM5ZJ3YVDj5JTu2qw
This is going to edit your existing config file. This edit is nondestructive,
but it's always good to have a backup.
Use the arrow keys to navigate: ↓ ↑ → ←
? Proceed?:
  ▸ Yes
    No

Hitting Yes writes the necessary file:

✔ Yes
The config file has been edited. To use it, make sure to start the node with the '--config-file'
option, e.g.

./build/metalgo --config-file config.json

(using your binary location). The node has to be restarted for the changes to take effect.

Restart the Node​

It's required to restart the node.

Setup Node Manually​

By choosing "Manual" instead, the tool prints instructions. The user is going to have to follow these instructions and apply them to the node. Note that the IDs for the VM and Subnet are different for each Subnet deployment and you shouldn't copy them from this tutorial.

✔ Manual

To setup your node, you must do two things:

1. Add your VM binary to your node's plugin directory
2. Update your node config to start validating the subnet

To add the VM to your plugin directory, copy or scp from /tmp/tGBrMADESojmu5Et9CpbGCrmVf9fiAJtZM5ZJ3YVDj5JTu2qw

If you installed metalgo manually, your plugin directory is likely
metalgo/build/plugins.

If you start your node from the command line WITHOUT a config file (e.g. via command
line or systemd script), add the following flag to your node's startup command:

--whitelisted-subnets=2b175hLJhGdj3CzgXENso9CmwMgejaCQXhMFzBsm8hXbH2MF7H
(if the node already has a whitelisted-subnets config, append the new value by
comma-separating it).

For example:
./build/metalgo --network-id=Mainnet --whitelisted-subnets=2b175hLJhGdj3CzgXENso9CmwMgejaCQXhMFzBsm8hXbH2MF7H

If you start the node via a JSON config file, add this to your config file:
whitelisted-subnets: 2b175hLJhGdj3CzgXENso9CmwMgejaCQXhMFzBsm8hXbH2MF7H

TIP: Try this command with the --metalgo-config flag pointing to your config file,
this tool is going to try to update the file automatically (make sure it can write to it).

After you update your config, you are going to need to restart your node for the changes to
take effect.

Going Live​

Once all of your validators have joined the network, you are ready to issue transactions to your Subnet.

For the safety of your validators, you should setup dedicated API nodes to process transactions, but for test purposes, you can issue transactions directly to one of your validator's RPC interface.