Skip to main content

How to Build a Simple Golang VM

This is part of a series of tutorials for building a Virtual Machine (VM):

Introduction

In this tutorial, we’ll create a very simple VM called the TimestampVM. Each block in the TimestampVM's blockchain contains a strictly increasing timestamp when the block was created and a 32-byte payload of data.

Such a server is useful because it can be used to prove a piece of data existed at the time the block was created. Suppose you have a book manuscript, and you want to be able to prove in the future that the manuscript exists today. You can add a block to the blockchain where the block’s payload is a hash of your manuscript. In the future, you can prove that the manuscript existed today by showing that the block has the hash of your manuscript in its payload (this follows from the fact that finding the pre-image of a hash is impossible).

Prerequisites

Make sure you're familiar with the previous tutorial in this series, which dives into what virtual machines are.

TimestampVM Implementation

Now we know the interface our VM must implement and the libraries we can use to build a VM.

Let’s write our VM, which implements block.ChainVM and whose blocks implement snowman.Block. You can also follow the code in the TimestampVM repository.

Codec

Codec is required to encode/decode the block into byte representation. TimestampVM uses the default codec and manager.

/timestampvm/codec.go
const (
// CodecVersion is the current default codec version
CodecVersion = 0
)

// Codecs do serialization and deserialization
var (
Codec codec.Manager
)

func init() {
// Create default codec and manager
c := linearcodec.NewDefault()
Codec = codec.NewDefaultManager()

// Register codec to manager with CodecVersion
if err := Codec.RegisterCodec(CodecVersion, c); err != nil {
panic(err)
}
}

State

The State interface defines the database layer and connections. Each VM should define their own database methods. State embeds the BlockState which defines block-related state operations.

/timestampvm/state.go
var (
// These are prefixes for db keys.
// It's important to set different prefixes for each separate database objects.
singletonStatePrefix = []byte("singleton")
blockStatePrefix = []byte("block")

_ State = &state{}
)

// State is a wrapper around avax.SingleTonState and BlockState
// State also exposes a few methods needed for managing database commits and close.
type State interface {
// SingletonState is defined in metalgo,
// it is used to understand if db is initialized already.
avax.SingletonState
BlockState

Commit() error
Close() error
}

type state struct {
avax.SingletonState
BlockState

baseDB *versiondb.Database
}

func NewState(db database.Database, vm *VM) State {
// create a new baseDB
baseDB := versiondb.New(db)

// create a prefixed "blockDB" from baseDB
blockDB := prefixdb.New(blockStatePrefix, baseDB)
// create a prefixed "singletonDB" from baseDB
singletonDB := prefixdb.New(singletonStatePrefix, baseDB)

// return state with created sub state components
return &state{
BlockState: NewBlockState(blockDB, vm),
SingletonState: avax.NewSingletonState(singletonDB),
baseDB: baseDB,
}
}

// Commit commits pending operations to baseDB
func (s *state) Commit() error {
return s.baseDB.Commit()
}

// Close closes the underlying base database
func (s *state) Close() error {
return s.baseDB.Close()
}

Block State

This interface and implementation provides storage functions to VM to store and retrieve blocks.

/timestampvm/block_state.go
const (
lastAcceptedByte byte = iota
)

const (
// maximum block capacity of the cache
blockCacheSize = 8192
)

// persists lastAccepted block IDs with this key
var lastAcceptedKey = []byte{lastAcceptedByte}

var _ BlockState = &blockState{}

// BlockState defines methods to manage state with Blocks and LastAcceptedIDs.
type BlockState interface {
GetBlock(blkID ids.ID) (*Block, error)
PutBlock(blk *Block) error

GetLastAccepted() (ids.ID, error)
SetLastAccepted(ids.ID) error
}

// blockState implements BlocksState interface with database and cache.
type blockState struct {
// cache to store blocks
blkCache cache.Cacher
// block database
blockDB database.Database
lastAccepted ids.ID

// vm reference
vm *VM
}

// blkWrapper wraps the actual blk bytes and status to persist them together
type blkWrapper struct {
Blk []byte `serialize:"true"`
Status choices.Status `serialize:"true"`
}

// NewBlockState returns BlockState with a new cache and given db
func NewBlockState(db database.Database, vm *VM) BlockState {
return &blockState{
blkCache: &cache.LRU{Size: blockCacheSize},
blockDB: db,
vm: vm,
}
}

// GetBlock gets Block from either cache or database
func (s *blockState) GetBlock(blkID ids.ID) (*Block, error) {
// Check if cache has this blkID
if blkIntf, cached := s.blkCache.Get(blkID); cached {
// there is a key but value is nil, so return an error
if blkIntf == nil {
return nil, database.ErrNotFound
}
// We found it return the block in cache
return blkIntf.(*Block), nil
}

// get block bytes from db with the blkID key
wrappedBytes, err := s.blockDB.Get(blkID[:])
if err != nil {
// we could not find it in the db, let's cache this blkID with nil value
// so next time we try to fetch the same key we can return error
// without hitting the database
if err == database.ErrNotFound {
s.blkCache.Put(blkID, nil)
}
// could not find the block, return error
return nil, err
}

// first decode/unmarshal the block wrapper so we can have status and block bytes
blkw := blkWrapper{}
if _, err := Codec.Unmarshal(wrappedBytes, &blkw); err != nil {
return nil, err
}

// now decode/unmarshal the actual block bytes to block
blk := &Block{}
if _, err := Codec.Unmarshal(blkw.Blk, blk); err != nil {
return nil, err
}

// initialize block with block bytes, status and vm
blk.Initialize(blkw.Blk, blkw.Status, s.vm)

// put block into cache
s.blkCache.Put(blkID, blk)

return blk, nil
}

// PutBlock puts block into both database and cache
func (s *blockState) PutBlock(blk *Block) error {
// create block wrapper with block bytes and status
blkw := blkWrapper{
Blk: blk.Bytes(),
Status: blk.Status(),
}

// encode block wrapper to its byte representation
wrappedBytes, err := Codec.Marshal(CodecVersion, &blkw)
if err != nil {
return err
}

blkID := blk.ID()
// put actual block to cache, so we can directly fetch it from cache
s.blkCache.Put(blkID, blk)

// put wrapped block bytes into database
return s.blockDB.Put(blkID[:], wrappedBytes)
}

// DeleteBlock deletes block from both cache and database
func (s *blockState) DeleteBlock(blkID ids.ID) error {
s.blkCache.Put(blkID, nil)
return s.blockDB.Delete(blkID[:])
}

// GetLastAccepted returns last accepted block ID
func (s *blockState) GetLastAccepted() (ids.ID, error) {
// check if we already have lastAccepted ID in state memory
if s.lastAccepted != ids.Empty {
return s.lastAccepted, nil
}

// get lastAccepted bytes from database with the fixed lastAcceptedKey
lastAcceptedBytes, err := s.blockDB.Get(lastAcceptedKey)
if err != nil {
return ids.ID{}, err
}
// parse bytes to ID
lastAccepted, err := ids.ToID(lastAcceptedBytes)
if err != nil {
return ids.ID{}, err
}
// put lastAccepted ID into memory
s.lastAccepted = lastAccepted
return lastAccepted, nil
}

// SetLastAccepted persists lastAccepted ID into both cache and database
func (s *blockState) SetLastAccepted(lastAccepted ids.ID) error {
// if the ID in memory and the given memory are same don't do anything
if s.lastAccepted == lastAccepted {
return nil
}
// put lastAccepted ID to memory
s.lastAccepted = lastAccepted
// persist lastAccepted ID to database with fixed lastAcceptedKey
return s.blockDB.Put(lastAcceptedKey, lastAccepted[:])
}

Block

Let’s look at our block implementation.

The type declaration is:

/timestampvm/block.go
// Block is a block on the chain.
// Each block contains:
// 1) ParentID
// 2) Height
// 3) Timestamp
// 4) A piece of data (a string)
type Block struct {
PrntID ids.ID `serialize:"true" json:"parentID"` // parent's ID
Hght uint64 `serialize:"true" json:"height"` // This block's height. The genesis block is at height 0.
Tmstmp int64 `serialize:"true" json:"timestamp"` // Time this block was proposed at
Dt [dataLen]byte `serialize:"true" json:"data"` // Arbitrary data

id ids.ID // hold this block's ID
bytes []byte // this block's encoded bytes
status choices.Status // block's status
vm *VM // the underlying VM reference, mostly used for state
}

The serialize:"true" tag indicates that the field should be included in the byte representation of the block used when persisting the block or sending it to other nodes.

Verify

This method verifies that a block is valid and stores it in the memory. It is important to store the verified block in the memory and return them in the vm.GetBlock method.

/timestampvm/block.go
// Verify returns nil iff this block is valid.
// To be valid, it must be that:
// b.parent.Timestamp < b.Timestamp <= [local time] + 1 hour
func (b *Block) Verify() error {
// Get [b]'s parent
parentID := b.Parent()
parent, err := b.vm.getBlock(parentID)
if err != nil {
return errDatabaseGet
}

// Ensure [b]'s height comes right after its parent's height
if expectedHeight := parent.Height() + 1; expectedHeight != b.Hght {
return fmt.Errorf(
"expected block to have height %d, but found %d",
expectedHeight,
b.Hght,
)
}

// Ensure [b]'s timestamp is after its parent's timestamp.
if b.Timestamp().Unix() < parent.Timestamp().Unix() {
return errTimestampTooEarly
}

// Ensure [b]'s timestamp is not more than an hour
// ahead of this node's time
if b.Timestamp().Unix() >= time.Now().Add(time.Hour).Unix() {
return errTimestampTooLate
}

// Put that block to verified blocks in memory
b.vm.verifiedBlocks[b.ID()] = b

return nil
}

Accept

Accept is called by the consensus to indicate this block is accepted.

/timestampvm/block.go
// Accept sets this block's status to Accepted and sets lastAccepted to this
// block's ID and saves this info to b.vm.DB
func (b *Block) Accept() error {
b.SetStatus(choices.Accepted) // Change state of this block
blkID := b.ID()

// Persist data
if err := b.vm.state.PutBlock(b); err != nil {
return err
}

// Set last accepted ID to this block ID
if err := b.vm.state.SetLastAccepted(blkID); err != nil {
return err
}

// Delete this block from verified blocks as it's accepted
delete(b.vm.verifiedBlocks, b.ID())

// Commit changes to database
return b.vm.state.Commit()
}

Reject

Reject is called by the consensus to indicate this block is rejected.

/timestampvm/block.go
// Reject sets this block's status to Rejected and saves the status in state
// Recall that b.vm.DB.Commit() must be called to persist to the DB
func (b *Block) Reject() error {
b.SetStatus(choices.Rejected) // Change state of this block
if err := b.vm.state.PutBlock(b); err != nil {
return err
}
// Delete this block from verified blocks as it's rejected
delete(b.vm.verifiedBlocks, b.ID())
// Commit changes to database
return b.vm.state.Commit()
}

Block Field Methods

These methods are required by the snowman.Block interface.

/timestampvm/block.go
// ID returns the ID of this block
func (b *Block) ID() ids.ID { return b.id }

// ParentID returns [b]'s parent's ID
func (b *Block) Parent() ids.ID { return b.PrntID }

// Height returns this block's height. The genesis block has height 0.
func (b *Block) Height() uint64 { return b.Hght }

// Timestamp returns this block's time. The genesis block has time 0.
func (b *Block) Timestamp() time.Time { return time.Unix(b.Tmstmp, 0) }

// Status returns the status of this block
func (b *Block) Status() choices.Status { return b.status }

// Bytes returns the byte repr. of this block
func (b *Block) Bytes() []byte { return b.bytes }

Helper Functions

These methods are convenience methods for blocks, they're not a part of the block interface.

// Initialize sets [b.bytes] to [bytes], [b.id] to hash([b.bytes]),
// [b.status] to [status] and [b.vm] to [vm]
func (b *Block) Initialize(bytes []byte, status choices.Status, vm *VM) {
b.bytes = bytes
b.id = hashing.ComputeHash256Array(b.bytes)
b.status = status
b.vm = vm
}

// SetStatus sets the status of this block
func (b *Block) SetStatus(status choices.Status) { b.status = status }

Virtual Machine

Now, let’s look at our timestamp VM implementation, which implements the block.ChainVM interface.

The declaration is:

/timestampvm/vm.go
// This Virtual Machine defines a blockchain that acts as a timestamp server
// Each block contains data (a payload) and the timestamp when it was created

const (
dataLen = 32
Name = "timestampvm"
)

// VM implements the snowman.VM interface
// Each block in this chain contains a Unix timestamp
// and a piece of data (a string)
type VM struct {
// The context of this vm
ctx *snow.Context
dbManager manager.Manager

// State of this VM
state State

// ID of the preferred block
preferred ids.ID

// channel to send messages to the consensus engine
toEngine chan<- common.Message

// Proposed pieces of data that haven't been put into a block and proposed yet
mempool [][dataLen]byte

// Block ID --> Block
// Each element is a block that passed verification but
// hasn't yet been accepted/rejected
verifiedBlocks map[ids.ID]*Block
}

Initialize

This method is called when a new instance of VM is initialized. Genesis block is created under this method.

/timestampvm/vm.go
// Initialize this vm
// [ctx] is this vm's context
// [dbManager] is the manager of this vm's database
// [toEngine] is used to notify the consensus engine that new blocks are
// ready to be added to consensus
// The data in the genesis block is [genesisData]
func (vm *VM) Initialize(
ctx *snow.Context,
dbManager manager.Manager,
genesisData []byte,
upgradeData []byte,
configData []byte,
toEngine chan<- common.Message,
_ []*common.Fx,
_ common.AppSender,
) error {
version, err := vm.Version()
if err != nil {
log.Error("error initializing Timestamp VM: %v", err)
return err
}
log.Info("Initializing Timestamp VM", "Version", version)

vm.dbManager = dbManager
vm.ctx = ctx
vm.toEngine = toEngine
vm.verifiedBlocks = make(map[ids.ID]*Block)

// Create new state
vm.state = NewState(vm.dbManager.Current().Database, vm)

// Initialize genesis
if err := vm.initGenesis(genesisData); err != nil {
return err
}

// Get last accepted
lastAccepted, err := vm.state.GetLastAccepted()
if err != nil {
return err
}

ctx.Log.Info("initializing last accepted block as %s", lastAccepted)

// Build off the most recently accepted block
return vm.SetPreference(lastAccepted)
}
initGenesis

initGenesis is a helper method which initializes the genesis block from given bytes and puts into the state.

/timestampvm/vm.go
// Initializes Genesis if required
func (vm *VM) initGenesis(genesisData []byte) error {
stateInitialized, err := vm.state.IsInitialized()
if err != nil {
return err
}

// if state is already initialized, skip init genesis.
if stateInitialized {
return nil
}

if len(genesisData) > dataLen {
return errBadGenesisBytes
}

// genesisData is a byte slice but each block contains an byte array
// Take the first [dataLen] bytes from genesisData and put them in an array
var genesisDataArr [dataLen]byte
copy(genesisDataArr[:], genesisData)

// Create the genesis block
// Timestamp of genesis block is 0. It has no parent.
genesisBlock, err := vm.NewBlock(ids.Empty, 0, genesisDataArr, time.Unix(0, 0))
if err != nil {
log.Error("error while creating genesis block: %v", err)
return err
}

// Put genesis block to state
if err := vm.state.PutBlock(genesisBlock); err != nil {
log.Error("error while saving genesis block: %v", err)
return err
}

// Accept the genesis block
// Sets [vm.lastAccepted] and [vm.preferred]
if err := genesisBlock.Accept(); err != nil {
return fmt.Errorf("error accepting genesis block: %w", err)
}

// Mark this vm's state as initialized, so we can skip initGenesis in further restarts
if err := vm.state.SetInitialized(); err != nil {
return fmt.Errorf("error while setting db to initialized: %w", err)
}

// Flush VM's database to underlying db
return vm.state.Commit()
}

CreateHandlers

Registered handlers defined in Service. See below for more on APIs.

/timestampvm/vm.go
// CreateHandlers returns a map where:
// Keys: The path extension for this blockchain's API (empty in this case)
// Values: The handler for the API
// In this case, our blockchain has only one API, which we name timestamp,
// and it has no path extension, so the API endpoint:
// [Node IP]/ext/bc/[this blockchain's ID]
// See API section in documentation for more information
func (vm *VM) CreateHandlers() (map[string]*common.HTTPHandler, error) {
server := rpc.NewServer()
server.RegisterCodec(json.NewCodec(), "application/json")
server.RegisterCodec(json.NewCodec(), "application/json;charset=UTF-8")
// Name is "timestampvm"
if err := server.RegisterService(&Service{vm: vm}, Name); err != nil {
return nil, err
}

return map[string]*common.HTTPHandler{
"": {
Handler: server,
},
}, nil
}

CreateStaticHandlers

Registers static handlers defined in StaticService. See below for more on static APIs.

/timestampvm/vm.go
// CreateStaticHandlers returns a map where:
// Keys: The path extension for this VM's static API
// Values: The handler for that static API
func (vm *VM) CreateStaticHandlers() (map[string]*common.HTTPHandler, error) {
server := rpc.NewServer()
server.RegisterCodec(json.NewCodec(), "application/json")
server.RegisterCodec(json.NewCodec(), "application/json;charset=UTF-8")
if err := server.RegisterService(&StaticService{}, Name); err != nil {
return nil, err
}

return map[string]*common.HTTPHandler{
"": {
LockOptions: common.NoLock,
Handler: server,
},
}, nil
}

BuildBock

BuildBlock builds a new block and returns it. This is mainly requested by the consensus engine.

/timestampvm/vm.go
// BuildBlock returns a block that this vm wants to add to consensus
func (vm *VM) BuildBlock() (snowman.Block, error) {
if len(vm.mempool) == 0 { // There is no block to be built
return nil, errNoPendingBlocks
}

// Get the value to put in the new block
value := vm.mempool[0]
vm.mempool = vm.mempool[1:]

// Notify consensus engine that there are more pending data for blocks
// (if that is the case) when done building this block
if len(vm.mempool) > 0 {
defer vm.NotifyBlockReady()
}

// Gets Preferred Block
preferredBlock, err := vm.getBlock(vm.preferred)
if err != nil {
return nil, fmt.Errorf("couldn't get preferred block: %w", err)
}
preferredHeight := preferredBlock.Height()

// Build the block with preferred height
newBlock, err := vm.NewBlock(vm.preferred, preferredHeight+1, value, time.Now())
if err != nil {
return nil, fmt.Errorf("couldn't build block: %w", err)
}

// Verifies block
if err := newBlock.Verify(); err != nil {
return nil, err
}
return newBlock, nil
}

NotifyBlockReady

NotifyBlockReady is a helper method that can send messages to the consensus engine through toEngine channel.

/timestampvm/vm.go
// NotifyBlockReady tells the consensus engine that a new block
// is ready to be created
func (vm *VM) NotifyBlockReady() {
select {
case vm.toEngine <- common.PendingTxs:
default:
vm.ctx.Log.Debug("dropping message to consensus engine")
}
}

GetBlock

GetBlock returns the block with the given block ID.

/timestampvm/vm.go
// GetBlock implements the snowman.ChainVM interface
func (vm *VM) GetBlock(blkID ids.ID) (snowman.Block, error) { return vm.getBlock(blkID) }

func (vm *VM) getBlock(blkID ids.ID) (*Block, error) {
// If block is in memory, return it.
if blk, exists := vm.verifiedBlocks[blkID]; exists {
return blk, nil
}

return vm.state.GetBlock(blkID)
}

proposeBlock

This method adds a piece of data to the mempool and notifies the consensus layer of the blockchain that a new block is ready to be built and voted on. This is called by API method ProposeBlock, which we’ll see later.

/timestampvm/vm.go
// proposeBlock appends [data] to [p.mempool].
// Then it notifies the consensus engine
// that a new block is ready to be added to consensus
// (namely, a block with data [data])
func (vm *VM) proposeBlock(data [dataLen]byte) {
vm.mempool = append(vm.mempool, data)
vm.NotifyBlockReady()
}

ParseBlock

Parse a block from its byte representation.

/timestampvm/vm.go
// ParseBlock parses [bytes] to a snowman.Block
// This function is used by the vm's state to unmarshal blocks saved in state
// and by the consensus layer when it receives the byte representation of a block
// from another node
func (vm *VM) ParseBlock(bytes []byte) (snowman.Block, error) {
// A new empty block
block := &Block{}

// Unmarshal the byte repr. of the block into our empty block
_, err := Codec.Unmarshal(bytes, block)
if err != nil {
return nil, err
}

// Initialize the block
block.Initialize(bytes, choices.Processing, vm)

if blk, err := vm.getBlock(block.ID()); err == nil {
// If we have seen this block before, return it with the most up-to-date
// info
return blk, nil
}

// Return the block
return block, nil
}

NewBlock

NewBlock creates a new block with given block parameters.

/timestampvm/vm.go
// NewBlock returns a new Block where:
// - the block's parent is [parentID]
// - the block's data is [data]
// - the block's timestamp is [timestamp]
func (vm *VM) NewBlock(parentID ids.ID, height uint64, data [dataLen]byte, timestamp time.Time) (*Block, error) {
block := &Block{
PrntID: parentID,
Hght: height,
Tmstmp: timestamp.Unix(),
Dt: data,
}

// Get the byte representation of the block
blockBytes, err := Codec.Marshal(CodecVersion, block)
if err != nil {
return nil, err
}

// Initialize the block by providing it with its byte representation
// and a reference to this VM
block.Initialize(blockBytes, choices.Processing, vm)
return block, nil
}

SetPreference

SetPreference implements the block.ChainVM. It sets the preferred block ID.

/timestampvm/vm.go
// SetPreference sets the block with ID [ID] as the preferred block
func (vm *VM) SetPreference(id ids.ID) error {
vm.preferred = id
return nil
}

Other Functions

These functions needs to be implemented for block.ChainVM. Most of them are just blank functions returning nil.

/timestampvm/vm.go
// Bootstrapped marks this VM as bootstrapped
func (vm *VM) Bootstrapped() error { return nil }

// Bootstrapping marks this VM as bootstrapping
func (vm *VM) Bootstrapping() error { return nil }

// Returns this VM's version
func (vm *VM) Version() (string, error) {
return Version.String(), nil
}

func (vm *VM) Connected(id ids.ShortID, nodeVersion version.Application) error {
return nil // noop
}

func (vm *VM) Disconnected(id ids.ShortID) error {
return nil // noop
}

// This VM doesn't (currently) have any app-specific messages
func (vm *VM) AppGossip(nodeID ids.ShortID, msg []byte) error {
return nil
}

// This VM doesn't (currently) have any app-specific messages
func (vm *VM) AppRequest(nodeID ids.ShortID, requestID uint32, time time.Time, request []byte) error {
return nil
}

// This VM doesn't (currently) have any app-specific messages
func (vm *VM) AppResponse(nodeID ids.ShortID, requestID uint32, response []byte) error {
return nil
}

// This VM doesn't (currently) have any app-specific messages
func (vm *VM) AppRequestFailed(nodeID ids.ShortID, requestID uint32) error {
return nil
}

// Health implements the common.VM interface
func (vm *VM) HealthCheck() (interface{}, error) { return nil, nil }

Factory

VMs should implement the Factory interface. New method in the interface returns a new VM instance.

/timestampvm/factory.go
var _ vms.Factory = &Factory{}

// Factory ...
type Factory struct{}

// New ...
func (f *Factory) New(*snow.Context) (interface{}, error) { return &VM{}, nil }

Static API

A VM may have a static API, which allows clients to call methods that do not query or update the state of a particular blockchain, but rather apply to the VM as a whole. This is analogous to static methods in computer programming. MetalGo uses Gorilla’s RPC library to implement HTTP APIs.

StaticService implements the static API for our VM.

/timestampvm/static_service.go
// StaticService defines the static API for the timestamp vm
type StaticService struct{}

Encode

For each API method, there is:

  • A struct that defines the method’s arguments
  • A struct that defines the method’s return values
  • A method that implements the API method, and is parameterized on the above 2 structs

This API method encodes a string to its byte representation using a given encoding scheme. It can be used to encode data that is then put in a block and proposed as the next block for this chain.

/timestampvm/static_service.go
// EncodeArgs are arguments for Encode
type EncodeArgs struct {
Data string `json:"data"`
Encoding formatting.Encoding `json:"encoding"`
Length int32 `json:"length"`
}

// EncodeReply is the reply from Encoder
type EncodeReply struct {
Bytes string `json:"bytes"`
Encoding formatting.Encoding `json:"encoding"`
}

// Encoder returns the encoded data
func (ss *StaticService) Encode(_ *http.Request, args *EncodeArgs, reply *EncodeReply) error {
if len(args.Data) == 0 {
return fmt.Errorf("argument Data cannot be empty")
}
var argBytes []byte
if args.Length > 0 {
argBytes = make([]byte, args.Length)
copy(argBytes, args.Data)
} else {
argBytes = []byte(args.Data)
}

bytes, err := formatting.EncodeWithChecksum(args.Encoding, argBytes)
if err != nil {
return fmt.Errorf("couldn't encode data as string: %s", err)
}
reply.Bytes = bytes
reply.Encoding = args.Encoding
return nil
}

Decode

This API method is the inverse of Encode.

/timestampvm/static_service.go
// DecoderArgs are arguments for Decode
type DecoderArgs struct {
Bytes string `json:"bytes"`
Encoding formatting.Encoding `json:"encoding"`
}

// DecoderReply is the reply from Decoder
type DecoderReply struct {
Data string `json:"data"`
Encoding formatting.Encoding `json:"encoding"`
}

// Decoder returns the Decoded data
func (ss *StaticService) Decode(_ *http.Request, args *DecoderArgs, reply *DecoderReply) error {
bytes, err := formatting.Decode(args.Encoding, args.Bytes)
if err != nil {
return fmt.Errorf("couldn't Decode data as string: %s", err)
}
reply.Data = string(bytes)
reply.Encoding = args.Encoding
return nil
}

API

A VM may also have a non-static HTTP API, which allows clients to query and update the blockchain's state.

Service's declaration is:

/timestampvm/service.go
// Service is the API service for this VM
type Service struct{ vm *VM }

Note that this struct has a reference to the VM, so it can query and update state.

This VM's API has two methods. One allows a client to get a block by its ID. The other allows a client to propose the next block of this blockchain. The blockchain ID in the endpoint changes, since every blockchain has an unique ID.

timestampvm.getBlock

Get a block by its ID. If no ID is provided, get the latest block.

getBlock Signature
timestampvm.getBlock({id: string}) ->
{
id: string,
data: string,
timestamp: int,
parentID: string
}
  • id is the ID of the block being retrieved. If omitted from arguments, gets the latest block
  • data is the base 58 (with checksum) representation of the block’s 32 byte payload
  • timestamp is the Unix timestamp when this block was created
  • parentID is the block’s parent
getBlock Example Call
curl -X POST --data '{
"jsonrpc": "2.0",
"method": "timestampvm.getBlock",
"params":{
"id":"xqQV1jDnCXDxhfnNT7tDBcXeoH2jC3Hh7Pyv4GXE1z1hfup5K"
},
"id": 1
}' -H 'content-type:application/json;' 127.0.0.1:9650/ext/bc/sw813hGSWH8pdU9uzaYy9fCtYFfY7AjDd2c9rm64SbApnvjmk
getBlock Example Response
{
"jsonrpc": "2.0",
"result": {
"timestamp": "1581717416",
"data": "11111111111111111111111111111111LpoYY",
"id": "xqQV1jDnCXDxhfnNT7tDBcXeoH2jC3Hh7Pyv4GXE1z1hfup5K",
"parentID": "22XLgiM5dfCwTY9iZnVk8ZPuPe3aSrdVr5Dfrbxd3ejpJd7oef"
},
"id": 1
}
getBlock Implementation
/timestampvm/service.go
// GetBlockArgs are the arguments to GetBlock
type GetBlockArgs struct {
// ID of the block we're getting.
// If left blank, gets the latest block
ID *ids.ID `json:"id"`
}

// GetBlockReply is the reply from GetBlock
type GetBlockReply struct {
Timestamp json.Uint64 `json:"timestamp"` // Timestamp of most recent block
Data string `json:"data"` // Data in the most recent block. Base 58 repr. of 5 bytes.
ID ids.ID `json:"id"` // String repr. of ID of the most recent block
ParentID ids.ID `json:"parentID"` // String repr. of ID of the most recent block's parent
}

// GetBlock gets the block whose ID is [args.ID]
// If [args.ID] is empty, get the latest block
func (s *Service) GetBlock(_ *http.Request, args *GetBlockArgs, reply *GetBlockReply) error {
// If an ID is given, parse its string representation to an ids.ID
// If no ID is given, ID becomes the ID of last accepted block
var (
id ids.ID
err error
)

if args.ID == nil {
id, err = s.vm.state.GetLastAccepted()
if err != nil {
return errCannotGetLastAccepted
}
} else {
id = *args.ID
}

// Get the block from the database
block, err := s.vm.getBlock(id)
if err != nil {
return errNoSuchBlock
}

// Fill out the response with the block's data
reply.ID = block.ID()
reply.Timestamp = json.Uint64(block.Timestamp().Unix())
reply.ParentID = block.Parent()
data := block.Data()
reply.Data, err = formatting.EncodeWithChecksum(formatting.CB58, data[:])

return err
}

timestampvm.proposeBlock

Propose the next block on this blockchain.

proposeBlock Signature
timestampvm.proposeBlock({data: string}) -> {success: bool}
  • data is the base 58 (with checksum) representation of the proposed block’s 32 byte payload.
proposeBlock Example Call
curl -X POST --data '{
"jsonrpc": "2.0",
"method": "timestampvm.proposeBlock",
"params":{
"data":"SkB92YpWm4Q2iPnLGCuDPZPgUQMxajqQQuz91oi3xD984f8r"
},
"id": 1
}' -H 'content-type:application/json;' 127.0.0.1:9650/ext/bc/sw813hGSWH8pdU9uzaYy9fCtYFfY7AjDd2c9rm64SbApnvjmk
proposeBlock Example Response
{
"jsonrpc": "2.0",
"result": {
"Success": true
},
"id": 1
}
proposeBlock Implementation
/timestampvm/service.go
// ProposeBlockArgs are the arguments to ProposeValue
type ProposeBlockArgs struct {
// Data for the new block. Must be base 58 encoding (with checksum) of 32 bytes.
Data string
}

// ProposeBlockReply is the reply from function ProposeBlock
type ProposeBlockReply struct{
// True if the operation was successful
Success bool
}

// ProposeBlock is an API method to propose a new block whose data is [args].Data.
// [args].Data must be a string repr. of a 32 byte array
func (s *Service) ProposeBlock(_ *http.Request, args *ProposeBlockArgs, reply *ProposeBlockReply) error {
bytes, err := formatting.Decode(formatting.CB58, args.Data)
if err != nil || len(bytes) != dataLen {
return errBadData
}

var data [dataLen]byte // The data as an array of bytes
copy(data[:], bytes[:dataLen]) // Copy the bytes in dataSlice to data

s.vm.proposeBlock(data)
reply.Success = true
return nil
}

Plugin

In order to make this VM compatible with go-plugin, we need to define a main package and method, which serves our VM over gRPC so that MetalGo can call its methods.

main.go's contents are:

/main/main.go
func main() {
log.Root().SetHandler(log.LvlFilterHandler(log.LvlDebug, log.StreamHandler(os.Stderr, log.TerminalFormat())))
plugin.Serve(&plugin.ServeConfig{
HandshakeConfig: rpcchainvm.Handshake,
Plugins: map[string]plugin.Plugin{
"vm": rpcchainvm.New(&timestampvm.VM{}),
},

// A non-nil value here enables gRPC serving for this plugin...
GRPCServer: plugin.DefaultGRPCServer,
})
}

Now MetalGo's rpcchainvm can connect to this plugin and calls its methods.

Executable Binary

This VM has a build script that builds an executable of this VM (when invoked, it runs the main method from above.)

The path to the executable, as well as its name, can be provided to the build script via arguments. For example:

./scripts/build.sh ../metalgo/build/plugins timestampvm

If no argument is given, the path defaults to a binary named with default VM ID: $GOPATH/src/github.com/!metal!blockchain/metalgo/build/plugins/tGas3T58KzdjLHhBDMnH2TvrddhqTji5iZAMZ3RXs2NLpSnhH

This name tGas3T58KzdjLHhBDMnH2TvrddhqTji5iZAMZ3RXs2NLpSnhH is the CB58 encoded 32 byte identifier for the VM. For the timestampvm, this is the string "timestampvm" zero-extended in a 32 byte array and encoded in CB58. See this for more details on how to create your own VM ID.

VM Aliases

Each VM has a predefined, static ID. For instance, the default ID of the TimestampVM is: tGas3T58KzdjLHhBDMnH2TvrddhqTji5iZAMZ3RXs2NLpSnhH.

It's possible to give an alias for these IDs. For example, we can alias TimestampVM by creating a JSON file at ~/.metalgo/configs/vms/aliases.json with:

{
"tGas3T58KzdjLHhBDMnH2TvrddhqTji5iZAMZ3RXs2NLpSnhH": [
"timestampvm",
"timestamp"
]
}

Installing a VM

MetalGo searches for and registers plugins under the plugins directory of the build directory.

To install the virtual machine onto your node, you need to move the built virtual machine binary under this directory. Virtual machine executable names must be either a full virtual machine ID (encoded in CB58), or a VM alias.

Copy the binary into the plugins directory.

cp -n <path to your binary> $GOPATH/src/github.com/!metal!blockchain/metalgo/build/plugins/

Node Is Not Running

If your node isn't running yet, you can install all virtual machines under your plugin directory by starting the node.

Node Is Already Running

Load the binary with the loadVMs API.

curl -sX POST --data '{
"jsonrpc":"2.0",
"id" :1,
"method" :"admin.loadVMs",
"params" :{}
}' -H 'content-type:application/json;' 127.0.0.1:9650/ext/admin

Confirm the response of loadVMs contains the newly installed virtual machine tGas3T58KzdjLHhBDMnH2TvrddhqTji5iZAMZ3RXs2NLpSnhH. You'll see this virtual machine as well as any others that weren't already installed previously in the response.

{
"jsonrpc": "2.0",
"result": {
"newVMs": {
"tGas3T58KzdjLHhBDMnH2TvrddhqTji5iZAMZ3RXs2NLpSnhH": [
"timestampvm",
"timestamp"
],
"spdxUxVJQbX85MGxMHbKw1sHxMnSqJ3QBzDyDYEP3h6TLuxqQ": []
}
},
"id": 1
}

Now, this VM's static API can be accessed at endpoints /ext/vm/timestampvm and /ext/vm/timestamp. For more details about VM configs, see here.

In this tutorial, we used the VM's ID as the executable name to simplify the process. However, MetalGo would also accept timestampvm or timestamp since those are registered aliases in previous step.

Wrapping Up

That’s it! That’s the entire implementation of a VM which defines a blockchain-based timestamp server.

In this tutorial, we learned:

  • The block.ChainVM interface, which all VMs that define a linear chain must implement
  • The snowman.Block interface, which all blocks that are part of a linear chain must implement
  • The rpcchainvm type, which allows blockchains to run in their own processes.
  • An actual implementation of block.ChainVM and snowman.Block.