Consistency in configuration of public EOS “Full Nodes”

in #eos6 years ago (edited)

Greymass

With the EOS mainnet launch underway, we need to start looking beyond launch at how block producers service the greater community of users and dApp developers.

One of the ways block producers have pledged to do this is by providing what is commonly known as an "API Endpoint" or a "Full Node". These are publicly available resources deployed to provide access and information about the EOS blockchain. These endpoints are then used by developers to build websites and applications that interact with the core EOS protocols.

This pledge to support the network is a great initiative, but in order to truly decentralize the network and provide the end-users with the best possible experience, a minimum configuration will have to be established. This document sets out to start the process of discovery on this topic.

Defining the Full Node

A "full node" in the context of many other blockchains simply means "all the plugins". This is probably what most block producers today would respond to you if you asked how they were planning on deploying their full nodes. This is largely because the demand for access to these EOS resources is very low right now and the ecosystem itself is just getting on its feet.

Very soon though there will be a flood of requests for different kinds of information to be surfaced through these services. In anticipation of this demand - we propose that a minimum standard configuration be put forth that all public node operators should meet to qualify as one of these publicly available servers.

This standard will not be related to networking or hardware configuration, but a method of configuring the account history plugin.

Account History

In a system with customizable smart contracts, the scope of something like the account history plugin could grow to be enormous. Block.one saw this coming and gave node operators a tool to deal with this, --filter-on, and then stripped the account history plugin bare by default.

When a system is run with the account history plugin, without any additional configuration, only a very basic set of information is actually indexed and exposed through the APIs. The problem with this is no history of EOS token transfers or any of the system contracts are actually returned. If a user is using a mobile or desktop wallet and connects to one of these unconfigured nodes their entire history will be empty. Even more confusing is the situation where a user switches an app between two different API endpoints and is shown different types of operations.

To prevent these types of situation - a minimum configuration is needed to establish consistency.

Fundamental system responsibilities

The basis of the configuration listed below makes the assumption that the following contracts are critical to the operation of the network. Each of these contracts should be maintained in full, with full history for every account. There can still be room to blacklist specific bad actors from indexing, but by default accounts should be included.

These contracts are:

  • eosio - The system contract.
  • eosio.token - The EOS token contract.
  • eosio.msig - The multisig contract.
  • ... (feel free to propose more)

This configuration will allow all token holders a basic history and access to the internals of what makes EOS tick.

Relevant nodeos parameters

The following is an example piece of the config.ini that a nodeos operator would use to comply with this proposed standard.


Edit/Changes


Draft

# Plugins required
plugin = eosio::chain_plugin
plugin = eosio::chain_api_plugin
plugin = eosio::history_plugin
plugin = eosio::history_api_plugin

# History of EOS token transfers
filter-on = eosio.token:transfer: 

# Creation of tokens
filter-on = eosio.token:issue: 
filter-on = eosio.token:create: 

# History of multisig transactions
filter-on = eosio.msig:propose:
filter-on = eosio.msig:approve:
filter-on = eosio.msig:unapprove:
filter-on = eosio.msig:cancel:
filter-on = eosio.msig:exec:

# System Contract Operations
filter-on = eosio:newaccount:
filter-on = eosio:updateauth:
filter-on = eosio:deleteauth:
filter-on = eosio:linkauth:
filter-on = eosio:unlinkauth:
filter-on = eosio:canceldelay:
filter-on = eosio:onerror:
filter-on = eosio:buyrambytes:
filter-on = eosio:buyram:
filter-on = eosio:sellram:
filter-on = eosio:delegatebw:
filter-on = eosio:undelegatebw:
filter-on = eosio:refund:
filter-on = eosio:regproducer:
filter-on = eosio:setram:
filter-on = eosio:bidname:
filter-on = eosio:unregprod:
filter-on = eosio:regproxy:
filter-on = eosio:voteproducer:
filter-on = eosio:claimrewards:
filter-on = eosio:setpriv:
filter-on = eosio:rmvproducer:
filter-on = eosio:setalimits:
filter-on = eosio:setglimits:
filter-on = eosio:setprods:
filter-on = eosio:reqauth:
filter-on = eosio:setparams:

QoL change to filter-on

This configuration is quite verbose, but to help address this we have opened issue/4068 on GitHub as a way to simplify the configuration options involved the filter-on parameter.

Try it

We have this configuration running on our full node available here:

https://eos.greymass.com

Try it out and lookup your account history.

./cleos -u https://eos.greymass.com get actions your_account_name

You can also look at our block producer account and see the regproducer actions we issued when submitting our candidacy.

./cleos -u https://eos.greymass.com get actions teamgreymass

Contribute and help formalize

This configuration of the account history plugin is an example of what this standard should eventually become. Arguments could be made for the addition of more contracts or the removal of others - and that conversation needs to be hashed out. We hope that starts now.

We encourage all block producers who plan on supporting the network with full nodes to think about this subject and join the conversation. Consensus must be reached between the block producers to create a seamless and consistent experience across the EOS network.

Sort:  

Thank you for your amazing work, we all thank you...

This is great! Appreciate your efforts and for continuing to improve around the voting process.

We are providing an API endpoint in Costa Rica which can be accessed using the following URL https://api.eosio.cr

As far as we know, this is the first detailed documentation on EOS full node configuration. Thank you greymass and jesta. We all appreciate your help towards the EOS community.

Great write up! I had a question about using filter-on. I assume this would require a replay to actually build up the proper indices after you add the configs?

Folks, i forgot my local password of the wallet, how do i "reset" the wallet to enter my private key again?

I've created exact same API endpoint on Tokenika side without thinking this should be widely adapted. Thank you for pointing that out!

In the meantime of issue #4068, could we agree on this full-node configuration on GitHub?

Excellent article. is there any way I can setup filter-on just to receive or save the transactions from only one account?

How long does it take to sync a fullnode? I've been running mine for 8h and the disk space used only increased by 10MB with the following config:

plugin = eosio::chain_plugin
plugin = eosio::chain_api_plugin
plugin = eosio::history_plugin
plugin = eosio::history_api_plugin
plugin = eosio::http_plugin

I'm only seeing produce_block output instead of sync messages:

eosd                | 2018-10-11T16:39:36.000 thread-0   producer_plugin.cpp:1419      produce_block        ] Produced block 000003b045facd81... #944 @ 2018-10-11T16:39:36.000 signed by eosio [trxs: 0, lib: 943, confirmed: 0]
eosd                | 2018-10-11T16:39:36.500 thread-0   producer_plugin.cpp:1419      produce_block        ] Produced block 000003b17a7ff506... #945 @ 2018-10-11T16:39:36.500 signed by eosio [trxs: 0, lib: 944, confirmed: 0]
eosd                | 2018-10-11T16:39:37.000 thread-0   producer_plugin.cpp:1419      produce_block        ] Produced block 000003b2f76ee17c... #946 @ 2018-10-11T16:39:37.000 signed by eosio [trxs: 0, lib: 945, confirmed: 0]
eosd                | 2018-10-11T16:39:37.500 thread-0   producer_plugin.cpp:1419      produce_block        ] Produced block 000003b3238a3d44... #947 @ 2018-10-11T16:39:37.500 signed by eosio [trxs: 0, lib: 946, confirmed: 0]
eosd                | 2018-10-11T16:39:38.000 thread-0   producer_plugin.cpp:1419      produce_block        ] Produced block 000003b443bf1b4a... #948 @ 2018-10-11T16:39:38.000 signed by eosio [trxs: 0, lib: 947, confirmed: 0]

Eep sorry I missed this.

Two things:

  1. It takes a while for history for EOS mainnet, and a lot longer if you're using lower end equipment. I'd estimate about a week on a 4.5ghz machine, with the shared_memory files loaded into RAM (avoiding needing to write to disk).
  2. It sounds like you've started your own chain, which is why you're producing blocks. There's no history until you start generating some on your network, or you can reconfigure your network with the mainnet peers and proper genesis files to start syncing the full chain.

I know those answers probably lead to a lot more questions :)

Coin Marketplace

STEEM 0.30
TRX 0.11
JST 0.033
BTC 64271.38
ETH 3157.43
USDT 1.00
SBD 4.25