ARK Core v2: Progress Update
Since our last post, the ARK team has been developing, testing and optimizing v2 and we want to bring you up to date with our current status.
The progress has been great and there have been many changes over the past two weeks. We will also review some of the challenges we are still facing.
Statistics
During the past 2 weeks (July 24th to August 7th) there have been 102 commits to the Core repository by 11 developers.
Progress & Plan
After the decision was made to abandon backwards compatibility with v1, testing immediately began with v2. Initially, we attempted to use old v1 genesis settings to quickly obtain a pre-generated configuration and to observe how some of the still running v1 nodes would interact with v2. Some v2 nodes were effectively adding available v1 peer nodes to the list — which should not happen. Following this, a new feature was implemented “Peer Common Blocks” which analyzes blocks from other peers and automatically recognizes if a peer is a valid one (on the correct chain) or not. For example, peer A has block ID XYZ, and then checks peer B for block ID XYZ. If peer B has it, it will download from that peer. If not, the peer gets suspended.
The 2nd step was to start v2 from zero, with a new genesis block and new configuration (new nethash) to test if things are being properly generated when starting a fresh network. This now essentially became our new, clean v2 DevNet, which is currently running and being constantly tested and improved.
Desktop Wallet Compatibility
Since the last update we have worked on making the current Desktop Wallet compatible with the changes brought by the new core. Everything on Core has been done from scratch which means a new version of API v2 as well. This brings new features and best practices for writing APIs and API documentation.
As the Desktop Wallet is reliant on the JavaScript client, some of it was rewritten to ensure compatibility. This new Desktop Wallet has been tested and all transaction types are working well on the new v2 DevNet.
The new wallet will be released simultaneously as the public DevNet. If you would like to be involved with testing you will need to update and register a delegate. You will still be able to use your delegate on MainNet where there will be no changes.
Challenges
There are still some bugs to squash before we open v2 to the public (although technically it is already available). One major bug to solve is a problem with transaction pool management. While things are smooth for a normal user (send, vote, unvote, register a delegate, register 2nd passphrase), if you spam the transaction pool in batches of a few hundred transactions per request, the transaction pool can not keep up. It begins duplicate transactions in the pool, effectively making a chain reaction as new transactions enter with the majority not being confirmed. For example, sending 100 transactions in a single request will duplicate the request and confirm just a fraction of the transactions).
The second problem relates to issues with peer propagation. During the startup of Core, it seems the collection of peers associated with the network is not complete, negatively impacting the network itself and blocking broadcasting.
Finally, a minor issue, the productivity and approval of a delegate are not properly calculated in certain cases due to the maximum integer size of JavaScript. This will be corrected by implementing BigNumber.JS for all numerical operations.
Fixed Bugs In The Last 14 Days
- Remove unneeded check for valid transactions in supposedly empty array.
- New Peer condition to accept peers others than ‘myself’.
- Fix P2P API before starting monitor to be able to be pinged by peers.
- Fixing various errors in transaction guard.
- Create unit test genesis block only after setup.
- Ignore IPv6 localhost addresses when searching network peers through the API client.
- Download size check on random peer.
- Pool wallet manager state & minor issues fixes.
- Get block ID from data.
- Append current round blocks before applying round.
- Apply missed blocks to wallet in SPV.
- Mark wallet dirty when missing blocks.
- Moving query logic to Sequelize connection.
- Unconfirmed Balance changed to balance in model, to fix issue with value being undefined.
- Broken transfer & vote commander in tester CLI has been fixed.
New Features & Improvements In The Last 14 Days
- Added throttling to the request P2P API to help prevent DDOS.
- Minimum peer version has been implemented making it possible to reject peers that don’t run specified minimum version of the core. There is an option to override this rule by white-listing specific peers in your config file.
- Added an option to blacklist peers in the config file.
- Implemented a way to find good peers via JavaScript client library and connect to the most healthy ones.
- Check common blocks when downloading blocks.
- Store recent block IDs.
- New method to get blocks for a round.
- Loading blocks from current round on start.
- Improving computational complexity when removing forged transactions.
- Addition of logging message for missed blocks as well as suspended peers by blacklist rule or throttle limit.
- Block processing logic has been rewritten.
When Can We Expect For Others To Get Involved?
As soon as we solve the transaction pool issue we will publicly launch the v2 DevNet. This will likely happen in the upcoming days and will be followed up with a blog post with all necessary instructions. Also outlined will be plans to get V2 up and running on MainNet as soon as possible.
We sincerely thank you for your patience and ask for your understanding as we leap these final few hurdles and pave the way for a stable new ARK Core v2.
How Can I Help With Development?
If you want to help, get involved and earn some extra ARK be sure to give ARK GitHub Development Bounty a read and get coding:
- ARK Core repository : https://github.com/ArkEcosystem/core
- ARK Core open issues : https://github.com/ArkEcosystem/core/issues
ARK.io | Github | Facebook | Twitter | Forum | Blog | Explorer | Shop