Building a Blockchain with Go - Persistence and Command Line - Part 3

in #utopian-io6 years ago

golang.jpg

Repository

https://github.com/golang/go

What Will I Learn?

  • You will learn how to use BadgerDB in a Go project
  • You will learn about building command line interfaces with Go
  • You will learn how to persist a Blockchain with a Key/Value database.
  • You will learn how to create an Iterator in Go
  • You will learn how to shutdown Go Routines
  • You will learn how defer works
  • You will learn how the Go module system imports 3rd party libraries

Requirements

System Requirements:

Operating System:

  • FreeBSD 10.3 or later
  • Linux 2.6.23 or later with glibc
  • macOS 10.10 or later
  • Windows 7, Server 2008R2 or later

Required Knowledge

  • A little understanding of the Go programming language
  • Go installed on your computer
  • A text editor or IDE like Gogland (VS Code used)

Resources for Go and this Project:

Credits/Sources:

Difficulty

  • Intermediate

Description

In this series, we have built a mostly functional blockchain thus far. However, blocks can only be added through hard coding them into the main function. Also, the blocks need to be re-signed upon every execution of the application. This also means that the blockchain can't be shared with other peers which defeats the purpose of the chain. To solve these problems, we need to build a persistence layer and a command line interface. Those are the main subjects of this Go video tutorial.

Blockchain Persistence with BadgerDB

If you read the original Bitcoin whitepaper, there is nothing specific written regarding the type of database that should be used for the persistence layer. The current implementation of Bitcoin makes use of a low level key-value storage database called LevelDB. LevelDB along with another database derived from LevelDB called RocksDB are both fairly common in the cryptocurrency scene. In this application, we use a native Go Key-Value database called BadgerDB for our persistence layer.

leveldb.png

Badger is an appropriate choice because it follows the conventions set up by LevelDB and RocksDB while also adding performance features to the architecture. One of the main design goals of BadgerDB is to write an extremely fast Key-Value database that can span across a very large amount of data. It is able to accomplish this task by using the latest SSD technologies and an ACID transaction technology called Serializable Snapshot Isolation.

Changing up the Blockchain Structure

With the addition of BadgerDB to our application, we need to refactor the Blockchain Structure from an array of blocks in memory to a pointer to the database and a cursor that signifies where in the chain we are. The BadgerDB API gives us a Database type to use as a pointer. Our current position in the Blockchain can be found using the block hashes. We can save the latest Hash value in memory and in the database for quick recall in a LastHash field.

chain-struct.png

The Database needs to store the blocks from our blockchain as well as the LastHash key. We have to serialize the Blocks into a slice of bytes before we can push them into the database. A simple serialization function can be written with the "encoding/gob" library. This same library can also be used to reclaim the data and deserialize it back into a block structure. A key called lhis also added to the database so that we can store and retrieve the LastHash.

Adding a Command Line Interface to the Blockchain

We are now able to build a command line interface which lets a user access our blockchain. In this tutorial, we add two primary commands to the interface; add and print. Add allows a user to add a new block to the blockchain which also automatically runs the Proof of Work algorithm upon each addition. The Print command allows a user to print out all of the blocks in the blockchain in order from the latest back to the genesis block. This is achieved through the use of of a blockchain iterator struct.

cli.png

As with most programming languages, Go treats the command line as an array of strings. Every item added to the command line after the initial execution is pushed into this array. We are able to control which items we want by adding flags and parsers to the values. We use a switch statement to determine if a user has typed in print or add and we execute the code based on this outcome.

main-defer.png

Also, inside of our main function, we add two defer statements, one to properly close our BadgerDB to stop it from corrupting the data on the disk and one to properly close down the program when a Goexit command is issued. The Go os library can also be used to shut down a program prematurely but there is not guarantee that the defer statements will be honored.

The Source Code for this video may be found here: https://github.com/tensor-programming/golang-blockchain/tree/part_3

Video Tutorial

Curriculum

Proof of Work Done

https://github.com/tensor-programming

Sort:  

I thank you for your contribution. Here are my thoughts. Note that, my thoughts are my personal ideas on your post and they are not directly related to the review and scoring unlike the answers I gave in the questionnaire;

  • This is different from my all other reviews. I'm just speechless. The effort you spent on this amazes me and the content you are showing is actually teaches something useful. The curriculum caught my attention and I plan to watch other videos of you whenever I have time, thanks for the effort!

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

Thank you for your review, @yokunjon! Keep up the good work!

Thank you very much for the high praise and for moderating my contribution. I hope you enjoy my other videos.

Hi @tensor, a little bit out of topic since you mention about LevelDB 🙂.
In some of Deep Learning framework I ever try, some of them use LevelDB while the others use HDF5 (not mistaken with HDFS). Do you know what's the key difference is?

Well firstly, LevelDB isn't really the same as HDF5 as one is a database and the other is a structure format. That being said, LevelDB uses LMDB which can kind of be compared with HDF5.

HDF5 requires many system calls to access the data. LMDB on the other hand requires zero system calls per data access instance. LMDB is made to be extremely fast and efficient with that being the primary focus, where as HDF5 has more of an emphasis on structure. Again though, its hard to compare the two since they aren't really the same thing.

Hmm...so HDF5 is suitable for storing a bunch of data with a lot of different structure while LMDB more like when performance is really important, plus if you need embeddable database since LMDB doesn't require system calls 🤔

Hi, @tensor!

You just got a 8.15% upvote from SteemPlus!
To get higher upvotes, earn more SteemPlus Points (SPP). On your Steemit wallet, check your SPP balance and click on "How to earn SPP?" to find out all the ways to earn.
If you're not using SteemPlus yet, please check our last posts in here to see the many ways in which SteemPlus can improve your Steem experience on Steemit and Busy.





This post has been voted on by the SteemSTEM curation team and voting trail in collaboration with @curie.

If you appreciate the work we are doing then consider voting both projects for witness by selecting stem.witness and curie!

For additional information please join us on the SteemSTEM discord and to get to know the rest of the community!

Hi @tensor!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

Hey, @tensor!

Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

Coin Marketplace

STEEM 0.28
TRX 0.13
JST 0.032
BTC 65920.41
ETH 3016.79
USDT 1.00
SBD 3.71