Extracting data from Steem Blockchain with SteemData and MongoDB, Part 1: Setting things up

in #steemdata6 years ago

In this tutorial I am going to show you how to extract data form the Steem-Blockchain with the help of SteemData and MongoDB.
I am new to this topic, but I am very excited about the Steem-Blockchain and also Steemit. I would like to contribute to the community by creating some tutorials, so users can get an even better experiance.

In this tutorial we will use a tool to look through the database and to extract the data. In future parts I will also use programming language to get the data directly from a self written program.

What is SteemData?

The website SteemData is collecting data from the Steem-Blockchain in easy accessible databases such as a MongoDB and also a MySQL database. They also provide some WebAPI resources. In this tutorial I will concentrate on the MonogDB database.

Fig. 1 - Screenshot of the SteemData website.

Setting things up

Accessing the data in the MongoDB is not straight away possible. We need a tool. I recommend the free tool Robo T3. After the installation we need to set things up.

  1. Create a new connection
  2. Set the connection settings as showed in Fig. 2
  3. Set the authentication settings as showed in Fig. 3
  4. Save it and click connect


    Fig. 2 - The connection tab.

    Fig. 3 - The authentications tab.
    On the left side you should see the database and its collections, like shown on Fig. 4.

    Fig. 4 - The database and its available collections.

Accessing data

Now to the more fun part. Getting actual data from the Blockchain.
If you click on "Accounts" the program is sending the following request to the database:

db.getCollection('Accounts').find({})

This will return all data to all accounts they have stored and show 50 accounts, starting the the oldest. Clicking on the first one shows the first ever created Steem account. It is @a-0.

Fig. 5 - The first ever created Steem account @a-0

Examples

Finding yourself

To find your own account you have to tell the find() command your account name. Like this:

db.getCollection('Accounts').find({name:"hawk399"})

This will return all the information about my account. To display your account change my name in the command to yours.

Getting the account with the most SBD at the moment

We will sort the accounts acording to the current SBD balance to see which account has the most. To do so we use following command:

db.getCollection('Accounts').find({'balances.total.SBD':{$gte:10000}}).sort({'balances.total.SBD':-1})

In the find() command we specify to only take accounts with more then 10'000 SBD, otherwise we have too much data and the server will throw an error. And we also sort the results by their current total SBD.
At the time of writing this article the five accounts with most SBD are:

Nr.NameSBD
1.bittrex6'366'653.209
2.poloniex471'508.689
3.freedom217'016.276
4.openledger-dex26'051.674
5.created19'205.262

It is not very surprising to see that the account with the most SBD is actually an exchange, as it has needs quite a lot of liquid funds, so people can trade with it.

Summary

With the help of SteemData and Robo T3 we are able to get data from the Steem-Blockchain and even to do some easy calculations.


Discloser

I am by no means a professional database programmer and have therefore only limited knowledge. I would like to thank @holger80 for his post about SteemData. I got some good inspiration from it.

If you liked the article I would be happy if you could give me an upvote and I would also not mind if you want to resteem the article and or follow me.
Any comments and feedback's are welcome and appreciated.

Coin Marketplace

STEEM 0.28
TRX 0.13
JST 0.032
BTC 61233.41
ETH 2934.46
USDT 1.00
SBD 3.56