#Blurtopian Development: Blocktivity Statistics API

in blurtopian •  8 months ago 


Have you ever wondered how much activity we have in Blurt in terms of operations[1] and transactions[2]? Well, that's exactly what Blocktivity measures. Thanks to @megadrive for wanting to add BLURT to the site.

From Operation Vs Transaction, the Blocktivity big debate:

[1] An operation is any signed message sent to the blockchain (including transactions)
[2] A transaction is any signed message sent to the blockchain that result in a token transfer

1. Background

Operations give us an idea of how active a blockchain is. Operations vary depending on the chain. In graphene-based blockchains like Steem, Hive, and Blurt, operations consist of upvotes, power-up, power-downs, etc.

With this in consideration, let's look at how the tool computed this for blocktivity.

2. Implementation

When tackling a project, the measure I use to estimate how long it will take to complete it is in the planning - the clearer the image I have on which tasks needs to be completed, the faster I'll be able to finish it.

For the blocktivity API, these were the tasks I identified:

  1. Blockchain Streaming
  2. Statistics Computation
  3. and, Data Storage

For #1, I already have existing projects which I can build upon.

#2 was more challenging because I didn't know from which data I can extract the information I needed. This will determine whether I needed to develop a tool from scratch or use an existing api.

And lastly, was making the decision where to store the data and for how long.

Let's look into the details on how I tackled each.

2.1. Streaming


Just last week, @yourking shared his codes for streaming the chain to check for plagiarized content. I checked its codes and found interesting lines which contained auto-switching rpc nodes should one fail. I wanted to port this function to blocktivity API as a fail-safe mechanism.

Initially, I used the get_block to stream the chain. Unfortunately, this didn't include virtual operations[3] which I believe constitutes block activity. The get_ops_in_block has this so I switched to it.

Once I had this down, the next step was the essence of the tool - computing blockchain operations.

[3]You can refer to Hive's documentation for the different virtual operations.

2.2. Statistics Computation


This is the critical piece of the project. This determined whether this project will succeed or not. The question I had to answer was:

Is there an existing function available to extract this data or do I need to build one from scratch?

As I mentioned in the previous section, fortunately, there was an existing API. It was just a matter of discovering which.

I initially though that this information can be retrieved from the get_dynamic_global_properties since from it's name I thought that since it's about getting dynamic global properties, then it should have this information, right? As I found it, it didn't. So I had to find another way.


Then it hit me. The result of get_ops_in_block contained an array of transactions included in a block, I can use its length (the number of items in the array) as the count for the number of operations, and I just needed to extraction the information from them as to which can be considered as transactions[1].

Getting this last piece of information was easy because it was just a matter of looping through the array, extracting the transaction type, and incrementing a count variable.

Which led me to the last piece of the puzzle - storing the data.

2.3. Data Storage


I initially thought that I had to store the history of the data I extracted. Fortunately, that wasn't what Blocktivity needed. They just needed that piece of information regarding the number of operations and transactions from the last hour. I even came to the point of overthinking of whether I had to consider the timezone that Blocktivity will call the API.


With God's help, I didn't continue that line of thought. Since they call the API every fifth minute of the hour, I needed not to consider the timezone because no matter the timezone, it's synchronized by the hour.


The fifth minute of the 12:05 UTC is the same fifth minute in, let's say, 21:05 Japan Standard Time.

I just decided to extend the delay for the API to respond with the data of the previous hour up to the 10th minute of the current hour. Meaning, if it's already 12:10 UTC, it will still return the data for the number of transactions of the previous hour - 11:00 to 11:59.

As for the storage, which is what this section is exactly about, I opted to use mongoDB since it's what I use in most of my projects. I also decided to only store the data for the last two hours - the current and the previous hours.

With this last piece in place, it was a matter of pushing and deploying it to my server.

Blurtopian's blocktivity API - this was what blocktivity needed.




3. Repository

The project's available in Blurtopian's Github repository.


@eastmael's Witness Projects


Find my contributions valuable? Please consider voting for my witness:



Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE BLURT!
Sort Order:  

Thanks for your help with this, I have added to https://gitlab.com/blurt/blurt/-/blob/dev/doc/devs/networknodes.md

Thanks. Network nodes looks more informative now.

Congratulations! This post has been upvoted by the @blurtcurator communal account,
You can request a vote every 12 hours from the #getupvote channel in the official Blurt Discord.Don't wait to join ,lots of good stuff happening there.

Just would like to mentioned that the code for streaming is not my code. It's copied from an open source hive program.

No worries. Yup, saw from the comments that it's from Rishi.

Again I'm impressed by what BLURT has accomplished. I just checked out https://blocktivity.info and we are already 20% as active as HIVE.


I did some analysis on BLURT and they turned out to be much better than what most people would imagine: https://blurt.blog/blurt365/@vimukthi/blurt-blockchain-front-end-statistics-are-more-incredible-than-hive-s I have becom an even bigger believer of BLURT after seeing those numbers. For comparison these are the numbers for HIVE: https://blurt.blog/blurtech/@vimukthi/dbuzz-microblogging-beats-leofinance-in-most-engagement-metrics-more-hive-front-end-statistics